Re: [Xen-devel] CONFIG_SPARSE_IRQ breaks single VCPU domain 0 between xen/master and xen/next

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Fri, 2010-02-26 at 17:28 +0000, Jeremy Fitzhardinge wrote:
> On 02/26/2010 08:17 AM, Ian Campbell wrote:
> > On Fri, 2010-02-26 at 12:05 +0000, Ian Campbell wrote:
> >    
> >> Which looks might suspicious to me... However simply removing that
> >> causes acpi_probe_gsi to return 16 (instead of 24) and I run out of
> >> interrupts for use by real hardware (specifically my disk controller).
> >> If I hack acpi_probe_gsi to return at least 24 everything works OK so
> >> it seems the error is only at the detection stage.
> >>      
> > So this seems to all relate to the removal of the xen_io_apic_(read|
> > write) stuff.
> >    
> 
> Yep.
> 
> > I can see that the GSI routing stuff is effective replaced by
> > PHYSDEVOP_setup_gsi but I don't see what replaces the IO APIC
> > enumeration. We still map a dummy page for FIX_IO_ACPI_* and
> > io_apic_(read|write) now go at that direct (and therefore get 0s back).
> > If the intention is not to enumerate the IO APICs in this way then what
> > seems to be missing is the part which discovers the number of GSIs in
> > the system and I'm not sure what is supposed to replace that.
> >    
> 
> Nothing, as yet.  The "+= 256" is definitely a hack, and we need to come 
> up with a sound way to resolve it.  There seem to be three possibilities:
> 
>     * Let the kernel see the IO APICs for the purposes of enumeration,
>       but nothing else (which seems to defeat the point of the exercise)
>     * Make up a fake Xen IO APIC mapping which just contains static
>       state for the config registers. (I don't think this will work,
>       because the IO APIC registers aren't simply memory-mapped)

Unfortunately IIRC they are index+data style, which is a pain.

>     * Add an interface to Xen so it can return the results of its own IO
>       APIC enumeration, and use that in dom0.  I think this is probably
>       most consistent with the idea that "Xen owns all the APICs", but
>       I'm not sure how to wire it into the Linux side.

Yes, that's the lines I was thinking along as well, but I wasn't sure
how to go about it either. For example I suppose you'd need to get in at
the acpi_parse_ioapic-ish level which isn't going to fly.

Could we nobble the APIC portion of the ACPI tables or otherwise arrange
for the generic code to find no IO APICs and instead enumerate and
register them in the Xen specific code? I noticed we have
xen_io_apic_init() which doesn't seem to be called from anywhere. There
is also a "skip_ioapic_setup" variable already defined which might be
useful?

Regardless of the mechanism for detecting number of hardware interrupts
required I think we need a mechanism to cause some extra interrupts to
be available for VIRQ and backend use. I think previously we just been
lucky that the core code overestimated the number of h/w interrupt
sources so we got a few free ones for our purposes. It looks like the
upstream x86 guys are doing some work to make interrupts be more
dynamically allocated (the radix tree irq_desc stuff) which looks like
it would be very useful for us once it lands. In 2.6.18 we explicitly
left space for a number of dynamic IRQs which seems like a reasonable
approach in the interim, I'll cook up a patch.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel