This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] [PATCH] VT-d: improve RMRR validity checking

To: Weidong Han <weidong.han@xxxxxxxxx>
Subject: Re: [Xen-devel] [PATCH] VT-d: improve RMRR validity checking
From: Alex Williamson <alex.williamson@xxxxxx>
Date: Tue, 09 Mar 2010 21:47:18 -0700
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Noboru Iwamatsu <n_iwamatsu@xxxxxxxxxxxxxx>, "Cihula, Joseph" <joseph.cihula@xxxxxxxxx>, "Kay, Allen M" <allen.m.kay@xxxxxxxxx>, "linux@xxxxxxxxxxxxxx" <linux@xxxxxxxxxxxxxx>, "keir.fraser@xxxxxxxxxxxxx" <keir.fraser@xxxxxxxxxxxxx>
Delivery-date: Tue, 09 Mar 2010 20:47:57 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4B971F57.9000903@xxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <C77E162B.6FE6%keir.fraser@xxxxxxxxxxxxx> <4B59098B.6000108@xxxxxxxxx> <4B590FA4.4000008@xxxxxxxxxxxxxx> <4B59132B.40607@xxxxxxxxx> <4B59188C.50901@xxxxxxxxxxxxxx> <4B59660F.4000909@xxxxxxxxx> <7162ab21003091339i4adb8669safd5e074607386a2@xxxxxxxxxxxxxx> <4B9706B3.3050903@xxxxxxxxx> <1268191101.3015.30.camel@xxxxxxxxxx> <4B9711CF.7030908@xxxxxxxxx> <1268192253.3015.37.camel@xxxxxxxxxx> <4B971F57.9000903@xxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Wed, 2010-03-10 at 12:25 +0800, Weidong Han wrote:
> Alex Williamson wrote:
> > On Wed, 2010-03-10 at 11:28 +0800, Weidong Han wrote:
> >   
> >> Alex Williamson wrote:
> >>     
> >>> On Wed, 2010-03-10 at 10:40 +0800, Weidong Han wrote:
> >>>   
> >>>       
> >>>> Alex Williamson wrote:
> >>>>     
> >>>>         
> >>>>> I have a system with what I consider to be a valid DRHD that's getting
> >>>>> tripped up on this patch.  The problem is that the DRHD includes an
> >>>>> IOAPIC scope, where the IOAPIC is not materialized on the PCI bus.  I
> >>>>> think Xen is being overzealous in it's validity checking and that this
> >>>>> is a valid configuration.  What do others think?  Are IOAPICs a
> >>>>> special case that we can allow to be non-existent on the PCI bus?
> >>>>>   
> >>>>>       
> >>>>>           
> >>>> Yes, IOAPIC can be not pci-discoverable. IOAPICs are only reported in 
> >>>> the "Include_all" DRHD, and our patch won't check if the device is 
> >>>> pci-discoverable or not for the "Include_all" DRHD. So I think the patch 
> >>>> is no problem unless IOAPIC is not included in the "Include_all" DRHD. 
> >>>> Can you post your boot logs?
> >>>>     
> >>>>         
> >>> Weidong,
> >>>
> >>> That's a very subtle restriction, and I'm not sure how it works in
> >>> practice.  If I have a multi-IOH system, each with VT-d hardware, each
> >>> supporting interrupt remapping, each with one or more IOAPICs below
> >>> them, how can interrupt remapping work if we can only associate an
> >>> IOAPIC with the "include all" DRHD?  I'm confused.  Thanks,
> >>>   
> >>>       
> >> Each IOH will have one "include all" DRHD which reports IOAPICs for each 
> >> IOH.
> >>     
> >
> > Wouldn't that imply multiple PCI segments?  The configuration I'm
> > looking at has multiple IOHs, all on the same PCI segment.  By my
> > reading of the spec, we're only allowed to declare INCLUDE_PCI_ALL for
> > one DRHD within the segment.  Am I incorrect?  Thanks,
> >   
> Currently multiple PCI segments are not supported in Xen yet. So you 
> encounter issue on multiple PCI segment system. We will support it after 
> xen 4.0.

Which is exactly why we have multiple IOHs on the *same* PCI segment on
this system.  How is it possible to support multiple IOHs, all on the
same PCI segment, each with VT-d hardware with interrupt remapping
support, each with one or more IOAPICs below them given the current
code?  We cannot list the IOAPICs only under the INCLUDE_PCI_ALL DRHD
because that wouldn't provide the right information in the right place
for interrupt remapping on the other DRHDs.  We cannot specify
INCLUDE_PCI_ALL on all of the DRHDs because the spec indicates we can
only have one INCLUDE_PCI_ALL DRHD per PCI segment (besides, we can't
have PCI sub-hierarchy scopes specified on INCLUDE_PCI_ALL DRHDs, which
means we'd have no way to associate PCI devices to a specific DRHD if
they all set this flag).

I'm inclined to believe the hardware actually works correctly if we
associate an IOAPIC to a non-INCLUDE_PCI_ALL DRHD, but this validity
checking code prevents Xen from even trying to use it.


Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>