This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] RE: co-assignment needs for PCI devices

To: "Cui, Dexuan" <dexuan.cui@xxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxxxx>
Subject: [Xen-devel] RE: co-assignment needs for PCI devices
From: Ross Philipson <Ross.Philipson@xxxxxxxxxx>
Date: Wed, 1 Apr 2009 09:09:52 -0400
Accept-language: en-US
Acceptlanguage: en-US
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Wed, 01 Apr 2009 06:10:16 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <EADF0A36011179459010BDF5142A457511D4698D@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <49D1DDEF.76E4.0078.0@xxxxxxxxxx> <EADF0A36011179459010BDF5142A457511A788E9@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> <49D1E917.76E4.0078.0@xxxxxxxxxx> <EADF0A36011179459010BDF5142A457511A78927@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> <831D55AF5A11D64C9B4B43F59EEBF7202311F51C6A@xxxxxxxxxxxxxxxxxxxxxxxxx> <EADF0A36011179459010BDF5142A457511D4698D@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Acmx1jsD0qvyawRKQLqGdahVpjCKsAAAF3awAAjhzDAAKYcdIAAKpvfw
Thread-topic: co-assignment needs for PCI devices
Ah I see now. I was looking at the trivial case where the first parent was 
DEV_TYPE_PCIe_BRIDGE but this would never really happen given the code leading 
up to the call to find_the_uppermost_pci_bridge(). Got it thanks.


-----Original Message-----
From: Cui, Dexuan [mailto:dexuan.cui@xxxxxxxxx] 
Sent: Wednesday, April 01, 2009 4:20 AM
To: Ross Philipson; Jan Beulich
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: co-assignment needs for PCI devices

Hi Ross,
find_the_uppermost_pci_bridge() is used to find the uppermost PCIe-to-PCI/PCI-X 
In the loop of the function, when dev_parent.dev_type == DEV_TYPE_PCIe_BRIDGE 
and the loop ends, dev.dev_type should be DEV_TYPE_PCI_BRIDGE.
Looks we have no issue here.

-- Dexuan

-----Original Message-----
From: Ross Philipson [mailto:Ross.Philipson@xxxxxxxxxx] 
Sent: 2009年3月31日 20:22
To: Cui, Dexuan; Jan Beulich
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: co-assignment needs for PCI devices

Thanks for the questions and answers regarding the below topics :) I was just 
working on related functionality and was arriving at some of the same questions 
(specifically the last one regarding PCIe one-device-per-secondary-bus).

I had another question/comment related to this topic and the xend code that is 
locating co-assigned PCI devices. It seems the routine 
find_the_uppermost_pci_bridge() is meant to find the uppermost bridge device 
that bridges *TO* a PCI/PCI-X bus. Per the logic this makes sense that it would 
be any non-PCIe bridge but also a PCIe-to-PCI/PCI-X bridge which is tested for 
earlier when the device type is determined. So far this all seems to make 
sense. But in the routine above, the logic seems like it would actually go past 
what I would think of as the topmost PCI/PCI-X bridge and find the next parent 
(note that if dev_type == DEV_TYPE_PCIe_BRIDGE, the loop ends but that device 
is taken to be the target bridge). Is this correct or is there an issue in this 


-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx 
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Cui, Dexuan
Sent: Tuesday, March 31, 2009 4:11 AM
To: Jan Beulich
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] RE: co-assignment needs for PCI devices

Jan Beulich wrote:
>>>> "Cui, Dexuan" <dexuan.cui@xxxxxxxxx> 31.03.09 09:48 >>>
>> Jan Beulich wrote:
>>> Also, being not really knowledgeable about all the differences
>>> between PCIe and PCI, I'd appreciate some clarification on why on
>>> PCIe it is possible and correct to do a secondary bus reset when
>>> targeting just the (PCIe) functions of a single device - to me this
>>> seems to imply that there's a one-device-per-non-root-bus
>>> restriction there. 
>> According to VT-d spec (section 3.6.1: Identifying Origination of
>> DMA Requests), PCI devices behind the same uppermost PCI/PCI-X
>> bridge must be co-assigned to the same guest. PCIe devices have not
>> such a constraint.  
>> FLR (Functional Level Reset) is used to quiesce an assigned device
>> when we destroy a guest or we detach the device from the guest. If a
>> PCIe device has no standard FLR capability, we'll try to use
>> SecondaryBusReset as a way to do FLR. Unluckily, SecondaryBusReset
>> resets all the devices behind the bridge, so for a multi-function
>> PCIe device without FLR capability, we require they have to be
>> co-assigned to the same guest --  certainly this is only for iommu
>> case and for non-iommu case this requirement may be not proper and
>> we shoud fix it...    
> But - for the sake of my education, if you forgive - you didn't
> answer whether there is a one-device-per-secondary-bus restriction on
> PCIe...  

My knowledge to the question is yes, and the "one-device" can be a 
single-function device or a multi-function device.

-- Dexuan

Xen-devel mailing list

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>