This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] PCI Passthrough and PV-OPS

On Thu, Mar 03, 2011 at 07:07:48PM +0000, Kinsella, Ray wrote:
> Hi Konrad,
> Thanks for your response.
> My understanding of this solution is that we are essentially switching off 
> VT-d and using a software solution instead (bounce buffers?).

> Am I correct?.

There are _two_ swiotlb implementation in the pvops kernel.

The baremetal is to be used under baremetal and it won't be turned
on when running DomU/Dom0.

The other, xen-swiotlb is turned on by default when dom0 boots. And
it is turned on for DomU if the user specified 'iommu=soft'.

The xen-swiotlb has two modes: bounce buffer for devices that need it (which 
nowadays is USB), and
translate the PFNs to MFNs (and vice-versa). We want to use the second 
which hooks up in the PCI DMA. You can turn on the bounce buffer for everything
if you specify 'swiotlb=force', but there is no point of doing that.

So in essence you have Xen hypervisor doing VT-D, dom0 and domU both using
the Linux DMA API wherein the Xen-SWIOTLB is hooked up. The Xen-SWIOTLB runs
in translation mode and when a device is setup using the DMA API it provides
the MFNs instead of the PFNs to the driver.

> Ray Kinsella
> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@xxxxxxxxxx] 
> Sent: Thursday, March 03, 2011 3:03 PM
> To: Kinsella, Ray
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-devel] PCI Passthrough and PV-OPS
> On Thu, Mar 03, 2011 at 02:25:47PM +0000, Kinsella, Ray wrote:
> > Hi all,
> > 
> > I am having a problem with Xen 4.0.1, PCI Passthrough with VT-d and Linux 
> > PV Guests, and I was wondering if anyone else had seen it.
> > In both Dom 0 and Dom U's I am using 64bit PV-OPS Kernels from 
> > Jeremy Fitzhartridge's Git repo.
> Do you have 'iommu=soft' in your guest configuration?
> > 
> > I am using dual port SR-IOV NIC's to test, passing through physical 
> > functions only.
> > Passthrough appears to work, I can passthrough the NIC and it will appear 
> > in the guest with lspci.
> > The guest detects the new device and loads the driver to service the NIC, 
> > as you would expect.
> > 
> > The guest is complaining about the NIC being hung, the message "Detected Tx 
> > Unit Hang..." is appearing in the system log on the guest.
> > In the Xen log, VT-d is producing errors similar to this one;
> > 
> > (XEN) [VT-D]iommu.c:824: iommu_fault_status: Primary Pending Fault
> > (XEN) [VT-D]iommu.c:799: DMAR:[DMA Read] Request device [03:00.0] fault 
> > addr 7c584000, iommu reg = ffff82c3fff57000
> > (XEN) DMAR:[fault reason 06h] PTE Read access is not set
> > (XEN) print_vtd_entries: iommu = ffff83016fffa5f0 bdf = 3:0.0 gmfn = 7c584
> > (XEN)     root_entry = ffff83016ff38000
> > (XEN)     root_entry[3] = 14d90a001
> > (XEN)     context = ffff83014d90a000
> > (XEN)     context[0] = 102_152209005
> > (XEN)     l4 = ffff830152209000
> > (XEN)     l4_index = 0
> > (XEN)     l4[0] = 152208003
> > (XEN)     l3 = ffff830152208000
> > (XEN)     l3_index = 1
> > (XEN)     l3[1] = 0
> > (XEN)     l3[1] not present
> > 
> > My understanding of what is happening here is that page table mappings are 
> > missing from VT-d's page table, mapping machine physical addresses to my PV 
> > guest's physical addresses. I have had a look at iommu_populate_page_table 
> > and I dumped the mappings as they are being setup but couldn't see a 
> > mapping for the GPA 0x7c584000 above?
> > 
> > Has anyone else encountered this issue?
> Yes. And it was fixed by running a PV guest with 'iommu=soft' as Linux kernel 
> command line argument.
> --------------------------------------------------------------
> Intel Shannon Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
> Business address: Dromore House, East Park, Shannon, Co. Clare
> This e-mail and any attachments may contain confidential material for the 
> sole use of the intended recipient(s). Any review or distribution by others 
> is strictly prohibited. If you are not the intended recipient, please contact 
> the sender and delete all copies.
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>