On Thu, Mar 03, 2011 at 07:07:48PM +0000, Kinsella, Ray wrote:
> Hi Konrad,
> Thanks for your response.
> My understanding of this solution is that we are essentially switching off
> VT-d and using a software solution instead (bounce buffers?).
> Am I correct?.
There are _two_ swiotlb implementation in the pvops kernel.
The baremetal is to be used under baremetal and it won't be turned
on when running DomU/Dom0.
The other, xen-swiotlb is turned on by default when dom0 boots. And
it is turned on for DomU if the user specified 'iommu=soft'.
The xen-swiotlb has two modes: bounce buffer for devices that need it (which
nowadays is USB), and
translate the PFNs to MFNs (and vice-versa). We want to use the second
which hooks up in the PCI DMA. You can turn on the bounce buffer for everything
if you specify 'swiotlb=force', but there is no point of doing that.
So in essence you have Xen hypervisor doing VT-D, dom0 and domU both using
the Linux DMA API wherein the Xen-SWIOTLB is hooked up. The Xen-SWIOTLB runs
in translation mode and when a device is setup using the DMA API it provides
the MFNs instead of the PFNs to the driver.
> Ray Kinsella
> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@xxxxxxxxxx]
> Sent: Thursday, March 03, 2011 3:03 PM
> To: Kinsella, Ray
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-devel] PCI Passthrough and PV-OPS
> On Thu, Mar 03, 2011 at 02:25:47PM +0000, Kinsella, Ray wrote:
> > Hi all,
> > I am having a problem with Xen 4.0.1, PCI Passthrough with VT-d and Linux
> > PV Guests, and I was wondering if anyone else had seen it.
> > In both Dom 0 and Dom U's I am using 64bit PV-OPS 22.214.171.124 Kernels from
> > Jeremy Fitzhartridge's Git repo.
> Do you have 'iommu=soft' in your guest configuration?
> > I am using dual port SR-IOV NIC's to test, passing through physical
> > functions only.
> > Passthrough appears to work, I can passthrough the NIC and it will appear
> > in the guest with lspci.
> > The guest detects the new device and loads the driver to service the NIC,
> > as you would expect.
> > The guest is complaining about the NIC being hung, the message "Detected Tx
> > Unit Hang..." is appearing in the system log on the guest.
> > In the Xen log, VT-d is producing errors similar to this one;
> > (XEN) [VT-D]iommu.c:824: iommu_fault_status: Primary Pending Fault
> > (XEN) [VT-D]iommu.c:799: DMAR:[DMA Read] Request device [03:00.0] fault
> > addr 7c584000, iommu reg = ffff82c3fff57000
> > (XEN) DMAR:[fault reason 06h] PTE Read access is not set
> > (XEN) print_vtd_entries: iommu = ffff83016fffa5f0 bdf = 3:0.0 gmfn = 7c584
> > (XEN) root_entry = ffff83016ff38000
> > (XEN) root_entry = 14d90a001
> > (XEN) context = ffff83014d90a000
> > (XEN) context = 102_152209005
> > (XEN) l4 = ffff830152209000
> > (XEN) l4_index = 0
> > (XEN) l4 = 152208003
> > (XEN) l3 = ffff830152208000
> > (XEN) l3_index = 1
> > (XEN) l3 = 0
> > (XEN) l3 not present
> > My understanding of what is happening here is that page table mappings are
> > missing from VT-d's page table, mapping machine physical addresses to my PV
> > guest's physical addresses. I have had a look at iommu_populate_page_table
> > and I dumped the mappings as they are being setup but couldn't see a
> > mapping for the GPA 0x7c584000 above?
> > Has anyone else encountered this issue?
> Yes. And it was fixed by running a PV guest with 'iommu=soft' as Linux kernel
> command line argument.
> Intel Shannon Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
> Business address: Dromore House, East Park, Shannon, Co. Clare
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). Any review or distribution by others
> is strictly prohibited. If you are not the intended recipient, please contact
> the sender and delete all copies.
> Xen-devel mailing list
Xen-devel mailing list