[Xen-devel] RE: MSI 890FXA-GD70: AMD IOMMU, VmError: Failed to assign device to IOMMU, (XEN) AMD_IOV: IO_PAGE_FALT: domain:10, device id:0x900, fault address:0x0

["Huang2, Wei" <Wei.Huang2@xxxxxxx>]

[cc Wei Wang here]

In the function reassign_device() in pci_amd_iommu.c, it seems 
pci_get_pdev_by_domain() fail to return a correct pci_dev. Here is the decode 
of IVRS table. It looks like 03:xx.x is behind a bridge 00:14.4 and uses this 
value as an alias ID. That might be a cause. From IVRS, I don't see anything 
wrong with 00:14.2 though. Maybe you can add more debug code to 
reassign_device().

-Wei

02 10 00 00                         DEV=00:2.0
03 00 0f 00 04 01 0f 00             DEV=[0f:1.0, 0f:1.0]
02 18 00 00                         DEV=00:2.0
02 00 0e 00                         DEV=0e:0.0
02 20 00 00                         DEV=00:4.0
02 00 0d 00                         DEV=0d:0.0
02 28 00 00                         DEV=00:5.0
02 00 0c 00                         DEV=0c:0.0
02 30 00 00                         DEV=00:6.0
02 00 0b 00                         DEV=0b:0.0
02 48 00 00                         DEV=00:9.0
02 00 0a 00                         DEV=0a:0.0
02 50 00 00                         DEV=00:a.0
02 00 08 00 00 00 00 00             DEV=08:0.0
43 08 09 00 00 00 09 00 04 ff 09 00 DEV=[09:1.0, 09:1f.7]
02 58 00 00                         DEV=00:b.0
02 00 04 00                         DEV=04:0.0
02 08 05 00                         DEV=05:1.0
02 00 07 00                         DEV=07:0.0
02 10 05 00                         DEV=05:2.0
03 00 06 00 04 00 06 00             DEV=06:0.0
02 88 00 00                         DEV=00:11.0
03 90 00 00 04 92 00 00             DEV=[00:12.0, 00:12.2]
03 98 00 00 04 9a 00 00             DEV=[00:13.0, 00:13.2]
02 a0 00 d7                         DEV=00:14.0
02 a1 00 00                         DEV=00:14.1
02 a2 00 00                         DEV=00:14.2
02 a3 00 00                         DEV=00:14.3
02 a4 00 00                         DEV=00:14.4
43 00 03 00 00 a4 00 00 04 ff 03 00 DEV=[03:0.0, 03:1f.7]
02 a5 00 00                         DEV=00:14.5
02 a8 00 00                         DEV=00:16.0
02 a9 00 00                         DEV=00:16.1
02 00 01 00                         DEV=01:0.0
03 b0 00 00 04 b2 00 00 00 00 00 00 DEV=[00:16.0, 00:16.2]
48 00 00 d7 00 a0 00 02
48 00 00 00 07 01 00 01



-----Original Message-----
From: Sander Eikelenboom [mailto:linux@xxxxxxxxxxxxxx] 
Sent: Tuesday, July 13, 2010 5:07 PM
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Cc: Przywara, Andre; Joerg Roedel; Huang2, Wei; Konrad Rzeszutek Wilk; Jeremy 
Fitzhardinge
Subject: MSI 890FXA-GD70: AMD IOMMU, VmError: Failed to assign device to IOMMU, 
(XEN) AMD_IOV: IO_PAGE_FALT: domain:10, device id:0x900, fault address:0x0

Hi All,

I have bought a MSI 890FXA-GD70, and have been able to enable the IOMMU with a 
beta-bios from MSI.
I succeed in passing through some pci-e devices to domU's (PV), but some other 
devices fail with both PV and HVM domains.

The message is allways the same "Failed to assign device to IOMMU", with some 
added printk's the return code from "iommu.c/assign_device" seems to be 
ffffffed instead of the desired 0.

Devices 00:14.2 and 03:06.* cause the domain(PV and HVM) to fail on creation 
with: Failed to assign device to IOMMU (BDF) Devices 09:01.* and 07:00.0 
passthrough fine ( visible and working in pv domU).

Since it's a beta bios, could it be the IVRS table supplied by the bios is not 
complete and therefore the iommu functions only for some devices ?
With amd_iommu_debug it sometimes also prints an "IO_PAGE_FALT":
         (XEN) [VT-D]iommu.c:144: SEIK: assign_device: domain 14 (0:14.2)!
         (XEN) [VT-D]iommu.c:162: SEIK: assign_device rc: ffffffed !
         (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device (0:14.2) 
failed
         (XEN) AMD_IOV: IO_PAGE_FALT: domain:10, device id:0x900, fault 
address:0x0


--
Sander







Xen version: xen-4.0-testing-pre4 (changeset:   21263:57859775f88f)
Dom0 kernel: 2.6.32.16 pvops (commit 006bfe8f594370f36f09d8e8ab13692d34e88533)

Attached:
- xm info
- xm dmesg
- dmesg
- lspci -vvvknn
- Dump of ACPI IVRS table
- Dump of ACPI DSDT table


serveerstertje:~/iommu-report# xm pci-list-assignable-devices 0000:0e:00.0 
0000:09:01.0
0000:09:01.1
0000:09:01.2
0000:03:06.0
0000:03:06.1
0000:03:06.2
0000:00:14.2


grub menu.lst:
title           xen-4.0.1-rc4-pre.gz / Debian GNU/Linux,  kernel 
2.6.32.16+xen-2.6.32.x-20100713-a
root            (hd0,0)
kernel          /xen-4.0.1-rc4-pre.gz dom0_mem=768M loglvl=all loglvl_guest=all 
com1=115200,8n1 console=com1,vga iommu=1,pv,verbose,amd_iommu_debug
module          /vmlinuz-2.6.32.16+xen-2.6.32.x-20100713-a 
root=/dev/mapper/serveerstertje-root ro earlyprintk=xen max_loop=255 
loop_max_part=63 
xen-pciback.hide=(00:14.2)(03:06.0)(03:06.1)(03:06.2)(07:00.0)(09:01.0)(09:01.1)(09:01.2)(0e:00.0)
 pci=resource_alignment=00:14.2;03:06.0;03:06.1;03:06.2;09:01.0;09:01.1;09:01.2
module          /initrd.img-2.6.32.16+xen-2.6.32.x-20100713-a




lspci:

00:00.0 Host bridge: ATI Technologies Inc RD890 Northbridge only single slot 
PCI-e GFX Hydra part (rev 02)
00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device 5a23 
00:02.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (PCI express 
gpp port B) 00:03.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge 
(PCI express gpp port C) 00:04.0 PCI bridge: ATI Technologies Inc RD890 PCI to 
PCI bridge (PCI express gpp port D) 00:05.0 PCI bridge: ATI Technologies Inc 
RD890 PCI to PCI bridge (PCI express gpp port E) 00:06.0 PCI bridge: ATI 
Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port F) 00:09.0 PCI 
bridge: ATI Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port H) 
00:0a.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (external gfx1 
port A) 00:0b.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (NB-SB 
link) 00:11.0 SATA controller: ATI Technologies Inc SB700/SB800 SATA Controller 
[IDE mode] (rev 40) 00:12.0 USB Controller: ATI Technologies Inc SB700/SB800 
USB OHCI0 Controller
00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller 
00:13.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller
00:13.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller 
00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 41)
00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller (rev 40)
00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller (rev 
40)
00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40)
00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2 Controller 
00:15.0 PCI bridge: ATI Technologies Inc Device 43a0 00:16.0 USB Controller: 
ATI Technologies Inc SB700/SB800 USB OHCI0 Controller
00:16.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller 
00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, 
Athlon64, Sempron] HyperTransport Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, 
Athlon64, Sempron] Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, 
Athlon64, Sempron] DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, 
Athlon64, Sempron] Miscellaneous Control
00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, 
Athlon64, Sempron] Link Control 03:06.0 USB Controller: VIA Technologies, Inc. 
VT82xxxxx UHCI USB 1.1 Controller (rev 62)
03:06.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 
Controller (rev 62)
03:06.2 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 65) 04:00.0 PCI 
bridge: PLX Technology, Inc. Device 8613 (rev ba) 05:01.0 PCI bridge: PLX 
Technology, Inc. Device 8613 (rev ba) 05:02.0 PCI bridge: PLX Technology, Inc. 
Device 8613 (rev ba) 06:00.0 SATA controller: Device 1b4b:9123 (rev 10) 07:00.0 
USB Controller: NEC Corporation Device 0194 (rev 03) 08:00.0 PCI bridge: Texas 
Instruments XIO2000(A)/XIO2200(A) PCI Express-to-PCI Bridge (rev 03) 09:01.0 
USB Controller: NEC Corporation USB (rev 43)
09:01.1 USB Controller: NEC Corporation USB (rev 43)
09:01.2 USB Controller: NEC Corporation USB 2.0 (rev 04) 0a:00.0 USB 
Controller: NEC Corporation Device 0194 (rev 03) 0b:00.0 Ethernet controller: 
Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet 
controller (rev 03) 0c:00.0 Ethernet controller: Realtek Semiconductor Co., 
Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03) 0d:00.0 
FireWire (IEEE 1394): VIA Technologies, Inc. Device 3403 0e:00.0 VGA compatible 
controller: nVidia Corporation GeForce 8400 GS (rev a1) 0f:00.0 VGA compatible 
controller: ATI Technologies Inc Mobility Radeon HD 3450
0f:00.1 Audio device: ATI Technologies Inc RV620 Audio device [Radeon HD 34xx 
Series]


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel