RE: [Xen-devel] Xen 3.4.1-rc8 iommu crash on Optiplex 760 ( Cor

Hi Michael,
After you change your BIOS's VT-d setting from Disabled to Enabled (or from Enabled to Disabled), can you try power cycle your host (unplug the power cord and wait for some time)?
If this works, it would be a BIOS issue.

Thanks,
-- Dexuan

From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Michael Opyr
Sent: 2009年7月25日 2:06
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] Xen 3.4.1-rc8 iommu crash on Optiplex 760 ( Core2 E8400, Q43 / ICH10D )

Xen 3.4.1-rc8 iommu crash on Optiplex 760 ( Core2 E8400, Q43 / ICH10D )

Using the following build procedure from a clean amd64 debian lenny install

On an optiplex 760 with the latest bios (A03)

apt-get install ssh less bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo pciutils-dev mercurial build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev gcc-multilib git-core

hg clone http://xenbits.xensource.com/xen-3.4-testing.hg

cd xen-3.4-testing.hg

make world -j2 debug=y debugger=y

make install -j2 debug=y debugger=y

Xen boots without a problem if iommu is disabled but

if I enable iommu no matter the boot time parameters xen crashes as follows

__ __ _____ _ _ _ ___

\ \/ /___ _ __ |___ /| || | / | _ __ ___( _ )

\ // _ \ '_ \ |_ \| || |_ | |__| '__/ __/ _ \

/ \ __/ | | | ___) |__ _|| |__| | | (_| (_) |

/_/\_\___|_| |_| |____(_) |_|(_)_| |_| \___\___/

(XEN) Xen version 3.4.1-rc8 (***@***) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Thu Jul 23 18:59:18 MDT 2009

(XEN) Latest ChangeSet: Thu Jul 23 09:08:13 2009 +0100 19702:0171f9316e25

(XEN) Command line: console=com1 com1=57600,8n1 iommu=passthrough iommu_inclusive_mapping=1

(XEN) Video information:

(XEN) VGA is text mode 80x25, font 8x16

(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds

(XEN) Disc information:

(XEN) Found 1 MBR signatures

(XEN) Found 1 EDD information structures

(XEN) Xen-e820 RAM map:

(XEN) 0000000000000000 - 000000000009ec00 (usable)

(XEN) 00000000000f0000 - 0000000000100000 (reserved)

(XEN) 0000000000100000 - 00000000cfdffc00 (usable)

(XEN) 00000000cfdffc00 - 00000000cfe53c00 (ACPI NVS)

(XEN) 00000000cfe53c00 - 00000000cfe55c00 (ACPI data)

(XEN) 00000000cfe55c00 - 00000000d0000000 (reserved)

(XEN) 00000000e0000000 - 00000000f0000000 (reserved)

(XEN) 00000000fec00000 - 00000000fed00400 (reserved)

(XEN) 00000000fed20000 - 00000000feda0000 (reserved)

(XEN) 00000000fee00000 - 00000000fef00000 (reserved)

(XEN) 00000000ffb00000 - 0000000100000000 (reserved)

(XEN) 0000000100000000 - 0000000128000000 (usable)

(XEN) System RAM: 3965MB (4060788kB)

(XEN) ACPI: RSDP 000FEC00, 0024 (r2 DELL )

(XEN) ACPI: XSDT 000FC7EF, 0074 (r1 DELL B10K 15 ASL 61)

(XEN) ACPI: FACP 000FC91F, 00F4 (r3 DELL B10K 15 ASL 61)

(XEN) ACPI: DSDT FFF1BE4D, 53BE (r1 DELL dt_ex 1000 INTL 20050624)

(XEN) ACPI: FACS CFDFFC00, 0040

(XEN) ACPI: SSDT FFF2132A, 00AA (r1 DELL st_ex 1000 INTL 20050624)

(XEN) ACPI: APIC 000FCA13, 0092 (r1 DELL B10K 15 ASL 61)

(XEN) ACPI: BOOT 000FCAA5, 0028 (r1 DELL B10K 15 ASL 61)

(XEN) ACPI: ASF! 000FCACD, 0096 (r32 DELL B10K 15 ASL 61)

(XEN) ACPI: MCFG 000FCB63, 003E (r1 DELL B10K 15 ASL 61)

(XEN) ACPI: HPET 000FCBA1, 0038 (r1 DELL B10K 15 ASL 61)

(XEN) ACPI: TCPA 000FCDFD, 0032 (r1 DELL B10K 15 ASL 61)

(XEN) ACPI: DMAR 000FCE2F, 00D8 (r1 DELL B10K 15 ASL 61)

(XEN) ACPI: SLIC 000FCBD9, 0176 (r1 DELL B10K 15 ASL 61)

(XEN) NUMA turned off

(XEN) Faking a node at 0000000000000000-0000000128000000

(XEN) Domain heap initialised

(XEN) found SMP MP-table at 000fe710

(XEN) DMI 2.5 present.

(XEN) Using APIC driver default

(XEN) ACPI: PM-Timer IO Port: 0x808

(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[804,0], pm1x_evt[800,0]

(XEN) ACPI: wakeup_vec[cfdffc0c], vec_size[20]

(XEN) ACPI: Local APIC address 0xfee00000

(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)

(XEN) Processor #0 7:7 APIC version 20

(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)

(XEN) Processor #1 7:7 APIC version 20

(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x05] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x07] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x00] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x01] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x02] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x03] disabled)

(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high level lint[0x1])

(XEN) ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])

(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23

(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)

(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)

(XEN) ACPI: IRQ0 used by override.

(XEN) ACPI: IRQ2 used by override.

(XEN) ACPI: IRQ9 used by override.

(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs

(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000

(XEN) [VT-D]dmar.c:485: Host address width 36

(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD

(XEN) [VT-D]dmar.c:349: dmaru->address = fedc1000

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0

(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD

(XEN) [VT-D]dmar.c:349: dmaru->address = fedc3000

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3

(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD

(XEN) [VT-D]dmar.c:349: dmaru->address = fedc4000

(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL

(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2

(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7

(XEN) Intel VT-d DMAR tables have been parsed.

(XEN) Using ACPI (MADT) for SMP configuration information

(XEN) Using scheduler: SMP Credit Scheduler (credit)

(XEN) Initializing CPU#0

(XEN) Detected 2992.547 MHz processor.

(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K

(XEN) CPU: L2 cache: 6144K

(XEN) CPU: Physical Processor ID: 0

(XEN) CPU: Processor Core ID: 0

(XEN) VMX: Supported advanced features:

(XEN) - APIC MMIO access virtualisation

(XEN) - APIC TPR shadow

(XEN) - Virtual NMI

(XEN) - MSR direct-access bitmap

(XEN) HVM: VMX enabled

(XEN) Intel machine check reporting enabled on CPU#0.

(XEN) CPU0: Thermal monitoring enabled (TM2)

(XEN) CMCI: CPU0 has no CMCI support

(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz stepping 0a

(XEN) Booting processor 1/1 eip 8c000

(XEN) Initializing CPU#1

(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K

(XEN) CPU: L2 cache: 6144K

(XEN) CPU: Physical Processor ID: 0

(XEN) CPU: Processor Core ID: 1

(XEN) Intel machine check reporting enabled on CPU#1.

(XEN) CPU1: Thermal monitoring enabled (TM2)

(XEN) CMCI: CPU1 has no CMCI support

(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz stepping 0a

(XEN) Total of 2 processors activated.

(XEN) ENABLING IO-APIC IRQs

(XEN) -> Using new ACK method

(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1

(XEN) checking TSC synchronization across 2 CPUs: passed.

(XEN) Platform timer is 14.318MHz HPET

(XEN) Brought up 2 CPUs

(XEN) microcode.c:73:d32767 microcode: CPU1 resumed

(XEN) Intel VT-d Snoop Control supported.

(XEN) Intel VT-d DMA Passthrough supported.

(XEN) Intel VT-d Queued Invalidation supported.

(XEN) Intel VT-d Interrupt Remapping supported.

(XEN) ----[ Xen-3.4.1-rc8 x86_64 debug=y Not tainted ]----

(XEN) CPU: 0

(XEN) RIP: e008:[<ffff828c8012db81>] clear_fault_bits+0x2d/0xb3

(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor

(XEN) rax: ffff828bfff5aff8 rbx: 0000000000004090 rcx: 0000000000000001

(XEN) rdx: 00000000ffffffff rsi: 0000000000000286 rdi: ffff830127fc80a0

(XEN) rbp: ffff828c80267db8 rsp: ffff828c80267db8 r8: ffff828bfff57000

(XEN) r9: ffff828bfff5700c r10: ffff828bfff57008 r11: 0000000000000020

(XEN) r12: ffff830127fc80a0 r13: 0000000000000090 r14: ffff830127fc80e0

(XEN) r15: ffff830127ff1540 cr0: 000000008005003b cr4: 00000000000026f0

(XEN) cr3: 00000000cfa6c000 cr2: ffff828bfff5aff8

(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008

(XEN) Xen stack trace from rsp=ffff828c80267db8:

(XEN) ffff828c80267df8 ffff828c8012e6f2 ffff828c80267df8 0000000000000002

(XEN) ffff830127fc8240 ffff828c80216e90 ffff828c80267e18 ffff828c80216ec0

(XEN) ffff828c80267e48 ffff828c8012effc ffff828c80267e38 ffff828c8012a456

(XEN) 000000000000001f 00000000ffffffed ffff828c80239e10 0000000000000001

(XEN) ffff828c80218a60 ffff828c80218960 ffff828c80267e68 ffff828c8012bb55

(XEN) ffff828c80267e68 ffff828c80239cf8 ffff828c80267f18 ffff828c8022d87c

(XEN) 0000000000000000 0000000000000000 000000000008bf40 0000000000000000

(XEN) 0000000000000000 ffff83000008bfc0 ffff83000008bf40 000000000163d108

(XEN) 0000000000000000 ffffffff00000000 0000000800000000 000000010000006e

(XEN) 0000000000000003 00000000000002f8 0000000000000000 0000000000000000

(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000

(XEN) 0000000000067e9c ffff828c801000b5 0000000000000000 0000000000000000

(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000

(XEN) ffff8300ce1c0000

(XEN) Xen call trace:

(XEN) [<ffff828c8012db81>] clear_fault_bits+0x2d/0xb3

(XEN) [<ffff828c8012e6f2>] init_vtd_hw+0x2d7/0x3e9

(XEN) [<ffff828c8012effc>] intel_vtd_setup+0x51e/0x638

(XEN) [<ffff828c8012bb55>] iommu_setup+0x38/0xfd

(XEN) [<ffff828c8022d87c>] __start_xen+0x2456/0x273a

(XEN)

(XEN) Pagetable walk from ffff828bfff5aff8:

(XEN) L4[0x105] = 00000000cfa72027 5555555555555555

(XEN) L3[0x02f] = 0000000127ffa063 5555555555555555

(XEN) L2[0x1ff] = 0000000127ff9063 5555555555555555

(XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff

(XEN)

(XEN) ****************************************

(XEN) Panic on CPU 0:

(XEN) FATAL PAGE FAULT

(XEN) [error_code=0000]

(XEN) Faulting linear address: ffff828bfff5aff8

(XEN) ****************************************

(XEN)

(XEN) Reboot in five seconds...

Various google searches yielded a similar seeming issue previously posted

http://www.nabble.com/Re:-Problem:-Xen-3.4-fails-to-boot-on-Dell-e6400,-with--iommu%3D1-p24250643.html

from the posts I attempted the described patches,

the first (print clear fault bits) yields the following change

(XEN) Intel VT-d Snoop Control supported.

(XEN) Intel VT-d DMA Passthrough supported.

(XEN) Intel VT-d Queued Invalidation supported.

(XEN) Intel VT-d Interrupt Remapping supported.

(XEN) clear_fault_bits: CAP_REG=0xffffffffffffffff

(XEN) ----[ Xen-3.4.1-rc8 x86_64 debug=y Not tainted ]----

(XEN) CPU: 0

(XEN) RIP: e008:[<ffff828c8012f000>] clear_fault_bits+0x5f/0xea

(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor

(XEN) rax: ffff828bfff5aff8 rbx: ffff830127ff1e90 rcx: 0000000000000000

(XEN) rdx: 00000000ffffffff rsi: 000000000000000a rdi: ffff828bfff57000

(XEN) rbp: ffff828c80267db8 rsp: ffff828c80267da8 r8: ffff828bfff5700c

(XEN) r9: ffff828bfff57008 r10: 00000000ffffffed r11: 0000000000000010

(XEN) r12: ffff830127ff1e90 r13: 0000000000000090 r14: ffff830127ff1ed0

(XEN) r15: ffff830127ff1540 cr0: 000000008005003b cr4: 00000000000026f0

(XEN) cr3: 00000000cfa6c000 cr2: ffff828bfff5aff8

(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008

(XEN) Xen stack trace from rsp=ffff828c80267da8:

(XEN) ffff830127ff1e90 0000000000004090 ffff828c80267df8 ffff828c8012fa2a

(XEN) ffff828c80267df8 0000000000000002 ffff830127fe0070 ffff828c8021c690

(XEN) ffff828c80267e18 ffff828c8021c6c0 ffff828c80267e48 ffff828c80130334

(XEN) ffff828c80267e38 ffff828c8012b756 000000000000001f 00000000ffffffed

(XEN) ffff828c8023ee30 0000000000002000 ffff828c8021e260 ffff828c8021e160

(XEN) ffff828c80267e68 ffff828c8012ce55 ffff828c80267e68 ffff828c8023ed18

(XEN) ffff828c80267f18 ffff828c802328ac 0000000000000000 0000000000000000

(XEN) 000000000008bf10 0000000000000000 0000000000000000 ffff83000008bfc0

(XEN) ffff83000008bf10 000000000169eb28 0000000000000000 ffffffff00000000

(XEN) 0000000800000000 000000010000006e 0000000000000003 00000000000002f8

(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000

(XEN) 0000000000000000 0000000000000000 0000000000067e9c ffff828c801000b5

(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000

(XEN) 0000000000000000 0000000000000000 ffff8300cf8fc000

(XEN) Xen call trace:

(XEN) [<ffff828c8012f000>] clear_fault_bits+0x5f/0xea

(XEN) [<ffff828c8012fa2a>] init_vtd_hw+0x2d7/0x3e9

(XEN) [<ffff828c80130334>] intel_vtd_setup+0x51e/0x638

(XEN) [<ffff828c8012ce55>] iommu_setup+0x38/0xfd

(XEN) [<ffff828c802328ac>] __start_xen+0x2456/0x273a

(XEN)

(XEN) Pagetable walk from ffff828bfff5aff8:

(XEN) L4[0x105] = 00000000cfa72027 5555555555555555

(XEN) L3[0x02f] = 0000000127ffa063 5555555555555555

(XEN) L2[0x1ff] = 0000000127ff9063 5555555555555555

(XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff

(XEN)

(XEN) ****************************************

(XEN) Panic on CPU 0:

(XEN) FATAL PAGE FAULT

(XEN) [error_code=0000]

(XEN) Faulting linear address: ffff828bfff5aff8

(XEN) ****************************************

(XEN)

(XEN) Reboot in five seconds...

And the second patch from the post (do not clear fault bits)

Gets us the following

(XEN) Intel VT-d Interrupt Remapping supported.

(XEN) DMAR_IQA_REG = ffffffffffffffff

(XEN) DMAR_IQH_REG = ffffffffffffffff

(XEN) DMAR_IQT_REG = ffffffffffffffff

(XEN)

(XEN) ****************************************

(XEN) Panic on CPU 0:

(XEN) queue invalidate wait descriptor was not executed

(XEN) ****************************************

(XEN)

(XEN) Reboot in five seconds...

Any ideas would be wonderfull as I would like to use passthrough with a windows hvm

Thank you

--Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

WARNING - OLD ARCHIVES

xen-devel

RE: [Xen-devel] Xen 3.4.1-rc8 iommu crash on Optiplex 760 ( Core2 E8400