RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, wi

When you meet with the panic, can you apply the printk log to print CAP_REG?

In your step 1 and 4, you just mean using the xen parameter iommu=0 and iommu=1, correct?

During theses steps, you don't change any BIOS setting and there is not any HW change, correct?

I see you're using "3.4.0 19629:b2128a88e3dd", but 19629's checksum should be 69293fc2096f (http://xenbits.xensource.com/xen-3.4-testing.hg?rev/69293fc2096f).

So I assume you made your own changes to the code?

Can you try the latest changeset 19677: b2ddeb734ca7(http://xenbits.xensource.com/xen-3.4-testing.hg)?

Sorry, I don't have the same type of host, so I can't try it myself. :-(

Thanks,
-- Dexuan

From: Tom Rotenberg [mailto:tom.rotenberg@xxxxxxxxx]
Sent: 2009年6月29日 21:09
To: Cui, Dexuan
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1

Well, it looks like my analysis was right.
This is how i reproduced it:
1. Disable the IOMMU
2. Turn off the laptop, and leave it turned off for a few seconds
3. Turn on the laptop
4. Enable the IOMMU (*DON'T* turn off the machine)
5. Continue
6. Xen crashes

and, now the log lines are:
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.

It looks like there is a problem when enabling IOMMU, without cold boot of the machine.

What do u say?

Here is the full log:
__ __            _____ _ _    ___
\ \/ /___ _ __   |___ /| || | / _ \
\ // _ \ '_ \    |_ \| || |_| | | |
/ \ __/ | | | ___) |__   _| |_| |
/_/\_\___|_| |_| |____(_) |_|(_)___/

(XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009
(XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd
(XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f000 (usable)
(XEN) 000000000009f000 - 00000000000a0000 (reserved)
(XEN) 0000000000100000 - 000000007c04d400 (usable)
(XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS)
(XEN) 000000007c04f400 - 000000007f000000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fed18000 - 00000000fed1c000 (reserved)
(XEN) 00000000fed20000 - 00000000fed90000 (reserved)
(XEN) 00000000feda0000 - 00000000feda6000 (reserved)
(XEN) 00000000fee00000 - 00000000fee10000 (reserved)
(XEN) 00000000ffe60000 - 0000000100000000 (reserved)
(XEN) System RAM: 1983MB (2031536kB)
(XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL )
(XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx     1001 INTL 20050624)
(XEN) ACPI: FACS 7C060C00, 0040
(XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL    M09            1 ASL        61)
(XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: APIC 7C052000, 0068 (r1 DELL    M09     27D9020D ASL        47)
(XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: TCPA 7C052300, 0032 (r1                        0 ASL         0)
(XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef    CpuPm     3000 INTL 20050624)
(XEN) NUMA turned off
(XEN) Faking a node at 0000000000000000-000000007c04d000
(XEN) Domain heap initialised
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0]
(XEN) ACPI:                  wakeup_vec[7c060c0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) [VT-D]dmar.c:485: Host address width 36
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed10000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed13000
(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
(XEN) Intel VT-d DMAR tables have been parsed.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2527.037 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) HVM: VMX enabled
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank3
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU     T9400 @ 2.53GHz stepping 06
(XEN) Booting processor 1/1 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU     T9400 @ 2.53GHz stepping 06
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.
(XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor
(XEN) rax: ffff828bfff5aff8   rbx: ffff830078f82f90   rcx: 0000000000000020
(XEN) rdx: 00000000ffffffff   rsi: ffff828bfff57000   rdi: ffff830078f82f90
(XEN) rbp: 0000000000000090   rsp: ffff828c8026fda0   r8: ffff83007be2c110
(XEN) r9: 000000000000001a   r10: 0000000000000005   r11: 0000000000000005
(XEN) r12: 0000000000000090   r13: 0000000000000090   r14: ffff830078f82fcc
(XEN) r15: 0000000000004800   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 000000007be74000   cr2: ffff828bfff5aff8
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen stack trace from rsp=ffff828c8026fda0:
(XEN)    ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490
(XEN)    ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000
(XEN)    ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f
(XEN)    00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60
(XEN)    ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 ffff828c80227978
(XEN)    ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    00000000002b23c0 ffffff8c80274000 0008bf3000000000 000000000008bf30
(XEN)    0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000
(XEN)    ffffffff00000000 0000000800000000 000000010000006e 0000000000000003
(XEN)    00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffff830079f18000
(XEN) Xen call trace:
(XEN)    [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN)    [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0
(XEN)    [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610
(XEN)    [<ffff828c8012a9d4>] iommu_setup+0x24/0x110
(XEN)    [<ffff828c8021a589>] __start_xen+0x1859/0x27c0
(XEN)    [<ffff828c801000b5>] __high_start+0xa1/0xa3
(XEN)
(XEN) Pagetable walk from ffff828bfff5aff8:
(XEN) L4[0x105] = 000000007be7a027 5555555555555555
(XEN) L3[0x02f] = 000000007c04b063 5555555555555555
(XEN) L2[0x1ff] = 000000007c04a063 5555555555555555
(XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: ffff828bfff5aff8
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

2009/6/29 Tom Rotenberg <tom.rotenberg@xxxxxxxxx>

I'm almost 100% sure, that this was from the same machine (since i have only 1 such Dell currently on my desk :).
However, this log was generated just after turning on the IOMMU (without cold boot). So, let me check if it happened because of this.

I'll check it now...

2009/6/29 Cui, Dexuan <dexuan.cui@xxxxxxxxx>

I have no idea about it.

> (XEN) Intel VT-d Snoop Control supported.
> (XEN) Intel VT-d DMA Passthrough not supported.
> (XEN) Intel VT-d Queued Invalidation supported.
> (XEN) Intel VT-d Interrupt Remapping supported.

This is from the log of your original mail. We can see SC/QI/IR are supported, but in your later log, we can see they are not supported.

I'm curious where the first log comes from. :-)

-- Dexuan

From: Tom Rotenberg [mailto:tom.rotenberg@xxxxxxxxx]
Sent: 2009年6月29日 20:38

To: Cui, Dexuan
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1

1) Yep, i can't reproduce it anymore. Maybe i'll try it with another Dell R6400 i have here.
2) On my Lenovo T400, i don't remember the exact crash, and i don't have the log of that crash, i just remember something about "DMAR ... failure", which happened, after i turned on the IOMMU, and didn't perform a cold reboot on that machine.

Do u have any idea why these problems happen?
Does anyone else working with Dell E6400 had this problem?

Tom

2009/6/29 Cui, Dexuan <dexuan.cui@xxxxxxxxx>

It's really weird... :-)

If I understand correctly, with iommu=1 in Xen:

1) on your "Dell E6400 machine", now you can't reproduce the panic any longer;

2) on your "Lenovo T400", you can reproduce the panic sometimes -- if so, can you post the log (with the printk patch applied) when you meet with the panic? Can you find out under what condition it's easy to reproduce the panic?

Thanks,
-- Dexuan

From: Tom Rotenberg [mailto:tom.rotenberg@xxxxxxxxx]
Sent: 2009年6月29日 20:15

To: Cui, Dexuan
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1

Hi,

Tried again, this time with cold reboot, it worked again. I also checked the lines you were talking about, and now they are:
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.

Could it be that there is some kind of problem with the hardware, which makes it sometimes report invalid values? (with the Lenovo T400 i'm testing, if you enable the IOMMU, without a cold-reboot, it may cause Xen to crash on boot, so you must do a cold reboot).

Here is the log:
__ __            _____ _ _    ___
\ \/ /___ _ __   |___ /| || | / _ \
\ // _ \ '_ \    |_ \| || |_| | | |
/ \ __/ | | | ___) |__   _| |_| |
/_/\_\___|_| |_| |____(_) |_|(_)___/

(XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009
(XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd
(XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f000 (usable)
(XEN) 000000000009f000 - 00000000000a0000 (reserved)
(XEN) 0000000000100000 - 000000007c04d400 (usable)
(XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS)
(XEN) 000000007c04f400 - 000000007f000000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fed18000 - 00000000fed1c000 (reserved)
(XEN) 00000000fed20000 - 00000000fed90000 (reserved)
(XEN) 00000000feda0000 - 00000000feda6000 (reserved)
(XEN) 00000000fee00000 - 00000000fee10000 (reserved)
(XEN) 00000000ffe60000 - 0000000100000000 (reserved)
(XEN) System RAM: 1983MB (2031536kB)
(XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL )
(XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx     1001 INTL 20050624)
(XEN) ACPI: FACS 7C060C00, 0040
(XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL    M09            1 ASL        61)
(XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: APIC 7C052000, 0068 (r1 DELL    M09     27D9020D ASL        47)
(XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: TCPA 7C052300, 0032 (r1                        0 ASL         0)
(XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef    CpuPm     3000 INTL 20050624)
(XEN) NUMA turned off
(XEN) Faking a node at 0000000000000000-000000007c04d000
(XEN) Domain heap initialised
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0]
(XEN) ACPI:                  wakeup_vec[7c060c0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) [VT-D]dmar.c:485: Host address width 36
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed10000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed11000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed12000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed13000
(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) Intel VT-d DMAR tables have been parsed.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2527.058 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) HVM: VMX enabled
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank3
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU     T9400 @ 2.53GHz stepping 06
(XEN) Booting processor 1/1 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU     T9400 @ 2.53GHz stepping 06
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) clear_fault_bits: CAP_REG=0xc0000020630270
(XEN) clear_fault_bits: CAP_REG=0xc0000020630260
(XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
(XEN) clear_fault_bits: CAP_REG=0xc9008020630260
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.:   0000000076000000->0000000078000000 (460259 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48
(XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400
(XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18
(XEN) Start info:    ffffffff81f28000->ffffffff81f284b4
(XEN) Page tables:   ffffffff81f29000->ffffffff81f3c000
(XEN) Boot stack:    ffffffff81f3c000->ffffffff81f3d000
(XEN) TOTAL:         ffffffff80000000->ffffffff82000000
(XEN) ENTRY ADDRESS: ffffffff80200000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 132kB init memory.

Tom

2009/6/29 Cui, Dexuan <dexuan.cui@xxxxxxxxx>

So, if I understand correctly, it's really strange you can't reproduce the panic now.

With only the first printk patch applied (and don't comment the clear_fault_bits()), can you try a cold rebooting the host (plug the power cord off from the host; wait several minutes; plug the cord in and tunn on the host)? And the log?

BTW, in your original mail, the log shows:

(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.

But in your this new log, it shows:

(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.

Are you sure you have been operating on the same host? :-)

-- Dexuan

From: Tom Rotenberg [mailto:tom.rotenberg@xxxxxxxxx]
Sent: 2009年6月29日 17:01
To: Cui, Dexuan
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1

Hi,

Well, just tried what you suggested, and here are the results:
* If i comment the 'clear_fault_bits()' call - then Xen boots ok, with iommu=1
The strange thing, is that after it booted one time ok (with the second patch), then it continued to boot ok, even if i reverted the second patch.

The relevant lines in the log are:
(XEN) clear_fault_bits: CAP_REG=0xc0000020630270
(XEN) clear_fault_bits: CAP_REG=0xc0000020630260
(XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
(XEN) clear_fault_bits: CAP_REG=0xc9008020630260

Anyway, here is the full Xen log, with the first patch (print cap's) applied:

__ __            _____ _ _    ___
\ \/ /___ _ __   |___ /| || | / _ \
\ // _ \ '_ \    |_ \| || |_| | | |
/ \ __/ | | | ___) |__   _| |_| |
/_/\_\___|_| |_| |____(_) |_|(_)___/

(XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009
(XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd
(XEN) Command line: console=com1 com1=115200,8n1 iommu=1 iommu_inclusive_mapping loglvl=all guest_loglvl=all
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f000 (usable)
(XEN) 000000000009f000 - 00000000000a0000 (reserved)
(XEN) 0000000000100000 - 000000007c04d400 (usable)
(XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS)
(XEN) 000000007c04f400 - 000000007f000000 (reserved)
(XEN) 00000000f8000000 - 00000000fc000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fed18000 - 00000000fed1c000 (reserved)
(XEN) 00000000fed20000 - 00000000fed90000 (reserved)
(XEN) 00000000feda0000 - 00000000feda6000 (reserved)
(XEN) 00000000fee00000 - 00000000fee10000 (reserved)
(XEN) 00000000ffe60000 - 0000000100000000 (reserved)
(XEN) System RAM: 1983MB (2031536kB)
(XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL )
(XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx     1001 INTL 20050624)
(XEN) ACPI: FACS 7C060C00, 0040
(XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL    M09            1 ASL        61)
(XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: APIC 7C052000, 0068 (r1 DELL    M09     27D9020D ASL        47)
(XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: TCPA 7C052300, 0032 (r1                        0 ASL         0)
(XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL    M09     27D9020D ASL        61)
(XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef    CpuPm     3000 INTL 20050624)
(XEN) NUMA turned off
(XEN) Faking a node at 0000000000000000-000000007c04d000
(XEN) Domain heap initialised
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0]
(XEN) ACPI:                  wakeup_vec[7c060c0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 7:7 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 7:7 APIC version 20
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) [VT-D]dmar.c:485: Host address width 36
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed10000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed11000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed12000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed13000
(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
(XEN) Intel VT-d DMAR tables have been parsed.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2527.068 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) HVM: VMX enabled
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank3
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU     T9400 @ 2.53GHz stepping 06
(XEN) Booting processor 1/1 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 6144K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) mce_init: init bank1
(XEN) mce_init: init bank2
(XEN) mce_init: init bank4
(XEN) mce_init: init bank5
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU     T9400 @ 2.53GHz stepping 06
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) clear_fault_bits: CAP_REG=0xc0000020630270
(XEN) clear_fault_bits: CAP_REG=0xc0000020630260
(XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
(XEN) clear_fault_bits: CAP_REG=0xc9008020630260
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
(XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff57000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff56000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff55000
(XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg = ffff828bfff54000
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8070ab48
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.:   0000000076000000->0000000078000000 (460259 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48
(XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400
(XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18
(XEN) Start info:    ffffffff81f28000->ffffffff81f284b4
(XEN) Page tables:   ffffffff81f29000->ffffffff81f3c000
(XEN) Boot stack:    ffffffff81f3c000->ffffffff81f3d000
(XEN) TOTAL:         ffffffff80000000->ffffffff82000000
(XEN) ENTRY ADDRESS: ffffffff80200000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 132kB init memory.

Tom

2009/6/29 Cui, Dexuan <dexuan.cui@xxxxxxxxx>

diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800
@@ -1693,7 +1693,7 @@ static int init_vtd_hw(void)
}
dma_msi_data_init(iommu, iommu->vector);
dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map)));
- clear_fault_bits(iommu);
+ //clear_fault_bits(iommu);
dmar_writel(iommu->reg, DMAR_FECTL_REG, 0);

/* initialize flush functions */

And, can you comment the invocation of clear_fault_bits() in init_vtd_hw() like above and see if the host can boot fine?

Thanks,
-- Dexuan

-----Original Message-----
From: Cui, Dexuan
Sent: 2009?6?29? 14:30
To: 'Tom Rotenberg'; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1

diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800
@@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm
{
u64 val;

+ printk("clear_fault_bits: CAP_REG=0x%llx\n",
+ (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG));
val = dmar_readq(
iommu->reg,
cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8);

Hi Tom,
Can you please apply the above patch and send us the entire log?
(I don't have the same host, so I can't try it myself.)

Thanks,
-- Dexuan

________________________________

From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Tom Rotenberg
Sent: 2009?6?28? 22:56
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu=1

Hi,

I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to boot xen with a 'iommu=1' parameter in the boot line, Xen fails to boot. I have also tried to boot with and without the 'iommu_inclusive_mapping' boot option, but it didn't seem to help at all.

Does anyone knows how to solve this issue?

These are the last messages in the UART:

(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush
(XEN) Intel VT-d Snoop Control supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.
(XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
(XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020
(XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90
(XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110
(XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005
(XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc
(XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff828c8026fda0:
(XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490
(XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000
(XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f
(XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60
(XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978
(XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40
(XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000
(XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003
(XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000
(XEN) Xen call trace:
(XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
(XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0
(XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610
(XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110
(XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0
(XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3
(XEN)
(XEN) Pagetable walk from ffff828bfff5aff8:
(XEN) L4[0x105] = 000000007be7a027 5555555555555555
(XEN) L3[0x02f] = 000000007c04b063 5555555555555555
(XEN) L2[0x1ff] = 000000007c04a063 5555555555555555
(XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: ffff828bfff5aff8
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

WARNING - OLD ARCHIVES

xen-devel

RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with iommu