Hmm i remember experiencing also problems with not cold booting after
changing the VT-D setting in BIOS. Since then i allways cold boot after
changing the setting (which is not that often)
Tuesday, June 30, 2009, 4:54:40 AM, you wrote:
>> CAP_REG=0xffffffffffffffff
> HW obviously doesn't report the value correctly.
> So, looks the issue is: when we change the iommu option in BIOS (e.g.,
> from disabled to enabled), to make it really take effect, we have to make
> a "cold shutdown (shutdown the power supply throughly)" to the host ; or
> else, though Xen can see the ACPI DMAR tables，the HW doesn't respond to
> the MMIO accesses to the iommu registers at all -- such read access would
> return -1 (all-'1's) and write access would be ignored.
> I remember you said after you commented the clear_fault_bits(), the host
> could boot fine with iommu=1 --- I guess in this case actually the DMA
> remapping is not enabled at all though Xen thinks HW tells it DMA
> remapping is enabled (Xen writes DMA_GCMD_TE into GCMD (this write is
> ignored) and polls the GSTS; since reading GSTS returns -1, Xen thinks
> TE is on, however the underlying HW doesn't enable DMA remapping!).
> Please note: this is only my deduction, anyway. :-) You can try to
> create some hvm guests with devices assigned, I think the host would die
> weirdly since the DMAs of the assigned devices of the hvm guests are
> corrupting the host memory! This can be a way to verify my deduction.
> And I remember you said after you commented the clear_fault_bits() and
> booted the host fine, you tried to uncomment the line and the host still
> boots fine! -- this is strange to me, I guess you might have done a cold
> shutdown before your uncommenting the line?
> One more thing: I think your HW doesn't support
> SnoopControl/QueuedInvalidation/InterruptRemapping, but after you
> commented the clear_fault_bits(), LOOKS the host could boot fine even
> after Xen tries to enable them! Maybe the specific HW implementation of your
> host could account for this...
> I think changing the iommu option in BIOS should not require a cold
> shutdown to the host. I'll try to raise this issue to the BIOS people.
> For now, you could work around the issue by remember doing a cold
> shutdown every time you change the iommu BIOS option. Hope this is acceptable
> to you for now. :-)
> Thanks,
> -- Dexuan
> ________________________________
> From: Tom Rotenberg [mailto:tom.rotenberg@xxxxxxxxx]
> Sent: 2009年6月29日 22:15
> To: Cui, Dexuan
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
> iommu=1
> in steps 1 & 4, i mean disabling and enabling IOMMU in the BIOS (and not
> in Xen boot parameter). And, this is what probably caused all of this.
> I am using my own version (above Xen 3.4.0), but i haven't made any
> changes in the hyervisor itself (but rather in the hvmloader and
> rombios), so it's not related to this. I prefer not to try the latest
> Xen, unless there is a very good reason for this, as it take long time to
> compile, and i can't do it right now.
> I have reproduced the panic with the print, and here is the relevant part of
> the log:
> (XEN) clear_fault_bits: CAP_REG=0xffffffffffffffff
> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]----
> (XEN) CPU: 0
> (XEN) RIP: e008:[<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0
> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000000
> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff828c801f0564
> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fd98 r8: 0000000000004000
> (XEN) r9: 0000000000003fff r10: ffff828c80288360 r11: 0000000080287ea0
> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc
> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0
> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> (XEN) Xen stack trace from rsp=ffff828c8026fd98:
> (XEN) ffff830078f82f90 ffff828c8012d35d ffff830078f824b0 ffff828c801e4257
> (XEN) ffff828c80204490 ffff828c801e4257 ffff828c802044c0 ffff828c8026fdf8
> (XEN) 0000000000000000 ffff828c8012deae ffff830078f82d40 ffff828c80228100
> (XEN) 000000000000001f 00000000ffffffed ffff828c80227a78 0000000000000001
> (XEN) ffff828c80205f60 ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4
> (XEN) ffff828c80227978 ffff828c8021a589 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 00000000002b23c0 ffffff8c80274000 0008bf3000000000
> (XEN) 000000000008bf30 0000000000000000 ffff83000008bfc0 0000000001ce08a0
> (XEN) 0000000000000000 ffffffff00000000 0000000800000000 000000010000006e
> (XEN) 0000000000000003 00000000000002f8 0000000000000000 0000000000000000
> (XEN) 0000000000067e9c 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 ffff828c801000b5 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) ffff830079f18000
> (XEN) Xen call trace:
> (XEN) [<ffff828c8012cac6>] clear_fault_bits+0x46/0xd0
> (XEN) [<ffff828c8012d35d>] init_vtd_hw+0x15d/0x3e0
> (XEN) [<ffff828c8012deae>] intel_vtd_setup+0x5ae/0x610
> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110
> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0
> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3
> (XEN)
> (XEN) Pagetable walk from ffff828bfff5aff8:
> (XEN) L4[0x105] = 000000007be7a027 5555555555555555
> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555
> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555
> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0000]
> (XEN) Faulting linear address: ffff828bfff5aff8
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
> 2009/6/29 Cui, Dexuan <dexuan.cui@xxxxxxxxx<mailto:dexuan.cui@xxxxxxxxx>>
> When you meet with the panic, can you apply the printk log to print CAP_REG?
> In your step 1 and 4, you just mean using the xen parameter iommu=0 and
> iommu=1, correct?
> During theses steps, you don't change any BIOS setting and there is not any
> HW change, correct?
> I see you're using "3.4.0 19629:b2128a88e3dd", but 19629's checksum
> should be 69293fc2096f
> (http://xenbits.xensource.com/xen-3.4-testing.hg?rev/69293fc2096f).
> So I assume you made your own changes to the code?
> Can you try the latest changeset 19677:
> b2ddeb734ca7(http://xenbits.xensource.com/xen-3.4-testing.hg)?
> Sorry, I don't have the same type of host, so I can't try it myself. :-(
> Thanks,
> -- Dexuan
> ________________________________
> From: Tom Rotenberg
> [mailto:tom.rotenberg@xxxxxxxxx<mailto:tom.rotenberg@xxxxxxxxx>]
> Sent: 2009年6月29日 21:09
> To: Cui, Dexuan
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx<mailto:xen-devel@xxxxxxxxxxxxxxxxxxx>
> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
> iommu=1
> Well, it looks like my analysis was right.
> This is how i reproduced it:
> 1. Disable the IOMMU
> 2. Turn off the laptop, and leave it turned off for a few seconds
> 3. Turn on the laptop
> 4. Enable the IOMMU (*DON'T* turn off the machine)
> 5. Continue
> 6. Xen crashes
> and, now the log lines are:
> (XEN) Intel VT-d Queued Invalidation supported.
> (XEN) Intel VT-d Interrupt Remapping supported.
> It looks like there is a problem when enabling IOMMU, without cold boot of
> the machine.
> What do u say?
> Here is the full log:
> __ __ _____ _ _ ___
> \ \/ /___ _ __ |___ /| || | / _ \
> \ // _ \ '_ \ |_ \| || |_| | | |
> / \ __/ | | | ___) |__ _| |_| |
> /_/\_\___|_| |_| |____(_) |_|(_)___/
> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian
> 4.3.2-1.1) ) Mon Jun 29 07:40:11 EDT 2009
> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd
> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1
> iommu_inclusive_mapping loglvl=all guest_loglvl=all
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN) Found 1 MBR signatures
> (XEN) Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009f000 (usable)
> (XEN) 000000000009f000 - 00000000000a0000 (reserved)
> (XEN) 0000000000100000 - 000000007c04d400 (usable)
> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS)
> (XEN) 000000007c04f400 - 000000007f000000 (reserved)
> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec10000 (reserved)
> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved)
> (XEN) 00000000fed20000 - 00000000fed90000 (reserved)
> (XEN) 00000000feda0000 - 00000000feda6000 (reserved)
> (XEN) 00000000fee00000 - 00000000fee10000 (reserved)
> (XEN) 00000000ffe60000 - 0000000100000000 (reserved)
> (XEN) System RAM: 1983MB (2031536kB)
> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL )
> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624)
> (XEN) ACPI: FACS 7C060C00, 0040
> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61)
> (XEN) ACPI: DMAR 7C060400, 00B0 (r1 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47)
> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0)
> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624)
> (XEN) NUMA turned off
> (XEN) Faking a node at 0000000000000000-000000007c04d000
> (XEN) Domain heap initialised
> (XEN) DMI 2.4 present.
> (XEN) Using APIC driver default
> (XEN) ACPI: PM-Timer IO Port: 0x1008
> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0]
> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20]
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 7:7 APIC version 20
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> (XEN) Processor #1 7:7 APIC version 20
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) ACPI: IRQ9 used by override.
> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
> (XEN) [VT-D]dmar.c:485: Host address width 36
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000
> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
> (XEN) Intel VT-d DMAR tables have been parsed.
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Initializing CPU#0
> (XEN) Detected 2527.037 MHz processor.
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 6144K
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 0
> (XEN) VMX: Supported advanced features:
> (XEN) - APIC MMIO access virtualisation
> (XEN) - APIC TPR shadow
> (XEN) - Virtual NMI
> (XEN) - MSR direct-access bitmap
> (XEN) HVM: VMX enabled
> (XEN) Intel machine check reporting enabled on CPU#0.
> (XEN) mce_init: init bank1
> (XEN) mce_init: init bank2
> (XEN) mce_init: init bank3
> (XEN) mce_init: init bank4
> (XEN) mce_init: init bank5
> (XEN) CPU0: Thermal monitoring enabled (TM2)
> (XEN) CMCI: CPU0 has no CMCI support
> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
> (XEN) Booting processor 1/1 eip 8c000
> (XEN) Initializing CPU#1
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 6144K
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 1
> (XEN) Intel machine check reporting enabled on CPU#1.
> (XEN) mce_init: init bank1
> (XEN) mce_init: init bank2
> (XEN) mce_init: init bank4
> (XEN) mce_init: init bank5
> (XEN) CPU1: Thermal monitoring enabled (TM2)
> (XEN) CMCI: CPU1 has no CMCI support
> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
(XEN) ->> Using new ACK method
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Brought up 2 CPUs
> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed
> (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush
> (XEN) Intel VT-d Snoop Control supported.
> (XEN) Intel VT-d DMA Passthrough not supported.
> (XEN) Intel VT-d Queued Invalidation supported.
> (XEN) Intel VT-d Interrupt Remapping supported.
> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]----
> (XEN) CPU: 0
> (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020
> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90
> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110
> (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005
> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc
> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0
> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> (XEN) Xen stack trace from rsp=ffff828c8026fda0:
> (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490
> (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000
> (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f
> (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60
> (XEN) ffff828c801f3400 ffff83000008bf30 ffff828c8012a9d4 ffff828c80227978
> (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 00000000002b23c0 ffffff8c80274000 0008bf3000000000 000000000008bf30
> (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000
> (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003
> (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000
> (XEN) Xen call trace:
> (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
> (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0
> (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610
> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110
> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0
> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3
> (XEN)
> (XEN) Pagetable walk from ffff828bfff5aff8:
> (XEN) L4[0x105] = 000000007be7a027 5555555555555555
> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555
> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555
> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0000]
> (XEN) Faulting linear address: ffff828bfff5aff8
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
> 2009/6/29 Tom Rotenberg
> <tom.rotenberg@xxxxxxxxx<mailto:tom.rotenberg@xxxxxxxxx>>
> I'm almost 100% sure, that this was from the same machine (since i have
> only 1 such Dell currently on my desk :).
> However, this log was generated just after turning on the IOMMU (without
> cold boot). So, let me check if it happened because of this.
> I'll check it now...
> 2009/6/29 Cui, Dexuan <dexuan.cui@xxxxxxxxx<mailto:dexuan.cui@xxxxxxxxx>>
> I have no idea about it.
>> (XEN) Intel VT-d Snoop Control supported.
>> (XEN) Intel VT-d DMA Passthrough not supported.
>> (XEN) Intel VT-d Queued Invalidation supported.
>> (XEN) Intel VT-d Interrupt Remapping supported.
> This is from the log of your original mail. We can see SC/QI/IR are
> supported, but in your later log, we can see they are not supported.
> I'm curious where the first log comes from. :-)
> -- Dexuan
> ________________________________
> From: Tom Rotenberg
> [mailto:tom.rotenberg@xxxxxxxxx<mailto:tom.rotenberg@xxxxxxxxx>]
> Sent: 2009年6月29日 20:38
> To: Cui, Dexuan
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx<mailto:xen-devel@xxxxxxxxxxxxxxxxxxx>
> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
> iommu=1
> 1) Yep, i can't reproduce it anymore. Maybe i'll try it with another Dell
> R6400 i have here.
> 2) On my Lenovo T400, i don't remember the exact crash, and i don't have
> the log of that crash, i just remember something about "DMAR ...
> failure", which happened, after i turned on the IOMMU, and didn't perform a
> cold reboot on that machine.
> Do u have any idea why these problems happen?
> Does anyone else working with Dell E6400 had this problem?
> Tom
> 2009/6/29 Cui, Dexuan <dexuan.cui@xxxxxxxxx<mailto:dexuan.cui@xxxxxxxxx>>
> It's really weird... :-)
> If I understand correctly, with iommu=1 in Xen:
> 1) on your "Dell E6400 machine", now you can't reproduce the panic any longer;
> 2) on your "Lenovo T400", you can reproduce the panic sometimes -- if
> so, can you post the log (with the printk patch applied) when you meet
> with the panic? Can you find out under what condition it's easy to reproduce
> the panic?
> Thanks,
> -- Dexuan
> ________________________________
> From: Tom Rotenberg
> [mailto:tom.rotenberg@xxxxxxxxx<mailto:tom.rotenberg@xxxxxxxxx>]
> Sent: 2009年6月29日 20:15
> To: Cui, Dexuan
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx<mailto:xen-devel@xxxxxxxxxxxxxxxxxxx>
> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
> iommu=1
> Hi,
> Tried again, this time with cold reboot, it worked again. I also checked
> the lines you were talking about, and now they are:
> (XEN) Intel VT-d Snoop Control not supported.
> (XEN) Intel VT-d DMA Passthrough not supported.
> (XEN) Intel VT-d Queued Invalidation not supported.
> (XEN) Intel VT-d Interrupt Remapping not supported.
> Could it be that there is some kind of problem with the hardware, which
> makes it sometimes report invalid values? (with the Lenovo T400 i'm
> testing, if you enable the IOMMU, without a cold-reboot, it may cause Xen
> to crash on boot, so you must do a cold reboot).
> Here is the log:
> __ __ _____ _ _ ___
> \ \/ /___ _ __ |___ /| || | / _ \
> \ // _ \ '_ \ |_ \| || |_| | | |
> / \ __/ | | | ___) |__ _| |_| |
> /_/\_\___|_| |_| |____(_) |_|(_)___/
> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian
> 4.3.2-1.1) ) Mon Jun 29 11:04:57 EDT 2009
> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd
> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1
> iommu_inclusive_mapping loglvl=all guest_loglvl=all
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN) Found 1 MBR signatures
> (XEN) Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009f000 (usable)
> (XEN) 000000000009f000 - 00000000000a0000 (reserved)
> (XEN) 0000000000100000 - 000000007c04d400 (usable)
> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS)
> (XEN) 000000007c04f400 - 000000007f000000 (reserved)
> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec10000 (reserved)
> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved)
> (XEN) 00000000fed20000 - 00000000fed90000 (reserved)
> (XEN) 00000000feda0000 - 00000000feda6000 (reserved)
> (XEN) 00000000fee00000 - 00000000fee10000 (reserved)
> (XEN) 00000000ffe60000 - 0000000100000000 (reserved)
> (XEN) System RAM: 1983MB (2031536kB)
> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL )
> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624)
> (XEN) ACPI: FACS 7C060C00, 0040
> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61)
> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47)
> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0)
> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624)
> (XEN) NUMA turned off
> (XEN) Faking a node at 0000000000000000-000000007c04d000
> (XEN) Domain heap initialised
> (XEN) DMI 2.4 present.
> (XEN) Using APIC driver default
> (XEN) ACPI: PM-Timer IO Port: 0x1008
> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0]
> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20]
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 7:7 APIC version 20
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> (XEN) Processor #1 7:7 APIC version 20
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) ACPI: IRQ9 used by override.
> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
> (XEN) [VT-D]dmar.c:485: Host address width 36
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000
> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
> (XEN) Intel VT-d DMAR tables have been parsed.
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Initializing CPU#0
> (XEN) Detected 2527.058 MHz processor.
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 6144K
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 0
> (XEN) VMX: Supported advanced features:
> (XEN) - APIC MMIO access virtualisation
> (XEN) - APIC TPR shadow
> (XEN) - Virtual NMI
> (XEN) - MSR direct-access bitmap
> (XEN) HVM: VMX enabled
> (XEN) Intel machine check reporting enabled on CPU#0.
> (XEN) mce_init: init bank1
> (XEN) mce_init: init bank2
> (XEN) mce_init: init bank3
> (XEN) mce_init: init bank4
> (XEN) mce_init: init bank5
> (XEN) CPU0: Thermal monitoring enabled (TM2)
> (XEN) CMCI: CPU0 has no CMCI support
> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
> (XEN) Booting processor 1/1 eip 8c000
> (XEN) Initializing CPU#1
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 6144K
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 1
> (XEN) Intel machine check reporting enabled on CPU#1.
> (XEN) mce_init: init bank1
> (XEN) mce_init: init bank2
> (XEN) mce_init: init bank4
> (XEN) mce_init: init bank5
> (XEN) CPU1: Thermal monitoring enabled (TM2)
> (XEN) CMCI: CPU1 has no CMCI support
> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
(XEN) ->> Using new ACK method
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Brought up 2 CPUs
> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed
> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush
> (XEN) Intel VT-d Snoop Control not supported.
> (XEN) Intel VT-d DMA Passthrough not supported.
> (XEN) Intel VT-d Queued Invalidation not supported.
> (XEN) Intel VT-d Interrupt Remapping not supported.
> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270
> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260
> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260
> (XEN) I/O virtualisation enabled
> (XEN) I/O virtualisation for PV guests disabled
> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
> (XEN) ACPI sleep modes: S3
> (XEN) mcheck_poll: Machine check polling timer started.
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
> ffff828bfff57000
> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
> ffff828bfff56000
> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
> ffff828bfff55000
> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
> ffff828bfff54000
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen kernel: 64-bit, lsb, compat32
> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 ->
> 0xffffffff8070ab48
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be
> allocated)
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48
> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400
> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18
> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4
> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000
> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000
> (XEN) TOTAL: ffffffff80000000->ffffffff82000000
> (XEN) ENTRY ADDRESS: ffffffff80200000
> (XEN) Dom0 has maximum 2 VCPUs
> (XEN) Scrubbing Free RAM: .done.
> (XEN) Xen trace buffers: disabled
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to
> Xen)
> (XEN) Freed 132kB init memory.
> Tom
> 2009/6/29 Cui, Dexuan <dexuan.cui@xxxxxxxxx<mailto:dexuan.cui@xxxxxxxxx>>
> So, if I understand correctly, it's really strange you can't reproduce the
> panic now.
> With only the first printk patch applied (and don't comment the
> clear_fault_bits()), can you try a cold rebooting the host (plug the
> power cord off from the host; wait several minutes; plug the cord in and tunn
> on the host)? And the log?
> BTW, in your original mail, the log shows:
> (XEN) Intel VT-d Queued Invalidation supported.
> (XEN) Intel VT-d Interrupt Remapping supported.
> But in your this new log, it shows:
> (XEN) Intel VT-d Queued Invalidation not supported.
> (XEN) Intel VT-d Interrupt Remapping not supported.
> Are you sure you have been operating on the same host? :-)
> -- Dexuan
> ________________________________
> From: Tom Rotenberg
> [mailto:tom.rotenberg@xxxxxxxxx<mailto:tom.rotenberg@xxxxxxxxx>]
> Sent: 2009年6月29日 17:01
> To: Cui, Dexuan
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx<mailto:xen-devel@xxxxxxxxxxxxxxxxxxx>
> Subject: Re: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
> iommu=1
> Hi,
> Well, just tried what you suggested, and here are the results:
> * If i comment the 'clear_fault_bits()' call - then Xen boots ok, with iommu=1
> The strange thing, is that after it booted one time ok (with the second
> patch), then it continued to boot ok, even if i reverted the second patch.
> The relevant lines in the log are:
> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270
> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260
> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260
> Anyway, here is the full Xen log, with the first patch (print cap's) applied:
> __ __ _____ _ _ ___
> \ \/ /___ _ __ |___ /| || | / _ \
> \ // _ \ '_ \ |_ \| || |_| | | |
> / \ __/ | | | ___) |__ _| |_| |
> /_/\_\___|_| |_| |____(_) |_|(_)___/
> (XEN) Xen version 3.4.0 (root@[unknown]) (gcc version 4.3.2 (Debian
> 4.3.2-1.1) ) Mon Jun 29 07:53:38 EDT 2009
> (XEN) Latest ChangeSet: Wed Jun 24 11:15:04 2009 +0300 19629:b2128a88e3dd
> (XEN) Command line: console=com1 com1=115200,8n1 iommu=1
> iommu_inclusive_mapping loglvl=all guest_loglvl=all
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN) Found 1 MBR signatures
> (XEN) Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009f000 (usable)
> (XEN) 000000000009f000 - 00000000000a0000 (reserved)
> (XEN) 0000000000100000 - 000000007c04d400 (usable)
> (XEN) 000000007c04d400 - 000000007c04f400 (ACPI NVS)
> (XEN) 000000007c04f400 - 000000007f000000 (reserved)
> (XEN) 00000000f8000000 - 00000000fc000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec10000 (reserved)
> (XEN) 00000000fed18000 - 00000000fed1c000 (reserved)
> (XEN) 00000000fed20000 - 00000000fed90000 (reserved)
> (XEN) 00000000feda0000 - 00000000feda6000 (reserved)
> (XEN) 00000000fee00000 - 00000000fee10000 (reserved)
> (XEN) 00000000ffe60000 - 0000000100000000 (reserved)
> (XEN) System RAM: 1983MB (2031536kB)
> (XEN) ACPI: RSDP 000FB9C0, 0024 (r2 DELL )
> (XEN) ACPI: XSDT 7C051E00, 006C (r1 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: FACP 7C051C9C, 00F4 (r4 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: DSDT 7C052400, 6A2B (r2 INT430 SYSFexxx 1001 INTL 20050624)
> (XEN) ACPI: FACS 7C060C00, 0040
> (XEN) ACPI: HPET 7C051F00, 0038 (r1 DELL M09 1 ASL 61)
> (XEN) ACPI: DMAR 7C060400, 0120 (r1 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: APIC 7C052000, 0068 (r1 DELL M09 27D9020D ASL 47)
> (XEN) ACPI: ASF! 7C051C00, 006A (r32 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: MCFG 7C051FC0, 003E (r16 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: TCPA 7C052300, 0032 (r1 0 ASL 0)
> (XEN) ACPI: SLIC 7C05209C, 0176 (r1 DELL M09 27D9020D ASL 61)
> (XEN) ACPI: SSDT 7C05032D, 066C (r1 PmRef CpuPm 3000 INTL 20050624)
> (XEN) NUMA turned off
> (XEN) Faking a node at 0000000000000000-000000007c04d000
> (XEN) Domain heap initialised
> (XEN) DMI 2.4 present.
> (XEN) Using APIC driver default
> (XEN) ACPI: PM-Timer IO Port: 0x1008
> (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[1004,0], pm1x_evt[1000,0]
> (XEN) ACPI: wakeup_vec[7c060c0c], vec_size[20]
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 7:7 APIC version 20
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> (XEN) Processor #1 7:7 APIC version 20
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) ACPI: IRQ9 used by override.
> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
> (XEN) [VT-D]dmar.c:485: Host address width 36
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed10000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed11000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed12000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed13000
> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:2.1
> (XEN) Intel VT-d DMAR tables have been parsed.
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Initializing CPU#0
> (XEN) Detected 2527.068 MHz processor.
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 6144K
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 0
> (XEN) VMX: Supported advanced features:
> (XEN) - APIC MMIO access virtualisation
> (XEN) - APIC TPR shadow
> (XEN) - Virtual NMI
> (XEN) - MSR direct-access bitmap
> (XEN) HVM: VMX enabled
> (XEN) Intel machine check reporting enabled on CPU#0.
> (XEN) mce_init: init bank1
> (XEN) mce_init: init bank2
> (XEN) mce_init: init bank3
> (XEN) mce_init: init bank4
> (XEN) mce_init: init bank5
> (XEN) CPU0: Thermal monitoring enabled (TM2)
> (XEN) CMCI: CPU0 has no CMCI support
> (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
> (XEN) Booting processor 1/1 eip 8c000
> (XEN) Initializing CPU#1
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 6144K
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 1
> (XEN) Intel machine check reporting enabled on CPU#1.
> (XEN) mce_init: init bank1
> (XEN) mce_init: init bank2
> (XEN) mce_init: init bank4
> (XEN) mce_init: init bank5
> (XEN) CPU1: Thermal monitoring enabled (TM2)
> (XEN) CMCI: CPU1 has no CMCI support
> (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz stepping 06
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
(XEN) ->> Using new ACK method
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Brought up 2 CPUs
> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed
> (XEN) [VT-D]iommu.c:1763: DMAR: Forcing write-buffer flush
> (XEN) Intel VT-d Snoop Control not supported.
> (XEN) Intel VT-d DMA Passthrough not supported.
> (XEN) Intel VT-d Queued Invalidation not supported.
> (XEN) Intel VT-d Interrupt Remapping not supported.
> (XEN) clear_fault_bits: CAP_REG=0xc0000020630270
> (XEN) clear_fault_bits: CAP_REG=0xc0000020630260
> (XEN) clear_fault_bits: CAP_REG=0xc9008020e30260
> (XEN) clear_fault_bits: CAP_REG=0xc9008020630260
> (XEN) I/O virtualisation enabled
> (XEN) I/O virtualisation for PV guests disabled
> (XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
> (XEN) ACPI sleep modes: S3
> (XEN) mcheck_poll: Machine check polling timer started.
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:0.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:3.3
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:19.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
> (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe: bdf = c:0.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:2.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
> (XEN) [VT-D]iommu.c:1252:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
> ffff828bfff57000
> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
> ffff828bfff56000
> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
> ffff828bfff55000
> (XEN) [VT-D]iommu.c:620: iommu_enable_translation: iommu->reg =
> ffff828bfff54000
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen kernel: 64-bit, lsb, compat32
> (XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 ->
> 0xffffffff8070ab48
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 0000000076000000->0000000078000000 (460259 pages to be
> allocated)
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff80200000->ffffffff8070ab48
> (XEN) Init. ramdisk: ffffffff8070b000->ffffffff81b94400
> (XEN) Phys-Mach map: ffffffff81b95000->ffffffff81f27f18
> (XEN) Start info: ffffffff81f28000->ffffffff81f284b4
> (XEN) Page tables: ffffffff81f29000->ffffffff81f3c000
> (XEN) Boot stack: ffffffff81f3c000->ffffffff81f3d000
> (XEN) TOTAL: ffffffff80000000->ffffffff82000000
> (XEN) ENTRY ADDRESS: ffffffff80200000
> (XEN) Dom0 has maximum 2 VCPUs
> (XEN) Scrubbing Free RAM: .done.
> (XEN) Xen trace buffers: disabled
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to
> Xen)
> (XEN) Freed 132kB init memory.
> Tom
> 2009/6/29 Cui, Dexuan <dexuan.cui@xxxxxxxxx<mailto:dexuan.cui@xxxxxxxxx>>
> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:24:31 2009 +0800
> @@ -1693,7 +1693,7 @@ static int init_vtd_hw(void)
> }
> dma_msi_data_init(iommu, iommu->vector);
> dma_msi_addr_init(iommu,
> cpu_physical_id(first_cpu(cpu_online_map)));
> - clear_fault_bits(iommu);
> + //clear_fault_bits(iommu);
> dmar_writel(iommu->reg, DMAR_FECTL_REG, 0);
> /* initialize flush functions */
> And, can you comment the invocation of clear_fault_bits() in
> init_vtd_hw() like above and see if the host can boot fine?
> Thanks,
> -- Dexuan
> -----Original Message-----
> From: Cui, Dexuan
> Sent: 2009?6?29? 14:30
> To: 'Tom Rotenberg';
> xen-devel@xxxxxxxxxxxxxxxxxxx<mailto:xen-devel@xxxxxxxxxxxxxxxxxxx>
> Subject: RE: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
> iommu=1
> diff -r 90391e2247fc xen/drivers/passthrough/vtd/iommu.c
> --- a/xen/drivers/passthrough/vtd/iommu.c Tue Jun 23 11:40:27 2009 +0100
> +++ b/xen/drivers/passthrough/vtd/iommu.c Mon Jun 29 10:18:23 2009 +0800
> @@ -1653,6 +1653,8 @@ void clear_fault_bits(struct iommu *iomm
> {
> u64 val;
> + printk("clear_fault_bits: CAP_REG=0x%llx\n",
> + (unsigned long long)dmar_readq(iommu->reg,DMAR_CAP_REG));
> val = dmar_readq(
> iommu->reg,
> cap_fault_reg_offset(dmar_readq(iommu->reg,DMAR_CAP_REG))+0x8);
> Hi Tom,
> Can you please apply the above patch and send us the entire log?
> (I don't have the same host, so I can't try it myself.)
> Thanks,
> -- Dexuan
> ________________________________
> From:
> xen-devel-bounces@xxxxxxxxxxxxxxxxxxx<mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx>
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx<mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx>]
> On Behalf Of Tom Rotenberg
> Sent: 2009?6?28? 22:56
> To: xen-devel@xxxxxxxxxxxxxxxxxxx<mailto:xen-devel@xxxxxxxxxxxxxxxxxxx>
> Subject: [Xen-devel] Problem: Xen 3.4 fails to boot on Dell e6400, with
> iommu=1
> Hi,
> I am trying to use xen 3.4 on a Dell E6400 machine. But, when i try to
> boot xen with a 'iommu=1' parameter in the boot line, Xen fails to boot.
> I have also tried to boot with and without the 'iommu_inclusive_mapping'
> boot option, but it didn't seem to help at all.
> Does anyone knows how to solve this issue?
> These are the last messages in the UART:
> (XEN) Brought up 2 CPUs
> (XEN) microcode.c:73:d32767 microcode: CPU1 resumed
> (XEN) [VT-D]iommu.c:1761: DMAR: Forcing write-buffer flush
> (XEN) Intel VT-d Snoop Control supported.
> (XEN) Intel VT-d DMA Passthrough not supported.
> (XEN) Intel VT-d Queued Invalidation supported.
> (XEN) Intel VT-d Interrupt Remapping supported.
> (XEN) ----[ Xen-3.4.0 x86_64 debug=n Not tainted ]----
> (XEN) CPU: 0
> (XEN) RIP: e008:[<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
> (XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
> (XEN) rax: ffff828bfff5aff8 rbx: ffff830078f82f90 rcx: 0000000000000020
> (XEN) rdx: 00000000ffffffff rsi: ffff828bfff57000 rdi: ffff830078f82f90
> (XEN) rbp: 0000000000000090 rsp: ffff828c8026fda0 r8: ffff83007be2c110
> (XEN) r9: 000000000000001a r10: 0000000000000005 r11: 0000000000000005
> (XEN) r12: 0000000000000090 r13: 0000000000000090 r14: ffff830078f82fcc
> (XEN) r15: 0000000000004800 cr0: 000000008005003b cr4: 00000000000026f0
> (XEN) cr3: 000000007be74000 cr2: ffff828bfff5aff8
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> (XEN) Xen stack trace from rsp=ffff828c8026fda0:
> (XEN) ffff828c8012d32d ffff830078f824b0 ffff828c801e422f ffff828c80204490
> (XEN) ffff828c801e422f ffff828c802044c0 ffff828c8026fdf8 0000000000000000
> (XEN) ffff828c8012de7e ffff830078f82d40 ffff828c80228100 000000000000001f
> (XEN) 00000000ffffffed ffff828c80227a78 0000000000000001 ffff828c80205f60
> (XEN) ffff828c801f3400 ffff83000008bf40 ffff828c8012a9d4 ffff828c80227978
> (XEN) ffff828c8021a589 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 00000000002b23c0 ffffff8c80274000 0008bf4000000000 000000000008bf40
> (XEN) 0000000000000000 ffff83000008bfc0 0000000001ce08a0 0000000000000000
> (XEN) ffffffff00000000 0000000800000000 000000010000006e 0000000000000003
> (XEN) 00000000000002f8 0000000000000000 0000000000000000 0000000000067e2c
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) ffff828c801000b5 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff830079f18000
> (XEN) Xen call trace:
> (XEN) [<ffff828c8012c951>] clear_fault_bits+0x21/0xa0
> (XEN) [<ffff828c8012d32d>] init_vtd_hw+0x15d/0x3e0
> (XEN) [<ffff828c8012de7e>] intel_vtd_setup+0x5ae/0x610
> (XEN) [<ffff828c8012a9d4>] iommu_setup+0x24/0x110
> (XEN) [<ffff828c8021a589>] __start_xen+0x1859/0x27c0
> (XEN) [<ffff828c801000b5>] __high_start+0xa1/0xa3
> (XEN)
> (XEN) Pagetable walk from ffff828bfff5aff8:
> (XEN) L4[0x105] = 000000007be7a027 5555555555555555
> (XEN) L3[0x02f] = 000000007c04b063 5555555555555555
> (XEN) L2[0x1ff] = 000000007c04a063 5555555555555555
> (XEN) L1[0x15a] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0000]
> (XEN) Faulting linear address: ffff828bfff5aff8
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
--
Best regards,
Sander mailto:linux@xxxxxxxxxxxxxx
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home