WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] Linux DomU halted very early after bootedwithVT-ddevice

To: 沈启龙 <shen.qilong@xxxxxxxxxxxx>, "Han, Weidong" <weidong.han@xxxxxxxxx>, "Jiang, Yunhong" <yunhong.jiang@xxxxxxxxx>, 'Konrad Rzeszutek Wilk' <konrad.wilk@xxxxxxxxxx>
Subject: RE: [Xen-devel] Linux DomU halted very early after bootedwithVT-ddevice
From: "Zhang, Xiantao" <xiantao.zhang@xxxxxxxxx>
Date: Mon, 21 Dec 2009 10:58:24 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Sun, 20 Dec 2009 18:59:45 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <000b01ca81e8$e37d0ea0$2705a8c0@freecloud>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <000001ca7a41$fc802580$2705a8c0@freecloud><20091211151008.GC19662@xxxxxxxxxxxxxxxxxxx><001801ca7c5f$33f4d0a0$2705a8c0@freecloud><C8EDE645B81E5141A8C6B2F73FD9265105AC73072A@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx><001701ca7d59$34a154b0$2705a8c0@freecloud><60E426D47DE8EA47AA104E65008A100D05AE7ED663@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx><EB8593BCECAB3D40A8248BE0B6400A382FAB2BF2@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx><000001ca81db$eab7b790$2705a8c0@freecloud> <60E426D47DE8EA47AA104E65008A100D05CD54608E@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx> <000b01ca81e8$e37d0ea0$2705a8c0@freecloud>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Acp6dd/r5qTld+cfRsiNJsQZImjGHwB5jQggADN1mDAACwcK0ACZVs/AAA+my6AAeHbAIAACLf+AAADUiTAAAFFQ8A==
Thread-topic: [Xen-devel] Linux DomU halted very early after bootedwithVT-ddevice
Could you paste the guest's log without vt-d devcie assignment ?
In you mail, you said: "And I use the command xm pci-list-assignable-devices, 
nothing show."
So when to run the command ? Before creating the vm or after ?
Xiantao


沈启龙 wrote:
> I had created a vm without device assignment with VT-d, and it could
> run normally.
>
> Regards,
> shenqilong
>
> -----邮件原件-----
> 发件人: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] 代表 Han, Weidong
> 发送时间: 2009年12月21日 10:25
> 收件人: 沈启龙; Zhang, Xiantao; Jiang, Yunhong; 'Konrad Rzeszutek Wilk'
> 抄送: xen-devel@xxxxxxxxxxxxxxxxxxx
> 主题: RE: [Xen-devel] Linux DomU halted very early after
> bootedwithVT-ddevice
>
> Did you try to create the guest without device assignment with VT-d?
>
> Regards,
> Weidong
>
> -----Original Message-----
> From: 沈启龙 [mailto:shen.qilong@xxxxxxxxxxxx]
> Sent: Monday, December 21, 2009 9:21 AM
> To: Zhang, Xiantao; Han, Weidong; Jiang, Yunhong; 'Konrad Rzeszutek
> Wilk' Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: 答复: [Xen-devel] Linux DomU halted very early after booted
> withVT-ddevice
>
> Thank you for you method, I get the domU's dump message as following:
>
> Linux version 2.6.18-128.el5 (mockbuild@xxxxxxxxxxxxxxxxxxxx) (gcc
> version
> 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Wed Jan 21 10:41:14 EST 2009
> Command line: ro root=/dev/VolGroup00/LogVol00 rhgb console=ttyS0,
> 11520, 8n1 BIOS-provided physical RAM map:
>
>
>  BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
>  BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
>  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
>  BIOS-e820: 0000000000100000 - 0000000040000000 (usable)
>  BIOS-e820: 00000000fc000000 - 0000000100000000 (reserved) DMI 2.4
> present. No NUMA configuration found
> Faking a node at 0000000000000000-0000000040000000 Bootmem setup node
> 0 0000000000000000-0000000040000000 Memory for crash kernel (0x0 to
> 0x0) notwithin permissible range disabling kdump
> ACPI: PM-Timer IO Port: 0x1f48
> ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) Processor #0 7:10
> APIC version 20
> ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
> IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-47
> ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 low level)
> ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 low level)
> ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 low level) Setting
> APIC routing to physical flat Using ACPI (MADT) for SMP configuration
> information Nosave address range: 000000000009f000 - 00000000000a0000
> Nosave address range: 00000000000a0000 - 00000000000e0000 Nosave
> address range: 00000000000e0000 - 0000000000100000 Allocating PCI
> resources starting at 50000000 (gap: 40000000:bc000000)
> SMP: Allowing 1 CPUs, 0 hotplug CPUs
> Built 1 zonelists.  Total pages: 257201
> Kernel command line: ro root=/dev/VolGroup00/LogVol00 rhgb
> console=ttyS0, 11520, 8n1 Initializing CPU#0 PID hash table entries:
> 4096 (order: 12, 32768 bytes)
> Console: colour VGA+ 80x25
> Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
> Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
> Checking aperture...
> Memory: 1023668k/1048576k available (2494k kernel code, 24520k
> reserved, 1263k data, 200k init) Calibrating delay using timer
> specific routine.. 3994.81 BogoMIPS (lpj=1997409) Security Framework
> v1.0.0 initialized SELinux:  Initializing.
> selinux_register_security:  Registering secondary module capability
> Capability LSM initialized as secondary Mount-cache hash table
> entries: 256 CPU: L1 I cache: 32K, L1 D cache: 32K
> CPU: L2 cache: 256K
> CPU: L3 cache: 4096K
> CPU: Physical Processor ID: 0
> CPU: Processor Core ID: 0
> SMP alternatives: switching to UP code
> Freeing SMP alternatives: 32k freed
> ACPI: Core revision 20060707
> activating NMI Watchdog ... done.
> Using local APIC timer interrupts.
> result 6250115
> Detected 6.250 MHz APIC timer.
> Brought up 1 CPUs
> testing NMI watchdog ... <4>WARNING: CPU#0: NMI appears to be stuck
> (0->0)! time.c: Using 3.579545 MHz WALL PM GTOD PIT/TSC timer.
> time.c: Detected 1995.040 MHz processor.
> checking if image is initramfs... it is
> Freeing initrd memory: 3229k freed
> NET: Registered protocol family 16
> No dock devices found.
> ACPI: bus type pci registered
> PCI: Using configuration type 1
> ACPI: Interpreter enabled
> ACPI: Using IOAPIC for interrupt routing
> ACPI: PCI Root Bridge [PCI0] (0000:00)
> PCI quirk: region 1f40-1f7f claimed by PIIX4 ACPI
> ACPI: PCI Interrupt Link [LNKA] (IRQs *5 10 11)
> ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
> ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
> ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11) Linux Plug and Play
> Support v0.97 (c) Adam Belay
> pnp: PnP ACPI init
> pnp: PnP ACPI: found 12 devices
> usbcore: registered new driver usbfs
> usbcore: registered new driver hub
> PCI: Using ACPI for IRQ routing
> PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post
> a report
> NetLabel: Initializing
> NetLabel:  domain hash size = 128
> NetLabel:  protocols = UNLABELED CIPSOv4
> NetLabel:  unlabeled traffic allowed by default
> PCI-GART: No AMD northbridge found.
> pnp: 00:02: ioport range 0x10c0-0x10e1 has been reserved
> pnp: 00:02: ioport range 0xb044-0xb047 has been reserved
> PCI: Ignore bogus resource 6 [0:0] of 0000:00:02.0
> NET: Registered protocol family 2
>
> The the vm hang!!
>
> And I used the option pci=routeirq as kernel parameter, but the
> output is the same.
>
> And I use the command xm pci-list-assignable-devices, nothing show.
> cat /sys/bus/pci/driver/pciback/slots, I can find the pci device what
> I needed.
> But if I use cat /sys/bus/pci/driver/pciback/permissive, nothing is
> showed out.
>
> Best Regards
> shenqilong
>
> -----邮件原件-----
> 发件人: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] 代表 Zhang, Xiantao
> 发送时间: 2009年12月18日 23:56
> 收件人: Han, Weidong; 沈启龙; Jiang, Yunhong; 'Konrad Rzeszutek Wilk'
> 抄送: xen-devel@xxxxxxxxxxxxxxxxxxx
> 主题: RE: [Xen-devel] Linux DomU halted very early after booted
> withVT-ddevice
>
> Another mothod is to add "console=ttyS0, 115200, 8n1" to guest's
> kernel cmdline in domU's grub and use "xm con $hvm-dom-id" in dom0's
> terminal at the same time. Then guest's kernel output should redirect
> to the dom0's terminal.  And once you see the kernel output, maybe
> you can paste it here, and it should help to identify this issue.
> Xiantao
>
>
> Han, Weidong wrote:
>> Can you create CentOS guest without device assignment? If so, you can
>> log in and remove "quiet" in grub.
>>
>> Regards,
>> Weidong
>>
>> -----Original Message-----
>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of 沈启龙 Sent:
>> Tuesday, December 15, 2009 3:36 PM
>> To: Jiang, Yunhong; 'Konrad Rzeszutek Wilk'
>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
>> Subject: 答复: [Xen-devel] Linux DomU halted very early after booted
>> withVT-d device
>>
>> I don't add "quiet" option in domU kernel parameter, and the Linux
>> domU is not completely booted. So I can't log in to dump kernel
>> message.
>>
>> Now I find some other information.
>> I have "iommu=1 loglvl=all guest_loglvl=all " in my dom0 kernel
>> parameter?
>>
>> When I boot centos DumU, I can get the following logger.
>>
>> (XEN) [VT-D]iommu.c:1357:d32767 domain_context_unmap:PCIe: bdf =
>> 2:0.1 (XEN) [VT-D]iommu.c:1245:d32767 domain_context_mapping:PCIe:
>> bdf = 2:0.1 (XEN) [VT-D]iommu.c:1357:d0 domain_context_unmap:PCIe:
>> bdf = 2:0.1 (XEN) [VT-D]iommu.c:1245:d0 domain_context_mapping:PCIe:
>> bdf = 2:0.1 (XEN) [VT-D]io.c:251:d0 VT-d irq bind: m_irq = ff device
>> = 3 intx = 0 (XEN) HVM2: HVM Loader
>> (XEN) HVM2: Detected Xen v3.4-091208
>> (XEN) HVM2: CPU speed is 1995 MHz
>> (XEN) irq.c:243: Dom2 PCI link 0 changed 0 -> 5
>> (XEN) HVM2: PCI-ISA link 0 routed to IRQ5
>> (XEN) irq.c:243: Dom2 PCI link 1 changed 0 -> 10
>> (XEN) HVM2: PCI-ISA link 1 routed to IRQ10
>> (XEN) irq.c:243: Dom2 PCI link 2 changed 0 -> 11
>> (XEN) HVM2: PCI-ISA link 2 routed to IRQ11
>> (XEN) irq.c:243: Dom2 PCI link 3 changed 0 -> 5
>> (XEN) HVM2: PCI-ISA link 3 routed to IRQ5
>> (XEN) HVM2: pci dev 01:3 INTA->IRQ10
>> (XEN) HVM2: pci dev 03:0 INTA->IRQ5
>> (XEN) HVM2: pci dev 04:0 INTA->IRQ5
>> (XEN) HVM2: pci dev 02:0 bar 10 size 02000000: f0000008
>> (XEN) HVM2: pci dev 03:0 bar 10 size 02000000: f2000004
>> (XEN) domctl.c:835:d0 memory_map:add: gfn=f2000 mfn=dc000
>> nr_mfns=2000 (XEN) domctl.c:845:d0 memory_map:remove: gfn=f200c
>> mfn=dc00c nr_mfns=1 (XEN) HVM2: pci dev 04:0 bar 14 size 01000000:
>> f4000008 (XEN) HVM2: pci dev 02:0 bar 14 size 00001000: f5000000
>> (XEN) HVM2: pci dev 04:0 bar 10 size 00000100: 0000c001
>> (XEN) HVM2: pci dev 01:1 bar 20 size 00000010: 0000c101
>> (XEN) HVM2: Multiprocessor initialisation:
>> (XEN) HVM2:  - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs
>> [2/8] ... done.
>> (XEN) HVM2: Writing SMBIOS tables ...
>> (XEN) HVM2: Loading ROMBIOS ...
>> (XEN) HVM2: 10300 bytes of ROMBIOS high-memory extensions:
>> (XEN) HVM2:   Relocating to 0xfc000000-0xfc00283c ... done
>> (XEN) HVM2: Creating MP tables ...
>> (XEN) HVM2: Loading Cirrus VGABIOS ...
>> (XEN) HVM2: Loading ACPI ...
>> (XEN) HVM2:  - Lo data: 000ea020-000ea04f
>> (XEN) HVM2:  - Hi data: fc002c00-fc00609f
>> (XEN) HVM2: vm86 TSS at fc006400
>> (XEN) HVM2: BIOS map:
>> (XEN) HVM2:  c0000-c8fff: VGA BIOS
>> (XEN) HVM2:  eb000-eb14a: SMBIOS tables
>> (XEN) HVM2:  f0000-fffff: Main BIOS
>> (XEN) HVM2: Invoking ROMBIOS ...
>> (XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
>> (XEN) stdvga.c:147:d2 entering stdvga and caching modes
>> (XEN) HVM2: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12
>> vruppert Exp $ (XEN) HVM2: Bochs BIOS - build: 06/23/99
>> (XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
>> (XEN) HVM2: Options: apmbios pcibios eltorito PMM
>> (XEN) HVM2:
>> (XEN) HVM2: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
>> (XEN) HVM2: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (10000
>> MBytes) (XEN) HVM2: IDE time out (XEN) HVM2:
>> (XEN) HVM2:
>> (XEN) HVM2:
>> (XEN) HVM2: Press F12 for boot menu.
>> (XEN) HVM2:
>> (XEN) HVM2: Booting from Hard Disk...
>> (XEN) HVM2: Booting from 0000:7c00
>> (XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=81
>> (XEN) HVM2: int13_harddisk: function 08, unmapped device for ELDL=81
>> (XEN) HVM2: *** int 15h function AX=00c0, BX=0000 not yet supported!
>> (XEN) HVM2: int13_harddisk: function 15, unmapped device for ELDL=81
>> (XEN) HVM2: *** int 15h function AX=ec00, BX=0002 not yet supported!
>> (XEN) HVM2: KBD: unsupported int 16h function 03
>> (XEN) HVM2: int13_harddisk: function 15, unmapped device for ELDL=81
>> (XEN) HVM2: int13_harddisk: function 02, unmapped device for ELDL=81
>> (XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=81
>> (XEN) domctl.c:835:d0 memory_map:add: gfn=f200c mfn=dc00c nr_mfns=1
>> (XEN) domctl.c:845:d0 memory_map:remove: gfn=f2000 mfn=dc000
>> nr_mfns=2000 (XEN) domctl.c:835:d0 memory_map:add: gfn=fe000
>> mfn=dc000 nr_mfns=2000 (XEN) domctl.c:845:d0 memory_map:remove:
>> gfn=fe00c mfn=dc00c nr_mfns=1 (XEN) domctl.c:835:d0 memory_map:add:
>> gfn=fe00c mfn=dc00c nr_mfns=1 (XEN) domctl.c:845:d0
>> memory_map:remove: gfn=fe000 mfn=dc000 nr_mfns=2000 (XEN)
>> domctl.c:835:d0 memory_map:add: gfn=f2000 mfn=dc000 nr_mfns=2000
>> (XEN) domctl.c:845:d0 memory_map:remove: gfn=f200c mfn=dc00c
>> nr_mfns=1 (XEN) irq.c:243: Dom2 PCI link 0 changed 5 -> 0 (XEN)
>> irq.c:243: Dom2 PCI link 1 changed 10 -> 0 (XEN) irq.c:243: Dom2 PCI
>> link 2 changed 11 -> 0 (XEN) irq.c:243: Dom2 PCI link 3 changed 5 ->
>> 0
>>
>> And I use command xm debug-key V (dump iommu info)
>>
>> (XEN) iommu 0: nr_pt_levels = 4.
>> (XEN)   Queued Invalidation: supported and enabled.
>> (XEN)   Interrupt Remapping: not supported.
>>
>> I think Interrupt Remapping is enable by default, does that mean
>> that the system don't support vtd completely.
>>
>> -----邮件原件-----
>> 发件人: Jiang, Yunhong [mailto:yunhong.jiang@xxxxxxxxx]
>> 发送时间: 2009年12月15日 9:58
>> 收件人: 沈启龙; 'Konrad Rzeszutek Wilk'
>> 抄送: xen-devel@xxxxxxxxxxxxxxxxxxx
>> 主题: RE: [Xen-devel] Linux DomU halted very early after booted
>> withVT-d device
>>
>> You have "quiet" in you kernel parameter?
>>
>> --jyh
>>
>>> -----Original Message-----
>>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of 沈启龙
>>> Sent: Monday, December 14, 2009 9:46 AM
>>> To: 'Konrad Rzeszutek Wilk'
>>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
>>> Subject: 答复: [Xen-devel] Linux DomU halted very early after booted
>>> withVT-d device
>>>
>>> For example
>>> DomU system is CentOS.
>>> There is nearly no any out from kernel.
>>> After grub configure info, the only output info is a logger as
>>> following. Memory for crash kernel (0x0 to 0x0) notwithin
>>> permissible range. Then the system halted.
>>>
>>> If I don't use vtd option, the following information is "PCI: PIIX3:
>>> Enabling Passive Release on 0000:00:01:0"
>>>
>>> -----邮件原件-----
>>> 发件人: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] 代表 Konrad Rzeszutek
>>> Wilk 发送时间: 2009年12月11日 23:10 收件人: 沈启龙
>>> 抄送: xen-devel@xxxxxxxxxxxxxxxxxxx; xen-users@xxxxxxxxxxxxxxxxxxx
>>> 主题: Re: [Xen-devel] Linux DomU halted very early after booted
>>> withVT-d device
>>>
>>> On Fri, Dec 11, 2009 at 05:11:55PM +0800, 沈启龙 wrote:
>>>> My windows2k3 server domU can run normally. I can sign into the
>>>> system and find the vt-d
>>>>
>>>> device, the device is not driverd, but the system can recognize it.
>>>>
>>>> However other linux domU can't start normally, it halt very early
>>>> after booted.
>>>
>>> Can you give me the kernel output of the domU?
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/xen-devel
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/xen-devel
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel