Re: [Xen-devel] 2.6.28 64-bit domU not booting

[Valtteri Kiviniemi <valtteri.kiviniemi@xxxxxxxxxxxx>]

Hi,

This is indeed a very strange problem. I just disabled _everything_ in 
the kernel menuconfig. No drivers, no filesystems, no nothing. I even 
disabled all in processor types and features.
Still the same error message. So it's probably a bug or something 
because its not just working no matter what options are enabled/disabled 
in kernel.
- Valtteri Kiviniemi

Valtteri Kiviniemi kirjoitti:
> Hi,
>
> Nice, I dont think that i have the necessary skillz for that :) but thnx 
> anyway
> - Valtteri Kiviniemi
>
> Ian Campbell kirjoitti:
> > On Wed, 2009-01-07 at 15:28 +0200, Valtteri Kiviniemi wrote:
> > > Hi,
> > >
> > > I have now upgraded to the latest Xen with debug=y and I dont still 
> > > get any specific error messages or how do I use the debug?
> > Looks like it is crashing really very early before even earlyprintk. I
> > guess you need to decode the RIP and any likely looking return addresses
> > on the stack manually as I described in a previous mail.
> >
> > Ian.
> > > In xm dmesg I have now this:
> > >
> > > root@xen-1:/# xm dmesg
> > > (XEN) mm.c:1482:d5 Bad L1 flags 800000
> > > (XEN) traps.c:437:d5 Unhandled invalid opcode fault/trap [#6] on VCPU 
> > > 0 [ec=0000]
> > > (XEN) domain_crash_sync called from entry.S
> > > (XEN) Domain 5 (vcpu#0) crashed on cpu#2:
> > > (XEN) ----[ Xen-3.3.1  x86_64  debug=y  Not tainted ]----
> > > (XEN) CPU:    2
> > > (XEN) RIP:    e033:[<ffffffff8020a7aa>]
> > > (XEN) RFLAGS: 0000000000000282   EM: 1   CONTEXT: pv guest
> > > (XEN) rax: 00000000ffffffea   rbx: 8000000000000161   rcx: 
> > > 0000000000000008
> > > (XEN) rdx: 00000000deadbeef   rsi: 00000000deadbeef   rdi: 
> > > 00000000deadbeef
> > > (XEN) rbp: 0000000001000000   rsp: ffffffff80599f58   r8:  
> > > 0000000000000000
> > > (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 
> > > 0000000000000000
> > > (XEN) r12: ffffffff80622000   r13: 0000000000000200   r14: 
> > > 0000000000001000
> > > (XEN) r15: 0000000000000800   cr0: 000000008005003b   cr4: 
> > > 00000000000026b0
> > > (XEN) cr3: 000000021a1bc000   cr2: 0000000000000000
> > > (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> > > (XEN) Guest stack trace from rsp=ffffffff80599f58:
> > > (XEN)    0000000000000008 0000000000000000 ffffffff8020a7aa 
> > > 000000010000e030
> > > (XEN)    0000000000010082 ffffffff80599f98 000000000000e02b 
> > > ffffffff8020a7a6
> > > (XEN)    ffffffff8020c18c 0000000000000200 0000000000000008 
> > > ffffffff805c4d3f
> > > (XEN)    ffffffff80710000 0000000000010000 0000000000000008 
> > > 0000000000000000
> > > (XEN)    0000000000000000 0000000000000000 0000000000000000 
> > > 0000000000000000
> > > (XEN)    0000000000000000 ffffffff8068d000 ffffffff8068e000 
> > > ffffffff8068f000
> > > (XEN)    ffffffff80690000 ffffffff80691000 ffffffff80692000 
> > > ffffffff80693000
> > > (XEN)    ffffffff80694000 ffffffff80695000 ffffffff80696000 
> > > ffffffff80697000
> > > (XEN)    ffffffff80698000 ffffffff80699000 ffffffff8069a000 
> > > ffffffff8069b000
> > > (XEN)    ffffffff8069c000 ffffffff8069d000 ffffffff8069e000 
> > > ffffffff8069f000
> > > (XEN)    ffffffff806a0000 ffffffff806a1000 ffffffff806a2000 
> > > ffffffff806a3000
> > > (XEN)    ffffffff806a4000 ffffffff806a5000 ffffffff806a6000 
> > > ffffffff806a7000
> > > (XEN)    ffffffff806a8000 ffffffff806a9000 ffffffff806aa000 
> > > ffffffff806ab000
> > > (XEN)    ffffffff806ac000 ffffffff806ad000 ffffffff806ae000 
> > > ffffffff806af000
> > > (XEN)    ffffffff806b0000 ffffffff806b1000 ffffffff806b2000 
> > > ffffffff806b3000
> > > (XEN)    ffffffff806b4000 ffffffff806b5000 ffffffff806b6000 
> > > ffffffff806b7000
> > > (XEN)    ffffffff806b8000 ffffffff806b9000 ffffffff806ba000 
> > > ffffffff806bb000
> > > (XEN)    ffffffff806bc000 ffffffff806bd000 ffffffff806be000 
> > > ffffffff806bf000
> > > (XEN)    ffffffff806c0000 ffffffff806c1000 ffffffff806c2000 
> > > ffffffff806c3000
> > > (XEN)    ffffffff806c4000 ffffffff806c5000 ffffffff806c6000 
> > > ffffffff806c7000
> > > (XEN) mm.c:1482:d6 Bad L1 flags 800000
> > > (XEN) traps.c:437:d6 Unhandled invalid opcode fault/trap [#6] on VCPU 
> > > 0 [ec=0000]
> > > (XEN) domain_crash_sync called from entry.S
> > > (XEN) Domain 6 (vcpu#0) crashed on cpu#3:
> > > (XEN) ----[ Xen-3.3.1  x86_64  debug=y  Not tainted ]----
> > > (XEN) CPU:    3
> > > (XEN) RIP:    e033:[<ffffffff8020a7aa>]
> > > (XEN) RFLAGS: 0000000000000282   EM: 1   CONTEXT: pv guest
> > > (XEN) rax: 00000000ffffffea   rbx: 8000000000000161   rcx: 
> > > 0000000000000008
> > > (XEN) rdx: 00000000deadbeef   rsi: 00000000deadbeef   rdi: 
> > > 00000000deadbeef
> > > (XEN) rbp: 0000000001000000   rsp: ffffffff80599f58   r8:  
> > > 0000000000000000
> > > (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 
> > > 0000000000000000
> > > (XEN) r12: ffffffff80622000   r13: 0000000000000200   r14: 
> > > 0000000000001000
> > > (XEN) r15: 0000000000000800   cr0: 000000008005003b   cr4: 
> > > 00000000000026b0
> > > (XEN) cr3: 0000000219dbc000   cr2: 0000000000000000
> > > (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> > > (XEN) Guest stack trace from rsp=ffffffff80599f58:
> > > (XEN)    0000000000000008 0000000000000000 ffffffff8020a7aa 
> > > 000000010000e030
> > > (XEN)    0000000000010082 ffffffff80599f98 000000000000e02b 
> > > ffffffff8020a7a6
> > > (XEN)    ffffffff8020c18c 0000000000000200 0000000000000008 
> > > ffffffff805c4d3f
> > > (XEN)    ffffffff80710000 0000000000010000 0000000000000008 
> > > 0000000000000000
> > > (XEN)    0000000000000000 0000000000000000 0000000000000000 
> > > 0000000000000000
> > > (XEN)    0000000000000000 ffffffff8068d000 ffffffff8068e000 
> > > ffffffff8068f000
> > > (XEN)    ffffffff80690000 ffffffff80691000 ffffffff80692000 
> > > ffffffff80693000
> > > (XEN)    ffffffff80694000 ffffffff80695000 ffffffff80696000 
> > > ffffffff80697000
> > > (XEN)    ffffffff80698000 ffffffff80699000 ffffffff8069a000 
> > > ffffffff8069b000
> > > (XEN)    ffffffff8069c000 ffffffff8069d000 ffffffff8069e000 
> > > ffffffff8069f000
> > > (XEN)    ffffffff806a0000 ffffffff806a1000 ffffffff806a2000 
> > > ffffffff806a3000
> > > (XEN)    ffffffff806a4000 ffffffff806a5000 ffffffff806a6000 
> > > ffffffff806a7000
> > > (XEN)    ffffffff806a8000 ffffffff806a9000 ffffffff806aa000 
> > > ffffffff806ab000
> > > (XEN)    ffffffff806ac000 ffffffff806ad000 ffffffff806ae000 
> > > ffffffff806af000
> > > (XEN)    ffffffff806b0000 ffffffff806b1000 ffffffff806b2000 
> > > ffffffff806b3000
> > > (XEN)    ffffffff806b4000 ffffffff806b5000 ffffffff806b6000 
> > > ffffffff806b7000
> > > (XEN)    ffffffff806b8000 ffffffff806b9000 ffffffff806ba000 
> > > ffffffff806bb000
> > > (XEN)    ffffffff806bc000 ffffffff806bd000 ffffffff806be000 
> > > ffffffff806bf000
> > > (XEN)    ffffffff806c0000 ffffffff806c1000 ffffffff806c2000 
> > > ffffffff806c3000
> > > (XEN)    ffffffff806c4000 ffffffff806c5000 ffffffff806c6000 
> > > ffffffff806c7000
> > > - Valtteri Kiviniemi
> > >
> > > Valtteri Kiviniemi kirjoitti:
> > > > Hi,
> > > >
> > > > Yeah I'll upgrade to the newest Xen with debug enabled later, so 
> > > > I'll will post the results then.
> > > > - Valtteri Kiviniemi
> > > >
> > > > Ian Campbell kirjoitti:
> > > > > On Tue, 2009-01-06 at 16:37 +0200, Valtteri Kiviniemi wrote:
> > > > > > Hi,
> > > > > >
> > > > > > No it does not print anything, just disappears immediately.
> > > > > >
> > > > > > This is what I get in xm dmesg after trying to launch the domU 
> > > > > > with 2.6.28
> > > > > > http://pastebin.com/m6ab14265
> > > > > Since they aren't huge please just paste the messages inline or attach
> > > > > them rather than sending me off to some website.
> > > > >
> > > > > Looks like a hypercall has failed and the kernel has done a BUG():
> > > > >
> > > > > (XEN) mm.c:1482:d71 Bad L1 flags 800000
> > > > > (XEN) traps.c:437:d71 Unhandled invalid opcode fault/trap [#6] on 
> > > > > VCPU 0 [ec=0000]
> > > > > (XEN) domain_crash_sync called from entry.S
> > > > > (XEN) Domain 71 (vcpu#0) crashed on cpu#0:
> > > > > (XEN) ----[ Xen-3.3.1-rc1-pre  x86_64  debug=n  Not tainted ]----
> > > > > (XEN) CPU:    0
> > > > > (XEN) RIP:    e033:[<ffffffff8020a7aa>]
> > > > > (XEN) RFLAGS: 0000000000000282   EM: 1   CONTEXT: pv guest
> > > > > (XEN) rax: 00000000ffffffea   rbx: 8000000000000161   rcx: 
> > > > > 0000000000000008
> > > > > (XEN) rdx: 0000000000000000   rsi: 80000001d9ddd061   rdi: 
> > > > > ffffffff80622000
> > > > > (XEN) rbp: 0000000001000000   rsp: ffffffff80599f58   r8:  
> > > > > 0000000000000000
> > > > > (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 
> > > > > 0000000000000000
> > > > > (XEN) r12: ffffffff80622000   r13: 0000000000000200   r14: 
> > > > > 0000000000001000
> > > > > (XEN) r15: 0000000000000800   cr0: 000000008005003b   cr4: 
> > > > > 00000000000026b0
> > > > > (XEN) cr3: 00000001d9cef000   cr2: 0000000000000000
> > > > > (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> > > > > (XEN) Guest stack trace from rsp=ffffffff80599f58:
> > > > > (XEN)    0000000000000008 0000000000000000 ffffffff8020a7aa 
> > > > > 000000010000e030
> > > > > (XEN)    0000000000010082 ffffffff80599f98 000000000000e02b 
> > > > > ffffffff8020a7a6
> > > > > (XEN)    ffffffff8020c18c 0000000000000200 0000000000000008 
> > > > > ffffffff805c4d3f
> > > > > (XEN)    ffffffff80710000 0000000000010000 0000000000000008 
> > > > > 0000000000000000
> > > > > (XEN)    0000000000000000 0000000000000000 0000000000000000 
> > > > > 0000000000000000
> > > > > (XEN)    0000000000000000 ffffffff8068d000 ffffffff8068e000 
> > > > > ffffffff8068f000
> > > > > (XEN)    ffffffff80690000 ffffffff80691000 ffffffff80692000 
> > > > > ffffffff80693000
> > > > > (XEN)    ffffffff80694000 ffffffff80695000 ffffffff80696000 
> > > > > ffffffff80697000
> > > > > (XEN)    ffffffff80698000 ffffffff80699000 ffffffff8069a000 
> > > > > ffffffff8069b000
> > > > > (XEN)    ffffffff8069c000 ffffffff8069d000 ffffffff8069e000 
> > > > > ffffffff8069f000
> > > > > (XEN)    ffffffff806a0000 ffffffff806a1000 ffffffff806a2000 
> > > > > ffffffff806a3000
> > > > > (XEN)    ffffffff806a4000 ffffffff806a5000 ffffffff806a6000 
> > > > > ffffffff806a7000
> > > > > (XEN)    ffffffff806a8000 ffffffff806a9000 ffffffff806aa000 
> > > > > ffffffff806ab000
> > > > > (XEN)    ffffffff806ac000 ffffffff806ad000 ffffffff806ae000 
> > > > > ffffffff806af000
> > > > > (XEN)    ffffffff806b0000 ffffffff806b1000 ffffffff806b2000 
> > > > > ffffffff806b3000
> > > > > (XEN)    ffffffff806b4000 ffffffff806b5000 ffffffff806b6000 
> > > > > ffffffff806b7000
> > > > > (XEN)    ffffffff806b8000 ffffffff806b9000 ffffffff806ba000 
> > > > > ffffffff806bb000
> > > > > (XEN)    ffffffff806bc000 ffffffff806bd000 ffffffff806be000 
> > > > > ffffffff806bf000
> > > > > (XEN)    ffffffff806c0000 ffffffff806c1000 ffffffff806c2000 
> > > > > ffffffff806c3000
> > > > > (XEN)    ffffffff806c4000 ffffffff806c5000 ffffffff806c6000 
> > > > > ffffffff806c7000
> > > > > What does ffffffff8020a7aa correspond to in your kernel? Do any of the
> > > > > other values on the stack correspond to interesting functions? I 
> > > > > usually
> > > > > pipe such stack traces into "| gdb-disas vmlinux" (attached, needs
> > > > > kernel with debug info), of course it would be better to update to 
> > > > > 3.3.1
> > > > > final and build with debugging enabled to see the actual stack trace
> > > > > from the domU kernel.
> > > > >
> > > > >
> > > > > > - Valtteri Kiviniemi
> > > > > >
> > > > > > Ian Campbell kirjoitti:
> > > > > > > On Mon, 2009-01-05 at 20:11 +0200, Valtteri Kiviniemi wrote:
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I don't have the opportunity of rebuilding my hypervisor right 
> > > > > > > > now, but
> > > > > > > > here is the .config:
> > > > > > > >
> > > > > > > > http://pastebin.com/m181cf183
> > > > > > > >
> > > > > > > > Is there something wrong with the config or am I missing 
> > > > > > > > something there?
> > > > > > > No I think your config is ok, I successfully built and booted 
> > > > > > > 2.6.28 as
> > > > > > > a 64 bit domU on my test system using it. My command line was
> > > > > > > "root=/dev/xvda1 ro 4 console=hvc0 earlyprintk=xen"
> > > > > > >
> > > > > > > Do you get anything on the Xen console (xm dmesg) when you try to 
> > > > > > > start
> > > > > > > the guest?
> > > > > > >
> > > > > > >
> > > > > > > > - Valtteri Kiviniemi
> > > > > > > >
> > > > > > > > Ian Campbell kirjoitti:
> > > > > > > > > On Mon, 2009-01-05 at 14:19 +0200, Valtteri Kiviniemi wrote:
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > Im using vanilla 2.6.28 kernel from kernel.org and when I try 
> > > > > > > > > > to boot my   domU with it, it just disappears without any 
> > > > > > > > > > error or warning.
> > > > > > > > > Can you build a debug=y hypervisor and add earlyprintk=xen to 
> > > > > > > > > the domU
> > > > > > > > > command line?
> > > > > > > > >
> > > > > > > > > > I have double checked that I have all the necessary options 
> > > > > > > > > > enabled on my kernel config.
> > > > > > > > > Can you supply the config pls.
> > > > > > > > >
> > > > > > > > > Ian.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > _______________________________________________
> > > > > > > > > Xen-devel mailing list
> > > > > > > > > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > > > > > > > > http://lists.xensource.com/xen-devel
> > > > > > > > _______________________________________________
> > > > > > > > Xen-devel mailing list
> > > > > > > > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > > > > > > > http://lists.xensource.com/xen-devel
> > > > > > > _______________________________________________
> > > > > > > Xen-devel mailing list
> > > > > > > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > > > > > > http://lists.xensource.com/xen-devel
> > > > _______________________________________________
> > > > Xen-devel mailing list
> > > > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > > > http://lists.xensource.com/xen-devel
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel