Re: [Xen-devel] 2.6.28 64-bit domU not booting

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Wed, 2009-01-07 at 15:28 +0200, Valtteri Kiviniemi wrote:
> Hi,
> 
> I have now upgraded to the latest Xen with debug=y and I dont still get 
> any specific error messages or how do I use the debug?

Looks like it is crashing really very early before even earlyprintk. I
guess you need to decode the RIP and any likely looking return addresses
on the stack manually as I described in a previous mail.

Ian.
> 
> In xm dmesg I have now this:
> 
> root@xen-1:/# xm dmesg
> (XEN) mm.c:1482:d5 Bad L1 flags 800000
> (XEN) traps.c:437:d5 Unhandled invalid opcode fault/trap [#6] on VCPU 0 
> [ec=0000]
> (XEN) domain_crash_sync called from entry.S
> (XEN) Domain 5 (vcpu#0) crashed on cpu#2:
> (XEN) ----[ Xen-3.3.1  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    2
> (XEN) RIP:    e033:[<ffffffff8020a7aa>]
> (XEN) RFLAGS: 0000000000000282   EM: 1   CONTEXT: pv guest
> (XEN) rax: 00000000ffffffea   rbx: 8000000000000161   rcx: 0000000000000008
> (XEN) rdx: 00000000deadbeef   rsi: 00000000deadbeef   rdi: 00000000deadbeef
> (XEN) rbp: 0000000001000000   rsp: ffffffff80599f58   r8:  0000000000000000
> (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
> (XEN) r12: ffffffff80622000   r13: 0000000000000200   r14: 0000000000001000
> (XEN) r15: 0000000000000800   cr0: 000000008005003b   cr4: 00000000000026b0
> (XEN) cr3: 000000021a1bc000   cr2: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> (XEN) Guest stack trace from rsp=ffffffff80599f58:
> (XEN)    0000000000000008 0000000000000000 ffffffff8020a7aa 000000010000e030
> (XEN)    0000000000010082 ffffffff80599f98 000000000000e02b ffffffff8020a7a6
> (XEN)    ffffffff8020c18c 0000000000000200 0000000000000008 ffffffff805c4d3f
> (XEN)    ffffffff80710000 0000000000010000 0000000000000008 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN)    0000000000000000 ffffffff8068d000 ffffffff8068e000 ffffffff8068f000
> (XEN)    ffffffff80690000 ffffffff80691000 ffffffff80692000 ffffffff80693000
> (XEN)    ffffffff80694000 ffffffff80695000 ffffffff80696000 ffffffff80697000
> (XEN)    ffffffff80698000 ffffffff80699000 ffffffff8069a000 ffffffff8069b000
> (XEN)    ffffffff8069c000 ffffffff8069d000 ffffffff8069e000 ffffffff8069f000
> (XEN)    ffffffff806a0000 ffffffff806a1000 ffffffff806a2000 ffffffff806a3000
> (XEN)    ffffffff806a4000 ffffffff806a5000 ffffffff806a6000 ffffffff806a7000
> (XEN)    ffffffff806a8000 ffffffff806a9000 ffffffff806aa000 ffffffff806ab000
> (XEN)    ffffffff806ac000 ffffffff806ad000 ffffffff806ae000 ffffffff806af000
> (XEN)    ffffffff806b0000 ffffffff806b1000 ffffffff806b2000 ffffffff806b3000
> (XEN)    ffffffff806b4000 ffffffff806b5000 ffffffff806b6000 ffffffff806b7000
> (XEN)    ffffffff806b8000 ffffffff806b9000 ffffffff806ba000 ffffffff806bb000
> (XEN)    ffffffff806bc000 ffffffff806bd000 ffffffff806be000 ffffffff806bf000
> (XEN)    ffffffff806c0000 ffffffff806c1000 ffffffff806c2000 ffffffff806c3000
> (XEN)    ffffffff806c4000 ffffffff806c5000 ffffffff806c6000 ffffffff806c7000
> (XEN) mm.c:1482:d6 Bad L1 flags 800000
> (XEN) traps.c:437:d6 Unhandled invalid opcode fault/trap [#6] on VCPU 0 
> [ec=0000]
> (XEN) domain_crash_sync called from entry.S
> (XEN) Domain 6 (vcpu#0) crashed on cpu#3:
> (XEN) ----[ Xen-3.3.1  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    3
> (XEN) RIP:    e033:[<ffffffff8020a7aa>]
> (XEN) RFLAGS: 0000000000000282   EM: 1   CONTEXT: pv guest
> (XEN) rax: 00000000ffffffea   rbx: 8000000000000161   rcx: 0000000000000008
> (XEN) rdx: 00000000deadbeef   rsi: 00000000deadbeef   rdi: 00000000deadbeef
> (XEN) rbp: 0000000001000000   rsp: ffffffff80599f58   r8:  0000000000000000
> (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
> (XEN) r12: ffffffff80622000   r13: 0000000000000200   r14: 0000000000001000
> (XEN) r15: 0000000000000800   cr0: 000000008005003b   cr4: 00000000000026b0
> (XEN) cr3: 0000000219dbc000   cr2: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> (XEN) Guest stack trace from rsp=ffffffff80599f58:
> (XEN)    0000000000000008 0000000000000000 ffffffff8020a7aa 000000010000e030
> (XEN)    0000000000010082 ffffffff80599f98 000000000000e02b ffffffff8020a7a6
> (XEN)    ffffffff8020c18c 0000000000000200 0000000000000008 ffffffff805c4d3f
> (XEN)    ffffffff80710000 0000000000010000 0000000000000008 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN)    0000000000000000 ffffffff8068d000 ffffffff8068e000 ffffffff8068f000
> (XEN)    ffffffff80690000 ffffffff80691000 ffffffff80692000 ffffffff80693000
> (XEN)    ffffffff80694000 ffffffff80695000 ffffffff80696000 ffffffff80697000
> (XEN)    ffffffff80698000 ffffffff80699000 ffffffff8069a000 ffffffff8069b000
> (XEN)    ffffffff8069c000 ffffffff8069d000 ffffffff8069e000 ffffffff8069f000
> (XEN)    ffffffff806a0000 ffffffff806a1000 ffffffff806a2000 ffffffff806a3000
> (XEN)    ffffffff806a4000 ffffffff806a5000 ffffffff806a6000 ffffffff806a7000
> (XEN)    ffffffff806a8000 ffffffff806a9000 ffffffff806aa000 ffffffff806ab000
> (XEN)    ffffffff806ac000 ffffffff806ad000 ffffffff806ae000 ffffffff806af000
> (XEN)    ffffffff806b0000 ffffffff806b1000 ffffffff806b2000 ffffffff806b3000
> (XEN)    ffffffff806b4000 ffffffff806b5000 ffffffff806b6000 ffffffff806b7000
> (XEN)    ffffffff806b8000 ffffffff806b9000 ffffffff806ba000 ffffffff806bb000
> (XEN)    ffffffff806bc000 ffffffff806bd000 ffffffff806be000 ffffffff806bf000
> (XEN)    ffffffff806c0000 ffffffff806c1000 ffffffff806c2000 ffffffff806c3000
> (XEN)    ffffffff806c4000 ffffffff806c5000 ffffffff806c6000 ffffffff806c7000
> 
> 
> - Valtteri Kiviniemi
> 
> Valtteri Kiviniemi kirjoitti:
> > Hi,
> > 
> > Yeah I'll upgrade to the newest Xen with debug enabled later, so I'll 
> > will post the results then.
> > 
> > 
> > - Valtteri Kiviniemi
> > 
> > Ian Campbell kirjoitti:
> >> On Tue, 2009-01-06 at 16:37 +0200, Valtteri Kiviniemi wrote:
> >>> Hi,
> >>>
> >>> No it does not print anything, just disappears immediately.
> >>>
> >>> This is what I get in xm dmesg after trying to launch the domU with 
> >>> 2.6.28
> >>>
> >>> http://pastebin.com/m6ab14265
> >>
> >> Since they aren't huge please just paste the messages inline or attach
> >> them rather than sending me off to some website.
> >>
> >> Looks like a hypercall has failed and the kernel has done a BUG():
> >>
> >> (XEN) mm.c:1482:d71 Bad L1 flags 800000
> >> (XEN) traps.c:437:d71 Unhandled invalid opcode fault/trap [#6] on VCPU 
> >> 0 [ec=0000]
> >> (XEN) domain_crash_sync called from entry.S
> >> (XEN) Domain 71 (vcpu#0) crashed on cpu#0:
> >> (XEN) ----[ Xen-3.3.1-rc1-pre  x86_64  debug=n  Not tainted ]----
> >> (XEN) CPU:    0
> >> (XEN) RIP:    e033:[<ffffffff8020a7aa>]
> >> (XEN) RFLAGS: 0000000000000282   EM: 1   CONTEXT: pv guest
> >> (XEN) rax: 00000000ffffffea   rbx: 8000000000000161   rcx: 
> >> 0000000000000008
> >> (XEN) rdx: 0000000000000000   rsi: 80000001d9ddd061   rdi: 
> >> ffffffff80622000
> >> (XEN) rbp: 0000000001000000   rsp: ffffffff80599f58   r8:  
> >> 0000000000000000
> >> (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 
> >> 0000000000000000
> >> (XEN) r12: ffffffff80622000   r13: 0000000000000200   r14: 
> >> 0000000000001000
> >> (XEN) r15: 0000000000000800   cr0: 000000008005003b   cr4: 
> >> 00000000000026b0
> >> (XEN) cr3: 00000001d9cef000   cr2: 0000000000000000
> >> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> >> (XEN) Guest stack trace from rsp=ffffffff80599f58:
> >> (XEN)    0000000000000008 0000000000000000 ffffffff8020a7aa 
> >> 000000010000e030
> >> (XEN)    0000000000010082 ffffffff80599f98 000000000000e02b 
> >> ffffffff8020a7a6
> >> (XEN)    ffffffff8020c18c 0000000000000200 0000000000000008 
> >> ffffffff805c4d3f
> >> (XEN)    ffffffff80710000 0000000000010000 0000000000000008 
> >> 0000000000000000
> >> (XEN)    0000000000000000 0000000000000000 0000000000000000 
> >> 0000000000000000
> >> (XEN)    0000000000000000 ffffffff8068d000 ffffffff8068e000 
> >> ffffffff8068f000
> >> (XEN)    ffffffff80690000 ffffffff80691000 ffffffff80692000 
> >> ffffffff80693000
> >> (XEN)    ffffffff80694000 ffffffff80695000 ffffffff80696000 
> >> ffffffff80697000
> >> (XEN)    ffffffff80698000 ffffffff80699000 ffffffff8069a000 
> >> ffffffff8069b000
> >> (XEN)    ffffffff8069c000 ffffffff8069d000 ffffffff8069e000 
> >> ffffffff8069f000
> >> (XEN)    ffffffff806a0000 ffffffff806a1000 ffffffff806a2000 
> >> ffffffff806a3000
> >> (XEN)    ffffffff806a4000 ffffffff806a5000 ffffffff806a6000 
> >> ffffffff806a7000
> >> (XEN)    ffffffff806a8000 ffffffff806a9000 ffffffff806aa000 
> >> ffffffff806ab000
> >> (XEN)    ffffffff806ac000 ffffffff806ad000 ffffffff806ae000 
> >> ffffffff806af000
> >> (XEN)    ffffffff806b0000 ffffffff806b1000 ffffffff806b2000 
> >> ffffffff806b3000
> >> (XEN)    ffffffff806b4000 ffffffff806b5000 ffffffff806b6000 
> >> ffffffff806b7000
> >> (XEN)    ffffffff806b8000 ffffffff806b9000 ffffffff806ba000 
> >> ffffffff806bb000
> >> (XEN)    ffffffff806bc000 ffffffff806bd000 ffffffff806be000 
> >> ffffffff806bf000
> >> (XEN)    ffffffff806c0000 ffffffff806c1000 ffffffff806c2000 
> >> ffffffff806c3000
> >> (XEN)    ffffffff806c4000 ffffffff806c5000 ffffffff806c6000 
> >> ffffffff806c7000
> >>
> >> What does ffffffff8020a7aa correspond to in your kernel? Do any of the
> >> other values on the stack correspond to interesting functions? I usually
> >> pipe such stack traces into "| gdb-disas vmlinux" (attached, needs
> >> kernel with debug info), of course it would be better to update to 3.3.1
> >> final and build with debugging enabled to see the actual stack trace
> >> from the domU kernel.
> >>
> >>
> >>> - Valtteri Kiviniemi
> >>>
> >>> Ian Campbell kirjoitti:
> >>>> On Mon, 2009-01-05 at 20:11 +0200, Valtteri Kiviniemi wrote:
> >>>>> Hi,
> >>>>>
> >>>>> I don't have the opportunity of rebuilding my hypervisor right now, 
> >>>>> but
> >>>>> here is the .config:
> >>>>>
> >>>>> http://pastebin.com/m181cf183
> >>>>>
> >>>>> Is there something wrong with the config or am I missing something 
> >>>>> there?
> >>>> No I think your config is ok, I successfully built and booted 2.6.28 as
> >>>> a 64 bit domU on my test system using it. My command line was
> >>>> "root=/dev/xvda1 ro 4 console=hvc0 earlyprintk=xen"
> >>>>
> >>>> Do you get anything on the Xen console (xm dmesg) when you try to start
> >>>> the guest?
> >>>>
> >>>>
> >>>>> - Valtteri Kiviniemi
> >>>>>
> >>>>> Ian Campbell kirjoitti:
> >>>>>> On Mon, 2009-01-05 at 14:19 +0200, Valtteri Kiviniemi wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> Im using vanilla 2.6.28 kernel from kernel.org and when I try to 
> >>>>>>> boot my   domU with it, it just disappears without any error or 
> >>>>>>> warning.
> >>>>>> Can you build a debug=y hypervisor and add earlyprintk=xen to the 
> >>>>>> domU
> >>>>>> command line?
> >>>>>>
> >>>>>>> I have double checked that I have all the necessary options 
> >>>>>>> enabled on my kernel config.
> >>>>>> Can you supply the config pls.
> >>>>>>
> >>>>>> Ian.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Xen-devel mailing list
> >>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>>>>> http://lists.xensource.com/xen-devel
> >>>>> _______________________________________________
> >>>>> Xen-devel mailing list
> >>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>>>> http://lists.xensource.com/xen-devel
> >>>>
> >>>> _______________________________________________
> >>>> Xen-devel mailing list
> >>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>>> http://lists.xensource.com/xen-devel
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel