The crash below happens every time I boot 32-bit dom0 pvops on 64-bit
xen-unstable, but only on some machines.
Circumstantial evidence, to be confirmed, suggests that 32-bit PAE
pvops dom0 on 32-bit PAE Xen doesn't work either, and that the XCP
kernel works fine on this hardware.
Crash dump below is from:
linux 2472a9ccf02b04a36300cf1d3515f3f14d40aa4b
qemu 706bc8c588cbda284a9579cdb7e494ad5c9d6383
xen 22267:00b92112b055
Massaging the oops file and feeding it and the dom0 kernel Symbols
file to ksymoops produces the output you find below.
Ian.
Oct 26 15:26:46 (XEN) trace.c:90:d32767 calc_tinfo_first_offset: NR_CPUs 128,
offset_in_bytes 25
Oct 26 15:26:46 (XEN) Xen trace buffers: disabled
Oct 26 15:26:46 (XEN) Std. Loglevel: All
Oct 26 15:26:46 (XEN) Guest Loglevel: All
Oct 26 15:26:46 (XEN) Xen is relinquishing VGA console.
Oct 26 15:26:46 (XEN) *** Serial input -> DOM0 (type 'CTRL-x' three times to
switch input to Xe)
Oct 26 15:26:46 (XEN) Freed 188kB init memory.
Oct 26 15:26:46 mapping kernel into physical memory
Oct 26 15:26:46 Xen: setup ISA identity maps
Oct 26 15:26:46 about to get started...
Oct 26 15:26:46 (XEN) mm.c:889:d0 Error getting mfn 3409e (pfn
5555555555555555) from L1 entry 0
Oct 26 15:26:46 (XEN) mm.c:4953:d0 ptwr_emulate: fixing up invalid PAE PTE
000000003409e023
Oct 26 15:26:46 (XEN) mm.c:889:d0 Error getting mfn 3409f (pfn
5555555555555555) from L1 entry 0
Oct 26 15:26:46 (XEN) mm.c:4953:d0 ptwr_emulate: fixing up invalid PAE PTE
000000003409f023
Oct 26 15:26:46 (XEN) d0:v0: unhandled page fault (ec=0000)
Oct 26 15:26:46 (XEN) Pagetable walk from 00000000c009fc00:
Oct 26 15:26:46 (XEN) L4[0x000] = 0000000235747027 0000000000001747
Oct 26 15:26:46 (XEN) L3[0x003] = 00000002358fd027 00000000000018fd
Oct 26 15:26:46 (XEN) L2[0x000] = 000000023620c067 000000000000220c
Oct 26 15:26:46 (XEN) L1[0x09f] = 0000000000000000 ffffffffffffffff
Oct 26 15:26:46 (XEN) domain_crash_sync called from entry.S
Oct 26 15:26:46 (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
Oct 26 15:26:46 (XEN) ----[ Xen-4.1-unstable x86_64 debug=y Not tainted ]----
Oct 26 15:26:46 (XEN) CPU: 0
Oct 26 15:26:46 (XEN) RIP: e019:[<00000000c17d500b>]
Oct 26 15:26:46 (XEN) RFLAGS: 0000000000000206 EM: 1 CONTEXT: pv guest
Oct 26 15:26:46 (XEN) rax: 000000000009fc00 rbx: 00000000c009fc00 rcx:
0000000000000001
Oct 26 15:26:46 (XEN) rdx: 0000000000000400 rsi: 0000000036dfe000 rdi:
0000000000000400
Oct 26 15:26:46 (XEN) rbp: 00000000c1745f00 rsp: 00000000c1745ee0 r8:
0000000000000000
Oct 26 15:26:46 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11:
0000000000000000
Oct 26 15:26:46 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14:
0000000000000000
Oct 26 15:26:46 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4:
00000000000026f0
Oct 26 15:26:46 (XEN) cr3: 000000023a3a9000 cr2: 00000000c009fc00
Oct 26 15:26:46 (XEN) ds: e021 es: e021 fs: 00d8 gs: 0000 ss: e021
cs: e019
Oct 26 15:26:46 (XEN) Guest stack trace from esp=c1745ee0:
Oct 26 15:26:46 (XEN) 00000000 c17d500b 0001e019 00010006 00000001 00000001
36dfe000 00000000
Oct 26 15:26:46 (XEN) c1745f0c c17d50fe 00000000 c1745fb0 c17cdd06 c1063e9a
00000000 c1745fc0
Oct 26 15:26:46 (XEN) 0218a600 00000000 01a68000 00000000 00722600 01a68000
c106439f 00000000
Oct 26 15:26:46 (XEN) 00000035 c1745f71 00000005 00000000 00000000 00000000
c1745f78 0000009e
Oct 26 15:26:46 (XEN) 205b0000 30202020 3030302e 5d303030 c1800020 00000000
00000000 c174f350
Oct 26 15:26:46 (XEN) c1745f8c c102d295 00000000 c1745fa0 c14e5178 00000000
c1800670 c174f350
Oct 26 15:26:46 (XEN) c1745fb0 00000000 c1800670 c174f350 c1745fc8 c17c75d9
c1659dee c14eb010
Oct 26 15:26:47 (XEN) c18019d0 00000000 c1745fd4 c17c70a2 c220b490 c1745ffc
c17cae57 1fc9dbf5
Oct 26 15:26:47 (XEN) 80980281 00100800 000106e5 00000001 00000000 c220b000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000
Oct 26 15:26:47 (XEN) Domain 0 crashed: rebooting machine in 5 seconds.
Oct 26 15:26:52 <Modem lines changed: -CTS -CD -DSR>
Oct 26 15:27:10 <Modem lines changed: +CTS +CD +DSR>
mariner:~> ksymoops -d -K -m t -L -V <u | grep -v 'Before first symbol'
DEBUG (parse): 'K' '(null)'
DEBUG (parse): 'm' 't'
DEBUG (parse): 'L' '(null)'
DEBUG (parse): 'V' '(null)'
DEBUG (convert_uname): /lib/modules/*r/ in
DEBUG (convert_uname): /lib/modules/2.6.26-2-amd64/ out
ksymoops 2.4.11 on x86_64 2.6.26-2-amd64. Options used
-V (specified)
-K (specified)
-L (specified)
-o /lib/modules/2.6.26-2-amd64/ (default)
-m t (specified)
DEBUG (main): level 1
DEBUG (read_env): default KSYMOOPS_NM=/usr/bin/nm
DEBUG (read_env): default KSYMOOPS_FIND=/usr/bin/find
DEBUG (read_env): default KSYMOOPS_OBJDUMP=/usr/bin/objdump
DEBUG (re_compile): '^([0-9a-fA-F]{4,}) +([^ ]) +([^ ]+)( +\[([^ ]+)\])?$' 5
sub expression(s)
DEBUG (re_compile): '^ *\[*<([0-9a-fA-F]{4,})>\]* *' 1 sub expression(s)
DEBUG (re_compile): '^ *<\[([0-9a-fA-F]{4,})\]> *' 1 sub expression(s)
DEBUG (re_compile): '^ *([0-9a-fA-F]{4,}) *' 1 sub expression(s)
No modules in ksyms, skipping objects
DEBUG (read_system_map): t
DEBUG (re_compile): '^(.*)_R.*[0-9a-fA-F]{8,}$' 1 sub expression(s)
DEBUG (ss_sort_na): System.map
DEBUG (merge_maps):
DEBUG (ss_sort_atn): merged
DEBUG (ss_sort_atn): merged
DEBUG (ss_sort_atn): merged
DEBUG (re_compile): '^( +|[^ ]{3} [ 0-9][0-9] [0-9]{2}:[0-9]{2}:[0-9]{2} [^ ]+(
kernel:)? +|<[0-9]+>|[0-9]+\|[^|]+\|[^|]+\|[^|]+\||[0-9][AC] [0-9]{3}
[0-9]{3}[cir][0-9]{2}:\+)' 2 sub expression(s)
DEBUG (re_compile): '^((Stack: )|(Stack from )|([0-9a-fA-F]{4,})|(Call Trace:
*)|(\[*<([0-9a-fA-F]{4,})>\]* *)|(Version_[0-9]+)|(Trace:)|(Call
backtrace:)|(bh:)|<\[([0-9a-fA-F]{4,})\]> *|(\([^ ]+\)
*\[*<([0-9a-fA-F]{4,})>\]* *)|([0-9]+ +base=)|(Kernel BackChain)|EBP
*EIP|0x([0-9a-fA-F]{4,}) *0x([0-9a-fA-F]{4,}) *|Process .*stackpage=|Code *:
|Kernel panic|In swapper task|kmem_free|swapper|Pid:|r[0-9]{1,2}
*[:=]|Corrupted stack page|invalid operand: |Oops: |Cpu:*
+[0-9]|current->tss|\*pde +=|EIP: |EFLAGS: |eax: |esi: |ds: |CR0:
|wait_on_|irq: |Stack dumps:|RAX: |RSP: |RIP: |RDX: |RBP: |FS: |CS: |CR2:
|PML4|pc[:=]|68060 access|Exception at |d[04]: |Frame format=|wb [0-9]
stat|push data: |baddr=|Arithmetic fault|Instruction fault|Bad unaligned
kernel|Forwarding unaligned exception|: unhandled unaligned exception|pc
*=|[rvtsa][0-9]+ *=|gp *=|spinlock stuck|tsk->|PSR: |[goli]0: |Instruction
DUMP: |spin_lock|TSTATE: |[goli]4: |Caller\[|CPU\[|MSR: |TASK = |last math
|GPR[0-9]+: |.* in .*\.[ch]:.*, line [0-9]+:|\$[0-9 ]+:|Hi *:|Lo *:|epc
*:|badvaddr *:|Status *:|Cause *:|Backtrace:|Function entered at|\*pgd
=|Internal error|pc :|sp
:|Flags:|Control:|WARNING:|this_stack:|i:|PSW|cr[0-9]+:|machine check|Exception
in |Program Check |System restart |IUCV |unexpected external |Kernel stack
|User process fault:|failing address|User PSW|User GPRS|User ACRS|Kernel
PSW|Kernel GPRS|Kernel ACRS|illegal operation|task:|Entering kdb|eax *=|esi
*=|ebp *=|ds *=|psr |unat |rnat |ldrs |xip |iip |ipsr |ifa |pr |itc |ifs |bsp
|[bfr][0-9]+ |irr[0-9] |General Exception|MCA|SAL|Processor State|Bank
[0-9]+|Register Stack|Processor Error Info|proc err map|proc state param|proc
lid|processor structure|check info|target identifier|IRP *: )' 18 sub
expression(s)
DEBUG (re_compile): 'Unable to handle kernel|Aiee|die_if_kernel|NMI |BUG
|\(L-TLB\)|\(NOTLB\)|\([0-9]\): Oops |: memory violation|: Exception at|:
Arithmetic fault|: Instruction fault|: arithmetic trap|: unaligned
trap|\([0-9]+\): (Whee|Oops|Kernel|.*Penguin|BOGUS)|kernel pc |trap at PC: |bad
area pc |NIP: | ra *==' 1 sub expression(s)
DEBUG (re_compile): '^(i[04]: |Instruction DUMP: |Caller\[)' 1 sub expression(s)
CPU: 0
DEBUG (re_compile): '^PSR: [0-9a-fA-F]+ PC: ([0-9a-fA-F]{4,}) *' 1 sub
expression(s)
DEBUG (re_compile): '^TSTATE: [0-9a-fA-F]{16} TPC: ([0-9a-fA-F]{4,}) *' 1 sub
expression(s)
DEBUG (re_compile): '(kernel pc |trap at PC: |bad area pc |NIP:
)([0-9a-fA-F]{4,}) *' 2 sub expression(s)
DEBUG (re_compile): '^epc *:+ *([0-9a-fA-F]{4,}) *' 1 sub expression(s)
DEBUG (re_compile): ' ip *:+ *\[*<([0-9a-fA-F]{4,})>\]* *' 1 sub expression(s)
DEBUG (re_compile): '[xi]ip *\([^)]*\) *: *0x([0-9a-fA-F]{4,}) *' 1 sub
expression(s)
DEBUG (re_compile): '(^0x([0-9a-fA-F]{4,}) *0x([0-9a-fA-F]{4,})
*.*+0x)|(^Entering kdb on processor.*0x([0-9a-fA-F]{4,}) *)' 5 sub expression(s)
DEBUG (re_compile): '^ *IRP *: *([0-9a-fA-F]{4,}) *' 1 sub expression(s)
DEBUG (re_compile): '^(EIP: +.*|RIP: +.*|PC *= *|pc *:
*)\[*<([0-9a-fA-F]{4,})>\]* *' 2 sub expression(s)
DEBUG (re_compile): '^spinlock stuck at ([0-9a-fA-F]{4,}) *.*owner.*at
([0-9a-fA-F]{4,}) *' 2 sub expression(s)
DEBUG (re_compile): '^spin_lock[^ ]*\(([0-9a-fA-F]{4,}) *.*stuck at
*([0-9a-fA-F]{4,}) *.*PC\(([0-9a-fA-F]{4,}) *' 3 sub expression(s)
DEBUG (re_compile): 'ra *=+ *([0-9a-fA-F]{4,}) *' 1 sub expression(s)
DEBUG (re_compile): '^(\$[0-9]{1,2}) *: *([0-9a-fA-F]{4,}) *' 2 sub
expression(s)
DEBUG (re_compile): '^(GPR[0-9]{1,2}) *: *([0-9a-fA-F]{4,}) *' 2 sub
expression(s)
DEBUG (re_compile): '^PSW *flags: *([0-9a-fA-F]{4,}) * *PSW addr:
*([0-9a-fA-F]{4,}) *' 2 sub expression(s)
DEBUG (re_compile): '^Kernel PSW: *([0-9a-fA-F]{4,}) *([0-9a-fA-F]{4,}) *' 2
sub expression(s)
DEBUG (re_compile): '^((Call Trace: *)|(Trace:)|(\[*<([0-9a-fA-F]{4,})>\]*
*)|(Call backtrace:)|([0-9a-fA-F]{4,}) *|Function entered at
(\[*<([0-9a-fA-F]{4,})>\]* *)|Caller\[([0-9a-fA-F]{4,})
*\]|(<\[([0-9a-fA-F]{4,})\]> *)|(\([0-9]+\) *(\[*<([0-9a-fA-F]{4,})>\]*
*))|([0-9]+ +base=0x([0-9a-fA-F]{4,}) *)|(Kernel BackChain.*))' 18 sub
expression(s)
DEBUG (re_compile):
'^((GP|o)?r[0-9]{1,2}|[goli][0-9]{1,2}|[eR][ABCD]X|[eR][DS]I|RBP|e[bs]p|[fsi]p|IRP|SRP|D?CCR|USP|MOF|ret_pc)
*[:=] *([0-9a-fA-F]{4,}) * *' 3 sub expression(s)
DEBUG (re_compile): '^(Kernel GPRS.*)' 1 sub expression(s)
DEBUG (re_compile): '^(IRP|SRP|D?CCR|USP|MOF): *([0-9a-fA-F]{4,}) *' 2 sub
expression(s)
DEBUG (re_compile): '^b[0-7] *(\([^)]*\) *)?: *(0x)?([0-9a-fA-F]{4,}) *' 3 sub
expression(s)
DEBUG (re_compile): '^([^ ]+) +[^ ] *([^ ]+).*\((L-TLB|NOTLB)\)' 3 sub
expression(s)
DEBUG (re_compile): '^((Instruction DUMP)|(Code:? *)): +((general
protection.*)|(Bad E?IP value.*)|(<[0-9]+>)|(([<(]?[0-9a-fA-F]+[>)]?
+)+[<(]?[0-9a-fA-F]+[>)]?))(.*)$' 10 sub expression(s)
RIP: e019:[<00000000c17d500b>]
DEBUG (re_compile): 'pc *: *\[*<([0-9a-fA-F]{4,})>\]* * *lr *:
*\[*<([0-9a-fA-F]{4,})>\]* *' 2 sub expression(s)
DEBUG (re_compile): 'pc *= *\[*<([0-9a-fA-F]{4,})>\]* * *ra *=
*\[*<([0-9a-fA-F]{4,})>\]* *' 2 sub expression(s)
Using defaults from ksymoops -t elf32-i386 -a i386
rax: 000000000009fc00 rbx: 00000000c009fc00 rcx: 0000000000000001
rdx: 0000000000000400 rsi: 0000000036dfe000 rdi: 0000000000000400
rbp: 00000000c1745f00 rsp: 00000000c1745ee0 r8: 0000000000000000
Warning (Oops_set_regs): garbage 'rsp: 00000000c1745ee0 r8:
0000000000000000' at end of register line ignored
r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000
r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000
r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0
Warning (Oops_set_regs): garbage 'cr0: 000000008005003b cr4:
00000000000026f0' at end of register line ignored
cr3: 000000023a3a9000 cr2: 00000000c009fc00
ds: e021 es: e021 fs: 00d8 gs: 0000 ss: e021 cs: e019
Trace: <00000000> <c17d500b> <0001e019> <00010006> <00000001> <00000001>
<36dfe000> <00000000>
DEBUG (re_compile): '^(\([0-9]+\) *)' 1 sub expression(s)
Trace: <c1745f0c> <c17d50fe> <00000000> <c1745fb0> <c17cdd06> <c1063e9a>
<00000000> <c1745fc0>
Trace: <0218a600> <00000000> <01a68000> <00000000> <00722600> <01a68000>
<c106439f> <00000000>
Trace: <00000035> <c1745f71> <00000005> <00000000> <00000000> <00000000>
<c1745f78> <0000009e>
Trace: <205b0000> <30202020> <3030302e> <5d303030> <c1800020> <00000000>
<00000000> <c174f350>
Trace: <c1745f8c> <c102d295> <00000000> <c1745fa0> <c14e5178> <00000000>
<c1800670> <c174f350>
Trace: <c1745fb0> <00000000> <c1800670> <c174f350> <c1745fc8> <c17c75d9>
<c1659dee> <c14eb010>
Trace: <c18019d0> <00000000> <c1745fd4> <c17c70a2> <c220b490> <c1745ffc>
<c17cae57> <1fc9dbf5>
Trace: <80980281> <00100800> <000106e5> <00000001> <00000000> <c220b000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000>
<00000000> <00000000>
Warning (Oops_read): Code line not seen, dumping what data is available
DEBUG (Oops_format):
>>EIP; c17d500b <smp_scan_config+35/ff> <=====
>>rbx; c009fc00 <phys_startup_32+bf09fc00/c0000000>
>>rsi; 36dfe000 <phys_startup_32+35dfe000/c0000000>
>>rbp; c1745f00 <init_thread_union+1f00/2000>
Trace; c17d500b <smp_scan_config+35/ff>
Trace; 36dfe000 <phys_startup_32+35dfe000/c0000000>
Trace; c1745f0c <init_thread_union+1f0c/2000>
Trace; c17d50fe <default_find_smp_config+29/5d>
Trace; c1745fb0 <init_thread_union+1fb0/2000>
Trace; c17cdd06 <setup_arch+84b/a4d>
Trace; c1063e9a <release_console_sem+1b0/1dd>
Trace; c1745fc0 <init_thread_union+1fc0/2000>
Trace; 0218a600 <phys_startup_32+118a600/c0000000>
Trace; 01a68000 <phys_startup_32+a68000/c0000000>
Trace; 01a68000 <phys_startup_32+a68000/c0000000>
Trace; c106439f <vprintk+30f/332>
Trace; c1745f71 <init_thread_union+1f71/2000>
Trace; c1745f78 <init_thread_union+1f78/2000>
Trace; 205b0000 <phys_startup_32+1f5b0000/c0000000>
Trace; 30202020 <phys_startup_32+2f202020/c0000000>
Trace; 3030302e <phys_startup_32+2f30302e/c0000000>
Trace; 5d303030 <phys_startup_32+5c303030/c0000000>
Trace; c1800020 <md_setup_args+f30/1400>
Trace; c174f350 <reboot_cpu+0/4>
Trace; c1745f8c <init_thread_union+1f8c/2000>
Trace; c102d295 <__raw_callee_save_xen_restore_fl+9/c>
Trace; c1745fa0 <init_thread_union+1fa0/2000>
Trace; c14e5178 <_spin_unlock_irqrestore+43/48>
Trace; c1800670 <map.24465+0/a00>
Trace; c174f350 <reboot_cpu+0/4>
Trace; c1745fb0 <init_thread_union+1fb0/2000>
Trace; c1800670 <map.24465+0/a00>
Trace; c174f350 <reboot_cpu+0/4>
Trace; c1745fc8 <init_thread_union+1fc8/2000>
Trace; c17c75d9 <start_kernel+86/31a>
Trace; c1659dee <kallsyms_token_index+4d6/c9a6>
Trace; c14eb010 <linux_banner+0/8c>
Trace; c18019d0 <command_line+0/800>
Trace; c1745fd4 <init_thread_union+1fd4/2000>
Trace; c17c70a2 <i386_start_kernel+91/96>
Trace; c220b490 <END_OF_CODE+7a3490/????>
Trace; c1745ffc <init_thread_union+1ffc/2000>
Trace; c17cae57 <xen_start_kernel+535/53d>
Trace; 1fc9dbf5 <phys_startup_32+1ec9dbf5/c0000000>
Trace; 80980281 <phys_startup_32+7f980281/c0000000>
Trace; c220b000 <END_OF_CODE+7a3000/????>
3 warnings issued. Results may not be reliable.
mariner:~> _______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|