WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] udev oops, and system boot failure, with 2.6.32.44 as PV

To: Jeremy Fitzhardinge <jeremy@xxxxxxxx>
Subject: Re: [Xen-devel] udev oops, and system boot failure, with 2.6.32.44 as PV guest
From: Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>
Date: Thu, 1 Sep 2011 16:35:51 +0100
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Thu, 01 Sep 2011 08:37:30 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4E4D81C8.7000900@xxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Newsgroups: chiark.mail.xen.devel
References: <20044.62811.430835.316774@xxxxxxxxxxxxxxxxxxxxxxxx> <4E4D81C8.7000900@xxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Jeremy Fitzhardinge writes ("Re: [Xen-devel] udev oops, and system boot 
failure, with 2.6.32.44 as PV guest"):
> On 08/18/2011 04:19 AM, Ian Jackson wrote:
> > This failure happens only on these two machines, for some reason.
> > I haven't tried 32-bit.

I see crashes with 32-on-64 too.

> At first glance it doesn't really look very Xen-related; alloc_fd isn't
> generally a place where anything Xen-specific happens.   Can you decode
> that to a specific line of code?

There doesn't seem to be much point, given that different crashes have
different locations.  I tried a number of boots and got the stack
backtraces you can see below.

Something is obviously completely buggered.

> I'm wondering if the access to "/sys/devices/virtual/bdi/1:13/uevent" is
> pertinent though; it could be one of our drivers which is doing the
> wrong thing which causes alloc_fd to explode.

No, it gives a different access each time.

> Is this expected, or does it indicate something wrong with your
> (initramfs?) confg?

I don't think anything is wrong with my initramfs.  It works fine with
other kernels :-).  The messages about volume group "rice-weevil"
being missing are simply because I reuse the host's initramfs, which
has had stuff about the host's disk layout encoded into it by the
host's initramfs-tools, and is harmless.

Ian.


Starting the hotplug events dispatcher: udevd[    1.240492] udev[835]: starting 
version 164
.
[    1.335497] BUG: unable to handle kernel NULL pointer dereference at (null)
[    1.335536] IP: [<c1051d38>] __wake_up_common+0x17/0x5c
[    1.335562] *pdpt = 000000000175c007 *pde = 0000000000000000 
[    1.335590] Oops: 0000 [#1] SMP 
[    1.335614] last sysfs file: /sys/kernel/uevent_seqnum
[    1.335627] Modules linked in: [last unloaded: scsi_wait_scan]
[    1.335653] 
[    1.335664] Pid: 844, comm: mv Not tainted (2.6.32.45 #1) 
[    1.335678] EIP: 0061:[<c1051d38>] EFLAGS: 00010093 CPU: 0
[    1.335692] EIP is at __wake_up_common+0x17/0x5c
[    1.335705] EAX: dfcb290c EBX: fffffff4 ECX: 00000001 EDX: 00000001
[    1.335720] ESI: dfcb0008 EDI: 00000001 EBP: dbb51e90 ESP: dbb51e78
[    1.335734]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[    1.335748] Process mv (pid: 844, ti=dbb50000 task=c4ef8f30 task.ti=dbb50000)
[    1.335763] Stack:
[    1.335772]  00000011 dbb51e84 00000001 dfcb2908 dfcb0008 00000011 dbb51eb0 
c1056e39
[    1.335828] <0> 00000001 c4ef8f30 00000001 00000001 c4ef8f30 00000011 
dbb51ebc c1062759
[    1.335889] <0> c4ef8f30 dbb51f5c c106f763 c4efef40 c4ef0044 c4eff444 
00000011 00000000
[    1.335958] Call Trace:
[    1.335976]  [<c1056e39>] ? __wake_up_sync_key+0x33/0x45
[    1.335996]  [<c1062759>] ? __wake_up_parent+0x1e/0x21
[    1.336015]  [<c106f763>] ? do_notify_parent+0x17e/0x19c
[    1.336036]  [<c10b880a>] ? perf_event_exit_task+0x1e/0x2b2
[    1.336059]  [<c146412e>] ? _write_lock_irq+0x18/0x2a
[    1.336070]  [<c1069d9a>] ? exit_ptrace+0xa3/0x10d
[    1.336070]  [<c1079c11>] ? switch_task_namespaces+0xf/0x3a
[    1.336070]  [<c106467f>] ? do_exit+0x553/0x608
[    1.336070]  [<c10647bc>] ? do_group_exit+0x88/0xab
[    1.336070]  [<c10647f2>] ? sys_exit_group+0x13/0x17
[    1.336070]  [<c102ea49>] ? syscall_call+0x7/0xb
[    1.336070] Code: 89 e5 e8 9b ff ff ff 5d c3 55 8b 80 88 02 00 00 89 e5 5d 
c3 55 89 e5 57 89 d7 56 53 83 ec 0c 89 4d f0 8b 58 04 83 c0 04 83 eb 0c <8b> 73 
0c 89 45 e8 83 ee 0c eb 2a 8b 03 89 fa ff 75 0c 8b 4d 08 
[    1.336070] EIP: [<c1051d38>] __wake_up_common+0x17/0x5c SS:ESP 0069:dbb51e78
[    1.336070] CR2: 0000000000000000
[    1.336070] ---[ end trace 59579aaa0506cac8 ]---
[    1.336070] Fixing recursive fault but reboot is needed!


Starting the hotplug events dispatcher: udevd[    1.200636] udev[839]: starting 
version 164
.
Synthesizing the initial hotplug events...done.
Waiting for /dev to be fully populated...[    1.546234] BUG: unable to handle 
kernel NULL pointer dereference at 00000008
[    1.546258] IP: [<c11c1c01>] rb_erase+0x72/0x208
[    1.546272] *pdpt = 000000001fdcc007 *pde = 0000000000000000 
[    1.546284] Oops: 0002 [#1] SMP 
[    1.546295] last sysfs file: /sys/devices/virtual/vtconsole/vtcon0/uevent
[    1.546302] Modules linked in: [last unloaded: scsi_wait_scan]
[    1.546314] 
[    1.546319] Pid: 855, comm: udevd Not tainted (2.6.32.45 #1) 
[    1.546325] EIP: 0061:[<c11c1c01>] EFLAGS: 00010046 CPU: 0
[    1.546332] EIP is at rb_erase+0x72/0x208
[    1.546337] EAX: dbbfd004 EBX: 00000000 ECX: c4da8b84 EDX: 00000000
[    1.546344] ESI: c5ff0388 EDI: 00000000 EBP: c52c5f04 ESP: c52c5eec
[    1.546350]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069
[    1.546357] Process udevd (pid: 855, ti=c52c4000 task=dbbab7b0 
task.ti=c52c4000)
[    1.546363] Stack:
[    1.546368]  00000000 00000000 00000000 dbbfd004 c5ff0380 00000000 c52c5f18 
c1078bd5
[    1.546393] <0> dbbfd004 048a2000 c52c5f84 c52c5f30 c1078c28 00000001 
c5ff0380 dbbfd004
[    1.546420] <0> ffffffff c52c5f44 c10793a4 c5ff0050 dbbab7b0 c52c5f94 
c52c5f7c c1064ae1
[    1.546451] Call Trace:
[    1.546461]  [<c1078bd5>] ? __remove_hrtimer+0x64/0x6c
[    1.546469]  [<c1078c28>] ? remove_hrtimer+0x4b/0x58
[    1.546478]  [<c10793a4>] ? hrtimer_try_to_cancel+0x24/0x3a
[    1.546488]  [<c1064ae1>] ? do_setitimer+0xaa/0x1f5
[    1.546497]  [<c10e93af>] ? __fput+0x161/0x169
[    1.546505]  [<c1064cd2>] ? alarm_setitimer+0x35/0x54
[    1.546515]  [<c106d1f6>] ? sys_alarm+0xb/0xd
[    1.546524]  [<c102ea49>] ? syscall_call+0x7/0xb
[    1.546530] Code: 8b 19 8b 51 04 89 5d ec 83 e3 fc 39 c3 89 5d f0 89 5d e8 
75 05 89 4d e8 eb 26 85 d2 74 0a 8b 3a 83 e7 03 0b 7d f0 89 3a 8b 7d f0 <89> 57 
08 8b 78 04 89 79 04 8b 58 04 8b 3b 83 e7 03 09 cf 89 3b 
[    1.546713] EIP: [<c11c1c01>] rb_erase+0x72/0x208 SS:ESP 0069:c52c5eec
[    1.546726] CR2: 0000000000000008
[    1.546733] ---[ end trace cafed11e7d7abcb5 ]---




Starting the hotplug events dispatcher: udevd[    1.260149] udev[838]: starting 
version 164
.
Synthesizing the initial hotplug events...done.
Waiting for /dev to be fully populated...[    1.647871] BUG: unable to handle 
kernel NULL pointer dereference at (null)
[    1.647896] IP: [<c1051d38>] __wake_up_common+0x17/0x5c
[    1.647909] *pdpt = 000000000175c007 *pde = 0000000000000000 
[    1.647922] Oops: 0000 [#1] SMP 
[    1.647933] last sysfs file: /sys/devices/virtual/input/input0/mouse0/uevent
[    1.647940] Modules linked in: [last unloaded: scsi_wait_scan]
[    1.647952] 
[    1.647957] Pid: 934, comm: grep Not tainted (2.6.32.45 #1) 
[    1.647964] EIP: 0061:[<c1051d38>] EFLAGS: 00010093 CPU: 0
[    1.647971] EIP is at __wake_up_common+0x17/0x5c
[    1.647977] EAX: dea9f20c EBX: fffffff4 ECX: 00000001 EDX: 00000001
[    1.647983] ESI: dea90008 EDI: 00000001 EBP: dead5e90 ESP: dead5e78
[    1.647990]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
[    1.647996] Process grep (pid: 934, ti=dead4000 task=dea98000 
task.ti=dead4000)
[    1.648003] Stack:
[    1.648008]  00000011 dead5e84 00000001 dea9f208 dea90008 00000011 dead5eb0 
c1056e39
[    1.648033] <0> 00000001 dea98000 00000001 00000001 dea98000 00000011 
dead5ebc c1062759
[    1.648053] <0> dea98000 dead5f5c c106f763 c4df39c0 c4df00c4 c4df3ec4 
00000011 00000000
[    1.648053] Call Trace:
[    1.648053]  [<c1056e39>] ? __wake_up_sync_key+0x33/0x45
[    1.648053]  [<c1062759>] ? __wake_up_parent+0x1e/0x21
[    1.648053]  [<c106f763>] ? do_notify_parent+0x17e/0x19c
[    1.648053]  [<c10b880a>] ? perf_event_exit_task+0x1e/0x2b2
[    1.648053]  [<c146412e>] ? _write_lock_irq+0x18/0x2a
[    1.648053]  [<c1069d9a>] ? exit_ptrace+0xa3/0x10d
[    1.648053]  [<c1079c11>] ? switch_task_namespaces+0xf/0x3a
[    1.648053]  [<c106467f>] ? do_exit+0x553/0x608
[    1.648053]  [<c10647bc>] ? do_group_exit+0x88/0xab
[    1.648053]  [<c10647f2>] ? sys_exit_group+0x13/0x17
[    1.648053]  [<c102ea49>] ? syscall_call+0x7/0xb
[    1.648053] Code: 89 e5 e8 9b ff ff ff 5d c3 55 8b 80 88 02 00 00 89 e5 5d 
c3 55 89 e5 57 89 d7 56 53 83 ec 0c 89 4d f0 8b 58 04 83 c0 04 83 eb 0c <8b> 73 
0c 89 45 e8 83 ee 0c eb 2a 8b 03 89 fa ff 75 0c 8b 4d 08 
[    1.648053] EIP: [<c1051d38>] __wake_up_common+0x17/0x5c SS:ESP 0069:dead5e78
[    1.648053] CR2: 0000000000000000
[    1.648053] ---[ end trace 6942a97668899ff4 ]---
[    1.648053] Fixing recursive fault but reboot is needed!



Using makefile-style concurrent boot in runlevel S.
[    1.133364] BUG: unable to handle kernel NULL pointer dereference at 00000004
[    1.133398] IP: [<c10767b5>] add_wait_queue+0x1b/0x36
[    1.133423] *pdpt = 000000001bfe4027 *pde = 0000000000000000 
[    1.133449] Oops: 0002 [#1] SMP 
[    1.133472] last sysfs file: /sys/kernel/uevent_seqnum
[    1.133485] Modules linked in: [last unloaded: scsi_wait_scan]
[    1.133510] 
[    1.133522] Pid: 808, comm: startpar Not tainted (2.6.32.45 #1) 
[    1.133536] EIP: 0061:[<c10767b5>] EFLAGS: 00010096 CPU: 0
[    1.133550] EIP is at add_wait_queue+0x1b/0x36
[    1.133563] EAX: c4f30208 EBX: c4f34908 ECX: dfce9f7c EDX: c4f3490c
[    1.133577] ESI: dfce9f70 EDI: 00000000 EBP: dfce9f20 ESP: dfce9f14
[    1.133591]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069
[    1.133604] Process startpar (pid: 808, ti=dfce8000 task=c4f28a20 
task.ti=dfce8000)
[    1.133619] Stack:
[    1.133629]  dfce9f58 00000000 00000000 dfce9f4c c1063dd3 c5384380 c5384980 
dfce9f70
[    1.133683] <0> dfce9f48 00000010 c4f28a20 ffffffff 00000000 00000000 
dfce9f94 c1063fcf
[    1.133744] <0> c10e82b9 00000003 00000007 00000000 00000000 bf87ee9c 
00000000 00000000
[    1.133811] Call Trace:
[    1.133829]  [<c1063dd3>] ? do_wait+0x61/0x1d5
[    1.133847]  [<c1063fcf>] ? sys_wait4+0x88/0xa1
[    1.133865]  [<c10e82b9>] ? rw_verify_area+0x98/0xbb
[    1.133884]  [<c10626dc>] ? child_wait_callback+0x0/0x5f
[    1.133902]  [<c1063ffb>] ? sys_waitpid+0x13/0x15
[    1.133922]  [<c102ea49>] ? syscall_call+0x7/0xb
[    1.133934] Code: 89 39 89 c2 89 d8 e8 00 d9 3e 00 5b 5e 5f 5d c3 55 89 e5 
57 56 89 d6 53 89 c3 83 22 fe e8 70 da 3e 00 8b 7b 0
4 8d 4e 0c 8d 53 04 <89> 4f 04 89 7e 0c 89 56 10 89 c2 89 d8 89 4b 04 e8 cb d8 
3e 00 
[    1.134058] EIP: [<c10767b5>] add_wait_queue+0x1b/0x36 SS:ESP 0069:dfce9f14
[    1.134058] CR2: 0000000000000004
[    1.134058] ---[ end trace 85d46112ef8f4b48 ]---
[    1.134895] ------------[ cut here ]------------
[    1.134912] kernel BUG at kernel/exit.c:84!
[    1.134924] invalid opcode: 0000 [#2] SMP 
[    1.134948] last sysfs file: /sys/kernel/uevent_seqnum
[    1.134960] Modules linked in: [last unloaded: scsi_wait_scan]
[    1.134984] 
[    1.134996] Pid: 805, comm: rc Tainted: G      D    (2.6.32.45 #1) 
[    1.135010] EIP: 0061:[<c1062ec1>] EFLAGS: 00010046 CPU: 0
[    1.135025] EIP is at release_task+0x73/0x3d4
[    1.135038] EAX: 00000000 EBX: c4f28a20 ECX: c1668980 EDX: 02218e31
[    1.135054] ESI: c4f34900 EDI: c174e2e0 EBP: dfde7ed0 ESP: dfde7eb8
[    1.135068]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069
[    1.135081] Process rc (pid: 805, ti=dfde6000 task=c4f29440 task.ti=dfde6000)
[    1.135096] Stack:
[    1.135105]  c4f28a20 dfde7ec4 c102ce90 bffffffd 00000328 c4f28a20 dfde7f38 
c1063967
[    1.135159] <0> 00003493 00000000 00000166 00000000 00000000 00000328 
00000001 00000000
[    1.135220] <0> 00000022 00000000 00000018 00000000 00000000 00000000 
00000000 00000000
[    1.135287] Call Trace:
[    1.135303]  [<c102ce90>] ? xen_spin_lock+0xa/0xe
[    1.135321]  [<c1063967>] ? wait_consider_task+0x745/0xb50
[    1.135340]  [<c1063e47>] ? do_wait+0xd5/0x1d5
[    1.135358]  [<c1063fcf>] ? sys_wait4+0x88/0xa1
[    1.135376]  [<c10626dc>] ? child_wait_callback+0x0/0x5f
[    1.135395]  [<c102ea49>] ? syscall_call+0x7/0xb
[    1.135405] Code: e8 d9 6c 00 00 8d 83 34 02 00 00 39 83 34 02 00 00 74 04 
0f 0b eb fe 8b b3 a8 03 00 00 85 f6 75 04 0f 0b eb f
e 8b 06 85 c0 75 04 <0f> 0b eb fe 8b 83 ac 03 00 00 89 45 f0 05 04 05 00 00 e8 
00 12 
[    1.135405] EIP: [<c1062ec1>] release_task+0x73/0x3d4 SS:ESP 0069:dfde7eb8
[    1.135405] ---[ end trace 85d46112ef8f4b49 ]---

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>
  • Re: [Xen-devel] udev oops, and system boot failure, with 2.6.32.44 as PV guest, Ian Jackson <=