WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 		  
Home Products Support Community News
 
   
 

xen-devel

[Top] [All Lists]
<prev [Date] next>
[Advanced]
 <prev [Thread] next>

Re: [Xen-devel] kernel BUG at arch/x86/xen/mmu.c:1860!

from [Teck Choon Giam] [Permanent Link][Original]
To: Sander Eikelenboom <linux@xxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] kernel BUG at arch/x86/xen/mmu.c:1860!
From: Teck Choon Giam <giamteckchoon@xxxxxxxxx>
Date: Sat, 12 Mar 2011 04:29:34 +0800
Cc: Andreas Olsowski <andreas.olsowski@xxxxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx, benco <benco@xxxxxxx>
Delivery-date: Fri, 11 Mar 2011 12:30:39 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=0djqjtuQDAbPjPVqbKwwAFgoKuZ2S90Kz/Ql+ppGm50=; b=Kcm+Hu2flaJUa/iypX5nyH7/vmLHs42xDobN7eqiNLHc5nqSahqDadg1lMYEVq6CrR UhM+j1zR00StWacwC9QuEP7ybxyQaudkqHtenKo2b7Lr9RLSdhbRcHRts3VLJ5beMOFR qNzt6n8IviizQlbBqdxcluxl0t+YgXleG2HkE=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Bbqld8RB8H/PwjcqEP5M8fteEAeEmGAm7PJAdUqXR4iLb1Z6Ix3etMSJyXDp6NS0xG KqSyOTh+kKvuUgmvbuteqVT2kY0OQSOFwCjwjGSR2aeNi4Xd1foEZeYRFqVpsL4yV/r1 PyiUALtgB1EMAK/lsWsEM34QrYqzvdAgWaDBg=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <295059341.20110311205909@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <AANLkTikYnh0xbHLtyAR71FAOT18+g6SPxo25CGz5gKwh@xxxxxxxxxxxxxx> <20110303221639.GB12175@xxxxxxxxxxxx> <AANLkTi=r+ErO+PkPWF=3L8+v9+TPbgVz-6qaycTgMo4c@xxxxxxxxxxxxxx> <AANLkTimTuXqoLe9VinpAdhwJPO3Z8HGU2+KOoHOBaUvq@xxxxxxxxxxxxxx> <AANLkTimin0OZZUmvUr2KcXugc_2GGuEhRHLdug1ufha6@xxxxxxxxxxxxxx> <20110308192950.GA4562@xxxxxxxxxxxx> <20110308201002.GA5721@xxxxxxxxxxxx> <AANLkTikdA0vnxYzU7MFZNk3m6SH6=ns-WGVry2zCfws+@xxxxxxxxxxxxxx> <1299617407852-3414620.post@xxxxxxxxxxxxx> <4D76C48F.2050006@xxxxxxxxxxx> <20110311183800.GD32084@xxxxxxx> <295059341.20110311205909@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx


On Sat, Mar 12, 2011 at 3:59 AM, Sander Eikelenboom <linux@xxxxxxxxxxxxxx> wrote:
Haven't seen this one on my systen. I do use LVM, but i'm only using LVM as device mapper, without software raid etc.
Also using debian lenny/squeeze together with custom compiled xen and xen-2.6.32.x pvops kernel

So I would suspect the multipathing / raid part.

From one of my test server, below is grep output:

# grep -A 45 'cut here' /var/log/messages*|grep 'kernel: Process'
/var/log/messages-Mar 10 00:53:00 xen06 kernel: Process mpath_wait (pid: 9513, threadinfo ffff880010f38000, task ffff880010c4c100)
/var/log/messages-Mar 10 02:52:33 xen06 kernel: Process clvmd (pid: 3240, threadinfo ffff880035a14000, task ffff88003b49c340)
/var/log/messages-Mar 10 02:52:33 xen06 kernel: Process syslogd (pid: 24279, threadinfo ffff88003bb3e000, task ffff880016f3a1c0)
/var/log/messages-Mar 10 03:40:52 xen06 kernel: Process mpath_wait (pid: 16487, threadinfo ffff88003becc000, task ffff88003bdb8480)
/var/log/messages.2-Feb 23 22:56:56 xen06 kernel: Process mpath_wait (pid: 15222, threadinfo ffff8800265b8000, task ffff88003b940200)
/var/log/messages.2-Feb 24 01:03:17 xen06 kernel: Process dmsetup (pid: 509, threadinfo ffff88003ced4000, task ffff88003025c4c0)
/var/log/messages.2-Feb 26 23:20:22 xen06 kernel: Process mpath_wait (pid: 10154, threadinfo ffff880025604000, task ffff8800254804c0)
/var/log/messages.4-Feb  8 04:30:10 xen06 kernel: Process mpath_wait (pid: 11646, threadinfo ffff88001e722000, task ffff880025222440)
/var/log/messages.4-Feb  9 16:43:11 xen06 kernel: Process lvremove (pid: 24469, threadinfo ffff88003b930000, task ffff880025ab03c0)
/var/log/messages.4-Feb  9 22:02:09 xen06 kernel: Process grep (pid: 10327, threadinfo ffff880025056000, task ffff880024814240)
/var/log/messages.4-Feb  9 22:13:37 xen06 kernel: Process dmsetup (pid: 6287, threadinfo ffff88003cf64000, task ffff880025fe8740)
/var/log/messages.4-Feb  9 22:16:55 xen06 kernel: Process clvmd (pid: 3204, threadinfo ffff88003ccb8000, task ffff88003a436440)

Whereby output related to 'kernel: BUG: unable to handle kernel paging request at':

# grep -B 20 -A 50 'kernel: BUG: unable to handle kernel paging request at' /var/log/messages*
/var/log/messages.2-Feb 24 00:59:13 xen06 lvm[2379]: No longer monitoring snapshot XenGroup-testcrash5--snapshot
/var/log/messages.2-Feb 24 00:59:14 xen06 lvm[2379]: Monitoring snapshot XenGroup-testcrash1--snapshot
/var/log/messages.2-Feb 24 00:59:14 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:14 xen06 kernel: EXT3 FS on dm-0, internal journal
/var/log/messages.2-Feb 24 00:59:14 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.2-Feb 24 00:59:14 xen06 lvm[2379]: No longer monitoring snapshot XenGroup-testcrash1--snapshot
/var/log/messages.2-Feb 24 00:59:15 xen06 lvm[2379]: Monitoring snapshot XenGroup-testcrash2--snapshot
/var/log/messages.2-Feb 24 00:59:16 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:16 xen06 kernel: EXT3 FS on dm-0, internal journal
/var/log/messages.2-Feb 24 00:59:16 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.2-Feb 24 00:59:16 xen06 lvm[2379]: No longer monitoring snapshot XenGroup-testcrash2--snapshot
/var/log/messages.2-Feb 24 00:59:17 xen06 lvm[2379]: Monitoring snapshot XenGroup-testcrash3--snapshot
/var/log/messages.2-Feb 24 00:59:17 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:17 xen06 kernel: EXT3 FS on dm-0, internal journal
/var/log/messages.2-Feb 24 00:59:17 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.2-Feb 24 00:59:18 xen06 lvm[2379]: No longer monitoring snapshot XenGroup-testcrash3--snapshot
/var/log/messages.2-Feb 24 00:59:19 xen06 lvm[2379]: Monitoring snapshot XenGroup-testcrash4--snapshot
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: EXT3 FS on dm-0, internal journal
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.2:Feb 24 00:59:19 xen06 kernel: BUG: unable to handle kernel paging request at ffff88002561f3f8
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: IP: [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: PGD 1002067 PUD 1006067 PMD 1f1067 PTE 801000002561f065
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Oops: 0003 [#1] SMP
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: last sysfs file: /sys/block/dm-13/dev
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: CPU 1
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Modules linked in: dlm configfs xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg ide_cd_mod cdrom tg3 serio_raw libphy button tpm_tis tpm tpm_bios pcspkr shpchp i2c_i801 i2c_core iTCO_wdt dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Pid: 16520, comm: udevd Not tainted 2.6.32.29-0.xen.pvops.choon.centos5 #1 PowerEdge 860
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RIP: e030:[<ffffffff8100e2d4>]  [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RSP: e02b:ffff880025655b88  EFLAGS: 00010246
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RAX: 0000000000000000 RBX: ffff88002561f3f8 RCX: 00007f9a10c63000
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88002561f3f8
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RBP: ffff880025655b98 R08: 00007f9a10fc0000 R09: ffff88003e64d609
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: R10: ffff88003cce22c0 R11: ffffffff813391a0 R12: 0000000000000000
/var/log/messages.2-Feb 24 00:59:19 xen06 lvm[2379]: No longer monitoring snapshot XenGroup-testcrash4--snapshot
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: R13: 00007f9a10000000 R14: ffff88003cc6a7f8 R15: ffff88002550c340
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: FS:  00007f9a10a41710(0000) GS:ffff88002806c000(0000) knlGS:0000000000000000
/var/log/messages.2-Feb 24 00:59:19 xen06 udevd-event[16460]: run_program: '/sbin/dmsetup' abnormal exit
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: CR2: ffff88002561f3f8 CR3: 000000003bb46000 CR4: 0000000000002660
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Process udevd (pid: 16520, threadinfo ffff880025654000, task ffff88003cce22c0)
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Stack:
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  ffff88002561f3f8 0000000025690067 ffff880025655c48 ffffffff810a60b0
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: <0> ffff880025655ac8 000000000000001b 00007f9a10e00000 0000000000000000
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: <0> 00007f9a10c63000 ffff88002807a5c0 00007f9a10c63000 ffff880025655cb8
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Call Trace:
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810a60b0>] free_pgd_range+0x3b3/0x401
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810a63b1>] free_pgtables+0xb6/0xce
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810a84a4>] exit_mmap+0xf0/0x14b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff81048888>] mmput+0x46/0xb2
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810c8cd9>] flush_old_exec+0x463/0x536
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810fbf99>] load_elf_binary+0x365/0x171b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810faa30>] ? load_misc_binary+0x61/0x322
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810a50dd>] ? get_user_pages+0x44/0x46
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810c795e>] ? put_arg_page+0x9/0xb
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810c7e0b>] search_binary_handler+0xde/0x268
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810c9208>] do_execve+0x193/0x262
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff810113af>] sys_execve+0x3e/0x58
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  [<ffffffff81012f1a>] stub_execve+0x6a/0xc0
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Code: 8b 3c 25 70 c5 00 00 ff 14 25 68 a2 5c 81 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 79 e3 ff ff 84 c0 75 05 <4c> 89 23 eb 0b 4c 89 e6 48 89 df e8 69 ff ff ff 5b 41 5c c9 c3
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RIP  [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel:  RSP <ffff880025655b88>
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: CR2: ffff88002561f3f8
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: ---[ end trace 5ea31e622470b518 ]---
/var/log/messages.2-Feb 24 00:59:20 xen06 lvm[2379]: Monitoring snapshot XenGroup-testcrash5--snapshot
/var/log/messages.2-Feb 24 00:59:21 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:21 xen06 kernel: EXT3 FS on dm-0, internal journal
/var/log/messages.2-Feb 24 00:59:21 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.2-Feb 24 00:59:21 xen06 lvm[2379]: No longer monitoring snapshot XenGroup-testcrash5--snapshot
--
/var/log/messages.4-Feb  8 03:58:30 xen06 lvm[3961]: Monitoring snapshot XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb  8 03:58:30 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  8 03:58:30 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  8 03:58:30 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  8 03:58:30 xen06 lvm[3961]: No longer monitoring snapshot XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb  8 03:58:31 xen06 lvm[3961]: Monitoring snapshot XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb  8 03:58:31 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  8 03:58:31 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  8 03:58:31 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  8 03:58:32 xen06 lvm[3961]: No longer monitoring snapshot XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb  8 03:58:33 xen06 lvm[3961]: Monitoring snapshot XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb  8 03:58:33 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  8 03:58:33 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  8 03:58:33 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  8 03:58:33 xen06 lvm[3961]: No longer monitoring snapshot XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb  8 03:58:34 xen06 lvm[3961]: Monitoring snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  8 03:58:34 xen06 lvm[3961]: No longer monitoring snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4:Feb  8 03:58:34 xen06 kernel: BUG: unable to handle kernel paging request at ffff880030081010
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: IP: [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: PGD 1002067 PUD 1006067 PMD 246067 PTE 8010000030081065
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: Oops: 0003 [#1] SMP
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: last sysfs file: /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: CPU 3
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: Modules linked in: ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp gfs2 dlm configfs xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg ide_cd_mod cdrom serio_raw tg3 button libphy tpm_tis tpm tpm_bios i2c_i801 pcspkr i2c_core iTCO_wdt shpchp dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: Pid: 32051, comm: mpath_wait Not tainted 2.6.32.28-0.xen.pvops.choon.centos5 #1 PowerEdge 860
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: RIP: e030:[<ffffffff8100e2d4>]  [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: RSP: e02b:ffff880030c19b88  EFLAGS: 00010246
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: RAX: 0000000000000000 RBX: ffff880030081010 RCX: 00000000008c3000
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff880030081010
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: RBP: ffff880030c19b98 R08: 0000000002150000 R09: ffff88003e6f3e89
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: R10: ffff880030e78040 R11: ffffffff813381a0 R12: 0000000000000000
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: R13: 0000000000600000 R14: ffff880036dc5000 R15: ffff880030994000
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: FS:  00007f147d9606e0(0000) GS:ffff8800280a6000(0000) knlGS:0000000000000000
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: CR2: ffff880030081010 CR3: 000000003ba55000 CR4: 0000000000002660
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: Process mpath_wait (pid: 32051, threadinfo ffff880030c18000, task ffff88003c93c280)
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: Stack:
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  ffff880030081010 000000003b7e4067 ffff880030c19c48 ffffffff810a5e32
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: <0> ffff880030c19ac8 000000000000001f 0000000002000000 0000000000000000
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: <0> 00000000008c3000 ffff8800280b45c0 00000000008c3000 ffff880030c19cb8
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: Call Trace:
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810a5e32>] free_pgd_range+0x3b3/0x401
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810a6133>] free_pgtables+0xb6/0xce
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810a8224>] exit_mmap+0xf0/0x14b
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff81048648>] mmput+0x46/0xb2
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810c8a56>] flush_old_exec+0x460/0x533
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810fbd15>] load_elf_binary+0x365/0x171c
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810fa7ac>] ? load_misc_binary+0x61/0x322
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810a4e5f>] ? get_user_pages+0x44/0x46
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810c76de>] ? put_arg_page+0x9/0xb
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810c7b8b>] search_binary_handler+0xde/0x268
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810c8f85>] do_execve+0x193/0x262
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff810113af>] sys_execve+0x3e/0x58
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  [<ffffffff81012f1a>] stub_execve+0x6a/0xc0
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: Code: 8b 3c 25 70 c5 00 00 ff 14 25 68 82 5c 81 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 79 e3 ff ff 84 c0 75 05 <4c> 89 23 eb 0b 4c 89 e6 48 89 df e8 69 ff ff ff 5b 41 5c c9 c3
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: RIP  [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel:  RSP <ffff880030c19b88>
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: CR2: ffff880030081010
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: ---[ end trace f2d7fcc13c4cdf6a ]---
/var/log/messages.4:Feb  8 03:58:34 xen06 kernel: BUG: unable to handle kernel paging request at ffff8800313e1b00
/var/log/messages.4-Feb  8 03:58:34 xen06 kernel: IP: [<ffffffff810377f2>] ptep_set_access_flags+0x27/0x4a
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: PGD 1002067 PUD 1006067 PMD 24f067 PTE 80100000313e1065
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: Oops: 0003 [#2] SMP
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: last sysfs file: /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: CPU 1
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: Modules linked in: ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp gfs2 dlm configfs xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg ide_cd_mod cdrom serio_raw tg3 button libphy tpm_tis tpm tpm_bios i2c_i801 pcspkr i2c_core iTCO_wdt shpchp dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: Pid: 32053, comm: mpath_wait Tainted: G      D    2.6.32.28-0.xen.pvops.choon.centos5 #1 PowerEdge 860
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: RIP: e030:[<ffffffff810377f2>]  [<ffffffff810377f2>] ptep_set_access_flags+0x27/0x4a
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: RSP: e02b:ffff8800307c7cb8  EFLAGS: 00210202
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: RAX: ffff8800304b4870 RBX: 00007f147d960770 RCX: 80000001f6d6f167
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: RDX: ffff8800313e1b00 RSI: 00007f147d960770 RDI: ffff8800304b4870
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: RBP: ffff8800307c7cd8 R08: 0000000000000001 R09: ffffea0000ac5948
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: R10: 3138666666666666 R11: 000000000002496a R12: 0000000000000001
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: R13: ffff8800304b4870 R14: 0000000000000008 R15: 0000000000000000
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: FS:  00007f147d9606e0(0000) GS:ffff88002806c000(0000) knlGS:0000000000000000
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: CR2: ffff8800313e1b00 CR3: 0000000031261000 CR4: 0000000000002660
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: Process mpath_wait (pid: 32053, threadinfo ffff8800307c6000, task ffff880030ece200)
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: Stack:
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  80000001f6d6f145 0000000000000000 ffffea0000ac5948 ffffea0000800f30
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: <0> ffff8800307c7d88 ffffffff810a2a19 0000000000000001 ffff8800307c7de8
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: <0> ffffffff8100f0b1 ffff8800307c7d98 ffff8800305a8f60 ffff8800313e1b00
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: Call Trace:
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff810a2a19>] do_wp_page+0x2fd/0x717
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff8100f0b1>] ? xen_force_evtchn_callback+0xd/0xf
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff8100cde0>] ? xen_pte_val+0x64/0x6e
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff8100c3e9>] ? __raw_callee_save_xen_pte_val+0x15/0x23
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff810a4933>] handle_mm_fault+0x84a/0x8b9
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff8131be2d>] do_page_fault+0x252/0x2e2
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff8100f7f2>] ? check_events+0x12/0x20
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff81319db5>] page_fault+0x25/0x30
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff8116dc9d>] ? __put_user_4+0x1d/0x30
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff81047d2c>] ? schedule_tail+0x98/0x9d
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  [<ffffffff810129c3>] ret_from_fork+0x13/0x80
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: Code: 41 5d c9 c3 55 48 89 e5 41 55 49 89 fd 41 54 45 31 e4 53 48 89 f3 48 83 ec 08 48 39 0a 41 0f 95 c4 45 85 e4 74 1d 45 85 c0 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 48 89 de 4c 89 ef e8 37
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: RIP  [<ffffffff810377f2>] ptep_set_access_flags+0x27/0x4a
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel:  RSP <ffff8800307c7cb8>
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: CR2: ffff8800313e1b00
/var/log/messages.4-Feb  8 03:58:35 xen06 kernel: ---[ end trace f2d7fcc13c4cdf6b ]---
/var/log/messages.4-Feb  8 04:26:35 xen06 syslogd 1.4.1: restart.
/var/log/messages.4-Feb  8 04:26:35 xen06 kernel: klogd 1.4.1, log source = /proc/kmsg started.
/var/log/messages.4-Feb  8 04:26:35 xen06 kernel: Linux version 2.6.32.28-0.xen.pvops.choon.centos5 (mockbuild@xxxxxxxxxxxxxxxxxx) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Tue Feb 8 03:26:17 SGT 2011
/var/log/messages.4-Feb  8 04:26:35 xen06 kernel: Command line: ro root=/dev/md1 panic=5 panic_timeout=5
/var/log/messages.4-Feb  8 04:26:35 xen06 kernel: KERNEL supported cpus:
/var/log/messages.4-Feb  8 04:26:35 xen06 kernel:   Intel GenuineIntel
/var/log/messages.4-Feb  8 04:26:35 xen06 kernel:   AMD AuthenticAMD
/var/log/messages.4-Feb  8 04:26:35 xen06 kernel:   Centaur CentaurHauls
/var/log/messages.4-Feb  8 04:26:35 xen06 kernel: released 0 pages of unused memory
--
/var/log/messages.4-Feb  9 07:44:11 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb  9 07:44:11 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 07:44:11 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 07:44:11 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 07:44:11 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb  9 07:44:12 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash3--snapshot
/var/log/messages.4-Feb  9 07:44:12 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 07:44:12 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 07:44:12 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 07:44:12 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash3--snapshot
/var/log/messages.4-Feb  9 07:44:13 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb  9 07:44:13 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 07:44:13 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 07:44:13 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 07:44:13 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb  9 07:44:14 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb  9 07:44:14 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 07:44:15 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 07:44:15 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 07:44:15 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash5--snapshot
/var/log/messages.4:Feb  9 07:53:59 xen06 kernel: BUG: unable to handle kernel paging request at ffff88003cc0c010
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: IP: [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: PGD 1002067 PUD 1006067 PMD 2ac067 PTE 801000003cc0c065
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: Oops: 0003 [#1] SMP
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: last sysfs file: /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: CPU 0
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: Modules linked in: dlm configfs xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg ide_cd_mod cdrom serio_raw tg3 libphy button tpm_tis tpm tpm_bios pcspkr i2c_i801 shpchp i2c_core iTCO_wdt dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: Pid: 11005, comm: mpath_wait Not tainted 2.6.32.28-0.xen.pvops.choon.centos5 #1 PowerEdge 860
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: RIP: e030:[<ffffffff8100e2d4>]  [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: RSP: e02b:ffff88003b7d3b88  EFLAGS: 00010246
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: RAX: 0000000000000000 RBX: ffff88003cc0c010 RCX: 00000000008c3000
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88003cc0c010
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: RBP: ffff88003b7d3b98 R08: 0000000001244000 R09: ffff88003f009f09
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: R10: ffff8800269e3ac0 R11: ffffffff813381a0 R12: 0000000000000000
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: R13: 0000000000600000 R14: ffff8800269ca000 R15: ffff8800268bb000
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: FS:  00007fa7f8de76e0(0000) GS:ffff88002804f000(0000) knlGS:0000000000000000
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: CR2: ffff88003cc0c010 CR3: 00000000306b0000 CR4: 0000000000002660
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: Process mpath_wait (pid: 11005, threadinfo ffff88003b7d2000, task ffff8800307aa480)
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: Stack:
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  ffff88003cc0c010 000000003cfee067 ffff88003b7d3c48 ffffffff810a5e32
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: <0> 000000000000e02b 0000000000000060 0000000001200000 0000000000000000
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: <0> 00000000008c3000 ffff88002805d5c0 00000000008c3000 ffff88003b7d3cb8
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: Call Trace:
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810a5e32>] free_pgd_range+0x3b3/0x401
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810a6133>] free_pgtables+0xb6/0xce
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810a8224>] exit_mmap+0xf0/0x14b
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff81048648>] mmput+0x46/0xb2
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810c8a56>] flush_old_exec+0x460/0x533
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810fbd15>] load_elf_binary+0x365/0x171c
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810fa7ac>] ? load_misc_binary+0x61/0x322
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810a4e5f>] ? get_user_pages+0x44/0x46
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810c76de>] ? put_arg_page+0x9/0xb
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810c7b8b>] search_binary_handler+0xde/0x268
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810c8f85>] do_execve+0x193/0x262
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff810113af>] sys_execve+0x3e/0x58
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  [<ffffffff81012f1a>] stub_execve+0x6a/0xc0
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: Code: 8b 3c 25 70 c5 00 00 ff 14 25 68 82 5c 81 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 79 e3 ff ff 84 c0 75 05 <4c> 89 23 eb 0b 4c 89 e6 48 89 df e8 69 ff ff ff 5b 41 5c c9 c3
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: RIP  [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel:  RSP <ffff88003b7d3b88>
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: CR2: ffff88003cc0c010
/var/log/messages.4-Feb  9 07:53:59 xen06 kernel: ---[ end trace b5fffc59f6c8f974 ]---
/var/log/messages.4-Feb  9 16:31:21 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb  9 16:31:21 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 16:31:21 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 16:31:21 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 16:31:21 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb  9 16:31:22 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb  9 16:31:22 xen06 kernel: kjournald starting.  Commit interval 5 seconds
--
/var/log/messages.4-Feb  9 16:35:34 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb  9 16:35:34 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 16:35:34 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 16:35:34 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 16:35:35 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb  9 16:35:36 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb  9 16:35:36 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 16:35:36 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 16:35:36 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 16:35:36 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb  9 16:35:37 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash3--snapshot
/var/log/messages.4-Feb  9 16:35:37 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 16:35:37 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 16:35:37 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 16:35:37 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash3--snapshot
/var/log/messages.4-Feb  9 16:35:38 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb  9 16:35:38 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 16:35:39 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash4--snapshot
/var/log/messages.4:Feb  9 16:35:39 xen06 kernel: BUG: unable to handle kernel paging request at ffff880026b00418
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: IP: [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: PGD 1002067 PUD 1006067 PMD 1fb067 PTE 8010000026b00065
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: Oops: 0003 [#2] SMP
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: last sysfs file: /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: CPU 2
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: Modules linked in: dlm configfs xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg ide_cd_mod cdrom serio_raw tg3 libphy button tpm_tis tpm tpm_bios pcspkr i2c_i801 shpchp i2c_core iTCO_wdt dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: Pid: 25953, comm: udevd Tainted: G      D    2.6.32.28-0.xen.pvops.choon.centos5 #1 PowerEdge 860
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: RIP: e030:[<ffffffff8100e2d4>]  [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: RSP: e02b:ffff88002682fb88  EFLAGS: 00010246
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: RAX: 0000000000000000 RBX: ffff880026b00418 RCX: 00007f71114ac000
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff880026b00418
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: RBP: ffff88002682fb98 R08: 00007f7112d14000 R09: ffff88003e6e5a89
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: R10: ffff88003d2d0580 R11: ffffffff813381a0 R12: 0000000000000000
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: R13: 00007f7110800000 R14: ffff880026b047f0 R15: ffff880025943e20
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: FS:  00007f711128a710(0000) GS:ffff880028089000(0000) knlGS:0000000000000000
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: CR2: ffff880026b00418 CR3: 0000000026946000 CR4: 0000000000002660
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: Process udevd (pid: 25953, threadinfo ffff88002682e000, task ffff8800269ae380)
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: Stack:
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  ffff880026b00418 000000003d32d067 ffff88002682fc48 ffffffff810a5e32
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: <0> ffff88002682fac8 000000000000001c 00007f7112c00000 0000000000000000
/var/log/messages.4-Feb  9 16:35:39 xen06 udevd-event[25941]: run_program: '/sbin/dmsetup' abnormal exit
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: <0> 00007f71114ac000 ffff8800280975c0 00007f71114ac000 ffff88002682fcb8
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: Call Trace:
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810a5e32>] free_pgd_range+0x3b3/0x401
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810a6133>] free_pgtables+0xb6/0xce
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810a8224>] exit_mmap+0xf0/0x14b
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff81048648>] mmput+0x46/0xb2
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810c8a56>] flush_old_exec+0x460/0x533
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810fbd15>] load_elf_binary+0x365/0x171c
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810fa7ac>] ? load_misc_binary+0x61/0x322
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810a4e5f>] ? get_user_pages+0x44/0x46
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810c76de>] ? put_arg_page+0x9/0xb
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810c7b8b>] search_binary_handler+0xde/0x268
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810c8f85>] do_execve+0x193/0x262
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff810113af>] sys_execve+0x3e/0x58
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  [<ffffffff81012f1a>] stub_execve+0x6a/0xc0
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: Code: 8b 3c 25 70 c5 00 00 ff 14 25 68 82 5c 81 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 79 e3 ff ff 84 c0 75 05 <4c> 89 23 eb 0b 4c 89 e6 48 89 df e8 69 ff ff ff 5b 41 5c c9 c3
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: RIP  [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel:  RSP <ffff88002682fb88>
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: CR2: ffff880026b00418
/var/log/messages.4-Feb  9 16:35:39 xen06 kernel: ---[ end trace b5fffc59f6c8f975 ]---
/var/log/messages.4-Feb  9 16:35:40 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb  9 16:35:40 xen06 kernel: kjournald starting.  Commit interval 5 seconds
/var/log/messages.4-Feb  9 16:35:40 xen06 kernel: EXT3 FS on dm-11, internal journal
/var/log/messages.4-Feb  9 16:35:40 xen06 kernel: EXT3-fs: mounted filesystem with ordered data mode.
/var/log/messages.4-Feb  9 16:35:40 xen06 lvm[3802]: No longer monitoring snapshot XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb  9 16:35:41 xen06 lvm[3802]: Monitoring snapshot XenGroup-testcrash1--snapshot

Thanks.

Kindest regards,
Giam Teck Choon
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
Previous by Date:  Re: [Xen-devel] kernel BUG at arch/x86/xen/mmu.c:1860!, Sander Eikelenboom
Next by Date:  Re: [Xen-devel] kernel BUG at arch/x86/xen/mmu.c:1860!, Teck Choon Giam
Previous by Thread:  Re: [Xen-devel] kernel BUG at arch/x86/xen/mmu.c:1860!, Sander Eikelenboom
Next by Thread:  Re: [Xen-devel] kernel BUG at arch/x86/xen/mmu.c:1860!, Teck Choon Giam
Indexes:  [Date] [Thread] [Top] [All Lists]
 
   

Copyright , Citrix Systems Inc. All rights reserved. Legal and Privacy