Re: [Xen-devel] kernel BUG at arch/x86/xen/mmu.c:1860!

[Teck Choon Giam <giamteckchoon@xxxxxxxxx>]

On Wed, Dec 29, 2010 at 2:01 AM, Teck Choon Giam <giamteckchoon@xxxxxxxxx> wrote:

It would be very good to track this down and get it fixed..
hopefully you're able to help a bit and try some things to debug it.

Konrad maybe has some ideas to try..

I would love to track this down and have it fixed or else this is a stopper for majority of us to deploy pvops stable kernel 2.6.32.x in production.  Well at least for me... ...

More related logs and it seems all is related to LVM since all will have the last sysfs file to /dev/block/dm-??/dev... ?

While waiting for others to offer any assistance... I created a script to basically test lvm create snapshot, mount, umount and remove snapshot... ... the result is quite sad as I am able to reproduce the crash with loop count 100 easily on two of my servers with different specs... ...

If anyone is willing to run the script to test whether can reproduce the crash on their deveopment xen servers or any LVM host servers would be great.  The below script assume that those snapshot LVM can be mounted as personally I am running all domUs.  If your LV Group Name is different, just ammend the LVGroupName=VolGroup or anything to suit your environment.

----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------
#!/bin/sh
#
# This script is to create lvm snapshot, mount it, umount it and remove in a
# specified number of loops to test whether it will crash the host server.
#
# Created by Giam Teck Choon
#

# The LV name and for this case we are using XenGroup
LVGroupName=XenGroup

# return 1 if is mounted otherwise return 0
check_mount() {
    local checkdir=${1}
    if [ -n "$checkdir" ] ; then
        local check=`grep "$checkdir" /proc/mounts`
        if [ -n "$check" ] ; then
            return 1
        fi
    fi
    return 0
}

do_lvm_create_remove() {
    # number of loops default is 1
    local loopcountlimit=${1:-1}
    # snapshot size default is 1G
    local snapshotsize=${2:-1G}
    # implement a sleep between create, mount, umount and remove (default is 0 which is no pause)
    local pauseinterval=${3:-0}
    # We filter out control, snapshot and swap
    local count=0
    if [ -d "/dev/${LVGroupName}" ] ; then
        while [ "$count" -lt "$loopcountlimit" ]
        do
            count=`expr $count + 1`
            echo "${count} ... ... "
            for i in `ls /dev/${LVGroupName} | grep -Ev 'snapshot$' | grep -Ev 'swap$'`; do
                if [ -h "/dev/${LVGroupName}/${i}" ] ; then
                    echo -n "lvcreate -s -v -n ${i}-snapshot -L ${snapshotsize} /dev/${LVGroupName}/${i} ... ... "
                    lvcreate -s -v -n ${i}-snapshot -L ${snapshotsize} /dev/${LVGroupName}/${i}
                    echo "done."
                    sleep ${pauseinterval}
                    mkdir -p /mnt/testlvm/${i}
                    if [ -h "/dev/${LVGroupName}/${i}-snapshot" ] ; then
                        check_mount /mnt/testlvm/${i}
                        local ismount=$?
                        if [ "$ismount" -eq 0 ] ; then
                            echo -n "mount /dev/${LVGroupName}/${i}-snapshot /mnt/testlvm/${i} ... ... "
                            mount /dev/${LVGroupName}/${i}-snapshot /mnt/testlvm/${i}
                            echo "done."
                            sleep ${pauseinterval}
                        fi
                        check_mount /mnt/testlvm/${i}
                        local ismount2=$?
                        if [ "$ismount2" -eq 1 ] ; then
                            echo -n "umount /mnt/testlvm/${i} ... ... "
                            umount /mnt/testlvm/${i}
                            echo "done."
                            sleep ${pauseinterval}
                        fi
                    fi
                    rm -rf /mnt/testlvm/${i}
                    echo -n "lvremove -f /dev/${LVGroupName}/${i}-snapshot ... ... "
                    lvremove -f /dev/${LVGroupName}/${i}-snapshot
                    echo "done."
                    sleep ${pauseinterval}
                fi
            done
            rm -fr /mnt/testlvm
        done
    else
        echo "/dev/${LVGroupName} directory not found!"
        exit 1
    fi
}

case $1 in
    loop)    shift
        do_lvm_create_remove "$@"
        ;;
    *)    echo "Usage: $0 loop loopcountlimit(default is 1) snapshotsize(default is 1GB) pauseinterval (default is 0)"
        ;;
esac
----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------

I name the above script as test.sh then run:

Below is without pause
# sh test.sh loop 100

Then I will get something printed to output as below:

Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: ------------[ cut here ]------------

Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: invalid opcode: 0000 [#1] SMP 

Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: last sysfs file: /sys/block/dm-16/dev

Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: Stack:

Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: Call Trace:

Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b 48 
  Unable to read from event server.

Then I can run the below command to get the log after it is back online:

# grep -A 50 'cut here' /var/log/messages
Dec 29 11:47:32 xen03 kernel: ------------[ cut here ]------------
Dec 29 11:47:32 xen03 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
Dec 29 11:47:32 xen03 kernel: invalid opcode: 0000 [#2] SMP 
Dec 29 11:47:32 xen03 kernel: last sysfs file: /sys/block/dm-16/dev
Dec 29 11:47:32 xen03 kernel: CPU 3 
Dec 29 11:47:32 xen03 kernel: Modules linked in: ext4 jbd2 crc16 xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi loop dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg tg3 libphy ide_cd_mod cdrom button serio_raw tpm_tis tpm tpm_bios pcspkr iTCO_wdt i2c_i801 shpchp i2c_core dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Dec 29 11:47:32 xen03 kernel: Pid: 30576, comm: mpath_wait Tainted: G      D    2.6.32.27-0.xen.pvops.choon.centos5 #1 PowerEdge 860
Dec 29 11:47:32 xen03 kernel: RIP: e030:[<ffffffff8100cb5b>]  [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 29 11:47:32 xen03 kernel: RSP: e02b:ffff880030a7fdc8  EFLAGS: 00010282
Dec 29 11:47:32 xen03 kernel: RAX: 00000000ffffffea RBX: 000000000002445a RCX: 0000000000000122
Dec 29 11:47:32 xen03 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
Dec 29 11:47:32 xen03 kernel: RBP: ffff880030a7fde8 R08: 00000000000002d0 R09: ffff880000000000
Dec 29 11:47:32 xen03 kernel: R10: 00000000deadbeef R11: 0000000000000246 R12: 0000000000000003
Dec 29 11:47:32 xen03 kernel: R13: 000000000002445a R14: ffff88002456e480 R15: 0000003db6c9a4d5
Dec 29 11:47:32 xen03 kernel: FS:  00007f45a01586e0(0000) GS:ffff8800280a6000(0000) knlGS:0000000000000000
Dec 29 11:47:32 xen03 kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Dec 29 11:47:32 xen03 kernel: CR2: 0000003db6c9a4d5 CR3: 000000001cc2d000 CR4: 0000000000002660
Dec 29 11:47:32 xen03 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Dec 29 11:47:32 xen03 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Dec 29 11:47:32 xen03 kernel: Process mpath_wait (pid: 30576, threadinfo ffff880030a7e000, task ffff88002456e480)
Dec 29 11:47:32 xen03 kernel: Stack:
Dec 29 11:47:32 xen03 kernel:  0000000000000000 00000000001f667f 000000013e6f3e18 000000000002445a
Dec 29 11:47:32 xen03 kernel: <0> ffff880030a7fe08 ffffffff8100e07c ffff88003d2fc040 ffff8800246c9db0
Dec 29 11:47:32 xen03 kernel: <0> ffff880030a7fe18 ffffffff8100e0af ffff880030a7fe58 ffffffff810a402f
Dec 29 11:47:32 xen03 kernel: Call Trace:
Dec 29 11:47:32 xen03 kernel:  [<ffffffff8100e07c>] xen_alloc_ptpage+0x64/0x69
Dec 29 11:47:32 xen03 kernel:  [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Dec 29 11:47:32 xen03 kernel:  [<ffffffff810a402f>] __pte_alloc+0x70/0xce
Dec 29 11:47:32 xen03 kernel:  [<ffffffff810a41cd>] handle_mm_fault+0x140/0x8b9
Dec 29 11:47:32 xen03 kernel:  [<ffffffff81319dd5>] ? page_fault+0x25/0x30
Dec 29 11:47:32 xen03 kernel:  [<ffffffff8131be4d>] do_page_fault+0x252/0x2e2
Dec 29 11:47:32 xen03 kernel:  [<ffffffff8116dd7d>] ? __put_user_4+0x1d/0x30
Dec 29 11:47:32 xen03 kernel:  [<ffffffff81319dd5>] page_fault+0x25/0x30
Dec 29 11:47:32 xen03 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b 48 
Dec 29 11:47:32 xen03 kernel: RIP  [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 29 11:47:32 xen03 kernel:  RSP <ffff880030a7fdc8>
Dec 29 11:47:32 xen03 kernel: ---[ end trace 900e639a50e97057 ]---
Dec 29 11:50:38 xen03 syslogd 1.4.1: restart.
Dec 29 11:50:38 xen03 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Dec 29 11:50:38 xen03 kernel: Linux version 2.6.32.27-0.xen.pvops.choon.centos5 (mockbuild@xxxxxxxxxxxxxxxxxx) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Sat Dec 25 09:19:44 SGT 2010
Dec 29 11:50:38 xen03 kernel: Command line: ro root=/dev/md1 panic=5 panic_timeout=5
Dec 29 11:50:38 xen03 kernel: KERNEL supported cpus:
Dec 29 11:50:38 xen03 kernel:   Intel GenuineIntel
Dec 29 11:50:38 xen03 kernel:   AMD AuthenticAMD
Dec 29 11:50:38 xen03 kernel:   Centaur CentaurHauls
Dec 29 11:50:38 xen03 kernel: released 0 pages of unused memory
Dec 29 11:50:38 xen03 kernel: BIOS-provided physical RAM map:
Dec 29 11:50:38 xen03 kernel:  Xen: 0000000000000000 - 00000000000a0000 (usable)
Dec 29 11:50:38 xen03 kernel:  Xen: 00000000000a0000 - 0000000000100000 (reserved)
Dec 29 11:50:38 xen03 kernel:  Xen: 0000000000100000 - 0000000040000000 (usable)
Dec 29 11:50:38 xen03 kernel:  Xen: 00000000dffc0000 - 00000000dffcfc00 (ACPI data)
--
Dec 29 11:52:30 xen03 kernel: ------------[ cut here ]------------
Dec 29 11:52:30 xen03 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
Dec 29 11:52:30 xen03 kernel: invalid opcode: 0000 [#1] SMP 
Dec 29 11:52:30 xen03 kernel: last sysfs file: /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
Dec 29 11:52:30 xen03 kernel: CPU 3 
Dec 29 11:52:30 xen03 kernel: Modules linked in: xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi loop dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg tg3 libphy ide_cd_mod cdrom serio_raw button tpm_tis tpm tpm_bios i2c_i801 i2c_core iTCO_wdt pcspkr shpchp dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Dec 29 11:52:30 xen03 kernel: Pid: 8000, comm: udevd Not tainted 2.6.32.27-0.xen.pvops.choon.centos5 #1 PowerEdge 860
Dec 29 11:52:30 xen03 kernel: RIP: e030:[<ffffffff8100cb5b>]  [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 29 11:52:30 xen03 kernel: RSP: e02b:ffff88001f0c9dc8  EFLAGS: 00010282
Dec 29 11:52:30 xen03 kernel: RAX: 00000000ffffffea RBX: 0000000000037fc2 RCX: 00000000000001bf
Dec 29 11:52:30 xen03 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
Dec 29 11:52:30 xen03 kernel: RBP: ffff88001f0c9de8 R08: 0000000000000e10 R09: ffff880000000000
Dec 29 11:52:30 xen03 kernel: R10: 00000000deadbeef R11: 00007fb810051ce0 R12: 0000000000000003
Dec 29 11:52:30 xen03 kernel: R13: 0000000000037fc2 R14: ffff88003257e100 R15: 00007fb80fc14158
Dec 29 11:52:30 xen03 kernel: FS:  00007fb81024e710(0000) GS:ffff8800280a6000(0000) knlGS:0000000000000000
Dec 29 11:52:30 xen03 kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Dec 29 11:52:30 xen03 kernel: CR2: 00007fb80fc14158 CR3: 0000000032ee7000 CR4: 0000000000002660
Dec 29 11:52:30 xen03 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Dec 29 11:52:30 xen03 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Dec 29 11:52:30 xen03 kernel: Process udevd (pid: 8000, threadinfo ffff88001f0c8000, task ffff88003257e100)
Dec 29 11:52:30 xen03 kernel: Stack:
Dec 29 11:52:30 xen03 kernel:  0000000000000000 000000000020a314 000000013e6f3e18 0000000000037fc2
Dec 29 11:52:30 xen03 kernel: <0> ffff88001f0c9e08 ffffffff8100e07c ffff880032c2a040 ffff880032aaa3f0
Dec 29 11:52:30 xen03 kernel: <0> ffff88001f0c9e18 ffffffff8100e0af ffff88001f0c9e58 ffffffff810a402f
Dec 29 11:52:30 xen03 kernel: Call Trace:
Dec 29 11:52:30 xen03 kernel:  [<ffffffff8100e07c>] xen_alloc_ptpage+0x64/0x69
Dec 29 11:52:30 xen03 kernel:  [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Dec 29 11:52:30 xen03 kernel:  [<ffffffff810a402f>] __pte_alloc+0x70/0xce
Dec 29 11:52:30 xen03 kernel:  [<ffffffff810a41cd>] handle_mm_fault+0x140/0x8b9
Dec 29 11:52:30 xen03 kernel:  [<ffffffff810d2ecc>] ? d_kill+0x3a/0x42
Dec 29 11:52:30 xen03 kernel:  [<ffffffff810c4cd1>] ? __fput+0x1cb/0x1da
Dec 29 11:52:30 xen03 kernel:  [<ffffffff8131be4d>] do_page_fault+0x252/0x2e2
Dec 29 11:52:30 xen03 kernel:  [<ffffffff81319dd5>] page_fault+0x25/0x30
Dec 29 11:52:30 xen03 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b 48 
Dec 29 11:52:30 xen03 kernel: RIP  [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 29 11:52:30 xen03 kernel:  RSP <ffff88001f0c9dc8>
Dec 29 11:52:30 xen03 kernel: ---[ end trace 2b8ce24b81556aa1 ]---
Dec 29 11:52:40 xen03 kernel: dmeventd[4328]: segfault at 90 ip 0000003db740bcd0 sp 00000000408080c8 error 4 in libpthread-2.5.so[3db7400000+16000]
Dec 29 11:55:35 xen03 syslogd 1.4.1: restart.
Dec 29 11:55:35 xen03 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Dec 29 11:55:35 xen03 kernel: Linux version 2.6.32.27-0.xen.pvops.choon.centos5 (mockbuild@xxxxxxxxxxxxxxxxxx) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Sat Dec 25 09:19:44 SGT 2010
Dec 29 11:55:35 xen03 kernel: Command line: ro root=/dev/md1 panic=5 panic_timeout=5
Dec 29 11:55:35 xen03 kernel: KERNEL supported cpus:
Dec 29 11:55:35 xen03 kernel:   Intel GenuineIntel
Dec 29 11:55:35 xen03 kernel:   AMD AuthenticAMD
Dec 29 11:55:35 xen03 kernel:   Centaur CentaurHauls
Dec 29 11:55:35 xen03 kernel: released 0 pages of unused memory
Dec 29 11:55:35 xen03 kernel: BIOS-provided physical RAM map:
Dec 29 11:55:35 xen03 kernel:  Xen: 0000000000000000 - 00000000000a0000 (usable)
Dec 29 11:55:35 xen03 kernel:  Xen: 00000000000a0000 - 0000000000100000 (reserved)
Dec 29 11:55:35 xen03 kernel:  Xen: 0000000000100000 - 0000000040000000 (usable)
--
Dec 29 12:13:19 xen03 kernel: ------------[ cut here ]------------
Dec 29 12:13:19 xen03 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
Dec 29 12:13:19 xen03 kernel: invalid opcode: 0000 [#1] SMP 
Dec 29 12:13:19 xen03 kernel: last sysfs file: /sys/block/dm-16/dev
Dec 29 12:13:19 xen03 kernel: CPU 1 
Dec 29 12:13:19 xen03 kernel: Modules linked in: ext4 jbd2 crc16 xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi loop dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg ide_cd_mod cdrom serio_raw tg3 button libphy tpm_tis tpm tpm_bios iTCO_wdt i2c_i801 pcspkr i2c_core shpchp dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Dec 29 12:13:19 xen03 kernel: Pid: 15285, comm: udevd Not tainted 2.6.32.27-0.xen.pvops.choon.centos5 #1 PowerEdge 860
Dec 29 12:13:19 xen03 kernel: RIP: e030:[<ffffffff8100cb5b>]  [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 29 12:13:19 xen03 kernel: RSP: e02b:ffff880025db5dc8  EFLAGS: 00010282
Dec 29 12:13:19 xen03 kernel: RAX: 00000000ffffffea RBX: 0000000000035093 RCX: 00000000000001a8
Dec 29 12:13:19 xen03 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
Dec 29 12:13:19 xen03 kernel: RBP: ffff880025db5de8 R08: 0000000000000498 R09: ffff880000000000
Dec 29 12:13:19 xen03 kernel: R10: 00000000deadbeef R11: 00007f7824d70600 R12: 0000000000000003
Dec 29 12:13:19 xen03 kernel: R13: 0000000000035093 R14: ffff880039ff0540 R15: 00007f7824936268
Dec 29 12:13:19 xen03 kernel: FS:  00007f78256fe710(0000) GS:ffff88002806c000(0000) knlGS:0000000000000000
Dec 29 12:13:19 xen03 kernel: CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Dec 29 12:13:19 xen03 kernel: CR2: 00007f7824936268 CR3: 0000000039fef000 CR4: 0000000000002660
Dec 29 12:13:19 xen03 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Dec 29 12:13:19 xen03 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Dec 29 12:13:19 xen03 kernel: Process udevd (pid: 15285, threadinfo ffff880025db4000, task ffff880039ff0540)
Dec 29 12:13:19 xen03 kernel: Stack:
Dec 29 12:13:19 xen03 kernel:  0000000000000000 0000000000207243 000000013e64d518 0000000000035093
Dec 29 12:13:19 xen03 kernel: <0> ffff880025db5e08 ffffffff8100e07c ffff88003a50fac0 ffff8800352a6920
Dec 29 12:13:19 xen03 kernel: <0> ffff880025db5e18 ffffffff8100e0af ffff880025db5e58 ffffffff810a402f
Dec 29 12:13:19 xen03 kernel: Call Trace:
Dec 29 12:13:19 xen03 kernel:  [<ffffffff8100e07c>] xen_alloc_ptpage+0x64/0x69
Dec 29 12:13:19 xen03 kernel:  [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Dec 29 12:13:19 xen03 kernel:  [<ffffffff810a402f>] __pte_alloc+0x70/0xce
Dec 29 12:13:19 xen03 kernel:  [<ffffffff810a41cd>] handle_mm_fault+0x140/0x8b9
Dec 29 12:13:19 xen03 kernel:  [<ffffffff813199ff>] ? _spin_unlock_irqrestore+0x11/0x13
Dec 29 12:13:19 xen03 kernel:  [<ffffffff8131be4d>] do_page_fault+0x252/0x2e2
Dec 29 12:13:19 xen03 kernel:  [<ffffffff81319dd5>] page_fault+0x25/0x30
Dec 29 12:13:19 xen03 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b 48 
Dec 29 12:13:19 xen03 kernel: RIP  [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 29 12:13:19 xen03 kernel:  RSP <ffff880025db5dc8>
Dec 29 12:13:19 xen03 kernel: ---[ end trace aa0f33d0cdc0d845 ]---
Dec 29 12:16:25 xen03 syslogd 1.4.1: restart.
Dec 29 12:16:25 xen03 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Dec 29 12:16:25 xen03 kernel: Linux version 2.6.32.27-0.xen.pvops.choon.centos5 (mockbuild@xxxxxxxxxxxxxxxxxx) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Sat Dec 25 09:19:44 SGT 2010
Dec 29 12:16:25 xen03 kernel: Command line: ro root=/dev/md1 panic=5 panic_timeout=5
Dec 29 12:16:25 xen03 kernel: KERNEL supported cpus:
Dec 29 12:16:25 xen03 kernel:   Intel GenuineIntel
Dec 29 12:16:25 xen03 kernel:   AMD AuthenticAMD
Dec 29 12:16:25 xen03 kernel:   Centaur CentaurHauls
Dec 29 12:16:25 xen03 kernel: released 0 pages of unused memory
Dec 29 12:16:25 xen03 kernel: BIOS-provided physical RAM map:
Dec 29 12:16:25 xen03 kernel:  Xen: 0000000000000000 - 00000000000a0000 (usable)
Dec 29 12:16:25 xen03 kernel:  Xen: 00000000000a0000 - 0000000000100000 (reserved)
Dec 29 12:16:25 xen03 kernel:  Xen: 0000000000100000 - 0000000040000000 (usable)
Dec 29 12:16:25 xen03 kernel:  Xen: 00000000dffc0000 - 00000000dffcfc00 (ACPI data)
Dec 29 12:16:25 xen03 kernel:  Xen: 00000000dffcfc00 - 00000000dffff000 (reserved)

My next crash test for this will be with a pause... to see whether I can reproduce the crash or not... then after that I will implement a sync before each crash to test crash... ...

Thanks.

Kindest regards,
Giam Teck Choon

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel