WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: Unable to handle kernel paging request

Another data point...  Dom0 seems to only want to crash when more than
3 or 4 domU's are running (each with their own DRBD root, with DRBD
running in dom0), and the below 'nc' command is run in the last
domU...

Still looking.

Steve

On Fri, Aug 11, 2006 at 09:34:12PM -0700, Steve Traugott wrote:
> Okay, for some reason I'm now able to crash dom0 pretty reliably in
> this configuration (domU root filesystem in DRBD) simply by running
> this in domU:
> 
>     nc -u -b 255.255.255.255 54321 < /dev/zero
> 
> Anyone else want to give that a try on 10868 or later, and see if they
> can duplicate it, with or without DRBD?
> 
> Steve
> 
> On Fri, Aug 11, 2006 at 06:50:50PM -0700, Steve Traugott wrote:
> > Uh oh....
> > 
> > On Sat, Aug 05, 2006 at 08:38:03AM +0100, Ian Pratt wrote:
> > > > So I built -unstable changeset 10868, and ran an even heavier workload
> > > > (the above, plus 'bonnie' in the guests) on dom0 and two guests
> > > > overnight, and they experienced no soft lockups; running -unstable,
> > > > changeset 10868, credit scheduler.  This same workload would have
> > > > caused soft lockups within seconds in -testing changeset 9732 using
> > > > the sedf scheduler; I may not have been able to get it started at all.
> > > > Response time remained subsecond under -unstable; -testing would have
> > > > been on its knees.
> > > 
> > > That's good to hear. 3.0.3 is going to be a big leap forward in many
> > > ways.
> > 
> > Same environment as we've discussed earlier in this thread, now on
> > unstable changeset 10868, Xen on top of DRBD, this one seems to be a
> > reliable crash when under combined disk and network load, still
> > isolating but would welcome suggestions...
> > 
> > Steve
> > 
> > 
> > root (hd0,0)
> >  Filesystem type is ext2fs, partition type 0x83
> > kernel /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1
> >    [Multiboot-elf, <0x100000:0x7a008:0x55ff8>, shtab=0x1d0078, 
> > entry=0x100000]
> > module /boot/vmlinuz-2.6.16.13-xen root=801 ro console=ttyS0 
> > ramdisk_size=32768
> >    [Multiboot-module @ 0x1d1000, 0x494ed0 bytes]
> > module /boot/initrd.img-2.6.16.13-xen
> >    [Multiboot-module @ 0x666000, 0x9a6400 bytes]
> >  __  __            _____  ___                     _        _     _
> >  \ \/ /___ _ __   |___ / / _ \    _   _ _ __  ___| |_ __ _| |__ | | ___
> >   \  // _ \ '_ \    |_ \| | | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> >   /  \  __/ | | |  ___) | |_| |__| |_| | | | \__ \ || (_| | |_) | |  __/
> >  /_/\_\___|_| |_| |____(_)___/    \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> > 
> >  http://www.cl.cam.ac.uk/netos/xen
> >  University of Cambridge Computer Laboratory
> > 
> >  Xen version 3.0-unstable (root@xxxxxxxxxxxxxxxxx) (gcc version 3.3.5 
> > (Debian 1:3.3.5-13)) Fri Aug  4 10:54:50 PDT 2006
> >  Latest ChangeSet: Sat Jul 29 06:05:59 2006 +0100 10868:d2bf1a7cc131
> > 
> > (XEN) Command line: /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1
> > (XEN) Physical RAM map:
> > (XEN)  0000000000000000 - 000000000009c000 (usable)
> > (XEN)  000000000009c000 - 00000000000a0000 (reserved)
> > (XEN)  00000000000e0000 - 0000000000100000 (reserved)
> > (XEN)  0000000000100000 - 000000003ffec340 (usable)
> > (XEN)  000000003ffec340 - 000000003fff0000 (ACPI data)
> > (XEN)  000000003fff0000 - 0000000040000000 (reserved)
> > (XEN)  00000000fec00000 - 0000000100000000 (reserved)
> > (XEN) System RAM: 1023MB (1048096kB)
> > (XEN) Xen heap: 10MB (10396kB)
> > (XEN) Using scheduler: SMP Credit Scheduler (credit)
> > (XEN) PAE disabled.
> > (XEN) found SMP MP-table at 0009c1d0
> > (XEN) DMI 2.3 present.
> > (XEN) Using APIC driver default
> > (XEN) ACPI: RSDP (v000 IBM                                   ) @ 0x000fdfd0
> > (XEN) ACPI: RSDT (v001 IBM    SEREMRLD 0x00001000 IBM  0x45444f43) @ 
> > 0x3ffeff80
> > (XEN) ACPI: FADT (v001 IBM    SEREMRLD 0x00001000 IBM  0x45444f43) @ 
> > 0x3ffeff00
> > (XEN) ACPI: MADT (v001 IBM    SEREMRLD 0x00001000 IBM  0x45444f43) @ 
> > 0x3ffefe80
> > (XEN) ACPI: DSDT (v001 IBM    SEREMRLD 0x00001000 MSFT 0x0100000b) @ 
> > 0x00000000
> > (XEN) ACPI: Local APIC address 0xfee00000
> > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
> > (XEN) Processor #3 6:11 APIC version 17
> > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> > (XEN) Processor #0 6:11 APIC version 17
> > (XEN) ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
> > (XEN) IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
> > (XEN) ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
> > (XEN) IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
> > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl)
> > (XEN) ACPI: IRQ3 used by override.
> > (XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
> > (XEN) Using ACPI (MADT) for SMP configuration information
> > (XEN) Initializing CPU#0
> > (XEN) Detected 1130.180 MHz processor.
> > (XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
> > (XEN) CPU: L2 cache: 512K
> > (XEN) Intel machine check architecture supported.
> > (XEN) Intel machine check reporting enabled on CPU#0.
> > (XEN) CPU0: Intel(R) Pentium(R) III CPU family      1133MHz stepping 01
> > (XEN) Booting processor 1/0 eip 90000
> > (XEN) Initializing CPU#1
> > (XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
> > (XEN) CPU: L2 cache: 512K
> > (XEN) Intel machine check architecture supported.
> > (XEN) Intel machine check reporting enabled on CPU#1.
> > (XEN) CPU1: Intel(R) Pentium(R) III CPU family      1133MHz stepping 01
> > (XEN) Total of 2 processors activated.
> > (XEN) ENABLING IO-APIC IRQs
> > (XEN)  -> Using new ACK method
> > (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=0 apic2=-1 pin2=-1
> > (XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
> > (XEN) ...trying to set up timer (IRQ0) through the 8259A ...  failed.
> > (XEN) ...trying to set up timer as Virtual Wire IRQ... works.
> > (XEN) checking TSC synchronization across 2 CPUs: passed.
> > (XEN) Platform timer is 1.193MHz PIT
> > (XEN) Brought up 2 CPUs
> > (XEN) Machine check exception polling timer started.
> > (XEN) *** LOADING DOMAIN 0 ***
> > (XEN) Domain 0 kernel supports features = { 0000001f }.
> > (XEN) Domain 0 kernel requires features = { 00000000 }.
> > (XEN) PHYSICAL MEMORY ARRANGEMENT:
> > (XEN)  Dom0 alloc.:   3c000000->3e000000 (8192 pages to be allocated)
> > (XEN) VIRTUAL MEMORY ARRANGEMENT:
> > (XEN)  Loaded kernel: c0100000->c05d5d74
> > (XEN)  Init. ramdisk: c05d6000->c0f7c400
> > (XEN)  Phys-Mach map: c0f7d000->c0f8d000
> > (XEN)  Start info:    c0f8d000->c0f8e000
> > (XEN)  Page tables:   c0f8e000->c0f94000
> > (XEN)  Boot stack:    c0f94000->c0f95000
> > (XEN)  TOTAL:         c0000000->c1400000
> > (XEN)  ENTRY ADDRESS: c0100000
> > (XEN) Dom0 has maximum 2 VCPUs
> > (XEN) Initrd len 0x9a6400, start at 0xc05d6000
> > (XEN) Scrubbing Free RAM: ...........done.
> > (XEN) Xen trace buffers: disabled
> > (XEN) Xen is relinquishing VGA console.
> > (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input 
> > to Xen).
> > Linux version 2.6.16.13-xen (root@n4h8) (gcc version 3.3.5 (Debian 
> > 1:3.3.5-13)) #3 SMP Fri Aug 4 10:57:44 PDT 2006
> > BIOS-provided physical RAM map:
> >  Xen: 0000000000000000 - 0000000004800000 (usable)
> > 0MB HIGHMEM available.
> > 72MB LOWMEM available.
> > DMI 2.3 present.
> > ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
> > ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> > ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
> > IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
> > ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
> > IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
> > ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl)
> > Enabling APIC mode:  Flat.  Using 2 I/O APICs
> > Using ACPI (MADT) for SMP configuration information
> > Allocating PCI resources starting at 50000000 (gap: 40000000:bec00000)
> > Built 1 zonelists
> > Kernel command line: root=801 ro console=ttyS0 ramdisk_size=32768
> > Enabling fast FPU save and restore... done.
> > Enabling unmasked SIMD FPU exception support... done.
> > Initializing CPU#0
> > PID hash table entries: 512 (order: 9, 8192 bytes)
> > Xen reported: 1130.113 MHz processor.
> > Console: colour VGA+ 80x25
> > Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
> > Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
> > Software IO TLB enabled:
> >  Aperture:     2 megabytes
> >  Kernel range: 0x00000000c10f0000 - 0x00000000c12f0000
> > vmalloc area: c5000000-fb7fe000, maxmem 33ffe000
> > Memory: 47400k/73728k available (3389k kernel code, 18068k reserved, 1044k 
> > data, 208k init, 0k highmem)
> > Checking if this processor honours the WP bit even in supervisor mode... Ok.
> > Calibrating delay using timer specific routine.. 2261.60 BogoMIPS 
> > (lpj=11308023)
> > Security Framework v1.0.0 initialized
> > Capability LSM initialized
> > Mount-cache hash table entries: 512
> > CPU: L1 I cache: 16K, L1 D cache: 16K
> > CPU: L2 cache: 512K
> > Checking 'hlt' instruction... OK.
> > ENABLING IO-APIC IRQs
> > Enabling SMP...
> > Brought up 2 CPUs
> > Initializing CPU#1
> > migration_cost=1895
> > checking if image is initramfs... it is
> > Freeing initrd memory: 9881k freed
> > Grant table initialized
> > NET: Registered protocol family 16
> > ACPI: bus type pci registered
> > PCI: Using configuration type 1
> > ACPI: Subsystem revision 20060127
> > ACPI: Interpreter enabled
> > ACPI: Using IOAPIC for interrupt routing
> > ACPI: PCI Root Bridge [PCI0] (0000:00)
> > ACPI: PCI Interrupt Link [LPE1] (IRQs *10)
> > ACPI: PCI Interrupt Link [LPE2] (IRQs *10)
> > ACPI: PCI Interrupt Link [LPVI] (IRQs) *0, disabled.
> > ACPI: PCI Interrupt Link [LPUS] (IRQs *7)
> > ACPI: PCI Root Bridge [PCI1] (0000:01)
> > ACPI: PCI Interrupt Link [LPSA] (IRQs *9)
> > ACPI: PCI Interrupt Link [LP1A] (IRQs) *0, disabled.
> > ACPI: PCI Interrupt Link [LP1B] (IRQs) *0, disabled.
> > ACPI: PCI Interrupt Link [LP2A] (IRQs *9)
> > ACPI: PCI Interrupt Link [LP2B] (IRQs) *0, disabled.
> > Linux Plug and Play Support v0.97 (c) Adam Belay
> > xen_mem: Initialising balloon driver.
> > SCSI subsystem initialized
> > usbcore: registered new driver usbfs
> > usbcore: registered new driver hub
> > PCI: Using ACPI for IRQ routing
> > PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a 
> > report
> > IA-32 Microcode Update Driver: v1.14-xen <tigran@xxxxxxxxxxx>
> > VFS: Disk quotas dquot_6.5.1
> > Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
> > JFS: nTxBlock = 512, nTxLock = 4096
> > SGI XFS with ACLs, security attributes, realtime, large block numbers, no 
> > debug enabled
> > Initializing Cryptographic API
> > io scheduler noop registered
> > io scheduler anticipatory registered (default)
> > io scheduler deadline registered
> > io scheduler cfq registered
> > PNP: No PS/2 controller found. Probing ports directly.
> > serio: i8042 AUX port at 0x60,0x64 irq 12
> > serio: i8042 KBD port at 0x60,0x64 irq 1
> > RAMDISK driver initialized: 16 RAM disks of 32768K size 1024 blocksize
> > Xen virtual console successfully installed as ttyS0
> > Event-channel device installed.
> > blkif_init: reqs=64, pages=704, mmap_vstart=0xc0800000
> > Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
> > ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
> > SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1
> > SvrWks OSB4: chipset revision 0
> > SvrWks OSB4: not 100% native mode: will probe irqs later
> >     ide0: BM-DMA at 0x0700-0x0707, BIOS settings: hda:DMA, hdb:DMA
> >     ide1: BM-DMA at 0x0708-0x070f, BIOS settings: hdc:DMA, hdd:DMA
> > hda: LG CD-ROM CRN-8245B, ATAPI CD/DVD-ROM drive
> > ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
> > hda: ATAPI 24X CD-ROM drive, 128kB Cache, (U)DMA
> > Uniform CD-ROM driver Revision: 3.20
> > ide-floppy driver 0.99.newide
> > ACPI: PCI Interrupt 0000:01:03.0[A] -> GSI 28 (level, low) -> IRQ 16
> > scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0
> >         <Adaptec aic7892 Ultra160 SCSI adapter>
> >         aic7892: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs
> > 
> > ACPI: PCI Interrupt 0000:01:05.0[A] -> GSI 20 (level, low) -> IRQ 17
> > scsi1 : IBM PCI ServeRAID 7.12.05  Build 761 <ServeRAID 4L>
> >   Vendor: IBM       Model: SERVERAID         Rev: 1.00
> >   Type:   Direct-Access                      ANSI SCSI revision: 02
> >   Vendor: IBM       Model: SERVERAID         Rev: 1.00
> >   Type:   Direct-Access                      ANSI SCSI revision: 02
> >   Vendor: IBM       Model: SERVERAID         Rev: 1.00
> >   Type:   Processor                          ANSI SCSI revision: 02
> >   Vendor: IBM       Model: FTlV1 S2          Rev: 0
> >   Type:   Processor                          ANSI SCSI revision: 02
> > Fusion MPT base driver 3.03.07
> > Copyright (c) 1999-2005 LSI Logic Corporation
> > Fusion MPT SPI Host driver 3.03.07
> > Fusion MPT misc device (ioctl) driver 3.03.07
> > mptctl: Registered with Fusion MPT base driver
> > mptctl: /dev/mptctl @ (major,minor=10,220)
> > usbmon: debugfs is not available
> > usbcore: registered new driver libusual
> > mice: PS/2 mouse device common for all mice
> > md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
> > md: bitmap version 4.39
> > NET: Registered protocol family 2
> > input: AT Translated Set 2 keyboard as /class/input/input0
> > IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
> > TCP established hash table entries: 4096 (order: 3, 32768 bytes)
> > TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
> > TCP: Hash tables configured (established 4096 bind 4096)
> > TCP reno registered
> > Initializing IPsec netlink socket
> > NET: Registered protocol family 1
> > NET: Registered protocol family 17
> > NET: Registered protocol family 8
> > NET: Registered protocol family 20
> > Using IPI No-Shortcut mode
> > Freeing unused kernel memory: 208k freed
> > Loading, please wait...
> > Begin: Loading essential drivers... ...
> > logips2pp: Detected unknown logitech mouse model 0
> > eepro100.c:v1.09j-t 9/29/99 Donald Becker 
> > http://www.scyld.com/network/eepro100.html
> > eepro100.c: $Revision: 1.36 $ 2000/11/17 Modified by Andrey V. Savochkin 
> > <saw@xxxxxxxxxxxxx> and others
> > ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 27 (level, low) -> IRQ 18
> > eth0: 0000:00:02.0, 00:02:55:C7:CA:D8, IRQ 18.
> >   Board assembly 754338-001, Physical connectors present: RJ45
> >   Primary interface chip i82555 PHY #1.
> >   General self-test: passed.
> >   Serial sub-system self-test: passed.
> >   Internal registers self-test: passed.
> >   ROM checksum self-test: passed (0x04f4518b).
> > ACPI: PCI Interrupt 0000:00:0a.0[A] -> GSI 25 (level, low) -> IRQ 19
> > eth1: 0000:00:0a.0, 00:02:55:C7:CA:D9, IRQ 19.
> >   Board assembly 754338-001, Physical connectors present: RJ45
> >   Primary interface chip i82555 PHY #1.
> >   General self-test: passed.
> >   Serial sub-system self-test: passed.
> >   Internal registers self-test: passed.
> >   ROM checksum self-test: passed (0x04f4518b).
> > Intel(R) PRO/1000 Network Driver - version 6.3.9-k4
> > Copyright (c) 1999-2005 Intel Corporation.
> > Done.
> > Begin: Running /scripts/init-premount ...
> > input: PS/2 Logitech Mouse as /class/input/input1
> > Done.
> > Begin: Mounting root file system... ...
> > Begin: Running /scripts/local-top ...
> > device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel@xxxxxxxxxx
> > /scripts/local-top/lvm: 36: vgchange: not found
> > Done.
> > SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB)
> > sda: assuming Write Enabled
> > sda: assuming drive cache: write through
> > SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB)
> > sda: assuming Write Enabled
> > sda: assuming drive cache: write through
> >  sda: sda1 sda2 < sda5 >
> > sd 1:0:0:0: Attached scsi disk sda
> > SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB)
> > sdb: assuming Write Enabled
> > sdb: assuming drive cache: write through
> > SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB)
> > sdb: assuming Write Enabled
> > sdb: assuming drive cache: write through
> >  sdb: unknown partition table
> > sd 1:0:1:0: Attached scsi disk sdb
> > Begin: Running /scripts/local-premount ...
> > Done.
> > EXT3-fs: INFO: recovery required on readonly filesystem.
> > EXT3-fs: write access will be enabled during recovery.
> > kjournald starting.  Commit interval 5 seconds
> > EXT3-fs: recovery complete.
> > EXT3-fs: mounted filesystem with ordered data mode.
> > Begin: Running /scripts/log-bottom ...
> > Done.
> > Done.
> > Begin: Running /scripts/init-bottom ...
> > Done.
> > INIT: version 2.86 booting
> > Starting the hotplug events dispatcher: udevd.
> > Synthesizing the initial hotplug events...done.
> > Waiting for /dev to be fully populated...done.
> > Loading /etc/console/boottime.kmap.gz
> > Activating swap.
> > Adding 1019880k swap on /dev/sda5.  Priority:-1 extents:1 across:1019880k
> > Checking root file system...
> > fsck 1.37 (21-Mar-2005)
> > /dev/sda1: clean, 58437/512000 files, 445341/1023996 blocks
> > EXT3 FS on sda1, internal journal
> > System time was Sat Aug 12 00:22:00 UTC 2006.
> > Setting the System Clock using the Hardware Clock as reference...
> > System Clock set. System local time is now Sat Aug 12 00:22:01 UTC 2006.
> > Calculating module dependencies...done.
> > Loading modules...
> >     auto
> > FATAL: Module auto not found.
> >     3c59x
> >     softdog
> > Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60 sec 
> > (nowayout= 0)
> >     tulip
> > Linux Tulip driver version 1.1.13 (May 11, 2002)
> >     e100
> > e100: Intel(R) PRO/100 Network Driver, 3.5.10-k2-NAPI
> > e100: Copyright(c) 1999-2005 Intel Corporation
> > All modules loaded.
> > Loading device-mapper support.
> > Starting Enterprise Volume Management System: device-mapper: dm-linear: 
> > Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > evms.
> > Checking all file systems...
> > fsck 1.37 (21-Mar-2005)
> > Setting kernel variables ...
> > ... done.
> > Loading the saved-state of the serial devices...
> > Cannot get serial info: Invalid argument
> > Cannot get serial info: Invalid argument
> > /dev/ttyS1: No such file or directory
> > /dev/ttyS1: No such file or directory
> > Mounting local filesystems...
> > Cleaning /tmp /var/run /var/lock.
> > Cleaning: /etc/network/ifstate.
> > Setting up IP spoofing protection: rp_filter.
> > Configuring network interfaces: done.
> > Starting portmap daemon: portmap.
> > 
> > Setting the System Clock using the Hardware Clock as reference...
> > System Clock set. Local time: Fri Aug 11 17:22:07 PDT 2006
> > 
> > Recovering jove files ... Done.
> > Running ntpdate to synchronize clock.
> > Initializing random number generator...done.
> > Recovering nvi editor sessions... done.
> > Setting up X server socket directory /tmp/.X11-unix...done.
> > Setting up ICE socket directory /tmp/.ICE-unix...done.
> > INIT: Entering runlevel: 2
> > Starting system log daemon: syslogd.
> > Starting kernel log daemon: klogd.
> > Starting virtual private network daemon:.
> > Starting portmap daemon: portmap.
> > Starting NFS common utilities: statd lockd.
> > Starting internet superserver: inetd.
> > * The mptctl module is missing. Please have a look at the README.Debian.gz.
> > .
> > Exporting directories for NFS kernel daemon...Installing knfsd (copyright 
> > (C) 1996 okir@xxxxxxxxxxxx).
> > 10.27.3.21:/export/xen/fs/stevegt/crashme1/root: No such file or directory
> > 10.27.3.18:/export/xen/fs/stevegt/xentest2/root: No such file or directory
> > 10.27.3.17:/export/xen/fs/stevegt/xentest1/root: No such file or directory
> > 10.27.3.8:/export/xen/fs/cclarke/umltest1/root: No such file or directory
> > hotaru.chaosring.Org:/export/xen/fs/stevegt/crashme1/root: No such file or 
> > directory
> > rn-2.refactored-networks.com:/export/xen/fs/stevegt/xentest2/root: No such 
> > file or directory
> > xentest1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/xentest1/root: No 
> > such file or directory
> > umltest1.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/umltest1/root: No 
> > such file or directory
> > 10.27.3.26:/export/xen/fs/stevegt/gforge/root: No such file or directory
> > 10.27.3.23:/export/xen/fs/baseline/sarge/root: No such file or directory
> > 10.27.3.22:/export/xen/fs/stevegt/build1/root: No such file or directory
> > 10.27.3.16:/export/xen/fs/baseline/woody/root: No such file or directory
> > bugs.t7a.Org:/export/xen/fs/stevegt/gforge/root: No such file or directory
> > sarge.baseline.TerraLuna.Org:/export/xen/fs/baseline/sarge/root: No such 
> > file or directory
> > build1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/build1/root: No such 
> > file or directory
> > woody.baseline.TerraLuna.Org:/export/xen/fs/baseline/woody/root: No such 
> > file or directory
> > 10.27.3.19:/export/xen/fs/cclarke/ebond/root: No such file or directory
> > ebond.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/ebond/root: No such file 
> > or directory
> > 10.27.3.29:/export/xen/fs/dmasten/mss0/root: No such file or directory
> > 10.27.3.27:/export/xen/fs/cclarke/ccms/root: No such file or directory
> > 10.27.3.25:/export/xen/fs/baseline/sid/root: No such file or directory
> > 10.27.3.24:/export/xen/fs/stevegt/cvs1/root: No such file or directory
> > sid.stevegt.TerraLuna.Org:/export/xen/fs/baseline/sid/root: No such file or 
> > directory
> > cvs1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/cvs1/root: No such file 
> > or directory
> > 10.27.3.28:/export/xen/fs/stevegt/t7a/root: No such file or directory
> > 10.27.3.20:/export/xen/fs/stevegt/tcx/root: No such file or directory
> > tcx.TerraLuna.Org:/export/xen/fs/stevegt/tcx/root: No such file or directory
> > n2h54.prd.TerraLuna.Org:/export/xen: No such file or directory
> > n2h53.prd.TerraLuna.Org:/export/xen: No such file or directory
> > n2h51.prd.TerraLuna.Org:/export/xen: No such file or directory
> > n2h41.prd.TerraLuna.Org:/export/xen: No such file or directory
> > done.
> > Starting NFS kernel daemon: nfsdNFSD: Using /var/lib/nfs/v4recovery as the 
> > NFSv4 state recovery directory
> > NFSD: unable to find recovery directory /var/lib/nfs/v4recovery
> > NFSD: starting 90-second grace period
> >  mountd.
> > Starting OpenBSD Secure Shell server: sshdNET: Registered protocol family 10
> > lo: Disabled Privacy Extensions
> > IPv6 over IPv4 tunneling driver
> > .
> > Starting NTP server: ntpd.
> > AFS module /lib/modules/2.6.16.13-xen/fs/openafs.mp.o does not exist. Not 
> > starting AFS.
> > Please consider building kernel modules using instructions in
> > /usr/share/doc/openafs-client/README.modules
> > Starting DRBD resources:    drbd: initialised. Version: 0.7.20 
> > (api:79/proto:74)
> > drbd: SVN Revision: 2260 build by root@n4h8, 2006-08-04 11:00:44
> > drbd: registered as block device major 147
> > [ d0 drbd7: resync bitmap: bits=1048576 words=32768
> > drbd7: size = 4095 MB (4194303 KB)
> > drbd7: 0 KB marked out-of-sync by on disk bit-map.
> > drbd7: No usable activity log found.
> > drbd7: drbdsetup [3825]: cstate Unconfigured --> StandAlone
> > d1 drbd8: resync bitmap: bits=1048576 words=32768
> > drbd8: size = 4095 MB (4194303 KB)
> > drbd8: 0 KB marked out-of-sync by on disk bit-map.
> > drbd8: Found 4 transactions (46 active extents) in activity log.
> > drbd8: Marked additional 156 MB as out-of-sync based on AL.
> > drbd8: drbdsetup [3829]: cstate Unconfigured --> StandAlone
> > d2 drbd9: resync bitmap: bits=2621440 words=81920
> > drbd9: size = 9 GB (10485759 KB)
> > drbd9: 0 KB marked out-of-sync by on disk bit-map.
> > drbd9: Found 4 transactions (192 active extents) in activity log.
> > drbd9: drbdsetup [3833]: cstate Unconfigured --> StandAlone
> > d3 drbd10: resync bitmap: bits=1835008 words=57344
> > drbd10: size = 7167 MB (7340031 KB)
> > drbd10: 0 KB marked out-of-sync by on disk bit-map.
> > drbd10: Found 4 transactions (192 active extents) in activity log.
> > drbd10: Marked additional 508 MB as out-of-sync based on AL.
> > drbd10: drbdsetup [3837]: cstate Unconfigured --> StandAlone
> > d4 drbd11: resync bitmap: bits=786432 words=24576
> > drbd11: size = 3071 MB (3145727 KB)
> > drbd11: 0 KB marked out-of-sync by on disk bit-map.
> > drbd11: Found 4 transactions (192 active extents) in activity log.
> > drbd11: drbdsetup [3841]: cstate Unconfigured --> StandAlone
> > d5 drbd12: resync bitmap: bits=786432 words=24576
> > drbd12: size = 3071 MB (3145727 KB)
> > drbd12: 0 KB marked out-of-sync by on disk bit-map.
> > drbd12: Found 4 transactions (136 active extents) in activity log.
> > drbd12: Marked additional 508 MB as out-of-sync based on AL.
> > drbd12: drbdsetup [3845]: cstate Unconfigured --> StandAlone
> > s0 s1 s2 s3 s4 s5 n0 drbd7: drbdsetup [3883]: cstate StandAlone --> 
> > Unconnected
> > drbd7: drbd7_receiver [3884]: cstate Unconnected --> WFConnection
> > n1 drbd8: drbdsetup [3891]: cstate StandAlone --> Unconnected
> > drbd8: drbd8_receiver [3892]: cstate Unconnected --> WFConnection
> > n2 drbd9: drbdsetup [3899]: cstate StandAlone --> Unconnected
> > drbd9: drbd9_receiver [3900]: cstate Unconnected --> WFConnection
> > n3 drbd10: drbdsetup [3907]: cstate StandAlone --> Unconnected
> > drbd10: drbd10_receiver [3908]: cstate Unconnected --> WFConnection
> > n4 drbd11: drbdsetup [3915]: cstate StandAlone --> Unconnected
> > drbd11: drbd11_receiver [3916]: cstate Unconnected --> WFConnection
> > n5 drbd12: drbdsetup [3923]: cstate StandAlone --> Unconnected
> > drbd12: drbd12_receiver [3924]: cstate Unconnected --> WFConnection
> > ].
> > ..........
> > ***************************************************************
> >  DRBD's startup script waits for the peer node(s) to appear.
> >  - In case this node was already a degraded cluster before the
> >    reboot the timeout is 120 seconds. [degr-wfc-timeout]
> >  - If the peer was available before the reboot the timeout will
> >    expire after 0 seconds. [wfc-timeout]
> >    (These values are for resource 'ds1.t7a.org'; 0 sec -> wait forever)
> >  To abort waiting enter 'yes' [  18]:drbd7: drbd7_receiver [3884]: cstate 
> > WFConnection --> WFReportParams
> > drbd7: Handshake successful: DRBD Network Protocol version 74
> > drbd7: Connection established.
> > drbd7: I am(S): 1:00000002:00000001:00000018:00000002:01
> > drbd7: Peer(S): 1:00000002:00000001:00000019:00000002:10
> > drbd7: drbd7_receiver [3884]: cstate WFReportParams --> WFBitMapT
> > drbd7: Secondary/Unknown --> Secondary/Secondary
> > drbd7: drbd7_receiver [3884]: cstate WFBitMapT --> SyncTarget
> > drbd7: Resync started as SyncTarget (need to sync 520192 KB [130048 bits 
> > set]).
> > drbd9: drbd9_receiver [3900]: cstate WFConnection --> WFReportParams
> > drbd9: Handshake successful: DRBD Network Protocol version 74
> > drbd9: Connection established.
> > drbd9: I am(S): 1:00000002:00000001:0000000d:00000001:01
> > drbd9: Peer(S): 1:00000002:00000001:0000000e:00000001:10
> > drbd9: drbd9_receiver [3900]: cstate WFReportParams --> WFBitMapT
> > drbd9: Secondary/Unknown --> Secondary/Secondary
> > drbd9: drbd9_receiver [3900]: cstate WFBitMapT --> SyncTarget
> > drbd9: Resync started as SyncTarget (need to sync 290816 KB [72704 bits 
> > set]).
> > drbd9: drbd9_receiver [3900]: cstate SyncTarget --> PausedSyncT
> > drbd9: Syncer waits for sync group.
> > drbd11: drbd11_receiver [3916]: cstate WFConnection --> WFReportParams
> > drbd11: Handshake successful: DRBD Network Protocol version 74
> > drbd11: Connection established.
> > drbd11: I am(S): 1:00000002:00000001:00000008:00000001:01
> > drbd11: Peer(S): 1:00000002:00000001:00000009:00000001:10
> > drbd11: drbd11_receiver [3916]: cstate WFReportParams --> WFBitMapT
> > drbd11: Secondary/Unknown --> Secondary/Secondary
> > drbd11: drbd11_receiver [3916]: cstate WFBitMapT --> SyncTarget
> > drbd11: Resync started as SyncTarget (need to sync 274432 KB [68608 bits 
> > set]).
> > drbd11: drbd11_receiver [3916]: cstate SyncTarget --> PausedSyncT
> > drbd11: Syncer waits for sync group.
> > drbd8: drbd8_receiver [3892]: cstate WFConnection --> WFReportParams
> > drbd8: Handshake successful: DRBD Network Protocol version 74
> > drbd8: Connection established.
> > drbd8: I am(S): 1:00000002:00000001:00000017:00000001:11
> > drbd8: Peer(S): 1:00000002:00000001:00000017:00000001:01
> > drbd8: drbd8_receiver [3892]: cstate WFReportParams --> WFBitMapS
> > drbd8: Secondary/Unknown --> Secondary/Secondary
> > drbd8: drbd8_receiver [3892]: cstate WFBitMapS --> SyncSource
> > drbd8: Resync started as SyncSource (need to sync 159744 KB [39936 bits 
> > set]).
> > drbd8: drbd8_receiver [3892]: cstate SyncSource --> PausedSyncS
> > drbd8: Syncer waits for sync group.
> > drbd12: drbd12_receiver [3924]: cstate WFConnection --> WFReportParams
> > drbd12: Handshake successful: DRBD Network Protocol version 74
> > drbd12: Connection established.
> > drbd12: I am(S): 1:00000002:00000001:00000001:00000001:11
> > drbd12: Peer(S): 1:00000002:00000001:00000001:00000001:01
> > drbd12: drbd12_receiver [3924]: cstate WFReportParams --> WFBitMapS
> > drbd12: Secondary/Unknown --> Secondary/Secondary
> > drbd12: drbd12_receiver [3924]: cstate WFBitMapS --> SyncSource
> > drbd12: Resync started as SyncSource (need to sync 520192 KB [130048 bits 
> > set]).
> > drbd12: drbd12_receiver [3924]: cstate SyncSource --> PausedSyncS
> > drbd12: Syncer waits for sync group.
> > drbd10: drbd10_receiver [3908]: cstate WFConnection --> WFReportParams
> > drbd10: Handshake successful: DRBD Network Protocol version 74
> > drbd10: Connection established.
> > drbd10: I am(S): 1:00000002:00000001:0000000b:00000001:11
> > drbd10: Peer(S): 1:00000002:00000001:0000000b:00000001:01
> > drbd10: drbd10_receiver [3908]: cstate WFReportParams --> WFBitMapS
> > 
> > drbd10: Secondary/Unknown --> Secondary/Secondary
> > drbd10: drbd10_receiver [3908]: cstate WFBitMapS --> SyncSource
> > drbd10: Resync started as SyncSource (need to sync 520192 KB [130048 bits 
> > set]).
> > drbd10: drbd10_receiver [3908]: cstate SyncSource --> PausedSyncS
> > drbd10: Syncer waits for sync group.
> > Heartbeat not configured: /etc/ha.d/ha.cf not found.
> >  Heartbeat failure [rc=1]. Failed.
> > Starting deferred execution scheduler: atd.
> > Starting periodic command scheduler: cron.
> > Starting watchdog daemon: watchdog.
> > openMosix: ERROR: Cannot find the /proc/hpc directory.
> > openMosix: ERROR: Looks like this is not an openMosix enabled kernel.
> > openMosix: ERROR: Configuration ABORTED.
> > openMosix: HINT: need to recompile your kernel with openMosix support 
> > enabled?
> > openMosix: HINT: need to update your boot manager (lilo, grub, etc)?
> > openMosix: HINT: have you read the docs at http://openMosix.sf.net/ ?
> > Bridge firewalling registered
> > device vif0.0 entered promiscuous mode
> > xenbr0: port 1(vif0.0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 1(vif0.0) entering forwarding state
> > device peth0 entered promiscuous mode
> > xenbr0: port 2(peth0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 2(peth0) entering forwarding state
> > Starting auto Xen domains: afs1.t7a.orgdrbd9: Secondary/Secondary --> 
> > Secondary/Primary
> > device vif1.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif1.0: link is not ready
> > ip_tables: (C) 2000-2006 Netfilter Core Team
> > Error: Device 2049 (vbd) could not be connected. Backend device not found.
> > xenbr0: port 3(vif1.0) entering disabled state
> > ! build2.t7a.orgdevice vif1.0 left promiscuous mode
> > xenbr0: port 3(vif1.0) entering disabled state
> > device vif2.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif2.0: link is not ready
> > drbd10: Secondary/Secondary --> Primary/Secondary
> > ADDRCONF(NETDEV_CHANGE): vif2.0: link becomes ready
> > xenbr0: port 3(vif2.0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 3(vif2.0) entering forwarding state
> >  ds1.t7a.orgdrbd7: Secondary/Secondary --> Secondary/Primary
> > device vif3.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif3.0: link is not ready
> > Error: Device 2049 (vbd) could not be connected. Backend device not found.
> > ! kdc1.t7a.orgxenbr0: port 4(vif3.0) entering disabled state
> > 
> > device vif3.0 left promiscuous mode
> > xenbr0: port 4(vif3.0) entering disabled state
> > 
> > 
> > 
> > 
> > 
> > 
> > drbd8: Secondary/Secondary --> Primary/Secondary
> > device vif4.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif4.0: link is not ready
> > ADDRCONF(NETDEV_CHANGE): vif4.0: link becomes ready
> > xenbr0: port 4(vif4.0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 4(vif4.0) entering forwarding state
> >  w1.cdint.comdrbd11: Secondary/Secondary --> Secondary/Primary
> > device vif5.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif5.0: link is not ready
> > Error: Device 2049 (vbd) could not be connected. Backend device not found.
> > xenbr0: port 5(vif5.0) entering disabled state
> > ! webkdc.t7a.orgdevice vif5.0 left promiscuous mode
> > xenbr0: port 5(vif5.0) entering disabled state
> > device vif6.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif6.0: link is not ready
> > drbd12: Secondary/Secondary --> Primary/Secondary
> > ADDRCONF(NETDEV_CHANGE): vif6.0: link becomes ready
> > xenbr0: port 5(vif6.0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 5(vif6.0) entering forwarding state
> >   [done]
> > 
> > 
> >          _sudZUZ#Z#XZo=_         DDDD   EEEEEE BBBB   IIIIII  AAAA   NN   NN
> >       _jmZZ2!!~---~!!X##wa       DD DD  EE     BB BB    II   AA  AA  NNN  NN
> >    .<wdP~~            -!YZL,     DD  DD EEEEE  BBBBB    II   AAAAAA  NNNN NN
> >   .mX2'       _%aaa__     XZ[.   DD DD  EE     BB  BB   II   AA  AA  NN NNNN
> >   oZ[      _jdXY!~?S#wa   ]Xb;   DDDD   EEEEEE BBBBB  IIIIII AA  AA  NN   NN
> >  _#e'     .]X2(     ~Xw|  )XXc
> > .2Z`      ]X[.       xY|  ]oZ(   Linux Version 2.6.16.13-xen
> > .2#;      )3k;     _s!~   jXf`   Compiled #3 SMP Fri Aug 4 10:57:44 PDT 2006
> >  1Z>      -]Xb/    ~    __#2(    Two 1.13GHz Intel Intel(R) Pentium(R) III 
> > CPU family      1133MHz Processors
> >  -Zo;       +!4ZwaaaauZZXY'      72M RAM
> >   *#[,        ~-?!!!!!!-~        4523.20 Bogomips Total
> >    XUb;.                         n4h34
> >     )YXL,,
> >       +3#bc,
> >         -)SSL,,
> >            ~~~~~
> > Updating the Linuxlogo... done.
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login: drbd7: Resync done (total 137 sec; paused 0 sec; 3796 K/sec)
> > drbd7: drbd7_worker [3826]: cstate SyncTarget --> Connected
> > drbd8: Syncer continues.
> > drbd8: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
> > drbd9: Syncer continues.
> > drbd9: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget
> > drbd10: Syncer continues.
> > drbd10: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
> > drbd11: Syncer continues.
> > drbd11: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget
> > drbd12: Syncer continues.
> > drbd12: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
> > drbd8: Resync done (total 211 sec; paused 137 sec; 2156 K/sec)
> > drbd8: drbd8_worker [3830]: cstate SyncSource --> Connected
> > drbd11: Resync done (total 239 sec; paused 137 sec; 2688 K/sec)
> > drbd11: drbd11_worker [3842]: cstate SyncTarget --> Connected
> > drbd9: Resync done (total 243 sec; paused 137 sec; 2740 K/sec)
> > drbd9: drbd9_worker [3834]: cstate SyncTarget --> Connected
> > 
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login: drbd12: Resync done (total 306 sec; paused 136 sec; 3056 K/sec)
> > drbd12: drbd12_worker [3846]: cstate SyncSource --> Connected
> > 
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login: drbd10: Resync done (total 310 sec; paused 136 sec; 2988 K/sec)
> > drbd10: drbd10_worker [3838]: cstate SyncSource --> Connected
> > 
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> > 
> > n4h34 login: Unable to handle kernel paging request at virtual address 
> > c0976590
> >  printing eip:
> > c03d6ed7
> > *pde = ma 3cf91067 pa 00f91067
> > *pte = ma 00000000 pa fffff000
> > Oops: 0000 [#1]
> > SMP
> > Modules linked in: iptable_filter ip_tables x_tables bridge drbd ipv6 nfsd 
> > lockd sunrpc e100 tulip softdog 3c59x evdev sd_mod dm_mod thermal processor 
> > fan e1000 eepro100 mii tg3
> > CPU:    0
> > EIP:    0061:[<c03d6ed7>]    Not tainted VLI
> > EFLAGS: 00010206   (2.6.16.13-xen #3)
> > EIP is at skb_copy_bits+0x127/0x280
> > eax: c0976000   ebx: 000005a8   ecx: 0000016a   edx: c3711720
> > esi: c0976590   edi: c37110e0   ebp: 000005a8   esp: c00818c8
> > ds: 007b   es: 007b   ss: 0069
> > Process drbd12_receiver (pid: 3924, threadinfo=c0080000 task=c3efda90)
> > Stack: <0>c1012ec0 00000002 c03d65ef c1d51a00 000005ea 00000042 00000000 
> > 00000000
> >        00000020 c3edc800 c376fd64 c03d6b0f c376fd64 00000042 c37110e0 
> > 000005a8
> >        c1a70000 c54690c0 00000000 c1d51ac0 c3edc800 c376fd64 c03dc478 
> > c376fd64
> > Call Trace:
> >  [<c03d65ef>] pskb_expand_head+0xdf/0x140
> >  [<c03d6b0f>] __pskb_pull_tail+0x7f/0x320
> >  [<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
> >  [<c03dc478>] dev_queue_xmit+0x328/0x370
> >  [<c5462f7e>] br_dev_queue_push_xmit+0xbe/0x140 [bridge]
> >  [<c5469212>] br_nf_post_routing+0x102/0x1c0 [bridge]
> >  [<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
> >  [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> >  [<c03f43f8>] nf_iterate+0x78/0x90
> >  [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> >  [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> >  [<c03f447e>] nf_hook_slow+0x6e/0x110
> >  [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> >  [<c5463061>] br_forward_finish+0x61/0x70 [bridge]
> >  [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> >  [<c5468995>] br_nf_forward_finish+0x75/0x130 [bridge]
> >  [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> >  [<c5468b38>] br_nf_forward_ip+0xe8/0x190 [bridge]
> >  [<c5468920>] br_nf_forward_finish+0x0/0x130 [bridge]
> >  [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> >  [<c03f43f8>] nf_iterate+0x78/0x90
> >  [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> >  [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> >  [<c03f447e>] nf_hook_slow+0x6e/0x110
> >  [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> >  [<c5463167>] __br_forward+0x77/0x80 [bridge]
> >  [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> >  [<c5463fbf>] br_handle_frame_finish+0xdf/0x160 [bridge]
> >  [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> >  [<c5467d89>] br_nf_pre_routing_finish+0xf9/0x370 [bridge]
> >  [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> >  [<c0322e3a>] loopback_start_xmit+0xba/0x110
> >  [<c0400d70>] ip_finish_output+0x0/0x220
> >  [<c03dc07e>] dev_hard_start_xmit+0x5e/0x130
> >  [<c03dc3b5>] dev_queue_xmit+0x265/0x370
> >  [<c03f43f8>] nf_iterate+0x78/0x90
> >  [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> >  [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> >  [<c03f447e>] nf_hook_slow+0x6e/0x110
> >  [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> >  [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> >  [<c54685fc>] br_nf_pre_routing+0x26c/0x520 [bridge]
> >  [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> >  [<c03f43f8>] nf_iterate+0x78/0x90
> >  [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> >  [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> >  [<c03f447e>] nf_hook_slow+0x6e/0x110
> >  [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> >  [<c546422d>] br_handle_frame+0x1ed/0x230 [bridge]
> >  [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> >  [<c03dcb21>] netif_receive_skb+0x1a1/0x330
> >  [<c03dcd87>] process_backlog+0xd7/0x190
> >  [<c03dcf2a>] net_rx_action+0xea/0x230
> >  [<c0125915>] __do_softirq+0xf5/0x120
> >  [<c01259d5>] do_softirq+0x95/0xa0
> >  [<c0125a42>] local_bh_enable+0x62/0xa0
> >  [<c0406bd1>] tcp_prequeue_process+0x71/0x80
> >  [<c04070e9>] tcp_recvmsg+0x349/0x750
> >  [<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
> >  [<c03d5085>] sock_common_recvmsg+0x55/0x70
> >  [<c03d11cf>] sock_recvmsg+0xef/0x110
> >  [<c03143fa>] force_evtchn_callback+0xa/0x10
> >  [<c0147163>] mempool_alloc+0x33/0xe0
> >  [<c01367d0>] autoremove_wake_function+0x0/0x60
> >  [<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
> >  [<c02b2ee0>] generic_make_request+0xf0/0x160
> >  [<c529c680>] drbd_recv+0x90/0x190 [drbd]
> >  [<c529cdec>] drbd_recv_header+0x2c/0xf0 [drbd]
> >  [<c529e580>] receive_DataRequest+0x0/0x7d0 [drbd]
> >  [<c52a045c>] drbdd+0x1c/0x150 [drbd]
> >  [<c52a105a>] drbdd_init+0x7a/0x1a0 [drbd]
> >  [<c52a7136>] drbd_thread_setup+0x86/0xf0 [drbd]
> >  [<c52a70b0>] drbd_thread_setup+0x0/0xf0 [drbd]
> >  [<c0102f75>] kernel_thread_helper+0x5/0x10
> > Code: 8b 4c 24 30 8b 7c 24 34 8b 91 a0 00 00 00 8b 4c 24 18 0f b7 74 ca 18 
> > 8b 4c 24 14 8d 34 06 01 fe 29 ce 8b 7c 24 38 89 d9 c1 e9 02 <f3> a5 89 d9 
> > 83 e1 03 74 02 f3 a4 89 04 24 ba 02 00 00 00 89 54
> >  <0>Kernel panic - not syncing: Fatal exception in interrupt
> >  (XEN) Domain 0 crashed: rebooting machine in 5 seconds.
> > 
> > -- 
> > Stephen G. Traugott  (KG6HDQ)
> > Managing Partner, TerraLuna LLC
> > stevegt@xxxxxxxxxxxxx -- http://www.t7a.org
> 
> -- 
> Stephen G. Traugott  (KG6HDQ)
> Managing Partner, TerraLuna LLC
> stevegt@xxxxxxxxxxxxx -- http://www.t7a.org

-- 
Stephen G. Traugott  (KG6HDQ)
Managing Partner, TerraLuna LLC
stevegt@xxxxxxxxxxxxx -- http://www.t7a.org

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel