WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] dom0 crash, require assistance interpretting logs and co

To: Ben Holt <ben@xxxxxxxxxxxxxxxx>
Subject: Re: [Xen-users] dom0 crash, require assistance interpretting logs and config suggestions
From: Pasi Kärkkäinen <pasik@xxxxxx>
Date: Mon, 12 Apr 2010 09:31:56 +0300
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Sun, 11 Apr 2010 23:33:14 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4BC22D19.2070801@xxxxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <4BC22D19.2070801@xxxxxxxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.18 (2008-05-17)
On Sun, Apr 11, 2010 at 01:12:09PM -0700, Ben Holt wrote:
> Hello,
>
> I have experienced a dom0 crash where the system became unreachable via  
> the network and the console was unresponsive.  I would appreciate help  
> interpretting the logs and any configuration change suggestions.
>
> It is a stock Debian Lenny dom0 running xen 3.2.1 with kernel  
> 2.6.26-2-xen-amd64 and an AMD Athlon IIx4 with 4 GB of RAM.  It is  
> running 4 VMs.  One VM has two PCI NICs being passed to it, while  
> another has three PCI MPEG encoders passed to it (physically they are  
> only two cards, but one, a hauppage pvr-500, has two encoders on it  
> connected via a PCI bridge).  Three of the VMs were migrated from  
> another machine and are running Debian kernel 2.6.26-2-xen-686 (32 bit),  
> the fourth is running the same kernel as the dom0 (64 bit).
>
> As you'll see from the logs the system has three hard drives, a PATA  
> (hdc) that holds the dom0 and VMs OS file systems, and 2 SATA (sda, sdb)  
> that are configured in a software RAID 1 and hold the VMs data  
> filesystems.  If I am reading things right all three drives appear to  
> experience interrupt / DMA issues during the crash.  The peth0 NIC goes  
> down and up during the crash.  There is also a mention of IRQ 22, which  
> is used by the SATA controller.
>
> Any pointers or suggests are greatly appreciated.
>

>
>
> From /var/log/syslog:
>
> Apr 11 04:22:29 xenserver kernel: [277113.497127] irq 22: nobody cared  
> (try booting with the "irqpoll" option)
> Apr 11 04:22:29 xenserver kernel: [277113.497144] Pid: 0, comm: swapper  
> Not tainted 2.6.26-2-xen-amd64 #1
>

So I assume this is the host/dom0 kernel.

What device is using IRQ 22? 
Try checking from /proc/interrupts or from lspci.

Is it a shared IRQ (multiple devices using the same IRQ)? 

-- Pasi


> Apr 11 04:22:29 xenserver kernel: [277113.497152]
> Apr 11 04:22:29 xenserver kernel: [277113.497153] Call Trace:
> Apr 11 04:22:29 xenserver kernel: [277113.497165]  <IRQ>   
> [<ffffffff8037ce18>] irq_ignore_unhandled+0x1c/0x32
> Apr 11 04:22:29 xenserver kernel: [277113.497188]  [<ffffffff8025f88f>]  
> __report_bad_irq+0x30/0x72
> Apr 11 04:22:29 xenserver kernel: [277113.497197]  [<ffffffff8025fb58>]  
> note_interrupt+0x287/0x2c7
> Apr 11 04:22:29 xenserver kernel: [277113.497207]  [<ffffffff80260440>]  
> handle_level_irq+0xc3/0x118
> Apr 11 04:22:29 xenserver kernel: [277113.497218]  [<ffffffff8020e146>]  
> do_IRQ+0x4e/0x9a
> Apr 11 04:22:29 xenserver kernel: [277113.497227]  [<ffffffff8037db2c>]  
> evtchn_do_upcall+0x13c/0x1fc
> Apr 11 04:22:29 xenserver kernel: [277113.497237]  [<ffffffff8020bbde>]  
> do_hypervisor_callback+0x1e/0x30
> Apr 11 04:22:29 xenserver kernel: [277113.497245]  <EOI>   
> [<ffffffff8020e79d>] xen_safe_halt+0x90/0xa6
> Apr 11 04:22:29 xenserver kernel: [277113.497263]  [<ffffffff8020a0ce>]  
> xen_idle+0x2e/0x66
> Apr 11 04:22:29 xenserver kernel: [277113.497271]  [<ffffffff80209d49>]  
> cpu_idle+0x97/0xb9
> Apr 11 04:22:29 xenserver kernel: [277113.497283]
> Apr 11 04:22:29 xenserver kernel: [277113.497288] handlers:
> Apr 11 04:22:29 xenserver kernel: [277113.497295] [<ffffffffa0099969>]  
> (ahci_interrupt+0x0/0x46b [ahci])
> Apr 11 04:22:29 xenserver kernel: [277113.497312] Disabling IRQ #22
> Apr 11 04:22:54 xenserver kernel: [277138.645875] hdc: dma_timer_expiry:  
> dma status == 0x64
> Apr 11 04:23:00 xenserver kernel: [277145.647409] ata2.00: exception  
> Emask 0x40 SAct 0x0 SErr 0x800 action 0x6 frozen
> Apr 11 04:23:00 xenserver kernel: [277145.647429] ata2: SError: { HostInt }
> Apr 11 04:23:00 xenserver kernel: [277145.647438] ata2.00: cmd  
> ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0
> Apr 11 04:23:00 xenserver kernel: [277145.647439]          res  
> 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x44 (timeout)
> Apr 11 04:23:00 xenserver kernel: [277145.647458] ata2.00: status: { DRDY }
> Apr 11 04:23:00 xenserver kernel: [277145.647467] ata2: hard resetting link
> Apr 11 04:23:00 xenserver kernel: [277145.647486] ata1.00: exception  
> Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
> Apr 11 04:23:00 xenserver kernel: [277145.647496] ata1.00: cmd  
> ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0
> Apr 11 04:23:00 xenserver kernel: [277145.647497]          res  
> 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> Apr 11 04:23:00 xenserver kernel: [277145.647515] ata1.00: status: { DRDY }
> Apr 11 04:23:00 xenserver kernel: [277145.647523] ata1: hard resetting link
> Apr 11 04:23:01 xenserver kernel: [277146.131362] ata2: softreset failed  
> (device not ready)
> Apr 11 04:23:01 xenserver kernel: [277146.131378] ata2: failed due to HW  
> bug, retry pmp=0
> Apr 11 04:23:01 xenserver kernel: [277146.131400] ata1: softreset failed  
> (device not ready)
> Apr 11 04:23:01 xenserver kernel: [277146.131409] ata1: failed due to HW  
> bug, retry pmp=0
> Apr 11 04:23:01 xenserver kernel: [277146.295250] ata1: SATA link up 3.0  
> Gbps (SStatus 123 SControl 300)
> Apr 11 04:23:01 xenserver kernel: [277146.295287] ata2: SATA link up 3.0  
> Gbps (SStatus 123 SControl 300)
> Apr 11 04:23:03 xenserver kernel: [277148.647303] hdc: DMA interrupt  
> recovery
> Apr 11 04:23:03 xenserver kernel: [277148.647316] hdc: lost interrupt
> Apr 11 04:23:24 xenserver kernel: [277168.647352] hdc: dma_timer_expiry:  
> dma status == 0x64
> Apr 11 04:23:31 xenserver kernel: [277176.295271] ata2.00: qc timeout  
> (cmd 0xec)
> Apr 11 04:23:31 xenserver kernel: [277176.295292] ata2.00: failed to  
> IDENTIFY (I/O error, err_mask=0x4)
> Apr 11 04:23:31 xenserver kernel: [277176.295301] ata2.00: revalidation  
> failed (errno=-5)
> Apr 11 04:23:31 xenserver kernel: [277176.295310] ata2: failed to  
> recover some devices, retrying in 5 secs
> Apr 11 04:23:31 xenserver kernel: [277176.295322] ata1.00: qc timeout  
> (cmd 0xec)
> Apr 11 04:23:31 xenserver kernel: [277176.295336] ata1.00: failed to  
> IDENTIFY (I/O error, err_mask=0x4)
> Apr 11 04:23:31 xenserver kernel: [277176.295344] ata1.00: revalidation  
> failed (errno=-5)
> Apr 11 04:23:31 xenserver kernel: [277176.295352] ata1: failed to  
> recover some devices, retrying in 5 secs
> Apr 11 04:23:33 xenserver kernel: [277178.647242] hdc: DMA interrupt  
> recovery
> Apr 11 04:23:33 xenserver kernel: [277178.647255] hdc: lost interrupt
> Apr 11 04:23:36 xenserver kernel: [277181.299161] ata1: hard resetting link
> Apr 11 04:23:36 xenserver kernel: [277181.299185] ata2: hard resetting link
> Apr 11 04:23:37 xenserver kernel: [277181.783130] ata2: softreset failed  
> (device not ready)
> Apr 11 04:23:37 xenserver kernel: [277181.783147] ata2: failed due to HW  
> bug, retry pmp=0
> Apr 11 04:23:37 xenserver kernel: [277181.783170] ata1: softreset failed  
> (device not ready)
> Apr 11 04:23:37 xenserver kernel: [277181.783178] ata1: failed due to HW  
> bug, retry pmp=0
> Apr 11 04:23:37 xenserver kernel: [277181.947140] ata1: SATA link up 3.0  
> Gbps (SStatus 123 SControl 300)
> Apr 11 04:23:37 xenserver kernel: [277181.947177] ata2: SATA link up 3.0  
> Gbps (SStatus 123 SControl 300)
> Apr 11 04:23:54 xenserver kernel: [277198.648158] hdc: dma_timer_expiry:  
> dma status == 0x64
> Apr 11 04:24:03 xenserver kernel: [277208.648119] hdc: DMA interrupt  
> recovery
> Apr 11 04:24:03 xenserver kernel: [277208.648131] hdc: lost interrupt
> Apr 11 04:24:05 xenserver kernel: [277210.437511] NETDEV WATCHDOG:  
> peth0: transmit timed out
> Apr 11 04:24:05 xenserver kernel: [277210.437533] ------------[ cut here  
> ]------------
> Apr 11 04:24:05 xenserver kernel: [277210.437541] WARNING: at  
> net/sched/sch_generic.c:222 dev_watchdog+0xa6/0x119()
> Apr 11 04:24:05 xenserver kernel: [277210.437553] Modules linked in:  
> xt_tcpudp xfs xt_physdev iptable_filter ip_tables x_tables bridge  
> netloop ipv6 raid1 md_mod loop snd_hda_intel snd_pcm snd_timer snd  
> psmouse soundcore pcspkr serio_raw snd_page_alloc i2c_piix4 i2c_core  
> button wmi shpchp pci_hotplug evdev ext3 jbd mbcache dm_mirror dm_log  
> dm_snapshot dm_mod ide_cd_mod cdrom ide_disk ide_pci_generic sd_mod  
> atiixp ide_core ata_generic ahci ehci_hcd libata scsi_mod dock ohci_hcd  
> r8169 thermal processor fan thermal_sys [last unloaded: scsi_wait_scan]
> Apr 11 04:24:05 xenserver kernel: [277210.437736] Pid: 0, comm: swapper  
> Not tainted 2.6.26-2-xen-amd64 #1
> Apr 11 04:24:05 xenserver kernel: [277210.437744]
> Apr 11 04:24:05 xenserver kernel: [277210.437745] Call Trace:
> Apr 11 04:24:05 xenserver kernel: [277210.437757]  <IRQ>   
> [<ffffffff8022c9a2>] warn_on_slowpath+0x51/0x78
> Apr 11 04:24:05 xenserver kernel: [277210.437774]  [<ffffffff8022d4f4>]  
> printk+0x4e/0x56
> Apr 11 04:24:05 xenserver kernel: [277210.437784]  [<ffffffff80235a70>]  
> lock_timer_base+0x26/0x4b
> Apr 11 04:24:05 xenserver kernel: [277210.437794]  [<ffffffff80235c16>]  
> __mod_timer+0xd4/0xe3
> Apr 11 04:24:05 xenserver kernel: [277210.437804]  [<ffffffff8023ca2f>]  
> queue_delayed_work_on+0xb8/0xc8
> Apr 11 04:24:05 xenserver kernel: [277210.437813]  [<ffffffff803d8161>]  
> dev_watchdog+0x0/0x119
> Apr 11 04:24:05 xenserver kernel: [277210.437822]  [<ffffffff803d8207>]  
> dev_watchdog+0xa6/0x119
> Apr 11 04:24:05 xenserver kernel: [277210.437831]  [<ffffffff802354df>]  
> run_timer_softirq+0x190/0x237
> Apr 11 04:24:05 xenserver kernel: [277210.437842]  [<ffffffff80231ad0>]  
> __do_softirq+0x77/0x103
> Apr 11 04:24:05 xenserver kernel: [277210.437852]  [<ffffffff8020c13c>]  
> call_softirq+0x1c/0x28
> Apr 11 04:24:05 xenserver kernel: [277210.437861]  [<ffffffff8020e092>]  
> do_softirq+0x55/0xbb
> Apr 11 04:24:05 xenserver kernel: [277210.437870]  [<ffffffff8020e175>]  
> do_IRQ+0x7d/0x9a
> Apr 11 04:24:05 xenserver kernel: [277210.437879]  [<ffffffff8037db2c>]  
> evtchn_do_upcall+0x13c/0x1fc
> Apr 11 04:24:05 xenserver kernel: [277210.437890]  [<ffffffff8020bbde>]  
> do_hypervisor_callback+0x1e/0x30
> Apr 11 04:24:05 xenserver kernel: [277210.437898]  <EOI>   
> [<ffffffff8020e79d>] xen_safe_halt+0x90/0xa6
> Apr 11 04:24:05 xenserver kernel: [277210.437916]  [<ffffffff8020a0ce>]  
> xen_idle+0x2e/0x66
> Apr 11 04:24:05 xenserver kernel: [277210.437924]  [<ffffffff80209d49>]  
> cpu_idle+0x97/0xb9
> Apr 11 04:24:05 xenserver kernel: [277210.437936]
> Apr 11 04:24:05 xenserver kernel: [277210.437942] ---[ end trace  
> 470e1e7e4a9eb321 ]---
> Apr 11 04:24:06 xenserver kernel: [277210.456294] r8169: peth0: link up
> Apr 11 04:24:07 xenserver kernel: [277211.948161] ata2.00: qc timeout  
> (cmd 0xec)
> Apr 11 04:24:07 xenserver kernel: [277211.948182] ata2.00: failed to  
> IDENTIFY (I/O error, err_mask=0x4)
> Apr 11 04:24:07 xenserver kernel: [277211.948191] ata2.00: revalidation  
> failed (errno=-5)
> Apr 11 04:24:07 xenserver kernel: [277211.948199] ata2: failed to  
> recover some devices, retrying in 5 secs
> Apr 11 04:24:07 xenserver kernel: [277211.948211] ata1.00: qc timeout  
> (cmd 0xec)
> Apr 11 04:24:07 xenserver kernel: [277211.948225] ata1.00: failed to  
> IDENTIFY (I/O error, err_mask=0x4)
> Apr 11 04:24:07 xenserver kernel: [277211.948233] ata1.00: revalidation  
> failed (errno=-5)
> Apr 11 04:24:07 xenserver kernel: [277211.948240] ata1: failed to  
> recover some devices, retrying in 5 secs
> Apr 11 04:24:12 xenserver kernel: [277216.952165] ata1: hard resetting link
> Apr 11 04:24:12 xenserver kernel: [277216.952189] ata2: hard resetting link
> Apr 11 04:24:12 xenserver kernel: [277217.436134] ata1: softreset failed  
> (device not ready)
> Apr 11 04:24:12 xenserver kernel: [277217.436150] ata1: failed due to HW  
> bug, retry pmp=0
> Apr 11 04:24:12 xenserver kernel: [277217.436173] ata2: softreset failed  
> (device not ready)
> Apr 11 04:24:12 xenserver kernel: [277217.436181] ata2: failed due to HW  
> bug, retry pmp=0
> Apr 11 04:24:12 xenserver kernel: [277217.600168] ata2: SATA link up 3.0  
> Gbps (SStatus 123 SControl 300)
> Apr 11 04:24:12 xenserver kernel: [277217.600210] ata1: SATA link up 3.0  
> Gbps (SStatus 123 SControl 300)
> Apr 11 04:24:17 xenserver kernel: [277222.438507] NETDEV WATCHDOG:  
> peth0: transmit timed out
> Apr 11 04:24:18 xenserver kernel: [277222.456219] r8169: peth0: link up
> Apr 11 04:24:24 xenserver kernel: [277228.648149] hdc: dma_timer_expiry:  
> dma status == 0x64
> Apr 11 04:24:29 xenserver kernel: [277234.439860] NETDEV WATCHDOG:  
> peth0: transmit timed out
> Apr 11 04:24:30 xenserver kernel: [277234.456241] r8169: peth0: link up
> Apr 11 04:24:33 xenserver kernel: [277238.648044] hdc: DMA interrupt  
> recovery
> Apr 11 04:24:33 xenserver kernel: [277238.648058] hdc: lost interrupt
> Apr 11 04:24:41 xenserver kernel: [277246.439954] NETDEV WATCHDOG:  
> peth0: transmit timed out
> Apr 11 04:24:42 xenserver kernel: [277246.456300] r8169: peth0: link up
> Apr 11 04:24:42 xenserver kernel: [277246.633366] INFO: task  
> md0_raid1:2194 blocked for more than 120 seconds.
> Apr 11 04:24:42 xenserver kernel: [277246.633382] "echo 0 >  
> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> Apr 11 04:24:42 xenserver kernel: [277246.633394] md0_raid1     D  
> ffffffff80449f00     0  2194      2
> Apr 11 04:24:42 xenserver kernel: [277246.633408]  ffff8800e0169cc0  
> 0000000000000246 0000000000000000 ffff8800e2a2fac0
> Apr 11 04:24:42 xenserver kernel: [277246.633429]  ffff8800e2ad2480  
> ffffffff804fe460 ffff8800e2ad2700 00000000e02c2528
> Apr 11 04:24:42 xenserver kernel: [277246.633450]  00000000ffffffff  
> ffff880093da90c0 000000000000ffff 000000007470593f
> Apr 11 04:24:42 xenserver kernel: [277246.633465] Call Trace:
> Apr 11 04:24:42 xenserver kernel: [277246.634648]  [<ffffffff802ff7a6>]  
> generic_make_request+0x2fe/0x339
> Apr 11 04:24:42 xenserver kernel: [277246.634664]  [<ffffffffa0232413>]  
> :md_mod:md_super_wait+0xee/0x107
> Apr 11 04:24:42 xenserver kernel: [277246.634674]  [<ffffffff8023f4e5>]  
> autoremove_wake_function+0x0/0x2e
> Apr 11 04:24:42 xenserver kernel: [277246.634688]  [<ffffffffa02328a3>]  
> :md_mod:md_update_sb+0x2a0/0x38f
> Apr 11 04:24:42 xenserver kernel: [277246.634702]  [<ffffffffa0237016>]  
> :md_mod:md_check_recovery+0x1c9/0x504
> Apr 11 04:24:42 xenserver kernel: [277246.634714]  [<ffffffffa0246239>]  
> :raid1:raid1d+0x26/0xd73
> Apr 11 04:24:42 xenserver kernel: [277246.634724]  [<ffffffff804369b7>]  
> error_exit+0x0/0x69
> Apr 11 04:24:42 xenserver kernel: [277246.634734]  [<ffffffff804369b7>]  
> error_exit+0x0/0x69
> Apr 11 04:24:42 xenserver kernel: [277246.634744]  [<ffffffff80434efe>]  
> schedule_timeout+0x1e/0xad
> Apr 11 04:24:42 xenserver kernel: [277246.634754]  [<ffffffff8030dc91>]  
> __next_cpu+0x19/0x26
> Apr 11 04:24:42 xenserver kernel: [277246.634766]  [<ffffffffa02357d2>]  
> :md_mod:md_thread+0xd7/0xed
> Apr 11 04:24:42 xenserver kernel: [277246.634775]  [<ffffffff8023f4e5>]  
> autoremove_wake_function+0x0/0x2e
> Apr 11 04:24:42 xenserver kernel: [277246.634788]  [<ffffffffa02356fb>]  
> :md_mod:md_thread+0x0/0xed
> Apr 11 04:24:42 xenserver kernel: [277246.634796]  [<ffffffff8023f3b7>]  
> kthread+0x47/0x74
> Apr 11 04:24:42 xenserver kernel: [277246.634805]  [<ffffffff8022817c>]  
> schedule_tail+0x27/0x5c
> Apr 11 04:24:42 xenserver kernel: [277246.634814]  [<ffffffff8020be28>]  
> child_rip+0xa/0x12
> Apr 11 04:24:42 xenserver kernel: [277246.634825]  [<ffffffff8023f370>]  
> kthread+0x0/0x74
> Apr 11 04:24:42 xenserver kernel: [277246.634833]  [<ffffffff8020be1e>]  
> child_rip+0x0/0x12
> Apr 11 04:24:42 xenserver kernel: [277246.634841]
> Apr 11 04:24:42 xenserver kernel: [277246.634856] INFO: task  
> blkback.2.sda6:4518 blocked for more than 120 seconds.
> Apr 11 04:24:42 xenserver kernel: [277246.634867] "echo 0 >  
> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> Apr 11 04:24:42 xenserver kernel: [277246.634878] blkback.2.sda D  
> 00000000000000ff     0  4518      2
> Apr 11 04:24:42 xenserver kernel: [277246.634891]  ffff880093e999c0  
> 0000000000000246 0000000000000001 ffff8800e0169ee0
> Apr 11 04:24:42 xenserver kernel: [277246.634911]  ffff8800e2057600  
> ffff8800e2ad2480 ffff8800e2057880 0000000000000000
> Apr 11 04:24:42 xenserver kernel: [277246.634932]  ffff880093e99980  
> ffffffff8023f4ee ffff8800e23d3ec0 ffffffff802213e5
> Apr 11 04:24:42 xenserver kernel: [277246.634947] Call Trace:
> Apr 11 04:24:42 xenserver kernel: [277246.634961]  [<ffffffff8023f4ee>]  
> autoremove_wake_function+0x9/0x2e
> Apr 11 04:24:42 xenserver kernel: [277246.634970]  [<ffffffff802213e5>]  
> __wake_up_common+0x41/0x74
> Apr 11 04:24:42 xenserver kernel: [277246.634981]  [<ffffffff802d54cc>]  
> sysfs_find_dirent+0x1b/0x2f
> Apr 11 04:24:42 xenserver kernel: [277246.634994]  [<ffffffffa02364f3>]  
> :md_mod:md_write_start+0x160/0x178
> Apr 11 04:24:42 xenserver kernel: [277246.635003]  [<ffffffff8023f4e5>]  
> autoremove_wake_function+0x0/0x2e
> Apr 11 04:24:42 xenserver kernel: [277246.635016]  [<ffffffffa00f43f8>]  
> :dm_mod:__split_bio+0x363/0x374
> Apr 11 04:24:42 xenserver kernel: [277246.635026]  [<ffffffffa0246faf>]  
> :raid1:make_request+0x29/0x5b6
> Apr 11 04:24:42 xenserver kernel: [277246.635036]  [<ffffffff803117f7>]  
> __up_read+0x13/0x8a
> Apr 11 04:24:42 xenserver kernel: [277246.635046]  [<ffffffff802ff7a6>]  
> generic_make_request+0x2fe/0x339
> Apr 11 04:24:42 xenserver kernel: [277246.635056]  [<ffffffff80266983>]  
> mempool_alloc+0x24/0xda
> Apr 11 04:24:42 xenserver kernel: [277246.635067]  [<ffffffff80300a80>]  
> submit_bio+0xdb/0xe2
> Apr 11 04:24:42 xenserver kernel: [277246.635077]  [<ffffffff80386e68>]  
> dispatch_rw_block_io+0x5b9/0x65d
> Apr 11 04:24:42 xenserver kernel: [277246.635102]  [<ffffffff802ff83b>]  
> blk_unplug+0x5a/0x60
> Apr 11 04:24:42 xenserver kernel: [277246.635115]  [<ffffffffa00f5b19>]  
> :dm_mod:dm_table_unplug_all+0x2a/0x3d
> Apr 11 04:24:42 xenserver kernel: [277246.635125]  [<ffffffff8038735b>]  
> blkif_schedule+0x3c7/0x4dc
> Apr 11 04:24:42 xenserver kernel: [277246.635137]  [<ffffffff80386f94>]  
> blkif_schedule+0x0/0x4dc
> Apr 11 04:24:42 xenserver kernel: [277246.635146]  [<ffffffff8023f3b7>]  
> kthread+0x47/0x74
> Apr 11 04:24:42 xenserver kernel: [277246.635155]  [<ffffffff8022817c>]  
> schedule_tail+0x27/0x5c
> Apr 11 04:24:42 xenserver kernel: [277246.635163]  [<ffffffff8020be28>]  
> child_rip+0xa/0x12
> Apr 11 04:24:42 xenserver kernel: [277246.635174]  [<ffffffff8023f370>]  
> kthread+0x0/0x74
> Apr 11 04:24:42 xenserver kernel: [277246.635183]  [<ffffffff8020be1e>]  
> child_rip+0x0/0x12
> Apr 11 04:24:42 xenserver kernel: [277246.635191]
> Apr 11 04:24:43 xenserver kernel: [277247.601345] ata1.00: qc timeout  
> (cmd 0xec)
> Apr 11 04:24:43 xenserver kernel: [277247.601366] ata1.00: failed to  
> IDENTIFY (I/O error, err_mask=0x4)
> Apr 11 04:24:43 xenserver kernel: [277247.601375] ata1.00: revalidation  
> failed (errno=-5)
> Apr 11 04:24:43 xenserver kernel: [277247.601383] ata1.00: disabled
> Apr 11 04:24:43 xenserver kernel: [277247.601396] ata2.00: qc timeout  
> (cmd 0xec)
> Apr 11 04:24:43 xenserver kernel: [277247.601410] ata2.00: failed to  
> IDENTIFY (I/O error, err_mask=0x4)
> Apr 11 04:24:43 xenserver kernel: [277247.601418] ata2.00: revalidation  
> failed (errno=-5)
> Apr 11 04:24:43 xenserver kernel: [277247.601425] ata2.00: disabled
> Apr 11 04:24:43 xenserver kernel: [277248.105346] ata2: hard resetting link
> Apr 11 04:24:43 xenserver kernel: [277248.105372] ata1: hard resetting link
> Apr 11 04:24:43 xenserver kernel: [277248.589302] ata1: softreset failed  
> (device not ready)
> Apr 11 04:24:43 xenserver kernel: [277248.589319] ata1: failed due to HW  
> bug, retry pmp=0
> Apr 11 04:24:43 xenserver kernel: [277248.589341] ata2: softreset failed  
> (device not ready)
> Apr 11 04:24:43 xenserver kernel: [277248.589350] ata2: failed due to HW  
> bug, retry pmp=0
> Apr 11 04:24:44 xenserver kernel: [277248.753334] ata2: SATA link up 3.0  
> Gbps (SStatus 123 SControl 300)
> Apr 11 04:24:44 xenserver kernel: [277248.753366] ata2: EH complete
> Apr 11 04:24:44 xenserver kernel: [277248.753384] sd 1:0:0:0: [sdb]  
> Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
> Apr 11 04:24:44 xenserver kernel: [277248.753399] end_request: I/O  
> error, dev sdb, sector 1953519935
> Apr 11 04:24:44 xenserver kernel: [277248.753410] end_request: I/O  
> error, dev sdb, sector 1953519935
> Apr 11 04:24:44 xenserver kernel: [277248.753421] md: super_written gets  
> error=-5, uptodate=0
> Apr 11 04:24:44 xenserver kernel: [277248.753431] raid1: Disk failure on  
> sdb1, disabling device.
> Apr 11 04:24:44 xenserver kernel: [277248.753432] raid1: Operation  
> continuing on 1 devices.
> Apr 11 04:24:44 xenserver kernel: [277248.753466] ata1: SATA link up 3.0  
> Gbps (SStatus 123 SControl 300)
> Apr 11 04:24:44 xenserver kernel: [277248.753479] ata1: EH complete
> Apr 11 04:24:44 xenserver kernel: [277248.753490] sd 0:0:0:0: [sda]  
> Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
> Apr 11 04:24:44 xenserver kernel: [277248.753505] end_request: I/O  
> error, dev sda, sector 1953519935
> Apr 11 04:24:44 xenserver kernel: [277248.753515] end_request: I/O  
> error, dev sda, sector 1953519935
> Apr 11 04:24:44 xenserver kernel: [277248.753524] md: super_written gets  
> error=-5, uptodate=0
> Apr 11 04:24:44 xenserver kernel: [277248.753551] sd 0:0:0:0: [sda]  
> Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
> Apr 11 04:24:44 xenserver kernel: [277248.753566] end_request: I/O  
> error, dev sda, sector 1953519935
> Apr 11 04:24:44 xenserver kernel: [277248.753576] end_request: I/O  
> error, dev sda, sector 1953519935
> Apr 11 04:24:44 xenserver kernel: [277248.753585] md: super_written gets  
> error=-5, uptodate=0
> Apr 11 04:24:44 xenserver kernel: [277248.753596] RAID1 conf printout:
> Apr 11 04:24:44 xenserver kernel: [277248.753603]  --- wd:1 rd:2
> Apr 11 04:24:44 xenserver kernel: [277248.753610]  disk 0, wo:0, o:1,  
> dev:sda1
> Apr 11 04:24:44 xenserver kernel: [277248.753618]  disk 1, wo:1, o:0,  
> dev:sdb1
> Apr 11 04:24:44 xenserver kernel: [277248.765416] RAID1 conf printout:
> Apr 11 04:24:44 xenserver kernel: [277248.765431]  --- wd:1 rd:2
>
>
> From lspci -v:
>
> 00:00.0 Host bridge: Advanced Micro Devices [AMD] RS780 Host Bridge
>        Subsystem: Advanced Micro Devices [AMD] RS780 Host Bridge
>        Flags: bus master, 66MHz, medium devsel, latency 0
>        Capabilities: [c4] HyperTransport: Slave or Primary Interface
>        Capabilities: [54] HyperTransport: UnitID Clumping
>        Capabilities: [40] HyperTransport: Retry Mode
>        Capabilities: [9c] HyperTransport: #1a
>        Capabilities: [f8] HyperTransport: #1c
>
> 00:01.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge  
> (int gfx) (prog-if 00 [Normal decode])
>        Flags: bus master, 66MHz, medium devsel, latency 64
>        Bus: primary=00, secondary=01, subordinate=01, sec-latency=64
>        I/O behind bridge: 0000b000-0000bfff
>        Memory behind bridge: fe700000-fe8fffff
>        Prefetchable memory behind bridge: 00000000c0000000-00000000cfffffff
>        Capabilities: [44] HyperTransport: MSI Mapping Enable+ Fixed+
>        Capabilities: [b0] Subsystem: Advanced Micro Devices [AMD] RS780  
> PCI to PCI bridge (int gfx)
>        Kernel modules: shpchp
>
> 00:04.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge  
> (PCIE port 0) (prog-if 00 [Normal decode])
>        Flags: bus master, fast devsel, latency 0
>        Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
>        I/O behind bridge: 0000c000-0000cfff
>        Memory behind bridge: fe900000-fe9fffff
>        Prefetchable memory behind bridge: 00000000d0000000-00000000d00fffff
>        Capabilities: [50] Power Management version 3
>        Capabilities: [58] Express Root Port (Slot+), MSI 00
>        Capabilities: [a0] Message Signalled Interrupts: Mask- 64bit-  
> Queue=0/0 Enable-
>        Capabilities: [b0] Subsystem: Advanced Micro Devices [AMD] Device 
> 9600
>        Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
>        Capabilities: [100] Vendor Specific Information <?>
>        Capabilities: [110] Virtual Channel <?>
>        Kernel driver in use: pcieport-driver
>        Kernel modules: shpchp
>
> 00:07.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge  
> (PCIE port 3) (prog-if 00 [Normal decode])
>        Flags: bus master, fast devsel, latency 0
>        Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
>        I/O behind bridge: 0000d000-0000dfff
>        Memory behind bridge: fea00000-feafffff
>        Prefetchable memory behind bridge: 00000000dff00000-00000000dfffffff
>        Capabilities: [50] Power Management version 3
>        Capabilities: [58] Express Root Port (Slot+), MSI 00
>        Capabilities: [a0] Message Signalled Interrupts: Mask- 64bit-  
> Queue=0/0 Enable-
>        Capabilities: [b0] Subsystem: Advanced Micro Devices [AMD] Device 
> 9600
>        Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
>        Capabilities: [100] Vendor Specific Information <?>
>        Capabilities: [110] Virtual Channel <?>
>        Kernel driver in use: pcieport-driver
>        Kernel modules: shpchp
>
> 00:11.0 SATA controller: ATI Technologies Inc SB700/SB800 SATA  
> Controller [IDE mode] (prog-if 01 [AHCI 1.0])
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 22
>        I/O ports at a000 [size=8]
>        I/O ports at 9000 [size=4]
>        I/O ports at 8000 [size=8]
>        I/O ports at 7000 [size=4]
>        I/O ports at 6000 [size=16]
>        Memory at fe6ff800 (32-bit, non-prefetchable) [size=1K]
>        Capabilities: [60] Power Management version 2
>        Capabilities: [70] SATA HBA <?>
>        Kernel driver in use: ahci
>        Kernel modules: ahci
>
> 00:12.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0  
> Controller (prog-if 10 [OHCI])
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 16
>        Memory at fe6fe000 (32-bit, non-prefetchable) [size=4K]
>        Kernel driver in use: ohci_hcd
>        Kernel modules: ohci-hcd
>
> 00:12.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller  
> (prog-if 10 [OHCI])
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 16
>        Memory at fe6fd000 (32-bit, non-prefetchable) [size=4K]
>        Kernel driver in use: ohci_hcd
>        Kernel modules: ohci-hcd
>
> 00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI  
> Controller (prog-if 20 [EHCI])
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 17
>        Memory at fe6ff000 (32-bit, non-prefetchable) [size=256]
>        Capabilities: [c0] Power Management version 2
>        Capabilities: [e4] Debug port: BAR=1 offset=00e0
>        Kernel driver in use: ehci_hcd
>        Kernel modules: ehci-hcd
>
> 00:13.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0  
> Controller (prog-if 10 [OHCI])
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 18
>        Memory at fe6fc000 (32-bit, non-prefetchable) [size=4K]
>        Kernel driver in use: ohci_hcd
>        Kernel modules: ohci-hcd
>
> 00:13.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller  
> (prog-if 10 [OHCI])
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 18
>        Memory at fe6fb000 (32-bit, non-prefetchable) [size=4K]
>        Kernel driver in use: ohci_hcd
>        Kernel modules: ohci-hcd
>
> 00:13.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI  
> Controller (prog-if 20 [EHCI])
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 19
>        Memory at fe6fa800 (32-bit, non-prefetchable) [size=256]
>        Capabilities: [c0] Power Management version 2
>        Capabilities: [e4] Debug port: BAR=1 offset=00e0
>        Kernel driver in use: ehci_hcd
>        Kernel modules: ehci-hcd
>
> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3a)
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: 66MHz, medium devsel
>        Capabilities: [b0] HyperTransport: MSI Mapping Enable- Fixed+
>        Kernel driver in use: piix4_smbus
>        Kernel modules: i2c-piix4
>
> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller  
> (prog-if 8a [Master SecP PriP])
>        Subsystem: ATI Technologies Inc SB700/SB800 IDE Controller
>        Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 16
>        I/O ports at 01f0 [size=8]
>        I/O ports at 03f4 [size=1]
>        I/O ports at 0170 [size=8]
>        I/O ports at 0374 [size=1]
>        I/O ports at ff00 [size=16]
>        Capabilities: [70] Message Signalled Interrupts: Mask- 64bit-  
> Queue=0/1 Enable-
>        Kernel driver in use: ATIIXP_IDE
>        Kernel modules: atiixp, ide-pci-generic, ata_generic
>
> 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA)
>        Subsystem: Biostar Microtech Int'l Corp Device 8218
>        Flags: bus master, slow devsel, latency 64, IRQ 16
>        Memory at fe6f4000 (64-bit, non-prefetchable) [size=16K]
>        Capabilities: [50] Power Management version 2
>        Kernel driver in use: HDA Intel
>        Kernel modules: snd-hda-intel
>
> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: bus master, 66MHz, medium devsel, latency 0
>
> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge  
> (prog-if 01 [Subtractive decode])
>        Flags: bus master, 66MHz, medium devsel, latency 64
>        Bus: primary=00, secondary=04, subordinate=05, sec-latency=64
>        I/O behind bridge: 0000e000-0000efff
>        Memory behind bridge: feb00000-febfffff
>        Prefetchable memory behind bridge: f0000000-fbffffff
>
> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2  
> Controller (prog-if 10 [OHCI])
>        Subsystem: Biostar Microtech Int'l Corp Device 3700
>        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 18
>        Memory at fe6f9000 (32-bit, non-prefetchable) [size=4K]
>        Kernel driver in use: ohci_hcd
>        Kernel modules: ohci-hcd
>
> 00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron,  
> Athlon64, Sempron] HyperTransport Configuration
>        Flags: fast devsel
>        Capabilities: [80] HyperTransport: Host or Secondary Interface
>
> 00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron,  
> Athlon64, Sempron] Address Map
>        Flags: fast devsel
>
> 00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron,  
> Athlon64, Sempron] DRAM Controller
>        Flags: fast devsel
>
> 00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron,  
> Athlon64, Sempron] Miscellaneous Control
>        Flags: fast devsel
>        Capabilities: [f0] Secure device <?>
>
> 00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron,  
> Athlon64, Sempron] Link Control
>        Flags: fast devsel
>
> 01:05.0 VGA compatible controller: ATI Technologies Inc Radeon HD 3300  
> Graphics (prog-if 00 [VGA controller])
>        Subsystem: Biostar Microtech Int'l Corp Device 0217
>        Flags: bus master, fast devsel, latency 0, IRQ 7
>        Memory at c0000000 (32-bit, prefetchable) [size=256M]
>        I/O ports at b000 [size=256]
>        Memory at fe8f0000 (32-bit, non-prefetchable) [size=64K]
>        Memory at fe700000 (32-bit, non-prefetchable) [size=1M]
>        Capabilities: [50] Power Management version 3
>        Capabilities: [a0] Message Signalled Interrupts: Mask- 64bit+  
> Queue=0/0 Enable-
>
> 02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.  
> RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01)
>        Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI  
> Express Gigabit Ethernet controller
>        Flags: bus master, fast devsel, latency 0, IRQ 16
>        I/O ports at c800 [size=256]
>        Memory at fe9ff000 (64-bit, non-prefetchable) [size=4K]
>        [virtual] Expansion ROM at d0000000 [disabled] [size=128K]
>        Capabilities: [40] Power Management version 2
>        Capabilities: [48] Vital Product Data <?>
>        Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+  
> Queue=0/1 Enable-
>        Capabilities: [60] Express Endpoint, MSI 00
>        Capabilities: [84] Vendor Specific Information <?>
>        Capabilities: [100] Advanced Error Reporting <?>
>        Capabilities: [12c] Virtual Channel <?>
>        Capabilities: [148] Device Serial Number 00-e0-4c-68-00-00-00-fd
>        Capabilities: [154] Power Budgeting <?>
>        Kernel driver in use: pciback
>        Kernel modules: r8169
>
> 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.  
> RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03)
>        Subsystem: Biostar Microtech Int'l Corp Device 2309
>        Flags: bus master, fast devsel, latency 0, IRQ 19
>        I/O ports at d800 [size=256]
>        Memory at dffff000 (64-bit, prefetchable) [size=4K]
>        Memory at dfff8000 (64-bit, prefetchable) [size=16K]
>        Expansion ROM at feae0000 [disabled] [size=128K]
>        Capabilities: [40] Power Management version 3
>        Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+  
> Queue=0/0 Enable-
>        Capabilities: [70] Express Endpoint, MSI 01
>        Capabilities: [ac] MSI-X: Enable- Mask- TabSize=4
>        Capabilities: [cc] Vital Product Data <?>
>        Capabilities: [100] Advanced Error Reporting <?>
>        Capabilities: [140] Virtual Channel <?>
>        Capabilities: [160] Device Serial Number 00-e0-4c-68-00-00-02-61
>        Kernel driver in use: r8169
>        Kernel modules: r8169
>
> 04:05.0 PCI bridge: Pericom Semiconductor Device 8140 (prog-if 00  
> [Normal decode])
>        Flags: bus master, 66MHz, medium devsel, latency 64
>        Bus: primary=04, secondary=05, subordinate=05, sec-latency=64
>        Prefetchable memory behind bridge: 00000000f4000000-00000000fbffffff
>        Capabilities: [80] Power Management version 2
>        Capabilities: [90] CompactPCI hot-swap <?>
>        Kernel modules: shpchp
>
> 04:06.0 Multimedia video controller: Internext Compression Inc iTVC16  
> (CX23416) MPEG-2 Encoder (rev 01)
>        Subsystem: Hauppauge computer works Inc. WinTV PVR 250
>        Flags: medium devsel, IRQ 21
>        Memory at f0000000 (32-bit, prefetchable) [disabled] [size=64M]
>        Capabilities: [44] Power Management version 2
>        Kernel driver in use: pciback
>        Kernel modules: ivtv
>
> 04:07.0 Ethernet controller: D-Link System Inc DGE-530T Gigabit Ethernet  
> Adapter (rev 11) (rev 11)
>        Subsystem: D-Link System Inc DGE-530T Gigabit Ethernet Adapter  
> (rev 11)
>        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 22
>        Memory at febfc000 (32-bit, non-prefetchable) [size=16K]
>        I/O ports at e800 [size=256]
>        Expansion ROM at feb00000 [disabled] [size=128K]
>        Capabilities: [48] Power Management version 2
>        Capabilities: [50] Vital Product Data <?>
>        Kernel driver in use: pciback
>        Kernel modules: skge
>
> 05:08.0 Multimedia video controller: Internext Compression Inc iTVC16  
> (CX23416) MPEG-2 Encoder (rev 01)
>        Subsystem: Hauppauge computer works Inc. WinTV PVR 500 (1st unit)
>        Flags: medium devsel, IRQ 20
>        Memory at f8000000 (32-bit, prefetchable) [disabled] [size=64M]
>        Capabilities: [44] Power Management version 2
>        Kernel driver in use: pciback
>        Kernel modules: ivtv
>
> 05:09.0 Multimedia video controller: Internext Compression Inc iTVC16  
> (CX23416) MPEG-2 Encoder (rev 01)
>        Subsystem: Hauppauge computer works Inc. WinTV PVR 500 (2nd unit)
>        Flags: medium devsel, IRQ 21
>        Memory at f4000000 (32-bit, prefetchable) [disabled] [size=64M]
>        Capabilities: [44] Power Management version 2
>        Kernel driver in use: pciback
>        Kernel modules: ivtv
>
>
> From /proc/interrupts:
>
>           CPU0       CPU1       CPU2       CPU3       1:         58       
>    0          0          0  Phys-irq-level     i8042
>  8:          0          0          0          0  Phys-irq-level     rtc0
>  9:          0          0          0          0  Phys-irq-level     acpi
> 12:          4          0          0          0  Phys-irq-level     i8042
> 14:          0          0          0          0  Phys-irq-level     ide0
> 15:      59700          0          0          0  Phys-irq-level     ide1
> 16:     339121          0          0          0  Phys-irq-level      
> ohci_hcd:usb1, ohci_hcd:usb2, HDA Intel
> 17:          0          0          0          0  Phys-irq-level      
> ehci_hcd:usb5
> 18:          3          0          0          0  Phys-irq-level      
> ohci_hcd:usb3, ohci_hcd:usb4, ohci_hcd:usb7
> 19:     174793          0          0          0  Phys-irq-level      
> ehci_hcd:usb6, peth0
> 22:     645023          0          0          0  Phys-irq-level     ahci
> 256:     450281          0          0          0  Dynamic-irq-level      
> timer0
> 257:       4704          0          0          0  Dynamic-irq-level      
> resched0
> 258:         43          0          0          0  Dynamic-irq-level      
> callfunc0
> 259:        731          0          0          0  Dynamic-irq-level      
> spinlock0
> 260:          0       2805          0          0  Dynamic-irq-level      
> resched1
> 261:          0        189          0          0  Dynamic-irq-level      
> callfunc1
> 262:          0        543          0          0  Dynamic-irq-level      
> spinlock1
> 263:          0      58565          0          0  Dynamic-irq-level      
> timer1
> 264:          0          0       4449          0  Dynamic-irq-level      
> resched2
> 265:          0          0        177          0  Dynamic-irq-level      
> callfunc2
> 266:          0          0        570          0  Dynamic-irq-level      
> spinlock2
> 267:          0          0      43025          0  Dynamic-irq-level      
> timer2
> 268:          0          0          0       3650  Dynamic-irq-level      
> resched3
> 269:          0          0          0        176  Dynamic-irq-level      
> callfunc3
> 270:          0          0          0        652  Dynamic-irq-level      
> spinlock3
> 271:          0          0          0      24529  Dynamic-irq-level      
> timer3
> 272:       5815          0          0          0  Dynamic-irq-level      
> xenbus
> 273:          0          0          0          0  Dynamic-irq-level      
> console
> 274:          0          0          0          0  Dynamic-irq-level      
> suspend
> 275:        406          0          0          0  Dynamic-irq-level      
> pciback
> 276:         36          0          0          0  Dynamic-irq-level      
> blkif-backend
> 277:        162          0          0          0  Dynamic-irq-level      
> blkif-backend
> 278:       1939          0          0          0  Dynamic-irq-level      
> blkif-backend
> 279:     178850          0          0          0  Dynamic-irq-level      
> vif1.0
> 280:      28670          0          0          0  Dynamic-irq-level      
> vif1.1
> 281:       4372          0          0          0  Dynamic-irq-level      
> blkif-backend
> 282:       1227          0          0          0  Dynamic-irq-level      
> blkif-backend
> 283:       8079          0          0          0  Dynamic-irq-level      
> blkif-backend
> 284:        177          0          0          0  Dynamic-irq-level      
> blkif-backend
> 285:        106          0          0          0  Dynamic-irq-level      
> blkif-backend
> 286:      43937          0          0          0  Dynamic-irq-level      
> blkif-backend
> 287:      28211          0          0          0  Dynamic-irq-level      
> vif2.0
> 288:       2950          0          0          0  Dynamic-irq-level      
> blkif-backend
> 289:       5669          0          0          0  Dynamic-irq-level      
> blkif-backend
> 290:         83          0          0          0  Dynamic-irq-level      
> blkif-backend
> 291:        148          0          0          0  Dynamic-irq-level      
> blkif-backend
> 292:         40          0          0          0  Dynamic-irq-level      
> blkif-backend
> 293:      98438          0          0          0  Dynamic-irq-level      
> blkif-backend
> 294:      64518          0          0          0  Dynamic-irq-level      
> vif3.0
> NMI:          0          0          0          0   Non-maskable interrupts
> RES:       4704       2805       4449       3650   Rescheduling interrupts
> CAL:         43        189        177        176   function call interrupts
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users