[Xen-users] PV HVM Linux drivers fail

Hi,

I don't see anyone having similar issues when running domU kernel in HVM. Hopefully it's just some very obvious stuff I missed.

I am using the 2.6.34-pvhvm-v6 branch from git://xenbits.xen.org/people/sstabellini/linux-pvhvm. I compiled the kernel with just about everything as modules, including platform-pci, xen-blkfront and xen-netfront.

When I use the kernel parameter xen_emul_unplug=ignore, both ata_piix and 8139cp drivers attach to the emulated devices and work just fine.

When I unplug either the nic or the ide-disk, the loading order of the 3 modules makes some difference. If I load platform-pci first before either of the frontend drivers, I get XENBUS Timeout. Note that loading the platform-pci module doesn't cause any error, it is loading the frontend drivers when I see this error: (This happens when I load the xen-netfront driver. platform-pci is loaded at 58s. I also trimmed the wait time from 300s to 30s)

[ 102.570177] Initialising Xen virtual ethernet driver.
[ 107.676025] XENBUS: Waiting for devices to initialise: 25s...20s...15s...10s...5s...0s...
[ 132.677038] XENBUS: Timeout connecting to device: device/vif/0 (local state 1, remote state 2)

So I have to load the frontend drivers before platform-pci, and none of this error like this occured. And because of that I cannot compile in all the three modules as they'll always be waiting to connect.

However I get some other error. When I unplug ide-disks, I get this:

<4>[   29.842072] ------------[ cut here ]------------
<4>[   29.842785] WARNING: at fs/sysfs/dir.c:451 sysfs_add_one+0xac/0xc0()
<4>[   29.843434] Hardware name: HVM domU
<4>[   29.844070] sysfs: cannot create duplicate filename '/block/xvda'
<4>[   29.844713] Modules linked in: 8139cp mii platform_pci xen_netfront xen_blkfront
<4>[   29.847272] Pid: 20, comm: xenwatch Not tainted 2.6.34-pvhvm #10
<4>[   29.847945] Call Trace:
<4>[   29.856027] [<ffffffff81146ebc>] ? sysfs_add_one+0xac/0xc0
<4>[   29.856722] [<ffffffff810424fa>] warn_slowpath_common+0x77/0x8f
<4>[   29.857372] [<ffffffff81042587>] warn_slowpath_fmt+0x64/0x66
<4>[   29.858015] [<ffffffff8106e6c8>] ? do_raw_spin_unlock+0x17/0x1b
<4>[   29.858722] [<ffffffff81146e32>] ? sysfs_add_one+0x22/0xc0
<4>[   29.859358] [<ffffffff812b3b94>] ? __mutex_lock_common+0x229/0x243
<4>[   29.860034] [<ffffffff810e433d>] ? __raw_local_irq_save+0x22/0x28
<4>[   29.860815] [<ffffffff81146d7a>] ? sysfs_pathname+0x37/0x3f
<4>[   29.861461] [<ffffffff81146d7a>] ? sysfs_pathname+0x37/0x3f
<4>[   29.862126] [<ffffffff81146ebc>] sysfs_add_one+0xac/0xc0
<4>[   29.862768] [<ffffffff811475f1>] create_dir+0x58/0x87
<4>[   29.863393] [<ffffffff810e433d>] ? __raw_local_irq_save+0x22/0x28
<4>[   29.864092] [<ffffffff81147658>] sysfs_create_dir+0x38/0x4f
<4>[   29.864833] [<ffffffff812b49f8>] ? _raw_spin_unlock+0x2d/0x38
<4>[   29.865525] [<ffffffff8119aa6d>] kobject_add_internal+0xdb/0x19b
<4>[   29.866186] [<ffffffff8119ac05>] kobject_add_varg+0x41/0x4e
<4>[   29.866858] [<ffffffff8119b0d0>] kobject_add+0x89/0x8b
<4>[   29.867590] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
<4>[   29.868280] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
<4>[   29.868936] [<ffffffff8119405c>] ? exact_lock+0x0/0x14
<4>[   29.869574] [<ffffffff810e71d8>] ? __kmalloc+0x106/0x118
<4>[   29.870232] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
<4>[   29.870866] [<ffffffff8119a98a>] ? kobject_get+0x1a/0x22
<4>[   29.871510] [<ffffffff81209ea0>] ? get_device+0x14/0x1a
<4>[   29.872209] [<ffffffff8120a451>] device_add+0xdc/0x60c
<4>[   29.872866] [<ffffffff8120ee8e>] ? kobj_map+0x68/0x12a
<4>[   29.873506] [<ffffffff8114082b>] register_disk+0x3c/0x11d
<4>[   29.874240] [<ffffffff81194db7>] add_disk+0xb8/0x119
<4>[   29.874922] [<ffffffffa000125b>] backend_changed+0x441/0x45b [xen_blkfront]
<4>[   29.875604] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
<4>[   29.876245] [<ffffffff811ec5ac>] otherend_changed+0x82/0x8b
<4>[   29.876864] [<ffffffff811eb7ea>] xenwatch_thread+0x118/0x152
<4>[   29.877481] [<ffffffff8105c560>] ? autoremove_wake_function+0x0/0x38
<4>[   29.878134] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
<4>[   29.878886] [<ffffffff8105c35e>] kthread+0x69/0x71
<4>[   29.879517] [<ffffffff8100aa04>] kernel_thread_helper+0x4/0x10
<4>[   29.880198] [<ffffffff8105c2f5>] ? kthread+0x0/0x71
<4>[   29.880923] [<ffffffff8100aa00>] ? kernel_thread_helper+0x0/0x10
<4>[   29.908017] ---[ end trace 3f99b54d0b8663be ]---
<3>[   29.909904] kobject_add_internal failed for xvda with -EEXIST, don't try to register things with the same name in the same directory.
<4>[   29.911103] Pid: 20, comm: xenwatch Tainted: G        W 2.6.34-pvhvm #10
<4>[   29.916378] Call Trace:
<4>[   29.917001] [<ffffffff8119a861>] ? kobject_put+0x47/0x4b
<4>[   29.917682] [<ffffffff8119aaeb>] kobject_add_internal+0x159/0x19b
<4>[   29.918389] [<ffffffff8119ac05>] kobject_add_varg+0x41/0x4e
<4>[   29.919124] [<ffffffff8119b0d0>] kobject_add+0x89/0x8b
<4>[   29.919834] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
<4>[   29.920474] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
<4>[   29.921191] [<ffffffff8119405c>] ? exact_lock+0x0/0x14
<4>[   29.921955] [<ffffffff810e71d8>] ? __kmalloc+0x106/0x118
<4>[   29.922690] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
<4>[   29.923578] [<ffffffff8119a98a>] ? kobject_get+0x1a/0x22
<4>[   29.924324] [<ffffffff81209ea0>] ? get_device+0x14/0x1a
<4>[   29.925178] [<ffffffff8120a451>] device_add+0xdc/0x60c
<4>[   29.925873] [<ffffffff8120ee8e>] ? kobj_map+0x68/0x12a
<4>[   29.926567] [<ffffffff8114082b>] register_disk+0x3c/0x11d
<4>[   29.927195] [<ffffffff81194db7>] add_disk+0xb8/0x119
<4>[   29.927821] [<ffffffffa000125b>] backend_changed+0x441/0x45b [xen_blkfront]
<4>[   29.928566] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
<4>[   29.929211] [<ffffffff811ec5ac>] otherend_changed+0x82/0x8b
<4>[   29.929855] [<ffffffff811eb7ea>] xenwatch_thread+0x118/0x152
<4>[   29.930490] [<ffffffff8105c560>] ? autoremove_wake_function+0x0/0x38
<4>[   29.931151] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
<4>[   29.931787] [<ffffffff8105c35e>] kthread+0x69/0x71
<4>[   29.932550] [<ffffffff8100aa04>] kernel_thread_helper+0x4/0x10
<4>[   29.933201] [<ffffffff8105c2f5>] ? kthread+0x0/0x71
<4>[   29.933834] [<ffffffff8100aa00>] ? kernel_thread_helper+0x0/0x10
<5>[   29.966562] SCSI subsystem initialized
<0>[   29.984037] ------------[ cut here ]------------
<2>[   29.984721] kernel BUG at fs/sysfs/group.c:65!
<0>[   29.985357] invalid opcode: 0000 [#1] PREEMPT SMP
<0>[   29.987151] last sysfs file: /sys/class/firmware/timeout
<4>[   29.987978] CPU 0
<4>[   29.988014] Modules linked in: sd_mod scsi_mod ext3 jbd mbcache 8139cp mii platform_pci xen_netfront xen_blkfront
<4>[   29.988014]
<4>[   29.988014] Pid: 20, comm: xenwatch Tainted: G        W 2.6.34-pvhvm #10 /HVM domU
<4>[   29.988014] RIP: 0010:[<ffffffff8114883a>] [<ffffffff8114883a>] internal_create_group+0x2a/0x127
<4>[   29.988014] RSP: 0018:ffff88001f97fd00 EFLAGS: 00010246
<4>[   29.988014] RAX: 00000000ffffffef RBX: ffff88001e200870 RCX: ffffffff8120a927
<4>[   29.988014] RDX: ffffffff81640280 RSI: 0000000000000000 RDI: ffff88001e200870
<4>[   29.988014] RBP: ffff88001f97fd40 R08: ffff88001f97e000 R09: ffff880001a14c20
<4>[   29.988014] R10: ffff88001f97fca0 R11: 0000000000000000 R12: ffff88001e0e88a8
<4>[   29.988014] R13: 000000000000000f R14: ffff88001e200860 R15: ffffffff81640280
<4>[   29.988014] FS: 0000000000000000(0000) GS:ffff880001a00000(0000) knlGS:0000000000000000
<4>[   29.988014] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<4>[   29.988014] CR2: 00007f151c82200f CR3: 000000001e7bc000 CR4: 00000000000006b0
<4>[   29.988014] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>[   29.988014] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>[   29.988014] Process xenwatch (pid: 20, threadinfo ffff88001f97e000, task ffff88001f975b80)
<0>[   29.988014] Stack:
<4>[   29.988014] 0000000000000000 0000000089700e79 ffff88001f97fd70 ffff88001e200800
<4>[   29.988014] <0> ffff88001e0e88a8 000000000000000f ffff88001e200860 0000000000000000
<4>[   29.988014] <0> ffff88001f97fd50 ffffffff81148958 ffff88001f97fd60 ffffffff810a39fe
<0>[   29.988014] Call Trace:
<4>[   29.988014] [<ffffffff81148958>] sysfs_create_group+0xe/0x12
<4>[   29.988014] [<ffffffff810a39fe>] blk_trace_init_sysfs+0x14/0x16
<4>[   29.988014] [<ffffffff8118ff11>] blk_register_queue+0x42/0xc8
<4>[   29.988014] [<ffffffff81194dbf>] add_disk+0xc0/0x119
<4>[   29.988014] [<ffffffffa000125b>] backend_changed+0x441/0x45b [xen_blkfront]
<4>[   29.988014] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
<4>[   29.988014] [<ffffffff811ec5ac>] otherend_changed+0x82/0x8b
<4>[   29.988014] [<ffffffff811eb7ea>] xenwatch_thread+0x118/0x152
<4>[   29.988014] [<ffffffff8105c560>] ? autoremove_wake_function+0x0/0x38
<4>[   29.988014] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
<4>[   29.988014] [<ffffffff8105c35e>] kthread+0x69/0x71
<4>[   29.988014] [<ffffffff8100aa04>] kernel_thread_helper+0x4/0x10
<4>[   29.988014] [<ffffffff8105c2f5>] ? kthread+0x0/0x71
<4>[   29.988014] [<ffffffff8100aa00>] ? kernel_thread_helper+0x0/0x10
<0>[   29.988014] Code: c3 55 48 89 e5 41 57 49 89 d7 41 56 41 55 41 54 53 48 89 fb 48 83 ec 18 48 85 ff 89 75 c0 74 0b 85 f6 75 11 48 83 7f 30 00 75 04 <0f> 0b eb fe 83 7d c0 00 74 11 48 83 7b 30 00 41 bd ea ff ff ff
<1>[   29.988014] RIP [<ffffffff8114883a>] internal_create_group+0x2a/0x127
<4>[   29.988014] RSP <ffff88001f97fd00>
<4>[   30.132020] ---[ end trace 3f99b54d0b8663bf ]---

Then when I do an fdisk on block device 202:0, I get this:

<4>[ 223.840262] ------------[ cut here ]------------
<4>[ 223.841061] WARNING: at fs/fs-writeback.c:1105 __mark_inode_dirty+0xed/0x130()
<4>[ 223.842008] Hardware name: HVM domU
<4>[ 223.842585] Modules linked in: ata_piix libata sd_mod scsi_mod ext3 jbd mbcache 8139cp mii platform_pci xen_netfront xen_blkfront
<4>[ 223.847533] Pid: 875, comm: busybox Tainted: G      D W 2.6.34-pvhvm #10
<4>[ 223.848259] Call Trace:
<4>[ 223.848994] [<ffffffff8110e33a>] ? __mark_inode_dirty+0xed/0x130
<4>[ 223.849761] [<ffffffff810424fa>] warn_slowpath_common+0x77/0x8f
<4>[ 223.850651] [<ffffffff81042521>] warn_slowpath_null+0xf/0x11
<4>[ 223.851509] [<ffffffff8110e33a>] __mark_inode_dirty+0xed/0x130
<4>[ 223.852195] [<ffffffff810b55ac>] ? sync_page_killable+0x0/0x3e
<4>[ 223.853449] [<ffffffff811041cd>] mark_inode_dirty_sync+0xe/0x10
<4>[ 223.854260] [<ffffffff81105554>] touch_atime+0xfb/0x120
<4>[ 223.855144] [<ffffffff810b41a6>] file_accessed+0x17/0x19
<4>[ 223.855956] [<ffffffff810b5ac4>] generic_file_aio_read+0x4da/0x53e
<4>[ 223.856686] [<ffffffff810f27c7>] do_sync_read+0xc2/0x106
<4>[ 223.857307] [<ffffffff810e433d>] ? __raw_local_irq_save+0x22/0x28
<4>[ 223.858023] [<ffffffff8115aca4>] ? security_file_permission+0x11/0x13
<4>[ 223.858993] [<ffffffff810f2e9d>] vfs_read+0xa8/0x102
<4>[ 223.859658] [<ffffffff810f313f>] sys_read+0x47/0x6d
<4>[ 223.860379] [<ffffffff81009c02>] system_call_fastpath+0x16/0x1b
<4>[ 223.861296] ---[ end trace 3f99b54d0b8663c0 ]---
<3>[ 223.861934] bdi-block not registered

Reading from the block device returns some random data: (here I am doing an fdisk on the block device 202:0), and it's returning the wrong disk size. The virtual disk is 15GiB.

Disk /dev/xvda: 4393 MB, 4393723904 bytes
255 heads, 63 sectors/track, 534 cylinders, total 8581492 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/xvda doesn't contain a valid partition table

If I unplug the nics, there is no error,

[   29.738321] Initialising Xen virtual ethernet driver.
[   29.748844]   alloc irq_desc for 28 on node -1
[   29.748847]   alloc kstat_irqs on node -1
[   29.748866] xen-platform-pci 0000:00:03.0: PCI INT A -> GSI 28 (level, low) -> IRQ 28
[   29.750216] Grant table initialized
[   29.753307] vif vif-0: 2 parsing device/vif/0/mac

8130cp, as expected, is not attached to eth0. However, it seems xen-netfront cannot get the MAC address of eth0:

eth0      Link encap:Ethernet HWaddr 00:00:00:00:00:00
          BROADCAST MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

Is it something very obvious that I missed?

Thanks!

Leo

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

WARNING - OLD ARCHIVES

xen-devel

[Xen-users] PV HVM Linux drivers fail