WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: [RFC PATCH linux-2.6.18-xen] pciback: clean up (MSI-X ve

To: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] Re: [RFC PATCH linux-2.6.18-xen] pciback: clean up (MSI-X vec, entrynr) list when resetting PCI device
From: Laszlo Ersek <lersek@xxxxxxxxxx>
Date: Wed, 01 Jun 2011 13:02:02 +0200
Cc: Paolo Bonzini <pbonzini@xxxxxxxxxx>, konrad.wilk@xxxxxxxxxx
Delivery-date: Wed, 01 Jun 2011 04:03:55 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4DE60EF8.5060902@xxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <4DE60EF8.5060902@xxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110428 Fedora/3.1.10-1.fc14 Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.10
(Perhaps it's best to write a separate mail about this.)

On 06/01/11 12:05, Laszlo Ersek wrote:
> Hi,
> 
> this is more of an RFC than a patch now for linux-2.6.18-xen. Describing the 
> situation captured in RHBZ#688673, in a nutshell:
> 
> - let's say we have an Intel 82576 card (igb),
> - the card exports virtual functions (igbvf),
> - one virtual function is passed through to a PV guest,
> - the igbvf driver, co-operating with pcifront, pciback, and the hypervisor, 
> sets up three MSI-X vectors for the virtual function PCI device,
> - when the domU is shut down, the MSI-X vectors are "leaked" in dom0, because 
> nobody ever reaches dom0's pci_disable_msix() / msi_remove_pci_irq_vectors() 
> during shutdown,
> - therefore configuration of the VF during the next boot of such a guest 
> doesn't succeed (dom0 thinks, based on its stale list, that MSI-X vectors are 
> already allocated/mapped for the device)

I tried to install a Fedora 15 PV guest as well (the host/hv was RHEL-5 
kernel-xen, -264 build). Surprisingly, when the igbvf driver called 
pci_enable_msix() in the PV guest at initialization time, it bugged out:

$ addr2line -p -f -i -e 
/usr/lib/debug/lib/modules/2.6.38.6-26.rc1.fc15.x86_64/vmlinux 
<<<ffffffff8125658d
readl at 
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/arch/x86/include/asm/io.h:58
 (inlined by) msix_program_entries at 
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/drivers/pci/msi.c:527
 (inlined by) msix_capability_init at 
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/drivers/pci/msi.c:578
 (inlined by) pci_enable_msix at 
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/drivers/pci/msi.c:806
 (inlined by) pci_enable_msix at 
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/drivers/pci/msi.c:773

Thanks,
lacos

[168711.841672] BUG: unable to handle kernel paging request at ffffc90001a9600c
[168711.841984] IP: [<ffffffff8125658d>] pci_enable_msix+0x270/0x354
[168711.842343] PGD 9c838067 PUD 9c839067 PMD 3216067 PTE 0
[168711.842675] Oops: 0000 [#1] SMP [168711.843016] last sysfs file: 
/sys/devices/virtual/block/dm-0/dev
[168711.843486] CPU 0 [168711.843507] Modules linked in: igbvf(+) joydev 
xen_pcifront xen_netfront ipv6 xen_blkfront
[168711.844504] [168711.845030] Pid: 449, comm: modprobe Not tainted 
2.6.38.6-26.rc1.fc15.x86_64 #1  [168711.845722] RIP: e030:[<ffffffff8125658d>]  
[<ffffffff8125658d>] pci_enable_msix+0x270/0x354
[168711.846440] RSP: e02b:ffff880098b4bc58  EFLAGS: 00010286
[168711.847172] RAX: ffffc90001a9600c RBX: ffff880003304000 RCX: 
0000000000000006
[168711.847960] RDX: 0000000000000006 RSI: ffff88009c800200 RDI: 
ffff88009c800200
[168711.848775] RBP: ffff880098b4bcc8 R08: ffff88009c80c000 R09: 
0000000000000002
[168711.849634] R10: ffff880098361400 R11: 0000000000000003 R12: 
ffff880002ceac80
[168711.850531] R13: ffff880096278280 R14: 0000000000000000 R15: 
ffff8800033048f0
[168711.851452] FS:  00007f2ee5360720(0000) GS:ffff88009fdf5000(0000) 
knlGS:0000000000000000
[168711.852434] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[168711.853454] CR2: ffffc90001a9600c CR3: 00000000033ce000 CR4: 
0000000000002620
[168711.854529] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
0000000000000000
[168711.855638] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
0000000000000000
[168711.856776] Process modprobe (pid: 449, threadinfo ffff880098b4a000, task 
ffff880003acae40)
[168711.857954] Stack:
[168711.859139]  ffff880096278280 0000000000000002 0000000000000000 
ffff8800033048f0
[168711.860445]  0000007298b4bcc8 ffffc9000000000c ffff880000000008 
c002ffff00000000
[168711.861777]  ffff880003304000 ffff880098350000 ffff880003304000 
ffff880003304090
[168711.863149] Call Trace:
[168711.864527]  [<ffffffffa0083400>] igbvf_probe+0x308/0x85b [igbvf]
[168711.865978]  [<ffffffff81006c3f>] ? xen_restore_fl_direct_end+0x0/0x1
[168711.867469]  [<ffffffff81080b0f>] ? arch_local_irq_restore+0xb/0xd
[168711.868984]  [<ffffffff812463ce>] local_pci_probe+0x44/0x75
[168711.870531]  [<ffffffff81246e60>] pci_device_probe+0xd0/0xff
[168711.872170]  [<ffffffff812e6af9>] driver_probe_device+0x124/0x1ff
[168711.873804]  [<ffffffff812e6c2e>] __driver_attach+0x5a/0x7e
[168711.875436]  [<ffffffff812e6bd4>] ? __driver_attach+0x0/0x7e
[168711.875441]  [<ffffffff812e5b9a>] bus_for_each_dev+0x53/0x89
[168711.875446]  [<ffffffff812e6706>] driver_attach+0x1e/0x20
[168711.875455]  [<ffffffff812e6318>] bus_add_driver+0xe2/0x235
[168711.875462]  [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875468]  [<ffffffff812e6e17>] driver_register+0x98/0x105
[168711.875474]  [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875480]  [<ffffffff81247723>] __pci_register_driver+0x56/0xc3
[168711.875486]  [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875493]  [<ffffffffa008904f>] igbvf_init_module+0x4f/0x51 [igbvf]
[168711.875500]  [<ffffffff810020a4>] do_one_initcall+0x7f/0x137
[168711.875506]  [<ffffffff810852dd>] sys_init_module+0xa6/0x1e4
[168711.875511]  [<ffffffff81009bc2>] system_call_fastpath+0x16/0x1b
[168711.875514] Code: 00 0f b7 42 04 c1 e0 04 83 c0 0c 89 45 b8 41 8b 44 24 0c 
89 02 41 8b 7c 24 0c 89 4d a0 e8 d0 80 e5 ff 48 63 45 b8 49 03 44 24 20 <8b> 00 
be 01 00 00 00 41 89 44 24 08 4c 89 e7 e8 c1 f2 ff ff 4d [168711.875552] RIP  
[<ffffffff8125658d>] pci_enable_msix+0x270/0x354
[168711.875558]  RSP <ffff880098b4bc58>
[168711.875560] CR2: ffffc90001a9600c
[168711.875564] ---[ end trace c82c0d02bae884a2 ]---
[168711.875570] BUG: sleeping function called from invalid context at 
kernel/rwsem.c:21
[168711.875573] in_atomic(): 0, irqs_disabled(): 1, pid: 449, name: modprobe
[168711.875577] Pid: 449, comm: modprobe Tainted: G      D     
2.6.38.6-26.rc1.fc15.x86_64 #1
[168711.875580] Call Trace:
[168711.875586]  [<ffffffff81047d33>] __might_sleep+0xeb/0xf0
[168711.875593]  [<ffffffff81474eee>] down_read+0x21/0x38
[168711.875598]  [<ffffffff8108be35>] acct_collect+0x4a/0x182
[168711.875603]  [<ffffffff81058627>] do_exit+0x216/0x732
[168711.875608]  [<ffffffff8147588c>] ? _raw_spin_unlock_irqrestore+0x17/0x19
[168711.875613]  [<ffffffff81006c52>] ? check_events+0x12/0x20
[168711.875618]  [<ffffffff81476b8e>] oops_end+0xbc/0xc5
[168711.875623]  [<ffffffff8146c0b4>] no_context+0x203/0x212
[168711.875628]  [<ffffffff81004415>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[168711.875633]  [<ffffffff8146c257>] __bad_area_nosemaphore+0x194/0x1b7
[168711.875637]  [<ffffffff8146b8e4>] ? pmd_val+0x10/0x12
[168711.875642]  [<ffffffff8146b945>] ? pte_offset_kernel+0x19/0x3f
[168711.875647]  [<ffffffff8146c28d>] bad_area_nosemaphore+0x13/0x15
[168711.875651]  [<ffffffff81478c5d>] do_page_fault+0x1c5/0x37a
[168711.875657]  [<ffffffff81476095>] page_fault+0x25/0x30
[168711.875662]  [<ffffffff8125658d>] ? pci_enable_msix+0x270/0x354
[168711.875667]  [<ffffffff81256584>] ? pci_enable_msix+0x267/0x354
[168711.875675]  [<ffffffffa0083400>] igbvf_probe+0x308/0x85b [igbvf]
[168711.875680]  [<ffffffff81006c3f>] ? xen_restore_fl_direct_end+0x0/0x1
[168711.875684]  [<ffffffff81080b0f>] ? arch_local_irq_restore+0xb/0xd
[168711.875689]  [<ffffffff812463ce>] local_pci_probe+0x44/0x75
[168711.875693]  [<ffffffff81246e60>] pci_device_probe+0xd0/0xff
[168711.875698]  [<ffffffff812e6af9>] driver_probe_device+0x124/0x1ff
[168711.875703]  [<ffffffff812e6c2e>] __driver_attach+0x5a/0x7e
[168711.875708]  [<ffffffff812e6bd4>] ? __driver_attach+0x0/0x7e
[168711.875713]  [<ffffffff812e5b9a>] bus_for_each_dev+0x53/0x89
[168711.875718]  [<ffffffff812e6706>] driver_attach+0x1e/0x20
[168711.875722]  [<ffffffff812e6318>] bus_add_driver+0xe2/0x235
[168711.875729]  [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875734]  [<ffffffff812e6e17>] driver_register+0x98/0x105
[168711.875741]  [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875746]  [<ffffffff81247723>] __pci_register_driver+0x56/0xc3
[168711.875752]  [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875759]  [<ffffffffa008904f>] igbvf_init_module+0x4f/0x51 [igbvf]
[168711.875764]  [<ffffffff810020a4>] do_one_initcall+0x7f/0x137
[168711.875769]  [<ffffffff810852dd>] sys_init_module+0xa6/0x1e4
[168711.875773]  [<ffffffff81009bc2>] system_call_fastpath+0x16/0x1b
[168711.879089] udevd-work[367]: '/sbin/modprobe -bv 
pci:v00008086d000010CAsv000015D9sd000010C9bc02sc00i00' unexpected exit with 
status 0x0009



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>