WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] pci device hotplug, race accessing xenstore

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] pci device hotplug, race accessing xenstore
From: Phung Te Ha <phungte@xxxxxxxxx>
Date: Tue, 13 Oct 2009 12:13:00 -0700
Cc: horms@xxxxxxxxxxxx
Delivery-date: Tue, 13 Oct 2009 12:13:31 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:cc:content-type; bh=8q4A3OpQIjR/3ZmW2dDjGN8srQFk4/mKW5T18ZVFdf4=; b=bhmQ9+23Md1RCDTHYFnDWVjNfYjDuoKwvj8632vcJEbAiBe7X5R6zylxM6rUFyfjLQ Gsq7gjhg/+rx9eQhAueP/tkeE14bQyyOao4n2Ln3uT7/2UsyNUYItzrr3wHWT6NFNVUr ybJ4v1hvmkOV/arfG1Kmva//0jonGBvQ+HpSo=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=FPF1jrq0etpQ5bnVrWutbE9I2bWsMjB8KcXR7gpKJFdsqsimvCw+NMf8c3ZFamDhOy MrJ6aq2PjSx5tS1hTE625U/C15YBdUKj+YVl8Scttu4ZjHTa/nrryQNQwnVJGhA/naZe +xiC7QCpkH6xuSngDTCFp5Jb7UrzpfB0JV5Gs=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Hello Simon,

I took the source as per you message: http://marc.info/?l=xen-devel&m=124748015304566&w=4

compiled and run it on an Intel-DQ35JO, Fedora-10.

When I try to pass pci device through at boot time in configuration file, there's a race between xend and qemu accessing xenstore.

Xend waits in signalDeviceModel(...) for qemu to declare 'running' then write to the dm-command pipe the devices to be passed-through.

On the qemu side, it poses a watch on  /local/domain/0/device-model/2/command and expects the dm-command from there, by calling xs_watch(...). xs_watch(...) causes xenstored to run do_watch(...) and at the end, run add_event(...) with the following comment:
          /* We fire once up front: simplifies clients and restart. */


The problem shows when xend is faster, detecting qemu 'running' state, and calls xstransact.Store adn writes to the command pipe, before qemu can call main_loop_wait(...) and run one empty loop on the command pipe. This write causes xenstored to run a fires_watch, thus another add_event(...).
The problem shows in qemu log by an extra dm-command, using wrong parameter and fails to initialize, for instance:

...
xs_read_watch: msg type 15 body /local/domain/0/device-model/3/command
read_message: msg type reply pci-ins
dm-command: hot insert pass-through pci dev
read_message: msg type reply 0000:00:1b.0@100
register_real_device: Assigning real physical device 00:1b.0 ...
pt_register_regions: IO region registered (size=0x00004000 base_addr=0x90420004)
pt_msi_setup: msi mapped with pirq ff
register_real_device: Real physical device 00:1b.0 registered successfuly!
IRQ type = MSI-INTx
read_message: msg type reply OK
read_message: msg type reply OK
xs_read_watch: msg type 15 body /local/domain/0/device-model/3/command
read_message: msg type reply pci-ins
dm-command: hot insert pass-through pci dev
read_message: msg type reply 0x20
hot add pci devfn -1 exceed.
read_message: msg type reply OK
...

On the xend side:

...
    (bdf_str, vdevfn))
VmError: Cannot pass-through PCI function '0000:00:1b.0@100'. Device model reported an error: no free hotplug devfn
[2009-10-13 10:45:10 4174] ERROR (XendDomainInfo:471) VM start failed
Traceback (most recent call last):
...


Thank you.

Phung-Te



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel