[Xen-devel] pci device hotplug, race accessing xenstore

[Phung Te Ha <phungte@xxxxxxxxx>]

Hello Simon,

I took the source as per you message: http://marc.info/?l=xen-devel&m=124748015304566&w=4

compiled and run it on an Intel-DQ35JO, Fedora-10.

When I try to pass pci device through at boot time in configuration file, there's a race between xend and qemu accessing xenstore.

Xend waits in signalDeviceModel(...) for qemu to declare 'running' then write to the dm-command pipe the devices to be passed-through.

On the qemu side, it poses a watch on  /local/domain/0/device-model/2/command and expects the dm-command from there, by calling xs_watch(...). xs_watch(...) causes xenstored to run do_watch(...) and at the end, run add_event(...) with the following comment:
          /* We fire once up front: simplifies clients and restart. */


The problem shows when xend is faster, detecting qemu 'running' state, and calls xstransact.Store adn writes to the command pipe, before qemu can call main_loop_wait(...) and run one empty loop on the command pipe. This write causes xenstored to run a fires_watch, thus another add_event(...).
The problem shows in qemu log by an extra dm-command, using wrong parameter and fails to initialize, for instance:

...
xs_read_watch: msg type 15 body /local/domain/0/device-model/3/command
read_message: msg type reply pci-ins
dm-command: hot insert pass-through pci dev 
read_message: msg type reply 0000:00:1b.0@100
register_real_device: Assigning real physical device 00:1b.0 ...
pt_register_regions: IO region registered (size=0x00004000 base_addr=0x90420004)
pt_msi_setup: msi mapped with pirq ff
register_real_device: Real physical device 00:1b.0 registered successfuly!
IRQ type = MSI-INTx
read_message: msg type reply OK
read_message: msg type reply OK
xs_read_watch: msg type 15 body /local/domain/0/device-model/3/command
read_message: msg type reply pci-ins
dm-command: hot insert pass-through pci dev 
read_message: msg type reply 0x20
hot add pci devfn -1 exceed.
read_message: msg type reply OK
...

On the xend side:

...
    (bdf_str, vdevfn))
VmError: Cannot pass-through PCI function '0000:00:1b.0@100'. Device model reported an error: no free hotplug devfn
[2009-10-13 10:45:10 4174] ERROR (XendDomainInfo:471) VM start failed
Traceback (most recent call last):
...


Thank you.

Phung-Te



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel