Re: [Xen-devel] qemu and xl semantics

[Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>]

On Fri, 2010-12-17 at 17:21 +0000, Christoph Egger wrote:
> On Friday 17 December 2010 11:32:41 Ian Campbell wrote:
> > On Fri, 2010-12-17 at 09:49 +0000, Christoph Egger wrote:
> > > On Friday 17 December 2010 10:15:14 Ian Campbell wrote:
> > > > On Fri, 2010-12-17 at 09:00 +0000, Christoph Egger wrote:
> > > > > Hi!
> > > > >
> > > > > When I start a guest with xm  the disk startup script assigns a
> > > > > loopback device for qemu to open it.
> > > > >
> > > > > Now it seems that qemu opens the disk image directly. Then when
> > > > > the loopback device wants to open the disk image then that fails
> > > > > with EBUSY.
> > > >
> > > > By "Now..." you mean "With xl..." ?
> > >
> > > no, I mean with xm.
> >
> > Hrm, I didn't think xm had changed in this area for ages, but I don't
> > really keep an eye on xm/xend.
> 
> It is not xm/xend that changed. It is qemu that changed and
> the startup scripts need an adaption to the new behaviour.
> Hence I am asking for the new semantics to know how to do that.

Which qemu changeset? (you obviously had this information to hand, why
not pro-actively help me to help you? Don't make me guess what you know
or have discovered)

The only recent qemu change which I am aware of in this area is the
backport of the blkback functionality from upstream Qemu. However this
should only be enabled in cases where blkback or blktap are not
available and furthermore is tied to libxl/xl so shouldn't have done
anything on xend (although shouldn't != doesn't).

Perhaps the problem is that this stuff is erroneously getting activated
on NetBSD?

> > > > I think this is all very specific to the precise disk type you have in
> > > > your config, i.e. tap: vs file: vs phy: etc. Which are you using?
> > >
> > > I use 'file'.
> >
> > If you are using xm and not xl this is not relevant but:
> >
> > AIUI libxl treats file: as tap:aio:, in other words it will fire up
> > blktap to provide the device backend. xend probably used a loop device
> > and treated it more like phy: i.e. invoked blkback.
> 
> Yes, I noticed that and I wonder how the algorithm works behind that.

See libxl_device.c:libxl__device_disk_backend_type_of_phystype and
libxl.c:libxl_device_disk_add.

A file: type device is really just a "degenerate" thing in the style of
qcow or vhd so using tapdisk to export it makes some sort of sense and
when it came to implementing libxl it was the simpler option compared
with integrating with the loopback subsystem.

> > Since NetBSD doesn't have blktap perhaps libxl needs to be updated to do
> > the NetBSD equivalent? (I thought I'd seen a patch from you which did
> > this?)
> 
> The disk access works on NetBSD - don't ask me how - otherwise I wouldn't
> be able to boot a guest at all. I have to run the network script manually
> in the dom0 to have network access from and to the guest.
> 
> > Recently libxl was also changed to use the qemu disk backend in cases
> > where blktap is not available -- perhaps that had an undesired knockon
> > effect on NetBSD?
> 
> Yes. Recently this message appears from the NetBSD disk backend driver:
> 
> xbdback backend/vbd/1/832: can't VOP_OPEN device 0xe13: 16
> 16 means EBUSY.

But it works other than this message?

I don't have much insight into how the NetBSD side of things works, I'm
afraid I think you will need to dig in and debug it.

> > Further to this there was a patch floating around (from Stefano) which
> > ensured that a qemu was launched when it was necessary for PV guests to
> > provide the disk backend. I'm not sure this got merged but ion the
> > meantime the workaround is to add a vfb which also causes qemu to get
> > launched for a PV guest.
> 
> What is the longterm solution?

The patch from Stefano.

Even longer term we may improve the libxl interface to make it more
explicit to the calling toolstack when it needs to start a qemu.

> 
> > > > > The network startup script adds the tap device to the bridge
> > > > > or assigns an ip address.
> > > > > With xl neither the disk nor the network script runs.
> > > > > So when I start the guest with xl then I have
> > > > > the tap device assigned to the guest but the
> > > > > tap device is not configured in the dom0.
> > > > >
> > > > > How does the 'xl' way work in respect to the network script
> > > > > used with 'xm' ?
> > > >
> > > > On Linux these are run from the hotplug event, via the udev rules. I
> > > > presume you are talking about on NetBSD though?
> > >
> > > Yes.
> > >
> > > > Under Linux I think it was always the same under xm too although there
> > > > have been some tweaks recently, e.g. the vif script is now always
> > > > /etc/xen/scripts/vif-setup which handles the indirection to the script
> > > > in the domain config or the default. Previously xend the hotplug rules
> > > > called the configured script directly. This change was
> > > > 21549:8bcaec29574e and was common to xm and xl I think.
> > >
> > > On NetBSD a xenbackendd starts along with xenstored.
> > > xenbackendd watches on /local/domain/0/backend
> > > and invokes the corresponding scripts when something changes
> > > beneath that.
> > >
> > > 'xend' is doing that in DevController.py. Since 'xl' is not interacting
> > > with 'xend' the scripts don't get launched at all.
> >
> > xl also creates stuff under /local/domain/0/backend (as it must) so why
> > is xenbackendd not firing up and running the scripts?
> 
> xenbackendd is firing up for things 
> like '/local/domain/0/backend/console/1/0/script'.
> 
> xenbackendd listens for changes on vec[XS_WATCH_PATH]/state.
> Then it reads vec[XS_WATCH_PATH]/hotplug-status.
> 
> And then it looks at '/local/domain/0/backend/vbd' 
> and '/local/domain/0/backend/vif' and starts the corresponding
> scripts.
> 
> > Perhaps xenbackendd is expecting some additional key which xend adds but
> > libxl doesn't?
> 
> Maybe. Is something missing from above?

I've no idea -- what does xenbackendd expect? What do you get in
xenstore? Howe do the two match up?

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel