This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Linux spin lock enhancement on xen

To: Jan Beulich <JBeulich@xxxxxxxxxx>
Subject: Re: [Xen-devel] Linux spin lock enhancement on xen
From: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
Date: Tue, 24 Aug 2010 17:11:32 +0100
Cc: Jeremy Fitzhardinge <jeremy@xxxxxxxx>, "Xen-devel@xxxxxxxxxxxxxxxxxxx" <Xen-devel@xxxxxxxxxxxxxxxxxxx>, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Delivery-date: Tue, 24 Aug 2010 09:12:22 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=eojh2doU7msHs+TC+9+tqchaUM2QR40GKkwrX8pGZ4w=; b=WjMO/+UPshwQPfqcwG58qPqI3eyXzGCm9NpVGBrxwGVr6obUc9r9TLVbCBhqlUNC1c UhKxb8LBJycRpbqeel2airVBcmJFZZ+U/ku85UU6mbhFOP9EXTHzgMbab4JZubt6n0p6 dYCS5tZU689phALxrDCfjj9NYnXUzf9Wn0Va0=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=GEoXEDg01/oaGjgFdAdb5PMvtcTvifv6K83UFsrttXqRzbyU6uoqeHKSURU+ebg4gJ js3th7BK5wbuuhqofBZXkpD9y4QHHrRUEWSavGRe1ezK/JfxGvw+MUrkleIFMRusHghH r58RR87P3a0bVEhJ+3celdnc3OU/29Vyhs2sM=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4C73E4850200007800011E31@xxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <AANLkTin_HTtxL9wB9JcxDWFeGGYHKHfBxGW4dPrYKDGb@xxxxxxxxxxxxxx> <C8993F5E.1EEDE%keir.fraser@xxxxxxxxxxxxx> <4C73A37B0200007800011D97@xxxxxxxxxxxxxxxxxx> <AANLkTinsXiOS5Sx6bBGnDGDMou_TO4wQGjq5gEqS4MX7@xxxxxxxxxxxxxx> <4C73E4850200007800011E31@xxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Tue, Aug 24, 2010 at 2:25 PM, Jan Beulich <JBeulich@xxxxxxxxxx> wrote:
> No, I understood it that way. What I was referring to is (as an
> example) the case where two vCPU-s on the sam pCPU's run queue
> both yield: They will each move after the other in the run queue in
> close succession, but neither will really make progress, and neither
> will really increase the likelihood of the respective lock holder to
> get a chance to run.

Ah, I see.  In order for this to be a waste, it needs to be the case that:
* Two vcpus from different domains grab a spinlock and are then preempted
* Two vcpus from different domains then fail to grab the spinlock
* The two vcpus holding the locks are kept from getting cpu by
{another vcpu, other vcpus} which uses a long time-slice
* The two waiting for the lock share a cpu with each other and no one else

Of course in this situation, it would be nice if Xen could migrate one
of the other vcpus to the cpu of the two yielding vcpus.  That
shouldn't be too hard to implement, at least to see if it has a
measurable impact on aggregate throughput.

> Immediately, or after a few (hundred) spin cycles?

It depends on the implementation.  The Citrix guest tools do binary
patching of spinlock routines for w2k3 and XP; I believe they spin for
1000 cycles or so.  The viridian enlightenments I believe would yield
immediately.  I think the pause instruction causes a yield immediately
as well.

Yielding immediately when the host is not overloaded is actually
probably not optimal: if the vcpu holding the lock is currently
running, it's likely that by the time the vcpu makes it to the
scheduler, the lock it's waiting for has already been released.
(Which is part of the reason it's a spinlock and not a semaphore.)

> And so I do with this. Apart from suspecting fairness issues with
> your yield_to proposal (as I wrote), my point just is - we won't
> know if a "complicated" solution outperforms a "simple" one if we
> don't try it.

Are you volunteering? :-)


Xen-devel mailing list