This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] [PATCH] Avoid endless loop for vcpu migration

To: Jan Beulich <JBeulich@xxxxxxxxxx>
Subject: Re: [Xen-devel] [PATCH] Avoid endless loop for vcpu migration
From: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Date: Tue, 15 Mar 2011 09:46:08 +0100
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Tue, 15 Mar 2011 01:47:10 -0700
Dkim-signature: v=1; a=rsa-sha256; c=simple/simple; d=ts.fujitsu.com; i=juergen.gross@xxxxxxxxxxxxxx; q=dns/txt; s=s1536b; t=1300178771; x=1331714771; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to; bh=YK5DlQO4Q6+G1gyB1OVagN8DhRD0Xtibwv72p9odeus=; b=K6LFo9mcg28JMITqnJKjDBR5iGiT5tPs+zI7Wrsxpt9HPIzvoQN2jg5L cUsZ2RUmUH1VEqj+X7Wbe0q2UvGdLIRS01u466e4RQZAJQKlLnezsk+K5 tb5enaoaQX4ZWtOxmYDWECkRT3y//mmOdW7K4eevG3jdoNUuPjwd9wGK1 Hp5jjSmpoYn8GwtRQ1IMOg2mpx7fvwH2vtcJC07ZEt0X/nIRTTo54B1Mt boIXQkG0NzHcTYVyvFkjYhTLzUEXy;
Domainkey-signature: s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=tlM+e6IbW+jQMggoHWQhj4yAFXdAONNWfqH3cLnkAiGHhZrGyXk9W6LE qPDrTqI3Vkh830tRX6LCCBvfgUWsIhvkR+u1SFAEotw7pk/rlNXwFgeeU AyFNiWl/8DKw0n5yon7UERH+jKMzLw9c7EIUNs+nQ50Pzrb7dOE7eXg7B 7cy2hKse3gbgaKVgn6XGKl2UM5UOwlBq6B7t+bOkgI3NavZ6yPcppUx/z tMUMehcmyjR+LB5Tdsi2kMPqZnCGL;
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4D7F29ED02000078000367F4@xxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: Fujitsu Technology Solutions
References: <9d164ce877a75cab847b.1300113594@nehalem1> <4D7E3C640200007800036564@xxxxxxxxxxxxxxxxxx> <4D7EFE43.7070900@xxxxxxxxxxxxxx> <4D7F29ED02000078000367F4@xxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20101226 Iceowl/1.0b1 Icedove/3.0.11
On 03/15/11 08:57, Jan Beulich wrote:
On 15.03.11 at 06:50, Juergen Gross<juergen.gross@xxxxxxxxxxxxxx>  wrote:
On 03/14/11 16:03, Jan Beulich wrote:
On 14.03.11 at 15:39, Juergen Gross<juergen.gross@xxxxxxxxxxxxxx>   wrote:
On multi-thread multi-core systems an endless loop can occur in vcpu_migrate()
with credit scheduler. Avoid this loop by changing the interface of pick_cpu
to indicate a repeated call in this case.

But you're not changing in any way the loop that doesn't get
exited - did you perhaps read my original description as the
pick function itself looping (which - afaict - it doesn't)?

I'm changing the way the pick_cpu function is reacting on multiple calls in
a loop. If I've understood the idle_bias correctly, updating it in each
loop iteration did result in returning another cpu for each call.
By updating idle_bias only once, it should return the same cpu in subsequent
calls. This should exit the loop in vcpu_migrate.

You're only decreasing the likelihood of a live lock, as the return
value of pick_cpu not only depends on idle_bias.

Hmm, then another solution would be to let pick_cpu really return the
proposed cpu from the first iteration, if it doesn't contradict the
allowed settings. It could be sub-optimal, but I don't think this is
critical, as vcpu_migrate is called rarely.

Patch attached.

Further, the change still isn't consistent with idle_bias - the
updating ought to happen on the last iteration (if you need
to call the function more than once), not the first one, which
creates a chicken-and-egg problem for you as you will know
it's the last one only when it returned.

Is it really so important idle_bias is reflecting the last cpu selected?
I was under the impression it should be okay when this is true in most
cases. With my patch idle_bias might be "wrong" if there is a race with
other cpus forcing a selection of a different cpu in the second iteration
of the loop in vcpu_migrate. Is this really critical? I doubt it.

It's not critical, and not affecting correctness. But with updating
idle_bias on the first invocation you're (on the right hardware)
basically guaranteeing the second invocation to return a
different CPU. That way, your loop will be run minimally three
times on such systems. I already find it odd to require two
iterations when previously this was a strait code path.

This was wrong. It was always required to hold the schedule lock of the
picked cpu as well, otherwise a race with cpu hotplug would be possible.

If there's really no way around the iterative approach, one
possibility might be to not take into consideration idle_bias
on non-initial invocations at all.

This would be a side effect of my suggestion.


Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@xxxxxxxxxxxxxx
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Attachment: vcpu_migrate.patch
Description: Text Data

Xen-devel mailing list