This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Cpu pools discussion

To: Tim Deegan <Tim.Deegan@xxxxxxxxxx>
Subject: Re: [Xen-devel] Cpu pools discussion
From: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Date: Tue, 28 Jul 2009 15:24:33 +0200
Cc: George Dunlap <dunlapg@xxxxxxxxx>, Zhigang Wang <zhigang.x.wang@xxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Keir Fraser <Keir.Fraser@xxxxxxxxxxxxx>
Delivery-date: Tue, 28 Jul 2009 06:25:03 -0700
Dkim-signature: v=1; a=rsa-sha256; c=simple/simple; d=ts.fujitsu.com; i=juergen.gross@xxxxxxxxxxxxxx; q=dns/txt; s=s1536b; t=1248787530; x=1280323530; h=from:sender:reply-to:subject:date:message-id:to:cc: mime-version:content-transfer-encoding:content-id: content-description:resent-date:resent-from:resent-sender: resent-to:resent-cc:resent-message-id:in-reply-to: references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:list-owner:list-archive; z=From:=20Juergen=20Gross=20<juergen.gross@xxxxxxxxxxxxxx> |Subject:=20Re:=20[Xen-devel]=20Cpu=20pools=20discussion |Date:=20Tue,=2028=20Jul=202009=2015:24:33=20+0200 |Message-ID:=20<4A6EFC11.9010404@xxxxxxxxxxxxxx>|To:=20Ti m=20Deegan=20<Tim.Deegan@xxxxxxxxxx>|CC:=20George=20Dunla p=20<dunlapg@xxxxxxxxx>,=20=0D=0A=20Zhigang=20Wang=20<zhi gang.x.wang@xxxxxxxxxx>,=0D=0A=20"xen-devel@xxxxxxxxxxxxx ce.com"=20<xen-devel@xxxxxxxxxxxxxxxxxxx>,=20=0D=0A=20Kei r=20Fraser=20<Keir.Fraser@xxxxxxxxxxxxx>|MIME-Version:=20 1.0|Content-Transfer-Encoding:=207bit|In-Reply-To:=20<200 90728130701.GJ5235@xxxxxxxxxxxxxxxxxxxxx>|References:=20< de76405a0907270820gd76458cs34354a61cc410acb@xxxxxxxxxxxxx m>=20<4A6E492D.201@xxxxxxxxxx>=20<20090728091929.GI5235@y ork.uk.xensource.com>=20<4A6ECFD3.4030709@xxxxxxxxxxxxxx> =20<de76405a0907280550j1ff82f1dq507f0258f138c477@xxxxxxxx il.com>=20<20090728130701.GJ5235@xxxxxxxxxxxxxxxxxxxxx>; bh=ds61HSprwkPeUQ2vBL+Jq8DM1jyleE309IXZratHpm0=; b=nbicYd41VF5qXIsyfzQLLKHtjIk932KM5qF71b/jdctt5NefVXy0SPN3 ShbdCwSPLAyfiADZyUpXDOqDuQTO1g8vFdXEYaqxGW+XpnuMfHow2RVC5 1XxCJvgVUf5ViCB51bYe3CEn1GhkxTJCBFtOVNmi6s541+3ep4dM1AqFr DofDdZUL3K8HitOHe+SuxD1wgDioST/9b520zWT329txd6Qf11DR9c5Tf e6/uqU0xZfjIN2MDcLyfp/AlVAXsJ;
Domainkey-signature: s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References: In-Reply-To:X-Enigmail-Version:Content-Type: Content-Transfer-Encoding; b=QNoVaaI5258eLDetYkATR9A9Ko/nipAkx68shcknfFyZyzXHIo1X1Eum Gbl7UspIvH3gLP4Vuli0awx1XSiipIC4u5WQOnqiaeSRgduden6hZ033Y qd/dAv9J5uInm9ufk3Ng8c8omG+63YnQ/0wMnkNngue54wRkWTOleEUTP 9lS9/LOKvt2hL8G3N4m9BJ0EZMgULeM8kBgfyLmctm39xaAyTkb++NBj9 w6or/zdhwNOX1Ha//UibKUkdE+d6t;
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20090728130701.GJ5235@xxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: Fujitsu Technology Solutions
References: <de76405a0907270820gd76458cs34354a61cc410acb@xxxxxxxxxxxxxx> <4A6E492D.201@xxxxxxxxxx> <20090728091929.GI5235@xxxxxxxxxxxxxxxxxxxxx> <4A6ECFD3.4030709@xxxxxxxxxxxxxx> <de76405a0907280550j1ff82f1dq507f0258f138c477@xxxxxxxxxxxxxx> <20090728130701.GJ5235@xxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla-Thunderbird (X11/20090707)
Tim Deegan wrote:
> At 13:50 +0100 on 28 Jul (1248789008), George Dunlap wrote:
>> On Tue, Jul 28, 2009 at 11:15 AM, Juergen
>> Gross<juergen.gross@xxxxxxxxxxxxxx> wrote:
>>> Tim Deegan wrote:
>>>> That's easily done by setting affinity masks in the tools, without
>>>> needing any mechanism in Xen.
>>> More or less.
>>> You have to set the affinity masks for ALL domains to avoid scheduling on 
>>> the
>>> "special" cpus.
> Bah.  You have to set the CPU pool of all domains to achieve the same
> thing; in any case this kind of thing is what toolstacks are good at. :)

If I have a dedicated pool for my "special domain" and all other domains are
running in the default pool 0, I only have to set the pool of my special
domain. Nothing else.

>>> You won't have reliable scheduling weights any more.
> That's a much more interesting argument.  It seems to me that in this
> simple case the scheduling weights will work out OK, but I can see that
> in the general case it gets entertaining.

Even in the relatively simple case of 2 disjunct subsets of domains/cpus
(e.g. 2 domains on cpu 0+1 and 2 domains on cpu 2+3) the consumed time
of the domains does not reflect their weights correctly.

>> Given that people want to partition a machine, I think cpu pools makes
>> the most sense:
>> * From a user perspective it's easier; no need to pin every VM, simply
>> assign which pool it starts in
> I'll say it again because I think it's important: policy belongs in the
> tools.  User-friendly abstractions don't have to extend into the
> hypervisor interfaces unless...
>> * From a scheduler perspective, it makes thinking about the algorithms
>> easier.  It's OK to build in the assumption that each VM can run
>> anywhere.  Other than partitioning, there's no real need to adjust the
>> scheduling algorithm to do it.
> ...unless there's a benefit to keeping the hypervisor simple.  Which
> this certainly looks like. 
> Does strict partitioning of CPUs like this satisfy everyone's
> requirements?  Bearing in mind that 
>  - It's not work-conserving, i.e. it doesn't allow best-effort
>    scheduling of pool A's vCPUs on the idle CPUs of pool B.
>  - It restricts the maximum useful number of vCPUs per guest to the size
>    of a pool rather than the size of the machine. 
>  - dom0 would be restricted to a subset of CPUs.  That seems OK to me
>    but occasionally people talk about having dom0's vCPUs pinned 1-1 on 
>    the physical CPUs.

You don't have to define other pools. You can just live with the default pool
extended to all cpus and everything is as today.

Pinning is still working in each pool as today.

If a user has domains with different scheduling requirements (e.g. sedf and
credit are to be used) he can use one partitioned machine instead two
dedicated machines. And he can shift resources between the domains (e.g.
devices, memory, single cores or even threads).
He can't do that without pools today.

With pools you have more possibilities without losing any function you have
today. The only restriction is that you might not be able to use ALL
features together with pools (e.g. complete load balancing), but the
alternative would be to either lose some other functionality (scheduling
weights) or to use different machines which won't give you load balancing


Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 636 47950
Fujitsu Technolgy Solutions               e-mail: juergen.gross@xxxxxxxxxxxxxx
Otto-Hahn-Ring 6                        Internet: ts.fujitsu.com
D-81739 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Xen-devel mailing list