WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: NUMA guest: best-fit-nodes algorithm (was Re: [PATCH 00/

To: Andre Przywara <andre.przywara@xxxxxxx>
Subject: [Xen-devel] Re: NUMA guest: best-fit-nodes algorithm (was Re: [PATCH 00/11] PV NUMA Guests)
From: Dulloor <dulloor@xxxxxxxxx>
Date: Sat, 24 Apr 2010 02:51:45 -0400
Cc: xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>, "Nakajima, Jun" <jun.nakajima@xxxxxxxxx>, "Cui, Dexuan" <dexuan.cui@xxxxxxxxx>
Delivery-date: Fri, 23 Apr 2010 23:52:31 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=vIl/yGPqWiIsY/Hv7tHpAUhOls+wG9Yby9bkoe68c88=; b=ooXFDVppynY0FEjyJV6U2P3e68t6W7IrSP27XkPAR65bbWb9i6OJJWNx8h2lUBt7wW 7VV1cKfraUOPrrfjRGS/+C61w7O2k9cgV7jcYt/q5+Aton1nUqfTlgFVHmCMXwvA729z wDGlYTaIw5PYVZxg+T0e4+pxJ0QOc64+kg26o=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=CwSVbKmeLZrX+vi+Q8/k9pudPOEyB+WwV7OTtXDUX0qNUSzcmj4HJhqBSq/Htmsv5/ /Z07xXHQ1VJXBGWS0kuHTxIIJyLCBQl1OjrydmPSbGFeAN6ObVbM2Y5BzubcOhXjYklH crEuOFfsFqwIh99tXqBAwbC5DsrBWW9rUL0zE=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4BD19686.1050602@xxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <4BD19686.1050602@xxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Fri, Apr 23, 2010 at 8:45 AM, Andre Przywara <andre.przywara@xxxxxxx> wrote:
> Dulloor wrote:
>> Cui, Dexuan <dexuan.cui@xxxxxxxxx> wrote:
>>> xc_select_best_fit_nodes() decides the "min-set" of host nodes that
>>> will be used for the guest. It only considers the current memory
>>> usage of the system. Maybe we should also condider the cpu load? And >>
>>> the number of the nodes must be 2^^n? And how to handle the case
>>> #vcpu is < #vnode?
>>> And looks your patches only consider the guest's memory requirement
>>> -- guest's vcpu requirement is neglected? e.g., a guest may not need
>>> a very large amount of memory while it needs many vcpus.
>>> xc_select_best_fit_nodes() should consider this when
>>> determining the number of vnode.
>> I agree with you. I was planning to consider vcpu load as the next
>> step. Also, I am looking for a good heuristic. I looked at the
>> nodeload heuristic (currently in xen), but found it too naive.
>> But, if you/Andre think it is a good heuristic, I will add the
>> support. Actually, I think in future we should do away with strict
>> vcpu-affinities and rely more on a scheduler with necessary NUMA
>> support to complement our placement strategies.
>>
>> As of now, we don't SPLIT, if #vcpu < #vnode. We use STRIPING in that
>> case.
> Determing the current load of a node is quite a hard thing to do currently
> in Xen. If guests are pinned to nodes (which I'd consider necessary with the
> current credit scheduler), then using this affinity is a good heuristic to
> find good nodes, at least the best I can think of. So until we have a NUMA
> aware scheduler, we should go with this solution. Of course it only measures
> the theoretical load of a node and doesn't distinguish between idle and
> loaded guests. One would need something like a permanently running xm top to
> gather statistics about the guest's load, but that is something for a future
> patch.
> (Or is there a guest load metric already measured in Xen?)
Yeah, for the current credit scheduler, looks like we could use only
affinity for load heuristics.
I will add that to the node selection algorithm -  similar to what you
do in calculating nodeload.
Also, gathering guest load statistics over a period of time could be
useful too. But, it is unclear
how any temporal behaviour could aid permanent memory placement.

I have started looking into load balancing and NUMA-related stuff for
credit2. I hope to send out
something in coming weeks.

>
> Regards,
> Andre.
>
>
> --
> Andre Przywara
> AMD-Operating System Research Center (OSRC), Dresden, Germany
> Tel: +49 351 448-3567-12
>
>
-dulloor

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>