This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Xen 3.4.1 NUMA support

To: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
Subject: Re: [Xen-devel] Xen 3.4.1 NUMA support
From: Dulloor <dulloor@xxxxxxxxx>
Date: Mon, 9 Nov 2009 07:51:11 -0500
Cc: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Keir Fraser <Keir.Fraser@xxxxxxxxxxxxx>, Papagiannis Anastasios <apapag@xxxxxxxxxxxx>
Delivery-date: Mon, 09 Nov 2009 04:51:38 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=oPnJdgF4d5j/b/IVMRzPoLv5ytAqvuAlWdGDBetevN0=; b=q4iN2AhZlRpbUtZsGqb/7H69eep6m/vFVqPaKx001S+pC3uyPDr8IuJqQ0FWKFqWSl RHmqYeQkfg3FbPHHXBQvqnXcdkCain/M3SizIMaDinlShqmROUMQxJe+8D4AH5EfJ/wv r+Ws5N5uhrnf/okHgCMvsG+3HXcfqjnL3DYVk=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=ABFaSEy6oiyW5xY7LhQO9bK0vUO9tdaPzT7ETmEOLwJbxq6lKX3p9otULdzv/oGur/ 40LuB2Z/XNVHxYoCwmysIW9fRySv1i2IDQZp6sFMoWUSF/E7QH5yjB7sx+Jr+PGG82WY YAqx/L8/uSKERoDiG440m4UNAsPsM62XjGz+k=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <de76405a0911090429i15e68ecbi52bae967ffae3ae8@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <bd4f4a54-5269-42d8-b16d-cbdfaeeba361@default> <4AF7FE15.6070503@xxxxxxxxxxxxx> <940bcfd20911090339m199e3893ke8e855e2c0ed3b9b@xxxxxxxxxxxxxx> <de76405a0911090429i15e68ecbi52bae967ffae3ae8@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Sure ! Let know when you have the patches ready. Also, that might be a
good time to see if runq-per-l2 works better.


On Mon, Nov 9, 2009 at 7:29 AM, George Dunlap
<George.Dunlap@xxxxxxxxxxxxx> wrote:
> On Mon, Nov 9, 2009 at 11:39 AM, Dulloor <dulloor@xxxxxxxxx> wrote:
>> What's the current scope and status of your scheduler work ? Is it
>> going to look similar to the Linux scheduler (with scheduling domains,
>> et al). In that case, topology is already accounted for, to a large
>> extent. It would be good to know so that I can work on something that
>> doesn't overlap.
> My plan was to do something similar to Linux, but with this
> difference: Instead of having one runqueue per logical processor (as
> both Xen and Linux currently do), and having "domains" all the way up
> (as Linux currently does), I had planned on having one runqueue per L2
> processor cache.  The main reason to avoid migration is to preserve a
> warm cache; but since L1's are replaced so quickly, there should be
> little impact to a VM migrating between different threads and cores
> which share the same L2.
> Above the L2s I was planning on having an idea similar to the Linux
> "domains" (although obviously it would need a different name to avoid
> confusion), and doing explicit load-balancing between them.  But as I
> have not had a chance to test this kind of load balancing yet, the
> plan may change somewhate before then.
> Problems to solve wrt NUMA, as I understand it, are to balance the
> performance cost of sharing a busy local CPU, vs the performance cost
> of non-local memory accesses.  This would involve adding the NUMA
> logic to the load balancing algorithm.  Which I guess would depend in
> part on having a load balancing algorithm to begin with. :-)
> Once I have the basic credit patches in working order, would you be
> interested in working on the load-balancing between runqueues?  I can
> then work on further testing of the credit algorithm.  My ultimate
> goal would be to have a basic regression test that people could use to
> measure how their changes to the scheduler affect a wide variety of
> workloads.
>  -George

Xen-devel mailing list