This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] [RFC][PATCH] domheap optimization for NUMA

To: Andre Przywara <andre.przywara@xxxxxxx>
Subject: Re: [Xen-devel] [RFC][PATCH] domheap optimization for NUMA
From: Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Date: Thu, 03 Apr 2008 11:58:09 +0100
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, "Edwin.Zhai" <edwin.zhai@xxxxxxxxx>
Delivery-date: Thu, 03 Apr 2008 03:59:32 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <47F4B3D2.1030909@xxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AciVeZoo2H2XegFsEd2x0gAX8io7RQ==
Thread-topic: [Xen-devel] [RFC][PATCH] domheap optimization for NUMA
User-agent: Microsoft-Entourage/
On 3/4/08 11:39, "Andre Przywara" <andre.przywara@xxxxxxx> wrote:

> By the way, can we solve the DMA_BITSIZE problem (your mail from 28th
> Feb) with this? If no node is specified, use the current behaviour of
> preferring non DMA zones, else stick to the given node.
> If you agree, I will implement this.

I don't think that gets us what we want. The fact is we specify NUMA node on
nearly 100% of allocations (either explicitly via MEMF_node() or via passing
a non-NULL domain pointer). So you would *always* prefer local DMA pages
over remote non-DMA pages. That's not necessarily better than the current

My point in my email of Feb 28th was that we should set dma_bitsize
'appropriately' (well, according to a slightly arbitrary policy :-) so that
*some* DMA memory is set aside and only used to satisfy allocations which
cannot be satisfied by a remote, while *some* memory is always made
available on every node for local allocations.

Does that make sense?

>> NUMA_NO_NODE probably needs to be pulled out of asm-x86/numa.h and made the
>> official arch-neutral way to specify 'don't care' for numa nodes.
> Is this really needed? I provided memflags=0 is all don't care cases,
> this should work and is more compatible. But beware that this silently
> assumes in page_alloc.c#alloc_domheap_pages that NUMA_NO_NODE is 0xFF,
> otherwise this trick will not work.

Yes it is needed if your patch is to work across all architectures, not just
x86! Your current patch is broken in this respect because you quite
unnecessarily define domain_to_node() and vcpu_to_node() in asm/numa.h
rather than xen/numa.h.

Please address architectural portability and re-send the patch. Apart from
that I think it's just about ready to go in.


> Attached again a diff against my last version and the full patch (for
> some reason a missing bracket slipped through my last one, sorry for that).
> This is only quick-tested (booted and created a guest on each node).

Xen-devel mailing list