Re: [Xen-devel] [RFC][PATCH] domheap optimization for NUMA

[Keir Fraser <keir.fraser@xxxxxxxxxxxxx>]

On 2/4/08 23:49, "Andre Przywara" <andre.przywara@xxxxxxx> wrote:

>> Yes, but it's a bad interface, particularlty when the function is called
>> alloc_domheap_pages_on_node(). Pass in a nodeid. Write a helper function to
>> work out the nodeid from the domain*.
> I was just looking at this code, too, so I fixed this. Eventually
> alloc_heap_pages is called, which deals with nodes only, so I replaced
> cpu with node everywhere else, too. Now __alloc_domheap_pages and
> alloc_domheap_pages_on_node are almost the same (except parameter
> ordering), so I removed the first one, since the naming of the latter is
> better. Passing node numbers instead of cpu numbers needs cpu_to_node
> and asm/numa.h, if you think there is a better way, I am all ears.

That's fine. If you reference numa stuff then you need numa.h.

But vcpu_to_node and domain_to_node as well as cpu_to_node, please. There's
no need to be open-coding v->processor everywhere. Also in future we might
care to pick node based on v's affinity map rather than just current
processor value. And usage of d->vcpu[0] without checking for != NULL is
asking to introduce edge-case bugs. We can easily do that NULL check in one
place if we implement domain_to_node().

And, while I'm thinking about the interfaces, let's just stick to
alloc_domheap_page() and alloc_domheap_pages(). Let's add a flags parameter
to the former (so it matches the latter in that respect) and let's add a
MEMF_node() flag subtype (similar to MEMF_bits). Semantics will be that if
MEMF_node(node) is provided then we try to allocate memory from node; else
we try to allocate memory from a node local to specified domain; else if
domain is NULL then we ignore locality.

Since zero is probably a valid numa nodeid we can define MEMF_node() as
something like ((((node)+1)&0xff)<<8). Then since NUMA_NO_NODE==0xff
everything works nicely: MEMF_node(NUMA_NO_NODE) is equivalent to not
specifying MEMF_node() at all, which is what we would logically expect.
NUMA_NO_NODE probably needs to be pulled out of asm-x86/numa.h and made the
official arch-neutral way to specify 'don't care' for numa nodes.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel