|
xen-devel
Re: [Xen-devel] Host Numa informtion in dom0
Dulloor wrote:
Beside that I have to oppose the introduction of sockets_per_node again.
Future AMD processors will feature _two_ nodes on _one_ socket, so this
variable should hold 1/2, but this will be rounded to zero. I think this
information is pretty useless anyway, as the number of sockets is mostly
interesting for licensing purposes, where a single number is sufficient.
I sent a similar patch (was using to enlist pcpu-tuples and in
vcpu-pin/unpin) and I didn't pursue it because of this same argument.
When we talk of cpu topology, that's how it is currently :
nodes-socket-cpu-core. Don't sockets also figure in the cache and
interconnect hierarchy ?
Not necessarily. Think of Intel's Core2Quad, they have two separate L2
caches each associated to two of the four cores in one socket. If you
move from core0 to core2 then AFAIK the cost would be very similar to
moving to another processor socket. So in fact the term socket does not
help here.
The situation is similar to the new AMD CPUs, just that it replaces "L2
cache" with "node" (aka shared memory controller, which also matches
shared L3 cache). In fact the cost of moving from one node to the
neighbor in the same socket is exactly the same as moving to another
socket.
What would be the hierarchy in those future AMD processors ? Even Keir
and Ian Pratt initially wanted the pcpu-tuples
to be listed that way. So, it would be helpful to make a call and move ahead.
You could create variables like cores_per_socket and cores_per_node,
this would solve this issue for now. Actually better would be an array
mapping cores (or threads) to {nodes,sockets,L[123]_caches}, as this
would allow asymmetrical configurations (useful for guests).
In the past there once was a socket_per_node value in physinfo, but it
has been removed. It was not used anywhere, and multiplying the whole
chain of x_per_y sometimes ended up in wrong values anyway.
Anyway, if you insist on this value it will hold bogus values for the
upcoming processors. If it will be zero, you end up in trouble when
multiplying or dividing with it, and letting it be one is also wrong.
I am sorry to spoil this whole game, but that it's how it is.
If you or Nitin show me how the socket_per_node variable should be used,
we can maybe find a pleasing solution.
Regards,
Andre.
On Mon, Feb 1, 2010 at 5:23 AM, Andre Przywara <andre.przywara@xxxxxxx> wrote:
Kamble, Nitin A wrote:
Hi Keir,
Attached is the patch which exposes the host numa information to dom0.
With the patch “xm info” command now also gives the cpu topology & host numa
information. This will be later used to build guest numa support.
What information are you missing from the current physinfo? As far as I can
see, only the total amount of memory per node is not provided. But one could
get this info from parsing the SRAT table in Dom0, which is at least mapped
into Dom0's memory.
Or do you want to provide NUMA information to all PV guests (but then it
cannot be a sysctl)? This would be helpful, as this would avoid to enable
ACPI parsing in PV Linux for NUMA guest support.
Beside that I have to oppose the introduction of sockets_per_node again.
Future AMD processors will feature _two_ nodes on _one_ socket, so this
variable should hold 1/2, but this will be rounded to zero. I think this
information is pretty useless anyway, as the number of sockets is mostly
interesting for licensing purposes, where a single number is sufficient.
For scheduling purposes cache topology is more important.
My NUMA guest patches (currently for HVM only) are doing fine, I will try to
send out a RFC patches this week. I think they don't interfere with this
patch, but if you have other patches in development, we should sync on this.
The scope of my patches is to let the user (or xend) describe a guest's
topology (either by specifying only the number of guest nodes in the config
file or by explicitly describing the whole NUMA topology). Some code will
assign host nodes to the guest nodes (I am not sure yet whether this really
belongs into xend as it currently does, or is better done in libxc, where
libxenlight would also benefit).
Then libxc's hvm_build_* will pass that info into the hvm_info_table, where
code in the hvmloader will generate an appropriate SRAT table.
An extension of this would be to let Xen automatically decide whether a
split of the resources is necessary (because there is not enough memory
available (anymore) on one node).
Looking forward to comments...
Regards,
Andre.
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
|
|
Home