Réf. : Re: [Xen-ia64-devel] [PATCH] Use saner dom0 memory and vcpu defaults, don't panic on over-allocation

[jean-paul.pigache@xxxxxxxx]

I am not sure it is wise to start dom0 with all available CPUs before we have a good optimization for NUMA machines in Xen,... unless we can rely on the embedded Linux kernel NUMA support for Dom0?.
My guess is that starting dom0 with 4 CPUs that can be mapped on a single board on all currently available hardware platforms should cover the large majority of use cases.
Jean-Paul

Jarod Wilson <jwilson@xxxxxxxxxx> Envoyé par : xen-ia64-devel-bounces@xxxxxxxxxxxxxxxxxxx 06/08/2007 16:11

Pour :        Alex Williamson <alex.williamson@xxxxxx>         cc :        Isaku Yamahata <yamahata@xxxxxxxxxxxxx>, xen-ia64-devel@xxxxxxxxxxxxxxxxxxx         Objet :        Re: [Xen-ia64-devel] [PATCH] Use saner dom0 memory and        vcpu        defaults,         don't panic on over-allocation

Alex Williamson wrote:
>> Based on empirical testing, it seems something along the lines of 1MB 
>> per 4GB of RAM, capped at like 64MB (though probably not necessary), 
>> might be sane. Its not exact of course, since memory amount doesn't 
>> necessarily indicate how many PCI root bridges there are to allocate, 
>> etc., but it seems to be a reasonable guess... The attached is a crack 
>> at that, and on your 96G machine, should decrease the default "all" 
>> memory allocation by about 24MB, allowing it to fully boot.
> 
>    I think we're going to have to go with something like this, but why
> would we reduce the cap to 64MB?  I usually think of ia64 systems as
> having "bigger" I/O than x86, so it seems like maybe we want to stick
> with at least 128MB(?)

I think my original thought was that since your 96G box only needed
really 17MB, 64MB was a ton to withhold. But after hitting send, I was
thinking that no cap at all might make more sense -- you'd need 288GB of
RAM to even get to 64MB here, and that's a tiny drop in the bucket when
you have that much. Even with 1TB of RAM, we would still withhold less
than 256MB. Until we have some 1TB+ systems to test on, we don't really
know if reserving more than 128MB makes sense or not... I'd have to lean
toward simply not capping this withholding at all, at least for right now.

>> Here's another thought: we could play it extra-safe, and slide that 
>> allocation down another 12-24MB, and possibly even switch to allocating 
>> "all" memory to dom0 by default... Then allocate all cpus too, and we'd 
>> have parity with x86 here... :)
> 
>    I'm not sure I see the value in that yet.  If you have a
> 32/64/128/etc-way system, do you really want to have all those extra
> vCPUs floating around?  Don't forget, XenSource is targeting 4-socket
> systems.  In that arena, it might make sense to allocate all CPUs, but
> as you scale the system up, I'm not sure it continues to make sense.

The thought is that you can always balloon dom0 down if you need/want
to, and take away cpus. In addition to x86 parity (we do have people
running xen on some pretty large x86_64 systems...), you've also got
parity with the bare-metal kernel. Also, my understanding is that if you
only allocate 4GB of RAM and 4 cpus to start out, that's the maximum
that dom0 can ever have. So if its doable, why not start out making all
resources available by default, and then reduce resources as needed?

In my particular case, the best system I've got direct access to is a
16-way, and I actually *do* like having all 16 vcpus available to dom0,
as I usually do development work (i.e. lots of compiling) in dom0, so
the more cpu power the better. I suppose I could just spin up a guest
with a ton of vcpus, but then my i/o performance isn't nearly as hot (I
usually use blktap w/disk images for guests, not dedicated partitions)...

But then I don't know what the typical end-user use case of ia64 xen is,
or what the average system size is, which would seem to be a fairly
relevant factor in deciding the most reasonable defaults... My intuition
says that most of our customers would like to have their systems boot up
using all available resources though. That, and I'm already starting to
feel like a 4G/4cpu default is a bit wimpy. :)


-- 
Jarod Wilson
jwilson@xxxxxxxxxx


_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel

signature.asc
Description: Binary data

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel