[Xen-ia64-devel] RE: Xen/ia64 - global or per VP VHPT

["Magenheimer, Dan (HP Labs Fort Collins)" <dan.magenheimer@xxxxxx>]

> In my opinion this is a moot point because in order to provide the
> appropriate semantics for physical mode emulation (PRS.dt, or 
> PSR.it, or
> PSR.rt == 0) it is necessary to support a 4K page size as the minimum
> (unless you special case translations for physical mode 
> emulation). Also in
> terms of machine memory utilization, it is better to have 
> smaller pages (I
> know this functionality is not yet available in Xen, but I 
> believe it will
> become important once people are done working on the basics).

In my opinion, performance when emulating physical mode is
a moot point.  It might make sense to simply not insert
metaphysical addresses into the VHPT and just rely on the
TLB (though perhaps a one-entry virtual TLB might be required
to ensure forward progress).

Remember, one major difference between full virtualization (VT)
and paravirtualization is that you have to handle any case that
any crazy OS designer might try, while I just have to ensure that
I can tell the crazy OS designer what crazy things need to be
removed to make sure it works on Xen :-)  This guarantees that
our design choices will sometimes differ.

> It is not just purging. Having a global VHPT is, in general, 
> really bad for scalability....

> Another important thing is hashing into the VHPT. If you have ...

> As you point out this is ALWAYS the case, but what matters is 
> what are your target workloads and target systems are...

All this just says that a global VHPT may not be good for a
big machine.  This may be true.  I'm not suggesting that
Xen/ia64 support ONLY a global VHPT or even necessarily that
it be the default, just that we preserve the capability to
configure either (or even both).

I wasn't present in the early Itanium architecture discussions
but I'll bet there were advocates for both lVHPT and sVHPT who
each thought it a terrible waste that the architecture support
both.  That was silicon and both are supported; this is a small
matter of software :-)

> Memory footprint is really not that big a deal for these 
> large machines, but
> in any case, the size of the VHPT is typically proportional 
> to the size of
> physical memory (some people suggest 4 PTEs per physical page 
> frame and some
> people suggest 2, but in any case, there is a linear 
> relationship between
> the two). If you follow this guide line, then individual 
> VHPTs for 5 guests
> should be 1/5 of the size of the combined VHPT for all 5 guests.

The point is that significant memory needs to be reserved in advance
or dynamically recovered whenever a domain launches.  Maybe this
isn't a big deal with a good flexible memory allocator and
"hidden ballooning" to steal physical memory from running domains.

E.g., assume an administrator automatically configures all domains
with a nominal 4GB but ability to dynamically grow up to 64GB.  The
per-guest VHPT would need to pre-allocate a shadow VHPT for the
largest of these (say 1% of 64GB) even if each of the domains never
grew beyond the 4GB, right?  (Either that or some kind of VHPT
resizing might be required whenever memory is "hot-plugged"?)

Again, there's a lot of interesting questions and discussion around
this... which means its best to preserve our options if possible.

Dan

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel