RE: [Xen-ia64-devel] RE: Xen/ia64 - global or per VP VHPT

["Magenheimer, Dan (HP Labs Fort Collins)" <dan.magenheimer@xxxxxx>]

 > > Wow, I hadn't even realized before you mentioned it... a lVHPT is
> > generally contiguous in physical memory because it has to be
> > pinned by a TR (or more than one TR, but there's not many to choose
> > from!).
> 
> No, the VHPT does not have to be pinned by a TR. People do 
> it, but it does
> not have to be that way. 

It doesn't architecturally, but it does practically, right?
If the the VHPT is greatly fragmented, I'll bet nearly all of
the performance advantage is gone due to extra misses and/or
loss of usable entries in the DTLB.  Not to mention the complexity
of the psr.ic-off code to handle this...

> - If you had a single global VHPT and you wanted to pin it 
> with a TR, then
> you could not grow it or shrink it dynamically without 
> running into the
> issue you mention about finding physically contiguous memory.
>
> - If you cannot grow or shrink your VHPT, then you need to 
> preallocate it to
> its maximum size at boot time regardless of how many domains 
> are running.
> 
> - If you are willing to preallocate memory at boot, you could 
> do this for
> per domains VHPTs also, by preallocating an area of 
> contiguous memory at
> boot that is used for allocating memory for VHPTs.

My points are:

Growing or shrinking is not necessary for a global VHPT because
it is scaled to the actual physical memory in the machine
rather than the sum of the virtual physical memory of N
domains.
 
Preallocation is much easier for the global VHPT because it need
not grow or shrink (ignoring hot-plug machine memory) nor is
it proportional to the number of domains.

If the number of domains is dynamic (especially wildly so),
allocating memory for per-domain VHPTs is going to be painful.
And if your solution to this is "if it hurts don't do that"
(meaning don't allow the number of domains to be dynamic or
the amount of (meta)physical memory to be dynamic),
I'd consider that a design problem with per-domain VHPT.

Dan

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel