Re: [Xen-devel] [PATCH] PoD: Handle operations properly when domain is dying

[George Dunlap <george.dunlap@xxxxxxxxxxxxx>]

Dan,

PoD memory is pre-allocated to a VM.  A PoD fault will never cause a 
domain to allocate any more memory than has already been allocated to 
it.  That's why it's called "populate on demand" instead of "allocate on 
demand".
(The failure mode in this case was this:
* domain_kill() called
* pod_empty_cache() called
* grant table op calls gfn_to_mfn on a PoD page
* pod_demand_populate finds the cache empty, and runs an emergency scan
* emergency scan pulls some unfreed pages from the p2m table to put in 
the pod cache
* p2m pages freed, p2m table torn down
* memory inthe pod cache keeps domain from dying.
So no new memory was being allocated; it was just being transferred 
from  a place which hadn't been destroyed yet to a place which had.)
WRT ballooning: the tools should be telling the balloon driver what to 
do; a well-behaved balloon driver will not allocate more memory unless 
the tools tell it to do so.  There are mechanisms in Xen available to 
the tools to limit the amount of memory a balloon driver can allocate 
even if it's not behaving.
So as long as the tools don't tell a VM to change its memory between 
deciding there's enough memory for the new domain and creating the new 
domain, you shouldn't have any problems.
-George

Dan Magenheimer wrote:
> Hi George --
>
> A possibly related issue came up in a team discussion
> yesterday.  It is not uncommon for management tools to
> check available memory to decide if there is sufficient
> space to provision a new domain, for example when choosing
> a machine on which to start a new domain.  But there is
> no "lock" so if the answer is yes and the machine
> is chosen, but moments later some memory becomes used
> (by PoD or ballooning or ???), domain creation might
> fail due to insufficient memory.
>
> Tmem has a "freeze" feature to avoid this problem, but
> tmem has the advantage that all domains will continue
> to function even if tmem is frozen... I'm not sure
> that's true for PoD.
>
> Clearly management tools can be very conservative
> and assume every domain on a machine is using its
> maxmem, but that defeats the purpose of much of the
> overcommitment work we are doing.  I know Citrix
> is working on hypervisor swap, but that has a pretty
> horrible performance penalty, as well as some
> interesting functionality limitations/challenges.
>
> Any clever ideas on how to deal with this problem
> in the future?  For example, assume someone wants
> to launch a sequence of PoD domains that will be
> idle when launched but eventually will utilize all
> (or even most) of maxmem.
>
> Thanks,
> Dan
>
>
>   
> > -----Original Message-----
> > From: George Dunlap [mailto:George.Dunlap@xxxxxxxxxxxxx]
> > Sent: Wednesday, November 11, 2009 8:08 AM
> > To: xen-devel@xxxxxxxxxxxxxxxxxxx
> > Subject: [Xen-devel] [PATCH] PoD: Handle operations properly 
> > when domain
> > is dying
> >
> >
> > Handle PoD operations properly when a domain is dying.
> >
> > No populate-on-demand activities should happen when a domain is dying.
> >  Especially, it is a bug for memory to be added to the PoD cache when
> > d->is_dying is non-zero, since if this happens after the cache has
> > been emptied, these pages will never be freed. This may cause "zombie
> > domains" to linger.
> >
> > Signed-off-by: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
> >
> >     
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel