Re: [Xen-devel] [PATCH] PoD: Handle operations properly when domain is d
PoD memory is pre-allocated to a VM. A PoD fault will never cause a
domain to allocate any more memory than has already been allocated to
it. That's why it's called "populate on demand" instead of "allocate on
(The failure mode in this case was this:
* domain_kill() called
* pod_empty_cache() called
* grant table op calls gfn_to_mfn on a PoD page
* pod_demand_populate finds the cache empty, and runs an emergency scan
* emergency scan pulls some unfreed pages from the p2m table to put in
the pod cache
* p2m pages freed, p2m table torn down
* memory inthe pod cache keeps domain from dying.
So no new memory was being allocated; it was just being transferred
from a place which hadn't been destroyed yet to a place which had.)
WRT ballooning: the tools should be telling the balloon driver what to
do; a well-behaved balloon driver will not allocate more memory unless
the tools tell it to do so. There are mechanisms in Xen available to
the tools to limit the amount of memory a balloon driver can allocate
even if it's not behaving.
So as long as the tools don't tell a VM to change its memory between
deciding there's enough memory for the new domain and creating the new
domain, you shouldn't have any problems.
Dan Magenheimer wrote:
Hi George --
A possibly related issue came up in a team discussion
yesterday. It is not uncommon for management tools to
check available memory to decide if there is sufficient
space to provision a new domain, for example when choosing
a machine on which to start a new domain. But there is
no "lock" so if the answer is yes and the machine
is chosen, but moments later some memory becomes used
(by PoD or ballooning or ???), domain creation might
fail due to insufficient memory.
Tmem has a "freeze" feature to avoid this problem, but
tmem has the advantage that all domains will continue
to function even if tmem is frozen... I'm not sure
that's true for PoD.
Clearly management tools can be very conservative
and assume every domain on a machine is using its
maxmem, but that defeats the purpose of much of the
overcommitment work we are doing. I know Citrix
is working on hypervisor swap, but that has a pretty
horrible performance penalty, as well as some
interesting functionality limitations/challenges.
Any clever ideas on how to deal with this problem
in the future? For example, assume someone wants
to launch a sequence of PoD domains that will be
idle when launched but eventually will utilize all
(or even most) of maxmem.
From: George Dunlap [mailto:George.Dunlap@xxxxxxxxxxxxx]
Sent: Wednesday, November 11, 2009 8:08 AM
Subject: [Xen-devel] [PATCH] PoD: Handle operations properly
Handle PoD operations properly when a domain is dying.
No populate-on-demand activities should happen when a domain is dying.
Especially, it is a bug for memory to be added to the PoD cache when
d->is_dying is non-zero, since if this happens after the cache has
been emptied, these pages will never be freed. This may cause "zombie
domains" to linger.
Signed-off-by: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
Xen-devel mailing list