This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] [PATCH] PoD: Handle operations properly when domain is d

To: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>
Subject: Re: [Xen-devel] [PATCH] PoD: Handle operations properly when domain is dying
From: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
Date: Wed, 11 Nov 2009 15:42:23 +0000
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Wed, 11 Nov 2009 07:42:46 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <d6b6a89e-3f42-499d-8a19-fd574dab7a16@default>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <d6b6a89e-3f42-499d-8a19-fd574dab7a16@default>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird (X11/20090817)

PoD memory is pre-allocated to a VM. A PoD fault will never cause a domain to allocate any more memory than has already been allocated to it. That's why it's called "populate on demand" instead of "allocate on demand".

(The failure mode in this case was this:
* domain_kill() called
* pod_empty_cache() called
* grant table op calls gfn_to_mfn on a PoD page
* pod_demand_populate finds the cache empty, and runs an emergency scan
* emergency scan pulls some unfreed pages from the p2m table to put in the pod cache
* p2m pages freed, p2m table torn down
* memory inthe pod cache keeps domain from dying.
So no new memory was being allocated; it was just being transferred from a place which hadn't been destroyed yet to a place which had.)

WRT ballooning: the tools should be telling the balloon driver what to do; a well-behaved balloon driver will not allocate more memory unless the tools tell it to do so. There are mechanisms in Xen available to the tools to limit the amount of memory a balloon driver can allocate even if it's not behaving.

So as long as the tools don't tell a VM to change its memory between deciding there's enough memory for the new domain and creating the new domain, you shouldn't have any problems.


Dan Magenheimer wrote:
Hi George --

A possibly related issue came up in a team discussion
yesterday.  It is not uncommon for management tools to
check available memory to decide if there is sufficient
space to provision a new domain, for example when choosing
a machine on which to start a new domain.  But there is
no "lock" so if the answer is yes and the machine
is chosen, but moments later some memory becomes used
(by PoD or ballooning or ???), domain creation might
fail due to insufficient memory.

Tmem has a "freeze" feature to avoid this problem, but
tmem has the advantage that all domains will continue
to function even if tmem is frozen... I'm not sure
that's true for PoD.

Clearly management tools can be very conservative
and assume every domain on a machine is using its
maxmem, but that defeats the purpose of much of the
overcommitment work we are doing.  I know Citrix
is working on hypervisor swap, but that has a pretty
horrible performance penalty, as well as some
interesting functionality limitations/challenges.

Any clever ideas on how to deal with this problem
in the future?  For example, assume someone wants
to launch a sequence of PoD domains that will be
idle when launched but eventually will utilize all
(or even most) of maxmem.


-----Original Message-----
From: George Dunlap [mailto:George.Dunlap@xxxxxxxxxxxxx]
Sent: Wednesday, November 11, 2009 8:08 AM
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] [PATCH] PoD: Handle operations properly when domain
is dying

Handle PoD operations properly when a domain is dying.

No populate-on-demand activities should happen when a domain is dying.
 Especially, it is a bug for memory to be added to the PoD cache when
d->is_dying is non-zero, since if this happens after the cache has
been emptied, these pages will never be freed. This may cause "zombie
domains" to linger.

Signed-off-by: George Dunlap <george.dunlap@xxxxxxxxxxxxx>

Xen-devel mailing list