This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] Re: non-zero order allocations in shadow code may prevent li

To: "Tim Deegan" <Tim.Deegan@xxxxxxxxxxxxx>
Subject: [Xen-devel] Re: non-zero order allocations in shadow code may prevent live migration
From: "Jan Beulich" <jbeulich@xxxxxxxxxx>
Date: Thu, 27 Sep 2007 12:19:21 +0100
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Thu, 27 Sep 2007 04:18:54 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <20070927104141.GA23760@xxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <46FB995B.76E4.0078.0@xxxxxxxxxx> <20070927104141.GA23760@xxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>Gosh.  Are you running with almost all memory in use and failing to
>allocate shadow memory?  Have you seen sh_set_allocation return -ENOMEM
>when there is enough memory on the page allocator's free lists?  Xend's
>ballooning rules have been wrong more than once before, and are what I'd
>suspect first.

Yes, that's the scenario (64G physical memory, one PV domain started with 4G
assigned, live migration back and forth between two hosts, perhaps intertwined
with other domain creation activities, plus perhaps ballooning Dom0 back up 
domain termination). So no, I verified there's about 21M free (xm info) before
migration starts (or after it failed), and the tools estimate the need 

DEBUG (balloon:146) Balloon: 21888 KiB free; need 21504; done.

while shadow still fails (some of the messages might be temporary debugging
aids of ours):

(XEN) sh error: set_sh_allocation(): current 0 target 4608
(XEN) sh error: set_sh_allocation(): failed to allocate shadow pages.
(XEN) sh error: set_sh_allocation(): current 4212 target 0
(XEN) sh error: shadow_one_bit_enable(): shadow_one_bit_enable() failed memory 
(XEN) sh error: shadow_log_dirty_enable(): shadow_log_dirty_enable() received 
(errno = -12) from shadow_one_bit_enabled()

4608 pages are just 18432k, so there is about 3M of fragmented and hence
unusable space.

So then I'll go ahead with implementing the described change (I'm actually
intending to have shadow_prealloc() take not just an order, but also a count
parameter - in a number of places it is being called with SHADOW_MAX_ORDER
for no reason other than wanting 3 or 4 single pages).


Xen-devel mailing list