WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] pre-reservation of memory for domain creation

To: Jan Beulich <JBeulich@xxxxxxxxxx>, Tim Deegan <Tim.Deegan@xxxxxxxxxx>
Subject: RE: [Xen-devel] pre-reservation of memory for domain creation
From: "Xu, Dongxiao" <dongxiao.xu@xxxxxxxxx>
Date: Tue, 9 Feb 2010 16:02:57 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Keir Fraser <Keir.Fraser@xxxxxxxxxxxxx>
Delivery-date: Tue, 09 Feb 2010 00:05:24 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4B70490D020000780002E4AD@xxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <4B4CA40C0200007800029657@xxxxxxxxxxxxxxxxxx> <C7724B76.6265%keir.fraser@xxxxxxxxxxxxx> <4B4CAEB9020000780002969A@xxxxxxxxxxxxxxxxxx> <6CADD16F56BC954D8E28F3836FA7ED7112A79326F2@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx> <4B4D8A9402000078000299C3@xxxxxxxxxxxxxxxxxx> <6CADD16F56BC954D8E28F3836FA7ED7112A7932EBF@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx> <4B4EEB300200007800029E45@xxxxxxxxxxxxxxxxxx> <20100114124619.GE30054@xxxxxxxxxxxxxxxxxxxxxxx> <4B70490D020000780002E4AD@xxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Acqo22KUiNwL8P3USGG4gy8V7RGdJAAbfk2w
Thread-topic: [Xen-devel] pre-reservation of memory for domain creation
Jan Beulich wrote:
>>>> Tim Deegan <Tim.Deegan@xxxxxxxxxx> 14.01.10 13:46 >>>
>> At 09:00 +0000 on 14 Jan (1263459616), Jan Beulich wrote:
>>>>>> "Xu, Dongxiao" <dongxiao.xu@xxxxxxxxx> 14.01.10 08:19 >>>
>>>>    Currently guest initialization process in xend
>>>> (XendDomainInfo.py) is: 
>>>> 
>>>>    _constructDomain() --> domain_create() --> domain_max_vcpus()
>>>>    ... --> _initDomain() --> shadow_mem_control() ...
>>> 
>>> While the patch certainly matches what I had in mind, with this
>>> sequence 
>>> it is clear that the tools still will need adjustment: The full
>>> ballooning only happens from _initDomain(), and hence the
>>> pre-reservation (from _constructDomain) of 4Mb would still be too
>>> small for large vCPU counts. 
>>> 
>>> I wonder though what all this memory is needed for before the domain
>>> (not to speak of secondary CPUs) actually gets started. If that
>>> could be 
>>> got under control, tools side adjustment would not be necessary.
>>> Tim? 
>> 
>> Hmmm.  Some shadow memory has to be allocated before the VCPUs are
>> initialized so that they can be given monitor pagetables etc.  Some
>> shadow memory has to be allocated before the guest's main memory is
>> assigned because the p2m is built out of shadow memory.
> 
> So is there a way to quantify that? In particular, is that *initial*
> amount in any way dependent on the number of vCPU-s?
> 
>> Fixing the first one should be enough, so long as xend assignd vcpus
>> and memory before assigning shadow memory properly (which I believe
>> it does).  Patch attached.
> 
> The full memory assignment happens after vCPU-s got assigned, but
> the initial assignment happens before. Dongxiao's patch tried to
> account for that, but neither was that patch accepted so far, nor am I
> convinced this is really correct or even necessary.

The patch I attached last time could not solve this issue. The reason is
the same that, at the point when shadow allocates memory for each vcpu,
xend hasn't ballooned out enough memory. 

I discussed this issue within our team, however we could't achieve a
good solution currently since Chinese New Year vacation will soon start.
Anyway I made a work-around patch for it, though you may not like it. 

Thanks!
Dongxiao

Xend: Enlarge the memory balloon size for domain creation since shadow
pre-allocation size has changed from 1M to 4M.

Signed-off-by: Dongxiao Xu <dongxiao.xu@xxxxxxxxx>

diff -r 5b895c3f4386 tools/python/xen/xend/XendDomainInfo.py
--- a/tools/python/xen/xend/XendDomainInfo.py   Mon Feb 08 10:18:51 2010 +0000
+++ b/tools/python/xen/xend/XendDomainInfo.py   Tue Feb 09 15:07:47 2010 +0800
@@ -2519,9 +2519,8 @@ class XendDomainInfo:
         # There is an implicit memory overhead for any domain creation. This
         # overhead is greater for some types of domain than others. For
         # example, an x86 HVM domain will have a default shadow-pagetable
-        # allocation of 1MB. We free up 4MB here to be on the safe side.
-        # 2MB memory allocation was not enough in some cases, so it's 4MB now
-        balloon.free(4*1024, self) # 4MB should be plenty
+        # allocation of 4MB. We free up 16MB here to be on the safe side.
+        balloon.free(16*1024, self) # 16MB should be plenty

         ssidref = 0
         if security.on() == xsconstants.XS_POLICY_USE:


> 
> I'm re-raising this question because we're not seeming to make any
> progress towards a satisfactory resolution of the regression c/s 20389
> introduced.
> 
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel