[Xen-devel] Populate-on-demand memory problem

To:	xen-devel@xxxxxxxxxxxxxxxxxxx
Subject:	[Xen-devel] Populate-on-demand memory problem
From:	Dietmar Hahn <dietmar.hahn@xxxxxxxxxxxxxx>
Date:	Tue, 27 Jul 2010 09:48:44 +0200
Cc:	George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
Delivery-date:	Tue, 27 Jul 2010 00:49:35 -0700
Dkim-signature:	v=1; a=rsa-sha256; c=simple/simple; d=ts.fujitsu.com; i=dietmar.hahn@xxxxxxxxxxxxxx; q=dns/txt; s=s1536b; t=1280216972; x=1311752972; h=from:to:subject:date:cc:mime-version: content-transfer-encoding:message-id; z=From:=20Dietmar=20Hahn=20<dietmar.hahn@xxxxxxxxxxxxxx> \|To:=20xen-devel@xxxxxxxxxxxxxxxxxxx\|Subject:=20Populate- on-demand=20memory=20problem\|Date:=20Tue,=2027=20Jul=2020 10=2009:48:44=20+0200\|Cc:=20George=20Dunlap=20<George.Dun lap@xxxxxxxxxxxxx>\|MIME-Version:=201.0 \|Content-Transfer-Encoding:=207bit\|Message-Id:=20<2010072 70948.44604.dietmar.hahn@xxxxxxxxxxxxxx>; bh=DhtG0TbuDh9zWbC+TX0bk2PQpFyyS5jF9ccPiSu0Mig=; b=QpGSg+FWt+nGhmNYRhL9I+BDQeq3T4Hb6kPsSl32R1XiUWZIVSwTOghj BgzFQrdFEqsZ1jAdjXyeWz5ckW2SSepJvvkUoTzqXPC6ezNwDFfIMjjE7 KqLbS9S5eUlFXuXLI+p0jn5+xCn9rcPEnNEjRHXzv4J+5j+jg1BeY0mIU c7Ar8PCDk4EfjCnG6eevxkj8TL9sPS9+BzGYPbWGgrf2Ue0O3wqyRgoSp PHca9dA3bhyHq4HGvNGsCX1G+eq3M;
Domainkey-signature:	s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:From:To:Subject:Date:User-Agent:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding: Message-Id; b=KRg+eWV9kL1RQRY0dTyDZU06Q7mRLy3jtkU3bmQ8X0FOIV44ByLRQNYC 7nAftOhMj9Q30TMuprhEyurF5GmzyNs5CHdG/gbrld4hbj36ju/K5BE1m fczq46RpPXy5xxQoOKkJs2V3nrlp9+EX4732WLXkl0XREcVXzqxGkV/nY JP23glObSNpqaWj4/awsF467Jhg+fI8FYBKil2Zm/+/WxCuJSApkBLacg 1y+h21oPsbxP52X+9MHFGaicuvQzr;
Envelope-to:	www-data@xxxxxxxxxxxxxxxxxxx
List-help:	<mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id:	Xen developer discussion <xen-devel.lists.xensource.com>
List-post:	<mailto:xen-devel@lists.xensource.com>
List-subscribe:	<http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe:	<http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender:	xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent:	KMail/1.13.5 (Linux/2.6.31.13-xen-hahn; KDE/4.4.5; x86_64; ; )

Hi list,

we ported our system from Novel SLES11 using xen-3.3 to SLES11 SP1 using
xen-4.0 and ran into some trouble with the pod stuff.
We have a HVM guest and already used target_mem < max_mem on startup of
the guest.
With the new xen version we get
(XEN) p2m_pod_demand_populate: Out of populate-on-demand memory! tot_pages 
792792 pod_entries 800
I did some code revisions and looking at pod patches 
(http://lists.xensource.com/archives/html/xen-devel/2008-12/msg01030.html)
to understand the behavior. We use the following configuration:
maxmem = 4096
memory = 3096
What I see is:
 - our guest boots with e820 map showing maxmem.
 - reading xenstore memory/target returns '3170304' means 3096MB, 792576 pages
Now our guest uses the target memory and gives back 1000MB via
hypervisor call XENMEM_decrease_reservation to the hypervisor.

Later I try to map the complete domU memory into dom0 kernel space and here I
get the 'Out of populate-on-demand memory' crash.

As far as I understand (ignoring the p2m_pod_emergency_sweep)
- on populating a page
   - the page is taken from the pod cache
   - p2md->pod.count--
   - p2md->pod.entry_count--
   - page gets type p2m_ram_rw
- decreasing a page
   - p2md->pod.entry_count--
   - page gets type p2m_invalid

So if the guest uses all the target memory and gave back all
the (maxmem-target) memory p2md->pod.count and p2md->pod.entry_count should be
zero.
I added some tracing in the hypervisor and see on start of the guest:
p2m_pod_set_cache_target: p2md->pod.count: 791264 tot_pages: 791744
This pod.count is lower then the target seen in the guest!
On the first call of p2m_pod_demand_populate() I can see
p2m_pod_demand_populate: p2md->pod.entry_count: 1048064 p2md->pod.count: 791264 
tot_pages: 792792
So pod.entry_count=1048064 (4096MB) complies to maxmem but
pod.count=791264 is lower then the target memory in xenstore.

Any help is welcome!
Thanks.
Dietmar.

-- 
Company details: http://ts.fujitsu.com/imprint.html

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

WARNING - OLD ARCHIVES

xen-devel

[Xen-devel] Populate-on-demand memory problem