This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] what happens when a PoD page is touched?

To: James Harper <james.harper@xxxxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] what happens when a PoD page is touched?
From: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
Date: Tue, 17 May 2011 14:00:47 +0100
Cc: Tim Deegan <Tim.Deegan@xxxxxxxxxxxxx>, Paul Durrant <Paul.Durrant@xxxxxxxxxx>, xen devel <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Tue, 17 May 2011 06:03:22 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=VwRgteNolBmAqnDlL3DhfqnRUHJxq7L7IxOZeNnu1SI=; b=PT8IGNtfCiFzbSTtOzfsb0AVIwphj/kqN3k6HsQJNUs7lgM4em66NjFrZ/tMMDnGXU wgI1skQlXmtCnZwxDgm59wLETHy3FAVMGqM8K6gY/NMlIKJ6DW8/KH/aFv9Jip4HVNRF ravrR1M5s6yjx83JRSbk+7HAy9MBgdomvyZNw=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=kh18BwAZJgivPZhRYNp30INJ63S1/ObGgmxLX/ZVoPbbtdlZ2mKh690rz9VpKtCV11 W4llrbK2dZpCO1VUHvbsoobWpudgnopGIq0BY2ZO1zYg9yNOEIYRnQ8MKXgOxfGVeXWv YLU7s4E0oIq9bEz1E0NysUnNx/wGv9wRmt46g=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <AEC6C66638C05B468B556EA548C1A77D01D5714A@trantor>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <AEC6C66638C05B468B556EA548C1A77D01D57078@trantor> <20110516083905.GP24068@xxxxxxxxxxxxxxxxxxxxxxx> <291EDFCB1E9E224A99088639C4762022B37FC178F9@xxxxxxxxxxxxxxxxxxxxxxxxx> <AEC6C66638C05B468B556EA548C1A77D01D570EB@trantor> <291EDFCB1E9E224A99088639C4762022B37FC178FA@xxxxxxxxxxxxxxxxxxxxxxxxx> <AEC6C66638C05B468B556EA548C1A77D01D570F6@trantor> <291EDFCB1E9E224A99088639C4762022B37FC17A16@xxxxxxxxxxxxxxxxxxxxxxxxx> <AEC6C66638C05B468B556EA548C1A77D01D5714A@trantor>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Tue, May 17, 2011 at 10:37 AM, James Harper
<james.harper@xxxxxxxxxxxxxxxx> wrote:
> I still can't quite get my head around why this happens at all... I
> thought it would go like this:
> 1. Allocate 1MB of memory
> 2. Still under our limit so xen populates the pages when Windows clears
> them
> 3. Hand them back to xen
> 4. Repeat
> I'm doing that in a tight loop very early in boot. If I keep handing
> back pages (and thus reducing my populated page count) why am I hitting
> any PoD limit at all and invoking the page scavenging code? Windows
> isn't doing anything else at this point, and even if it was, I'm the
> boot driver so it has to wait for me before the boot can progress so
> it's not like it would be consuming gigabytes of memory.

Suppose for simplicity we're using 1G pages and not splintering them.

Suppose we boot a VM with 8G maxmem, 4G memory.

* Domain built
pages 0-7 are PoD, 4 pages in PoD pool.

* HVMLoader loaded into memory
Page 0 populated; 1-7 PoD, 3 pages in PoD pool

* Windows boots, starts scrubbing (from top of memory)
 + page 7 populated, 2 pages in PoD pool.
 + Page 6 populated, 1 page in PoD pool
 + Page 5 populated, 0 pages in PoD pool
 + Page 4 touched
  - emergency sweep marks page 7 PoD; page 4 populated
 + Page 3 touched
  - emergency sweep marks page 6 PoD; page 3 populated
 + page 1 touched
  - emergency sweep marks page 5 PoD; page 1 populated
 + Result at the end: pages 0-4 populated, 5-7 PoD, no pages in PoD pool

* Balloon driver starts.
 Let's suppose Windows happens to do all allocation from the PoD region.
 + Asks windows for 1 page
  - Windows chooses page 6, scrubs it.
  - emergency sweep marks page 4 PoD, populates page 6
  - Balloon driver hands page 6 back to Xen
  - page 5 marked empty; page put in PoD pool
 + Asks Windows for 1 more page
  - Windows chooses page 5, scrubs it.
  - Xen fills page 5 with the page in the PoD pool; PoD now empty
  - Balloon driver hands page 5 back to Xen
  - page 5 marked empty; page put in PoD pool
 + Repeat previous

So in theory, once you get your initial allocation from Windows, you
shouldn't have to do any sweeps.

The exception to this would be the case where you're allocating N
pages, where N > (used_pages - total allocation).

I guess try allocating fewer pages before returning them to Xen, and
seeing if that helps at all.

Hmm -- looks like I didn't upstream my PoD tracing patch, otherwise we
could see what was going on.  If you're interested in taking a look at
what's happening at a PoD level, I can try to dig it up and send it to
you.  (Probably need some porting because of the p2m restructuring Tim
Deegan has been doing.)


Xen-devel mailing list