This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Shadow Page Tables in Xen

To: priya sehgal <priyagps@xxxxxxxxxxx>
Subject: Re: [Xen-devel] Shadow Page Tables in Xen
From: Gianluca Guida <glguida@xxxxxxxxx>
Date: Wed, 22 Apr 2009 15:18:39 +0200
Cc: Tim Deegan <Tim.Deegan@xxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Wed, 22 Apr 2009 06:19:12 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=jtYjE4RMGYAnRGYS/eawHxQYMl9dPhGJkWNyV3vc+5M=; b=wA6Y4OrOYVyjLRYJ6CEHfMFOZJVxiD5Qp17VLY5+Mg3OKqJydHMnJyqS6jD6J60iD4 KEHXVtBIX68ZjRnM94Ye+Dp4JpLj+cgLj/OKepKpTEXVoZs2TPM4gokJbgcW+T2PidEh rrlTJdNc0nPmssntSrz6rIvF+A99U5BesX798=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=Wfq0DbG9ltl2P7b62GyAB+4l2I9A7kSq5z41oxQQ0NJjbhE+PI1OLVR91XTlyXv8EU z4jo8LY33PAneDH3bHB6Ad3ucyeRWnmKQi46Ao9Z7RDvtV6J8RU0/03LsFBRF58J1RSQ aQe1GI/34TOmqZqVAq3ZGHS5PG1FzU752O2h0=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <573035.12343.qm@xxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <573035.12343.qm@xxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx

On Apr 22, 2009, at 3:02 PM, priya sehgal wrote:

Hi Gianluca,

There are various problems I can see with this approach:

- As Tim suggested, this will make the bandwidth required
to do live
migration much bigger (you're talking about increasing the
of memory to be sent from 1 to 1000 pages). So you should
take into
account that yes, making bigger logdirty chunks will
decrease the
pagefaults, but will increase the required network
bandwidth, which is
a very important parameter for live migration.

I think there is one thing missing in what I explained.
If a page in a group is written for the first time, all its neighbors
are marked as RW. But, their dirty bit might be off. Now, we maintain
a dirty group bitmap, which just stores the page groups that are dirty.
During the end of an epoch, when we are about to ship the pages
to the destination machine, we first check the groups which are dirty. Scan through *each* page's PTE corresponding to that group to determine
if it is DIRTY. Only if the page is DIRTY is it sent over the network.
So, we do not send unwanted pages , only the ones marked DIRTY in the
RW-enabled groups.

Ok that seems, but please note that at the moment shadows always set the DIRTY bit (unless we're tracking a VRAM address), so you might have to set the dirty bit clean when propagating the shadow, which is a bit too intrusive perhaps. But yes, that would work.

Also, you mentioned that the main performance bottleneck is due to blowing up of the shadow page tables. Why is this required? Can't the page table entries that were migrated in the last epoch be cleaned up only and reset. Could you please elaborate on this ?

Yes. We could do that, as Ian suggested. There was no effort in making live migration's shadow code fast, so nobody ever did it. Might be good to do it, if you're interested in this case, and will be much cleaner than your suggested effort, and perhaps way more interesting.

This comment, found in shadow_clean_dirty_bitmap might explain you a lot:

    /* Need to revoke write access to the domain's pages again.
     * In future, we'll have a less heavy-handed approach to this,
     * but for now, we just unshadow everything except Xen. */


Xen-devel mailing list