This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] [PATCH][RFC]Remove lock on first guest table walk

To: "Tim Deegan" <Tim.Deegan@xxxxxxxxxx>
Subject: [Xen-devel] [PATCH][RFC]Remove lock on first guest table walk
From: "Tian, Kevin" <kevin.tian@xxxxxxxxx>
Date: Thu, 21 Feb 2008 18:37:06 +0800
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Thu, 21 Feb 2008 02:38:06 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Ach0dbQB1GtH5UXySRuhL+6iMEA7jQ==
Thread-topic: [PATCH][RFC]Remove lock on first guest table walk
Just found this patch in house which I thought sent out
already. :-)

Still one thing remaining is to special case for UP guest
which suffers overhead from this change...

Remove lock on first guest table walk

Existing shadow fault path grabs big lock before walking
guest tables, to ensure consistency with shadow content
lest concurrent change from other vcpu in a bad OS.
But this lock brings more lock contention when scaled up
for a good guest which already prevents above case happen.
So this patch tries to remove the lock on first guest
table walk, and then delay check at some special points.

The key is to check whether any guest table update happens
between 1st walk and holding shadow lock. Here we take
two hints for guest table update:
    * write permission removal
    * write emulation
If any above two operations are observed within the race
window, it indicates possiblity that previous walk result
may be inaccurate and re-check is requried. If mismatch,
simply return to trigger another fault.

I made some experiment to sample perfc count for kernel
<64bit guest>
3.7% of gwalks are re-checked
For re-check, 68% comes from write permission removal
<32bit pae guest>
7.2% of gwalks are re-checked
For re-check, 54.9% comes from write permission removal

Actually previous fast emulation optimization already skip
lots of guest table walks, and thus above ratio can be
smaller if compared to total shadow fault count.

Basically shadow promotion with write permision removal
does suffer higher overhead, but the benefit to reduce
lock contention is more obvious.

Improvement on kernel compile for this patch is:
(64bit Xen, with 2 vcpus guest and 2 physical cpus)
32bit guest: 1.1%
pae guest:   0.4%
64bit guest: 0.5%

Signed-off-by Kevin Tian <kevin.tian@xxxxxxxxx>


Attachment: gwalk_no_lock.patch
Description: gwalk_no_lock.patch

Xen-devel mailing list
<Prev in Thread] Current Thread [Next in Thread>