This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Re: Poor performance on HVM (kernbench)

To: "Gianluca Guida" <gianluca.guida@xxxxxxxxxxxxx>
Subject: Re: [Xen-devel] Re: Poor performance on HVM (kernbench)
From: "George Dunlap" <dunlapg@xxxxxxxxx>
Date: Thu, 11 Sep 2008 19:07:56 +0100
Cc: Muli Ben-Yehuda <MULI@xxxxxxxxxx>, deshantm@xxxxxxxxx, Anthony Liguori <aliguori@xxxxxxxxxx>, xen-devel mailing list <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Thu, 11 Sep 2008 11:08:26 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=oKZ4FvSSg1b7JPvBHBqn9+uJivTjWz+7WTdyvx8dcFU=; b=O4HWiW3ZUrEtS9EdbpbVla99Ujxyrc7gjxZlkuKQ+UhreFDaUUwBXRfur2URzMfrog 0EkZgSos158EMkWGberZMrAHl87xLvR2IkZomHoJiHbuybbmnupVWoJseV7Rx2+iQFef gSXHSmCYKfnkBkZIXUbn0mYjZ8fzKmMvUyqws=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=Tgfc5Zf2qO2YNOy+xzhznLu8fOTRc8VMJf+sx1cd4U4n9mXHy7qIaGmjC/5yWUPsNz iXaY1ZCMyZiFnLZJmgUXEtdY0IosA3nz9e2QxPss4gHedyrcruVRwqrGlD3/WFpXntT2 83EIOFpS1K1ah66zKGu6oUdbfkhJpySGi10cY=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <48C9548C.4060709@xxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <1e16a9ed0809101123m71a12030v7d06501f6467f93@xxxxxxxxxxxxxx> <1e16a9ed0809101422p6a58304dxaa1a92847109a518@xxxxxxxxxxxxxx> <48C8EC48.6080507@xxxxxxxxxxxxx> <48C9367B.6090003@xxxxxxxxxxxxx> <1e16a9ed0809110835p73165150t3f78d75b2f2ca8eb@xxxxxxxxxxxxxx> <48C9548C.4060709@xxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
So, the problem appears to be with a ton of brute-force searches to
remove writable mappings, both during resync and promotion.  My
analysis tool is reporting that of the 30 seconds or so in the trace
from xen-unstable, the guest spent a whopping 67% in the hypervisor:
 * 26% doing resyncs as a result of marking another page out-of-sync
 * 9% promoting pages
 * 27% resyncing as a result of cr3 switches
And almost the entirety of all of those can be attributed to
brute-force searches to remove writable mappings.

(Caveat emptor: My tool was designed to analyze XenServer product
traces, which have a different trace file format than xen-unstable.
I've just taught it to read the xen-unstable trace formats, so the
exact percentages may be incorrect still.  But the preponderance of
brute-force searches is unmistakable.)

The good news is that if we can finger the cause of the brute-force
searches, we should be able to reduce all those numbers down to
respectable levels; my guess is totaling not more than 5%.


On Thu, Sep 11, 2008 at 6:25 PM, Gianluca Guida
<gianluca.guida@xxxxxxxxxxxxx> wrote:
> Todd Deshane wrote:
>> Can you also run kernbench on native for comparison?
> Here tehy are, with a two CPUs dom0.
> Thu Sep 11 13:03:57 EDT 2008
> Average Optimal load -j 8 Run (std deviation):
> Elapsed Time 181.51 (0.550318)
> User Time 300.494 (0.965572)
> System Time 54.198 (0.784391)
> Percent CPU 195 (0.707107)
> Context Switches 26611.8 (205.029)
> Sleeps 29778.8 (330.637)
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>