This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Performance difference between Xen versions

To: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] Performance difference between Xen versions
From: Keir Fraser <keir.xen@xxxxxxxxx>
Date: Fri, 29 Apr 2011 15:58:55 +0100
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Fri, 29 Apr 2011 08:01:07 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:user-agent:date:subject:from:to:cc:message-id :thread-topic:thread-index:in-reply-to:mime-version:content-type :content-transfer-encoding; bh=J/3WNhk23KT6dfGc1Fp+f5QURtvP2n6GfXgm9VgFqyQ=; b=nGoQHyHxtlqdY7faqN8BW3dnzZefSdt+/W/nN9OGWEShrdv9CdGCDEyXFMOcmGYZQv QeBaApPlL517FVDT9YKr8002lnvLekP1RJeV/2PCc+tyYuMw2wxumDq7Twg8TCF/yzQ3 b/LWyaNDDM/SoVav2rIPrRGKhk2m06OCEhPyo=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :thread-index:in-reply-to:mime-version:content-type :content-transfer-encoding; b=GNExBB/zJ2qnc8cL5Pjoqu5kwA6DL+HA+A/grHO3BKDZ7EwTPWfPSgwk6igxH0l78H 4g1GWg8VZ9WN2YNw41DOrRze1fnutMpazDVcrBCGmgVDkjT1RTAfVNgOfsCTcWUSVaXA nmzJQi9xNHzIwO+Q319GPfXtuSJDki/DgUfmo=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4DBABEAB.6090906@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcwGffW3XWrLXuHIu026yA8vF9Oomw==
Thread-topic: [Xen-devel] Performance difference between Xen versions
User-agent: Microsoft-Entourage/
On 29/04/2011 14:35, "Juergen Gross" <juergen.gross@xxxxxxxxxxxxxx> wrote:

> On 04/29/11 15:28, Keir Fraser wrote:
>> Are you sure TSC runs at the same rate in the guest on both hypervisor
>> versions? Xen 4.0 might trap and emulate a more consistent but slower rate
>> TSC by default. 'tsc_mode=2' in your domain config file on 4.0 might be a
>> quick fix.
> Already done :-), so yes, I am sure the tsc rate is the same. The debug key
> 's' (softTSC stats) shows that no tsc is emulated.
> BTW: different tsc rate is improbable as the memory access loop shows
> nearly the same tsc difference...

Then I'm not sure. Maybe something got added to the VMEXIT/VMENTRY path that
is unexpectedly slow. You'll have to do a bit of digging.

 -- Keir

> Juergen
>>   -- Keir
>> On 29/04/2011 13:32, "Juergen Gross"<juergen.gross@xxxxxxxxxxxxxx>  wrote:
>>> Hi,
>>> comparing performance of different Xen versions with BS2000 as HVM guest
>>> showed some weird data I'd like to understand.
>>> All measurements were done on an Intel Xeon E7220 box. We used a disk-
>>> benchmark and found the cpu utilization was much higher with Xen 4.0
>>> compared
>>> to Xen 3.3. I did some more investigation and narrowed things down to calls
>>> of
>>> the hypervisor (implicit or explicit).
>>> Following is a table with timing data for different low-level functions, all
>>> timing values are tsc ticks obtained via rdtsc:
>>> Xen 3.3     Xen 4.0      Function
>>>         88        165      just the measurement overhead
>>>        176        330      rdtsc-instruction + cli/sti
>>>       5896      11044      lapic timer query
>>>       7381      13519      setting lapic timer
>>>       4653       8987      reload of cr3
>>>       3124       5709      invlpg instruction
>>>     792253     792264      wbinvd instruction
>>>        748       1375      int + iret
>>>       5203       9317      hypervisor yield call
>>> 12598102   12597882      memory access loop
>>> All operations involving the hypervisor take nearly twice the time on 4.0.
>>> Operations not involving the hypervisor (wbinvd and memory access loop) are
>>> the same on both systems (this rules out the possibility of different rdtsc
>>> behavior).
>>> Is there any easy explanation for this? Both Xen versions are from SLES
>>> (SLES11 or SLES11 SP1).
>>> Juergen

Xen-devel mailing list