This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Performance difference between Xen versions

To: Keir Fraser <keir.xen@xxxxxxxxx>
Subject: Re: [Xen-devel] Performance difference between Xen versions
From: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Date: Fri, 29 Apr 2011 15:35:39 +0200
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Fri, 29 Apr 2011 06:36:37 -0700
Dkim-signature: v=1; a=rsa-sha256; c=simple/simple; d=ts.fujitsu.com; i=juergen.gross@xxxxxxxxxxxxxx; q=dns/txt; s=s1536b; t=1304084143; x=1335620143; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=Jt6rrZnc718xY984YqsELgnJPvOuyHGKGUkR4fR24kc=; b=PCyT5KQFDvrojf4d82YspPsjYK8pzTa476CcbNt1m/u8SUlkNg2VMf/7 qswYVMdrjWPcrIo6b0fJhBn9fLtPGnP8JFAyO9Q7v64+O60RlYtldHch8 ogESCc/a7ALea1h6BPkwYvi+b42UCC11axCSsVei7IMxEhIFQzF0JTeDo Nbq+1i44JsOgzIe9jhkcvjExvcK3NQ1QE93VbG5jK66p03+dBWqtTUFVe Arc59TJs4XB0rggQQjdMDiA6JOeTh;
Domainkey-signature: s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=bcX6sl5kjv/TcbOO9raHRo/wgkMCNlFknjlcIoAOZcOkMg4JrbsteTPP wHZsmwAY1hdYRMvW1kMbaCI8ek0qZlkGScKnh+nrliq5dRUo+1XdW/ohb LHYq/ste6ePHZ4u3jfTJROz1Jk9LUtqcCfYIC31NwXFLCwYeSqj8N3ipQ 7ZIUDxuXSuQlLLknfCHF2ko725Vvg3Y1qDwyMGISbD0T3gBjlEuRVwWkN rnaKACjPnhwM0eoT0ApS0Vzi3zVDl;
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <C9E07B7A.16FBD%keir.xen@xxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: Fujitsu Technology Solutions
References: <C9E07B7A.16FBD%keir.xen@xxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20110402 Iceowl/1.0b2 Icedove/3.1.9
On 04/29/11 15:28, Keir Fraser wrote:
Are you sure TSC runs at the same rate in the guest on both hypervisor
versions? Xen 4.0 might trap and emulate a more consistent but slower rate
TSC by default. 'tsc_mode=2' in your domain config file on 4.0 might be a
quick fix.
Already done :-), so yes, I am sure the tsc rate is the same. The debug key
's' (softTSC stats) shows that no tsc is emulated.

BTW: different tsc rate is improbable as the memory access loop shows
nearly the same tsc difference...


  -- Keir

On 29/04/2011 13:32, "Juergen Gross"<juergen.gross@xxxxxxxxxxxxxx>  wrote:


comparing performance of different Xen versions with BS2000 as HVM guest
showed some weird data I'd like to understand.

All measurements were done on an Intel Xeon E7220 box. We used a disk-
benchmark and found the cpu utilization was much higher with Xen 4.0 compared
to Xen 3.3. I did some more investigation and narrowed things down to calls of
the hypervisor (implicit or explicit).

Following is a table with timing data for different low-level functions, all
timing values are tsc ticks obtained via rdtsc:

Xen 3.3     Xen 4.0      Function
        88        165      just the measurement overhead
       176        330      rdtsc-instruction + cli/sti
      5896      11044      lapic timer query
      7381      13519      setting lapic timer
      4653       8987      reload of cr3
      3124       5709      invlpg instruction
    792253     792264      wbinvd instruction
       748       1375      int + iret
      5203       9317      hypervisor yield call
12598102   12597882      memory access loop

All operations involving the hypervisor take nearly twice the time on 4.0.
Operations not involving the hypervisor (wbinvd and memory access loop) are
the same on both systems (this rules out the possibility of different rdtsc

Is there any easy explanation for this? Both Xen versions are from SLES
(SLES11 or SLES11 SP1).

Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@xxxxxxxxxxxxxx
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Xen-devel mailing list