WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] Poor HVM performance with 8 vcpus

To: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] Poor HVM performance with 8 vcpus
From: Gianluca Guida <gianluca.guida@xxxxxxxxxxxxx>
Date: Wed, 7 Oct 2009 18:37:59 +0200
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Wed, 07 Oct 2009 09:38:18 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=MegafYm4Y3EjX2CTlRZhgZqaoEe8yhABm594e3jxovY=; b=Sq0efixPZkbDF0VhWlfllcaxzOl3BCfs9e9uX0ks4gk7DKWvjAqnvtF6gzOEwXFYTG 3eGJHhcOn1vN6dleyhN4pzzYhpK1mkNt9K6RRQovEzTLZ7LwPcs1Nv/Sm90x41ly4L0j Jb6+d6VLBXG1qiTrG2+tMMCpRYKbJellhwK88=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=VbScYxJ/LxlvMn4L8JWl89zVB5TtofJSt6w3yJ8/qD/cAohPkIMgRb/AFY+Mc7Wpuf Z6Q8aqz21Hr+BtdyT5xeeXuh7OTaVNWLjoAK5CD/N0dtQS4UnllGfHfwN0L3DneaEVZ7 AbcZmsteH1zzIr7J8oxlqzSsCaVkEvUNpCA8k=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4ACC3B49.4060500@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <4ACC3B49.4060500@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Hi,

On Wed, Oct 7, 2009 at 8:55 AM, Juergen Gross
<juergen.gross@xxxxxxxxxxxxxx> wrote:
> we've got massive performance problems running a 8 vcpu HVM-guest (BS2000)
> under XEN (xen 3.3.1).
>
> With a specific benchmark producing a rather high load on memory management
> operations (lots of process creation/deletion and memory allocation) the 8
> vcpu performance was worse than the 4 vcpu performance. On other platforms
> (/390, MIPS, SPARC) this benchmark scaled rather well with the number of cpus.
>
> The result of the usage of the software performance counters of XEN seemed
> to point to the shadow lock being the reason. I modified the Hypervisor to
> gather some lock statistics (patch will be sent soon) and found that the
> shadow lock is really the bottleneck. On average 4 vcpus are waiting to get
> the lock!
>
> Is this a known issue?

Acutally, I think so. The OOS optimization is widely known not to be
too scalable at 8vcpus in the current state, since its weak point is
the CR3 switching time increasing linearly with the number of cpus. If
you have lot of processes switches together with lot of PTE writings
(as it seems to be the case for your benchmark) then that's probably
the cause.

Could you try disabling the OOS optimization from the
SHADOW_OPTIMIZATIONS definition?

Thanks,
Gianluca

> Is there a chance to split the shadow lock into sub-locks or to use a
> reader/writer lock instead?
> I just wanted to ask before trying to understand all of the shadow code :-)
>
>
> Juergen
>
> --
> Juergen Gross                 Principal Developer Operating Systems
> TSP ES&S SWE OS6                       Telephone: +49 (0) 89 636 47950
> Fujitsu Technolgy Solutions               e-mail: juergen.gross@xxxxxxxxxxxxxx
> Otto-Hahn-Ring 6                        Internet: ts.fujitsu.com
> D-81739 Muenchen                 Company details: ts.fujitsu.com/imprint.html
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>



-- 
It was a type of people I did not know, I found them very strange and
they did not inspire confidence at all. Later I learned that I had been
introduced to electronic engineers.
                                                  E. W. Dijkstra

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>