WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Large server, Xen limitations

To: morten@xxxxxxxxxxxxxx
Subject: Re: [Xen-users] Large server, Xen limitations
From: Steve Thompson <smt@xxxxxxxxxxxx>
Date: Thu, 1 Jan 2009 10:09:09 -0500 (EST)
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Thu, 01 Jan 2009 07:09:55 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <9a5476f637202a5db554f67960ca4199@localhost>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <9a5476f637202a5db554f67960ca4199@localhost>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
On Thu, 1 Jan 2009, morten@xxxxxxxxxxxxxx wrote:

we're contemplating getting a large new server, where we will run a number of virtual servers. Are there any things we need to keep in mind in that case? Are there limitations on what a Xen system can manage?

We're talking about a 4 x Quad core CPU server with 64 GBs of RAM and a
couple of terabytes of RAIDed SATA storage.

I have a 2 x quad core Dell PE2900 server with 24 GB memory and a couple of TB of RAID-1 SATA disk running CentOS 5.2 x86_64 and xen 3.0.3, and currently have 33 file-based guests running on it (mostly 32-bit and 64-bit linux, some Windows). All guests have 1 vcpu and 512 MB memory. This setup has been running solidly for about 6 months. Some things I have noticed:

- HVM guests were a lot slower than PV guests, and there is a _lot_ of qemu-dm overhead on Dom0. In particular, 32-bit HVM guests were much slower than 64-bit HVM guests. Avoid HVM as much as possible, and if you can't (Windows), use the Xen PV drivers. My workload consists mostly of a software development environment, so I run a lot of make's (on Windows, under cygwin, sources and objects on Samba shares). I found that the Xen PV drivers on Windows improved performance by over three times (reduced one compilation from 6 hours to 100 minutes); Windows compile performance with cygwin is now about 80% of native speed.

- I converted most of my Linux guests to diskless guests with an NFS root, with the Dom0 as NFS server. Not only do the formerly-HVM guests run much faster, but they also run faster than file-based PV guests, and now 32-bit guests are a little faster than 64-bit guests (less stuff to read from NFS, I presume). The qemu-dm overhead on Dom0 is now essentially zero.

- With a large number of guests, you have to be more careful with their start/stop/start order to avoid the 'out of memory' error.

- Set xen.independent_wallclock on Linux guests if possible (Xen kernel), otherwise it will be next to impossible to keep the clocks in sync.

Steve

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users