WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

[Xen-users] LVM Read/Write speed <10% drive's normal speed

To: xen-users@xxxxxxxxxxxxxxxxxxx, linux-lvm@xxxxxxxxxx
Subject: [Xen-users] LVM Read/Write speed <10% drive's normal speed
From: Stephen Hamer <Stephen.Hamer@xxxxxxxxxx>
Date: Wed, 16 Sep 2009 21:49:33 +0000
Cc:
Delivery-date: Wed, 16 Sep 2009 14:50:29 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Organization: JHUAPL
Reply-to: Stephen.Hamer@xxxxxxxxxx
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Hello all,

I've been searching through the archives and the internet for a while now, and can't seem to find anything that helps me out. I hope you don't my posting this to both xen-users and linux-lvm simultaneously, but i figured it'd help keep the solution in one place... even though it'll be the same across two places... Anyways:

My setup is this:
        Xen 3.3 hypervisor (from the ubuntu repositories)
        Ubuntu 9.04 server
        Kernel and module listed here: http://www.infohit.net/blog/post/running-xen-on-ubuntu-intrepid-and-jaunty.html

The dom0 and domUs in my Xen setup are all Ubuntu 9.04, using that kernel. (I'm not sure if the kernel has something to do with it, so i included it just in case.) All the domUs are set up to use LVM. The dom0 uses a straight ext3 fs on /dev/sda1. The domUs use ext3 fs's within LVMs (using LVM2) set up on /dev/sda5.

I have noticed a massive file I/O problem on all of my domUs. While I can peak file operations around 100 MB/s within the dom0, I can't get anything more than 3 MB/s read OR write out of a file operation on the domUs. I am using values reported by "iotop" to make this distinction. I'm using both the deployment of a Jboss server and the "dd" command to benchmark this. The Jboss slowdown is how I found this. On the dom0, the server takes 56s to come up. On a domU, the same operation takes 15 minutes.

It should also be noted that I attempted to change the filesystems to ext2, because I noticed that 'kjournald' was chugging away taking up most of the i/o percentage, but only writing at KB/s! Actually, when I removed the journaling is when the process slowed from 7 minutes to 15. Even without the kjournald there, the java process took forever to load and never wrote more than 3 MB/s.

I have messed around with the 'blockdev' and 'hdparm' commands (the second of which doesn't even interact with any partition on my computer, lvm or no, instead simply failing with an "input/output error" message), but have had no performance increases after messing around with the read ahead speed of the devices. I also tried the "lvchange -r" command to set the readahead higher. I think there's something else missing here, though.

I'm fairly fresh to xen, this being only my second installation, and I'm extremely fresh to lvm. I've played with what seems like every option within "xm" and the 'lv*' commands. The LVMs themselves I haven't played with all too much yet, as I don't want to nuke a certain domU before it needs to be used heavily tomorrow. I've already killed a test domU that I brought up just for this.

-Stephen
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users