WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Xen4.0.1 : slow Disk IO on DomU

To: Joost Roeleveld <joost@xxxxxxxxxxxx>
Subject: Re: [Xen-users] Xen4.0.1 : slow Disk IO on DomU
From: Erwan RENIER <erwan.renier@xxxxxxxxxxx>
Date: Thu, 17 Mar 2011 18:31:10 +0100
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Thu, 17 Mar 2011 10:32:15 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20110317083208.AE3901131@xxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <AANLkTi=mJHRZ_iiLLU6aTX6tHQXZT27+BZ20irSzPrDB@xxxxxxxxxxxxxx> <AANLkTikMu6Qa5fofBs6YN=pESsczYyEEn3b930OaLAT6@xxxxxxxxxxxxxx> <4D813A43.9090608@xxxxxxxxxxx> <20110317083208.AE3901131@xxxxxxxxxxxxxxxxx>
Reply-to: erwan.renier@xxxxxxxxxxx
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9
Le 17/03/2011 09:31, Joost Roeleveld a écrit :
On Wednesday 16 March 2011 23:31:31 Erwan RENIER wrote:
Hi,
When i test the IO bandwidth it's pretty much slower on DomU :

Dom0 read  : 180MB/s write  : 60MB/s
DomU read  : 40MB/s write :  6MB/s
Just did the same tests on my installation (not yet on Xen4):
Dom0:
# hdparm -Tt /dev/md5
/dev/md5:
  Timing cached reads:   6790 MB in  1.99 seconds = 3403.52 MB/sec
  Timing buffered disk reads:  1294 MB in  3.00 seconds = 430.94 MB/sec

(md5 = 6-disk RAID-5 software raid)

# hdparm -Tt /dev/vg/domU_sdb1
/dev/vgvg/domU_sdb1:
  Timing cached reads:   6170 MB in  2.00 seconds = 3091.21 MB/sec
  Timing buffered disk reads:  1222 MB in  3.00 seconds = 407.24 MB/sec

DomU:
# hdparm -Tt /dev/sdb1
/dev/sdb1:
  Timing cached reads:   7504 MB in  1.99 seconds = 3761.93 MB/sec
  Timing buffered disk reads:  792 MB in  3.00 seconds = 263.98 MB/sec

Like you, I do see some drop in performance, but not as severe as you are
experiencing.

DomU disks are Dom0 logical volumes, i use paravirtualized guests, the
fs type is ext4.
How do you pass the disks to the domU?
I pass them as such:
disk = ['phy:vg/domU_sda1,sda1,w',
(rest of the partitions removed for clarity)

My DomU conf is like this :
kernel =  "vmlinuz-2.6.32-5-xen-amd64"
ramdisk = "initrd.img-2.6.32-5-xen-amd64"
root = "/dev/mapper/pvops-root"
memory = "512"
disk = [ 'phy:vg0/p2p,xvda,w' , 'phy:vg0/mmd,xvdb1,w', 'phy:sde3,xvdb1,w' ]
vif = [ 'bridge=eth0' ]
vfb = [ 'type=vnc,vnclisten=0.0.0.0' ]
keymap = 'fr'
serial = 'pty'
vcpus = 2
on_reboot = 'restart'
on_crash = 'restart'

I already tried the ext4 options barrier=0,data=writeback , it doesn't
realy change anything.
I tried too with ext2 and ext3 , it's the same.
To avoid any "issues" with the filesystem, what does "hdparm -Tt<device>" give
you?

Dom0 :
/dev/sde( single disk ):
 Timing cached reads:   6086 MB in  2.00 seconds = 3050.54 MB/sec
 Timing buffered disk reads: 270 MB in  3.01 seconds =  89.81 MB/sec
/dev/md127 ( raid 5 of 5 disks ):
 Timing cached reads:   6708 MB in  1.99 seconds = 3362.95 MB/sec
 Timing buffered disk reads: 1092 MB in  3.00 seconds = 363.96 MB/sec

DomU :
/dev/xvda:
 Timing cached reads:   5648 MB in  2.00 seconds = 2830.78 MB/sec
 Timing buffered disk reads:  292 MB in  3.01 seconds =  97.16 MB/sec
/dev/xvda2:
 Timing cached reads:   5542 MB in  2.00 seconds = 2777.66 MB/sec
 Timing buffered disk reads:  274 MB in  3.01 seconds =  90.94 MB/sec
/dev/xvdb1:
 Timing cached reads:   5526 MB in  2.00 seconds = 2769.20 MB/sec
 Timing buffered disk reads:  196 MB in  3.02 seconds =  64.85 MB/sec
/dev/xvdb2:
 Timing cached reads:   5334 MB in  2.00 seconds = 2672.47 MB/sec
 Timing buffered disk reads:  166 MB in  3.03 seconds =  54.70 MB/sec

Is this normal ?
Some drop, yes. loosing 90% performance isn't

If not what do you think the problem is?
Either you are hitting a bug or it's a configuration issue.
What is the configuration for your domU? And specifically the way you pass the
LVs to the domU.
As you can see :
xvda is a lv exported as a whole disk with lvm on it, so xvda2 is a lv from a vg in a lv ( ext4 => lv => vg => pv => virtual disk => lv =>vg =>pv => raid5 =>disk ) xvdb1 is a lv exported as a partition ( ext4 => virtual part => lv => vg => pv => raid5 => disk ) xvdb2 is a physical partition exported as a partition ( ext3 => virtual part => disk )

Curiously it seems the more complicated, the better it is :/

Thanks.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users