WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Poor write-performance with nfs

To: "Xen-devel@xxxxxxxxxxxxxxxxxxx" <Xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] Poor write-performance with nfs
From: Jens Krehbiel-Gräther <jens.krehbiel-graether@xxxxxxxxxxxxxxxxxx>
Date: Wed, 24 Nov 2010 19:38:45 +0100
Cc:
Delivery-date: Wed, 24 Nov 2010 10:39:29 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6
Hi!

I have a question about nfs-performance. I don't where the problem is.

My setup:

Host dom0 with 8TB harddisk space 8GB RAM and 6 core CPU.
The hard disk raid is fast:

<cut>
root@midas-phalanx:/data# time dd if=/dev/zero of=test.null bs=1M count=1024
1024+0 Datensätze ein
1024+0 Datensätze aus
1073741824 Bytes (1,1 GB) kopiert, 3,27874 s, 327 MB/s

real    0m3.303s
user    0m0.004s
sys     0m1.792s
</cut>

so the write performance is very good on harddisk.

I created several domU guests which do a nfs-boot from the local host. (strange setup, I know). I don't want to define the size of guests disk so I don't want to use a image as hard disk image. This is why I do a nfs-boot.

The dom0 exports the file system for the guest. I use network bridging. My guest config file is the following:

<cut>
kernel          = '/boot/vmlinuz-2.6.32-5-xen-amd64'
ramdisk         = '/boot/initrd.img-2.6.32-5-xen-amd64'
memory          = 1024
name            = 'ufp-mysql-01'
hostname        = 'ufp-mysql-01'
#dhcp           = 'dhcp'
vif             = [ 'mac=00:16:3e:00:42:06, bridge=lan' ]
interface       = 'eth0'
ip              = '10.10.42.206'
netmask         = '255.255.255.0'
gateway         = '10.10.42.204'
nfs_server      = '10.10.42.108'
nfs_root        = '/data/nfs-root/ufp-mysql-01'
root            = '/dev/nfs'
extra           = '2 console=hvc0 xencons=tty boot=nfs rootfstype=nfs'
</cut>

The guest boots without error. But the write performance over the nfs is very poor:

<cut>
root@ufp-mysql-01:~# time dd if=/dev/zero of=test.null bs=1M count=512
512+0 Datensätze ein
512+0 Datensätze aus
536870912 Bytes (537 MB) kopiert, 2159,08 s, 249 kB/s

real    35m59.133s
user    0m0.000s
sys     0m1.396s
</cut>

The read-performance is good:

<cut>
root@ufp-mysql-01:~# dd if=test.null of=/dev/zero bs=1M
512+0 Datensätze ein
512+0 Datensätze aus
536870912 Bytes (537 MB) kopiert, 0,705893 s, 761 MB/s
</cut>

But it is not the performance of the server itself:

<cut>
root@ufp-mysql-01:~# dd if=/dev/zero of=/dev/null bs=1M count=512
512+0 Datensätze ein
512+0 Datensätze aus
536870912 Bytes (537 MB) kopiert, 0,0655885 s, 8,2 GB/s
</cut>

OK, this is the setup where the nfs-server is on dom0 and exports the filesystem for domU.

When I copy the rootfs to another computer and export it from there (with same export options) I get a lot of more performance.
This is the guest bootet from nfs on other server:

<cut>
root@ufp-mysql-01:~# time dd if=/dev/zero of=test.null bs=1M count=512
512+0 Datensätze ein
512+0 Datensätze aus
536870912 Bytes (537 MB) kopiert, 15,506 s, 34,6 MB/s

real    0m15.617s
user    0m0.004s
sys     0m1.260s
</cut>

The "external" nfs-server has a single hard disk, so this performance is OK for this config. Can you give me a hint why the perfmance is so bad when I export the fs from dom0? How can i solve this?

The server has no I/O jobs while this test was running so the whole performance should be available.

Jens


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-devel] Poor write-performance with nfs, Jens Krehbiel-Gräther <=