rsize and wsize are the most important ones to reduce your
iowait. Those can dramatically change performance.
The theory is that the best performance is reached when you
match the geometry of the actual disks to the rsize/wsize nfs parameter.
Use dd to emulate different block sizes and check the best
performance for your environment.
Emerson
-----Original Message-----
From: Geoff Galitz
[mailto:geoff@xxxxxxxxxx]
Sent: Tuesday, September 14, 2010
11:33 AM
To: Ribeiro Emerson Gomes;
xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-users] IOwaits
over NFS
We`ve used various NFS parameters
in our testing to resolve this. Here are three different examples
of what we have used on a single machine, with no significant improvement:
1)
defaults,auto,tcp,bg,hard,intr,noatime 0 0
2) auto,noatime,rsize=32768,wsize=32768,bg,nfsvers=3,intr,tcp,retrans=0
0 0
3) noatime,bg,intr,nolock,tcp
0 0
Sent: Tuesday,
September 14, 2010 4:20 PM
Subject: RE: [Xen-users] IOwaits over NFS
Please
inform the parameters you use in fstab when you mount your NFS share.
Something
like this:
<server>:/<share>
<mountpoint> nfs rsize=8192, wsize=8192, timeo=14, intr, tcp, bg, rw, bg,
hard, nointr, rsize=32768, wsize=32768, tcp, actimeo=0, vers=3,timeo=600
Emerson
-----Original Message-----
From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-users-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Geoff Galitz
Sent: Tuesday,
September 14, 2010 11:04 AM
To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] IOwaits over
NFS
Hello.
We have a number of Xen 3.4.2. boxes which have constant iowaits at around
10% with spikes up to 100% when accessing data over NFS. We have been
unable to nail down the issue. Any advice?
System info:
release
: 2.6.18-194.3.1.el5xen
version
: #1 SMP Thu May 13 13:49:53 EDT 2010
machine
: x86_64
nr_cpus
: 16
nr_nodes
: 1
cores_per_socket : 4
threads_per_core : 2
cpu_mhz
: 2261
hw_caps
:
bfebfbff:28100800:00000000:00000340:009ce3bd:00000000:00000001:00000000
virt_caps
: hvm
total_memory :
32758
free_memory :
3974
node_to_cpu :
node0:0-15
node_to_memory : node0:3974
xen_major
: 3
xen_minor
: 4
xen_extra
: .2
xen_caps
: xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params :
virt_start=0xffff800000000000
xen_changeset :
unavailable
cc_compiler :
gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)
cc_compile_by : root
cc_compile_domain : gitco.tld
cc_compile_date : Wed Nov 11 21:16:28 CET 2009
xend_config_format : 4
And lsmod from a VM:
Module
Size Used by
nfsd
284041 17
exportfs
38849 1 nfsd
auth_rpcgss
81889 1 nfsd
ipv6
424737 24
xfrm_nalgo
43333 1 ipv6
crypto_api
42945 1 xfrm_nalgo
i2c_dev
43849 0
i2c_core
56129 1 i2c_dev
nfs
289549 1
lockd
99185 3 nfsd,nfs
fscache
52385 1 nfs
nfs_acl
36673 2 nfsd,nfs
sunrpc
197513 18 nfsd,auth_rpcgss,nfs,lockd,nfs_acl
iptable_filter 36161 1
ip_tables
55329 1 iptable_filter
x_tables
50377 1 ip_tables
dm_mirror
54345 0
dm_multipath
55385 0
scsi_dh
41665 1 dm_multipath
scsi_mod
197337 1 scsi_dh
parport_pc
62313 0
lp
47121 0
parport
73293 2 parport_pc,lp
pcspkr
36289 0
xennet
61129 0 [permanent]
dm_raid45
98897 0
dm_message
36161 1 dm_raid45
dm_region_hash 46401 1
dm_raid45
dm_log
44993 3 dm_mirror,dm_raid45,dm_region_hash
dm_mod
100753 4 dm_mirror,dm_multipath,dm_raid45,dm_log
dm_mem_cache
39617 1 dm_raid45
ext3
168529 1
jbd
94257 1 ext3
uhci_hcd
57561 0
ohci_hcd
56053 0
ehci_hcd
65869 0
xenblk
50729 1
Home