Re: [Xen-users] Massive iowait with Xen 3.2

[Antoine Benkemoun <antoine.benkemoun@xxxxxxxxx>]

Thank you for your answer. Sorry for the headers ! 

4MB/s read speed is just plain ridiculous for a hard drive or is it just me ? Smartctl gives me the following output :

=== START OF INFORMATION SECTION ===
Device Model:     Hitachi HDP725025GLA380
Serial Number:    GEK234RS1DT3RA
Firmware Version: GM2OA5CA
User Capacity:    250 059 350 016 bytes
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Tue Apr 28 14:57:48 2009 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

It gives me the following google result : http://www.newegg.com/Product/Product.aspx?Item=N82E16822145211 . Smells like a decent hard drive to me ... I'll just go and whip my server provider...

These domains are actually stored on LVM so that's already in place but iostat doesn't seem to differenciate... 

Thank you for your help,

Antoine

On Tue, Apr 28, 2009 at 2:43 PM, Fajar A. Nugraha <fajar@xxxxxxxxx> wrote:

On Tue, Apr 28, 2009 at 6:51 PM, Antoine Benkemoun

<antoine.benkemoun@xxxxxxxxx> wrote:

> The dom0 is a pretty minimal install of Debian with nothing else then Xen,
> SSH and fail2ban running. It can easily fit on 256MB of RAM. In fact, it
> averagely uses 100MB of RAM.

Hmm, interesting. I should test Debian or Ubuntu for firewall domUs sometimes.

> I have reproduced the problem by generating IOs with 3 torrents and 1 file
> copy.
>
> XM top shows :
>
>     commun ------       2959    3.5     262144   12.7     262144
> 12.7     1    1  1828352  1567395    3    16267  3411443  1848929   32
>   Domain-0 -----r       2617    2.7     262240   12.7   no limit
> n/a     1    0        0        0    0        0        0        0   32

Since you ommit the headers, the output is quite confusing :P
What are vbd stats for the domU, does it have lots of reads/writes?

>
> Load average for "commun", the domain being loaded, is above 5. Memory is
> only used at about 50% and only 440k of swap is used. Along goes the usual
> symptoms, really slow web page loads and really slow reactivity of about
> everything in the domain. Reactivity in the other domains is rather normal.

So it's not swap problem. However looking at your previous iostat
output %util is over 99 (which means your disk is busy), r/s over 180,
and rsec/s almost 8000 (meaning about 4 MB/s). Based on this numbers
I'm GUESSING you only have one lowend disk (not raid), and it has
reached its IOPS caps. In other words, get more disks :P

A suggestion, it would probably be easier to manage if you use LVM and
put dom0 and domU each on its own LV. That way iostat can show per-LV
staticstic so you can easily determine who's using most disk I/O.

Regards,

Fajar

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users