Re: [Xen-users] dom0 hangs when doing heavy I/O on domU

[Kiefer Chang <zapchang@xxxxxxxxx>]

Hi Florian,

- are they sharing a disk?

  Yes, 3 VMs' images are stored in the same disk sdb. sda is used for dom0 root filesystem.
- can you check what I/O scheduler you are using?

  Default to CFQ in for sdb. I tried to ionice all blkback processes to class 2 and still have no luck.

  I also tried deadline scheduler before.

Right now I found a cure is to make sdb as physical volume and setup volume groups/logical volumes on it.

Attach logical volumes to VMs by "phy" method.

The symptom is gone when 3 VMs perform the same I/Os.

I know XEN manual suggest using blktap and phy method for VM storages.

But we think it's much easier to manage VM's image files then LVM volumes since we provision VMs by downloading their images from servers.

Thanks!

--

Kiefer Chang

2011/4/21 Florian Heigl <florian.heigl@xxxxxxxxx>

Hi Chang,

2011/4/20 Kiefer Chang <zapchang@xxxxxxxxx>:

> Hi all,
> We are using XEN as hypervisor to setup our private cloud.
> The framework is Eucalyptus and using CentOS 5.4 as dom0 OS.
> Sometimes we find some machines' dom0 become unresponsive, the symptoms are:
> (1) We can't log into dom0 via ssh. After typing password, it just stops
> there.
> (2) We can ping dom0 successfully.
> (3) We can log into domU without problem.
> The unresponsive dom0 eventually "alive" after a period of time. Maybe half
> hour or even several hours.

So one of your domUs is trashing the disks and dom0 can't get enough
performance, right?
- are they sharing a disk?
- can you check what I/O scheduler you are using?
 (with cfq you can then use ionice to lower prio on all blkback
threads a little. that way dom0 will "win the race")

In general, your dom0 is privileged in terms of IO access rights, but
not in IO peformance. So if one domU goes crazy, it will affect
anything.
... until you take measures :)
I'd suggest you switch to deadline scheduler and re-test.
dom0 on a different disk media is also very advisable imho.


Flo

--
the purpose of libvirt is to provide an abstraction layer hiding all
xen features added since 2006 until they were finally understood and
copied by the kvm devs.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users