Re: [Xen-users] domUs with iSCSI disks... connect in dom0 or domU?

[xenusers <m+xenusers@xxxxxxxxxx>]

Hello Bart,

On Mon, Oct 17, 2011 at 3:01 AM, Bart Coninckx <bart.coninckx@xxxxxxxxxx> wrote:

On 10/16/11 23:14, xenusers wrote:

Hi List,

I don't know if this is something that is a simple matter of opinion or
if there are strong reasons to take one route or the other.  I have dom0
nodes with dedicated bond interfaces that connect to a storage
traffic-only VLAN.  Currently I have a few domUs running that have
large-ish volumes on the iSCSI SAN, and to present the volumes I'm
connecting the dom0s to the storage, then in the domU config I give it
the iSCSI volumes as
'phy:/dev/disk/by-path/ip-10.10.10.5-iqn....:volume1' and so on.

I'm wondering if instead it's considered better to simply have the domU
make those iSCSI connections, by giving it a second interface that
connects to a bridge with access to the iSCSI VLAN?  I set one up that
way instead, and it works as expected though it seems to perform less
well in some admittedly simple tests using dd with 'oflag=direct' set.
 The dom0 can write at about 110MB/s sustained, but if the domU makes
the iSCSI connection instead it's doing well to manage 60MB/s.

So I'm looking to the wisdom of the list, is it crazy to set things up
one way or the other, or are they both an OK approach?  It seems like
having the dom0 handle the iSCSI connection is a big win
performance-wise, but perhaps that comes at a big negative cost?  Having
the domU make the connection simplifies the Pacemaker configuration, but
with an apparent loss in disk throughput.

Do you use HVM or PV DomUs? I wonder if this performance drop should hebben with PV.
Anyway, I definitely use the iSCSI connections in Dom0. As far as complexity is concerned: I combine iSCSI with multipath, allowing me to change the SCSI disks' names to the names of the DomUs that actually use them. Complexity for the bigger part gone. It also provides you with the benefit of redundant paths towards your iSCSI.

Yes, I should have mentioned that, these are PV domUs, so I think that 60MB/s is about as good as I'd be able to hope for if setting up iSCSI within the domU.  My preference is to have all of the iSCSI configuration done within dom0, anyhow.

I may fiddle around with multipathing again.  I tried both and bonding with LACP seemed to be the fastest recovery for a lost link (1 second or so, while muiltipath was taking two minutes).  I know there are some timers you're supposed to be able to adjust for multipath to get a much faster failover, so if I get some time I'll experiment.

Thanks,
Mark

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users