WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] easy high availability storage

To: Javier Guerra <javier@xxxxxxxxxxx>
Subject: Re: [Xen-users] easy high availability storage
From: Peter Braun <xenware@xxxxxxxxx>
Date: Thu, 26 Nov 2009 15:57:20 +0100
Cc: "Fajar A. Nugraha" <fajar@xxxxxxxxx>, xen-users@xxxxxxxxxxxxxxxxxxx, Roland Frei <frei@xxxxxxxxxxxx>
Delivery-date: Thu, 26 Nov 2009 06:58:23 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=SvuzeATj4RqZuGpEgpvRmfOe8bhLwGbxLkbLt6ghNLA=; b=m3VA4nQazE9SRD54YkyK1HmgpFEacwJ9szq/LtU4SHWHnt1kJV5gplYZ8DfdQlkjOz c+YmUOCtVONkrVjQ6NUODU7AOEI+EIkYh3rBh3TneNPlz2OfKYng/4DTBXRVeLIP9t+K QLpHlxJQOm4GhNV3tqVmtrN6y66Z4Rb9JD888=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=fxGqeuGx/Td3T9lY+8GYCbZ0pZvBRY9qFD6+SMdXlL/pgsoVMEvzJAEmoWK0VuJHiI jhmkGDmHHwZdqExCBAxMrN3Vif4yIvrXDASgB33LX9WrGQQ4TIsxYJ7OsElS5ltqymyN 6OAJlORv0mkAhAfbD/UZdO/A37BqmaYAT1kAM=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <90eb1dc70911260622y7c28f999xf1ec0b0dd52194bd@xxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <op.u3wq0lzpcwzw3p@xxxxxxxxxxxxxxxxx> <1259081515.2241.4.camel@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx> <op.u3x0s8axcwzw3p@xxxxxxxxxxxxxxxxx> <7207d96f0911250114p1b4db8f3y26b053f926f08bcb@xxxxxxxxxxxxxx> <acbf498f0911250201h30a3a9b5lf1fdb59637780e7d@xxxxxxxxxxxxxx> <7207d96f0911251917x7b30575fj7dc0534bcf16fed5@xxxxxxxxxxxxxx> <acbf498f0911260126s5b9461cambab8266c9c5fa3cc@xxxxxxxxxxxxxx> <90eb1dc70911260622y7c28f999xf1ec0b0dd52194bd@xxxxxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Hi Javier,

thank you very much for your thoughts.

SAN should be providing HDD space for several hundreds of VM at
several Xen boxes over iSCSI.

This should provide:
- easy offline migration of VMs between hosts (just stop VM on xen01
an start on xen04 - matter of seconds)
- higher utilization of expensive 15k harddrives
- ? possible live migration in future
- higher speed than current RAID1 local drive
- easy expandable
- ? VM HDD snapshot on the fly?


Technology summary:
2 SAN hosts
DRBD
cLVM
highly available iSCSI TARGET
HEARTBEAT


Whats clear difference between LVM and cLVM?


Br

Peter


2009/11/26 Javier Guerra <javier@xxxxxxxxxxx>:
> On Thu, Nov 26, 2009 at 4:26 AM, Peter Braun <xenware@xxxxxxxxx> wrote:
>> Am looking for opensource solution of H/A SAN - and this is close to my goal.
>
> there are lots of options, some simply bad; most good for some things
> and bad for other things
>
>>
>>
>> The basics:
>>
>> 1)    2 machines with some HDD space synchronized between them with DRBD.
>
> ok
>
>>
>> 2)    Now am little bit confused what shall I use above the DRBD block 
>> device?
>>       - some cluster FS like OCFS2 or GFS?
>>       - LVM?
>
> first question: what would be the clients, and how many??
>
> if the client(s) would store files, you need a filesystem.  if the
> client(s) are Xen boxes, the best would be block devices, shared via
> iSCSI (or AoE, or FC, or nbd...) and split with LVM.   or split with
> LVM and shared.
>
> if you want a single client, for filesystem anyone would do and for
> block devices LVM would be enough.  if you want several clients, you
> need a cluster filesystem or cLVM.  or you could split with plain LVM
> and share each LV via iSCSI.
>
> pruning some branches off the decision tree, you get two main options:
>
> 1: two storage boxes, synced with DRBD, split with (plain) LVM, share
> each LV via iSCSI.
>  pros:
>    - easy to admnister
>    - no 'clustering' software (apart from DRBD)
>    - any number of clients
>  cons:
>    - you can grow by adding more storage pairs; but a single LV can't
> span two pairs
>    - no easy way to move LVs between boxes
>    - if you're not careful you can get 'hot spots' of unbalanced load
>
> 2: any number of 'pairs', each synced with DRBD. no LVM,  share the
> full block via iSCSI. set cLVM on the clients, using each 'pair' as a
> PV.
>  pros:
>    - very scalable
>    - lots of flexibility, the clients see a single continous expanse
> of storage split in LVs
>  cons:
>    - somewhat more complex to setup well
>    - cLVM has some limitations: no pvmove, no snapshots (maybe soon
> will be fixed?)
>
>
>> 3)    create files with DD on cluster FS and export them with iSCSI?
>>       create LVM partitions and export them with iSCSI?
>
> if you're exporting image files with iSCSI, then the only direct
> client of these files is iSCSI itself. no need for a cluster FS, any
> plain FS would do.
> ... and of course, LVM is more efficient than any FS
>
>>
>> 4)    how to make iSCSI target highly available?
>>       - configure   iSCSI on virtual IP/another IP and run it as HA service
>>       - configure separate iSCSI targets on both SAN hosts and
>> connect it to Xen server as multipath?
>
> no experience here.  i'd guess multipath is nicer; but any delay in
> DRBD replication would be visible as read inconsistencies.  a
> migratable IP number might be safer.
>
>> 5)    hearbeat configuration
>
> yes, this can be a chore.
>
>> VM machines with iSCSI HDD space on SAN should survive reboot/non
>> availability of one SAN hosts without interruption nor noticing that
>> SAN is degraded.
>>
>> Is that even possible?
>
> yes, but plan to spend lots of time to get it right.
>
>
> --
> Javier
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users