Re: [Xen-users] High Number of VMs

[Ciro Iriarte <cyruspy@xxxxxxxxx>]

2011/9/9 John Madden <jmadden@xxxxxxxxxxx>:
> On 09/09/2011 03:07 PM, Ciro Iriarte wrote:
>>
>> Hi, I'm curious about how you guys deal with big virtualization
>> installations. To this date we only dealt with a small number of VM's
>> (~10)on not too big hardware (2xquad xeons+16GB ram). As I'm the
>> "storage guy" I find it quite convenient to present to the dom0s one
>> LUN per VM that makes live migration possible but without the cluster
>> file system or cLVM complexity. The thing is that Linux has a 255 SCSI
>> device limit apparently (255/2= ~128 with multipath) and that won't
>> scale in big installations (300 VMs for example).
>>
>> Any experiences on this scale?
>
> I run a couple hundred VM's across a handful of blades.  I recommend going
> to fewer, larger LUNs, carving them up with LVM, and handing out LV's to
> your VM's.  You don't actually need cLVM to do this!  All the cluster infra
> does (for its nasty administrative overhead) is keep the LVM metadata (not
> your actual data) consistent through cluster-wide locks.  You can manage
> yourself by, for example, making changes on one node and refreshing the
> other nodes with things like 'vgscan -ay'.  I typically allocate LUNs 500GB
> at a time.  You can invent means of keeping things consistent that work for
> your environment, just test them first.
>
> If you really want data security -- preventing one node from hosing data
> that's accessible from another node -- you'll have to go with a cluster
> filesystem.  That still doesn't help keep your multipathing and LVM configs
> consistent though, so I think you're better off just skipping that step.
>
> John
>
>
Any advantage on using large luns+LVM instead of independent LUNs
appart from snapshots? (according to Novell support LVM on top of LVM
is a bad thing...). I remember reading that Xen itself implements some
kind of locking...

Regards,

-- 
Ciro Iriarte
http://cyruspy.wordpress.com
--

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users