El Martes 27 Enero 2009, Tait Clarridge escribió:
> Hello Everyone,
Hi, I'm no expert but I'm in the same path as you, so let's try to help each
other... and get help from others as we go :)
> I recently had a question that got no responses about GFS+DRBD clusters for
> Xen VM storage, but after some consideration (and a lot of Googling) I have
> a couple of new questions.
>
> Basically what we have here are two servers that will each have a RAID-5
> array filled up with 5 x 320GB SATA drives, I want to have these as useable
> file systems on both servers (as they will both be used for Xen VM storage)
> but they will be replicating in the background for disaster recovery
> purposes over a GbE link.
OK,
> First of all, I need to know if this is good practice because I can see a
> looming clusterf**k if both machines are running VMs from the same shared
> storage location.
Well, it shouldn't happen if you're using GFS or another cluster aware
filesystem.
> Second, I ran a test on two identical servers with DRBD and GFS in a
> Primary/Primary cluster setup and the performance numbers were appalling
> compared to local ext3 storage, for example:
Yes, cluster filesystem have lower performance than non-cluster filesystems,
due to the former performing lokcs on files/dirs.
Add DRBD replication on top of that and performance will be lower.
> 5 Concurrent Sessions in iozone gave me the following:
>
> Average Throughput for Writers per process:
> EXT3: 41395.96 KB/s
> DRBD+GFS (2 nodes): 10884.23 KB/s
>
> Average Throughput for Re-Writers per process:
> EXT3: 91709.05 KB/s
> DRBD+GFS (2 nodes): 15347.23 KB/s
>
> Average Throughput for Readers per process:
> EXT3: 210302.31 KB/s
> DRBD+GFS (2 nodes): 5383.27 KB/s <-------- a bit ridiculous
Ridiculous indeed
> And more of the same where basically it can range from being 4x to however
> many times slower reading was. I can only assume that this would be a
> garbage setup for Xen VM storage and was wondering if anyone could point me
> to a solution that may be more promising. We currently are running out of
> space on our NetApp (that does snapshots for backups) for VMs not to
> mention the I/O available for multiple VMs on a single NetApp directory is
> already dangerously low.
>
> Anyone have thoughts as to what might solve my problems?
Have you tried any GFS optimizations? e.g. use noatime and nodiratime, disable
gfs quotas, etc. The first two should improve reading performance.
> I am thinking a few things:
>
> - Experiment with DRBD again with another Filesystem (XFS?) and have it
> re-exported as NFS to both machines (so they can both bring up VMs from the
> "pool")
I guess NFS could work, unless you have too many machines using it (Linux's
NFS sucks)
> - Export one of the machines as iSCSI and software raid it on a primary (not
> really what I want but might work)
This one sound interesting.
> - Write a custom script that will backup the VM storage directories to a 3rd
> server (don't really have the budget for a redundant backup server) using
> something like rsync
>
> And finally, what kind of redundant server to server storage do most people
> use here?
From what I'been reading on the list, most people uses some form of DRBD + AoE
or iSCSI.
Check the thread with subject "disk backend performance" from November 27,
2008. There started a very nice discussion involving Thomas Halinka and
Stefan de Konink about AoE vs. iSCSI (thank you both!).
Also, the thread with subject "lenny amd64 and xen" will be of your interest,
on November 27 Thomas started a description of his self-build SAN which is
very insightful.
> Thanks a lot for reading my novel of a question :)
>
> Best,
>
> Tait Clarridge
Best regards,
--
Ricardo J. Barberis
Senior SysAdmin - I+D
Dattatec.com :: Soluciones de Web Hosting
Su Hosting hecho Simple..!
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home