WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Re: LVM on DRBD -OR- DRBD on LVM?

Simon,

I think Matthew was making a recommendation based upon how you have your domU's setup ... i.e. are your domU's living on LVM LV's or disk files in one big LV, multiple PV's (i.e. 1 for the boot/root of the dom0 and one (or more) for the domU's), etc.

We need to know how you have your primary server configured ... I'm not sure you can reliably mirror your entire root PV since the kernel needs to be up and have the DRBD driver in place to mount the device the kernel will be booting on (chicken and egg) ... it may be possible, but I've not thought about it long enough to decide.

What we've done is set up a pair of physically identical servers with a GigE crossover cable for the DRBD devices to sync across (and a 100M Eth for the public interface), and each guest has a set of LV's dedicated to it. Each LV is created identically on both servers and are bound under a DRBD device. The guest mounts the DRBD device, so all writes to the domU's "disk" gets replicated via DRBD to both servers.

Now, that said, that's not our entire story, as we have some more mirroring stuff under the LV's, but you get the point for where we have DRBD in place.

Each guest generally has 3 LV's associated with it, and only 2 of those replicated via DRBD:

   xen0:GuestA-root ->   drbd    ->  xen1:GuestA-root      (2gb)
   xen0:GuestA-swap <not replicated> xen1:GuestA-swap      (1gb)
xen0:GuestA-data -> drbd -> xen1:GuestA-data (anywhere from 5gb to 100gb)

I felt that replicating swap was just a waste of resources as we can't do hot migration anyway, so upon reboot, anything in swap is gone anyhow.

Currently, we are running with 14 DRBD devices, but none of the guests are in production yet, so I don't have good load and performance data yet ... I'm still hunting for good performance monitoring tools (got one suggestion for ganglia, and I also found bandwidthd as well as the usual friends like cacti+snmpd), and I've been watching the privage GigE as well as /proc/drbd.

I'll let the list know how performance seems to be after it's all up and running! :)

Oh, and currently, we are running all guests on the primary server, but we plan to distribute the guests across both servers after all is running.

Hope this helps,
Sam


Simon wrote:
Hi Matthew,

On 6/20/06, Matthew Palmer <mpalmer@xxxxxxxxxxx> wrote:
On Tue, Jun 20, 2006 at 02:49:48PM +1200, Simon wrote:
> In creating a "backup" dom0 server using drbd and using lvm partitions
> for the domU data, whats the best way to implement the drbd part?
>
> Should i have drbd mirroring the lvm partitions them selves or the
> data within them?

Depends.  Primarily, on whether you've going to be sharing the entire PV
between both machines, and never sharing some LVs with any other host. If
you have a separate VG (on it's own PV) for your domUs, and are just
mirroring your domUs for availability, then mirroring the PVs is fine.
Otherwise (if you've got the dom0 disk on the same VG) then you'll have to
mirror the LVs individually, and incur the overhead of lots-o-drbd.

Im only concerned with two individual servers here:

dom0(1) will hold all the domUs
dom0(2) will only be used in a DR case when dom0(1) is down

So reading your above comment.... LVM on DRBD?

Simon

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users