WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Yet another backup proposal (file based vbd's & lvm)

To: tim.post@xxxxxxxxxxxxxxx, roger@xxxxxxxxxxxxx, lsc@xxxxxxxxx
Subject: Re: [Xen-users] Yet another backup proposal (file based vbd's & lvm)
From: "Kenneth Kalmer" <kenneth.kalmer@xxxxxxxxx>
Date: Wed, 18 Oct 2006 07:33:07 +0200
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Tue, 17 Oct 2006 22:33:49 -0700
Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=YTG0wMTQ/1ZdWndHvLuPPsot8YqbBclGeXIkTsjaDWKfmMQK5ivYIcEQDMaU9xybC4qx4/TlQLr6+ofHIZ7oCZOs5iKUtMnMI67mjNRq7zwB51/pLkzwyNUGK6p5GHjiP6T+75rhqrsIEhhK3fKbFFvoFym8+Du3YYv05UepIoM=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <1161145802.2109.159.camel@tower>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <fad9d4840610171532v7fe12ee4k36057b606fa4e68f@xxxxxxxxxxxxxx> <1161145802.2109.159.camel@tower>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
On 10/18/06, Tim Post <tim.post@xxxxxxxxxxxxxxx> wrote:
On Wed, 2006-10-18 at 00:32 +0200, Kenneth Kalmer wrote:
> Greeting list

Hello :)

Hi Tim, and Roger and Roger as well

> This is yet another request for feedback on a proposed backup solution
> for a xen environment, and I'll appreciate any all scrutiny of what I
> want to attempt.

Be careful what you ask for :P

I love a good sense of humour :)

> What I'd like to attempt to setup (based on the feedback I receive) is
> the following.
>
> 1. Pause the domU using "xm pause"
> 2. Sync the domU using "xm sysrq"
> 3. Use rdiff-backup to make a local backup of the file VBD
> 4. Resume the domU using "xm unpause"
>

I'm going to ask the same 'why not use lvm' question here, and toss in
the idea of replacing NFS with AoE. LVM read only snapshots are rather
safe.

I'll be making the transition from file backed to lvm as we get more
hardware in... But the AoE looks promising as well. We used the
file-backed VBD's to be ready for live migrations in the future...

If you don't want to use LVM, setup ocfs2 and join dom-0 and the nas to
the same cluster.

Format a big partition on the nas (ocfs2) , create your loops there as
you would normally with ext3 file systems. Export it as 0 0 and all of
your xen nodes (dom-0) have access.

Mount the big partition via AoE on dom-0, get the loop and boot it, then
use the sysrq method to sync and rdiff to backup as you said. Doing
this, however is only really basically emulating what a lvm/clvm
snapshot would do. However I admire your drive toward simplicity, I have
no love for LVM either, but realize its use in this type of setting,
despite bad past experiences.

This would of course work with NFS, but nfs will be a bottleneck.

> After this I can rsync the backup directory to another server while
> the domU continues to run. This is based almost entirely on the
> xen-server-tools backup script by Christian Wieke from
> xmlvalidation.com.

Again, use AoE. Faster, easier ..

>
> We have all the domU's on the NFS to ease migration in case of
> hardware failures, and I need to be able to restore a backup on
> another machine within minutes of any critical failure. I believe the
> above solution will work for file-based VBD's, albeit not the best or
> fastest solution around.

I think with a better network fs and medium (aoe and ocfs2) you'd get
the performance increase you want and simplify things, while removing
the hassle of nfs. Migration would also be very easy. Remember, loops
can be exported as block devices via AoE too :) I'm still going to
officially recommend LVM + AoE as it solves all of your problems, but
completely understand a reluctance to use it and the need for something
a little different.

I'll definitely look into this setup. I was trying to avoid LVM two
reasons (Roger, this one's for you), the apparent snapshot issue that
everyone seems to complain about throughout the list and the issue of
live migrations between hosts. But this is only using LVM as a backend
for the domU's. I use LVM on the NAS with ext3 for the ability to grow
my partitions as needed and resize the ext3 fs on top of the volumes,
and it works beautifully.

Luke, I'll setup a test server here using read-only LVM snapshots and
keep on backing them up to see if I get the lockups that the previous
posts in the archive (and the kernel archives) mention. I'll report my
successes here when I'm done.

Thanks all!

--

Kenneth Kalmer
kenneth.kalmer@xxxxxxxxx

Folding@home stats
http://fah-web.stanford.edu/cgi-bin/main.py?qtype=userpage&username=kenneth%2Ekalmer

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users