WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot.

To: Pasi Kärkkäinen <pasik@xxxxxx>
Subject: Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot.
From: J Nb <j_nwb@xxxxxxxxx>
Date: Thu, 14 Jan 2010 10:34:30 -0800 (PST)
Cc: xen-users <xen-users@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Thu, 14 Jan 2010 10:36:31 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1263494070; bh=D2M15xN8ajl648WnGHRORnWnAuER57sK4n2KKZCfBqA=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=r4wnJsZZYCqrgqmDDmZd39EJvLHzXacE1U2DwepOLCinRcTZdhVG18hdRtf5EX58CPYltpzlXaSyELsRqkUTjU/TOcKWnR+5sp84Q0sSkR/k0FEiFHlppKSyULgVpIy6Nr7/sI7Kk56qa0mRsirh77Qjgo+sRBzAPdVhRm3iu18=
Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=W50qnLjDepweio4GxTsNnvUKgsLfCBiDNVIKlqhtOWMI2EnVC6geGotOA1EAuY530N1W8TMAFc15vIt2St1bo3+HIoe9KkWf1aM+/PU64O9TltEXRTYfbus19y55B1aCl0aWzvbfdXB37BacktusaWKrxgCG+SxRsyyVXIfzug8=;
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20100114171552.GL25902@xxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
I will look in to XCP.. but my guess is that they use their pv drivers to 
communicate via xenstore..etc. I am looking for something that would work with 
vanilla open source xen. (well non XCP  :) )

But my plan is same as I described in skeleton and organization very similar to 
yours. but to have something that works with ssh/wmi. This way scripts etc can 
be centrally managed.

What I was looking for was 
 -- For xen is there a flush command for PV and HVM vms. (hypervisor level)
 -- Is sync sufficient as a starting point at OS level.
 -- Any repository of application specific scripts. (Mysql,oracle..etc)
 -- And anything to control cpu/io consumption of the backup process. (Probably 
start with cstream / nice)

-nb

--- On Thu, 1/14/10, Pasi Kärkkäinen <pasik@xxxxxx> wrote:

> From: Pasi Kärkkäinen <pasik@xxxxxx>
> Subject: Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot.
> To: "J Nb" <j_nwb@xxxxxxxxx>
> Cc: "xen-users" <xen-users@xxxxxxxxxxxxxxxxxxx>
> Date: Thursday, January 14, 2010, 9:15 AM
> On Wed, Jan 13, 2010 at 09:11:47AM
> +0200, Pasi Kärkkäinen wrote:
> > On Tue, Jan 12, 2010 at 05:44:18PM -0800, J Nb wrote:
> > > Anyone ?
> > > 
> > 
> > Now that XenServer is opensourced as XCP (Xen Cloud
> Platform), you could
> > take a look at the source.. 
> > 
> > I believe XenServer supports Windows VSS coordination
> for snapshots.
> > "xe vm-snapshot-with-quiesce"
> > 
> 
> Did you get anywhere with this? 
> 
> For a long time I've been thinking of writing some kind of
> 
> "VSS" library/daemon for Linux.. similar to Microsoft
> Windows 
> Volume Snapshot Services.
> 
> It could (and should) be called something different than
> VSS, 
> but I'll just use that name for now..
> 
> The interface/usage could be something like:
> 
>     1. backup begins
>     2. execute in the guest: "vss_control
> pre-snapshot xvda"
>     3. take snapshot of the lvm volume
> having guest disk xvda
>     4. execute in the guest: "vss_control
> post-snapshot xvda"
>     5. do the actual backup from the lvm
> snapshot
>     6. backup ends
> 
> This would have minimal downtime for the applications
> running in the guest.
> 
> pre-snapshot action would go through all the application
> specific pre-scripts, 
> execute them, preparing the applications for taking the
> snapshot, and then 
> it would flush the filesystem, kernel and disk caches as
> needed to prepare for
> a consistent snapshot.
> 
> Preparing the application for taking the snapshot might
> mean something
> like placing locks to database tables preventing disk
> writes from happening, 
> or shutting down the application, or whatever the specific
> application requires.
> 
> post-snapshot action would again go through and execute all
> the application specific
> post-scripts releasing the applications back to normal
> running state.
> 
> The application specific pre/post scripts could be stored
> like this:
> 
>      
> /etc/vss/scripts/xvda/001-mysql.pre
>     /etc/vss/scripts/xvda/001-mysql.post
> 
>     /etc/vss/scripts/xvda/002-pgsql.pre
>     /etc/vss/scripts/xvda/002-pgsql.post
> 
>    
> /etc/vss/scripts/xvda/003-my_custom_app.pre
>    
> /etc/vss/scripts/xvda/003-my_custom_app.post
> 
> Or you could have "scripts/xvda2/*" if you have lvm volumes
> directly as
> partitions to the guest. You get the idea.
> 
> Then the next step would be to create a daemon ("vssd")
> that listens for
> requests from the network, and/or possibly from Xen dom0
> through the hypervisor.
> 
> Or then the backup server might issue those pre-snapshot
> and post-snapshot 
> commands through ssh.. many options how to implement it.
> 
> Does this make any sense? Comments?
> 
> -- Pasi
> 
> > 
> > > 
> > > --- On Sun, 1/3/10, J Nb <j_nwb@xxxxxxxxx>
> wrote:
> > > 
> > > > From: J Nb <j_nwb@xxxxxxxxx>
> > > > Subject: coordinating VM snapshot with LVM
> snapshot.
> > > > To: xen-devel@xxxxxxxxxxxxxxxxxxx
> > > > Date: Sunday, January 3, 2010, 6:17 PM
> > > > Hi
> > > >    How to co-ordinate VM snapshots and
> disk
> > > > snapshots ? Here is a sequence I am thinking
> of
> > > > automating...
> > > > 
> > > >   1.Tell VM to sync (say via ssh or some
> special
> > > > agent)
> > > >      * This will do "sync" and any
> > > > application specific hook to make it
> quiescent.
> > > > 
> > > >   2. Tell VM/Hypervisor to flush all disk
> buferes
> > > > ?  (HOW ??? )
> > > >      * Anything at this level?
> > > > 
> > > >   3. Pause the VM
> > > > 
> > > >   4. xm save -c (To save the VMs state in
> to
> > > > checkpoint file.)
> > > >       (This may take more time !)
> > > > 
> > > >   5. Take LVM snapshot for each disk. 
> > > > 
> > > >   6. Unpause the VM
> > > > 
> > > >   7. Application level resume if any.
> > > >   
> > > >   7. Use the snapshot to copy each disks
> ?!!!! (either
> > > > dd or mount followed by cp/rsync)  (This
> can be big!
> > > > Any optimization suggestions )   
> > > > 
> > > >   8. discard snapshot.
> > > > 
> > > > 
> > > > Would the above sequence work ? 
> > > > 
> > > > Any suggestions on individual steps ?
> > > > 
> > > > Anything specific for PV and HVM virtual
> machines?
> > > > 
> > > > Is there a backdoor to qemu-dm console ?
> > > > 
> > > > Does anyone know how similar functionality
> is implemented
> > > > in XenServer product or Virtual Iron or
> anyone else? 
> > > > 
> > > > -nb
> > > > 
> > > > 
> > > >       
> > > > 
> > > 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
> 





_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users