WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] RDMA based Live Migration

To: "Ian Pratt" <Ian.Pratt@xxxxxxxxxxxxx>, <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-devel] RDMA based Live Migration
From: "Robert Dunkley" <Robert@xxxxxxxxx>
Date: Fri, 2 Jan 2009 12:11:33 -0000
Cc:
Delivery-date: Fri, 02 Jan 2009 04:15:36 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <C1EAC9C5E752D24C968FF091D446D8232DEEFB@ALTERNATEREALIT> <DD74FBB8EE28D441903D56487861CD9D3BFC5324@xxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AclsxiL6DiOYyEaARe6jVPximpr6HwABTK5gAAHjKsA=
Thread-topic: [Xen-devel] RDMA based Live Migration
Here are some results comparing Ininiband with IPOIB TCP/IP migration to
Infiniband native RDMA migration:
http://nowlab.cse.ohio-state.edu/publications/conf-papers/2007/huang-clu
ster07.pdf

4 to 5 times quicker is about the average so it seems it could bring big
advantages for those that use Infiniband or IWarp NICs. Any chance of it
being considered for a future version?

Rob

-----Original Message-----
From: Ian Pratt [mailto:Ian.Pratt@xxxxxxxxxxxxx] 
Sent: 02 January 2009 12:07
To: Robert Dunkley; xen-devel@xxxxxxxxxxxxxxxxxxx
Cc: Ian Pratt
Subject: RE: [Xen-devel] RDMA based Live Migration

> I saw someone had done patches for this and run benchmarks a while ago
> but still no mention of support within Xen. Is support for RDMA based
> migration planned? (Or even better is it already present?)

I don't think I've seen any patches for doing live relocation using
RDMA, but it wouldn't be hard to create them.

However, the benefits may not actually be that great:  when doing a live
relocation you're usually moving a VM from a loaded machine to a
less-loaded one, hence it's the sending machine that you care most about
CPU usage. Assuming the NIC you're using has TSO, it should be
relatively efficient to send the data over TCP. However, this could
definitely be optimized by adding a zero copy TX mechanism to avoid the
copy from user space to the kernel, and this would benefit all NICs, not
just RDMA. It requires a little bit of work to do this as sendfile isn't
a good fit and the foreign pages will likely need page structs which
isn't the case at present.

Ian




> 
> 
> Thanks,
> 
> Rob
> 
> The SAQ Group
> 
> Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ
> SAQ is the trading name of SEMTEC Limited. Registered in England &
> Wales
> Company Number: 06481952
> 
> http://www.saqnet.co.uk AS29219
> 
> SAQ Group Delivers high quality, honestly priced communication and
I.T.
> services to UK Business.
> 
> Broadband : Domains : Email : Hosting : CoLo : Servers : Racks :
> Transit : Backups : Managed Networks : Remote Support.
> 
> ISPA Member
> 
> Find us in http://www.thebestof.co.uk/petersfield
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>