Re: [Xen-devel] A migration framework for external devices

[Stefan Berger <stefanb@xxxxxxxxxx>]

aliguori@xxxxxxxxxxxxxxxxxxxxxxx wrote on 02/09/2006
10:05:37 AM:

> Hi Stefan,
> 
> Thanks for sending this note out.  Comments inlined.
> 
> Stefan Berger wrote:
> >
> > Hello!
> >
> >  As part of our off-list discussion on how to migrate the
virtual TPM 
> > in support of virtual machine migration (live migration), we
came up 
> > with the idea of having a migration framework that could be used
for 
> > general migration of 'external devices' such as disk images and
> > possibly serialized state of device models. I was going to look
into 
> > this now and was wondering whether a framework like this is the
right 
> > approach, particularly since it would exist next to XenD, which
I 
> > believe is handling live migration ?
> I'm immediately wary of any framework for migration.  There's
been talk 
> (at least) of moving to a push/pull migration model which would avoid
> having to constantly listen for incoming migrations (and also add
a bit 
> of security too).  Any sort of framework seems like its just
going to 
> make those sort of changes harder.

The problem that I would see with a framework living
outside XenD is that you now have two different entities taking care of
migration. Certainly it should be one piece of code that does everything.
I don't understand your argument about a push/pull
migration model. I mean in certain ways everything is push/pull and push
is certainly what we have today with a command like "xm migrate <DomID>
<Host>", which effectively pushes the vm onto another machine.
What would be different in the new model?


> >  To be a bit clearer on the idea of the framework: It would
consist of 
> > a deamon running on the target machine whose different plug-ins
know 
> > how to handle the migration of different pieces of state information
> > and know how to de-serialize them (which mere 'scp' would not
do).

> Plug-ins concern me even more as it implies a
stable interface.  There's 

Plug-ins  will need to exist in some form on
another since knowledge is needed about how to migarte a specific technology
and prepare the target system for it - and maybe check the target system
first whether that technology is supported there or migration requirements
can be met. In a way they do exist today with classes like xen/xend/{pciif,netif,blkif,usbif,tpmif}.py
which are all implementing technology-specific code - not for migration,
though. 

> a lot of churning going on in Xend right now and we're just not there
> yet.  Also, I'd like to see us move away from using a daemon
for 
> migration anyway.

It depends on what you understand under 'daemon'.
The target system for migration will have to have something running that
listens for an incoming migration-request. I would call that a 'daemon'.

> 
> I'm assuming TPM migration requires bidirectional communication right?
 

Yes, it will.

> Is it static throughout the lifetime of the domain or does it change?
 

The TPM state itself is not static throughout the
lifetime of a domain. It does change - if that's what you mean.

> How much state are we talking about migrating?

It's not going to be much in terms of kilobytes or
so, but it might end up being the first device that lives outside a domain
an needs to be migrated.

> 
> I think simple patches that introduce TPM migration would be a 
> preferable start.  If we end up having a lot of code that handles
> individual device migration than we can abstract it.

My gut feeling is that we need to design a flexible
migration protocol that is is extensibile. So I am just looking around
what other poeple think, although I am doing some coding as well :-).

   Stefan

> 
> Regards,
> 
> Anthony Liguori
> > Clients on the source machine would communicate with that daemon
and 
> > transfer the state. The clients would have to be triggered by
XenD 
> > after a partition is not scheduled anymore and be given the IP
address 
> > of the target machine. Afterwards there needs to be some 
> > synchronization on resuming the scheduling on the target machine
after 
> > all state has been deserialized.
> >   The plugable deamon itself would handle the communication
sockets, a 
> > low-level protocol which the plugins and clients would use, have
> > support for timing and protocol time-outs and provide threading.
The 
> > plugins would have to do the rest of what's necessary to communicate
> > with the infrastructure and the higher-level protocol shared
with the 
> > clients.
> >   Comments?
> >
> >   Stefan
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
> >   
> 
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel