Re: [Xen-devel] Live migration with MMIO pages

[Kieran Mansley <kmansley@xxxxxxxxxxxxxx>]

On Wed, 2007-10-31 at 13:32 +0000, Keir Fraser wrote:
> 
> 
> On 31/10/07 12:19, "Kieran Mansley" <kmansley@xxxxxxxxxxxxxx> wrote:
> 
> > However there's no equivalent test for a non-HVM domain.  I think adding
> > such a test to just ignore any MMIO pages would help.  The MMIO pages
> > will be unmapped when the live migration finally suspends the domain
> > (and so the migration should be possible), but accessing them in the
> > mean time is probably not a good idea.  Non-live migration is fine
> > because the domain (and so all its devices) are suspended in advance,
> > and the MMIO pages are unmapped.
> 
> Is mapping the page as cacheable and then reading from it not supported by
> the hardware device? E.g., does it need to be uncacheable, or do reads have
> side effects?

The page is mapped with ioremap_nocache, and needs to be uncacheable.
Reads shouldn't have side effects (although shouldn't and don't aren't
always synonymous).  More generally I can think of devices where reads
do have side effects though.

The symptom of failure is that once live migration has started, trying
to write to an IO page results in it getting stuck in (or a perpetual
loop into) page_fault().  This only happens very occasionally.  I've
interpreted it getting stuck in page_fault() as a result of the shadow
paging which (as I understand it) marks the normal page table entries as
read only so that writes to pages trap into the hypervisor and it can
update its dirty set.

So the live migration page reading doesn't quite explain the problem
(unless the read from hardware does have side effects) but as there was
a check there for HVM MMIO pages it seemed to me that adding something
equivalent for PV would be a good idea, and if that didn't fix the
problem I could try looking at the shadow page tables and how they might
interact badly. 

Kieran


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel