[Xen-devel] Re: non-zero order allocations in shadow code may prevent live migration

[Tim Deegan <Tim.Deegan@xxxxxxxxxxxxx>]

Hi,

At 10:51 +0100 on 27 Sep (1190890315), Jan Beulich wrote:
> after a lot of walking dead end routes with a customer issue stating that he
> can't reliably run live migration I finally concluded that the problem can 
> only be
> explained by the non-zero order allocations done in shadow code (on x86-64
> and x86-32/pae).

Gosh.  Are you running with almost all memory in use and failing to
allocate shadow memory?  Have you seen sh_set_allocation return -ENOMEM
when there is enough memory on the page allocator's free lists?  Xend's
ballooning rules have been wrong more than once before, and are what I'd
suspect first.

> However, from a PV-domain-live-migration perspective it
> would seem to me that these order 2 allocations are entirely
> pointless; 

You're absolutely right.  The order-2 allocations are only used for
shadowing 32bit non-PAE guests on PAE or 64bit Xen, which only happens
for HVM guests.  (They were originally used for shadowing both kinds of
PAE guests as well but that's done differently now).

shadow_alloc_p2m_pages() only does order-2 allocations to prevent it from 
fragmenting the shadow pool and undermining the assertion that you can
always free up an order-2 block.

> So the bottom line is - sh_set_allocation() really shouldn't need to allocate
> non-zero order pages except for hvm domains.

Yep.

> As this implies quite a few changes, before going that route I'd like to
> understand whether I'm mistaken with anything here.

No, you're quite right.  It should be possible to have the shadow
allocator work with single pages for PV guests.  The shadow_prealloc()
call still needs to be able to make sure that four free pages are
available so that we can populate an entire walk of the shadow pagetable
without needing to reclaim in-use pages.  Apart from that I can't think
of any other problems right now.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@xxxxxxxxxxxxx>, XenSource UK Limited
Registered office c/o EC2Y 5EB, UK; company number 05334508

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel