Re: [Fwd: Re: [Xen-devel] [RFC] XI Shadow Page Table Mechanism]

["Robert Phillips" <rsp.vi.xen@xxxxxxxxx>]

I am replying for Ben.
-- rsp

On 6/22/06, Ben Thomas < bthomas@xxxxxxxxxxxxxxx> wrote:

-------- Original Message --------
Subject: Re: [Xen-devel] [RFC] XI Shadow Page Table Mechanism
Date: Wed, 21 Jun 2006 15:31:59 -0500
From: Anthony Liguori < aliguori@xxxxxxxxxx >
To: Ben Thomas <bthomas@xxxxxxxxxxxxxxx>
CC: xen-devel@xxxxxxxxxxxxxxxxxxx
References: < 44998B2C.2000109@xxxxxxxxxxxxxxx>

Ben Thomas wrote:
> This post contains the design document for what is currently known
> as the "XI Shadow Mechanism".  This is a design for shadow
> page table code for fully virtualized HVM domains running on a 64-bit
> Xen hypervisor.
>
> This work was undertaken to address a number of goals.  These are
> enumerated in the document and include:
>
> - ability to run fully virtualized 32, 32PAE and 64 bit guest
>   domains concurrently on a 64-bit hypervisor

This isn't supported currently?  Since an HVM must go through 16 bit, 32
bit, and 64 bit mode to boot up, how can we start more than one guest at
a time currently if this doesn't already work?

Ed Smith's daily test results show there have been problems with SMP.  We haven't diagnosed these problems but, from reading the pre-XI shadow code, it's not clear how it copes with multiple VCPUs in the same domain running in different modes.

> - support live migration of fully virtualized domains

Sweet.  What about the current shadow page code limited this?

Nothing intrinsic.  Just a SMOP.

> - provide good performance and robustness
>
> This design has been implemented and is currently being tested.
> It has been supporting the variety of memory models as noted above,
> and using widely used Windows and Linux distributions (SuSe,
> RedHat and others). At a point in the near future, a patch
> will be available.
>
> This design center is the x86-64 architecture. It is not our
> intent to completely replace all shadow page management, and
> we've attempted to limit the scope of change.
>
> A preliminary version of this design concept has undergone
> brief review with some members of the Xen community. We hope
> that this is of value to the Xen community and welcome your
> feedback and comments.

Can't really comment as it's not quotable.  Some questions that
immediately come to mind are:

- how do you deal with large pages within the hypervisor?  do you
coalesce or just hope there is contiguous pages available?

We just hope that contiguous pages are available.  However we allocate pages for the guest in large extents to maximize this likelihood.  In practice this is very effective for guests created soon after boot time.  There may be fragmentation problems later.

- what is the performance benefit in saving the shadow pages for each
domain?  there's clearly a memory trade-off here so understanding the
performance gain seems important.

The performance benefit appears to be substantial but we have not done a thorough study yet.

- OOM can be dealt with in the existing code by just invalidating
existing mappings to free up pages.  what advantages do your approach
have to this?  (i realize we don't do this today but in theory, we could).

Ultimately XI deals with OOM  by tearing down cached shadow pages, just as you say.  But it uses LRU to pick the victims.

Interesting stuff.  I'm eager to see the code.

Regards,

Anthony Liguori

> Thanks,
> -b
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel


--
------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
bthomas@xxxxxxxxxxxxxxx                             Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                    Lowell, MA 01851

------------------------------------------------------------------------
Robert S. Phillips                          Virtual Iron Software
rsp.vi.xen@xxxxxxxxxxxxxxx             Tower 1, Floor 2
978-849-1220                                 900 Chelmsford Street
                                                    Lowell, MA 01851

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel