[Xen-devel] Re: Page fault is 4 times faster with XI shadow mechanism

["Robert Phillips" <rsp.vi.xen@xxxxxxxxx>]

Keir et al have not given any feedback.  Not a peep.  To be generous, though, it is a large body of code to digest.
-- rsp

On 7/2/06, zhu < vanbas.han@xxxxxxxxx> wrote:

Really thorough explanation. Now I understand all of your concerns for
the design. All of us can tune the code if it check in the unstable tree.
BTW: How about the feedbacks from the Cambridge guys?

 

Robert Phillips 写道:
> In XI, the idea is to have a pool of SPTIs all ready to go.  When a page
> needs to be shadowed, I simply pull a SPTI  off the list, zero its pages,
> and it is ready for use.  No further memory allocation is needed.  This is
> the critical path and I want it as short as possible.
That's quite reasonable. Another classic example of space to time trade-off.
>
> One reason I have backlinks on all guest pages is because one can't know
> ahead of time which guest pages are (or will become) GPTs.  When the code
> first detects a guest page being used as a guest page table, it would have
> to do a linear search to find all SPTEs that point to the new guest page
> table, so it can mark them as readonly.
When the first time we shadow it, we could know it's a GPT and then we
could connect the backlinks with the SPTE. However, the disadvantages is
just as you have noted, it will increase the complexity of the critical
shadow fault path.
>
> One could do without backlinks altogether if one were willing to put up
> with
> linear searching.  It's a space/performance tradeoff.  I think, with
> machines now having many megabytes of memory, users are more concerned
> about
> performance than a small memory overhead.
>
> -- rsp
>
>
> On 7/2/06, zhu < vanbas.han@xxxxxxxxx> wrote:
>>
>>
>> Robert Phillips ♨彔牝»:
>> > Well I don't.  I simply pre-allocate a pool of SPTI's.  It can be quite
>> a
>> > large pool but certainly not one-SPTI per MFN.  SPTIs are allocated on
>> > demand (when a guest page needs to be shadowed) and, when the pool runs
>> > low,
>> > the LRU SPTs are torn down and their SPTIs recycled.
>> >
>> Well what I mean is that we should not connect a snapshot page with a
>> SPTI at the first time the SPTIs are reserved. It would be better to
>> manage these snapshot pages in another dynamic pool.
>> BTW: What do you think of the backlink issue mentioned in my previous
>> mail?
>> > Currently I allocate about 5% of system memory for this purpose (this
>> > includes the SPT, its snapshot and the backlink pages) and, with that
>> > reasonable investment, we get very good performance.  With more study,
>> I'm
>> > sure things could be tuned even better.  (I hope I have properly
>> understood
>> > your questions.)
>> >
>> > -- rsp
>> >
>> > On 7/1/06, zhu < vanbas.han@xxxxxxxxx > wrote:
>> >>
>> >> Hi,
>> >> After taking some time to dig into your patch about XI Shadow page
>> >> table, I have to say it's really a good design and implementation
>> IMHO,
>>
>> >> especially the parts about the clear hierarchy for each smfn,decision
>> >> table and how to support 32nopae in a rather elegant way. However, I
>> >> have several questions to discuss with you.:-)
>> >> 1) It seems XI shadow pgt reserve all of the possible resources at the
>> >> early stage for HVM domain(the first time to create the asi). It could
>> >> be quite proper to reserve the smfns and sptis. However, do we really
>> >> need to reserve one snapshot page for each smfn at first and retain it
>> >> until the HVM domain is destroyed? I guess a large number of gpts will
>> >> not been modified frequently after them are totally set up. IMHO, it
>> >> would be better to manage these snapshot pages dynamic. Of course,
>> this
>> >> will change the basic logistic of the code, e.g. you have to sync the
>> >> shadow pgt when invoke spti_make_shadow instead of leaving it out of
>> >> sync, you can't set up the total low level shadow pgt when invoke
>> >> resync_spte  since it could cost a lot of time.
>> >> 2) GP back link plays a very important role in XI shadow pgt. However,
>> >> it will also cause high memory pressure for the domain(2 pages for
>> each
>> >> smfn). For these normal guest pages instead of GPT pages, I guess its
>> >> usage is limited. Only when invoke xi_invld_mfn, divide_large_page or
>> >> dirty logging, we will refer to the back link for these normal guest
>> >> pages. Is it reasonable to implement the back link only for the GPT
>> >> pages? Of course, this will increase the complexity of the code a
>> little.
>> >> 3) Can you show us the statistics between the current shadow pgt
>> and XI
>> >> pgt for some critical operations, such as shadow_resync_all,
>> gva_to_gpa,
>> >> shadow_fault and so on. I'm really curious about it.
>> >>
>> >> I have to say I'm not very familiar with the current shadow pgt
>> >> implementation so I could miss some important considerations when I
>> post
>> >> these questions. Please point it out.
>> >> Thanks for sharing your idea and code with us. :-)
>> >>
>> >> _______________________________________________________
>> >> Best Regards,
>> >> hanzhu
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Xen-devel mailing list
>> >> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> >> http://lists.xensource.com/xen-devel
>> >>
>> >
>> >
>> >
>>
>
>
>

-- 
--------------------------------------------------------------------
Robert S. Phillips                          Virtual Iron Software
rphillips@xxxxxxxxxxxxxxx                Tower 1, Floor 2
978-849-1220                                 900 Chelmsford Street
                                                    Lowell, MA 01851
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel