Re: [Xen-devel] Why is 'emulate' as good as writable PT's?

[Andrew Theurer <habanero@xxxxxxxxxx>]

Ian Pratt wrote:
> > I was wondering, perhaps we are not just triggering writable pagetables when we 
> > shouldn't, but maybe we are flushing them back too early.  I
> > added some xen perf counters to get an idea of why we are flushing back
> > wtpt's (run on SDET again):
> Are these numbers taken on a uniprocessor guest (or dom0?)
Yes.
> >                         modified:  0 <=10  <=20  <=30  <=40  <=50
> > 1 writable pt updates    T=1086   0  612   194   111    49    85
> > 2 ptwr_fl: called from ptwr_emulated_update because wtpt exists T=0
> > 3 ptwr_fl: called from ptwr_do_page_fault because wtpt is used T=338
> > 4 ptwr_fl: called from spurious_page_fault  T=0
> > 5 ptwr_fl: called from fixup_page_fault  T=0
> > 6 ptwr_fl: called from cleanup_wpt, do_mmuext_op (active) T=467
> > 7 ptwr_fl: called from cleanup_wpt, do_mmuext_op (inactive) T=0
> > 8 ptwr_fl: called from cleanup_wpt, update_va_mapping (active) T=280
> > 9 ptwr_fl: called from cleanup_wpt, update_va_mapping (inactive) T=0
> > 10 ptwr_flush: called from cleanup_wpt, do_mmu_update (active) T=1
> > 11 ptwr_flush: called from cleanup_wpt, do_mmu_update (inactive) T=0
> > line 3:  I think we can just goto emulate instead of flushing back the
> > wtpt here, right?  I've tried this, but no real difference in
> > performance.  Could we increase the number of wtpt's we keep track of,
> > so we don't have to flush back or emulate?
> This will happen as part of a fork when we move on to the next page in
> the PT. It should be harmless unless we're flopping back and forth.
OK
> > line 6: We seem to call cleanup_writable_pagetables unconditionally
> > here, and if either of the active or inactive pages are used, they get
> > flushed back.  Do we always need to do this?
> What's the op? is it a TLB flush, invplg, or cr3 load?
I don't know, but I'll find out.
>  
> > line 8: Also call cleanup_writable_pagetables unconditionally here.
> > Do the wtpt's always need this to happen?  Is is possible the
> > update_va_mapping call is for an address space which does not affect
> > the wtpt?
> It's interesting to understand what the interaction is here. I'd like to
> know 
>
> > line 10: Not seeing many flushes here, so I guess it's not an issue.
> >
> > Sorry if these questions seem odd.  There's a good chance I am not
> > "getting it" :)
> This is useful work. It's been on our todo list to re-profile this on
> newer kernels. Once upon a time we had it quite nicely tuned...
>
> Could you find out all the kernel EIPs that are triggering writeable
> pagetables with any frequency and list them for us. It might be good to
> turn everything into using update mmuop and then just turn on direct
> writes just for the fork case which is where we know need it.
I think I can do that.  I'll just use something similar to the EIP 
logging Xen has for finding out what triggered the wtpt flushes.
Thanks,

Andrew


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel