RE: [Xen-devel] [PATCH] pvcpuid: mask TSC invariant bit for various circumstances

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx] 
> On 27/10/2009 01:09, "Dan Magenheimer" 
> <dan.magenheimer@xxxxxxxxxx> wrote:
> 
> >> This only affects PV domUs. I suggest patching pv_cpuid() to
> >> catch dom0, and
> >> then really all domUs should be handled by
> >> xc_cpuid_apply_policy(), which
> >> pushes CPUID info down to the hypervisor, as called from xend's
> >> image.py:setCpuid(). The advantage of this is that the
> >> hypervisor does not
> >> need to be informed of the 'disable_migrate' flag at all 
> -- it can be
> >> assumed to be true for dom0 -- and it can be handled entirely
> >> in the tools.
> >> Hence I can get rid of the new domctl, which is nice.
> > 
> > I'm not done using the disable_migrate flag yet though.
> > This was just low-hanging fruit.
> 
> Okay. Well I nack this new patch anyway.

Frankly, I'm  not very thrilled with the hack-y pvcpuid
requirement anyway.  It seemed a nice architected interface
if cpuid would work, but would be too easy for a well-intentioned
app developer to use cpuid instead of pvcpuid and make
incorrect assumptions that would result in app breakage.

So let me reiterate my objective and perhaps we can come
up with something better:

An app needs to definitively determine programmatically
once when it is starting up if it can always safely use
rdtsc or must fall back to always use an OS API.

It can safely use rdtsc (and rdtsc will be faster than
the OS API) if any of the following are true:

1) tsc_native==0
2) nomigrate==1 AND physical-TSC-Invariant==1 AND
a)   ( tsc_native==1 OR
b)      ( tsc_native==0 BUT Xen-forces-tsc_native-to-1 ) )
3) tsc_native==1 AND nomigrate==0 AND 
a)     ( ( physical-TSC-Invariant==1 AND incarnation==0 ) OR
b)       ( incarnation>0 BUT Xen-forces-tsc_native-to-0 ) )

It can safely use rdtscp by post-applying pvclock scaling if:

4) pvrdtscp==1

It must use the OS API if:

5) tsc_native==1 AND
     ( physical-TSC-Invariant==0 OR
        ( incarnation>0 AND XEN-never-overrides-tsc_native ) )

Choice 3b follows the Intel proposed HVM approach of using
native TSC if no migration has occurred and switching to
emulated TSC after any migration.

Choice 2b is where I was headed with the nomigrate option:
Since TSC is "safe" when physical-TSC-invariant and nomigrate
is set, there's no need to emulate rdtsc.

I'm beginning to wonder if tsc_native should be a tri-state
with the default being "adaptive":  Xen guarantees rdtsc
is always monotonically-increasing regardless of the physical
machine and regardless of migration but if at all possible
uses native rdtsc for best performance.

In fact, by making it a quad-state, the pvrdtscp option could
be incorporated as well.

tsc_native==0: "adaptive", always correct but performance
                when possible
tsc_native==1: best performance, correctness be d*mned
tsc_native==2: always emulate for correctness, reduced but
                consistent performance
tsc_native==3: best performance, all apps must use rdtscp+pvclock
                to guarantee correctness

I can describe these more completely if this seems like
a promising direction.

Dan

P.S. In case it's not obvious, "incarnation==0" means
"domain-has-never-migrated" and "incarnation>0" means
"domain-has-migrated-at-least-once".  Also save/restore
is considered a migrate.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel