[Xen-devel] TSC scaling and softtsc reprise, and PROPOSAL

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

While at Linux Symposium last week, I heard a rumor that VMware ESX
always traps and emulates all rdtsc instructions.  (Can anyone confirm
or deny this?)

This reminded me that I'm not sure we came to any conclusion
for proper handling of TSC in Xen, though I think that the
scaling patch was taken into xen-unstable, meaning that some
users will unknowingly be using softtsc (all rdtsc instructions
fully emulated) when live migrating between machines with
different Hz rates.  This could lead to the bizarre situation
where a time-sensitive SMP app might fail in cryptic ways if
it has never migrated, but work fine if it has.

(Here's the last discussion I think:
http://lists.xensource.com/archives/html/xen-devel/2009-06/msg00980.html)

I dug up some old measurements from when we first implemented
softtsc that I think showed that emulating TSC averages around
one microsecond on my Conroe box.  John Levon's measurements showed
that Solaris' mstate accounting was doing rdtsc at a frequency of
about 3000/sec (per processor on an idle system), which would
translate to a fraction of a percent of CPU time, even for this
very excessive use of rdtsc.

While I'd like to see my measurement independently confirmed
(and on a wider variety of old and new systems); and some better
(heavy-workload) data on mstate accounting tsc frequency; and
a rerun of the oltp workload that showed poor (10%?) results
to prove that this number is real and not just apocryphal;
this raw data leads me to the following:

PROPOSAL:

The default mode for all xen systems should be that all rdtsc
instructions should be emulated by xen using xen system time
as the timestamp counter (i.e. nanosecond frequency).

The no-softtsc Xen boot option remains available to force the
non-trapping mechanism if desired.  It might make sense to
add a per-guest config option to override per guest.

The Xen CPU info emulation should reflect that tsc is constant
and safe to use on an SMP.

Comments?  I think someone at Intel (Eddie?) was studying the
TSC emulation path to see if it could be faster, but I'm not
sure where that ended up.

Thanks,
Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel