RE: [Xen-devel] [PATCH] replace rdtsc emulation-vs-native xen boot option with per-domain (hypervisor part)

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

>     Any other necessary reasons to introduce it except fixing 
> skew issue between cpus?

I think the reasons were discussed at length in a previous
thread.  To summarize, some applications already do (or can
or will) use rdtsc as a legal non-privileged instruction
and expect to get the Intel SDM-defined behavior of the
instruction.  There are some hardware+software environments
that do not correctly provide this behavior, including
some older SMP machines and early generation AMD desktop
multi-core machines.  There are many more that DO correctly
provide this behavior, including all recent generation
Intel machines and nearly all recent generation AMD
machines... and, very importantly, any VM running on
VMware.   Because rdtsc is "unsafe" on some machines
does not stop application programmers from using it
on machines where it is safe; it will be increasingly
likely that an application programmer may never
experience an "unsafe" hardware/software environment.

We cannot tell app programmers that they cannot use rdtsc,
only warn them that it is risky for some older machines.
Some will use it anyway and I have personally talked to
some that are.  We cannot predict or legislate
how rdtsc will be used in the future.  It is easy to
imagine a scenario where a transaction-oriented application
timestamps transactions with rdtsc and then, when an
infrequent error condition occurs, tries to replay the
transactions; this would work fine on any new hardware
and on VMware and even on Xen for awhile... but without
rdtsc-emulaion could mysteriously fail and cause data
corruption, perhaps only after a migration (or two or
three) and only when the infrequent error condition
occurs after a few migrations.  Would you want to be
on the support team that tries to diagnose that?

VMware does not have this problem.  The cost for Xen is
some performance.  I do not take loss of performance
lightly and have spent weeks now looking for a better
solution.  I have not found one and am open to any
creative alternative.  But pretending that apps, today
and in the future will NOT use rdtsc, or that they
will use it only following Xen-prescribed constraints,
is just wishful thinking and not appropriate for
hardware/software vendors selling to enterprise
customers (or selling to cloud providers that expect to
provide services for enterprise customers).

The patch provided records and reports frequency of
rdtsc emulations.  If an administrator cares to improve
performance and can verify that all apps that are (now
or ever will be) running on this VM are using
rdtsc safely, a per-domain option can be specified
to get the performance back.  The opposite is nearly
impossible to ascertain.

So I believe, for rdtsc, correctness is more important
than a small amount of performance.

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel