[Xen-devel] rdtsc in userspace

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

I'm continuing to investigate the usage of rdtsc in
userspace and whether there are programs "out there"
that use it "unsafely" that might randomly break
under Xen if rdtsc is not emulated, e.g. across a migration.
Some have argued that nobody should use rdtsc and any
programs that use rdtsc directly are "fundamentally broken"
so the default for rdtsc emulation should be off.

In measuring rdtsc usage in the kernel, both Jeremy
and I noticed that compiling the kernel causes a
large number of userland rdtsc's.  At first I thought
that this meant that gcc was using rdtsc, but gcc
sources do not show any use of rdtsc.  Next I suspected
bash, but it doesn't either.  Finally, it appears that
the calls are occurring from glibc, and searching for
uses of rdtsc, I found that glibc has its
own implementation of clock_gettime that uses rdtsc
directly!

The manpage for clock_gettime ("man 3 clock_gettime")
has a lengthy caveat about using clock_gettime on
an SMP system BUT provides a mechanism to test to
see if your SMP system is safe, clock_getcpuclockid(0).
See for example (near the end):

http://linux.die.net/man/3/clock_gettime

But testing this on a system I know is unsafe, the
test says my system is safe!  Looking at the source
code, I can see why... the test is a compile-time
test, not a run-time test, and on x86 it appears that
it will always pass.

So I wrote a simple test program that uses clock_gettime().
It does indeed do userland rdtsc's.

So... any program that uses clock_gettime to approximate
the passage of time is subject to break on Xen across
a migration.  Even if the programmer uses the prescribed
method for testing "safeness", it is subject to break.

True I still can't point to a specific example of breakage,
but this appears to be a much broader problem than code
where the programmer has explicitly used asm-like
constructs to directly utilize the rdtsc instruction,
in spite of dire warnings to not use it.

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel