RE: [Xen-devel] rdtscP and xen (and maybe the app-tsc answer I've been looking for)

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> > By constant_tsc I mean that CPUID:0x80000007:EDX:8
> > is set.
> 
> Well, if it is at least true for 99% of systems, then it 
> might be worth

Well I'm not sure how to count, but I'd venture to guess
that close to 99% of servers out there (that are new enough
to have this CPUID bit set) are single socket.  So
as long as constant_tsc applies across all cores in
a socket, your 99% test applies.  But according to
Intel and AMD, it should also apply across multiple
sockets, and according to HP, it applies on one big
NUMA machine even across enclosures.

> enabling constant_tsc support by default, and detect TSC divergence at
> runtime and disbale dynamically. I think that's what Linux 
> does (i.e., it
> has a fallback at runtime if its TSC assumptions turn out to 
> be wrong).

Indeed Linux does, and the code looks easy to leverage.
See arch/x86/kernel/tsc_sync.c where check_tsc_sync_*
is defined, used by start_secondary() and native_cpu_up()
in arch/x86/kernel/smpboot.c.  It may actually too
aggressively test for TSC reliability as it can fail
if TSC's differ by more than "a cacheline bounce",
which is a lot more restrictive than Xen cares about
(or any userland algorithm that post-processes for
monotonicity).

In fact, Linux no longer does any kind of write_tsc(0)
at processor boot but apparently instead assumes
that the BIOS has done the synchronization.  I
don't know if/how the BIOS could do a better job than
Xen's current rendezvous algorithm, but if it does,
Xen's code may not only be superfluous but also
making problems worse.  We should probably test
for divergence and only write_tsc if the
test fails?

P.S. I was looking at 2.6.30 and 2.6.31 though it
looks like check_tsc_sync been around since at least
2.6.24:  http://lwn.net/Articles/211051/

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel