Re: [Xen-devel] Time skew on HP DL785 (and possibly other boxes)

[Keir Fraser <keir.fraser@xxxxxxxxxxxxx>]

On 03/04/2009 23:23, "Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx> wrote:

> I think I still have a real concern here.  Let me see if
> I can explain.
> 
> The goal for Xen timekeeping is to ensure that if a guest
> could somehow magically read any of its virtual clocks
> (tsc, pit, hpet, pmtimer, ??) on all its virtual processors
> simultaneously, the values read must always obey this
> "virtual clock law":

We can do this for all except TSC for HVM guests because there virtual TSC
is hardwired onto the physical TSC (plus a configurable offset). If TSCs run
at significantly different rates then that will be hard to hide from the
guest. Luckily Windows is pretty robust to iffy timers, and no doubt
particularly suspicious of TSCs in multiprocessor environments.

Everything else builds on Xen system time, and Xen system time should just
require each CPU's TSC to be individually stable. This is true even with
your 3.3 patch to rendezvous and snapshot all TSCs at the same instant in
time. This doesn't rely on all TSCs running at the same rate! The approach
should work just as well if they run at their own separate stable rates off
separate crystals. I think the benefit of your patch was in sync'ing system
time across all CPUs at the same time, which significantly reduced maximum
divergence.

One concern I have however, is Intel's X86_FEATURE_CONSTANT_TSC logic. This
was added by them to prevent TSCs from diverging due to Cx deep sleep
states, by observing that usually all TSCs will tick at the same exact rate,
so all that needs to be done is to rewrite all AP TSCs to that of the BP
periodically. This seems to work well on small systems, but the trigger for
this mode is rather suspicious. CONSTANT_TSC feature means that a CPU's TSC
is invariant across frequency/voltage changes -- it *doesn't* mean that all
TSCs across a large MP box are at matched frequency! I wonder whether this
optimisation will bite us on big iron? Probably it ought to disable itself
if it detects significant TSC divergence, or at the very least maybe we
should add a command-line option to disable (or enable?) it.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel