This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


RE: Xen system skew MUCH worse than tsc skew (was RE: [Xen-devel]RE: [PA

To: "Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>, "Keir Fraser" <Keir.Fraser@xxxxxxxxxxxxx>, "Xen-Devel (E-mail)" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: Xen system skew MUCH worse than tsc skew (was RE: [Xen-devel]RE: [PATCH] record max stime skew (was RE: [PATCH] strictly increasinghvm guest time))
From: "Ian Pratt" <Ian.Pratt@xxxxxxxxxxxxx>
Date: Wed, 23 Jul 2008 00:07:53 +0100
Cc: Ian Pratt <Ian.Pratt@xxxxxxxxxxxxx>, Dave Winchell <dwinchell@xxxxxxxxxxxxxxx>
Delivery-date: Tue, 22 Jul 2008 16:08:26 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20080722162717359.00000001992@djm-pc>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <C4AA0829.245DF%keir.fraser@xxxxxxxxxxxxx> <20080722162717359.00000001992@djm-pc>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-topic: Xen system skew MUCH worse than tsc skew (was RE: [Xen-devel]RE: [PATCH] record max stime skew (was RE: [PATCH] strictly increasinghvm guest time))
> I'm not positive yet, but I think I have an explanation for
> this.  The issue is not HOW LONG it takes to execute the
> calibration function but WHEN relative to other processors
> the calibration function executes.  If jitter on the platform
> timer occurs and the (e.g. two) calibration functions are triggered
> "temporally maximally distant" (e.g. cpu0 at 1.0, 2.0, 3.0
> and cpu1 at 1.5, 2.5, 3.5), their differing slope during the
> interim partial-second could result in greater skew.  Since activity
> on a processor will result in different locks held, interrupts
> on/off, etc, system load differences between processors is more
> likely to cause distance to vary between the scheduled calibration
> functions on each processor.

If you want to test this theory, you can easily get all the CPUs to
recalibrate at the same instant, though it's a bit expensive:

Get one CPU to issue an smp_call_function on all CPUs (including
itself). The called function should atomic_inc a variable and then spin
waiting reading the count until all CPUs have reached this point. When
this happens, turn interrupts off, atomic_dec the same counter, spin
until it hits zero, then read the TSC, re-enable interrupts, finish.

The TSC reads should all happen very close to each other. One of the
CPUs could read the platform timer after the TSC to tie everything

The only thing that could mess this up would be NMI's or SMI's. You
could at least detect that by reading the TSC after all CPUs have
incremented the counter, and check that only a "reasonable" amount of
time had elapsed. If not, set a flag to indicate that a recalibration is
required (you'd need to add another gather loop to enable all CPUs to
vote on whether they're happy).


Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>