This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


RE: [xen-devel] System time monotonicity

To: "Tian, Kevin" <kevin.tian@xxxxxxxxx>, "Keir Fraser" <keir.fraser@xxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [xen-devel] System time monotonicity
From: "Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>
Date: Tue, 8 Apr 2008 19:55:04 -0600
Delivery-date: Tue, 08 Apr 2008 22:37:47 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <D470B4E54465E3469E2ABBC5AFAC390F024D9192@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: Oracle Corporation
Reply-to: "dan.magenheimer@xxxxxxxxxx" <dan.magenheimer@xxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> >But I also observe that all of the hvm platform timer (pit,
> >hpet, and pmtimer) code is built on top of the physical TSC
> >plus the vmx/svm tsc_offset which doesn't seem to be affected
> >by the Xen TSC synchronization.  True?
> For cpus on same system bus driven by one crystal, TSC drift among
> cpus may be just dozen of cycles after boot time sync, which is
> negligible enough compared to migration overhead and thus
> it's unlikely
> to have HVM guest to observe a non-monotonic behavior after resume.

I agree this case is not much of a problem.

> The issue comes with cpus running on different frequency, like driven
> by multiple crystals or on-demand frequency change which affects TSC
> too. HVM guest can be configured to avoid migrating among cpus with
> different TSC freq, like limiting its cpu affinity to cpus on
> same system bus.

These are the cases I am worried about.  The linux kernel seems
to have a number of cases that mark TSC as unstable, but
Xen does not, nor (I think) does Xen expose this information
anywhere.  So it seems SMP guests need to be pinned to physical
CPUs that are measured to have sync'ed TSC's to guarantee that
the (virtual) platform timer is monotonic.

> Or you have to configure HVM guest to not trust TSC...

Yes, that's what I'm thinking... like Linux, Xen could/should
build virtual platform timers on a physical clocksource other
than tsc if all of the potential vcpu->pcpu mappings are not
on sync'd-TSC-pcpus.

I assume this problem is worse with multi-socket Hypertransport
and future Intel QPI boxes?  Or is TSC (and frequency changing)
synchronized for such systems?

Xen-devel mailing list