This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


RE: [Xen-devel] RE: [PATCH] rendezvous-based local time calibration WOW!

To: "John Levon" <levon@xxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-devel] RE: [PATCH] rendezvous-based local time calibration WOW!
From: "Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>
Date: Wed, 6 Aug 2008 09:09:06 -0600
Cc: Ian Pratt <Ian.Pratt@xxxxxxxxxxxxx>, "Xen-Devel \(E-mail\)" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Dave Winchell <dwinchell@xxxxxxxxxxxxxxx>, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Delivery-date: Wed, 06 Aug 2008 08:12:10 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20080806133829.GA28204@xxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: Oracle Corporation
Reply-to: "dan.magenheimer@xxxxxxxxxx" <dan.magenheimer@xxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Acj31l1i9fQPnh48TwS19VlBd8EbMA==
> I wonder if we couldn't do something when we know that we're scheduling
> a VPCU onto a different CPU to ensure time can't go backwards.

Again no guarantees but I think we are now under the magic
threshold where the skew is smaller than the time required
for scheduling a VCPU onto a different CPU.  If so,
consecutive gethrtime's by the same thread in a domain
should always be monotonic.

The overhead of measuring the inter-CPU stime skew is
too large to do at every cross-PCPU-schedule so doing
any kind of adjustment would be difficult.
But it might make sense for the Xen scheduler to do a
get_s_time() before and after a cross-PCPU-schedule
to detect the problem and printk if it occurs
(possibly rate-limited in case it happens a lot on
some badly-behaved machine).

> -----Original Message-----
> From: John Levon [mailto:levon@xxxxxxxxxxxxxxxxx]
> Sent: Wednesday, August 06, 2008 7:38 AM
> To: Dan Magenheimer
> Cc: Ian Pratt; Xen-Devel (E-mail); Dave Winchell; Keir Fraser
> Subject: Re: [Xen-devel] RE: [PATCH] rendezvous-based local time
> calibration WOW!
> On Wed, Aug 06, 2008 at 07:25:50AM -0600, Dan Magenheimer wrote:
> > > > I'm not sure its possible to guarantee monotonicity in
> > > > PV domains (without a global lock) except by doing a trap
> > > > or hypercall at each "get time".
> > >
> > > That's a shame.
> >
> > Further followup on this...
> >
> > I'd encourage you to put some test code in your lock to
> > see if time ever measurably goes backwards.  It may never,
> > or it may only on some ill-behaved-tsc machines or when
> > cpufreq changes occur... needs testing.  Even if it
> > does, it may be by a smaller delta than all but the
> > most sophisticated SMP applications can detect.
> I believe the normal (metal) Solaris algorithm expects any 
> inter-CPU TSC
> differences to remain static (that is, no drift), so any machine that
> breaks that is problematic:
> http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/



The presumption is that gethrtimef() is monotonically increasing, which
at least Xen 3.0.4 regularly broke. If the hypervisor has been fixed to
give as much guarantees as we got already then great.

A monotonic gethrtime() is part of the ABI so I'm not sure we can avoid
a lock even on well-behaved machines if Xen isn't correct.

I wonder if we couldn't do something when we know that we're scheduling
a VPCU onto a different CPU to ensure time can't go backwards.

Anyway, some more testing sounds like it would be interesting.


Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>