IMHO, this all comes down to how bad the tsc drift gets
between calibrations. If this is agreed, let me propose
the following: I see local_time_calibration() has
some old printk's. I propose re-enabling them (with
some rate-limiting) to record to a log how bad the
skew gets. Then we can request feedback from anyone
running xen-unstable (and maybe xen-3.1-latest and
xen-3.2-latest also) so we can "measure" a broad set of
machines.
One nice approach to rate-limit is to printk each time
the value exceeds the next higher power of two. Even
though this printk gets output for each processor,
I'd think this would be low overhead and sufficient
information for our needs.
A nice touch would be to include a little script to
run in domain0 that collects the cpuinfo, the relevant
log lines, and emails (or UDPs?) them to a pre-set address
(which is viewable via http). And maybe a once-per-log
printk that says "please run this script".
Even tied to a boot parameter, this would be better
than the lack-of-information we have now.
Dan
> -----Original Message-----
> From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx]
> Sent: Friday, April 11, 2008 3:21 PM
> To: Dave Winchell
> Cc: dan.magenheimer@xxxxxxxxxx; Ian Pratt; Tian, Kevin;
> xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [xen-devel] System time monotonicity
>
>
> On 11/4/08 21:00, "Dave Winchell" <dwinchell@xxxxxxxxxxxxxxx> wrote:
>
> > I turned to the hpet as I became frustrated trying to solve
> the problem
> > in xen with pit.
> > One of the solutions proposed to the customer was a
> max(curr time, last
> > time) modification to Linux.
> > They didn't want that.
> > (Keir, what Linux version are you looking at when you say
> Linux already
> > has this modification?)
>
> This is part of our own Xen-specific time patches for Linux.
>
> > I had tried hpet before to solve the time backwards problem
> and knew it
> > was effective.
> > But the accuracy of hpet was very poor. When I looked into
> the hpet I
> > was surprised that it was
> > based on tsc, as I was tring to get away from tsc. But
> note, even based
> > on tsc the time was not
> > going backwards, at least for this simple test case.
>
> Yes, having all the virtual timers based on 'guest TSC'
> (which really is
> basically host TSC + an offset) is not great.
>
> > Its a fairly simple matter to base the hpet on the physical
> hpet. Its
> > easy to share it among guests
> > as no one really writes the physical hpet. Offsets are kept in each
> > vhpet such that each guest thinks
> > he owns the hpet.
>
> This is really no better than basing on Xen system time.
> Actually it's worse
> since most systems don't even expose the HPET, so we can't
> probe it (without
> hacks) and so we can't use it. Xen's system time abstraction,
> perhaps with
> the max(last, curr) addition, is perfectly good enough.
>
> > This goes along with some of the experiences
> > Keir has had with drift, I think. I'm not sure why this
> happens - can
> > the hpet hardware be that poor in quality?
>
> It does appear to be, and I have no idea why.
>
> > There are three factors that give hpet its great accuracy,
> in my opinion.
> > 1) The hardware is very stable.
> > 2) There is only one of them in the system, not one per cpu.
> > 3) The Linux implementation for clock and hpet is very clean. It
> > calculates missed ticks and offsets without
> > including the interrupt delay.
>
> Encouraging the guest to use HPET makes sense. It's a nice
> wide counter
> which hence does not have the wrap issues of the 16-bit PIT
> counters. Also
> in some cases the guest OS interface to the HPET is saner
> (for our purposes
> at least) than the equivalent code to interface to PIT/TSC.
> This doesn't
> mean it has to be plumbed right down to the physical HPET.
> HVM time sources
> can be fixed for drift by moving them away from guest/host
> TSC and onto the
> Xen system time abstraction.
>
> -- Keir
>
> > Items 2 and 3 here are important factors in why the time stays
> > monotonic. Another reason is that
> > gettimeofday reads the hpet main counter for extrapolation,
> eliminating
> > extrapolation error since
> > the same counter is the sole determinator for the next
> interrupt time
> > stamp. Furthermore, Linux can take the
> > clock interrupt on any processor and the monotonicity is preserved
> > because of item 2.
> >
> > Thanks for reading this far!
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home