RE: [Xen-ia64-devel] SMP guest and itc

[Alex Williamson <alex.williamson@xxxxxx>]

On Tue, 2006-02-14 at 09:26 +0800, Dong, Eddie wrote:

> Base on my understanding, the ITC drift between different processor
> after fixup done in Linux or Xen today is less than 100ns. So I think
> that is not a big issue as if we guarantee the guest doesn't  see
> backward time. (As VP migration usually take longer than 100ns)

Hi Eddie,

  This is why I suggest pre-sync'd ITC are probably sufficient for now.
However, we will need to support systems where the ITC between
processors drift.  In such a case, we either need to expose a better
time source to the guest (perhaps a paravirtualized time interpolator)
or fabricate ITC values for the guest which make the ITCs appear
synchronized.  The latter feels like it could be a bottleneck.

> For the gettimeofday() concern, I don't agree. Because even we support
> full virtualization, an paravirtualized guest can still get guest ITC
> quickly by exposing the formula to guest or accessing share memory (X86
> use share memory).

   Have a look at kernel/timer.c:time_interpolator_get_counter().  ITCs
cannot be perfectly synchronized, therefore all ITC time interpolators
have "jitter".  We ensure that we never see time go backwards by keeping
track of the last cycle count we returned.  Storing this cycle counter
requires a cmpxchg.  As soon as we get multiple CPUs doing
gettimeofday(), we get contention in the cmpxchg.  Multiple CPUs doing
gettimeofday() simultaneously can potentially cause us to read the ITC
many, many, many times.  If each read causes a trap into xen, the
performance implications could be severe.  This is why larger systems
provide HPETs or other similar platform time sources.  The ITC based
time interpolator does not scale well to large SMP systems.

>       Anyway, gettimeofday is not frequently accessed, so it is not a
> big cake:-)

   I disagree and note that Linux/ia64 implements fsys_gettimeofday to
avoid even entering C code to make a fast gettimeofday call.  Thanks,

        Alex


_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel