[Xen-ia64-devel] RE: Timer merge

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

>From: Magenheimer, Dan (HP Labs Fort Collins)
>
>The current (non-VTI) code is not perfect but it *is* fast.  This
>is due a great deal to the fact that all the operations are
>handled either with "fast reflection" or "fast hyperprivop"
>code.  This code reflects/emulates operations without requiring
>all the state save/restore and stack switching that is necessary
>to call C.  Indeed it is done without even turning on psr.ic
>because all data that is accessed is pinned by TRs.  As a
>result each operation takes on the order of 100 cycles, as
>opposed to 1000-2000 cycles if C must be called.  And there are
>several (8-10 IIRC) operations per guest timer tick.

Yes, this is a fast path to reflect guest timer interrupt, which we
didn't note before. But considering that IPF Linux can catch up losing
ticks based on ITC as a monotonic increasing timer source, the
requirement for accuracy of virtual timer injection may not look so
strictly. To some extent, to let guest catch up may have better
performance than triggering as many machine interrupts as what guest
wants. Because you can save much cycles to do context switches in that
way. Drawback of this way may let guest application observe time
stagnant within small time slot. Of course actual performance difference
needs future benchmark data. But this is a factor we need to balance.
;-)

>
>The core Xen code for handling timers is all in C so using
>any of it will slow every guest timer tick substantially,
>thus slowing the entire guest substantially.  It may be
>possible to write the semantic equivalent of the Xen ac_timer
>code in ia64 assembly, but this defeats the purpose of sharing
>the core Xen code.  Also, I'm doubtful that walking timer
>queues can be done with psr.ic off.

It's cleaner to consolidate all places modifying machine itm into one
uniform interface (Ac_timer here). This conforms to common interface and
also benefits merge process. If using above policy to inject less
interrupt, the benefit of assembly is a bit reduced and instead show
more error-prone.

>
>Note that hypervisor ticks (primarily used for scheduling
>timeouts) are much less frequent (32 hz?) so not as
>performance-sensitive.  The current code does call C for
>these ticks.

Now HZ is defined as 100 in config.h, however current itm modification
policy actually makes this periodic value useless. Even when itm_delta
is added and set into itm, an immediately following ac_timer softirq
will reprogram the itm to the closest time point in the ac timer list. 

>
>In short, I am open to rearchitecting the timer code to
>better merge with VTI.  However the changes should not have
>a significant effect on performance.  And anything that calls
>C code multiple times at 1024hz almost certainly will.

Agree. Actually this area is the one missing enough discussion for a
long time. We need to make progress without breaking anything. Since we
begin this discussion, broader usage model should also be considered for
future support:
        - When guest is presented with multiple vcpus, current guest
linux smp boot code will try to sync itc and thus write to itc.
        - When vcpu is allowed to be scheduled to different physical cpu
(host smp), itc on different physical cpu is unlikely to be exactly same
even after sync.
        - For large machine like NUMA, the itc is completely
un-synchronized driven by different ratio. People need to access global
platform timer for different cpus to have a base monotonic time base.

All these cases in my head just pose the importance of a scalable and
well-organized time mechanism for both system time keep and virtual time
emulation. To implement all in assembly code seems frighten me. Without
virtualized itc (by offset) and itm, it's difficult to handle above
cases. This is why Eddie gave the proposal as the below of this thread.

However, current assembly approach is also a good research direction to
consider. Whether we can short-circuit in some special case is also the
way to gain maximum performance. We just need balance, but let's draw
out a achievable goal first. ;-)

Thanks,
Kevin
>
>Dan
>
>> -----Original Message-----
>> From: Dong, Eddie [mailto:eddie.dong@xxxxxxxxx]
>> Sent: Tuesday, August 23, 2005 9:07 PM
>> To: Magenheimer, Dan (HP Labs Fort Collins)
>> Cc: Yang, Fred; Tian, Kevin; Xu, Anthony; Mallick, Asit K; Dong,
Eddie
>> Subject: Timer merge
>>
>> Dan:
>>      We are looking for simplest way to merge the timer code
>> together now. Probably you still remember the discussion
>> several months ago, now it looks like we are much easier to
>> merge together as the XEN ac_timer is moved to one-shot timer.
>>
>>
>>      Current code is setting machine ITM to smallest one of
>> HV timer (HZ=1024), guest ITM (suppose 1024HZ) and next
>> ac_timer. How about following changes to support both VTI & non-VTI?
>>      1:  machine ITM is set only by next ac_timer (current
>> non VTI HV also do this)
>>      2: Setting guest ITM will be done by adding a new
>> ac_timer for that ITM. (expire = requested_iTM - current ITC)
>>      3: HV timer probably is not necessary now, but up to you.
>>      4: xen_timer_interrupt do corresponsive modification to
>> reflect above changes.
>>
>>
>>      Any suggestion?
>> Thx,eddie
>>
>>

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel