[Xen-ia64-devel] RE: Timer merge

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

(Continue with Eddie's mail with some of my thoughts)

>From: Magenheimer, Dan (HP Labs Fort Collins)
>
>Delivering all ticks to all guests is certainly not
>scalable.  Say there are 1000 lightly-loaded guests sharing
>a single processor server.  The entire processor would be
>utilized just delivering all the ticks to each guest!
>
>Is this what Xen/x86 does?

Eddie explained the difference in last mail. Since you also agree
there's no need to inject all ticks into all guest, this factor somehow
reduces the necessity to always inject virtual timer interrupt that
quick in fast path. (Yes, a clean fast path is always welcomed. ;-)

>
>> Drawback of this way may let guest
>> application observe time stagnant within small time slot.
>
>Hmmm.... can you explain?  Are you talking about a guest
>application that is making system calls to count jiffies
>(which I think is a Linux-wide problem) or a guest application
>that is reading the itc?  In the current model, the itc
>is always monotonically increasing unless the guest operating
>system sets itc.

Please forget this incautious assertion when I'm sleepy... ;-)

>>
>> Now HZ is defined as 100 in config.h, however current itm
>> modification policy actually makes this periodic value
>> useless. Even when itm_delta is added and set into itm, an
>> immediately following ac_timer softirq will reprogram the itm
>> to the closest time point in the ac timer list.
>
>This sounds like a bug (but on the path for scheduling domains,
>not delivering guest ticks, correct?)

This is a bug if you want to keep a periodic HZ concept in hypervisor.
(Maybe we can abandon it and only keep a one-shot timer model later).
It's unrelated to this discussion, and just raised it since mentioned in
the context.

>
>>      - When vcpu is allowed to be scheduled to different
>> physical cpu (host smp), itc on different physical cpu is
>> unlikely to be exactly same even after sync.
>
>This is much less frequent so doing extra work here is OK.

IMO, the extra work should at least keep a global monotonic increasing
variable to hold itc value at last time interrupt. Then the delta value
of guest itc should base on this global variable, instead of local itc.
When emulating read_from_itc, we may return by formula like "(now_itc >
monotonic_itc) ? (now_itc + vitc_delta) : (monotonic_itc + vitc_delta)".
This can promise itc never going backward when vcpu migration. En,
anyway this is just some rough thought immediately raised, and we
definitely need elaborate and detail discussion in design for vcpu
migration. Currently we just need a guest ITM together with ITC delta,
as Eddie suggested.

>
>>      - For large machine like NUMA, the itc is completely
>> un-synchronized driven by different ratio. People need to
>> access global platform timer for different cpus to have a
>> base monotonic time base.
>
>Agreed.  But this is an operating system problem that
>is currently being discussed and solved in the Linux
>community.  I would prefer to see the problem solved
>by Linux and then (to the extent possible) leverage
>that solution.

Since SGI guys are also doing something on XEN/IPF, I'm afraid that
we'll see possible issue on SN ia64 platform soon. IFIRC, SN platform
has un-synchronized ITC, and thus requires a platform RTC timer source
to adjust system time. Also SN box has been supported well on current
linux release. Maybe some SGI people can jump out to give expertise idea
here. ;-) 

I agree we should enhance current code step by step without anything
regressed, (even linux timer source has been modified so many times). We
can keep such parallel discussion to learn more ideas from community, at
same time with agreement to merge current time code with minimal impact
in first step.

Thanks,
Kevin

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel