[Xen-devel] RE: TSC scaling and softtsc reprise, and PROPOSAL

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

>    I am still confused about why need to emulate rdtsc is 
> necessary. Even if emulating it in software, we still need to 
> find a stable time source, right?  If you think TSC is not 
> stable on SMP system, and I think the issue should exist in 
> native OS which depends on TSC as time source instead of 
> Xen-self issue.  If host's TSC is stable enough, I think the 
> hardware's TSC offset feature should be the right way to go ?  
> 
> I have a proposal on it. If Xen finds hardware's TSC is not 
> reliable, it can tell guest about the info at guest's boot 
> stage, and guest should use other  time sources(eg, hpet) 
> instead of TSC . And if the TSC is reliable in hardware, I 
> think we should let Xen try the best to use hardware's 
> feature and just leave it as current implementation. If users 
> know hardware's TSC is not reliable from his knowledge, it 
> may set softtsc to solve the possible issue manually.  So 
> maybe we only need to create a way how to tell guest the 
> TSC's status in Xen hypervisor. 

Hi Xiantao --

Thanks for the info in your previous reply.

The issue is that there's no easy way for Xen to
determine for sure if the hardware has a reliable TSC.
The TSC_CONSTANT bit in the MSR only applies to the
socket, not to the entire system.  Even if
it is possible on one box, it is not possible to
determine it for the whole data center (to handle
live migration).  Even if it is possible for the
whole data center, new machines may be live-added
to a data center that might be different.  So
no SMP application in a virtualized data center
can assume TSC is monotonic.  But SMP applications
designed on smaller multi-core physical systems
CAN assume TSC is monotonic; when these apps
are moved to virtual systems, problems will
occur (including possible data corruption).

As you've pointed out, there are other issues
with migration and power management where the
TSC frequency changes.  Unless/until there is
a TSC-scaling feature as well as a TSC-offset
feature, frequency changes will have to be
handled in software.

A virtual TSC (always trapping all rdtsc instructions)
allows us to guarantee monotonicity and provide a
constant rate (Xen time*, 1GHz) across all processors
in a machine and all machines in a data center.
There is a performance impact for applications
that execute RDTSC at a high frequency but I hope
that we can reduce this penalty somewhat.

I am proposing that the virtual TSC be the default.
We should provide a per-VM option and a Xen boot
option to allow VMs to NOT trap rdtsc, but this
should have a warning that it is not recommended
and may result in data corruption in some apps.

* Xen time by itself is not monotonic across multiple
processors but can be supplemented with a global
variable to always provide "last_tsc + 1" to
enforce monotonicity.

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel