[Xen-devel] RE: Live migration fails due to c/s 20627

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> Add rdtscp emulation has such problem that, in Intel VMX, the 
> vmexit control for rdtsc and rdtscp is the same, so if we trap
> rdtscp for emulation, OS will suffer from looooots of rdtsc vmexit,
> which will bring performance downgrade.

OK, I see... you probably are not aware of all the recent
work in Xen in this area.  Machines that do not have rdtscp
support are highly likely to be emulating rdtsc anyway.
This is true for both PV and HVM domains.

Please check docs/misc/tscmode.txt in a xen tree from
the last few days.

But in any case, I am not suggesting any change in
RDTSC_EXITING... that code is fine.  I am suggesting
adding code similar to the emulate_invalid_rdtscp
for PV in c/s 20504 to catch the illegal instruction
traps on machines where the instruction is illegal.

Dan

> -----Original Message-----
> From: Xu, Dongxiao [mailto:dongxiao.xu@xxxxxxxxx]
> Sent: Tuesday, December 15, 2009 9:10 AM
> To: Dan Magenheimer; Keir Fraser
> Cc: Jeremy Fitzhardinge; xen-devel@xxxxxxxxxxxxxxxxxxx; Kurt Hackel;
> Dugger, Donald D; Nakajima, Jun; Zhang, Xiantao
> Subject: RE: Live migration fails due to c/s 20627
> 
> 
> Dan Magenheimer wrote:
> > Hi Dongxiao --
> > 
> > Why would you disable live migraton between two
> > very widely used Intel processors for ALL HVM domains
> > just because some domains use the rdtscp instruction?
> 
> Dan, I won't disable the migration. As Keir said, I will put
> the cpuid logic in xc_cpuid_x86.c so that admin can use
> configuration file to mask rdtscp feature through cpuid. 
> This is the common usage model for live migration 
> between two different hosts.
> 
> > 
> > Why not just add the code to do rdtscp emulation,
> > which would NOT break live migration?
> 
> Add rdtscp emulation has such problem that, in Intel VMX, the 
> vmexit control for rdtsc and rdtscp is the same, so if we trap
> rdtscp for emulation, OS will suffer from looooots of rdtsc vmexit,
> which will bring performance downgrade.
> 
> > 
> > There are many cases where rdtsc/rdtscp instructions
> > are emulated and so most of the code is already there.
> > You only need to intercept illegal instruction traps,
> > so there is not a significant performance issue.
> > And the code to do the emulation is necessary
> > to implement the pvrdtscp algorithm on hvm anyway
> 
> I think in HVM environment, we should respect the native
> behavior. Moreover, it would be valuable for
> guest if it could get the node/cpu info which reflects
> hardware topology.
> 
> Thanks!
> Dongxiao
> 
> > (which I think was the reason this whole discussion
> > started).
> > 
> > Dan
> > 
> >> -----Original Message-----
> >> From: Xu, Dongxiao [mailto:dongxiao.xu@xxxxxxxxx]
> >> Sent: Monday, December 14, 2009 9:40 PM
> >> To: Keir Fraser; Dan Magenheimer
> >> Cc: Jeremy Fitzhardinge; xen-devel@xxxxxxxxxxxxxxxxxxx; 
> Kurt Hackel;
> >> Dugger, Donald D; Nakajima, Jun; Zhang, Xiantao
> >> Subject: RE: Live migration fails due to c/s 20627
> >> 
> >> 
> >> Keir Fraser wrote:
> >>> On 14/12/2009 18:02, "Dan Magenheimer" 
> <dan.magenheimer@xxxxxxxxxx>
> >>> wrote: 
> >>> 
> >>>> This may be true in concept, but existing tools (including
> >>>> the default xm tools) do NOT check for this... I just
> >>>> tested a live migration between a Nehalem (which supports
> >>>> rdtscp) and a Conroe (which does not).  The live migration
> >>>> works fine and the app using rdtscp runs fine on the
> >>>> Nehalem and then crashes when the live migration completes
> >>>> on the Conroe.  I *know* of existing code in Oracle
> >>>> that will be broken by this!
> >>> 
> >>> This is a general problem for migration between dissimilar
> >>> processors. The solution is to 'level' the feature sets, 
> by masking
> >>> CPUID flags from the more-featured processor. In this 
> case you would
> >>> mask out RDTSCP (and perhaps others too). This does need 
> the RDTSCP
> >>> flag setting/clearing to be moved to xc_cpuid_x86.c, as currently
> >>> the user cannot override the policy wedged into the hypervisor
> >>> itself. That's an easy thing to fix.
> >> 
> >> I will write a patch for this. Thanks!
> >> 
> >> -- Dongxiao
> >> 
> >>> 
> >>>  -- Keir
> >>> 
> >>> 
> >>> 
> >>> _______________________________________________
> >>> Xen-devel mailing list
> >>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel