RE: [Xen-devel] RE: Saving/Restoring IA32_TSC_AUX MSR

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

Well, although it might be nice to be able to use
rdtscp and TSC_AUX to determine pcpu/vcpu/pnode/vnode
information, I think Jeremy and Jan convinced me in
another thread a couple of months ago that in userland:

x = vgetcpu()
do_other_stuff();
y = vgetcpu()

if x==1 and y==2, there's no way to determine that
do_other_stuff() was executed on cpu 1 vs cpu 2,
or (though unlikely) even on cpu 3.  And if
x==y==4, there's  no guarantee that do_other_stuff()
is executed on cpu 4.

If this is true the only safe use of TSC_AUX is for
its originally designed intent: To determine if two
successive rdtscp instructions were or were not
executed on the same processor.  Since this cannot
be guaranteed in a VM, that's a reasonable argument
that TSC_AUX shouldn't be exposed at all (meaning the
rdtscp bit in cpuid should be turned off by Xen).

True, as long as the information is ONLY used
heuristically to obtain pcpu/vcpu/pnode/vnode info,
and no guarantee of correctness is implied or expected,
it might be useful some of the time.

But frankly, if "performance sucks" when the heuristic
fails due to the fact that the app is running on
a VM instead of native OS, I'd see that as a problem
and suggest the proper way to fix that is to define
more App-to-Xen ABIs so that the app can get the
real information, not a heuristic.  Which also argues
for Xen leaving the rdtscp bit in cpuid turned off

Dan

> -----Original Message-----
> From: Nakajima, Jun [mailto:jun.nakajima@xxxxxxxxx]
> Sent: Friday, December 11, 2009 12:30 PM
> To: Jeremy Fitzhardinge; Dan Magenheimer
> Cc: Keir Fraser; Zhang, Xiantao; Xu, Dongxiao;
> xen-devel@xxxxxxxxxxxxxxxxxxx; Dugger, Donald D
> Subject: RE: [Xen-devel] RE: Saving/Restoring IA32_TSC_AUX MSR
> 
> 
> Jeremy Fitzhardinge wrote on Fri, 11 Dec 2009 at 10:50:29:
> 
> > On 12/11/09 10:35, Dan Magenheimer wrote:
> >>> However, the vcpu number is definitely useful to usermode
> >>> apps, so they
> >>> can get some idea how they're moved between (v)cpus.  I don't
> >>> think it
> >>> will matter to them that it isn't pcpu.
> >>> 
> >> My point is that an app running on native Linux can
> >> safely assume that, if TSC_AUX==3 at time T1 and
> >> TSC_AUX is still 3 at time T2,it is running
> >> on the same processor and the same node at both T1
> >> and T2.  In a virtual environment it cannot even
> >> assume it is running on the same machine.
> >> Further if the app sees that TSC_AUX==2 at time T3
> >> and TSC_AUX==3 at time T4, on native Linux it
> >> can safely assume that it is running on a different
> >> processor.  While rarer, in a virtual environment,
> >> this may also be a false assumption.
> >> 
> >> That's why I say the information is misleading.
> >> 
> >  Sure, but that info is, at best, of heuristic value, and 
> won't cause
> > any correctness problems if it is wrong.  The performance 
> may suck, but
> > that's part of the larger problem of running NUMA-aware code in a
> > virtual environment.
> > 
>  
> And to utilize various NUMA optimizations in the kernel/apps 
> in the guest, we need "the virtual numa info bears some vague 
> resemblance to the real topology" (from Jeremy's email) with 
> the vcpus bound to the CPU/node.
> 
> I understand that enabling RDTSCP in HVM will disable the 
> pvrdtscp algorithm if used by the kernel. One way is to mask 
> off the feature in CPUID (by default). Then kernel won't use it. 
> 
> Jun
> ___
> Intel Open Source Technology Center
> 
> 
> 
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel