Just a followup on my attempt to track down existing
uses of rdtsc in userspace in booting an OS (for at
least one PV guest, 32-bit EL5u2). See below for
previous status.
I removed all the uses of rdtsc in glibc, which were
mostly in the dynamic linker (used to gather stats
for LD_DEBUG=statistics output). After that, there
remains 5 different "offsets" collected for userland
rdtsc (see below) and only one increments at every
process launch. Three others have small totals
(102, 8, 14) that don't change after boot. The
final/fifth is also small but seems to randomly
increment by one about once every 20-40 seconds.
While there remains a non-zero probability that any
one of these remaining rdtsc's could "break" on
an ill-timed migration, I don't think there's any
low-hanging fruit here to prove my point so am
abandoning this investigation.
> -----Original Message-----
> From: Dan Magenheimer
> Sent: Saturday, October 24, 2009 7:21 PM
> To: Jeremy Fitzhardinge
> Cc: Xen-Devel (E-mail); Keir Fraser; Tim Deegan; Avi Kivity;
> Kurt Hackel
> Subject: RE: rdtsc in userspace
>
>
> > From: Jeremy Fitzhardinge [mailto:jeremy@xxxxxxxx]
> > On 10/23/09 15:51, Dan Magenheimer wrote:
> > > In measuring rdtsc usage in the kernel, both Jeremy
> > > and I noticed that compiling the kernel causes a
> > > large number of userland rdtsc's. At first I thought
> > > that this meant that gcc was using rdtsc, but gcc
> > > sources do not show any use of rdtsc. Next I suspected
> > > bash, but it doesn't either.
> >
> > I think the dynamic linker may use rdtsc as a piece of
> randomness for
> > randomizing the addresses of libraries and other mappings.
>
> Interesting. I'll look into that. I've collected
> some other data that might appear somewhat contradictory:
> I record the page offset of eip for each emulated rdtsc
> and there are about 28 different values collected,
> of which about 20 of them increment for each
> exec'ed process. One would think/hope that there
> would only be one place where randomness is generated
> for this kind of purpose, but I suppose if it is
> inlined, it's hard to say.
>
> Anyway, thanks for the tip, I will dig further.
>
> > > Finally, it appears that
> > > the calls are occurring from glibc, and searching for
> > > uses of rdtsc, I found that glibc has its
> > > own implementation of clock_gettime that uses rdtsc
> > > directly!
> > >
> >
> > When I run clock_gettime on my system it uses the vsyscall version
> > (which is either a syscall or, with my recent patches, all in
> > usermode).
>
> I could be mistaken but I don't think my test guest,
> which is EL5u2-32bit (2.6.18-based kernel) has vsyscall.
> At least the detection mechanisms that I am aware
> of ("sysctl -a | grep vsyscall" and "ls /proc/sys/kernel/vsys*"
> don't work. Is there another way on older kernels?
>
> > Using strace on your test program should show whether its really
> > bypassing the syscall or not.
>
> You're right! Strace does show syscalls for
> each clock_gettime(), and I was mistaken... these
> rdtsc's are in the kernel, not userland.
>
> > > The manpage for clock_gettime ("man 3 clock_gettime")
> > > has a lengthy caveat about using clock_gettime on
> > > an SMP system BUT provides a mechanism to test to
> > > see if your SMP system is safe, clock_getcpuclockid(0).
> >
> > My manpages don't have anything like that for clock_gettime().
>
> RHEL5 does. Though now I am not sure how to use the
> section 3 calls.
>
> > That caveat is for CLOCK_PROCESS_CPUTIME_ID which doesn't seem
> > reasonable to count on for monotonicity (my manpages
> > CLOCK_PROCESS_CPUTIME_ID just refer to as "High resolution
> per-process
> > timer."). CLOCK_REALTIME or CLOCK_MONOTONIC is a different matter.
> >
> > Using what CLOCK_?
> >
> > I always see clock_gettime(CLOCK_MONOTONIC or CLOCK_REALTIME,
> > ...) make
> > a syscall or vsyscall. If you're testing any of the
> > thread-local times
> > then I think its less interesting.
> >
> > J
>
> I'll dig further next week.
>
> Dan
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home