WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] LTTng Xen port : finally in a repository near you

* Mark Williamson (mark.williamson@xxxxxxxxxxxx) wrote:
> Mathieu,
> 
> Nice one, thank you for getting involved in this.  Are you planning to 
> eventually submit this for inclusion in the Xen mainline?
> 

I did the original port, but my current focus is more oriented towards
having LTTng included in Linux mainline. As you probably guess, it is
changing quite a bit in the process. I plan to first get the kernel
tracing polished, and afterward to port this "polished" tracer to
userspace tracing and hypervisor. Since a first port has been done, it
will not represent too much work to do it again.

> Is there anything we (the community) can do to help?  E.g. patch review / 
> advise / etc?
> 

You could go through a code review of the Xen port I have done. Not the
tracing infrastructure itself, since it is scheduled for major changes,
but mostly the low-level calls I did in lttd-xen, lttctl-xen and the Xen
ltt-tracer.



Note that you have to expect the current "Linux Kernel Markers" to be
ported to the Xen hypervisor eventually.

Some advice about how I currently (fail) to do the teardown of traces
correctly: I depend on synchronize_sched() under Linux to prodive a
race-less efficient teardown (using read-copy-update for my data
structures). It is a core concept I depend on to provide fast and
efficient tracing. Has RCU finally been implement in Xen since the last
time I checked?

Thanks,

Mathieu

> Cheers,
> Mark
> 
> On Monday 25 June 2007, INAKOSHI Hiroya wrote:
> > Hi Mathieu,
> >
> > I am interested in LTTng-xen because I thought that it would be nice if
> > I can get traces on both xen and guest linux at the same time.  I
> > reviewed LTTng-xen and found that
> >
> > * LTTng and LTTng-xen have a quite similar structure,
> > * a trace buffer resides in a hypervisor for LTTng-xen,
> > * it is currently impossible to get traces from guest linux because
> > there is no LTTng for 2.6.18-xen kernel, as you mentioned.
> >
> > I had coarsely ported LTTng to 2.6.18-xen, though it is only for
> > i386.  Now I can get traces on xen and guest linux simultaneously, even
> > though they put records in different trace buffers.  Then I thought that
> > it would be more useful if they put records in xen's trace buffer and I
> > can analyze events from xen and linux guests with a single lttd and
> > lttctl running on Domain-0.  Do you have an opinion about that?
> >
> > Regards,
> > Hiroya
> >
> > Mathieu Desnoyers wrote:
> > > Hello,
> > >
> > > I made a working version of the LTTng tracer for xen-unstable for x86.
> > > Here is the pointer to my repository so you can try it out :
> > >
> > > hg clone http://ltt.polymtl.ca/cgi-bin/hgweb.cgi xen-unstable-lttng.hg
> > >
> > > Basic usage :
> > >
> > > (see lttctl-xen -h)
> > >
> > > lttctl-xen -c
> > >
> > > (in a different console)
> > > lttd-xen -t /tmp/xentrace1
> > >
> > > (in the 1st console)
> > > lttctl-xen -s
> > >
> > > (tracing is active)
> > >
> > > lttctl-xen -q
> > > lttctl-xen -r
> > >
> > > lttd-xen should automatically quit after writing the last buffers as
> > > soon as lttctl-xen -r is issued.
> > >
> > > Then, you must copy the XML facilities :
> > >
> > > (see the http://ltt.polymtl.ca > QUICKSTART to see how to install the
> > > ltt-control package which contains the XML facilities in your system)
> > >
> > > lttctl-xen -e -t /tmp/xentrace1
> > >
> > > View in the visualiser : (see the QUICKSTART to see how to install it)
> > >
> > > lttv -m textDump -t /tmp/xentrace1
> > >
> > > (not tested yet) : to visualize a dom0 trace with the xen hypervisor
> > > information, one would have to collect the dom0 kernel trace and the Xen
> > > trace and open them together with :
> > > lttv -m textDump -t /tmp/xentrace1 -t /tmp/dom0trace
> > >
> > > The current Linux kernel instrumentation is for 2.6.20. A backport might
> > > be needed to 2.6.18 if there is no proper Xen support in 2.6.20 (I have
> > > not followed the recent developments).
> > >
> > >
> > > Currently broken/missing :
> > >
> > > - Ressources are not freed when the trace channels are destroyed. So you
> > >   basically have to reboot between taking different traces.
> > > - My code in the hypervisor complains to the console that subbuffers
> > >   have not been fully read when the trace channels are destroyed. The
> > >   error printing is just done too fast : lttd-xen is still there and
> > >   reading the buffers at that point. It will get fixed with proper
> > >   ressource usage tracking of both Xen and lttd-xen (same as the first
> > >   point above).
> > > - x86_64 not tested, powerpc local.h and ltt.h missing (should be ripped
> > >   from my Linux kernel LTTng).
> > >
> > >
> > > Cheers,
> > >
> > > Mathieu
> > >
> > > * Mathieu Desnoyers (compudj@xxxxxxxxxxxxxxxxxx) wrote:
> > >> Hi,
> > >>
> > >> My name is Mathieu Desnoyers, I am the current maintainer of the Linux
> > >> Trace Toolkit project, known as LTTng. This is a tracer for the 2.6
> > >> Linux kernels oriented towards high performance and real-time
> > >> applications.
> > >>
> > >> I have read your tracing thread and I am surprised to see how much
> > >> things you would like in a tracer are already implemented and tested in
> > >> LTTng. I am currently porting my tracer to Xen, so I think it might be
> > >> useful for you to know what it provides. My goal is to do not duplicate
> > >> the effort and save everyone some time.
> > >>
> > >> Here follows some key features of LTTng :
> > >>
> > >> Architecture independant data types
> > >> Extensible event records
> > >> Self-describing traces
> > >> Variable size records
> > >> Fast (200 ns per event record)
> > >> Highly reentrant
> > >> Does not disable interrupts
> > >> Does not take lock on the critical path
> > >> Supports NMI tracing
> > >> Analysis/visualization tool (LTTV)
> > >>
> > >> Looking at the integration of the existing LTTng implementation into
> > >> Xen, I came up with those two points for my Christmas whichlist :
> > >>
> > >> Additionnal functionnalities that would be nice to have in Xen :
> > >>
> > >> - RCU-style updates : would allow freeing the buffers without impact on
> > >> tracing. * I guess I could currently use :
> > >>       for_each_domain( d )
> > >>         for_each_vcpu( d, v )
> > >>           vcpu_sleep_sync(v);
> > >>       I think it will have a huge impact on the system, but it would
> > >> only be performed before trace buffers free.
> > >>
> > >> - Polling for data in Xen from a dom0 process.
> > >>   Xentrace currently polls the hypervisor each 100ms to see if there is
> > >> data that needs to be consumed. Instead of an active polling, it would
> > >> be nice to use the dom0 OS capability to put a process to sleep while
> > >> waiting for a resource. It would imply creating a module, loaded in
> > >> dom0, that would wait for a specific virq coming from the Hypervisor to
> > >> wake up such processes. We could think of exporting a complete poll()
> > >> interface through sysfs or procfs that would be a directory filled with
> > >> the resources exported from the Hypervisor to dom0 (which could include
> > >> wait for resource freed, useful when shutting down a domU instead of
> > >> busy looping). It would help dom0 to schedule other processes while a
> > >> process is waiting for the Hypervisor.
> > >>
> > >>
> > >> You might also be interested in looking at :
> > >> - the website (http://ltt.polymtl.ca)
> > >> - LTTng Xen port design document (this one is different from the one
> > >> posted by Jimi)
> > >>  
> > >> (http://ltt.polymtl.ca/svn/ltt/branches/poly/doc/developer/lttng-xen.txt
> > >>) - OLS 2006 paper "The LTTng tracer : A Low Impact Performance and
> > >> Behavior Monitor for GNU/Linux"
> > >>   (http://ltt.polymtl.ca/papers/desnoyers-ols2006.pdf)
> > >>
> > >>
> > >> Questions and constructive comments are welcome.
> > >>
> > >> Mathieu
> > >>
> > >>
> > >> OpenPGP public key:             
> > >> http://krystal.dyndns.org:8080/key/compudj.gpg Key fingerprint:     8CD5
> > >> 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> > >>
> > >> _______________________________________________
> > >> Xen-devel mailing list
> > >> Xen-devel@xxxxxxxxxxxxxxxxxxx
> > >> http://lists.xensource.com/xen-devel
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
> 
> 
> 
> -- 
> Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
> Mark: To answer a question with a question: What use is a skateboard?
> Dave: Skateboards have wheels.
> Mark: My wheel has a wheel!
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel