[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] LTTng Xen port

I have read your tracing thread and I am surprised to see how much things you would like in a tracer are already implemented and tested in LTTng.

The Xen tracer was originally a fairly simple hack, although it has been updated a number of times to improve the scope of its functionality.

A more advanced tracing tool would be nice.

I am currently porting my tracer to Xen, so I think it might be useful for you to know what it provides. My goal is to do not duplicate the effort and save everyone some time.

Here follows some key features of LTTng :

Architecture independant data types

This should please those developing with cross compilers, so would be nice.

Extensible event records

Definitely good.

Self-describing traces
Variable size records

Definitely good. Self-describing traces would be particularly useful - for our current xentrace, I had proposed adding a hypercall that would return event ID -> event description string mappings.

Fast (200 ns per event record)
Highly reentrant
Does not disable interrupts
Does not take lock on the critical path

Cool. We've been running with xentrace compiled by default for about a year now, although trace buffers are not usually allocated on production systems. It would be nice to keep the overhead low enough to continue to do this. Refinements like being able to enable specific event types might be nice, but aren't critical.

Supports NMI tracing
Analysis/visualization tool (LTTV)

Visualisation would be really useful - poring through trace files (even once formatted) is pretty hairy. Some kind of timeline visualisation tool (I think I've seen one for LTT at some point) would rock.

- Polling for data in Xen from a dom0 process.
Xentrace currently polls the hypervisor each 100ms to see if there is data that needs to be consumed. Instead of an active polling, it would be nice to
 use the dom0 OS capability to put a process to sleep while waiting for a
resource. It would imply creating a module, loaded in dom0, that would wait
 for a specific virq coming from the Hypervisor to wake up such processes.
 We could think of exporting a complete poll() interface through sysfs or
procfs that would be a directory filled with the resources exported from the Hypervisor to dom0 (which could include wait for resource freed, useful when shutting down a domU instead of busy looping). It would help dom0 to schedule
 other processes while a process is waiting for the Hypervisor.

We can actually do this already, it's just that xentrace doesn't implement it ;-) See xenbaked's code: it sleeps waiting for an event channel, which gets fired by Xen when the buffer has had a chance to fill up a bit. The nice bit is that the /dev/evtchn driver is compiled into Xen by default *anyhow*, so a new kernel module isn't required.

You might also be interested in looking at : - the website (http://ltt.polymtl.ca) - LTTng Xen port design document (this one is different from the one posted by
- OLS 2006 paper "The LTTng tracer : A Low Impact Performance and Behavior
 Monitor for GNU/Linux"

A few questions: How much commonality will this have with the Linux LLTng? Will there be direct code sharing, or a patchqueue, or simply code copying? Will existing userspace LTTng tools be applicable? Is there already a working visualiser? Is there a source repository online that we could take a look at?

It would be really great if you came up with something interesting on the tracing front: quite a lot of people seem to use it these days.


Questions and constructive comments are welcome.


OpenPGP public key: http://krystal.dyndns.org:8080/key/compudj.gpg Key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68

Xen-devel mailing list

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.