|
xen-users
Re: [Xen-devel] Hi,something about the xentrace tool
George Dunlap wrote:
There's no copying from the HV to the xentrace process. But there is
copying from xentrace to the dom0 kernel for the output file. Some
copying is necessary right now, because rather than writing out the
pages verbatim, xentrace writes out the pcpu before writing out each
record:
void write_rec(unsigned int cpu, struct t_rec *rec, FILE *out)
{
size_t written = 0;
written += fwrite(&cpu, sizeof(cpu), 1, out);
written += fwrite(rec, sizeof(*rec), 1, out);
if ( written != 2 )
{
PERROR("Failed to write trace record");
exit(EXIT_FAILURE);
}
}
If we wanted to make it zero copy all the way from the HV to the disk,
we could have the xentrace process one stream per cpu, and do
whatever's necessary to use DMA. (Does anyone know if O_DIRECT will
do direct DMA, or if one would have to use a raw disk?)
So you're saying if we didn't have to write the cpu number, then we
could bypass stdio, and directly do a write() using the trace buffer?
And this would be better because it would avoid a memory to memory copy,
and use DMA immediately on the trace buffer memory? Do I understand you
correctly? Assuming this is what you mean, allow me to correct a slight
logic flaw. Stdio is there for a reason; Doing lots of raw I/O using
very small buffers is highly inefficient. There's the overhead of kernel
entry/exit and of setting up and tearing down DMA transactions. And
writing to a block device will result in I/O's that are multiples of the
devices' block size, so writing a 32 byte trace record will probably
cause a 512-byte block to actually be written to disk. So bypassing
stdio in this case will result in lots more disk accesses, lots more dma
setup/teardown, and lots more system calls. In other words, the
performance is going to horrible. The Stdio library greatly reduces all
this overhead by buffering stuff in memory until there's enough to make
a genuine I/O relatively efficient. In this case, the memory copies are
intentional and beneficial; We do not want to eliminate them in our
quest for "zero copy".
Rob
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
|
|
Home