Re: [Xen-devel] Xen cluster n/w performance (again!)

[Ian Pratt <Ian.Pratt@xxxxxxxxxxxx>]

> Hey everyone,
> 
> A few weeks back there was some discussion on Xen's I/O performance in
> clusters on the list. I did some experiments today myself using iperf
> (not ttcp):
> 
> o Xen dom0 talking to another machine in the cluster running native
> linux: b/w around 904Mbps, thats nice :)
> 
> o Xen VM (running on the same machine as the dom0 in the previous
> experiment) talking to another machine running native linux (again,
> same as in previous experiment) only achieves 128 Mbps
> 
> I read on the list that you folks at Cambridge got upto 800+ Mbps
> across VMs? Did you guys do any special optimizations or set any
> special parameters? I read something about socket buffer size?

We did our measurements with a 128KB socket buffer on a 3.0GHz dual
Xeon. 

It does make a difference as to whether dom0 and domU are sharing
the same CPU, on different CPUs, or on different hyperthreads of
the same package.

At least in our experiments, with an MTU of 1500 bytes things
were pretty good regardless of the CPU allocation, but with an
artificially reduced MTU of 552 bytes we were definitely seeing
the advantage of having two CPUs.

There have been a couple of reports on the list of people seeing
unexpectedly low numbers, so something odd must be going on on
some systems. Please can you give more information about your
setup. When doing the experiments it would be interesting to know
the number of interrupts per second reported by the dom0 and domU
in each configuration.

I guess the best approach to solving this is probably to add more
instrumentation to the netfront/back drivers and export the data
via a /proc/interface. It's possible that we're getting into a
situation whereby the pipelining is breaking down and we're only
transferring a couple of packets per context switch. 

Ian

Here's the actual data we recorded:
                                                                                
                MTU 1500        MTU 552
                TX      RX      TX      RX
Linux SMP       897     897     808     808
dom0            897     898     718     769
domU UP         897     843     436     379
domU HT         897     897     651     577
domU SMP        897     897     778     663
(VMWare)        291     651     101     137
(UML)           165     203     61      91


-------------------------------------------------------
This SF.Net email is sponsored by: InterSystems CACHE
FREE OODBMS DOWNLOAD - A multidimensional database that combines
robust object and relational technologies, making it a perfect match
for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/xen-devel