Re: [Xen-devel] MPI benchmark performance gap between native linux anddomU

[xuehai zhang <hai@xxxxxxxxxxxxxxx>]

Ian,

Thanks for the quick response! Explainations to your comments are inline below.

> > I did the following experiments to explore the MPI 
> > application execution performance on both native linux 
> > machines and inside of unpriviledged Xen user domains. I use 
> > 8 machines with identical HW configurations (498.756 MHz dual 
> > CPU, 512MB memory, on a 10MB/sec LAN) and I use Pallas MPI 
> > Benchmarks (PMB).
> > The expreiment results show, running a same MPI benchmark in 
> > user domains usually results in a worse (sometimes very bad) 
> > performance comparing with on native linux machines. The 
> > following are the results for PMB SendRecv benchmark for both 
> > experiments (table1 and table2 report throughput and latency 
> > respectively). As you may notice, SendRecv can achieve a 
> > 14.9MB/sec throughput on native linux machines but can get a 
> > maximum 7.07 MB/sec throughput if running inside of user 
> > domains. The latency results also have big gap.
> > I will appreciate your help if you had the similar experience 
> > and wanna share your insights.
> Xen (or any kind of virtualization) is not particularly well suited to
> MPI type applications, at least unless you're using Inifiniband or some
> other smart NIC that avoids having to use dom0 to do the IO
> virtualization.
>
> However, the results you are seeing are lower than I'd expect.
>
> Are you running dom0 and the domU on the same CPU or different CPUs. How
> does changing this effect the results?
I did not specify "cpu" option in Xen's configuration file, so I think both dom0 and domU run on the 
 same CPU (1st CPU). I will try to assign them to different CPUs later.
> Also, are you sure the MTU is the same in all cases?
The outputs of "ifconfig" show MTU is 1500 in all cases.

> Further, please can you repeat the experiements with just a dom0 running
> on each node.
I will do it and update you later.

Thanks again for the help.

Xuehai

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel