WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

[Xen-users] Network traffic stalling between domUs on different servers

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] Network traffic stalling between domUs on different servers
From: Adam Wilbraham <adam.wilbraham@xxxxxxxxxxxxxxxx>
Date: Mon, 26 Apr 2010 12:23:45 +0100
Delivery-date: Mon, 26 Apr 2010 04:25:12 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9pre) Gecko/20100217 Shredder/3.0.3pre
Up until now I've been using a single DL380 running Xen 3.3.1 on top of Debian Lenny (using the Debian Etch Xen kernel) to host a number of domUs, however I have had to install a second dom0 (this time a DL360, again Lenny + Etch kernel, this time Xen 3.4.2) to take over some of the domUs due to load issues.

Network connectivity between domUs on the same host has always been absolutely fine, however I am having a problem with network traffic between domUs on the new dom0 when connecting to domUs on the original dom0. Basically, connections can be established but as soon as data starts to flow then the connection stalls and eventually drops.

The network on each dom0 is configured as a series of VLAN bridges, eg:

 auto eth1.38
 iface eth1.38 inet manual

 auto vlan38
 iface vlan38 inet manual
        pre-up ip link set eth1.38 down
        pre-up brctl addbr vlan38
        pre-up brctl addif vlan38 eth1.38
        pre-up ip link set eth1.38 up
        pre-up ip link set vlan38 up
        post-down ip link set vlan38 down
        post-down brctl delbr vlan38

My head keeps telling me that this issue is related to TCP checksum offloading, but I've already turned that off and set the MTU to 1496 in case the network card doesn't cope with the VLAN header properly:

        pre-up ethtool -K eth1 tx off
        pre-up ifconfig eth1 mtu 1496

The NICs in both servers are Broadcom ones, bnx2 driver.

I managed to get an NFS export mounted and vaguely stable between domUs when forcing the it to use UDP and an rsize and wsize of 1024, however thats hardly workable and only for NFS. If I try and SCP a file between boxes then only a few k gets transferred before it stalls, similar from other services.

Does anyone have any experience of similar issues?

--
Adam Wilbraham - Senior Systems Administrator
TechnoPhobia Limited
The Workstation
15 Paternoster Row
SHEFFIELD
England
S1 2BX
t: +44 (0)114 2212123
f: +44 (0)114 2212124
e: adam.wilbraham@xxxxxxxxxxxxxxxx
w: http://www.technophobia.com/

Registered in England and Wales Company No. 3063669
VAT registration No. 598 7858 42
ISO 9001:2000 Accredited Company No. 21227
ISO 14001:2004 Accredited Company No. E997
ISO 27001:2005 (BS7799) Accredited Company No. IS 508906
Investor in People Certified No. 101507

The contents of this email are confidential to the addressee
and are intended solely for the recipients use. If you are not
the addressee, you have received this email in error.
Any disclosure, copying, distribution or action taken in
reliance on it is prohibited and may be unlawful.

Any opinions expressed in this email are those of the author
personally and not TechnoPhobia Limited who do not accept
responsibility for the contents of the message.

All email communications, in and out of TechnoPhobia,
are recorded for monitoring purposes.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>