WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] network problems

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-users] network problems
From: Mike Lovell <mike@xxxxxxxxxxxx>
Date: Thu, 23 Jul 2009 19:37:52 -0600
Delivery-date: Thu, 23 Jul 2009 18:38:45 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4A67976D.1020502@xxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <4A67976D.1020502@xxxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird 2.0.0.22 (X11/20090608)
Mike Lovell wrote:
So, I have been running into network problems for a while on 4 boxes that I installed xen on so that some engineers have places to test code. This particular problem is happening on all 4 of these boxes. (although, it isn't happening on an older box running xen from debian etch).

What appears to be the problem is that traffic is getting dropped between the vif#.0 interface in dom0 and the eth0 interface in the guest. To find this out, I started a ping flood from one domU that was pinging another domU. About every 10 minutes, there will be a lot of ping requests going out but no replies coming back. I think it really weird that it happens like ever 10 minutes plus about 2 seconds. While the ping was going, I did tcpdumps from the domU starting the ping, from the vif#.0 of the pinging machine, from the virtual bridge, from the vif#.0 for the receiving guest, and then from the receiving domU. The packets are making it all the way to the dom0 vif for the receiving guest but not making it to the eth0 in the guest. I have no clue why this is happening and it happens in rather regular intervals. The same thing happens in pinging a different guest and it happens in about the same interval but at different times. Also, during the ping flood, there is never a pause in the sending of packets out of the guest. Only a pause on the packets going from the host to the guest.

I am running this on 64 bit Debian Lenny using the distribution's packages. xen-hypervisor-3.2-1-amd64 version 3.2.1-2 and linux-image-2.6.26-2-xen-amd64 version 2.6.26-17. Here are the networking configs.

---------
dom0# cat /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
   address 10.135.7.34
   netmask 255.255.255.224
   network 10.135.7.32
   broadcast 10.135.7.63
   gateway 10.135.7.33
# dns-* options are implemented by the resolvconf package, if installed
   dns-nameservers 10.135.7.34
   dns-search qa1.mozyops.com

auto vmnet
iface vmnet inet static
       address 10.135.2.71
       netmask 255.255.255.224
       bridge_ports eth1
#        bridge_stp off
#        bridge_fd 9
#        bridge_hello 2
#        bridge_maxage 12

---------
DomU# cat /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp
  post-up ethtool -K eth0 tx off

---------
Dom0# brctl show vmnet
bridge name    bridge id        STP enabled    interfaces
vmnet        8000.003048c8166d    no        eth1
                           vif1.0
                           vif10.0
                           <other interfaces>
---------

Does anyone have any ideas as to what is going on here? Or more importantly, any ideas on how to solve this? I have tried building a newer domU kernel from scratch but I haven't been able to make any progress there. The guest fails to boot without showing anything on the console. It then goes into this loop of trying to reboot the guest but failing. I would really like to stay with the debian kernels.

I have been banging my head against a wall for a week or so on this and desperately need some help to get this working. I have engineers that are getting held up by this bug.

This problem still exists.

I tried setting an independent wallclock on all of the virtual machines. I also managed to miss that I had the wrong netmask configured for the vmnet bridge. It should have been 255.255.255.128. The vms were able to talk to each other before changing the netmask and I saw traffic flowing past the switch.

Does anyone have any clue as to what might be going on? I am great need of some help here.

Thanks

mike

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>