WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Stability of XEN under high load

Thomas,

thanks for your input, but we are sure we do not have an arp problem.

The phenomenon is that suddenly under heavy load a vif stops working - no connection to others on the bridge is possible any more then, it does not send out anything (not even arp requests are sent, then). Since this is happening under heavy load the mac of this vif is well listed in all concerned arp caches of others on the bridge.

In the mean time we found out that we can trigger this phenomenon with certain tests that create massive network I/O. Is there any known bugs with xen 3.0.3 on ia64 concerning network load?

Dirk

--On 1. Dezember 2008 19:41:02 +0800 Thomas Goirand <thomas@xxxxxxxxxx> wrote:

On Sun, Nov 30, 2008 at 12:13 AM, Bernd Gotschy
<Bernd.Gotschy@xxxxxxxxxxxxxx <mailto:Bernd.Gotschy@xxxxxxxxxxxxxx>>
wrote: Major problem is  that under high load we loose some of the
    network interfaces and cannot get them working unless we reboot the
    guest.

We have find out quite the opposite way. If one of the VM has no
activity, the ARP vs IP cache of most switch will discard the MAC of the
VMs. So we have made a very simple python script that we deploy on all
of our Xen servers, that simply does a ping of all VMs it finds in the
/etc/xen/auto folder. This way, the ARP cache gets refreshed, and
everything is working fine. Maybe this is what you are experiencing, no?
Anyway, bellow is the script, I hope that helps (take care, the procs =
line might appear on 2 lines as I didn't do an attachment).

Thomas

# !/usr/bin/env python

import glob
import re
import subprocess
import os
import sys

pathspec = "/etc/xen/auto/*"
regexp = re.compile(r"vif.*?=.*?ip=([0-9\. ]+)'")

files = glob.glob(pathspec)

contents = ( file(t).read(-1) for t in files )

def ips(contents):
        matches = regexp.findall("\n".join(list(contents)))
        for match in matches:
                mips = match.split(" ")
                for ip in mips: yield ip

devnull = file("/dev/null","w")
procs = (
subprocess.Popen(["ping",ip,"-c","1"],stdin=devnull,stdout=devnull,stderr
=devnull) for ip in ips(contents) )

returncodes = ( proc.wait() for proc in list(procs) )

sys.exit(sum(returncodes))


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users



--------------------------------------------------------------
Dirk H. Schulz
IT Systems Service
Wiesenweg 12, 85567 Grafing
Tel. 0 80 92/86 25 68
Fax. 0 80 92/86 25 72
--------------------------------------------------------------
Technik vom Feinsten - und das nötige Tuning

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users