WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

RE: [Xen-users] Strange network issue; Guest/DomU outgoing traffic

To: eric van blokland <ericvanblokland@xxxxxxxxx>, "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-users] Strange network issue; Guest/DomU outgoing traffic
From: "Fischer, Anna" <anna.fischer@xxxxxx>
Date: Wed, 25 Mar 2009 22:23:14 +0000
Accept-language: en-US
Acceptlanguage: en-US
Cc:
Delivery-date: Wed, 25 Mar 2009 15:24:47 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4467577e0903250736v4f88fe60jcb65ea290e52e933@xxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <4467577e0903250736v4f88fe60jcb65ea290e52e933@xxxxxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcmtVycI5hytxTRxRvasd2o8CnP1WgAP8oMg
Thread-topic: [Xen-users] Strange network issue; Guest/DomU outgoing traffic
> Subject: [Xen-users] Strange network issue; Guest/DomU outgoing traffic
> 
> Hello all,
> 
> I've recently observed some strange behaviour on several DomU (spread
> over several xen setups). While I've found a few reports on similar
> issues, the discussions were fairly old and discontinued without
> providing a cause or solution.
> 
> Every once in a while (sometimes with an interval of several months) I
> get a "host down" alert for a guest host. The host is actually very
> responsive and working fine in every aspect but for the network. Due
> to lack of time I reboot those guests and everything is fine again.
> (Read on, ofcourse I did some checks to identify the issue but
> production servers have to come back up.)
> 
> Today a development guest is having exactly the same problem and I
> started to investigate. Tcpdump on the guest shows incomming traffic,
> however, no outgoing traffic is to be observed:
> 
> Arp requests (who-has) are being received, but we never reply.

You are tracing this within the guest OS? Or are you tracing on the bridge 
within Dom0?


> After manually adding addresses to arp:
> 
> ICMP ping/TCP testpackets/UDP testpackets are being received, never
> replied to.
> 
> Trying to ping from the specific guest or sending anything through the
> network interface doesn't show up in tcpdump. (writing to the socket
> does not give an error)

Again, you are tracing this within the guest OS? What do the stats on the guest 
network interface say? Do you see packet counters increasing (both within the 
guest OS eth0, and on vifX.0 in Dom0?) when you are sending packet? 

 
> Things I tried but did nothing:
> 
> - Restarting the network (interface).
> - Changing the interface hw address.
> - Bringing the Dom0 virtual interface down and up again. (As suggested
> as temporary fix in an old discussion).
> 
> Nothing of interest can be found in any log (for either the guest or
> Dom0).
> 
> This particalur setup is Xen 3.1.? running CentOS5 as Dom0 and DomU
> using a network-bridge setup. Other guest(s) running simultaneous on
> the same hypervisor continue to work without any (networking) issue.

If you only have a single guest failing on Xen, then I would guess this is an 
issue with the guest OS and not with anything relating to Xen networking. What 
kernel does the guest OS run?

 
> Does anyone have an idea about the cause or a solution to this problem?

I have seen strange things with libpcap under certain guest OS versions when 
used with Xen's netfront driver... libpcap is used for tcpdump and some network 
testing tools (potentially the ones you are using). It might be worth trying to 
also trace packets on the backend devices in Dom0, and also have a look at 
packet counters on all interfaces (Dom0/DomU).

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users