[Xen-users] High network latency on first packet

[Luca Corti <luca@xxxxxxxxxxxx>]

Hi all,

My setup is Debian testing dom0/domUs with a 2.6.32 pvops kernel from
Debian unstable. Hypervisor is 3.4.2 from Debian testing.
I use network- and vif- route with a default route in domU pointing to
the nic because multiple IPs with bridge would trigger port shutdown on
the switch (only one mac-address allowed per port). A subnet is routed
to the dom0, which then knows which addresses the domUs have.

The problem is that every first packet between a domU and a host not in
the arp cache has a very high delay. This is protocol independent
(happens for http and dns too) but I'll provide ping as an example:

Route in domU:
--------------

$ ip route
default dev eth0  scope link 


Ping from domU to remote host:
------------------------------

$ ping 1.2.3.4
PING 1.2.3.4 (1.2.3.4) 56(84) bytes of data.
64 bytes from 1.2.3.4: icmp_seq=1 ttl=54 time=304 ms
64 bytes from 1.2.3.4: icmp_seq=2 ttl=54 time=22.7 ms
64 bytes from 1.2.3.4: icmp_seq=3 ttl=54 time=22.7 ms
^C
--- 1.2.3.4 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 22.702/116.691/304.574/132.853 ms

Immediately replying remote host on second try.

$ ping 1.2.3.4
PING 1.2.3.4 (1.2.3.4) 56(84) bytes of data.
64 bytes from 1.2.3.4: icmp_seq=1 ttl=54 time=23.0 ms
64 bytes from 1.2.3.4: icmp_seq=2 ttl=54 time=22.8 ms
64 bytes from 1.2.3.4: icmp_seq=3 ttl=54 time=23.1 ms
^C
--- 1.2.3.4 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 22.895/23.010/23.103/0.151 ms



Ping from remote host to domU:
------------------------------

Initial ping to domU from remote host.

$ ping 5.6.7.8
PING 5.6.7.8 (5.6.7.8) 56(84) bytes of data.
64 bytes from 5.6.7.8: icmp_seq=1 ttl=51 time=458 ms
64 bytes from 5.6.7.8: icmp_seq=2 ttl=51 time=24.9 ms
64 bytes from 5.6.7.8: icmp_seq=3 ttl=51 time=25.3 ms
^C
--- 5.6.7.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 24.936/169.512/458.272/204.184 ms

Immediately replying remote host on second try.

$ ping 5.6.7.8
PING 5.6.7.8 (5.6.7.8) 56(84) bytes of data.
64 bytes from 5.6.7.8: icmp_seq=1 ttl=51 time=24.6 ms
64 bytes from 5.6.7.8: icmp_seq=2 ttl=51 time=25.3 ms
64 bytes from 5.6.7.8: icmp_seq=3 ttl=51 time=25.2 ms
^C
--- 5.6.7.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 24.612/25.068/25.300/0.371 ms


Initial time goes up as high as 800ms. My guess is that this has
something to do with arp. Since when a host is in the arp cache, the
delay disappears, which is why the second ping gets an immediate reply.

I tried specifing mac addresses in the domU cfg files and playing with
domU routing flags without apparent benefit.

Is this an issue with Xen? Is this related to my setup?

thanks

Luca


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users