Problem Description:
In the configuration below, xen is booted in a bridge configuration with firewalls disabled and forwarding enabled,
which is what I want for layer 2 testing. The output below illustrates
what happens on a ping from the Virtual Router to the traffic
generator. The results are the same when the ping originates from the
traffic generator. You can see the ICMP request pass from vif1.0, across
the eth2 bridge and finally across the peth2 nic i/f. You can then see the
reply packet returned from the traffic generator, pass through peth2 then
arrive at eth2 at which point the bridge appears to send an arp request to
vif1.0 to resolve the 1.7.0.3 destination
address on Virtual Router. This occurs despite the fact the arp entry for
1.7.0.3 was manually added to the arp table and exists when the request is
made. Virtual Router sends its response to the arp request, which shows
up on the vif1.0 trace. The packet is never forwarded to 1.7.0.3 and the
‘destination
unreachable’ message is sent by linux as a reply from 1.3.0.1 to the
traffic generator at 1.3.0.2. This really seems to be a configuration issue at the bridge, but I have sort of run out
of ideas at the moment.
Topology:
Traffic
|
Linux
Dom0
| Virt Router
Gen
|
|
| Intel 82576 Bridge Virtual
I/F |
|
|
1.3.0.2 -----
1.3.0.1 -----eth2----- 1.7.0.1 ---------1.7.0.3
|
peth2
vif1.0 | Eth0/0
|
|
1.5.0.2 -------1.5.0.1
----- eth3-----1.9.0.1 ----------1.9.0.3
|
peth3
vif1.1 | Eth1/0
[root@localhost vr]# brctl
show
bridge
name bridge
id
STP enabled interfaces
eth2
8000.001b21242104
no
vif1.0
peth2
eth3
8000.001b21242105
no
vif1.1
peth3
[root@localhost vr]# arp -v
Address
HWtype
HWaddress Flags
Mask Iface
1.3.0.2
ether 00:10:94:00:00:02
C
eth2
192.168.102.65
ether 00:1C:F6:85:99:71
C
eth0
1.7.0.3
*
* MP
vif1.0
1.5.0.2
*
* MP
eth3
1.9.0.3
*
* MP
vif1.1
Ping
is sent from Virtual Router 1.7.0.3:
(vif1.0 trace)
[root@localhost vr]# tcpdump
-i 2 tcpdump: verbose output suppressed, use -v or -vv for full
protocol decode
listening on vif1.0,
link-type EN10MB (Ethernet), capture size 96 bytes
15:54:58.004454
15:55:08.004507
15:55:13.256470 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 0, length 80
15:55:13.261149 arp who-has
1.7.0.3 tell 1.7.0.1
15:55:13.264651 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:14.261145 arp who-has
1.7.0.3 tell 1.7.0.1
15:55:14.264040 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:15.002408 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 1, length 80
15:55:15.261237 arp who-has
1.7.0.3 tell 1.7.0.1
15:55:15.263594 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:17.001942 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 2, length 80
15:55:17.011211 arp who-has
1.7.0.3 tell 1.7.0.1
15:55:17.014329 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:18.005692
15:55:18.011224 arp who-has
1.7.0.3 tell 1.7.0.1
15:55:18.013886 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:19.001560 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 3, length 80
15:55:19.011189 arp who-has
1.7.0.3 tell 1.7.0.1
15:55:19.013990 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:21.000988 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 4, length 80
15:55:21.011187 arp who-has
1.7.0.3 tell 1.7.0.1
15:55:21.013402 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:22.011245 arp who-has
1.7.0.3 tell 1.7.0.1
15:55:22.013058 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:23.011214 arp who-has
1.7.0.3 tell 1.7.0.1
15:55:23.013193 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
(peth2 trace)
[root@localhost vr]# tcpdump
-i 4
tcpdump: WARNING: peth2: no
IPv4 address assigned
tcpdump: verbose output
suppressed, use -v or -vv for full protocol decode
listening on peth2,
link-type EN10MB (Ethernet), capture size 96 bytes
15:55:13.256497 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 0, length 80
15:55:13.256785 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 0, length 80
15:55:15.002427 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 1, length 80
15:55:15.002665 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 1, length 80
15:55:16.261339 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:16.261347 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:17.001960 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 2, length 80
15:55:17.002207 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 2, length 80
15:55:18.256067 arp who-has
1.3.0.1 tell 1.3.0.2
15:55:18.256082 arp reply
1.3.0.1 is-at 00:1b:21:24:21:04 (oui Unknown)
15:55:19.001577 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 3, length 80
15:55:19.001834 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 3, length 80
15:55:20.011308 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:20.011316 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:21.001005 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 4, length 80
15:55:21.001237 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 4, length 80
15:55:24.011181 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:25.011241 arp who-has
1.3.0.2 tell 1.3.0.1
15:55:25.011438 arp reply
1.3.0.2 is-at 00:10:94:00:00:02 (oui Unknown)
15:55:25.012261 arp reply
1.3.0.2 is-at 00:10:94:00:00:02 (oui Unknown)
(eth2 trace)
[root@localhost vr]# tcpdump
-i 5
tcpdump: verbose output
suppressed, use -v or -vv for full protocol decode
listening on eth2, link-type
EN10MB (Ethernet), capture size 96 bytes
15:55:13.256470 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 0, length 80
15:55:13.256785 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 0, length 80
15:55:13.264651 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:14.264040 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:15.002408 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 1, length 80
15:55:15.002665 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 1, length 80
15:55:15.263594 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:16.261331 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:16.261346 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:17.001942 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 2, length 80
15:55:17.002207 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 2, length 80
15:55:17.014329 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:18.005692
15:55:18.013886 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:18.256067 arp who-has
1.3.0.1 tell 1.3.0.2
15:55:18.256079 arp reply
1.3.0.1 is-at 00:1b:21:24:21:04 (oui Unknown)
15:55:19.001560 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 3, length 80
15:55:19.001834 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 3, length 80
15:55:19.013990 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:20.011302 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:20.011315 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:21.000988 IP 1.7.0.3
> 1.3.0.2: ICMP echo request, id 16, seq 4, length 80
15:55:21.001237 IP 1.3.0.2
> 1.7.0.3: ICMP echo reply, id 16, seq 4, length 80
15:55:21.013402 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:22.013058 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:23.013193 arp reply
1.7.0.3 is-at 00:16:3e:08:1c:9c (oui Unknown)
15:55:24.011175 IP 1.3.0.1
> 1.3.0.2: ICMP host 1.7.0.3 unreachable, length 108
15:55:25.011235 arp who-has
1.3.0.2 tell 1.3.0.1
15:55:25.011438 arp reply
1.3.0.2 is-at 00:10:94:00:00:02 (oui Unknown)
15:55:25.012261 arp reply
1.3.0.2 is-at 00:10:94:00:00:02 (oui Unknown
[root@localhost vr]#
ifconfig
eth0
Link encap:Ethernet HWaddr 00:15:17:46:EF:CA
inet addr:192.168.102.94 Bcast:192.168.102.95 Mask:255.255.255.224
inet6 addr: fe80::215:17ff:fe46:efca/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9075 errors:0 dropped:0 overruns:0 frame:0
TX packets:5180 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:1644721 (1.5 MiB) TX bytes:810984 (791.9 KiB)
Base address:0x2020 Memory:98820000-98840000
eth2
Link encap:Ethernet HWaddr 00:1B:21:24:21:04
inet addr:1.3.0.1 Bcast:1.3.0.255 Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fe24:2104/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:429 errors:0 dropped:0 overruns:0 frame:0
TX packets:158 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:30865 (30.1 KiB) TX bytes:21598 (21.0 KiB)
eth3
Link encap:Ethernet HWaddr 00:1B:21:24:21:05
inet addr:1.5.0.1 Bcast:1.5.0.255 Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fe24:2105/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:71 errors:0 dropped:0 overruns:0 frame:0
TX packets:57 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8500 (8.3 KiB) TX bytes:10139 (9.9 KiB)
lo
Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:12 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:964 (964.0 b) TX bytes:964 (964.0 b)
peth2
Link encap:Ethernet HWaddr 00:1B:21:24:21:04
inet6 addr: fe80::21b:21ff:fe24:2104/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:143 errors:0 dropped:0 overruns:0 frame:0
TX packets:299 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:12904 (12.6 KiB) TX bytes:43150 (42.1 KiB)
peth3
Link encap:Ethernet HWaddr 00:1B:21:24:21:05
inet6 addr: fe80::21b:21ff:fe24:2105/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:31 errors:0 dropped:0 overruns:0 frame:0
TX packets:145 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2420 (2.3 KiB) TX bytes:25561 (24.9 KiB)
vif1.0
Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet addr:1.7.0.1 Bcast:1.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1726 errors:0 dropped:0 overruns:0 frame:0
TX packets:183 errors:0 dropped:16 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:87769 (85.7 KiB) TX bytes:10749 (10.4 KiB)
vif1.1
Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet addr:1.9.0.1 Bcast:1.9.0.255 Mask:255.255.255.0
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1527 errors:0 dropped:0 overruns:0 frame:0
TX packets:29 errors:0 dropped:17 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:75672 (73.8 KiB) TX bytes:4214 (4.1 KiB)