[Xen-users] Trouble with DomU networking.

["Chris Heller" <heller@xxxxxxxxxxxx>]

I am building a LAN made up of all DomU hosts. Everything has been going
quite nicely until I tried to bring up a new host on the network. This
new host is able to get an IP address from the DHCP server on this
network, I can ping the host from other hosts, but this new host appears
to be un-routable across the NAT interface.

Let me see if I can explain things in some more detail.

Dom0 has a single physical device which is attached onto a local network
(10.10.10.0). The first thing I did was create a DomU (DomSvc) which is
bridged onto this physical device and given an IP address from the DHCP
server for 10.10.10.0.

Now my goal is to have all other DomU hosts use DomSvc as their gateway.

To do this I created a bridge in Dom0 like so:

brctl addbr svcBr0
brctl stp svcBr0 off
brctl setfd svcBr0 0
ifconfig svcBr0 up

(Note I have not given the bridge an IP address, I want it to act as a
pure switch).

I then added a second interface to DomSvc bridged to svcBr0 in Dom0.

Inside DomSvc, I added eth1, with a static IP on 192.168.240.1. This
will be the gateway IP address for the 192.168.240.0/27 network.

I also setup a DHCP server on DomSvc which gives out IP addresses for
the 192.168.240.0 network and listens only on eth1.

I order to allow hosts using DomSvc as a gateway access to the
10.10.10.0 (and beyond) I added the following NAT rules via iptables on
DomSvc:

iptables --flush            
iptables --table nat --flush
iptables --delete-chain
iptables --table nat --delete-chain
iptables --table nat --append POSTROUTING --out-interface eth0 -j
MASQUERADE
iptables --append FORWARD --in-interface eth1 -j ACCEPT

Then I turned on IP forwarding in the DomSvc kernel:

echo 1 > /proc/sys/net/ipv4/ip_forward

This was all set-up to configure on reboot.

Anyway, I then went and setup a new DomU (DomH1) which was a HVM guest
install of Debian. I used a minimal installer which downloaded most of
the packages from an apt repository. This worked perfectly! The host
DomH1 rebooted and was now able to see the internet via DomSvc (tcpdump
confirms this).

Figuring I had worked everything out I began to setup my next DomU
(DomH2). Things were going fine: I configured the network, and grabbed
an IP from the DHCP server on DomSvc. When I went to being downloading
from the network that's when things went wrong.

DomH2 was able to send an initial packet up to the apt repository (its
being hosted on the 10.10.10.0 network), and the repo did reply (as seen
with tcpdump). But then that's it, nothing more has transferred over the
network.

>From DomH1 I can still ping DomH2 (and from DomSvc) but the network has
simply stalled.

However from DomH1 I can still access the internet without any problems.

Argh!

Where should I begin to start debugging an issue such as this? I
currently think it will be resolved somewhere in the bridge device
set-up, but bridging seems so simple and straight forward that there
doesn't seem to be much I can do there.

Perhaps its an iptables misconfiguration, but if that is the case why
does the other host behind DomSvc still work just fine?

One other thing to note is that eventually DomSvc seems to lock up after
DomH2 stops sending packets. Terminating DomH2 seems to un-hang DomSvc.

Also all hosts have static MAC addresses so that udev doesn't keep
adding new Ethernet devices to the hosts.

Hopefully someone much more Xen aware than I will have come across these
problems before, and reveal just how trivial they are to resolve.

Regards,
Chris Heller

-- 
_/_/_/_/ Chris Heller                                       Network
Systems |
_/_/_/   Teragram Corporation                 e-mail:
<heller@xxxxxxxxxxxx> |
_/_/_/   10 Fawcett St. 2nd Flr.               web:
http://www.teragram.com |
_/_/     Cambridge, Ma 02138   phone: 617.576.6800 x237 ~ fax:
617.576.7227 v

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users