[Xen-users] Xen 3.0.0 AMD64 - Can ping through bridge/vif - but only host (domain0) can use XenU guests services

[Kevin Jackson <uksysadmin@xxxxxxxxx>]

Dear all,
I had a working Xen 2.0.0 32-Bit setup on RHEL 4 with one server
(Opteron) hosting multiple Xen guests.  Networking works fully and
I'm able to use these Xen guests like any otehr server as expected.

I've now got a second server setup using Xen 3.0.0 running under 64-Bit RHEL.

This Domain0 Xen server has its networking setup using VLANs on eth0 - so eth0 doesn't have a network address.

eth0
eth0.4086 172.18.138.48/255.255.0.0
eth0.4092 172.19.138.48/255.255.0.0

I have 2 bridges set up
bridge name     bridge
id              
STP enabled     interfaces
xen-br-back            
8000.00093d10d5fc      
no             
eth0.4092
                                                       
vif1.1
xen-br-front           
8000.00093d10d5fc      
no             
eth0.4086
                                                       
vif1.0


The guests have eth0 bridged through vif1.0 and packets leave the
Domain0 server through eth0.4086 to other machines on the real network.
Same with eth1, but for the second interface.
They have a network address like:  eth0 172.18.135.10/255.255.0.0
and have a default route through the Domain0's eth0.4086 interface
[with forwarding enabled on this box].

I can ping to and from the Xen guests from any machine on the phyiscal
network, but - I can't, for example, SSH from the physical network to
this Xen guest and vice versa.
But I can go from the Domain0 server and ssh to the Xen box.  So I
think I've some network level issue with the actual Domain0 server.

What is interesting is that I can - say - ssh from 172.19.138.1
(physical box on same network as Xen guest) to 172.19.135.10 (Xen
Guest) -- and can see a netstat ESTABLISHED connection and eventually
it times out.


Can anybody shed any light on what is going on?
I had a problem getting this to work with VLANs - I edited
/etc/xen/scripts/network-bridge script ops_start() function to not
'exit 1' if Link $vdev is missing as this gets created fine (it seems!)
and make sure the script executes:

# old style without ${vdev}
transfer_addrs  ${netdev} ${bridge}
transfer_routes ${netdev} ${bridge}

as it was taking down my VLANs and not bringing them up again!

Domain0 (with one XenU Guest running):

Linux NLWAPP049 2.6.12.6-xen0 #3 Fri Dec 9 11:33:59 GMT 2005 x86_64 x86_64 x86_64 GNU/Linux

eth0      Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:812699 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11094 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:59039386 (56.3 MiB)  TX bytes:1438590 (1.3 MiB)
          Interrupt:25

eth0.4086 Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          inet addr:172.18.138.49  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:515633 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1192 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:24148579 (23.0 MiB)  TX bytes:115282 (112.5 KiB)

eth0.4092 Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          inet addr:172.19.138.49  Bcast:172.19.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:295755 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9925 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:16590729 (15.8 MiB)  TX bytes:1214086 (1.1 MiB)

eth1      Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FD
          inet addr:172.20.138.49  Bcast:172.20.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:110431 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:12261024 (11.6 MiB)  TX bytes:128 (128.0 b)
          Interrupt:26

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:217 errors:0 dropped:0 overruns:0 frame:0
          TX packets:217 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1150665 (1.0 MiB)  TX bytes:1150665 (1.0 MiB)

vif2.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7 errors:0 dropped:0 overruns:0 frame:0
          TX packets:705 errors:0 dropped:13 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:486 (486.0 b)  TX bytes:42945 (41.9 KiB)

vif2.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:101 errors:0 dropped:0 overruns:0 frame:0
          TX packets:454 errors:0 dropped:137 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:7854 (7.6 KiB)  TX bytes:33697 (32.9 KiB)

xen-br-ba Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          inet addr:172.19.138.49  Bcast:172.19.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:292321 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7858 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:15423109 (14.7 MiB)  TX bytes:1056275 (1.0 MiB)

xen-br-fr Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          inet addr:172.18.138.49  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:515367 errors:0 dropped:0 overruns:0 frame:0
          TX packets:688 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:24120245 (23.0 MiB)  TX bytes:44070 (43.0 KiB)



Bridge info

bridge name     bridge
id              
STP enabled     interfaces
xen-br-back            
8000.00093d10d5fc      
no             
eth0.4092
                                                       
vif2.1
xen-br-front           
8000.00093d10d5fc      
no             
eth0.4086
                                                       
vif2.0

net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 1
kernel.sysrq = 0
kernel.core_uses_pid = 1



Guest xenU (NLWXEN010)

Linux NLWXEN010 2.6.12.6-xenU #1 SMP Thu Dec 8 15:15:30 GMT 2005 x86_64 x86_64 x86_64 GNU/Linux

eth0      Link encap:Ethernet  HWaddr 00:16:3E:67:B0:35
          inet addr:172.18.135.10  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1252 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:76284 (74.4 KiB)  TX bytes:708 (708.0 b)

eth1      Link encap:Ethernet  HWaddr 00:16:3E:20:6E:75
          inet addr:172.19.135.10  Bcast:255.255.0.0  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:758 errors:0 dropped:0 overruns:0 frame:0
          TX packets:123 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:55771 (54.4 KiB)  TX bytes:9526 (9.3 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:560 (560.0 b)  TX bytes:560 (560.0 b)


/etc/xen/NLWXEN010
kernel = "/boot/vmlinuz-2.6-xenU"
memory = 512
name = "NLWXEN010"
nics = 1
ip = "172.18.135.10"
vif = [ 'bridge=xen-br-front','bridge=xen-br-back' ]
gateway = "172.18.138.49"
netmask = "255.255.0.0"
disk = ['file:/xen/NLWXEN010_rootfs01,sda1,w', 'file:/xen/NLWXEN010_swap01,sda2,w']
root = "/dev/sda1 ro"


Regards,

Kev
-- 
Kevin Jackson


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users