| |
|
 |
|
 |
|
|
|
| |
|
|
xen-users
RE: [Xen-users] Random swap of interfaces at domU startup
>
> > Hello,
> >
> > I'm using xen 3.0.4 with a custom kernel (with drbd) and
> > quite often xen inverts eth0 and eth1. In order to have correct
> > interfaces I need to restart the domU
>
> I think that's complete nonsense: XEN does not do the naming of the
> network
> interfaces in DomU, but udev does. In SLES10 there's a file named
> "/etc/udev/rules.d/30-net_persistent_names.rules". Do you have that?
>
> >
> > The virtual interfaces vifX.0 and vifX.1 should be connected
> > to xenbr0 and xenbr1 respectively but sometimes they are connected
> > to xenbr1 and xenbr0.
>
> I don't understand that.
>
> [...]
>
> Regards,
> Ulrich
>
Similar problems here with Xen-3.0.4 on Ubuntu Edgy (with Xen binaries).
We have a Dom-0 with two network interfaces and added a Dom-U with VIFs onto
both networks. In the Dom-0, eth0 is the 192.168.0.X subnet with a DHCP
server on it whilst eth1 is the 192.168.1.X network with another DHCP server
on it. Xen's the VIF assignment on some DomUs doesn't follow the order of
the VIF declarations in the DomU configuration file. This was unexpected
and caused us problems with our MAC/ethX assignements. Our solution was to
assume that Xen's VIF ordering was random and to expect to have to use
"/etc/iftab" files in the DomUs to tie the MACs to specific network
interfaces.
For those that want to dig deeper, I have provided the output from the
ifconfig, brctl and config files so that you can see what is happening.
Having had problems with ethernet assignments in the past, I consider
setting up "/etc/iftab" to be good practice to ensure that networks don't
move about underneath you...
Best regards,
Roger
P.S. The Xen Dom-0 is called "borg" because it assimilated all our old
servers :-)
-----
Our Xen configuration is below:
root@borg:/etc/xen# xm info
host : borg
release : 2.6.16.33-xen
version : #1 SMP Mon Jan 8 14:39:10 GMT 2007
machine : i686
nr_cpus : 2
nr_nodes : 1
sockets_per_node : 1
cores_per_socket : 2
threads_per_core : 1
cpu_mhz : 2799
hw_caps :
bfebfbff:20100000:00000000:00000180:0000e49d:00000000:00000001
total_memory : 2550
free_memory : 983
xen_major : 3
xen_minor : 0
xen_extra : .4-1
xen_caps : xen-3.0-x86_32p
xen_pagesize : 4096
platform_params : virt_start=0xf5800000
xen_changeset : Mon Jan 08 14:08:36 2007 +0000 13131:90d67d032114
cc_compiler : gcc version 3.4.4 20050314 (prerelease) (Debian
3.4.3-13)
cc_compile_by : shand
cc_compile_domain : localdomain
cc_compile_date : Mon Jan 8 14:09:48 GMT 2007
xend_config_format : 3
root@borg:/etc/xen# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:D0:B7:18:73:47
inet addr:192.168.0.17 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::2d0:b7ff:fe18:7347/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5129 errors:0 dropped:0 overruns:0 frame:0
TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:447967 (437.4 KiB) TX bytes:748 (748.0 b)
root@borg:/etc/xen# ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:15:58:36:CB:48
inet addr:192.168.1.139 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::215:58ff:fe36:cb48/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4895060 errors:0 dropped:0 overruns:0 frame:0
TX packets:2534262 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2746219551 (2.5 GiB) TX bytes:195800585 (186.7 MiB)
root@borg:/etc/xen#
------
We created a Dom-U as below. We originally expected to have this bound
across both network interfaces and expected that DomU.eth0 -> Dom0.vifX.0 ->
xenbr0 and DomU.eth1 -> Dom0.vifX.1 -> xenbr1.
------
root@borg:/etc/xen# cat test
kernel = "/boot/vmlinuz-2.6.16.33-xen"
memory = 256
name = "testdomu"
vif = ['mac=aa:00:00:FF:00:0a,bridge=xenbr0',
'mac=aa:00:00:FF:01:0a,bridge=xenbr1']
disk = ['phy:/dev/raidarray/testroot,hda,w']
hostname = "testdomu"
root = "/dev/hda ro"
extra = "4"
root@borg:/etc/xen# xm create test
root@borg:/etc/xen# xm list
Name ID Mem VCPUs State
Time(s)
Domain-0 0 257 2 r----- 846.5
bigserver 4 256 1 -b---- 101.8
fossa 3 256 1 -b---- 1445.1
harpseal 1 256 1 -b---- 160.5
tarantula 2 256 1 -b---- 37.6
testdomu 5 256 1 -b---- 12.7
root@borg:/etc/xen# brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.feffffffffff no vif5.1
vif3.0
vif2.0
vif1.0
peth0
vif0.0
xenbr1 8000.feffffffffff no vif5.0
vif4.0
vif3.1
peth1
vif0.1
xenbr2 8000.000000000000 no
root@borg:/etc/xen# brctl showmacs xenbr0
port no mac addr is local? ageing timer
2 00:0d:87:b9:89:27 no 0.64
2 00:0d:88:53:28:a4 no 0.68
2 00:0f:b5:cb:45:ac no 130.45
3 aa:00:00:00:00:08 no 50.53
5 aa:00:00:00:00:0a no 0.64
6 aa:00:00:ff:00:0a no 130.45
1 fe:ff:ff:ff:ff:ff yes 0.00
root@borg:/etc/xen# brctl showmacs xenbr1
port no mac addr is local? ageing timer
2 00:09:5b:66:51:42 no 52.05
2 00:0e:a6:2f:ca:bb no 52.05
2 00:13:8f:00:5a:72 no 0.00
1 00:15:58:36:cb:48 no 0.00
4 aa:00:00:00:01:15 no 111.32
5 aa:00:00:ff:01:0a no 175.92
1 fe:ff:ff:ff:ff:ff yes 0.00
root@borg:/etc/xen#
-----
?!?!?!?!?
The MACs are correct. As per the VIF configuration, MAC aa:00:00:ff:00:0a
is attached to xenbr0 and MAC aa:00:00:ff:01:0a is attached to xenbr1. The
surprise is that the vif5 numbers are not what you would expect from the
order of the declaration in the "/etc/xen/test" line.
-----
root@borg:/etc/xen# xm console test
root@testdomu:~# ifconfig eth0
eth0 Link encap:Ethernet HWaddr AA:00:00:FF:01:0A
inet addr:192.168.1.133 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::a800:ff:feff:10a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:243 errors:0 dropped:0 overruns:0 frame:0
TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:25024 (24.4 KiB) TX bytes:1936 (1.8 KiB)
root@testdomu:~# ifconfig eth1
eth1 Link encap:Ethernet HWaddr AA:00:00:FF:00:0A
inet addr:192.168.0.5 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::a800:ff:feff:a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:90 errors:0 dropped:0 overruns:0 frame:0
TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6843 (6.6 KiB) TX bytes:1874 (1.8 KiB)
root@testdomu:~#
-----
This is now as we would have hoped. For whatever reason Xen chose to number
the VIFs in the order it did, this has upset our expectations for the DomU.
I followed the traditional Debian approach of using "/etc/iftab" to fix the
network interface assignment. DomU now has the file "/etc/iftab":
root@testdomu:~# cat /etc/iftab
eth0 mac AA:00:00:FF:00:0A
eth1 mac AA:00:00:FF:01:0A
root@testdomu:~#
After a reboot, the VIF assigment is unchanged by Xen in the Dom0:
root@borg:/etc/xen# xm list
Name ID Mem VCPUs State
Time(s)
Domain-0 0 257 2 r----- 851.5
bigserver 4 256 1 -b---- 101.9
fossa 3 256 1 -b---- 1445.4
harpseal 1 256 1 -b---- 160.8
tarantula 2 256 1 -b---- 37.9
testdomu 6 256 1 -b---- 12.7
root@borg:/etc/xen# brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.feffffffffff no vif6.1
vif3.0
vif2.0
vif1.0
peth0
vif0.0
xenbr1 8000.feffffffffff no vif6.0
vif4.0
vif3.1
peth1
vif0.1
xenbr2 8000.000000000000 no
root@borg:/etc/xen# brctl showmacs xenbr0
port no mac addr is local? ageing timer
2 00:0d:87:b9:89:27 no 1.80
2 00:0d:88:53:28:a4 no 0.04
2 00:0f:b5:cb:45:ac no 4.35
3 aa:00:00:00:00:08 no 46.73
5 aa:00:00:00:00:0a no 0.04
6 aa:00:00:ff:00:0a no 4.35
1 fe:ff:ff:ff:ff:ff yes 0.00
root@borg:/etc/xen# brctl showmacs xenbr1
port no mac addr is local? ageing timer
2 00:09:5b:66:51:42 no 42.74
2 00:0d:88:2e:8a:1e no 254.10
2 00:0e:a6:2f:ca:bb no 42.74
2 00:13:8f:00:5a:72 no 0.00
1 00:15:58:36:cb:48 no 0.00
2 aa:00:00:00:01:01 no 4.87
4 aa:00:00:00:01:15 no 162.07
5 aa:00:00:ff:01:0a no 4.87
1 fe:ff:ff:ff:ff:ff yes 0.00
root@borg:/etc/xen#
Looking within the DomU:
root@testdomu:~# ifconfig
eth0 Link encap:Ethernet HWaddr AA:00:00:FF:00:0A
inet addr:192.168.0.5 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::a800:ff:feff:a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:63 errors:0 dropped:0 overruns:0 frame:0
TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5607 (5.4 KiB) TX bytes:1856 (1.8 KiB)
eth1 Link encap:Ethernet HWaddr AA:00:00:FF:01:0A
inet addr:192.168.1.133 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::a800:ff:feff:10a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:34 errors:0 dropped:0 overruns:0 frame:0
TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3895 (3.8 KiB) TX bytes:2576 (2.5 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
root@testdomu:~#
----
Once you fix the MAC assignments in "/etc/iftab" in the DomU, all is fixed!
Hope this helps...
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
|
|
| |
|
Home