[Xen-users] Unable to get Networking on HVM Domain

[Alberto Munoz <amunoz0221@xxxxxxxxxxx>]

Hello everyone,

I grabbed xen-3.3.1 sources from the Mercurial repository (xen-3.3-testing.hg, changeset 475:1cf755387728, from Oct. 23, 2008) and built Xen from these sources on a 32-core (8 quad core Barcelona CPUs) SunFire machine, running an Ubuntu 8.04 (Hardyy Heron) distribution. Everything seemed to be going OK until I booted an HVM (also Ubuntu 8.04) domain. That domain boots normally but it is unable to go over the network (and I cannot reach it over the network either). My configuration file is as follows:

import os, re
arch = os.uname()[4]
if re.search('64', arch):
       arch_libdir = 'lib64'
else:
       arch_libdir = 'lib'

def vmid_check(var, val):
   val = int(val)
   if val <= 0:
       raise ValueError
   return val

# Define the 'vmid' variable so that 'xm create' knows about it.
xm_vars.var('vmid', use="Virtual machine id. Integer greater than 0.", check=vmid_check)

# Check the defined variables have valid values..
xm_vars.check()

builder='hvm'
kernel = "/usr/lib/xen/boot/hvmloader"
memory = 256
shadow_memory = 8
name = "tdf_%d" % vmid
vif = ['type=ioemu, bridge=eth0']
disk = ['file:/mnt/sdb1/vm_diskimgs/hda%d.img,ioemu:hda,w'%vmid]
cdrom="/root/ubuntu-8.04-server-amd64.iso"
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
boot = 'c'
sdl = 0
extra = 'console=xvc0'
serial='pty'


The bridge on dom0 seems normal to me (/etc/xen/scripts/network-bridge status):

root@cadr19:~# /etc/xen/scripts/network-bridge status
============================================================
7: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
    link/ether 00:14:4f:eb:42:2c brd ff:ff:ff:ff:ff:ff
    inet 10.245.144.19/24 brd 10.245.4.255 scope global eth0
    inet6 fe80::214:4fff:feeb:422c/64 scope link
       valid_lft forever preferred_lft forever
7: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
    link/ether 00:14:4f:eb:42:2c brd ff:ff:ff:ff:ff:ff
    inet 10.245.144.19/24 brd 10.245.4.255 scope global eth0
    inet6 fe80::214:4fff:feeb:422c/64 scope link
       valid_lft forever preferred_lft forever

bridge name     bridge id               STP enabled     interfaces
eth0            8000.00144feb422c       no              peth0
                                                        tap2.0
                                                        vif2.0

10.245.144.0/24 dev eth0  proto kernel  scope link  src 10.245.144.19
default via 10.245.144.254 dev eth0  metric 100

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.245.144.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         10.245.144.254  0.0.0.0         UG    100    0        0 eth0
============================================================
ifconfig on dom0 show the following:

eth0      Link encap:Ethernet  HWaddr 00:14:4f:eb:42:2c
          inet addr:10.245.144.19  Bcast:10.245.4.255  Mask:255.255.255.0
          inet6 addr: fe80::214:4fff:feeb:422c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:315581 errors:0 dropped:0 overruns:0 frame:0
          TX packets:155437 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:283680407 (270.5 MB)  TX bytes:14947657 (14.2 MB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:34 errors:0 dropped:0 overruns:0 frame:0
          TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2592 (2.5 KB)  TX bytes:2592 (2.5 KB)

peth0     Link encap:Ethernet  HWaddr 00:14:4f:eb:42:2c
          inet6 addr: fe80::214:4fff:feeb:422c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:328879 errors:0 dropped:0 overruns:0 frame:0
          TX packets:155791 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:290216829 (276.7 MB)  TX bytes:15597861 (14.8 MB)
          Base address:0xbc00 Memory:fe2e0000-fe300000

tap2.0    Link encap:Ethernet  HWaddr 2e:e0:09:2f:97:41
          inet6 addr: fe80::2ce0:9ff:fe2f:9741/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:88 errors:0 dropped:0 overruns:0 frame:0
          TX packets:17536 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:4416 (4.3 KB)  TX bytes:1804090 (1.7 MB)

vif2.0    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:17614 overruns:0 carrier:0
                                             ^^^^^^^^^^^^^
              !!!!!!!!!!!!!!!!!!!!!!! Notice the number of dropped packets!!!!!!!!!!!

          collisions:0 txqueuelen:32
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

iptables -L on dom0 shows the following:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere            PHYSDEV match --physdev-in vif2.0

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

The log form xend (whcih also looks pretty normal to me) is as follows:

[2008-11-06 16:57:29 5915] DEBUG (XendDomainInfo:89) XendDomainInfo.create(['vm', ['name', 'tdf_1'], ['memory', 256], ['shadow_memory', 8], ['vcpus', 1], ['on_xend_start', 'ignore'], ['on_xend_stop', 'ignore'], ['image', ['hvm', ['kernel', '/usr/lib/xen/boot/hvmloader'], ['args', 'console=xvc0'], ['device_model', '/usr/lib64/xen/bin/qemu-dm'], ['pae', 1], ['vcpus', 1], ['boot', 'c'], ['fda', ''], ['fdb', ''], ['timer_mode', 0], ['localtime', 0], ['serial', 'pty'], ['stdvga', 0], ['isa', 0], ['nographic', 0], ['soundhw', ''], ['vncunused', 1], ['sdl', 0], ['display', 'localhost:10.0'], ['xauthority', '/root/.Xauthority'], ['rtc_timeoffset', '0'], ['monitor', 0], ['acpi', 1], ['apic', 1], ['usb', 0], ['usbdevice', ''], ['keymap', ''], ['pci', []], ['hpet', 0], ['guest_os_type', 'default'], ['hap', 1], ['cpuid', []], ['cpuid_check', []]]], ['device', ['vbd', ['uname', 'file:/mnt/sdb1/vm_diskimgs/hda1.img'], ['dev', 'ioemu:hda'], ['mode', 'w']]], ['device', ['vif', ['bridge', 'eth0'], ['type', 'ioemu']]]])
[2008-11-06 16:57:29 5915] DEBUG (XendDomainInfo:2028) XendDomainInfo.constructDomain
[2008-11-06 16:57:29 5915] DEBUG (balloon:132) Balloon: 291384 KiB free; need 2048; done.
[2008-11-06 16:57:30 5915] DEBUG (XendDomain:449) Adding Domain: 2
[2008-11-06 16:57:30 5915] DEBUG (XendDomainInfo:2134) XendDomainInfo.initDomain: 2 256
[2008-11-06 16:57:30 5915] DEBUG (image:712) cmdline        = console=xvc0
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: boot, val: c
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: fda, val: None
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: fdb, val: None
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: soundhw, val: None
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: localtime, val: 0
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: serial, val: pty
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: std-vga, val: 0
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: isa, val: 0
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: acpi, val: 1
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: usb, val: 0
[2008-11-06 16:57:30 5915] DEBUG (image:743) args: usbdevice, val: None
[2008-11-06 16:57:30 5915] DEBUG (XendDomainInfo:2215) _initDomain:shadow_memory=0x8, memory_static_max=0x10000000, memory_static_min=0x0.
[2008-11-06 16:57:30 5915] DEBUG (balloon:132) Balloon: 290092 KiB free; need 279552; done.
[2008-11-06 16:57:30 5915] INFO (image:166) buildDomain os=hvm dom=2 vcpus=1
[2008-11-06 16:57:30 5915] DEBUG (image:795) domid          = 2
[2008-11-06 16:57:30 5915] DEBUG (image:796) image          = /usr/lib/xen/boot/hvmloader
[2008-11-06 16:57:30 5915] DEBUG (image:797) store_evtchn   = 2
[2008-11-06 16:57:30 5915] DEBUG (image:798) memsize        = 256
[2008-11-06 16:57:30 5915] DEBUG (image:799) vcpus          = 1
[2008-11-06 16:57:30 5915] DEBUG (image:800) acpi           = 1
[2008-11-06 16:57:30 5915] DEBUG (image:801) apic           = 1
[2008-11-06 16:57:30 5915] INFO (XendDomainInfo:1900) createDevice: vbd : {'uuid': '28bc24dc-9476-f1a5-232a-f43c16109168', 'bootable': 1, 'driver': 'ioemu', 'dev': 'ioemu:hda', 'uname': 'file:/mnt/sdb1/vm_diskimgs/hda1.img', 'mode': 'w'}
[2008-11-06 16:57:30 5915] DEBUG (DevController:122) DevController: writing {'backend-id': '0', 'virtual-device': '768', 'device-type': 'disk', 'state': '1', 'backend': '/local/domain/0/backend/vbd/2/768'} to /local/domain/2/device/vbd/768.[2008-11-06 16:57:30 5915] DEBUG (DevController:124) DevController: writing {'domain': 'tdf_1', 'frontend': '/local/domain/2/device/vbd/768', 'uuid': '28bc24dc-9476-f1a5-232a-f43c16109168', 'dev': 'hda', 'state': '1', 'params': '/mnt/sdb1/vm_diskimgs/hda1.img', 'mode': 'w', 'online': '1', 'frontend-id': '2', 'type': 'file'} to /local/domain/0/backend/vbd/2/768.
[2008-11-06 16:57:30 5915] INFO (XendDomainInfo:1900) createDevice: vif : {'bridge': 'eth0', 'mac': '00:16:3e:40:31:b9', 'type': 'ioemu', 'uuid': '88ea2839-0cf0-46c8-0a67-9a1f6eebcbc4'}
[2008-11-06 16:57:30 5915] DEBUG (DevController:122) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vif/2/0'} to /local/domain/2/device/vif/0.
[2008-11-06 16:57:30 5915] DEBUG (DevController:124) DevController: writing {'bridge': 'eth0', 'domain': 'tdf_1', 'handle': '0', 'uuid': '88ea2839-0cf0-46c8-0a67-9a1f6eebcbc4', 'script': '/etc/xen/scripts/vif-bridge', 'mac': '00:16:3e:40:31:b9', 'frontend-id': '2', 'state': '1', 'online': '1', 'frontend': '/local/domain/2/device/vif/0', 'type': 'ioemu'} to /local/domain/0/backend/vif/2/0.
[2008-11-06 16:57:30 5915] INFO (image:375) spawning device models: /usr/lib64/xen/bin/qemu-dm ['/usr/lib64/xen/bin/qemu-dm', '-d', '2', '-domain-name', 'tdf_1', '-nographic', '-vcpus', '1', '-append', 'console=xvc0', '-boot', 'c', '-serial', 'pty', '-acpi', '-net', 'nic,vlan=1,macaddr=00:16:3e:40:31:b9,model=rtl8139', '-net', 'tap,vlan=1,ifname=tap2.0,bridge=eth0', '-M', 'xenfv']
[2008-11-06 16:57:30 5915] INFO (image:422) device model pid: 29365
[2008-11-06 16:57:30 5915] DEBUG (XendDomainInfo:2723) Storing VM details: {'on_xend_stop': 'ignore', 'shadow_memory': '8', 'uuid': '8879f608-fb38-4443-97d2-06992c6f9aec', 'on_reboot': 'restart', 'start_time': '1226019450.35', 'on_poweroff': 'destroy', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '1', 'vcpu_avail': '1', 'image': "(hvm (kernel ) (args 'console=xvc0') (hpet 0) (stdvga 0) (loader /usr/lib/xen/boot/hvmloader) (serial pty) (vncunused 1) (boot c) (rtc_timeoffset 0) (pci ()) (pae 1) (hap 1) (acpi 1) (localtime 0) (timer_mode 0) (nographic 0) (guest_os_type default) (apic 1) (sdl 0) (monitor 0) (device_model /usr/lib64/xen/bin/qemu-dm) (usb 0) (xauthority /root/.Xauthority) (isa 0) (display localhost:10.0) (notes (SUSPEND_CANCEL 1)))", 'name': 'tdf_1'}
[2008-11-06 16:57:30 5915] INFO (image:511) waiting for sentinel_fifo
[2008-11-06 16:57:30 5915] DEBUG (XendDomainInfo:1359) Storing domain details: {'console/port': '3', 'name': 'tdf_1', 'console/limit': '1048576', 'store/port': '2', 'vm': '/vm/8879f608-fb38-4443-97d2-06992c6f9aec', 'domid': '2', 'image/suspend-cancel': '1', 'cpu/0/availability': 'online', 'memory/target': '262144', 'control/platform-feature-multiprocessor-suspend': '1', 'store/ring-ref': '65533', 'console/type': 'ioemu'}
[2008-11-06 16:57:30 5915] DEBUG (DevController:122) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/console/2/0'} to /local/domain/2/device/console/0.
[2008-11-06 16:57:30 5915] DEBUG (DevController:124) DevController: writing {'domain': 'tdf_1', 'frontend': '/local/domain/2/device/console/0', 'uuid': 'ce131513-2252-8f38-a1cb-a206a685004a', 'frontend-id': '2', 'state': '1', 'location': '3', 'online': '1', 'protocol': 'vt100'} to /local/domain/0/backend/console/2/0.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices vif.
[2008-11-06 16:57:30 5915] DEBUG (XendDomainInfo:1443) XendDomainInfo.handleShutdownWatch
[2008-11-06 16:57:30 5915] DEBUG (DevController:171) Waiting for 0.
[2008-11-06 16:57:30 5915] DEBUG (DevController:656) hotplugStatusCallback /local/domain/0/backend/vif/2/0/hotplug-status.
[2008-11-06 16:57:30 5915] DEBUG (DevController:670) hotplugStatusCallback 1.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices vscsi.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices vbd.
[2008-11-06 16:57:30 5915] DEBUG (DevController:171) Waiting for 768.
[2008-11-06 16:57:30 5915] DEBUG (DevController:656) hotplugStatusCallback /local/domain/0/backend/vbd/2/768/hotplug-status.
[2008-11-06 16:57:30 5915] DEBUG (DevController:670) hotplugStatusCallback 1.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices irq.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices vkbd.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices vfb.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices console.
[2008-11-06 16:57:30 5915] DEBUG (DevController:171) Waiting for 0.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices pci.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices ioports.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices tap.
[2008-11-06 16:57:30 5915] DEBUG (DevController:166) Waiting for devices vtpm.
[2008-11-06 16:57:30 5915] INFO (XendDomain:1174) Domain tdf_1 (2) unpaused.


Does anyone know what may be gogin on? I have searched for the past couple of days. but I have not seen anyone else with a problem like this one.

Any help would be greatly appreciated.

Alberto
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users