[Xen-users] Re: [Xen-devel] xen 4.1.2* dhcp issue/bug when installing/booting HVM domU domains (CentOS 6, unbuntu 11.04 server). Debian/OpenSolaris work fine.

[Mark Schneider <ms@xxxxxxxxxxxxxxxxxxxxxx>]

Thanks a lot for your hints Pasi.

Due to your suggestion I open a new thread here.

I have tested with model=e1000 (HVMs). As HVM configs are identical (not iso images and LVM volumes) I guess it is a xen 4.1.2* issue .. or just involved domU OS-kernels (however as it is HVM it should not play any role)

Tested and correctly working OSes (getting an IP address and pinging of "external" IPs):
- debian squeeze
- debian wheezy
- OpenSolaris (live mode)

Tested and *nonworking* working OSes (no IP address):
- CentOS 6.0

# ---
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  1536     2     r-----   1063.7
centoshvm.born2b3.net                        5  8192     4     -b----     41.9
netbsdhvm.born2b3.net                        7  4096     2     -b----     17.6
opensolarishvm.born2b3.net                   3  4096     2     -b----    179.6
# ---


Pings from *centos-hvm* to dom0 interface*no ip-traffic on peth0*
# ---
root@xen411dom0:/ftp/HVM# tcpdump  'icmp[icmptype] = icmp-echo and icmp[icmptype] != icmp-echoreply'
tcpdump: WARNING: peth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on peth0, link-type EN10MB (Ethernet), capture size 65535 bytes
^C
0 packets captured
6 packets received by filter
0 packets dropped by kernel


root@xen411dom0:/ftp/HVM# tcpdump  -i eth0 'icmp[icmptype] = icmp-echo and icmp[icmptype] != icmp-echoreply'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
19:45:30.283679 IP 192.168.1.180 > 192.168.1.124: ICMP echo request, id 61445, seq 23, length 64
19:45:31.285560 IP 192.168.1.180 > 192.168.1.124: ICMP echo request, id 61445, seq 24, length 64
# ---

Pings from *centos-hvm* to default gateway *no ip-traffic on peth0*
# ---
root@xen411dom0:/ftp/HVM# tcpdump  'icmp[icmptype] = icmp-echo and icmp[icmptype] != icmp-echoreply'
tcpdump: WARNING: peth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on peth0, link-type EN10MB (Ethernet), capture size 65535 bytes

^C
0 packets captured
6 packets received by filter
0 packets dropped by kernel


root@xen411dom0:/ftp/HVM# tcpdump  -i eth0 'icmp[icmptype] = icmp-echo and icmp[icmptype] != icmp-echoreply'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
19:54:00.770442 IP 192.168.1.180 > 192.168.1.1: ICMP echo request, id 518, seq 23, length 64
19:54:01.770958 IP 192.168.1.180 > 192.168.1.1: ICMP echo request, id 518, seq 24, length 64
# ---

OpenSolaris HVM (live mode) works as expected:
# ---
Sun Microsystems Inc.   SunOS 5.11      snv_111b        November 2008
root@opensolaris:~#

root@opensolaris:~# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 2
        inet 0.0.0.0 netmask ff000000
        ether 0:16:3e:27:20:67
xnf0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 3
        inet 192.168.1.100 netmask ffffff00 broadcast 192.168.1.255
        ether 0:16:3e:27:20:67
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128
xnf0: flags=2004841<UP,RUNNING,MULTICAST,DHCP,IPv6> mtu 1500 index 3
        inet6 fe80::216:3eff:fe27:2067/10
        ether 0:16:3e:27:20:67

root@opensolaris:~# ping 192.168.1.1
192.168.1.1 is alive
# ---

How can I debug why I can see ICMP or dhcp packages on eth0 (dom0) but not peth0?

Thank you in advance for any hints.

regards, Mark


Am 14.08.2011 16:33, schrieb Pasi Kärkkäinen:

This is probably something for xen-users mailinglist instead..
It sounds like a configuration problem to me.


   On Sun, Aug 14, 2011 at 03:52:57PM +0200, Mark Schneider wrote:

As there are quite a lot of involved details I would use links to keep
better overview of the problem.

The problem is that CentOS (tested with 6.0) and ubuntu (tested with
11.04 server) *can't get* an IP address during the installation of HVMs.
I had the same problem with NetBSD 5.1 however didn't repeat this test
yesterday again. I have tested with "model=e1000" and "model=rtl8139"
(s. the HVM config file below)
http://www.it-infrastrukturen.com/fileadmin/linux/paste/xen-centos-issues/hvm-centos1Gbit.born2b3.net.cfg

Debian's squeeze and wheezy images *don't have this issue* and are
getting an IP address (dhpc) during the installation and booting.

The following text describes the test with CentOS 6.0 and involves only
it's logfiles or kernel output.
During the installation CentOS was not able to get an IP address (dhcp).
After booting from LVM-drive "lspci" listed the correct PCI device:
00:04.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
Controller (rev 03)

There was NO network configuration .. even for dhcp (in ifcfg-eth0).
After setting it manually "service network restart" hang a while after
like "Determining of information for eth0 .. " and gave "failed" back.
There were three dhcp-Discover requests on eth0 in dom0 but no responce
for requests from CentOS HVM.

tcpdump -i eth0 -vvv -s 1500 '((port 67 or port 68) and (udp[8:1] = 0x1))'

    

Ok, so the question is "why your dhcp server is not responding?"

Did you also tcpdump on "peth0" interface in dom0, to make sure the dhcp requests
actually go *out* from the physical interface?

Yes - I traced with tcpdump. dhcp requests or icmp don't go out.

Do you see the dhcp requests on the dhcp server? run tcpdump on the dhcp server aswell.

No.

I made the same test with "ubuntu 11.04 server" as HVM .. with the same
"failed" message. After I have installed this ubuntu iso image on bare
metal (another HP DL385g7) and didn't run in such troubles. The
installation has got an IP address (dhcp) and after boot there was also
an IP address for eth0.

How can I trace the dhcp install issue? Below are some details and traces:

HVM config file:
http://www.it-infrastrukturen.com/fileadmin/linux/paste/xen-centos-issues/hvm-centos1Gbit.born2b3.net.cfg

Details about the test server / xen installation (dom0):
http://www.it-infrastrukturen.com/fileadmin/linux/paste/xen-centos-issues/dom0_details.txt
http://www.it-infrastrukturen.com/fileadmin/linux/debian-live-xen/config-3.0.1

Trace details (for CentOS domU):
http://www.it-infrastrukturen.com/fileadmin/linux/paste/xen-centos-issues/grub.conf_CentOS_kernel-debug-output-on
http://www.it-infrastrukturen.com/fileadmin/linux/paste/xen-centos-issues/centos-kernel-output.txt
http://www.it-infrastrukturen.com/fileadmin/linux/paste/xen-centos-issues/config-2.6.32-71.el6.x86_64


As I don't like to give up I have set a static IP for eth0 in ifcfg-eth0
and restarted network service.
Now CentOS domU has got an IP address and "route" showed the default
gateway however it is only possible to ping dom0 (and not default
gateway). Probably I am missing here something else.

    

Are you sure you're using free/unused static MAC address for the VM ?

Yes. I use different MAC addresses for vif interfaces.

And again, run tcpdump in dom0 on "peth0" interface and make sure the
icmp packets go *out* from dom0 to physical network.

There are *no* packages (ICMP) on peth0 .. they are only on eth0

Also run tcpdump on the router/defaultgateway. Can you see the icmp packets there?

No (for CentOS HVM)

Some details:
http://www.it-infrastrukturen.com/fileadmin/linux/paste/xen-centos-issues/ip-issue-centos-hvm.txt

For tests I have used xen 4.1.2* debian live image (kernel 3.0.1 final
with some xen settings) booting dom0 from an USB stick (s. details
below).
http://www.it-infrastrukturen.com/fileadmin/linux/debian-live-xen/README.xen-live

-- 
ms@xxxxxxxxxxxxxxxxxxxxxx

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users