WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] PV networking issue in 2.6.38.6 and 2.6.39-rc7 pvops dom

To: Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>
Subject: Re: [Xen-users] PV networking issue in 2.6.38.6 and 2.6.39-rc7 pvops dom0 kernels
From: Mario <mario@xxxxxxxxxxxxxx>
Date: Mon, 16 May 2011 13:04:11 +0200
Cc: "todd.deshane@xxxxxxx" <todd.deshane@xxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx>, Pasi Kärkkäinen <pasik@xxxxxx>, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Delivery-date: Mon, 16 May 2011 04:06:08 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4DD0E85F.7070105@xxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <4DCFC8BB.9030201@xxxxxxxxxxxxxx> <20110515155322.GM32595@xxxxxxxxxxx> <4DCFF8FA.8020409@xxxxxxxxxxxxxx> <20110515161203.GN32595@xxxxxxxxxxx> <4DCFFE2B.4040302@xxxxxxxxxxxxxx> <20110515163444.GO32595@xxxxxxxxxxx> <4DD00235.1090207@xxxxxxxxxxxxxx> <20110515170345.GP32595@xxxxxxxxxxx> <20110515172030.GQ32595@xxxxxxxxxxx> <4DD00C66.7080100@xxxxxxxxxxxxxx> <20110516065141.GR32595@xxxxxxxxxxx> <1305532695.31488.168.camel@xxxxxxxxxxxxxxxxxxxxxx> <4DD0E6CE.4050004@xxxxxxxxxxxxxx> <1305536377.31488.213.camel@xxxxxxxxxxxxxxxxxxxxxx> <4DD0E85F.7070105@xxxxxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110429 Thunderbird/3.1.10
On 05/16/2011 11:03 AM, Mario wrote:
On 05/16/2011 10:59 AM, Ian Campbell wrote:
On Mon, 2011-05-16 at 09:56 +0100, Mario wrote:
On 05/16/2011 09:58 AM, Ian Campbell wrote:
On Mon, 2011-05-16 at 07:51 +0100, Pasi Kärkkäinen wrote:
Hello,

Ian: Any idea what could cause this error message (more info below):

"XENBUS: Timeout connecting to device: device/vif/0 (state 1)"

Is this from the domU or the dom0?

This is from the domU.


Do other devices work ok?


Yes, everything else seems to be working fine.


User says replacing 2.6.38/2.6.39 pvops dom0 kernel with xenlinux one
fixes the problem..

2.6.38 didn't have netback, only 2.6.39-rc1 did. Mario, please can you
supply the .config you used with the 2.6.39 kernel and the exact commit
id you are using. It would also be useful to see a complete log from
both dom0 and domU.




Pvops kernel configs are here:

http://dev.slackverse.org/tmp/config-2.6.39-rc7-xen
http://dev.slackverse.org/tmp/config-2.6.37.6-xenU

They are both based on default configs shipped with Slackware, with minimal xen-only options enabled.


Well, when i saw that 2.6.39-rc7 did not work, and i did not expect
anyone to care, i simply wiped all of the pvops stuff that i was
testing, and replaced it with a gentoo patched kernel wich works.
If it will not be in vain, i will compile it again and give you more
details.

Yes please, we care very much about pvops to working well for everyone.
(I'm not sure why you would think otherwise, this has been the focus of
our development efforts for several years now).

Ian.


Fair enough, it is compiling as we speak. Report to follow shortly.

m.



Do the vif entries appear in XenStore? Using:
xenstore-ls /local/domain/$(xl domid<name>)/devices/vif
you should see some entries and those entries should reference a
backend
path which should also contains entries, please post both.




##################
This is from dom0:
##################

# xenstore-ls /local/domain/$(xl domid slackware)/devices/vif
xenstore-ls: xs_directory (/local/domain/1/devices/vif): No such file or directory


However it works when i do this:

# xenstore-ls /local/domain/0/backend/vif/1/0
frontend = "/local/domain/1/device/vif/0"
frontend-id = "1"
online = "1"
state = "1"
script = "/etc/xen/scripts/vif-bridge"
mac = "00:16:3e:00:00:01"
bridge = "xenbr0"
handle = "0"


# xl block-list 1
Vdev  BE  handle state evt-ch ring-ref BE-path
51713 0   1      4     8      8        /local/domain/0/backend/qdisk/1/51713
268440065 0 1 4 9 9 /local/domain/0/backend/qdisk/1/268440065


# xl network-list 1
Idx BE Mac Addr. handle state evt-ch tx-/rx-ring-ref BE-path 0 0 00:16:3e:00:00:01 0 1 -1 -1/-1 /local/domain/0/backend/vif/1/0


# brctl show
bridge name     bridge id               STP enabled     interfaces
xenbr0          8000.00219bfbf76e       no              eth0


# ifconfig -a | grep -v '^ '
eth0      Link encap:Ethernet  HWaddr 00:21:9b:fb:f7:6e
eth1      Link encap:Ethernet  HWaddr 00:21:9b:fb:f7:6f
lo        Link encap:Local Loopback
xenbr0    Link encap:Ethernet  HWaddr 00:21:9b:fb:f7:6e


# cat /var/log/xen/qemu-dm-slackware.log
domid: 1
Warning: vlan 0 is not connected to host network
-videoram option does not work with cirrus vga device model. Videoram set to 4M. /tmp/SBo/xen-4.1.0/tools/ioemu-qemu-xen/hw/xen_blktap.c:628: Init blktap pipes
Could not open /var/run/tap/qemu-read-1
char device redirected to /dev/pts/1
xs_read(): target get error. /local/domain/1/target.
(qemu) (qemu)


- /var/log/xen/xend.log - empty
- /var/log/xen/xen-hotplug.log - also empty


# dmesg | grep -i xen
[    1.354427] xen/balloon: Initialising balloon driver.
[    1.368850] xen-balloon: Initialising balloon driver.
[    1.370157] Switching to clocksource xen
[    1.371868] xen: registering gsi 13 triggering 1 polarity 0
[    1.371872] xen_map_pirq_gsi: returning irq 13 for gsi 13
[    1.371876] xen: --> pirq=13 -> irq=13
[    1.372379] xen: registering gsi 8 triggering 1 polarity 0
[    1.372382] xen_map_pirq_gsi: returning irq 8 for gsi 8
[    1.372386] xen: --> pirq=8 -> irq=8
[    1.372912] xen: registering gsi 4 triggering 1 polarity 0
[    1.372915] xen_map_pirq_gsi: returning irq 4 for gsi 4
[    1.372919] xen: --> pirq=4 -> irq=4
[    1.386083] xen: registering gsi 16 triggering 0 polarity 1
[    1.386091] xen: --> pirq=16 -> irq=16
[    1.386119] xen: registering gsi 16 triggering 0 polarity 1
[    1.386122] xen_map_pirq_gsi: returning irq 16 for gsi 16
[    1.386126] xen: --> pirq=16 -> irq=16
[    1.386151] xen: registering gsi 16 triggering 0 polarity 1
[    1.386153] xen_map_pirq_gsi: returning irq 16 for gsi 16
[    1.386157] xen: --> pirq=16 -> irq=16
[    1.386182] xen: registering gsi 17 triggering 0 polarity 1
[    1.386186] xen: --> pirq=17 -> irq=17
[    5.175257] xen: registering gsi 23 triggering 0 polarity 1
[    5.175265] xen: --> pirq=23 -> irq=23
[   10.135114] xen: registering gsi 21 triggering 0 polarity 1
[   10.135125] xen: --> pirq=21 -> irq=21
[   10.159141] xen: registering gsi 16 triggering 0 polarity 1
[   10.159146] xen_map_pirq_gsi: returning irq 16 for gsi 16
[   10.159150] xen: --> pirq=16 -> irq=16
[   10.191968] usb usb1: Manufacturer: Linux 2.6.39-rc7-xen ehci_hcd
[   10.195303] xen: registering gsi 17 triggering 0 polarity 1
[   10.195308] xen_map_pirq_gsi: returning irq 17 for gsi 17
[   10.195312] xen: --> pirq=17 -> irq=17
[   10.486610] xen: registering gsi 21 triggering 0 polarity 1
[   10.486615] xen_map_pirq_gsi: returning irq 21 for gsi 21
[   10.486619] xen: --> pirq=21 -> irq=21
[   10.500379] usb usb2: Manufacturer: Linux 2.6.39-rc7-xen uhci_hcd
[   10.500601] xen: registering gsi 20 triggering 0 polarity 1
[   10.500608] xen: --> pirq=20 -> irq=20
[   10.540318] usb usb3: Manufacturer: Linux 2.6.39-rc7-xen uhci_hcd
[   10.540557] xen: registering gsi 21 triggering 0 polarity 1
[   10.540563] xen_map_pirq_gsi: returning irq 21 for gsi 21
[   10.540567] xen: --> pirq=21 -> irq=21
[   10.570445] usb usb4: Manufacturer: Linux 2.6.39-rc7-xen uhci_hcd
[   16.992548] ADDRCONF(NETDEV_UP): xenbr0: link is not ready
[   19.463299] xenbr0: port 1(eth0) entering forwarding state
[   19.463303] xenbr0: port 1(eth0) entering forwarding state
[   19.463363] ADDRCONF(NETDEV_CHANGE): xenbr0: link becomes ready
[   29.870099] xenbr0: no IPv6 routers present
[   34.470061] xenbr0: port 1(eth0) entering forwarding state
[   57.355646] XENBUS: Unable to read cpu state
[   57.356049] XENBUS: Unable to read cpu state

# xl dmesg
 __  __            _  _    _   ___
 \ \/ /___ _ __   | || |  / | / _ \
  \  // _ \ '_ \  | || |_ | || | | |
  /  \  __/ | | | |__   _|| || |_| |
 /_/\_\___|_| |_|    |_|(_)_(_)___/

(XEN) Xen version 4.1.0 (root@xxxxxxxxxxx) (gcc version 4.5.2 (GCC) ) Fri May 13 20:30:34 CEST 2011
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: mbootpack v0.6 (alpha)
(XEN) Command line: auto BOOT_IMAGE=Linux.XEN ro root=900 dom0_mem=512M
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: none; EDID transfer time: 2 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 00000000000a0000 (usable)
(XEN)  0000000000100000 - 000000007feb2000 (usable)
(XEN)  000000007feb2000 - 000000007fec8000 (reserved)
(XEN)  000000007fec8000 - 000000007fee7c00 (ACPI data)
(XEN)  000000007fee7c00 - 0000000080000000 (reserved)
(XEN)  00000000e0000000 - 00000000f0000000 (reserved)
(XEN)  00000000fe000000 - 0000000100000000 (reserved)
(XEN) System RAM: 2046MB (2095432kB)
(XEN) ACPI: RSDP 000F2980, 0024 (r2 DELL  )
(XEN) ACPI: XSDT 000F2A08, 0094 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: FACP 000F2B20, 00F4 (r3 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: DSDT 7FEC8000, 2545 (r1 DELL   PE_SC3          1 INTL 20050624)
(XEN) ACPI: FACS 7FEE7C00, 0040
(XEN) ACPI: APIC 000F2C14, 0072 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: SPCR 000F2C93, 0050 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: HPET 000F2CE3, 0038 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: MCFG 000F2D1B, 003C (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: WD__ 000F2D57, 0134 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: SLIC 000F2E8B, 0024 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: ERST 7FECB114, 0210 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: HEST 7FECB324, 027C (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: BERT 7FECAF94, 0030 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: EINJ 7FECAFC4, 0150 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: SSDT 7FECA545, 032C (r1 DELL   PE_SC3         11 INTL 20050624)
(XEN) ACPI: SSDT 7FECA9BB, 032C (r1 DELL   PE_SC3         11 INTL 20050624)
(XEN) ACPI: SSDT 7FECAE31, 0162 (r1 DELL   PE_SC3         10 INTL 20050624)
(XEN) Domain heap initialised
(XEN) Processor #0 6:15 APIC version 20
(XEN) Processor #1 6:15 APIC version 20
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ERST table is invalid
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2400.141 MHz processor.
(XEN) Initing memory sharing.
(XEN) I/O virtualisation disabled
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) Brought up 2 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1b72000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 000000007a000000->000000007c000000 (121577 pages to be allocated)
(XEN)  Init. ramdisk: 000000007f4e9000->000000007f9ff200
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81b72000
(XEN)  Init. ramdisk: ffffffff81b72000->ffffffff82088200
(XEN)  Phys-Mach map: ffffffff82089000->ffffffff82189000
(XEN)  Start info:    ffffffff82189000->ffffffff821894b4
(XEN)  Page tables:   ffffffff8218a000->ffffffff8219f000
(XEN)  Boot stack:    ffffffff8219f000->ffffffff821a0000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82400000
(XEN)  ENTRY ADDRESS: ffffffff818dd200
(XEN) Dom0 has maximum 2 VCPUs
(XEN) Scrubbing Free RAM: ...............done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 220kB init memory.


And domU config:

# cat slackware
name = "slackware"
memory = 256
vcpus = 1
vif = ["bridge=xenbr0,mac=00:16:3e:00:00:01"]
disk = ["file:/var/test/xvda1.img,xvda1,w","file:/var/test/xvds1.img,xvds1,w"]
bootloader = "/usr/bin/pygrub"
extra = "fastboot"
root = "/dev/xvda1"




#####################
Some stuff from domU:
#####################

# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)


# dmesg | grep -i xen
[ 0.000000] Linux version 2.6.37.6-xenU (root@orion) (gcc version 4.5.2 (GCC) ) #1 SMP Sun Apr 24 14:37:22 CEST 2011
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000010800000 (usable)
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.1.0 (preserve-AD)
[    0.000000] Xen: using vcpuop timer interface
[    0.000000] installing Xen timer for CPU 0
[    0.060643] PCI: setting up Xen PCI frontend stub
[    1.250219] xen_balloon: Initialising balloon driver.
[    1.261020] Switching to clocksource xen
[    1.308267] Initialising Xen virtual ethernet driver.
[ 6.410098] XENBUS: Waiting for devices to initialise: 295s...290s...285s...280s...275s...270s...265s...260s...255s...250s...245s...240s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s...120s...115s...110s...105s...100s...95s...90s...85s...80s...75s...70s...65s...60s...55s...50s...45s...40s...35s...30s...25s...20s...15s...10s...5s...0s... [ 301.410245] XENBUS: Timeout connecting to device: device/vif/0 (local state 1, remote state 1)


Hope it helps, i might not be able to power this box on any more, but if you need something else i will keep the drives and boot them up in another server on your request..


Best regards,
mario




Well, from what i remember, only vhd was being shown, but i will try and
get you exact info today.


Ian.


-- Pasi

On Sun, May 15, 2011 at 07:24:54PM +0200, Mario wrote:


On 05/15/2011 07:20 PM, Pasi Kärkkäinen wrote:
On Sun, May 15, 2011 at 08:03:45PM +0300, Pasi Kärkkäinen wrote:
On Sun, May 15, 2011 at 06:41:25PM +0200, Mario wrote:
On 05/15/2011 06:34 PM, Pasi Kärkkäinen wrote:
On Sun, May 15, 2011 at 06:24:11PM +0200, Mario wrote:


On 05/15/2011 06:12 PM, Pasi Kärkkäinen wrote:
On Sun, May 15, 2011 at 06:02:02PM +0200, Mario wrote:
On 05/15/2011 05:53 PM, Pasi Kärkkäinen wrote:
On Sun, May 15, 2011 at 02:36:11PM +0200, Mario wrote:
Hey everyone,

I have been trying to make Xen-4.1.0 work on kernels
mentioned in the
subject, without much sucess. HVM guests are not
affected, only PV.


Below you mention gentoo xenlinux patches..

So please be more specific about which kernel was vanilla
(pvops),
and which kernels had some external patches applied..

-- Pasi


Sorry, both kernels in subject were vanilla from
kernel.org, with xen
options selected for dom0 usage.
I only wanted to point out that networking works if i use
kernel patched
with rebased gentoo patches.
I am talking only about dom0 kernel here, domU kernel does
not seem to
be the source of the problem.


OK. What kernel is the PV domU running? What Xen version are
you using? 4.1 ?

-- Pasi


Well, i have tried following kernels in domU:

2.6.31.6 (i got this one from stacklet)
2.6.37.6-domU
2.6.33.4-domU

Are these pvops/vanilla or with some patches?

Those are pvops/vanilla, no patches.



2.6.38.6-xen (dom0 kernel)
2.6.34.7-xen (dom0 kernel)
2.6.39-rc7-xen (dom0 kernel)

None of them made any difference, they were all complaining
about vif.
Yeah, I am running Xen 4.1


Are you using xm/xend or xl/libxl toolstack?

I am using xl/libxl with Xen 4.1


Ok. Have you tried Jeremy's xen.git xen/stable-2.6.32.x as a
dom0 kernel?
It would be interesting to know if that works for you or not.


Btw. Did you realize upstream 2.6.38 / 2.6.39 does not yet have
xen-blkback driver included? also blktap2 driver is missing.

So you you can only use the phy: or file: backends, assuming Xen
4.1,
which has the (slow) fallback qemu-based userspace blkback
implementation.

-- Pasi




Yes i have, but seeing that HVM networking works, i wanted to
report the
PV one that doesn't.


m.





Error that i see in dmesg on the domU:
XENBUS: Timeout connecting to device: device/vif/0 (state 1)

ifconfig -a on domU shows MAC address: 00:00:00:00:00:00

dom0 doesn't have any vif/tap interfaces assigned to domU
when i do
brctl show or ifconfig -a

I don't see any errors worth mentioning in /var/log/xen,
except mybe
these two:

/tmp/SBo/xen-4.1.0/tools/ioemu-qemu-xen/hw/xen_blktap.c:628:
Init blktap
pipes
Could not open /var/run/tap/qemu-read-2

Problem goes away when i boot 2.6.34.7 (it is the last
kernel that i
have built with gentoo rebased patches).

If there is something i can do to help debug this please
do let me know.


Best regards,
mario

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users





_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>