WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Remus: network buffering problem

To: Shriram Rajagopalan <rshriram@xxxxxxxxx>
Subject: Re: [Xen-users] Remus: network buffering problem
From: Wenda Ni <wonda.ni@xxxxxxxxx>
Date: Mon, 21 Feb 2011 16:08:01 -0500
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Mon, 21 Feb 2011 13:09:36 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=1q1moR2qHDQ5LomnezDelA9VMgusVjd/IDapqVYTcS0=; b=jaz8FFWxURWD3mm2tFWEFUmZxmCDPFWKd+/o/D6GYID1Pyt1/ZXZIklpLIdeqMx+yb EY9I80+s5P08D/oquFyWksSj2pEu2nntgmddYMm3IWs3JbA22SR9JoV7ZW44ZiiBX/lJ yzLIFDdZSYirXSO0I4tk0NjMA6wuVIAFG9xxw=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=VLY4Gwrq5fADUJrRDmtrGhO3hBLAuJ01qoYcx2Lb8nR4khzkWL3FFKSZCG4ajIPnK0 LgtyB9TKMprzvwFYQ8H8dMF2h17+PbnbRA3frln/curXAsPj8s50jSyvrGN1FraOUXzz 7hJgxLNRcahUJ9HRmAy1a5oN/qp/ZvMEdfM4E=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <1639665C-89B7-411B-A3F5-E27DD8238432@xxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <AANLkTi=7WtfPsHvUvGy101eAd6gi+pSXjckjoMDruiW4@xxxxxxxxxxxxxx> <AANLkTimS1qs7kkngSgfXrYNt-QkKxE4q=-LqdA4+4hj+@xxxxxxxxxxxxxx> <AANLkTin-pUCWDMXvfWnh2v+bECzCGochveKKJcTVZTbO@xxxxxxxxxxxxxx> <AANLkTi=DphJnXFCi1EvUmW8HhwS_CBwhkWhOzvpv5dDs@xxxxxxxxxxxxxx> <AANLkTi=e1OV8srp8WJU5VBiqL9mh0Q-Mnj0VM89FS2VE@xxxxxxxxxxxxxx> <AANLkTi=FzaSve5P+xR-bx1Rbg8TYw3yppmNP3N_XUZW8@xxxxxxxxxxxxxx> <AANLkTik_6KatpsVdxLGAo1t3zrarCMMV_39N7r_QOReR@xxxxxxxxxxxxxx> <AANLkTinK8=LsFMCJuh=U5GxO3+mpi3OZFeeSfvEhk8et@xxxxxxxxxxxxxx> <AANLkTima1TVHzmrqc1eE-mYGR4Y9BgEFxKwBE1YtB0Tj@xxxxxxxxxxxxxx> <AANLkTim89DwyiB=qN8nnQs_q+wPNQsBSwZpB3sFjU4PK@xxxxxxxxxxxxxx> <1639665C-89B7-411B-A3F5-E27DD8238432@xxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Thanx, Shriram.

This is the most confusing part and the biggest worry in my mind. So far, I have not managed to figure out what the difference between PVOPS and HVM is, particularly to the extent that how I can identify or create a domain with clear view of whether it is PVOPS or HVM. I understand that in CentOS 5.5, the first question of "virt-install --prompt" is on whether your want your domU to be created fully virtualized or not. But when using xen-tools (on Ubuntu), it is not so obvious.


I have an previous post on xen-users discussing my confusion, and soliciting knowledge. Unfortunately, it is still left unanswered so far. The post is attached as follows (with modifications):

I have Xen 4.0.1 pvops on Ubuntu 10.10 (The original Ubuntu kernel is 2.6.35-25-generic) following the steps on https://help.ubuntu.com/community/Xen
I use xen-tools (debootstrap) to configure domU. In xen-tools.config, I specify
"
kernel = /boot/vmlinuz-2.6.18.8
initrd = /boot/initrd.img-2.6.18.8
"
It seems that the actual .cfg file generated for domU uses
"
bootloader = '/usr/bin/pygrub'
"
There are some explanations about pygrub on xenwiki, so I believe this should be fine. The .cfg created is as follows:

#
# Configuration file for the Xen instance Alice, created
# by xen-tools 4.2 on Mon Feb 21 00:30:22 2011.
#

#
#  Kernel + memory size
#


bootloader = '/usr/bin/pygrub'

vcpus       = '1'
memory      = '128'

#
#  Disk device(s).
#
root        = '/dev/sda2 ro'
disk        = [
                  'file:/mnt/vm-nfs-storage/domains/Alice/disk.img,sda2,w',
                  'file:/mnt/vm-nfs-storage/domains/Alice/swap.img,sda1,w',
              ]


#
#  Physical volumes
#


#
#  Hostname
#
name        = 'Alice'

#
#  Networking
#
vif         = [ 'ip=155.98.38.230,mac=00:16:3E:B1:BE:76' ]

#
#  Behaviour
#
>
on_reboot   = 'restart'
on_crash    = 'restart'

And when the domU starts, the "uname -a" gives kernel as 2.6.35-25-generic, which is exactly the same as the original Ubuntu kernel version.


So my question is 1) how to find whether domU is pv-ops or HVM?


Next, According to http://wiki.xen.org/xenwiki/XenParavirtOps?, I checked the ..../build-linux-2.6-pvops_x86_32/./config,
  1. If building 32 bit kernel make sure you have CONFIG_X86_PAE enabled (which is set by selecting CONFIG_HIGHMEM64G)
    • non-PAE mode doesn't work in 2.6.25, and has been dropped altogether from 2.6.26 and newer kernel versions.
  2. Enable these core options:
    1. CONFIG_PARAVIRT_GUEST
    2. CONFIG_XEN
  3. And Xen pv device support
    1. CONFIG_HVC_DRIVER and CONFIG_HVC_XEN
    2. CONFIG_XEN_BLKDEV_FRONTEND
    3. CONFIG_XEN_NETDEV_FRONTEND
These options are all enabled, so I believe the dom0 built does support pv-ops domU. Then 2) how do I build a pv-ops domU?

I have been struggling to find out the answer, but no progress so far in part due to my limited knowledge on OS. I am new to this area, so please forgive any of my simple questions, and hope the reply can be easy to follow.






On Mon, Feb 21, 2011 at 11:11 AM, Shriram Rajagopalan <rshriram@xxxxxxxxx> wrote:
Yes absolutely. As the Remus wiki in xensource says, it still needs 2.6.18 domU kernel (or novel suse 2.6.27) to operate in fast mode. 
 What type of guest are u running? PV or hvm?

Shriram

On 2011-02-20, at 11:44 PM, Wenda Ni <wonda.ni@xxxxxxxxx> wrote:

Thank you so much, Shriram. This error has been well handled. Now I meet with the following error:

WARNING: suspend event channel unavailable, falling back to slow xenstore signalling
Had 0 unexplained entries in p2m table
 1: sent 32768, skipped 0, delta 1477ms, dom0 38%, target 0%, sent 726Mb/s, dirtied 0Mb/s 0 pages
 2: sent 0, skipped 0, Start last iteration
PROF: suspending at 1298269217.559698
timeout polling fd
ERROR Internal error: Suspend request failed
ERROR Internal error: Domain appears not to have suspended
Save exit rc=1
tc filter del dev vif2.0 parent ffff: proto ip pref 10 u32
RTNETLINK answers: Invalid argument
We have an error talking to the kernel
Exception xen.remus.util.PipeException: PipeException('tc failed: 2, No such file or directory',) in <bound method BufferedNIC.__del__ of <xen.remus.device.BufferedNIC object at 0x228cd90>> ignored

It is related to the guest kernel?


On Sun, Feb 20, 2011 at 11:56 PM, Shriram Rajagopalan <rshriram@xxxxxxxxx> wrote:
On Sun, Feb 20, 2011 at 8:39 PM, Wenda Ni <wonda.ni@gmail.com> wrote:
> Hi Shriram,
>
> I have disabled NET_SCH_PLUG.
>
> I have modified the tools/remus/kmod/Makefile by removing "grep -q
> ^CONFIG_IMQ= $(KERNELDIR)/.config 2>/dev/null" as
>
> XEN_ROOT=../../..
> include $(XEN_ROOT)/tools/Rules.mk
>
> # Should make makefiles export linux build directory!
> # This is a fragile hack to tide us over
> ifeq ($(KERNELS),linux-2.6-xen)
> LINUX_VER=2.6.18-xen
> endif
> ifeq ($(KERNELS),linux-2.6-xen0)
> LINUX_VER=2.6.18-xen0
> endif
> ifeq ($(KERNELS),linux-2.6-pvops)
> LINUX_VER=2.6-pvops
> endif
>
> KERNELDIR ?= $(XEN_ROOT)/build-linux-$(LINUX_VER)_$(XEN_TARGET_ARCH)
>
> .PHONY: all
> all:
>         if test -d $(KERNELDIR); then $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd`
> modules; fi
>
> .PHONY: install
> install: all
>         if test -d $(KERNELDIR); then $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd`
> INSTALL_MOD_PATH=$(DESTDIR) modules_install; fi
>
> clean::
>         -rm -rf *.o *.ko *.mod.c *.mod.o Module.symvers .*.cmd .tmp_versions
>
My bad. Just edit the Kbuild file to
obj-m := sch_queue.o

This will prevent the ebt_imq modules from being built, since we are using
IFB (not IMQ). The rest is usual make and make install stuff.
> When I do a "sudo make" in the same directory, it gives the following error.
> Is there a a model called ebt_imq in the menuconfig? Where can i find it? I
> have searched a lot, but failed.
>
> /mnt/local/xen-4.0.1/tools/remus/kmod/ebt_imq.c: In function ‘ebt_imq_tg’:
> /mnt/local/xen-4.0.1/tools/remus/kmod/ebt_imq.c:62: error: ‘struct sk_buff’
> has no member named ‘imq_flags’
> /mnt/local/xen-4.0.1/tools/remus/kmod/ebt_imq.c:62: error: ‘IMQ_F_ENQUEUE’
> undeclared (first use in this function)
> /mnt/local/xen-4.0.1/tools/remus/kmod/ebt_imq.c:62: error: (Each undeclared
> identifier is reported only once
> /mnt/local/xen-4.0.1/tools/remus/kmod/ebt_imq.c:62: error: for each function
> it appears in.)
> make[4]: *** [/mnt/local/xen-4.0.1/tools/remus/kmod/ebt_imq.o] Error 1
> make[3]: *** [_module_/mnt/local/xen-4.0.1/tools/remus/kmod] Error 2
> make[2]: *** [sub-make] Error 2
> make[1]: *** [all] Error 2
> make[1]: Leaving directory
> `/mnt/local/xen-4.0.1/build-linux-2.6-pvops_x86_64'
> make: *** [all] Error 2
>
> You said I need to manually configure the KERNELDIR. Could you be more
> specific? I am new to this area. please make it easy to implement. Thank you
> very much.
>
>
> On Sun, Feb 20, 2011 at 1:10 AM, Shriram Rajagopalan <rshriram@xxxxxxxxx>
> wrote:
>>
>> On Sat, Feb 19, 2011 at 9:14 PM, Wenda Ni <wonda.ni@gmail.com> wrote:
>> > Hi Shriram,
>> >
>> > Thank you so much for your advice.
>> >
>> > I have enabled IFB (Intermediate functional block) module. I have
>> > enabled
>> > Ingress, redirecting, mirroring, NET_SCH_PLUG, and Netfilter. I have
>> > moduled
>> > or enabled the rest of the selections under QoS and/or fair queueing.
>> > Now it
>> > give the following error.
>> >
>> > I cannot find the sch_queue module under QoS and/or fair queueing. I
>> > find a
>> > few previous discussions about the sch_queue module, but have not found
>> > a
>> > good solution. Could you help a bit further? I am using XEN 4.0.1
>> > release
>> > pvops on Ubuntu 10.10 (2.6.32.27).
>> >
>> You only need either sch_plug or the sch_queue. not both.
>>  this issue has been fixed in the latest xen-unstable. But it still
>> lingers in 4.0.1. Here is what I suggest
>>
>> option 1:
>> Go to tools/remus/kmod/
>>  Hack the Makefile. I mean, manually set the KERNELDIR,
>>  remove the "grep CONFIG_IMQ..".
>>  if you do a make now, you should be able to compile the sch_queue module
>>  and then a make install.
>>  Basically, fix the makefile and compile/install the sch_queue module
>>
>> Option 2:
>>  (I suggest you avoid this). Look at device.py and qdisc.py from
>> xen-unstable
>> and fix your local repo's version (from xen 4.0.1).
>>
>> BTW, you are probably going to be raising the next issue - Disk
>> replication fails.
>>  This is a bad hack but probably saves u time, for the moment, in
>> 4.0.1. Edit the device.py near
>>  if disk.uname.startswith("tap:remus..") to "tap2:remus"...
>>  Unfortunately, 4.0.1 decided that users should explicitly specify
>> tap2:... to use blktap2 driver,
>> which meant that using tap:aio leads to loading blktap1
>> driver(deprecated/doesnt work in pvops).
>> So, one would have to use tap2:aio (loads blktap2 driver), and thus
>> similarly tap2:remus:..
>>        - but the remus scripts werent fixed soon enough to reflect
>> this, in 4.0.1. :(
>>
>> shriram
>> >
>> > modprobe -q sch_queue
>> >
>> > Traceback (most recent call last):
>> >   File "/usr/bin/remus", line 210, in <module>
>> >     run(cfg)
>> >   File "/usr/bin/remus", line 119, in run
>> >     bufs.append(BufferedNIC(vif))
>> >   File "/usr/local/lib/python2.6/dist-packages/xen/remus/device.py",
>> > line
>> > 275, in __init__
>> >     self.setup()
>> >   File "/usr/local/lib/python2.6/dist-packages/xen/remus/device.py",
>> > line
>> > 314, in setup
>> >     raise BufferedNICException('could not load sch_queue module')
>> > xen.remus.device.BufferedNICException: could not load sch_queue module
>> >
>> >
>> >
>> > On Sat, Feb 19, 2011 at 6:25 PM, Shriram Rajagopalan
>> > <rshriram@xxxxxxxxx>
>> > wrote:
>> >>
>> >> On 2011-02-18, at 7:04 PM, Wenda Ni <wonda.ni@gmail.com> wrote:
>> >>
>> >> Hi, Todd
>> >>
>> >> I realize what you mean. <330.gif> Hope Shriram can shed some light to
>> >> us!
>> >>
>> >> On Fri, Feb 18, 2011 at 9:44 PM, Todd Deshane <todd.deshane@xxxxxxx>
>> >> wrote:
>> >>>
>> >>> Adding our new remus maintainer to the CC
>> >>>
>> >>> On Fri, Feb 18, 2011 at 6:24 AM, Wenda Ni <wonda.ni@gmail.com> wrote:
>> >>> > Hello,
>> >>> >
>> >>> > Many thanks to Simon for his trouble shooting on live migration. I
>> >>> > have
>> >>> > now
>> >>> > started testing remus according to the remus link
>> >>> > http://nss.cs.ubc.ca/remus/doc.html
>> >>> >
>> >>> > 1) Remus in its simplest mode (--no-net option), without disk
>> >>> > replication or
>> >>> > network protection, is now working good.
>> >>> >
>> >>> > 2) When we add the network buffering protection (remove the --no-net
>> >>> > option), remus gives the error as follows:
>> >>> >
>> >>> > modprobe -q ifb
>> >>> >
>> >>> > modprobe -q imq
>> >>> >
>> >>
>> >> Have you installed the ifb module?
>> >> You have to enable it while configuring the dom0 kernel.
>> >>  Device Drivers-->network devices-->Intermediate Frame Buffer
>> >> You also need to enable a whole bunch of other things
>> >>  Networking ->QoS ->Ingress, actions, mirred, redirect, etc
>> >>   and Plug Queue (NET_SCH_PLUG).
>> >>
>> >> You are better off at the moment to just enable all or most of the
>> >> stuff
>> >> under Netfilter and Qos as modules. Might make your task easier.
>> >>
>> >> Btw thanks for pointing this out. These should probably be enabled by
>> >> the
>> >> kernel build scripts (buildconfigs/enable-xen-config ) or some such
>> >> place.
>> >>
>> >> Shriram
>> >>>
>> >>> > Traceback (most recent call last):
>> >>> >   File "/usr/bin/remus", line 210, in <module>
>> >>> >     run(cfg)
>> >>> >   File "/usr/bin/remus", line 119, in run
>> >>> >     bufs.append(BufferedNIC(vif))
>> >>> >   File "/usr/local/lib/python2.6/dist-packages/xen/remus/device.py",
>> >>> > line
>> >>> > 272, in __init__
>> >>> >     self.pool = Netbufpool(selectnetbuf())
>> >>> >   File "/usr/local/lib/python2.6/dist-packages/xen/remus/device.py",
>> >>> > line
>> >>> > 174, in selectnetbuf
>> >>> >     raise BufferedNICException('no net buffer available')
>> >>> > xen.remus.device.BufferedNICException: no net buffer available
>> >>> > Exception AttributeError: "'BufferedNIC' object has no attribute
>> >>> > 'bufdev'"
>> >>> > in <bound method BufferedNIC.__del__ of
>> >>> > <xen.remus.device.BufferedNIC
>> >>> > object
>> >>> > at 0xd9dd90>> ignored
>> >>> >
>> >>> > I googled, but find little useful info. Could anyone know my
>> >>> > problem?
>> >>> >
>> >>> > --
>> >>> > Cheers,
>> >>> >
>> >>> > Wenda Ni, Ph.D.
>> >>> > Dept. of Computer Science and Engineering
>> >>> > State University of New York (SUNY) at Buffalo
>> >>> >
>> >>> > _______________________________________________
>> >>> > Xen-users mailing list
>> >>> > Xen-users@xxxxxxxxxxxxxxxxxxx
>> >>> > http://lists.xensource.com/xen-users
>> >>> >
>> >>
>> >>
>> >>
>> >> --
>> >> Cheers,
>> >>
>> >> Wenda Ni, Ph.D.
>> >> Dept. of Computer Science and Engineering
>> >> State University of New York (SUNY) at Buffalo
>> >>
>> >
>> >
>> >
>> > --
>> > Cheers,
>> >
>> > Wenda Ni, Ph.D.
>> > Dept. of Computer Science and Engineering
>> > State University of New York (SUNY) at Buffalo
>> >
>> >
>
>
>
> --
> Cheers,
>
> Wenda Ni, Ph.D.
> Dept. of Computer Science and Engineering
> State University of New York (SUNY) at Buffalo
>
>
>
>
> --
> Cheers,
>
> Wenda Ni, Ph.D.
> Dept. of Computer Science and Engineering
> State University of New York (SUNY) at Buffalo
>
>



--
Cheers,
 
Wenda Ni, Ph.D.
Dept. of Computer Science and Engineering
State University of New York (SUNY) at Buffalo





--
Cheers,
 
Wenda Ni, Ph.D.
Dept. of Computer Science and Engineering
State University of New York (SUNY) at Buffalo


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
<Prev in Thread] Current Thread [Next in Thread>