WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] Re: [Xen-users] XEN - networking and performance

To: "D. Duckworth" <donduq@xxxxxxxxxxx>
Subject: Re: [Xen-devel] Re: [Xen-users] XEN - networking and performance
From: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Date: Mon, 10 Oct 2011 13:03:30 -0400
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Mon, 10 Oct 2011 10:07:10 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20111008190701.50DFBAE098@xxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20111006203821.CD70CE9342@xxxxxxxxxxxxxxxxxxxxx> <20111008190701.50DFBAE098@xxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.21 (2010-09-15)
On Sat, Oct 08, 2011 at 07:07:01PM +0000, D. Duckworth wrote:
> Salutations,
> 
> From the xen-users list:
> > I would like some advice from people how are/were using  Xen 3.4.2 -
> > it should be a rather stable release. Dom0 is CentOS 5.5 64bit with
> > Xen 3.4.2 installed with the default settings (minus cpu pinning to
> > Dom0 and memory setup for 2GB). There are 2 built-in nics (Broadcom)
> > and an add-on network card (Intel) with another 4 nics. Currently
> > only one NIC is used for all network access, and as far as
> > networking, the default setings are used - xend-config.sxp:
> > (network-script network-bridge) (vif-script vif-bridge)
> > 
> > The questions are:
> > 
> > How can I improve the network performance (now all the VM are sharing
> > one bridge):
> > 
> > a. creating multiple bridges and assigning a VM (DomU) per bridge
> > b. trying to hide the NICs from Dom0 using something like "pciback
> > hide" - (pointers/example of how one would do this in Centos 5.5
> > would be highly appreciated...)
> 
> Xen Networking has been a thorn in my eye and a similar question has
> been with me for a long time now. So prepare, for this response contains
> rage.
> 
> Xen networking has room for many different approaches, yet the best
> thing about its scripts is that they are not mandatory to use. You can
> fully adjust the scripts to your needs or even replace them with your
> own. You can find modifications on forums and blogs although most of
> them just seem to be copies of few suggestions made by few people.
> Logical, because quite frankly it's a pain to grope what the Xen
> scripts and udev rules really do, let alone grok most of what they
> do out of the box.
> 
> Right now I just care about creating my ideal networking solution, i.e.
> routing, bridging and firewall stuff for vms with different roles. I am
> running Xen 4.1.2-rc3-pre non-professionally on a quad core single cpu
> 1U server. with 4 hard drives in RAID10 configuration. The server has
> one usable Ethernet port with multiple globally routable IPs. I can't
> use the other ethernet port; the server has no IPMI and the ISP declines
> use of two ports by one system because the data center is a no smoking
> zone for both humans and routers and switches.
> 
> So the highest priority is to reach dom0 from the Internet and
> therefore my grub has fallback options, one of which is a boot to Linux
> with no Xen. In turn this means that dom0's networking boot scripts
> may not depend on Xen at all, and Xen may not change networking in any
> way unless specified. My dom0 is a minimal system that only controls vms
> and networking. I want dom0 to be small and simple so the obvious
> choice is Arch Linux. Dom0 should be separated from the domUs in that
> the domUs cannot reach dom0 and one domU (domN) should do all
> networking for the other domUs.
> 
> I tried to use xl with xen4 for a while but due to bugs and missing
> features I had to go back to xm and xend. This is where the fun
> begins. In the past I used xend with network-bridge and for some strange
> reason (voodoo probably) I blindly accepted that script in the past and
> blamed myself for not appreciating it. But let's be blunt and honest:
> the scripts, in particular the script that *modifies dom0 networking
> during xend startup* is the biggest piece of sh!# idea I have ever seen
> in Xen. It creates bridges, takes eth0 down, tortures dom0 with occult
> ip addr/route, brctl, sysctl and iptables awk/sed manipulations and then
> it has you looking at your screen yearning for the moment that ping
> timeouts become ping replies, telling you that your box is reachable
> again. This script is a malevolent demon from the sewers of Norman the
> Golgothan and the worst part is that network-bridge is also still the

<laughs>
> recommended default!

Can you point me to where it mentions that please?
> 
> On the more positive side there was a fantastic update in Xen 4 where
> network-bridge changed a bit so that "it will not bring a bridge up if
> one already exists". Whoever wrote it should get a corporate medal for
> that and then a long vacation to a deserted island with an MSX II and
> no floppies. How can this even be approved by Xen's senior project
> manager, or is that a vacant position?

We realized that the networking setup is quite complex and would be best
left in the hands of the admins. The problem is that..
> 
> It surely seems so. Xen's /etc/xen/scripts (another design fail, why
> not /usr?) and udev scripts are confusing ad-hoc bloatware routines and
> are not transparent at all. With the current xen4 I saw the premature
> advice to more or less 'prepare for migration from xm to xl'. Yet, xl
> supports less and is conflicting: there is no vifname, no 'xl
> new/delete', no more python, no relocation and suddenly there is a
> conflict between 'xm start domain' versus 'xl start /etc/xen/domain'.
> 
> So new features emerge, adding to the confusion of the end user, while
> old problems are not being fixed properly. I wonder why, especially
> because it does not seem that xm and xend are the broken parts that
> need to be replaced by an unstable interface.
> 
> What needs attention first and foremost are two things, first of which
> is real and wise effort into one simple, minimal script that just
> handles the minimum in a transparent way e.g. control the hypervisor,
> manage vms, manage the backend. Of course networking can be done on
> domain start too, but this has happen in an entirely different way from
> what it does and how it does it. This is so important because it gives
> more control to the user that runs Xen. It's also a good moment to
> build in proper and mature support for IPv6.
> 
> Secondly, the website and documentation should be cleaned up and
> revised where appropriate. The current situation is a mess that has
> a much too steep and incompatible learning curve right now - for
> example, a bridge should just not be named eth0 and a physical device
> should not be renamed at all. It's fundamentally wrong, stupid, mad as
> hell and a PR failure for Xen to do it this way out of the box. No
> matter how often and detailed it has been documented on the website. 

.. the documentation and setup is sometimes quite hard. BTW, we are
going to do on Oct 26th a Documentation Day to clean up some of this mess.
Would you be intereted in helping along - perhaps in the networking Wiki?
> 
> I propose something like the following for xen networking:
> 
> * Xen will not manipulate non-xen devices or a firewall under any
>   circumstance, it might only add or substract routes and/or rules from
>   the routing tables,

Uh, what is 'non-xen' devices? Like bridges?

> * Allow for networking configuration per domU. For example let
>   networking per device be nat, routed, bridged or custom, where
>   all name the interface and bring it up; nat only adds the ip to the
>   routing table; routed could be an array of routes and rules that need
>   to be added or subtracted from various routing tables and it might
>   support proxyarp; bridged turns off arp, sets the mac on the vif and
>   then adds the interface to a bridge that should already be created by
>   the user; and custom is a custom set of unmanaged commands after
>   creating and destroying a domain.

You lost me. <sigh> I am using a bridge configuration and just
do:

auto lo
iface lo inet loopback

auto switch
iface switch inet static
        address 192.168.101.16
        netmask 255.255.255.0
        gateway 192.168.101.1
        bridge_ports eth2

And just use that 'bridge=switch' in all my configuration. And that
seems to work just fine - wouldn't that be best way of providing
the first network setup to users? I would think the majority of folks
do something akin to this?

> 
> I am aware that this can already be done with Xen. However, that
> process is quite arbitrary and it does things no one asked for. So one
> has to read the scripts. For example with the iptables part of
> vif-bridge. It is not handled transparently, it is quite arbitrary and
> it automatically executes for all vms that are being started. This leads
> you to wonder what more it does without you knowing it...
> 
> So, with that off my chest and the second line of my network-bridge
> being the words "exit 0" Xen lets my dom0 configuration alone
> like it is supposed to do. While KVM is becoming a 'next cool
> thing' for many people I would still prefer a separate hypervisor so now
> the fat just has to be removed from Xen.
> 

I am all for removing fat. Do you have links to some of particularly
bad Wiki pages that should be heavily audited?

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>