WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] Unable to start more than 103 VMs

To: "Carb, Brian A" <Brian.Carb@xxxxxxxxxx>, <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] Unable to start more than 103 VMs
From: Keir Fraser <Keir.Fraser@xxxxxxxxxxxx>
Date: Wed, 04 Jul 2007 09:35:11 +0100
Delivery-date: Wed, 04 Jul 2007 01:30:05 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <B05D2E415E8CC94897BB44233D14EE6805DC9ECC@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Ace9uBgO7yju7uF/ToKDz9Fsie+b3AAXiQLD
Thread-topic: [Xen-devel] Unable to start more than 103 VMs
User-agent: Microsoft-Entourage/11.3.3.061214
Christian pointed out that this is probably running into the default dynamic irq limit in dom0, which is 256. You should try raising NR_DYNIRQS in include/asm/mach-xen/irq_vectors.h. If you are hitting this limit you should get a warning in dom0’s dmesg (or /var/log/messages) unless the loglevel is set too low.

 K.

On 3/7/07 22:21, "Carb, Brian A" <Brian.Carb@xxxxxxxxxx> wrote:

Unable to start more than 103 VMs. Any ideas?

Running Xen Unstable (changeset 15445) on a Unisys ES7000/one on SLES10 Release, 64gb memory, 16 cpu, dom0_mem=512M, xenheap_megabytes=64. VMs are SLES10 para-virtual guests, each on their own lun on san storage.

We can start 103 VMs successfully. When starting the 104th, the VM times out waiting for its disk to appear:
XENBUS: Timeout connecting to device: device/vbd/2048 (state 6)
XENBUS: Timeout connecting to device: device/vif/0 (state 6)
XENBUS: Device with no driver: device/console/0
BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
EDD information not available.
Freeing unused kernel memory: 172k freed
Starting udevd
Creating devices
Loading jbd
jbd: no version for "struct_module" found: kernel tainted.
Loading ext3
Loading reiserfs
Loading jbd
Loading ext3
Waiting for device /dev/sda2 to appear: ..............................not found -- exiting to /bin/sh
sh: no job control in this shell
$

Noteworthy:
  - domain-0 can see the lun of the failing VM correctly, and there are no errors if we run kpartx and mount the VMs root partition.
  - we can reproduce the problem on xen-3.1 as well as xen-unstable.
  - problem still occurs if we split the VMs across 2 bridges - the 104th still fails.
  - problem also occurs if we start the VMs in any order - once 103 are started, the next one fails.
  - if we comment out the "vif = " statement in the VM config files, we can start more than 103
brian carb
unisys corporation - malvern, pa


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
<Prev in Thread] Current Thread [Next in Thread>