[Xen-users] xen_net: Failed to connect all virtual interfaces: err=-100

[Mark Doll <doll@xxxxxxxxx>]

Hi all!

I'm using Xen to emulate huge networks (30 domains with 4 to 20
interfaces each), so I need lots of interfaces, which in turn need a lot
of (dynamic) interrupts (one per interface). But when I'm exceeding the
256 interrupt limit of the kernel, domain0 panics giving a me hint to
increase NR_IRQS. So I did what I was told and changed NR_IRQS from 256
to 512: Since in
/usr/src/linux/include/asm-xen/asm-i386/mach-xen/irq_vectors.h line 111,
NR_IRQS is defined as

#define NR_IRQS                 (NR_PIRQS + NR_DYNIRQS)

I've instead increased NR_DYNIRQS from 128 to 384 to change NR_IRQS from
256 to 512. The xen0 does not panic anymore, fine, but things still do
not work.

Now. when booting a xenU domain when all interrupts of xen0 below 256
are already in use, the network frontend complaints about beeing unable
to connect the virtual interfaces. These are the last lines during the
boot of xenU:

[...]
Xen virtual console successfully installed as tty
Event-channel device installed.
xen_blk: Initialising virtual block device driver
xen_net: Initialising virtual ethernet driver.
xen_net: Failed to connect all virtual interfaces: err=-100
NET: Registered protocol family 2
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 4096 bind 8192)
NET: Registered protocol family 8
NET: Registered protocol family 20

That's all. Aftes this, the xenU domain hangs. Taking a look at the
allocated interrupts of domain 0, it seems as if the block backend makes
use of the new interrupts while the network backend doesn't:

root@tb1:~ >cat /proc/interrupts |tail -33
235:       1776     Dynamic-irq  vif76.18
236:       2149     Dynamic-irq  vif76.19
237:      11213     Dynamic-irq  blkif-backend
238:          3     Dynamic-irq  vif77.0
239:         30     Dynamic-irq  vif77.1
240:       1788     Dynamic-irq  vif77.2
241:       2145     Dynamic-irq  vif77.3
242:      13147     Dynamic-irq  vif77.4
243:        708     Dynamic-irq  vif77.5
244:        303     Dynamic-irq  vif77.6
245:       1793     Dynamic-irq  vif77.7
246:       2128     Dynamic-irq  vif77.8
247:      11376     Dynamic-irq  vif77.9
248:          8     Dynamic-irq  vif77.10
249:         30     Dynamic-irq  vif77.11
250:       1813     Dynamic-irq  vif77.12
251:       2132     Dynamic-irq  vif77.13
252:       3916     Dynamic-irq  vif77.14
253:          8     Dynamic-irq  vif77.15
254:        165     Dynamic-irq  vif77.16
255:        755     Dynamic-irq  vif77.17
256:          0     Dynamic-irq  vif77.18
257:          0     Dynamic-irq  vif77.19
258:          0     Dynamic-irq  blkif-backend
259:          0     Dynamic-irq  blkif-backend
260:          0     Dynamic-irq  blkif-backend
261:          0     Dynamic-irq  blkif-backend
262:          0     Dynamic-irq  blkif-backend
263:          0     Dynamic-irq  blkif-backend
264:          0     Dynamic-irq  blkif-backend
265:          0     Dynamic-irq  blkif-backend
NMI:          0
ERR:          0

One can see that the last 8 domains can't allocate interrupts for their
network interfaces. They only allocate one interupts for the block
device backend of the root filesystem.

What have I done wrong? What do I have to do to get things working? I've
tried to get some hints from the (kernel) sources, but that's not that
easy, though...

Mark.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users