This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-users] Very large latency during network activity

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] Very large latency during network activity
From: Pepe Barbe <elventear@xxxxxxxxx>
Date: Fri, 10 Apr 2009 23:43:51 -0500
Delivery-date: Fri, 10 Apr 2009 21:44:43 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to :content-type:content-transfer-encoding:mime-version:subject:date :x-mailer; bh=SS36CGjpIgBOO1UNlsKAEr8+NEXgkCXCgNedXazjpv0=; b=NGfVCfUE+nb6cnijF3yWjG2l3ln1Yi240AguxcXjAj70q7VPtYFWObmGzxfRV+ApcY GV0/JqxNobpFXkAcaS+Sl6RyGsLPuMW8p/6s/0Bd4Cp2MBObJPlzL/SYzzldFR+kOCKD b8Lt4sjbahEizXaldmO81EU5Sv0jq1yOEqZrg=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:content-type:content-transfer-encoding :mime-version:subject:date:x-mailer; b=sIUFvVSr25pqCEFgQg3fsdGwXjFDRGTsI5JwirrTueIjUEi7HwSGjZleO14kvHj17D ZLTS+df6Ou7A0HHqCqYg2exufyurnUfkNW53R8MK/iIE/Z1UKbtKJEif5EGdshji3M6q zPO8W8oAOWYQ7amzEoZKHs3oXYq3/FABEOOKY=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx

I am running Xen 3.3.1 with a Centos 5.3 dom0 and with 3 Ubuntu 8.04 and 1 Centos 5.3 dom0. One of the Ubuntu dom0s has two interfaces, one real PCI assigned via passthrough and one virtual; this host works as the router for the Network it serves.

The problem that I have is that when network traffic picks up, and by this I mean rates close to 1Mbps, the network latency starts to creep up. From an average of a 100ms, it goes up to 3s, with spiks up to 10s. This is unbearable given that is a fair amount of interactive usage in the network.

What can I do to figure out what is the issue here, and if there is a way to mitigate this problem? I have already limited the rate of the vifs to 5Mbps and I have QoS on the router limiting the outgoing speeds considerably below the max and prioritizing small packets to improve interactive sessions.

I've checked the load of the entire system using xentop and the overall CPU usage does not go above 10%, nor the virtual machines are memory starved.

I don't know if this is related, but I get the following error in my kernel log:

irq 21: nobody cared (try booting with the "irqpoll" option)

Call Trace:
 <IRQ>  [<ffffffff802b1483>] __report_bad_irq+0x30/0x7d
 [<ffffffff802b16ba>] note_interrupt+0x1ea/0x22b
 [<ffffffff802b0bb2>] __do_IRQ+0xbd/0x103
 [<ffffffff8028e122>] _local_bh_enable+0x61/0xc5
 [<ffffffff8026cf04>] do_IRQ+0xe7/0xf5
 [<ffffffff803a6cca>] evtchn_do_upcall+0x13b/0x1fb
 [<ffffffff8025f8d6>] do_hypervisor_callback+0x1e/0x2c
 <EOI>  [<ffffffff8026df02>] monotonic_clock+0x35/0x7b
 [<ffffffff802063aa>] hypercall_page+0x3aa/0x1000
 [<ffffffff802063aa>] hypercall_page+0x3aa/0x1000
 [<ffffffff8026e4e5>] raw_safe_halt+0x84/0xa8
 [<ffffffff8026ba22>] xen_idle+0x38/0x4a
 [<ffffffff8024a803>] cpu_idle+0x97/0xba
 [<ffffffff80634b09>] start_kernel+0x21f/0x224
 [<ffffffff806341e5>] _sinittext+0x1e5/0x1eb

[<ffffffff803dab93>] (usb_hcd_irq+0x0/0x55)
Disabling IRQ #21

I've already passed the irq=poll parameter to the kernel, but I keep getting this error. Is it related?


Xen-users mailing list

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-users] Very large latency during network activity, Pepe Barbe <=