This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Paravirtualised drivers for fully virtualised domains, r

To: Steven Smith <sos22-xen@xxxxxxxxxxxxx>
Subject: Re: [Xen-devel] Paravirtualised drivers for fully virtualised domains, rev9
From: Steve Dobbelstein <steved@xxxxxxxxxx>
Date: Fri, 11 Aug 2006 12:04:13 -0500
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Fri, 11 Aug 2006 10:04:56 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <20060811101704.GA3476@xxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Steven Smith <sos22@xxxxxxxxxxxxxxxx> wrote on 08/11/2006 05:17:04 AM:

> > Here is what I have found so far in trying to chase down the cause of
> > slowdown.
> > The qemu-dm process is running 99.9% of the CPU on dom0.
> That seems very wrong.  When I try this, the device model is almost
> completely idle.  Could you see what strace says, please, or if there
> are any strange messages in the /var/log/qemu-dm. file?

Looks like I jumped the gun in relating the 99.9% CPU usage for qemu-dm and
the network.  I start up the HVM domain and without running any tests
qemu-dm is chewing up 99.9% of the CPU in dom0.  So it appears that the
100% CPU qemu usage is a problem by itself.  Looks like the same problem
Harry Butterworth is seeing.

> > It appears that a lot of time is spent running timers and getting the
> > current time.  Not being familiar with the code, I am now crawling
> > it to see how timers are handled and how the xen-vnif PV driver uses
> Timer handling isn't really changed by any of these patches.  Patch
> 02.ioemu_xen_evtchns.diff is in vaguely the same area, but I can't see
> how it could cause the problems you're seeing, assuming your
> hypervisor and libxc are up to date.
> What changeset of xen-unstable did you apply the patches to?


> > P.S.  This just in from a test running while I typed the above.  I
> > that qemu will start a "gui_timer" when VNC is not used.  I normally
> > without graphics (nographic=1 in the domain config file).  I changed
> > config file to use VNC. The qemu-dm CPU utilization in dom0 dropped to
> > below 10%.   The network performance improved from 0.19 Mb/s to 9.75
> > (still less than the 23.07 Mb/s for a fully virtualized domain).
> When I try this, I see about 1600Mb/s between dom0 and a
> paravirtualised domU, about 30Mb/s between dom0 and an ioemu domU, and
> about 1200Mb/s between dom0 and an HVM domU running these drivers, all
> collected using netpipe-tcp.  That is a regression, but much smaller
> than you're seeing.
> There are a couple of obvious things to check:
> 1) Do the statistics reported by ifconfig show any errors?

No errors.

> 2) How often is the event channel interrupt firing according to
>    /proc/interrupts?  I see about 50k-150k/second.

I'm seeing ~ 500/s when netpipe-tcp reports decent throughput at smaller
buffer sizes and then ~50/s when the throughput drops at larger buffer

> 3) Is there any packet loss when you ping a domain?  Start your test
>    and run a ping in parallel.

No packet loss.

> The other thing is that these drivers seem to be very sensitive to
> kernel debugging options in the domU.  If you've got anything enabled
> in the kernel hacking menu it might be worth trying again with that
> switched off.

Kernel debugging is on.  I also have Oprofile enabled.  I'll build a kernel
without those and see if it helps.

> > It appears there is some interaction between using the xen-vnif
> > driver and the qemu timer code.  I'm still exploring.
> I'd be happier if I could reproduce this problem here.  Are you
> running SMP?  PAE?  64 bit?  What kernel are you running in the domU?

UP kernels in both the domU and dom0 (although the scheduler likes to move
the 1 vcpu in dom0 around to different physical CPUs).  64-bit kernels on
Steve D.

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>