WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] Xen, Windows 2003 and APIC

To: "Gary Pennington" <gary.pennington@xxxxxxx>
Subject: Re: [Xen-devel] Xen, Windows 2003 and APIC
From: "George Dunlap" <George.Dunlap@xxxxxxxxxxxxx>
Date: Fri, 5 Dec 2008 17:06:28 +0000
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Delivery-date: Fri, 05 Dec 2008 09:06:55 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=c0tbcxOJhgPmW2UjTM1O/WZxpFe/k1kQKzi6cqXuSBw=; b=u7vrM6+kDGtD12ONfacyPaKsPNgpRs3rlOUvFuKoIyzMzaxGEU+J5q23AmS8lLx7ho 1b9uHJw0+VQR3j/8g7+gwte3LcaG7elv+lTn6di47A0rKvfm3ISBLnW0vfvcaMJ/7red 21jIvj1DQP1xQm2mwAuTmH26n+z7lJw6NEIQs=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=RDBB0RMbDVnZSoynf6ZnwOlKvLKc9QQXGO1dlFKwMJzy603Ozf6UNLbwLxMCsg9VUc 72EuVBwfLherrsPNvpPtZZtNWxVz4JkTgFPOEQdxw49aKX3Uq0V3G9u4LORvVQHStzlG ZT51SIDtr5yxWmMDsaSXioO8LMWV6MjBdCs3U=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20081205165719.GI29148@xxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20081205140341.GD29148@xxxxxxxxxxxxxxxxx> <C55EEF13.9EA%keir.fraser@xxxxxxxxxxxxx> <de76405a0812050842u6c1fcdbeldf46ab5e77350df6@xxxxxxxxxxxxxx> <20081205165719.GI29148@xxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Fri, Dec 5, 2008 at 4:57 PM, Gary Pennington <Gary.Pennington@xxxxxxx> wrote:
> However, what I was mostly curious about was why specifying apic when the
> guest was created made so much different to performance.

Because of how the interrupt blocking works with APIC vs non-apic.
For APIC (on Intel boxes), you set the level of interrupts that you
want to receive (called the TPR, or Task Priority Register) by writing
to an MMIO register.  Under Xen, this means a vmexit every time the
TPR is written.  w2k3sp0 writes to the TPR every time the priority
changes, which happens a lot.  Each write causes a
vmexit/update/vmenter cycle, which is pretty expensive.

w2k3sp2 has a technique called "lazy TPR", where it manages to avoid
the vast majority of TPR writes.  This results in a much lower
overhead, probably on bare hardware as well, but especially in
HVM-virtualized environments.

If you take a xen trace of w2k3sp0, you will see a ton of MMIO writes
to ffe00080; that's the TPR register (if I recall correctly from the
top of my head).  xenalyze (Coming Soon[tm]) would look at the trace
and tell you that you were spending a non-negligible percentage of
your time handling MMIO writes to the TPR register.

If you disable the APIC, the processor uses other methods of setting
interrupt levels (don't know these off the top of my head) which are
virtualized as a part of the HVM spec, and are therefore don't involve
the hyperivsor at all.  As Keir said, newer Intel hardware has TPR
virtualization, so even with the APIC accessing the TPR doesn't
involve the hypervisor.

 -George

>
> Gary
>
>> Hopefully, sometime in the next few weeks, I'll be able to release my
>> 'xenalyze' tool, which will help a lot with analyzing what's really
>> going on with these kinds of workloads.
>>
>>  -George
>>
>> On Fri, Dec 5, 2008 at 2:45 PM, Keir Fraser <keir.fraser@xxxxxxxxxxxxx> 
>> wrote:
>> > On 05/12/2008 14:03, "Gary Pennington" <Gary.Pennington@xxxxxxx> wrote:
>> >
>> >> My next question is: What is really happening when APIC is specified for
>> >> a windows guest and why does performance vary so much according to whether
>> >> it's specified or not?
>> >
>> > Older Windows kernels update the APIC TPR a lot, and unless you have a very
>> > modern Intel processor every one of those TPR updates causes a vmexit.
>> >
>> > Modern Windows (including possibly latest w2k3 service pack, but I'm not
>> > totally certain) includes lazy TPR, which gets rid of the vast majority of
>> > TPR updates, and hence will go much faster.
>> >
>> >  -- Keir
>> >
>> >
>> >
>> > _______________________________________________
>> > Xen-devel mailing list
>> > Xen-devel@xxxxxxxxxxxxxxxxxxx
>> > http://lists.xensource.com/xen-devel
>> >
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
>
> --
> Gary Pennington
> Solaris Core OS
> Sun Microsystems
> Gary.Pennington@xxxxxxx
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>