On Fri, Dec 5, 2008 at 4:57 PM, Gary Pennington <Gary.Pennington@xxxxxxx> wrote:
> However, what I was mostly curious about was why specifying apic when the
> guest was created made so much different to performance.
Because of how the interrupt blocking works with APIC vs non-apic.
For APIC (on Intel boxes), you set the level of interrupts that you
want to receive (called the TPR, or Task Priority Register) by writing
to an MMIO register. Under Xen, this means a vmexit every time the
TPR is written. w2k3sp0 writes to the TPR every time the priority
changes, which happens a lot. Each write causes a
vmexit/update/vmenter cycle, which is pretty expensive.
w2k3sp2 has a technique called "lazy TPR", where it manages to avoid
the vast majority of TPR writes. This results in a much lower
overhead, probably on bare hardware as well, but especially in
HVM-virtualized environments.
If you take a xen trace of w2k3sp0, you will see a ton of MMIO writes
to ffe00080; that's the TPR register (if I recall correctly from the
top of my head). xenalyze (Coming Soon[tm]) would look at the trace
and tell you that you were spending a non-negligible percentage of
your time handling MMIO writes to the TPR register.
If you disable the APIC, the processor uses other methods of setting
interrupt levels (don't know these off the top of my head) which are
virtualized as a part of the HVM spec, and are therefore don't involve
the hyperivsor at all. As Keir said, newer Intel hardware has TPR
virtualization, so even with the APIC accessing the TPR doesn't
involve the hypervisor.
-George
>
> Gary
>
>> Hopefully, sometime in the next few weeks, I'll be able to release my
>> 'xenalyze' tool, which will help a lot with analyzing what's really
>> going on with these kinds of workloads.
>>
>> -George
>>
>> On Fri, Dec 5, 2008 at 2:45 PM, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
>> wrote:
>> > On 05/12/2008 14:03, "Gary Pennington" <Gary.Pennington@xxxxxxx> wrote:
>> >
>> >> My next question is: What is really happening when APIC is specified for
>> >> a windows guest and why does performance vary so much according to whether
>> >> it's specified or not?
>> >
>> > Older Windows kernels update the APIC TPR a lot, and unless you have a very
>> > modern Intel processor every one of those TPR updates causes a vmexit.
>> >
>> > Modern Windows (including possibly latest w2k3 service pack, but I'm not
>> > totally certain) includes lazy TPR, which gets rid of the vast majority of
>> > TPR updates, and hence will go much faster.
>> >
>> > -- Keir
>> >
>> >
>> >
>> > _______________________________________________
>> > Xen-devel mailing list
>> > Xen-devel@xxxxxxxxxxxxxxxxxxx
>> > http://lists.xensource.com/xen-devel
>> >
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
>
> --
> Gary Pennington
> Solaris Core OS
> Sun Microsystems
> Gary.Pennington@xxxxxxx
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home