This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] Issues using xenpm in 3.3.1 on Intel Nehalem cpu

To: xen-devel@xxxxxxxxxxxxxxxxxxx, ke.yu@xxxxxxxxx, kevin.tian@xxxxxxxxx
Subject: [Xen-devel] Issues using xenpm in 3.3.1 on Intel Nehalem cpu
From: Andrew J Younge <ajy4490@xxxxxxx>
Date: Mon, 6 Apr 2009 14:29:43 -0400
Cc: Lizhe Wang <lizhe.wang@xxxxxxxxx>, Gregor von Laszewski <laszewski@xxxxxxxxx>
Delivery-date: Mon, 06 Apr 2009 11:30:21 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Eq/Mwgm9EkqxTrxLlpGqJwDIwPLmezeasSl3eIJFGUM=; b=q9YBEDI/7K45zY/ro7Mez2+GmtFT+5T8YFj25bIVNf60l4v9f7QChgevPKK6h1WbxV yOaOanLNT7Og3YruePw8p/cJTBGK4T2IfmXRW4rWyzJlxgKuQCcW9HDbCjuDpd8jbVwu lMNkQSP/RJAGFrc1rBztbX8/hJPNgk7NWpgPQ=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; b=Z1H8P0NnVs1HSPyvlbfThTDYNwc2RCwLDZlEPEd7Cf1daBqTsxai6gLe9yrIMsqOZK 7hN4i4BWXBhelBzf4VZ7NlQA1cV7kMlUJms3oHq6XLd4ukU53M9HkOc0H7qQ1XVFgPdH Htu3Z4Y74/xsJ+2bNDGanCKKgFCBMTqDw7VNk=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx

I am having some issues with using the new Xen Power management features in Xen 3.3.1 and I'm need of some help. If this question should be directed elsewhere, please let me know. 

My goal is to be able to dynamically turn the CPU frequency up or down for each core in dom0, depending on the VM load.  I am hoping to do this either with an automatic governor (ondemand) or write my own custom scripts to control this dynamically.  To my knowledge, the new Core i7 Nehalem processors are able to dynamically control each core's frequency and voltage independent of another, which is why I chose the CPU.  In a normal non-xen environment (vanilla Ubuntu 8.10) I can see this working with various governors using the cpufreq-set and cpufreq-info tools that in turn rely on the acpi-cpufreq module.  When Xen announced it had the same support in 3.3.1, I was very excited to be able to recreate this effect with virtual machine loads in Xen.

My current setup is a Intel Core i7 2.66GHz (with hyperthreading) in a X58 motherboard with 6Gb DDR3 ram and a 300GB raptor SATA HD.  I have installed Ubuntu 8.10 server x86_64 as the default OS and then compiled and installed Xen 3.3.1 from source [1],[2].  Furthermore, I am using the xen kernel [3] for Dom0 as I had issues booting with the 2.6.18-8-xen kernel that came with the source.  I have enabled my dom0 to boot with cpufreq=xen cpuidle parameters in my grub menu.conf [4] to enable use of the xenpm tool.

While xenpm seems to be working on my system by reading all C states and P states (with hyperthreading enabled too), the controls do nothing.  For instance, if I try to set any governor (ex: "xenpm set-scaling-governor performance"), it seems to have no effect.  No matter what command I give, the default behavior is the same.  If the system is idle, then it will show all cores as running at the lowest P-state of 1600MHz, which is correct.  However when I create load that only would use 1 core (burnP5 &), the all cores report to run at 2668Mhz (highest setting).  This is not what I want, obviously.  Like I said before, the acpi-cpufreq module only scales up 1 core in such a situation.  If I "xm create" a DomU (ex: Debian Lenny) and put load on that machine (burnP5), the result is no different, all P-states report at 2668Mhz.  I don't know if its actually turning all the CPUs up to 2.6Ghz, or xenpm just reporting at 2.6Ghz but doing the right thing under the hood. 

After a bit of google searching, I found a previous mailing list thread [5] that describes a similar issue and provides a patch fix.  However it looks like this patch was issued before the release of 3.3.1 so I don't think it applies to me. I did take a look at the code, and the patch doesn't match up with the current code structure that I have in 3.3.1, so the patch wouldn't install even if I tried.  It also looks that there were still problems being reported after the patch that are similar to mine (all cores reporting the same freq when maybe they aren't at the same freq).  To be honest some of the details discussed in the later part of this article are beyond my current level of understanding, so if I am missing something please let me know.  I have attached the output of "xm dmesg" after booting up in Dom0 and running "xenpm" once as it may be of some use. 

Any help with this issue would be greatly appreciated.  I am really looking forward to have dynamic control of each core operating frequency in Xen as I believe some valuable research can come from this.  I am not sure if there is an issue with Xen 3.3.1 and the new Core i7 system I have, or just my level of understanding.  I would be happy to test any patches or experimental versions if necessary. My lab has also ordered a dual-socket quad-core Xeon 5520 system (also Nehalem based) that I plan to use in a similar way, so if there is testing that can be done on this machine too I would also be volunteer its use.

Thank you in advance for any help you can provide me.


[1] Xen 3.3.1 source code http://bits.xensource.com/oss-xen/release/3.3.1/xen-3.3.1.tar.gz
[2] Instructions on installing Xen 3.3.1 on Debian http://www.howtoforge.com/virtualization-with-xen-3.3.1-on-debian-etch
[3] Instructions to compile 2.6.27 dom0 kernel http://www.howtoforge.com/installing-xen-3.3-with-kernel-2.6.27-on-ubuntu-8.10-x86_64
[4] Xenpm manual http://wiki.xensource.com/xenwiki/xenpm
[5] C-state and P-state conrol in Xen http://markmail.org/message/2ivrwafrue2krdox#query:depends%20on%20!PROCESSOR_EXTERNAL_CONTROL+page:1+mid:digdksyggm7ofbdp+state:results

Andrew J Younge
Rochester Institute of Technology
102 Lomb Memorial Drive
Rochester, NY 14623

Attachment: dmesg_04.06.2009.out
Description: Binary data

Xen-devel mailing list
<Prev in Thread] Current Thread [Next in Thread>