This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] Optimize credit scheduler __runq_tickle to reduce IPIs

To: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] Optimize credit scheduler __runq_tickle to reduce IPIs
From: "Wei, Gang" <gang.wei@xxxxxxxxx>
Date: Fri, 19 Mar 2010 16:22:12 +0800
Accept-language: zh-CN, en-US
Acceptlanguage: zh-CN, en-US
Cc: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>, "Tian, Kevin" <kevin.tian@xxxxxxxxx>, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>, "Yu, Ke" <ke.yu@xxxxxxxxx>
Delivery-date: Fri, 19 Mar 2010 01:25:27 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcrHPUalciZ52kqPSSS8mkLvqOuGTg==
Thread-topic: Optimize credit scheduler __runq_tickle to reduce IPIs
I used to find for multiple idle vms case, there are a lot of break events come 
from IPIs which are used to raise SCHEDULE_SOFTIRQ to wake up idle cpus to do 
load balancing -- csched_vcpu_wake ->__runq_tickle->cpumask_raise_softirq. In 
__runq_tickle(), if there are at least two vcpus runable, it will try to tickle 
all idle cpus which have affinity with the waking up vcpu to let them pull this 
vcpu away.

I am thinking about an optimization, limiting the number of idle cpus tickled 
for vcpu migration purpose to ONLY ONE to get rid of a lot of IPI events which 
may impact the average cpu idle residency time.

There are two concerns about this optimization:
1. if the only one target cpu failed to pull this vcpu (for the reason such as 
it just has been scheduled for another vcpu), this vcpu may stay on the 
original cpu for a long period until suspend/wakeup again and keep system cpus 
2. if first_cpu() was used as the way to choose the target among all possible 
idle cpus, will it cause overall unbalanced cpu utilization? i.e. cpu 0 > cpu 1 
> ... > cpu N

Do my concerns make sense? Or any comments, suggestions, ...

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>