[Xen-devel] Optimize credit scheduler __runq_tickle to reduce IPIs

["Wei, Gang" <gang.wei@xxxxxxxxx>]

I used to find for multiple idle vms case, there are a lot of break events come 
from IPIs which are used to raise SCHEDULE_SOFTIRQ to wake up idle cpus to do 
load balancing -- csched_vcpu_wake ->__runq_tickle->cpumask_raise_softirq. In 
__runq_tickle(), if there are at least two vcpus runable, it will try to tickle 
all idle cpus which have affinity with the waking up vcpu to let them pull this 
vcpu away.

I am thinking about an optimization, limiting the number of idle cpus tickled 
for vcpu migration purpose to ONLY ONE to get rid of a lot of IPI events which 
may impact the average cpu idle residency time.

There are two concerns about this optimization:
1. if the only one target cpu failed to pull this vcpu (for the reason such as 
it just has been scheduled for another vcpu), this vcpu may stay on the 
original cpu for a long period until suspend/wakeup again and keep system cpus 
unbalanced.
2. if first_cpu() was used as the way to choose the target among all possible 
idle cpus, will it cause overall unbalanced cpu utilization? i.e. cpu 0 > cpu 1 
> ... > cpu N

Do my concerns make sense? Or any comments, suggestions, ...

Jimmy
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel