WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] [PATCH] scheduler rate controller

To: "Lv, Hui" <hui.lv@xxxxxxxxx>
Subject: RE: [Xen-devel] [PATCH] scheduler rate controller
From: Dario Faggioli <raistlin@xxxxxxxx>
Date: Fri, 28 Oct 2011 10:10:25 +0200
Cc: "Tian, Kevin" <kevin.tian@xxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, "keir@xxxxxxx" <keir@xxxxxxx>, George Dunlap <George.Dunlap@xxxxxxxxxxxxx>, "Dong, Eddie" <eddie.dong@xxxxxxxxx>, "Duan, Jiangang" <jiangang.duan@xxxxxxxxx>
Delivery-date: Fri, 28 Oct 2011 01:11:41 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAIAAADYYG7QAAAAA3NCSVQICAjb4U/gAAAVx0lEQVRYwzWY6a/d953XP9/tt51zfme/+37t692OtzhxtzTuJFPaTDVt0cDQeTIajToSKBJICBASQkI8Q1Q8ACQEhUGgCsF0OrSkaRLaxknqOl7i9dq++3727bd/Vx5k+CNe7+WFvvkvbvoywAZF1BMYOFNFbqWEW1gho42yB9jJq1QihEGVCh7BxrXpeNlxbJZmcmNvkArIhAIMJd+drlmA0aiXuBbN5x3Pxq6NS6X85l5/rxNnGtJYgAYhgVCa8UxqAwgQIABAGGMD1JWtS7/48Vy0W1k5GxXnmh41br1y6fxXv3LCEPXLu3vv3N3t2kXPAi0EIjTjnDGcKawzHceZl3N0mIExrmfnGS1iSij2pxgDxJhlsOFID8LUsVjecXvNPiJg2ZYySiODGUVcAwBCCAAMAIAhNc82C2xvuLG8uX422VvZfHrx9Mz5P7kRR51wby188YD+8metqQWBLa00F1JIlQk5imIhle+XBsHI9/PVUoFCBp7JOZbSjGiS92mx7IE0SICRkC/Yw2GSZECpIsYYwEmWGkDGIAAAAIQAY4QMEA6Nvc7G6aaYzKQlUs/x/O99VywsZcZd33w++l9/NbN6Z905MSqPGYO10QDYGCI1EIQQGMdzkzCo+G4pb5eY7blWKhKadiZ92yYsFWoYpwhRv0A4h/4oK5Zcl9nGAMLUgEYYMAZAmhBkM6KkoYpVIRjU47SRRUuVlXRlUZ08nxDyi50na+2jV1r9tTjK/DwYqgFRLHMuYdhCMou1bg64a0M15yKiCyCcLJlJspl+e6x3QNtuv1TbKIwHxpHIZDEYorGFh32eK3h+yc06XYqA2MyyLCmk0lobAFC04NsTyCGjOAbrp5svrs/N1sqeRegKscrF2aiz1ZfJweZPjLzojB0fdPcqFcv481592Y15KiKOLBDyd OPw1b37JZ3Qosdspk2Eh71iZ3MmJafLE/2l4w/pjBiiRMVJKjOtLItypb18LkxCmSilAIAorQCA0nLl0lGSqY5kyCP55bFpOwvW1vo79+7vb+z5yjSF1Wg9RaNNjKxRlObPnd/c+UurPFMZPzO5cPpcIt/c/HApaZp+Jzoa6FKhcu4kogzZFGmEGgczG6v1O+8uLp1eQ0VdOrXuLFqu0mAwwZkUCBOMmJAKDAIgAJq88f1/NucwtbZqaXP9pfNnFxY++ujmI4OZVzoYHfKtxnrY2PFAS46yuJTPNzvNYXvbDJsvTfr/5PSJl3/676cbhzvPV4ccbN9KTCoHHYa5xQoZRPpglw1SEim5tzN58KSc4b1LFzEmFiMEYwCSca01MsYAAoyRUYrGgjXOXfgbg8aE5hPV8vsPPr7XHJaPn4x6MY0GjsUWvnxy+dLSB7+43W6zN77y9Q/e/3ENvMvI/dO4WftX/xzidDvrO7Mnxqt+P0kshcCyRj2pe89zKSKxVjrWWKs4balUO/s26MwgqZU2JsmkUhjAAPpr0ACA3vjKydHuAa56B2u7Hz59qDFOkD149x1n9VkW81LevvCH54b5ztiY/+rLr7zxu1+TwdEr99deD0Ln1tMw4ixVhVpuemZGYT5W8JQCgyAMR1whY1IGqRI8E1ks0m2pnqq0k6ogUpgRroBLDQgjDBgTjAEUUgDUJI16Se7sbcWdfoxdzrRUwj7aV1I7165Mnjhx+oLz2+CTyclL3/2D74ApfOFr3w4Hf7778/dmSclIzoXKTy6n2QBrjRQyBPdGQ6RVlCQ8TXScMGk8iwWErsV8Z/58J9KZ0IgbBJgQiok2BghBCCFjMACQlesvrzjY/tWtOM0EoZwCU0Ym2ag+VZ5fLOf9xZXph61s8dibtdpYnApmFe3JmX6372UByfg+wKNOt+hYSgHBNjiesW3H9SyhqEKJlE2ZHiSjrVS2c8Xm639nhEoACAzGhDCGKSVKGQ AwBgECxQXNuz7uD1QvQFJEeeOm0iiFBbDu/uEzwW8O+P826hu/d6v0/PTx5Y8/ee/l66/lFqdqb9xQ//pRJ+KBwfmp8U1tmkgnXqaxbPd6/V4fJ1kJ6JQhderZ1OvE/aYSfeKBAYSQQVgbI5Q2UgMgBMgYAxgBAE1U6uQKuS9e7310qzwQ2pDUghR4JU3UZji07BfFamHtIF00O839zx7ceeXaFxGY/J1Hz1sHPEGHDlnTreLkyeNXrpQLDgjtBHF+GI6i4VEYfnpwpAejvNBnUdG2qkBdw4w2Bhv0eXeZzwMfIYQ+7zKgc7VynCux73176ktXV//ix6Onq67RxIhEGTDSHybQH4Xt6LU3317f2pycmGJUVxudOZ+o6Zn764dB0YHx6dCwp88eIYSIxpLLIIziLJbSOAKEW9QlZ7Posijv5awsSw1yDMIAGMAgZBBCGGNjDEJGAdCJnI9thwHLFr3pt9+e3Wl2W41fbj1NlD51sHvmxeP1YfqMya9fvfaf//E/3T/cfPP6K8vv3Wz/3w/SxiBPaDFUeGZ6V2Sd7UOtACmkpeRSaI0wMogZYzNk05KfG5Z8q5a9bBeaA90P0kRixGyuPqddY4wwQgKAFosVQjEANjF1lFIz4/1x50g0kyR+uVi79jtf/vmHL4LKhZ7WreYWMbH5b/8zu3vHGkUZlxjrSOkzy/Px/n5fYxAxNUCNIdhIojEhmDFM3By2sWdJNlajeKmUnqiwQNuRZJ0I398LDVBkABPABgEA7o/CMEhlplzHKlZL1WJVY4LB5Ci6+p2rvRv1uxm8evba/Qe/UaPuDUEXP/kNTdJewqtV/+zlK9yQnfWtdrsVp+koEWEmpQELUwcTqrGMVZpEiUqVU43PflUzLR2pbFqkenncKniZsRNjcaAcocRmGQBQh1kIISmllJLrbG/3oCujE7PHahYo8eDu46fUqawsLzx4+vEZN/eHQcJHyVBjz5DqsYWOwDm/nmP52TF vY79x1O4bKcp5b7JWIQoNklRhYhvKjBkfCrf3xJ7+ijVKo+TAsQuO4wKlDsMGpRRjlYm8U+wBp57jWrYlpFRKGa7G56b/4i//x0Hj4OKU9/03Tj3e3BuvFvP1QhJmf2Zcuz/UhGiNWK3a7US8Ovvaq7/zcXenmfWqrmdKpTRLGCO9KELGhJmIOHcZLY9XGo3BBTrUJxuzCtJ8refnuDY6EwRjbaRnA3OxjQ0AUK2U4EIZrYw2gIYyzVQyNV48e9wzRt6729p5sl+2ci8/+Gy80x5hMFIBsZOyf4TIrC4Uld05bChP+kpZNo4RywAERgyTPCJgO1bBmfALzb3B5msXrg0UqxaG5QJgy0UMrIxYjAhHZ4HtJVW3uAaCUkYNGAxGKYkBd/pNC2AizxYq/LPHn9hWfb4eFxob59cakhpXmIaCnGu9u7td9MdajTsvnbxAmY94YEFojOEGEcYQAYSY4VEx79J8QQ+yiPlBFq/aE1Gt7BmTwwo7jh5wjDlxGDWKMGKMAgBsW45t2TazGWHI0hs7G1oJIgavXjmTCKcXwJeuXsv+zwexyhLmxoRix3rEUAv0++3tDweHdzafvlZfxH6RS8cYAkAPW/0Xe612t2sAwNC0nz67t+rmcgEPW82+yEgqdZImo2Gnfbge9p6AaliuGsUj5lAAoJQxKYUCDYAEhkEYFPzCXA0q9eLme0m7K6auTgUf3ynYrCzprhJHBSdYnLX3jvSoNwrMzfbW7Y9aIaCwQMHlOS83XvWTRGDLihCkoyTujpiiBAOSYTKAOAjdXAFjLGTWae0/W3skwa3WJiYnpse9BQCgwyhCCAhBwOgoDErVCd1tLc2ge4+fBKHDjJk/fsrR76BhHBoIy6VoeSlFEmxKHNpN0rGJqbDbV9KwgWj3IlWQhGGlTXuUacvKMxtJhA3oLGZJL+Nht1M/eeZaq90wGDtu7tj8cYUsIU3J8yzKAAQVimdSICmlUQ93Xxw1W8cnxhYW wkKhY/C8X+ufq891Wy3P4Nsg26eWolx9dLC9P+wpZBKbNnv9AiGWlipNXEP1gGNKjJIFg0lmLJOWmS2UylMU91qWXw663X4nUAYjkNRxX9zd0oj4lclykTNCAIDatqORMVpJhNY21raP9lS0+bdeWd7ePXr8wlqaXiK//qDX3P20Nr41tZASRtSwhSRCUHVsZkO3F9qkgFKJDC4QprRSmcgTSjHFBjEwjtR5hEm7xxzmunkZhL1Od2F5th+2DQJDqF2ezo8vAcbUpABAkyTjUhjJRzrRQe9CsXj9st0ZbDVeDK8M7b+Nk+7qnaevfvVAC8G5ATTx2mtLRt7+d/+WpKHrWEnRO2gPxqUtMU4UV0bbzMKISCmBIKAUAGyAzvZWrmjZ1SJS/OHde/MzE2OVeqHgX3n5qnLHO/3UIO55GAAoF0pppbTuBoOEGkqsGqPXotlLvUp1pto8ffGdXuPFbz+lsSL5YovoW+++c2q8rmyLaocinCN6VLaToZFcYoM0xpHiHFAOU8aIBpAAgFEmZNLpmFpJsk5hciYeDmsTi/fuf7bX2auWx0rjc47v7j2+BXCRUoK4QQDQaB0Zg7SRhVSxx0NfzHWPrzxfXlj76EObxxqhv1p7fOLiJd8Ujp890z46HG5tGNAIKdtloRCJUjnMMAKsMEK4RbhCAhuMElkkdozAjIb5YZToZiDM5XNntQ2u5VKtGzsbe/tbpXye1SbBvkgJAMVYEzIajTyEj83YQ7/104Xhb37WvLGwEm3shUnAMhUiM1AZz9KjVmtydrpFCAeFMaYWtQBpn0mtBpkCpZAFiGLm2KAhHaY5REEjhJUDhAcxx05hLBcnaa1UKjALE+aVqpHIJienbBAAgEt+frxURBZRBkOuNF2lC2P4fif7JXa2uD7Y3+VxQlO5GwfMyecK+Xq1WB8b6yaBJEhqhRBGGGML56s+rnhW3XeqBavoEERFkFmAkdIIQR4wjlMRxx5zUgG3769qo U7MzLS7w04QGcs2zPbLPgDg3qAXh8PV7fXEoPm5xWIt543nRe585djS1vN7+MXDwfZGe9Dc63XSRBwcNOIoAgQjkQqGhdGAkNEaaUW0ylnUw9g1BIdKdsK8op4hNiUSa8VFvVI5ff6sSFOG2YOnG0kii26+6BeJxbjRSZpg1wMAbDMgFjxZvb+/u/X81ntJZysYos7hcNbLjy1MPtp6gkbdqD+oF6vFSs3NeTNT04NePwQtsNZAldYGIaUUaIkkRxrCME0GSQk7HmCiDQWEAQMCGcYXjh2zjQShohiGcXb1pQvH5ueLOafgOkaK2bEiAFDFZTcc2CpayMPSWO3kUn5zbXtwNNjfffTW3/9Hd29+mISBkWRhYf7333xzZqKUBaNOp9/ziqUgwSBSJGxqy0xrYUAZLkUUpXVs5zHJQEstkAFsMGNOOgiS++s05AiACp7G8Vi1xMPk5LFjbrGUQzA3UYOHQA1mz7e3D1qDAtNTp+cZNd2B+NaXvtjvqGrRs4pETXuHR73NezcDHTsMX7n2ansYzc2drnO7K5+vzEwqrtf3okAbkxgeZyCMTy0bUY1U9v/FmMUoaHPc5LOZY7/urk9Yo3H2eu+ou1ir/urxqtHZialp98wUAGAA9XhrPaH+SJPKhB+lfL2de//uemblk0hcml34g7feOHN+5eyZk6eXpoP2YaXsdzdXv3ugb9SXSi75va9d+4f/4I+/89aNotb9ILYFnrYLHsYUAUMYGQAD1CCCSd0tfPrg/hwH0ttctgN7dDjGu0sOdqmZKlvnj88SpAGAPtnYyESKVFTP8Zw16rUOW116FOPr5fqz7b2fP36hP+7zWAkplxdmEUEF1748O/7wg59MaOv7J065NzcOnrW8nKeo48lwxStSLi3QgCkiaATYGMMMgNG1QmlfxxO9Xtn1Ro2+UnDz5z+LgzTP0Hgtl7MhzmIAwI/XVgNBAqNXTh2v5Mwwg3Z3wEfJ7OxyHHPb9SNuEi 64Ed1RmAjm58rbHz85qdCfzi2fHfH67sj+6EX6s0/hoPvW5Mmq5MwimlCikUXtIYOYIYWUz1impU75ve6WzeHK+OKf/+AHYchVKuJRsLrTfHHY6A77AEAnZ6bBXc6ePj+1UOmPntxfkx3t5yzI5Uu7e3sYEDKmXirisisJnqxPH7Q68aD3rYVllkQJF6GUsRQmiE4j9/Wpkz9tHhGtDSUKACgOmcmQdhS1LDvLMoOwjJJ4d/ir4cfP0qaA23pxWc5N0FKhnQiLMQCgFceSFl24dPwLF4s/efe2P3llmoAbB5ZLw3gYxFE/jkdZNl8udA47KyunB72jb1Qn7c5gmGVcqJTzkVbAzY1TV89fvHR398nBYMgNikD5mhSQHWJuXM/OFaJhKLHhowAIWfjGl7b3nwbx8Nw3f9cCEDJjXAeDEQCQOGjFRzsny8FkRe43i0+fH2jIn5qZkxG/fft+rlDNlSYzxOanZxuj/hfPnLu+erCyvquSmKd6oOORkoMw+VTEb37hm6oXzdq5o2DU0akUMnbJhhgUHHeCelRBnGXDLAGDLMcrvHJ58dtvfXC4tdntCQG+bU/MTPl8sD5cIlMztSAOvvX6ceJEO3tHn7z3JG20XJleu3w1ToPdvQ0j+GS9/vWv3Xjl1Onru935W59ClnZGQ4sxi7FA8N+OBs8QOZqvD20z7fpoGDbikIMwMiNSlRG1DVWZiHiCCHUtG1vW+T/6fXDtrfZRnGSDqN8bdMbLpaS909EXaYBYKafPXl367PH9nd0gzTRRoWPnKWJ/81vfPnNy5Yc//OGt28/nHHhjoz03AD4cZdgUixULcJylFnZ7AOlYuXjj5R/9x//033cPr1VnRjKyLUtkkRYyFIJDUsv5DsWI0ZFUr/2977Gx+iCJDnu9IAgRNYVqaTjomFEEOaA2QV//8qLD0t2do62dWChIs2R8cqbgl0DJ4/NLb//dtzvd7kub2xceHQUqjJDqRmHQjQuOqyn ZiaKpyoRz4wuXzr70/MTyi8bR+72NEqPVTGL4a1NvIQRK5Cw3MDBz+RI6tpAAhJIzy65WEGOEYEhF5iv6+VE0F05P+f7Ui61Od4iV1hb1p6amM5USQFoh368t5IvjP/m1ylIuJAhFCGXMotQyjGCbbzX39p88Pvyv/8V2XZT3yED2sHYskteYCcIwBg0DwTVzZLkQ1XyTSuvzgUCpEglFVGMMmNSrzpEAim310sWX7j5ZI9Zc5/ARF6w651dqVSllKIWJhxF3is9XaaPNNOQsBkAdLRLCNMbNLBYGIbcgbdJu7iEpjO+aOE45fxIPl73SuFeORBpgaSwmPUp8ZutU89S1LRwazKgyFkLYY7bHnIXp4sNtoCtLtYnx8n/40Y+2d5U21PfL/+YH/5JqY7v5Ti950DksKlV4+CQnpaY441xzJUBigjMpM6N247ANOGKUKAnaOPliRILBsJ8geDDq1ZhjuzatFsMoKlBdMUbE8aDfY5S5tnvp8pWUZzzL0iR2bdbpHALA/wMUjQteOh3xIgAAAABJRU5ErkJggg==
In-reply-to: <C10D3FB0CD45994C8A51FEC1227CE22F3428CB5EF9@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <C10D3FB0CD45994C8A51FEC1227CE22F340768D793@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> <CAFLBxZZ9nqeb7CVqTZCsEtJRjgGMTHF2Ak929kvauj2KUFSOyg@xxxxxxxxxxxxxx> <C10D3FB0CD45994C8A51FEC1227CE22F3428CB5EF9@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Fri, 2011-10-28 at 10:07 +0800, Lv, Hui wrote:
> Yes, agree that, there are more things to do to make a more delicate solution 
> for this in the next step. 
>
Hi Hui,

Firt of ll, I took a look at the slides and it really seems there's a
huge amount of work there about benchmarking, analyzing the results and
chasing the culprits of the various sources of overhead... Great job,
indeed! :-)

> For example, we can consider per VM status to decide whether to turn on/off 
> the control to make it more fair, such as your point three.
> 
> However, as the first step, the current solution is straightforward and 
> effective. 
> 1) Most importantly, it happens when the scheduling frequency is excessive. 
> User can decide which degree is excessive by setting parameter 
> "opt_sched_rate_high"(default 50). If the system is crucial for latency 
> sensitive tasks, you can choose a higher value that this patch will have 
> little impact on it. User can decide which value is good for their 
> environment. However, from our experience, if the scheduling frequency is too 
> excessive, it also impairs the Qos of latency sensitive tasks due to frequent 
> interrupts by other VMs.
> 2) Considering the excessive scheduling condition, the preemption is turning 
> off entirely. If current running vcpu, yielded frequently, it cannot run for 
> the full 10ms. If current running vcpu, not yielded frequently, it can 
> possible run as long as up to 10ms. That means, this algorithm roughly 
> protect the un-yielded vcpu to run a long time slice without preemption. This 
> is something similar to your point 3 but in a roughly way. :)
> 3) Finally, this patch aimed to solve the issue when scheduling frequency is 
> excessive but not influence the normal case (less frequency). We should treat 
> these two cases separately. Since excessive scheduling case cannot guarantee 
> neither performance or Qos.
> 
Just something crossed my mind reading the patch and the comments, would
it make sense to rate-limit the calls coming from (non-timer) interrupt
exit paths while still letting the tick able to trigger a scheduling
decision? This just to be sure that at least the time slice enforcing
(if any) happens how expected... Could it make sense?

More generally speaking, I see how this feature can be useful, and I
also think it could live in the generic schedule.c code, but (as George
was saying) the algorithm by which rate-limiting is happening needs to
be well known, documented and exposed to the user (more than by means of
a couple of perf-counters).

For example this might completely destroy the time guarantees a
scheduler  like sEDF would give, and in such case it must be easy enough
to figure out what's going on and why the scheduler is not behaving as
expected!

For that reason, although again, this could be made general enough to be
sensible and meaningful for all the various schedulers, it might be
worthwhile to have it inside credit1 for now, where we know it will
probably yield the most of its benefits.

Just my 2 cents. :-)

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
----------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa  (Italy)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel