Re: [Xen-devel] RE: [RFC][PATCH 0/4] Modification of credit scheduler rev2

[NISHIGUCHI Naoki <nisiguti@xxxxxxxxxxxxxx>]

Hi Disheng,

Thank you for evaluating patches and reporting results.

Su, Disheng wrote:
> Hi Naoki,
>         Thanks for your excellent work.
>         This days, I tested the playing audio/video with your patches. With the 
> default credit scheduler, the audio effect is really bad(a lot of audio 
> glitches). But I got a better result with your patches. I list my findings 
> here, FYI.
>
>         1. What's the latency requirement for audio? I am not good at this 
> one:) I find some links regarding to it( 
> http://www.soundonsound.com/sos/jan05/articles/pcmusician.htm and 
> http://www.podcomplex.com/blog/setting-buffers-and-latency-for-your-audio-interface/).
>   In native env, setting the buffer size of audio hardware to produce a latency 
> of 23ms is acceptable even for many musicians. It's safe to say we have to 
> schedule in the VM for each 23ms for such case in virtual env when playing 
> audio in VM. Even worse for Vista, which has 10ms requirement ( 
> http://blogs.technet.com/markrussinovich/archive/2007/08/27/1833290.aspx ). 
> Apparently, the default credit scheduler can't handle well for this case.
Thanks for your information.
I'll see these links.

>         2. Test env:
> 		hardware: 
> 			Cpu: INTEL Core 2 Duo E6850
>                         Chipset: 82G33
>                         Memory: 2G
>                 software:
>                         Xen upstream(cs: 18881)
>                 doms configuration:
>                         guest A: primary HVM guest(integreted graphic card, 
> sound, USB controller directly assigned), playing mp3 with WMP in foreground + 
> copying large files(e.g. 2G) in background. 2 vcpus, 1G memory. Guest OS is 
> Windows XP or Vista.
>                         guest B: secondary HVM guest(also copying large files 
> in guest, no devices assigned). 2 vcpus, 128M memory. Guest OS is Windows XP.
>         
>         3. Configure the scheduler and Xen:
> 		a. the weight of guest B must be lower as much as possible(e.g. 10 for it, but 256 for guest A and dom0). Guest B is competing with Guest A for dom0. The lower the weight, the lesser chance to be scheduled in. 
> 		b. the boost credit needs to be larger as much as possible.(e.g 1000 for both primary guest and dom0). To make sure the guest A stays in boost priority longer when doing heavy I/O.
>                 c. vcpus of guest A need to be pinned to physical cpu. Without 
> pinned and guest is smp, the scheduler will dynamically migrate vcpus between 
> physcial cpus, and the audio glitches is also obvious. One of possible reason 
> is high freq of migration and the small runtime when the vcpu be scheduled in. 
> The migration rate is about 60~110 per second, and each migration has the 
> migration cost(such as cache, TLB miss, etc..). And the runtime is small, 90% 
> of runtime is less than 30us. It sounds not reasonable to migrate a vcpu, but 
> it just runs for a tens of microseconds.
>         With this configuration, both xp/vista guest works well, no glitches 
> usually.
>         
>         4. issues left:
>                 a. Abrupt glitches are still generated when the QEMU emulated 
> mouse being used and moving mouse quickly in guest A. Passing-through USB 
> mouse/keyboard to guest A, then no glitches.
I also noticed that. Though I don't know the precise cause, I found that 
dom0 and guest A would consume largely CPU time (hundreds of 
milliseconds) in such situation. In this case, the priority of dom0 and 
guest A falls rapidly, then guest B runs until the priority of dom0 and 
guest A becomes BOOST. In worst case, it will take about 120ms.
I tried to solve this issue as follows, but the scheduler correctly 
didn't schedule according to the weight of a domain.
- In csched_schedule(), if a vcpu runs over current time slice then the 
time slice is subtracted from the vcpu's credit.
I think to need investigate deeply.

>                 b. vcpu migration. As said before, without vcpu pinned, 
> glitches are obvious.
I think that this issue would be solved by adding the condition for 
migrating the vcpu.
e.g. If the vcpu has boost credit, don't migrate the vcpu.

I'll try to test.

>                 c. the limitation of weight for guest B. I have to set the 
> weight of guest B to 10. It may not be reasonable in real usage case.
Is copying large files in background on guest A indispensable?
In my test, guest A runs only video playing.
I think that my approach couldn't solve this issue.

>         Do you have the experience with audio? I don't know I have properly 
> configured your scheduler or not. Hope the your scheduler can solve the audio 
> issues also.
Sorry, I don't have the experience with audio.
But I'll try to reproduce your configuration and investigate.

Regards,
Naoki Nishiguchi

> NISHIGUCHI Naoki wrote:
> > Hi all,
> >
> > The patchset is revised version of patches that I was posted 10 days
> > ago. This patchset is consist of the following 4 patches.
> >
> > 1. Subtract credit consumed accurately and shorten cpu time per one
> > credit 
> > 2. Change the handling of credits over upper bound.
> > 3. Balance credits of each vcpu of a domain
> > 4. Introduce boost credit for latency-sensitive domain
> >
> > It was not possible to separate these cleanly.
> > Please apply these patches in numerical order.
> >
> > Please review these patches.
> > Any comments are appreciated.
> >
> > Best regards,
> > Naoki Nishiguchi
>
> Best Regards,
> Disheng, Su
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel