Re: [Xen-devel] Credit scheduler

Thanks for the explanations. Some more questions

Please see my comments inline:

Thanks

Prabha

-------------- Original message --------------
From: Mats Petersson <mats@xxxxxxxxxxxxxxxxx>

> At 21:46 12/07/2007, pak333@xxxxxxxxxxx wrote:
> >Hi Atsushi
> >
> >I am still confused, so let me explain what i think should happen
> >and understand from you why it may not happen.
> >
> >I have 2 cpu intensive VMs and 1 IO intensive VM. My system has 4
> >physical cpus and 8 virtual cpus.
> >
> >Now using default parameters for the credit scheduler, the cpu
> >intersive vcpu should run for 30 millsec. It does not, it runs for
> >less (few microsecs).
> >
> >You said this could be because of softirq which are raised after a
> >system call. Can i disable the softirqs. What happens if i do? If i
> >cannot disable is there a way to see what VM is raising the softirq.
> >If the cpu-VM raises the softirq it ge ts preempted or does it
> >continue to run. How can I montior the softirqs raised by this VM.
> >Can I tell if the premption is from a softirq or from something else.
>
> I don't believe you can "disable" a softirq - you could of course
> avoid doing the call to "do_softirq", but that is most likely just
> going to completely remove any scheduling capability at all in the
> system, making the system "single-tasking". Not a recommended way, in
> my opinion.
>

Agreed

if softirq is indeed a problem, and if a cpu-intensive VM is being preempted because of a softirq, then maybe affintizing the interrupts of dom0 to a pcpu will reduce this preemption. Comments?

> >
> >Also, if the IO VM requests an IO, it will block and dom0 wakes up
> >and gets scheduled to run to service the IO. Will it preempt the cpu
> >intensive VM. If so why? Shouldn't the cpuintensiveV M get its
> >quantum of time.
> >Or does the IOVM get higher priority to preempt the cpu intensive
> >VM. How does the scheduler pick which cpu to run dom0 on ( if all
> >the vcpus running are cpu intensive)? If there is a mix of cpu-vcpus
> >and Io-vcpus, which will be the victim
>
>
> The whole idea of having separate queues for IO and CPU intensive VMs
> (or processes in a normal kernel scenario) is to allow the
> IO-intensive tasks to avoid waiting for the next time-slot when a
> CPU-hogging VM/process is running [1]. The normal behaviour for the
> scheduler is to determine dynamically if the current VM/process is IO
> or CPU intensive.
>
> Unless Dom0 is used for doing some silly CPU-intensive task
> (calculating PI with a million decimal points or compiling Xen +
> Linux kernel, for example), it will most likely look to the scheduler
> like a IO-intensive VM. So it will have the same priority as any
> other IO-intensive VM (assuming Dom0 has equal scheduler parameters
> as other guests, which I'm pretty sure is the default behaviour).
>

Does dom0 have a higher pritority than any CPU intensive VM
> If, like in your example, there are a number of processors busy with
> CPU-intensive tasks, and others with IO-intensive tasks, the most
> likely scenario is that any new IO-request will go be run on a
> "previously CPU-intensive" CPU - but on the other hand, if you have a
> lot of IO-intensive processing, there should be some processor(s)
> that are "asleep" - unless all CPU's are busy running CPU-intensive tasks...

But will it preempt a CPU intensive vcpu if no cpu is available
>
> Note that scheduling is a complex subject, and there have been dozens
> of PhD dissertations written on the subject.

Yes, but what does the Xen credit scheduler actually do
>
> [1] The idea being that if we process the IO-request that just
> completed, we can set up another IO-request, and this will get better
> IO-throughput than waiting a long time (relatively speaking 30ms is
> an eternity to the processor).

Don't understand what you mean here.

>
> --
> Mats
> >
> >As you see i have a lot of questions, so please bear with me and
> >help me understand the scheduling behavior of Xen
> >
> >Thanks
> >Prabha
> >
> >
> >
> >
> >-------------- Original message --------------
> >From: Atsushi SAKAI
> >
> > > Hi Prabha
> > >
> > > Please check do_softirq in assembler.
> > > for x86
> > > call do_softirq
> > > is in.
> > >
> > > And usually do_softirq is executed at the end of system call.
> > >
> > > Thanks
> > > Atsushi SAKAI
> > >
> > >
> > >
> > > pak333@xxxxxxxxxxx wrote:
> > >
> > > > Thanks for the explanation, i am not sure i und erstood it correctly.
> > > >
> > > >
> > > > when does softirq get set. So when softirq is set a processor enters the
> > > scheduler every 3 micro secs? where in the source code is this
> > handled. please
> > > could you send me some pointers
> > > >
> > > > thanks
> > > > -Prabha
> > > >
> > > >
> > > >
> > > >
> > > > -------------- Or iginal message --------------
> > > > From: Atsushi SAKAI
> > > >
> > > > > Hi,$B!!(BPrabha
> > > > >
> > > > > $B!!(B30msec is the maximum time slice.
> > > > > And I guess your 3 microsecond is the response after
> > > > > SCHEDULE_SOFTIRQ is set.
> > > > > As you know,
> > > > ; > If SCHEDULE_SOFTIRQ is set,
> > > > > it waits softirq calls schedule(). (this time interval is 3microsec)
> > > > >
> > > > > And I/O intensive has higher priority than CPU intensive.
> > > > > So I/O intensive job is first dispatched domain in runq.
> > > > > This is because latency improvement for I/O intensive guest.
> > > > >
> > > > > So your behavior is not strange.
> > > > >
> > > > > Thanks
> > > > > Atsushi SAKAI
> > > > >
> > > > >
> > > > > pak333@xxxxxxxxxxx wrote:
> > > &g
> >t; >
> > > > > > Hi
> > > > > >
> > > > > > I have noticed that VMs are typically spending 3 microsecs
> > or less before
> > > they
> > > > > are being prempted. I thought that the credit schduler time
> > slice was 30 ms.
> > > I
> > > > > have 4 VMs running and they are all cpu intensive except for
> > 1 (which is IO
> > > > > intensive) but having a VM spend max 3 micro secs before
> > being kicked out
> > > seems
> > > > > strange.
> > > > > >
> > > > > > Is there something else going on that i am not aware of. Is
> > the time slice
> > > > > really 30 millisecs? I am using default parameters of the
> > credit scheduler.
> > > > > >
> > > > > > Thanks
> > > > > > -Prabha
> > > > >
> > > > >
> > >
> > >
> > >
> > > ________________________ _______________________
> > > Xen-devel mailing list
> > > Xen-devel@xxxxxxxxx source.com
> > > http://lists.xensource.com/xen-devel
> >
> >_______________________________________________
> >Xen-devel mailing list
> >Xen-devel@xxxxxxxxxxxxxxxxxxx
> >http://lists.xensource.com/xen-devel
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

WARNING - OLD ARCHIVES

xen-devel

Re: [Xen-devel] Credit scheduler