|
xen-devel
[Xen-devel] RE: Credit scheduler vs SEDF scheduler
Sorry for unreadable subject in earlier mail ~~~~~~~~~~~~~~~~~~~~~~~~ Hi all, I mean N/W for network send and receive statistics in xentop. I also wish to know the actual credit status of each domain with time granularity like after each 30 ms. Is there anything helpful available.
Thanks Gaurav 2009/5/11 Zhiyuan Shao <zyshao.maillist@xxxxxxxxx>
Some points I think that
may help to your doubts:
1) how the scheduler of
guest OS and VMM cooperate together.
Frankly, I also think this
is a rather interesting topic to research, and currently working on that. However,
before putting hands on this topic, we need some answers on several other tightly
related problems. First problem is time keeping mechanism of Xen VMM. Not yet
reading all the related codes of Xen, I guess for PV guests (do not consider
the HVM case, since it is more complicated than PV), each time that a VCPU is
scheduled in, one of the members of VCPU structure that stores current time
will be updated (see struct vcpu_time_info), while the time precedes when the
VCPU is actually is scheduled-out is simply “lost”. This will definitely cause
problems by confusing the scheduler of the guest OS, however, hardly find documents
that elaborate this problem or analyze what will happen in deep.
The second problem is load-balancing.
Unfortunately, this is a more complicated problem. The guest OS (for the SMP
guest cases) has its own load balancing strategy, and I think the bottom line
is that it will not waste processing time. Taking an example here, running a
single thread process (such as “dd if=/dev/zero of=./test.file bs=64
count=3200K”, it writes a 200MB file to virtual disk) inside a two-vcpu PV
guest. The process originally running on top of single VCPU, but after a while,
the VCPU is scheduled out (blocked and waiting for the data actually write to
the virtual disk file), and another VCPU is scheduled in, the process will
migrate to the other VCPU naturally. This case will result in extra overhead,
and I got performance data to prove this. Besides understanding the load
balancing strategy of guest OS, load balancing for scheduler of the VMM deserves
also careful consideration and design. Current default scheduler of Credit
actually do not doing very well on this point. The PCPU will steal VCPU from
its neighbors once it found it is idle. Smarter solutions are needed in later
schedulers (as George had mentioned).
2) what types of events
are ocurring during scheduling
I think it is good for you
to check the xentrace data for this problem.
3) xenmon problems.
I had not used that and
can not give you any suggestions, sorry for that.
4) Xentop
Do not know what you mean
for N/W?
Best,
Zhiyuan
发送时间: 2009年5月10日 20:48
主题: Re: 答复: 答复:
[Xen-devel] Credit scheduler vs SEDF scheduler
Hi list,
Some doubts.
(1) How the guest kernel scheduling strategies (linux kernel scheduling like
DEADLINE and IO schedulers in guests) affect the VM Scheduling?
(2) I want to know what types of events are ocurring during scheduling- which
domain is scheduled and what types of tasks it is running.
I am using xenmon- some probs.
(1) it does not show me --iocount option correctly. it always shows iocount= 0,
although i am running dd utility in one of the guset domain.
(2) when using xentop - it does not show any N/W data for domain 0.
am i missing something in configuration.
2009/5/7
Zhiyuan Shao <zyshao.maillist@xxxxxxxxx>
Believe your can use BCredit for this job.
However, you can only set the attributes of a domain statically, which means
the scheduler can not adjust its behavior according to the types of the work
loads.
Remember there is some publication in proceedings
of VEE 2009, which had addressed your problem. The solution continuously guests
the performance characteristics of the application, and adjust the preemption
decisions of the scheduler itself. You can take a look at that, but
unfortunately, they did not provide the patch behind the solution. Besides, I
am not sure that whether such solution is really helpful to solve real problems
on real hardware, since it is based its decisions on possibilities.
Zhiyuan
Hi Zhiyuan and George,
Thanks for the comments.
you are right i was getting spikes with more than 100ms and some times 400ms.
I will be trying with the patch u suggetsed.
one more aspect, can we classify schedulers according to applications they
run.?
Like compute intensive on Credit and Latency intensive on sEDF or some trade
off between them (In a cluster of servers).
Thanks
Gaurav
2009/5/6 Zhiyuan Shao
<zyshao.maillist@xxxxxxxxx>
Hi,
Your experiment results are actually “expected”.
There are several reasons make the results happen.
Firstly, Credit has long time slice (i.e., 30
millisec by default), and this make the observed close-to 30ms RTT to happen,
actually, during such long RTT (the spike), the domain in test (domain 1 of
your experiment) is just scheduled-out (actually, I guess the spikes
illustrated in your figure 1 should read higher than 30ms!). At the same time,
the scheduled-in vcpus (may executing the TEST program, which is cpu-hogs), and
they will not give up their 30-ms time slice if they are not preempted.
Secondly, as regulated by Credit scheduler, if a
VCPU has no more credits, it will not be BOOSTed even it do have some I/O
requests to handle. This explains why your figure1, which has little spikes in
the first 150 second, and crowed spikes follow. Since vcpu of domain 1 simply
used out their credits, and can not preempt the other running vcpus of other
domains, even they do have I/O operations to accomplish (i.e., PING).
Lastly, compared with Credit, SEDF generally has
short time slices, and it has bigger chances to preempt current running vcpu
when another vcpu found it has some I/O operations to be accomplished.
(actually, I donot understand SEDF very well, but conclude these features by
reading the trace data)
Actually, you can use BCredit patch to eliminate
the spikes, I can guarantee that it will work very well! J And I agree with George that you should not use PING
to justify whether a scheduler is good or not.
Best,
Zhiyuan
Hi list,
I am evaluating the scheduler behavior in xen.
I am using Xen 3.3.0
Dom0 and Dom1,2,3 and 4 all are opensuse 11.
I have one CPU Intensive TEST
which has no. of arithmatic instruction in an infinite while() loop.
i am pinging domain1 with an external machine. and noting the RTT values.
i have below experiments
time (s)
domain state
0
dom0,1,2,3,4 all idle
50
dom2 TEST started
100
dom3 TEST started
150
dom4 TEST started
200
dom0 TEST started
250
dom2 TEST stopped
300
dom3 TEST stopped
350
dom4 TEST stopped
400 dom0 TEST stopped
For these 400 seconds time, i have performed experiments with Credit and SEDF
sceduler.
the configuration is
Credit configuration - weight 256, cap 0
Domain
VCPU
0
2
1
2
2
2
3
2
4
2
all vcpu0s are pinned to pcpu0 and vcpu1s to pcpu1.
SEDF configuration - Period 10ms slice - 1.9ms
Domain
VCPU
0
2
1
2
2
2
3
2
4
2
all vcpu0s are pinned to pcpu0 and vcpu1s to pcpu1.
the results of RTT values are attached herewith. the performance of Credit is
very bad in comparison to SEDF in this scenario.
Please provide me some thought on it.
Thanks and Regards
Gaurav somani
M.Tech (ICT)
Dhribhai Ambani Institute of ICT,
INDIA
http://dcomp.daiict.ac.in/~gaurav
--
Gaurav somani
M.Tech (ICT)
Dhribhai Ambani Institute of ICT,
Gandhinagar
Gujarat, INDIA
onlineengineer@xxxxxxxx
gaurav_somani@xxxxxxxxxxxx
http://dcomp.daiict.ac.in/~gaurav
http://www.linkedin.com/in/gauravsomani
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
|
|
Home