WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: [RFC][PATCH] Scheduler interface changes for credit2

To: Keir Fraser <Keir.Fraser@xxxxxxxxxxxxx>
Subject: [Xen-devel] Re: [RFC][PATCH] Scheduler interface changes for credit2
From: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
Date: Mon, 22 Feb 2010 16:16:34 +0000
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Mon, 22 Feb 2010 08:17:15 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <C7A85593.AE86%keir.fraser@xxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <C7A85593.AE86%keir.fraser@xxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird 2.0.0.23 (X11/20090817)
Keir Fraser wrote:
They're all fine, even probably sched-context_switch-callback.diff, which I
suppose is your new alternative to having vcpus-which-arent-yet-schedulable
left on the shared runqueue? I suppose, although I reckon it could still be
done another way and using the vcpu_migrate logic, that this is a smaller
and neater way to do it really.
Exactly. The migrate logic calls cpu_pick, which is also called from other contexts. Having the same function try to do both, and setting the "migrating" flag when the vcpu isn't migrating, just seems hackish and likely to be confusing to me. It does add another "Check to see if this function exists" on the hot path for the other schedulers.

I suppose we could think about changing things around, so that there's a generic "callback on context switch" flag. Then we'd only have one callback in the context switch code, and the check would only happen if the flag was set (reducing cache footprint). It would be a slightly larger patch, and make changes to the other schedulers, but those would be pretty minimal.

I'll play around with it and see how it looks.

-George
All this are apply-able after 4.0.

 -- Keir

On 22/02/2010 15:22, "George Dunlap" <George.Dunlap@xxxxxxxxxxxxx> wrote:

Keir, any thoughts?  Does this seem like a reasonable approach?

Thanks,
 -George

On Mon, Feb 15, 2010 at 5:20 PM, George Dunlap
<George.Dunlap@xxxxxxxxxxxxx> wrote:
The two attached patches change the scheduler interface to allow
credit2 to have several cpus share the same runqueue.  The patches
should have almost no impact on the current schedulers.  The patches
and their reasonings are below.  I've also attached the patches for
the prototype credit2 scheduler, for reference.

* Add a context swich callback (sched-context_switch-callback.diff)

Add a callback to tell a scheduler that a vcpu has been completely
context-switched off a cpu.

When sharing a runqueue, we can't put a scheduled-out vcpu back on the
runqueue until it's been completely de-scheduled, because it may be
grabbed by another processor before it's ready.  This callback allows
a scheduler to detect when a vcpu on its way out is completely off the
processor, so that it can put the vcpu on the runqueue.

* Allow sharing of locks between cpus (sched-spin_lock-pointers.diff)

Have per-cpu pointers, initialized to per-cpu locks, which the
scheduler may change during its init to reconfigure locking
granularity.

There are a number of race conditions having to do with updating of
v->is_running and v->processor, all having to do with the fact that
vcpus may change cpus without an explicit migrate.  Furthermore, the
scheduler needs runqueues to be covered by a lock as well.  The
cleanest way to solve all of these is to have the scheduler lock and
the runqueue lock coincide.

* Add a "scheduler" trace class (trace-sched-class.diff)
Uses defined on a per-scheduler basis

I've been running parallel kernel compiles on a 16-way box (2x4x2) for
several hours now without deadlocks or BUG()s.  As far as I'm
concerned, with these changes, credit2 is now ready to be checked in,
as long as it's not set to the default scheduler.

All of the above:
Signed-off-by: George Dunlap <george.dunlap@xxxxxxxxxxxxx>





_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel