WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] Fast inter-VM signaling using monitor/mwait

To: Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Subject: Re: [Xen-devel] Fast inter-VM signaling using monitor/mwait
From: Michael Abd-El-Malek <mabdelmalek@xxxxxxx>
Date: Thu, 7 May 2009 12:25:26 -0400
Cc: "Tian, Kevin" <kevin.tian@xxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Ian Pratt <Ian.Pratt@xxxxxxxxxxxxx>
Delivery-date: Thu, 07 May 2009 09:26:02 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <C62765D1.A19E%keir.fraser@xxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <C62765D1.A19E%keir.fraser@xxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On May 6, 2009, at 11:05 AM, Keir Fraser wrote:

On 06/05/2009 15:38, "Michael Abd-El-Malek" <mabdelmalek@xxxxxxx> wrote:

In Xen actual vcpu scheduling happens at the point before resuming
back to VM, instead of in timer interrupt ISR. So as long as your
monitor/mwait loop in hypercall doesn't exit before update is
observed,
scheduling won't happen.

I'm not an expert on Xen scheduling, so please correct my following
understanding.  For the credit scheduler, csched_tick sets the next
timer interrupt.  So after the mwait hypercall executes the mwait
instruction and is waiting for a memory write, I observe the timer
interrupt eventually causing the mwait instruction to return.  The
mwait hypercall can then run the scheduler.

The issue is, what if another VM's VCPUs are runnable? Xen should prefer to run those rather than pause the CPU on an MWAIT, right? But if it does that
it will lose the memory-access wakeup property of the MWAIT and cannot
schedule the 'MWAIT'ing VM back in promptly when the relevant memory access occurs. I don't see that MWAIT can used effectively in a guest idle loop
unless you are happy to bin the work-conserving property of the Xen
scheduler (e.g., by dedicating a physical CPU to the VM).

I see the issue, thanks for explaining this. Yes, I have dedicated a physical CPU to my VM, which does throw away the work-conserving property. As a solution, what if the source VM continues to use the "send event" hypercall to signal the destination VM, but we modify the "send event" hypercall as follows. If the destination VM is currently executing _and_ is blocked in an mwait hypercall, then we simply write to its monitored memory address. Otherwise, we fall back to the existing event channel mechanism.

How does that sound?

Thanks,
Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>