WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] Fast inter-VM signaling using monitor/mwait

To: Michael Abd-El-Malek <mabdelmalek@xxxxxxx>
Subject: RE: [Xen-devel] Fast inter-VM signaling using monitor/mwait
From: "Tian, Kevin" <kevin.tian@xxxxxxxxx>
Date: Wed, 6 May 2009 14:22:48 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc: Ian Pratt <Ian.Pratt@xxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Tue, 05 May 2009 23:24:55 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <FDA4E0F2-1C18-4F3E-A266-459DFB5EFE9F@xxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <F4E6A08B-E8A4-425B-A8FD-46A16D882A65@xxxxxxx> <4FA716B1526C7C4DB0375C6DADBC4EA34172EC1D7B@xxxxxxxxxxxxxxxxxxxxxxxxx> <0A882F4D99BBF6449D58E61AAFD7EDD613324E9D@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> <9FE7E996-1D69-4289-A8B5-FC37FEF487C9@xxxxxxx> <0A882F4D99BBF6449D58E61AAFD7EDD61370D71F@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> <FDA4E0F2-1C18-4F3E-A266-459DFB5EFE9F@xxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcnNjdNT7NCPEjZFR/e4SkpdUY4vVAAg01JA
Thread-topic: [Xen-devel] Fast inter-VM signaling using monitor/mwait
>From: Michael Abd-El-Malek [mailto:mabdelmalek@xxxxxxx] 
>Sent: 2009年5月5日 22:29
>
>On Apr 26, 2009, at 9:04 AM, Tian, Kevin wrote:
>
>>> From: Michael Abd-El-Malek [mailto:mabdelmalek@xxxxxxx]
>>> Sent: 2009年4月24日 5:48
>>>
>>> On Apr 21, 2009, at 5:01 AM, Tian, Kevin wrote:
>>>
>>>>> From: Ian Pratt
>>>>> Sent: 2009年4月21日 11:19
>>>>>
>>>>>> The mwait instruction is privileged.  So I added a new hypercall
>>>>>> that
>>>>>> wraps access to the mwait instruction.  Thus, my code has a Xen
>>>>>> component (the new hypercall) and a guest kernel component
>>> (code for
>>>>>> executing the hypercall and for turning off/on the timer  
>>>>>> interrupts
>>>>>> around the hypercall).  For this code to be merged into Xen, it
>>>>>> would
>>>>>> need to add security checks and check whether the
>>> processor supports
>>>>>> such a feature.
>>>>>
>>>>> I seem to recall that some newer CPUs have an mwait
>>>>> instruction accessible from ring3, using a different opcode --
>>>>> you might want to check this out.
>>>>>
>>>>> How do you deal with atomicity of the monitor and mwait? i.e.
>>>>> how do you stop the hypervisor pre-empting the VM and using
>>>>> monitor for its own purposes or letting another guest use it?
>>>>
>>>> That's a true concern. To use monitor/mwait sanely, software is
>>>> required
>>>> to not add voluntary context switch in between, however to
>>> ensure that
>>>> atomicity at hypercall level, I'm not sure about overall efficiency
>>>> when
>>>> multiple VMs are all active...
>>>
>>> I'm executing the montior and mwait instructions together in the
>>> hypercall.  The hypercall also takes an argument specifying the old
>>> value of the memory location.  When the mwait instruction
>>> returns, the
>>> hypervisor can check and handle any interrupts.  I 
>currently return a
>>> continuation so that the mwait hypercall is rexecuted at the end of
>>> handling interrupts.  I haven't really thought about what if the VM
>>> gets scheduled out.  These are the kinds of issues that I'd like to
>>> fix if the community wants to add this hypercall.  For my
>>
>> Maybe the reverse that you need consider those issues to persuade
>> the community or else it's like a very limited usage in real world.  
>> This
>> is something to hold the cpu exclusively with unknown time, unless
>> you also ensure producer, which writes to monitored address, not
>> being scheduled out too, which then further limits the 
>actual benefit.
>
>Interrupts will cause the mwait instruction to return.  So the same  
>periodic timer interrupts that are used for VM scheduling will  
>continue to be useful.  The CPU is not held exclusively for unbounded  
>time.

In Xen actual vcpu scheduling happens at the point before resuming
back to VM, instead of in timer interrupt ISR. So as long as your
monitor/mwait loop in hypercall doesn't exit before update is observed, 
scheduling won't happen.

>
>>> benchmarking
>>> purposes, I'm not worrying about this :)
>>>
>>>>> Have you thought about HVM guests as well as PV?
>>>>>
>>>>
>>>> For HVM guest, both vmexit and vmentry clears any address range
>>>> monitoring in effect and thus that won't work.
>>>
>>> I imagine this would cause the mwait instruction to execute before a
>>> write occurs to the memory address?  If so, the guest OS can check
>>> this (by comparing the memory address's value to the previous saved
>>> value), and reexecute the mwait hypercall.  Users of mwait already
>>> have to check whether their terminating condition has 
>occurred, since
>>> interrupts cause mwait to return.
>>
>> yes, then why do you need monitor/mwait, compared to a simple loop
>> checking data directly? :-)
>
>The simple spin-poll loop prevents the core from going into a low- 
>energy mode.  My motivation in using monitor/mwait is to get the  
>latency of spin-poll but with the energy efficiency of Xen events  
>(i.e., the CPU can go to sleep if the VM is waiting for a signal).

That's obvious a wrong model to go. There could be other runnable
threads with VM. Here it's not "if VM is waiting for a singal", instead
it's just "if one thread in VM is waiting for a signal".  

Thanks
Kevin
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel