This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] Re: One (possible) x86 get_user_pages bug

To: Jan Beulich <JBeulich@xxxxxxxxxx>
Subject: [Xen-devel] Re: One (possible) x86 get_user_pages bug
From: Jeremy Fitzhardinge <jeremy@xxxxxxxx>
Date: Thu, 27 Jan 2011 10:27:25 -0800
Cc: Kaushik Barde <kbarde@xxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Kenneth Lee <liguozhu@xxxxxxxxxx>, Nick Piggin <npiggin@xxxxxxxxx>, Marcelo Tosatti <mtosatti@xxxxxxxxxx>, linux-kernel@xxxxxxxxxxxxxxx, wangzhenguo@xxxxxxxxxx, Xiaowei Yang <xiaowei.yang@xxxxxxxxxx>, linqaingmin <linqiangmin@xxxxxxxxxx>, fanhenglong@xxxxxxxxxx, Avi Kivity <avi@xxxxxxxxxx>, Wu Fengguang <fengguang.wu@xxxxxxxxx>, Peter Zijlstra <a.p.zijlstra@xxxxxxxxx>
Delivery-date: Thu, 27 Jan 2011 10:28:06 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4D419416020000780002ECB7@xxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <4D416D9A.9010603@xxxxxxxxxx> <4D419416020000780002ECB7@xxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14 Lightning/1.0b3pre Thunderbird/3.1.7
On 01/27/2011 06:49 AM, Jan Beulich wrote:
> However, going through all the comments in gup.c again I wonder
> whether pv Xen guests don't violate the major assumption: There
> is talk about interrupts being off preventing (or sufficiently
> deferring) remote CPUs doing TLB flushes. In pv Xen guests,
> however, non-local TLB flushes do not happen by sending IPIs -
> the hypercall interface gets used instead

Yes, I was aware of that synchronization mechanism, and I think I'd
convinced myself we were OK.  But I can't think was that reasoning was -
perhaps it was something as simple as "gupf isn't used under Xen" (which
may have been true at the time, but isn't now).

As clever as it is, the whole "disable interrupts -> hold off IPI ->
prevent TLB flush -> delay freeing" chain seems pretty fragile.  I guess
its OK if we assume that x86 will forever have IPI-based cross-cpu TLB
flushes, but one could imagine some kind of "remote tlb shootdown using
bus transaction" appearing in the architecture.

And even just considering virtualization, having non-IPI-based tlb
shootdown is a measurable performance win, since a hypervisor can
optimise away a cross-VCPU shootdown if it knows no physical TLB
contains the target VCPU's entries.  I can imagine the KVM folks could
get some benefit from that as well.

So is there some way we can preserve the current scheme's benefits while
making it a bit more general?  (If anyone else has non-IPI-based
shootdown, it would be s390; is there some inspiration there?  An
instruction perhaps?)


Xen-devel mailing list