This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] stale TLB contents?

To: Tim Deegan <Tim.Deegan@xxxxxxxxxx>
Subject: Re: [Xen-devel] stale TLB contents?
From: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Date: Mon, 24 Jan 2011 14:29:42 +0100
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Mon, 24 Jan 2011 05:30:31 -0800
Dkim-signature: v=1; a=rsa-sha256; c=simple/simple; d=ts.fujitsu.com; i=juergen.gross@xxxxxxxxxxxxxx; q=dns/txt; s=s1536b; t=1295875784; x=1327411784; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; z=Message-ID:=20<4D3D7EC6.3060000@xxxxxxxxxxxxxx>|Date:=20 Mon,=2024=20Jan=202011=2014:29:42=20+0100|From:=20Juergen =20Gross=20<juergen.gross@xxxxxxxxxxxxxx>|MIME-Version: =201.0|To:=20Tim=20Deegan=20<Tim.Deegan@xxxxxxxxxx>|CC: =20"xen-devel@xxxxxxxxxxxxxxxxxxx"=20<xen-devel@xxxxxxxxx source.com>|Subject:=20Re:=20[Xen-devel]=20stale=20TLB=20 contents?|References:=20<4D3D780A.4030001@xxxxxxxxxxxxxx> =20<20110124131340.GN8286@xxxxxxxxxxxxxxxxxxxxxxx> |In-Reply-To:=20<20110124131340.GN8286@xxxxxxxxxxxxxxxxxx e.com>|Content-Transfer-Encoding:=207bit; bh=6+uklFtse7/2qZ1EWx2w938vx2uWtA6LaTSDR4ow4MI=; b=Rw+3JUsJGp06Q8ctdvXrpNAZQ3SeYBhbx99MCcOQ+NAjaT3ZTE/zlWa/ jDn0Wsnp6Jpz111WbeXmdwwecI9lJ0/AOEVp8Mzlp10Rm+e4SX9s/1iQX 2yDN6aSx9cyzV9jdJqqtjuWHPusJ2lq3U8mmfsMRSt1vUYL7NGF+P9aK/ mmOhZxU8e+47QmE4urxXmS68Bh/Gkrtu6Efi2wFMfqTSVzTvwZxik1QNi K1o8b8+iKRowiDO90JkNZ5tAfVxjC;
Domainkey-signature: s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=tINbXvWEGkcPzg3hRLfQVizjcDpofHXcp879zsrR2G4Y4iyFZwCr/Yu/ 4sEFvfpeXQtQ1klhG6O1WHVaQVVSXvxMwRVZvRCxfOUuuAhaeaDc2gg3K g69EMIFa3Ao9LiBdUYyYWOg/4n5t1WD74jDfPd1SBNVJTUNgAfLsday/e Tsmo9Xk4aasfCaf1qoKAIj8XS2D0WAdEDLRc1AgWxxFX5OoEpMFRso5W6 iT2l/vi0sFl65cQzHKAiDT7RbGry2;
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20110124131340.GN8286@xxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: Fujitsu Technology Solutions
References: <4D3D780A.4030001@xxxxxxxxxxxxxx> <20110124131340.GN8286@xxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20101226 Iceowl/1.0b1 Icedove/3.0.11
On 01/24/11 14:13, Tim Deegan wrote:
At 13:00 +0000 on 24 Jan (1295874058), Juergen Gross wrote:

in our BS2000 guest running as HVM with EPT on x86_64 I have a problem which
seems to be related to stale TLB entries. I'm pretty sure I have invalidated
the TLB correctly after a change of the page tables, so I've searched for
possible problems in the hypervisor.

Xen is version 4.0 from SLES 11 SP1.

If I have read the sources correctly, neither INVLPG nor reload of CR3 are
handled by the hypervisor. And I didn't find an explicit clearing of the TLB
when a vcpu is switching physical cpus. So I think the following scenario is

- a vcpu is running on physical cpu A creating a TLB entry
- the vcpu is scheduled on physical cpu B, while physical cpu A is left idle
- on physical cpu B the TLB entry is cleared by INVLPG or load CR3
- the vcpu is scheduled on physical cpu A again (no other vcpu was active
    there in between), CR3 is same as when vcpu left cpu A
- the old TLB entry from the vcpu is still valid there!

Do I miss something?

vmx_do_resume() calls hvm_asid_flush_vcpu() if the VCPU is migrating
onto this CPU, so the VCPU should get a fresh ASID when it comes back to
CPU A.  Processors with no ASID support flush their TLBs on every
VMENTER and VMEXIT, so I don't see where we could leak TLB entries.

Ah, this was the missing information I needed!
Thanks, I'll keep on searching...


Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@xxxxxxxxxxxxxx
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>