This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] [PATCH, RFC] x86: make the GDT per-CPU

To: Jan Beulich <jbeulich@xxxxxxxxxx>, <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] [PATCH, RFC] x86: make the GDT per-CPU
From: Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Date: Thu, 11 Sep 2008 11:54:49 +0100
Delivery-date: Thu, 11 Sep 2008 03:55:44 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <48C7F75C.76E4.0078.0@xxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AckT/M90DbeuJH/wEd2NXQAX8io7RQ==
Thread-topic: [Xen-devel] [PATCH, RFC] x86: make the GDT per-CPU
User-agent: Microsoft-Entourage/
On 10/9/08 15:35, "Jan Beulich" <jbeulich@xxxxxxxxxx> wrote:

> The major issue with supporting a significantly larger number of physical
> CPUs appears to be the use of per-CPU GDT entries - at present, x86-64
> could support only up to 126 CPUs (with code changes to also use the
> top-most GDT page, that would be 254). Instead of trying to go with
> incremental steps here, by converting the GDT itself to be per-CPU,
> limitations in that respect go away entirely.

Two thoughts:

Firstly, we don't really need the LDT and TSS GST slots to be always valid.
Actually we always initialise the slot immediately before LTR or LLDT. So we
could even have per-CPU LDT and TSS initialisation share a single slot.
Then, with the extra reserved page, we'd be good for nearly 512 CPUs.

Secondly: Actually your patch looks not too bad. But the double LGDT in
context switch is nasty. But also I do not see why it is necessary?
Presumably your fear is about using the prev->vcpu_id's mapped GDT in
next->vcpu_id's page tables? But we should only be relying on GDT entries
(HYPERVISOR_CS, HYPERVISOR_DS, for example) which are identical in all
per-CPU GDTs. So why do you need to add that LGDT before CR3 switch at all?

You would need to use l1e_write_atomic() in the context-switch code, to make
sure all VCPU's hypervisor reserved GDT mappings are always valid. Actually
you must at least use l1e_write() in any case -- it is not safe to not use
one of those macros on a live pagetable (by which I mean possibly in use by
some CPU) because a direct write of a PAE pte is not atomic and can cause
the pte to pass through a bogus intermediate state (which could be bogusly
prefetched by a CPU into its TLB. Yuk!).

 -- Keir

Xen-devel mailing list