This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Re: [PATCH 1/6][RESEND] xen: Add NUMA support to Xen

To: Keir Fraser <Keir.Fraser@xxxxxxxxxxxx>
Subject: Re: [Xen-devel] Re: [PATCH 1/6][RESEND] xen: Add NUMA support to Xen
From: Ryan Harper <ryanh@xxxxxxxxxx>
Date: Fri, 19 May 2006 16:08:58 -0500
Cc: Xen Devel <xen-devel@xxxxxxxxxxxxxxxxxxx>, Andi Kleen <ak@xxxxxxx>, Ryan Grimm <grimm@xxxxxxxxxx>
Delivery-date: Fri, 19 May 2006 14:09:26 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <403a4dbd4c7329e3de93e78054d5ffaf@xxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20060501215648.GU16776@xxxxxxxxxx> <200605152246.30374.ak@xxxxxxx> <25076f74e546e868c489e384ee838e2e@xxxxxxxxxxxx> <200605161457.09187.ak@xxxxxxx> <403a4dbd4c7329e3de93e78054d5ffaf@xxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.6+20040907i
* Keir Fraser <Keir.Fraser@xxxxxxxxxxxx> [2006-05-16 08:03]:
> On 16 May 2006, at 13:57, Andi Kleen wrote:
> >>Yes, my gut feeling looking at x86_64's numa.c is that it's going to
> >>need some heavier surgery than srat.c. I wouldn't worry so much about
> >>keeping that one close to the Linux original: if we end up pulling 
> >>down
> >>more Linux memory bookkeeping code later then we can always go back 
> >>and
> >>sync the file more closely. Keep it as clean as possible though,
> >>obviously (e.g., replacing whole functions is nicer than functions 
> >>that
> >>are a hacky halfway house between Linux and Xen, etc).
> >
> >If it helps I can split these functions into smaller ones in the 
> >mainline
> >sources. That could isolate the pglists in only very small functions.
> Thanks: I guess we'll see how the new patch turns out. It would 

I've got the ACPI numa.c parser function using
linux/arch/x86_64/mm/srat.c almost entirely unmodified.
linux/arch/x86_64/mm/numa.c was gutted retainly only the calls between
itself and srat.c.  I kept the memnodemap which has a nice phys_to_nid()
function.  There is still some cleanup to do, but I'd like some feedback
on what I used and didn't.

I didn't bring in mmzone.h but I did pull a few ideas and defines from
there.  We have a simple node_data structure with start_pfn,
spanned_pages and node_id.

I attempted to follow x86_64 numa.c which stashes the structure on
node-local memory.  This was problematic for 32-bit NUMA support.  On
the opteron  2-way that I had, the starting physical address for the
second node is 80000000, and when turned into a virtual address
(PAGE_OFFSET is FF000000), the resulting va is 17F000000, which
overflows unsigned long in 32-bit.  I started to use u64's, but several
functions (like map_pages_to_xen()) only take unsigned longs.  Rather
than go through getting that function working, I think it is perfectly
fine to just have a static array.  The structure we store out there
isn't accessed in any fast path.  If the structure becomes more
complicated in the future, or someone not having the structure on
node-local memory becomes a performance issue we can revisit.

The patch is function on 32-bit and 64-bit boxes and parse the SRAT
table and fills out the node_data array.  I installed a simple
keyhandler 'u' to dump the info to check that it was function after
booting up.

Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253

Attachment: acpi_numa.tgz
Description: GNU Unix tar archive

Xen-devel mailing list
<Prev in Thread] Current Thread [Next in Thread>