WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] [PATCH] Fix hypervisor crash with unpopulated NUMA nodes

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] [PATCH] Fix hypervisor crash with unpopulated NUMA nodes
From: Andre Przywara <andre.przywara@xxxxxxx>
Date: Wed, 7 Oct 2009 11:45:42 +0200
Delivery-date: Wed, 07 Oct 2009 02:46:39 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird 2.0.0.21 (X11/20090329)
Hi Keir,

on NUMA systems with memory-less nodes Xen crashes quite early in the hypervisor (while initializing the heaps). This is not an issue if this happens to be the last node, but "inner" nodes trigger this reliably.
On multi-node processors it is much more likely to leave a node unequipped.
The attached patch fixes this by enumerating the node via the node_online_map instead of counting from 0 to num_nodes.

The resulting NUMA setup is still somewhat strange, but at least it does not crash. In lowlevel/xc/xc.c there is again this enumeration bug, but I suppose we cannot access the HV's node_online_map from this context, so the xm info output is not correct (but xm debug-keys H is).
I plan to rework the handling of memory-less nodes later.

Please also consider committing this to the -testing trees.

Regards,
Andre.

Signed-off-by: Andre Przywara <andre.przywara@xxxxxxx>

--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448 3567 12
----to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
diff -r 440d22c01652 xen/common/page_alloc.c
--- a/xen/common/page_alloc.c   Wed Oct 07 08:07:06 2009 +0100
+++ b/xen/common/page_alloc.c   Wed Oct 07 11:40:54 2009 +0200
@@ -294,7 +294,6 @@
         node = cpu_to_node(smp_processor_id());
 
     ASSERT(node >= 0);
-    ASSERT(node < num_nodes);
     ASSERT(zone_lo <= zone_hi);
     ASSERT(zone_hi < NR_ZONES);
 
@@ -323,8 +322,9 @@
         } while ( zone-- > zone_lo ); /* careful: unsigned zone may wrap */
 
         /* Pick next node, wrapping around if needed. */
-        if ( ++node == num_nodes )
-            node = 0;
+        node = next_node(node, node_online_map);
+        if (node == MAX_NUMNODES)
+            node = first_node(node_online_map);
     }
 
     /* Try to free memory from tmem */
@@ -466,7 +466,6 @@
 
     ASSERT(order <= MAX_ORDER);
     ASSERT(node >= 0);
-    ASSERT(node < num_online_nodes());
 
     for ( i = 0; i < (1 << order); i++ )
     {
@@ -817,13 +816,13 @@
 static unsigned long avail_heap_pages(
     unsigned int zone_lo, unsigned int zone_hi, unsigned int node)
 {
-    unsigned int i, zone, num_nodes = num_online_nodes();
+    unsigned int i, zone;
     unsigned long free_pages = 0;
 
     if ( zone_hi >= NR_ZONES )
         zone_hi = NR_ZONES - 1;
 
-    for ( i = 0; i < num_nodes; i++ )
+    for_each_online_node(i)
     {
         if ( !avail[i] )
             continue;
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
<Prev in Thread] Current Thread [Next in Thread>