Re: [Xen-devel] NUMA problem with AMD G34 system

[Andre Przywara <andre.przywara@xxxxxxx>]

Brian Marcotte wrote:
> I'm having a problem with NUMA on an AMD G34 system. Xen seems to think
> that out of the 16 total cores, there are 12 on node0 and 4 on node3.
>
> The box is:
>
>     SuperMicro H8DGU (dual socket G34)
>     Pair of AMD 6128 CPUs (8 core each)
>     32 GB of RAM.
>     BIOS has been upgraded three times - no help.
>
>     xen-unstable from Feb 12 (same problem on 3.4.x).
>
> I'm including the output from the "u" command below to show Xen's idea
> of the CPU->node mapping. Complete boot log is here:
>
>     http://www.panix.com/~marcotte/g34/boot.txt
>
> Any help would be appreciated.
Have you checked that you populated the DIMM slots properly?
The SRAT CPU table seems sane, but the memory table seems to say that 
there is no memory on node 1 and 3, thus Xen will simply put those CPUs 
to node 0 as it cannot cope with memory-less nodes.
If I understand the manual right, you first have to populate all the 
blue slots, then the black ones.
Hope that helps,
Andre.


> Thanks.
>
> - Brian
>
> (XEN) Xen version 4.1.0-rc5-pre (root@xxxxxxxxxxxxxx) (gcc version 4.3.2 
> (Debian 4.3.2-1.1) ) Sat Feb 12 18:53:49 EST 2011
> (XEN) Command line: dom0_mem=600M com1=57600,8n1 console=com1 no-mce 
> loglvl=debug
>
> (XEN) 'u' pressed -> dumping numa info (now-0x2:5A61B881)
> (XEN) idx0 -> NODE0 start->0 size->4325376
> (XEN) phys_to_nid(0000000000001000) -> 0 should be 0
> (XEN) idx3 -> NODE3 start->4325376 size->4194304
> (XEN) phys_to_nid(0000000420001000) -> 3 should be 3
> (XEN) CPU0 -> NODE0
> (XEN) CPU1 -> NODE0
> (XEN) CPU2 -> NODE0
> (XEN) CPU3 -> NODE0
> (XEN) CPU4 -> NODE0
> (XEN) CPU5 -> NODE0
> (XEN) CPU6 -> NODE0
> (XEN) CPU7 -> NODE0
> (XEN) CPU8 -> NODE3
> (XEN) CPU9 -> NODE3
> (XEN) CPU10 -> NODE3
> (XEN) CPU11 -> NODE3
> (XEN) CPU12 -> NODE0
> (XEN) CPU13 -> NODE0
> (XEN) CPU14 -> NODE0
> (XEN) CPU15 -> NODE0
> (XEN) Memory location of each domain:
> (XEN) Domain 0 (total: 153600):
> (XEN)     Node 0: 132232
> (XEN)     Node 3: 21368
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel