Hi Akio,
I guess this patch is part of NUMA support.
Have we considered the whole NUMA support?
Such as,
1. We should add NUMA support into Xen.
2. Do we need to provide physical NUMA information to dom0?
1. the node topology.
2. the cost of one hop memory access
3. Do we provide physical node topology to Guest domain?
4. Do Xen scheduler need to consider node topolohy?
....
Yes, this patch fixes LSAPIC ID issue.
Maybe we need to generally consider the NUMA support.
Thanks,
Anthony
>-----Original Message-----
>From: xen-ia64-devel-bounces@xxxxxxxxxxxxxxxxxxx
>[mailto:xen-ia64-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Akio Takebe
>Sent: 2007年7月18日 15:04
>To: xen-ia64-devel
>Subject: [Xen-ia64-devel] [Patch] Pass the bare LSAPIC ID to dom0
>
>Hi, all
>
>This patch fix a issue which dom0 cannot boot with dom0_max_vcpus.
>Currently LSAPIC IDs are create by xen,
>but ACPI SRAT table is the bare table.
>So on some boxes node_cpuid[].phys_id are different from cpu_physical_id()s,
>and we cannot boot dom0.
>I think xen should pass the bare LSAPIC ID to dom0.
>How about you?
>
>Detail;
>
>I added the debug printks in build_cpu_to_node_map() on dom0,
>and get the following message.
>
> build_cpu_to_node_map: node_cpuid[0].phys_id=0 cpu_physical_id(0)=0
> build_cpu_to_node_map: node_cpuid[1].phys_id=1024 cpu_physical_id(1)=256
> build_cpu_to_node_map: node_cpuid[2].phys_id=768 cpu_physical_id(2)=512
> build_cpu_to_node_map: node_cpuid[3].phys_id=1792 cpu_physical_id(3)=768
> build_cpu_to_node_map: node_cpuid[4].phys_id=32768
>cpu_physical_id(4)=1024
> build_cpu_to_node_map: node_cpuid[5].phys_id=33792
>cpu_physical_id(5)=1280
> build_cpu_to_node_map: node_cpuid[6].phys_id=33536
>cpu_physical_id(6)=1536
> build_cpu_to_node_map: node_cpuid[7].phys_id=34560
>cpu_physical_id(7)=1792
> build_cpu_to_node_map: node_cpuid[8].phys_id=1 cpu_physical_id(8)=-1
> build_cpu_to_node_map: node_cpuid[9].phys_id=1025 cpu_physical_id(9)=-1
> build_cpu_to_node_map: node_cpuid[10].phys_id=769 cpu_physical_id(10)=-1
> build_cpu_to_node_map: node_cpuid[11].phys_id=1793
>cpu_physical_id(11)=-1
> build_cpu_to_node_map: node_cpuid[12].phys_id=32769
>cpu_physical_id(12)=-1
> build_cpu_to_node_map: node_cpuid[13].phys_id=33793
>cpu_physical_id(13)=-1
> build_cpu_to_node_map: node_cpuid[14].phys_id=33537
>cpu_physical_id(14)=-1
> build_cpu_to_node_map: node_cpuid[15].phys_id=34561
>cpu_physical_id(15)=-1
>
>At that time, some cpus cannot get node id like the below.
>
> build_cpu_to_node_map: cpu=0 node_cpuid[0].phys_id=0 is , node=0
> build_cpu_to_node_map: cpu=1, node=-1
> build_cpu_to_node_map: cpu=2, node=-1
> build_cpu_to_node_map: cpu=3 node_cpuid[2].phys_id=768 is , node=0
> build_cpu_to_node_map: cpu=4 node_cpuid[1].phys_id=1024 is , node=0
> build_cpu_to_node_map: cpu=5, node=-1
> build_cpu_to_node_map: cpu=6, node=-1
> build_cpu_to_node_map: cpu=7 node_cpuid[3].phys_id=1792 is , node=0
> build_cpu_to_node_map: cpu=8, node=-1
> build_cpu_to_node_map: cpu=9, node=-1
> build_cpu_to_node_map: cpu=10, node=-1
> build_cpu_to_node_map: cpu=11, node=-1
> build_cpu_to_node_map: cpu=12, node=-1
> build_cpu_to_node_map: cpu=13, node=-1
> build_cpu_to_node_map: cpu=14, node=-1
> build_cpu_to_node_map: cpu=15, node=-1
>
>
>Finally, dom0 get NULL pointer access like the below.
>
>ELILO boot:
>Uncompressing Linux... done
>Loading file initrd-2.6.18-xen.img-takebe...done
>Loading file vmlinuz-2.6.18-xen-takebe...done
>Uncompressing... done
> __ __ _____ ___ _ _ _
> \ \/ /___ _ __ |___ / / _ \ _ _ _ __ ___| |_ __ _| |__ | | ___
> \ // _ \ '_ \ |_ \| | | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> / \ __/ | | | ___) | |_| |__| |_| | | | \__ \ || (_| | |_) | | __/
> /_/\_\___|_| |_| |____(_)___/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
>
> http://www.cl.cam.ac.uk/netos/xen
> University of Cambridge Computer Laboratory
>
> Xen version 3.0-unstable (root@xxxxxxxxxxxxxxxx) () Thu Jul 12 21:57:50 JST
>2007
> Latest ChangeSet: Wed Jul 11 11:32:30 2007 -0600 15558:f536eb8576ee
>
>(XEN) Xen command line: BOOT_IMAGE=scsi0:EFI\redhat\xen.gz-takebe
>dom0_mem=8G
>dom0_max_vcpus=8
>(XEN) xen image pstart: 0x4000000, xenheap pend: 0x8000000
>(XEN) Xen patching physical address access by offset: 0x0
>(XEN) find_memory: efi_memmap_walk returns max_page=5400000
>(XEN) Before xen_heap_start: f0000000041c6fb0
>(XEN) After xen_heap_start: f000000004c4c000
>(XEN) Init boot pages: 0x10000d8 -> 0x4000000.
>(XEN) Init boot pages: 0x8000000 -> 0x69cf0000.
>(XEN) Init boot pages: 0x6ac6d098 -> 0x6b290010.
>(XEN) Init boot pages: 0x6b290070 -> 0x6b293fb0.
>(XEN) Init boot pages: 0x6b293ff9 -> 0x6b296000.
>(XEN) Init boot pages: 0x6b48d37d -> 0x6b49a010.
>(XEN) Init boot pages: 0x6b49a6a0 -> 0x6d000000.
>(XEN) Init boot pages: 0x100000000 -> 0x1000000000.
>(XEN) Init boot pages: 0x4080000000 -> 0x4100000000.
>(XEN) Init boot pages: 0x14004000000 -> 0x15000000000.
>(XEN) System RAM: 130688MB (133824512kB)
>(XEN) size of virtual frame_table: 326928kB
>(XEN) virtual machine to physical table: f6ffffffd6000000 size: 65424kB
>(XEN) max_page: 0x5400000
>(XEN) allocating frame table/mpt table at mfn 0.
>(XEN) Xen heap: 51MB (52944kB)
>(XEN) Reserving non-aligned node boundary @ mfn 262144
>(XEN) Reserving non-aligned node boundary @ mfn 16908288
>(XEN) Domain heap initialised: DMA width 32 bits
>(XEN) avail:0x3170074000000000,
>status:0x74000000000,control:0x3170000000000000,
>vm?0x10000000000
>(XEN) WARNING: no opcode provided from hardware(0)!!!
>(XEN) vm buffer size: 1048576, order: 6
>(XEN) vm_buffer: 0xf000000004d00000
>(XEN) Using scheduler: SMP Credit Scheduler (credit)
>(XEN) Time init:
>(XEN) .... System Time: 217700ns
>(XEN) .... scale: 28000001A
>(XEN) num_online_cpus=1, max_cpus=64
>(XEN) Brought up 16 CPUs
>(XEN) xenoprof: using perfmon.
>(XEN) perfmon: version 2.0 IRQ 238
>(XEN) perfmon: Montecito PMU detected, 27 PMCs, 35 PMDs, 12 counters (47 bits)
>(XEN) Maximum number of domains: 63; 18 RID bits per domain
>(XEN) *** LOADING DOMAIN 0 ***
>(XEN) Dom0 kernel: 64-bit, lsb, paddr 0x4000000 -> 0x53a6028
>(XEN) METAPHYSICAL MEMORY ARRANGEMENT:
>(XEN) Kernel image: 4000000->53a6028
>(XEN) Entry address: 400ff20
>(XEN) Init. ramdisk: 53ac000 len 1f737d
>(XEN) Start info.: 53a8000->53ac000
>(XEN) Dom0 max_vcpus=8
>(XEN) Dom0: 0xf000000007d54080
>(XEN) enable lsapic entry: 0xf00000007f92e7e4
>(XEN) enable lsapic entry: 0xf00000007f92e7f0
>(XEN) enable lsapic entry: 0xf00000007f92e7fc
>(XEN) enable lsapic entry: 0xf00000007f92e808
>(XEN) enable lsapic entry: 0xf00000007f92e814
>(XEN) enable lsapic entry: 0xf00000007f92e820
>(XEN) enable lsapic entry: 0xf00000007f92e82c
>(XEN) enable lsapic entry: 0xf00000007f92e838
>(XEN) DISABLE lsapic entry: 0xf00000007f92e844
>(XEN) DISABLE lsapic entry: 0xf00000007f92e850
>(XEN) DISABLE lsapic entry: 0xf00000007f92e85c
>(XEN) DISABLE lsapic entry: 0xf00000007f92e868
>(XEN) DISABLE lsapic entry: 0xf00000007f92e874
>(XEN) DISABLE lsapic entry: 0xf00000007f92e880
>(XEN) DISABLE lsapic entry: 0xf00000007f92e88c
>(XEN) DISABLE lsapic entry: 0xf00000007f92e898
>(XEN) ACPI_INTERRUPT_CPEI disabled for Domain0
>(XEN) Domain0 EFI passthrough: ACPI 2.0=0x7f924000 SMBIOS=0xf0000
>(XEN) Scrubbing Free RAM: ..................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>............................................................
>.............................!
> ...........................................................
>............................................................
>............................................................
>...........................................................................
>....
>done.
>(XEN) Xen trace buffers: disabled
>(XEN) Std. Loglevel: Errors and warnings
>(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
>(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to
>Xen).
>(XEN) Freed 128kB init memory.
>(XEN) vcpu_set_itc: Setting ar.itc is currently disabled (this message is only
>displayed once)
>Linux version 2.6.18-xen (root@xxxxxxxxxxxxxxxxxxxxxxxxxxxx) (gcc バ?`ジョン
>4.1.
>1 20070105 (Red Hat 4.1.1-52)) #6 SMP Fri Jul 13 01:01:11 JST 2007
>EFI v1.00 by Xen/ia64: SALsystab=0x2178 ACPI 2.0=0x7f924000 SMBIOS=0xf0000
>booting generic kernel on platform dig
>Number of logical nodes in system = 2
>Number of memory chunks in system = 5
>Initial ramdisk at: 0xe0000000053ac000 (2061181 bytes)
>SAL 0.1: Xen/ia64 Xen/ia64 version 0.0
>SAL: AP wakeup using external interrupt vector 0xf3
>No logical to physical processor mapping available
>ACPI: Local APIC address c0000000fee00000
>iosapic_register_platform_intr: invalid int type 0xff
>8 CPUs available, 16 CPUs total
>Running on Xen! start_info_pfn=0x14ea nr_pages=524288 flags=0x3
>MCA related initialization done
>Virtual mem_map starts at 0xa0007ffffdc04000
>SMP: Allowing 16 CPUs, 8 hotplug CPUs
>Built 2 zonelists. Total pages: 518806
>Kernel command line: rhgb root=LABEL=/ ro
>PID hash table entries: 4096 (order: 12, 32768 bytes)
>Console: colour VGA+ 80x25
>Placing software IO TLB between 0x55a4000 - 0x95a4000
>Memory: 8201712k/8300896k available (10915k code, 121360k reserved, 4877k data,
>672k init)
>McKinley Errata 9 workaround not needed; disabling it
>Dentry cache hash table entries: 1048576 (order: 9, 8388608 bytes)
>Inode-cache hash table entries: 524288 (order: 8, 4194304 bytes)
>Mount-cache hash table entries: 1024
>ACPI: Core revision 20060707
>Boot processor id 0x0/0x0
>Fixed BSP b0 value from CPU 1
>CPU 1: synchronized ITC with CPU 0 (last diff -1 cycles, maxerr 139 cycles)
>CPU 2: synchronized ITC with CPU 0 (last diff 4 cycles, maxerr 139 cycles)
>CPU 3: synchronized ITC with CPU 0 (last diff 2 cycles, maxerr 391 cycles)
>CPU 4: synchronized ITC with CPU 0 (last diff 5 cycles, maxerr 574 cycles)
>CPU 5: synchronized ITC with CPU 0 (last diff 5 cycles, maxerr 574 cycles)
>CPU 6: synchronized ITC with CPU 0 (last diff 5 cycles, maxerr 574 cycles)
>CPU 7: synchronized ITC with CPU 0 (last diff 4 cycles, maxerr 573 cycles)
>Brought up 8 CPUs
>Total of 8 processors activated (25473.84 BogoMIPS).
>Unable to handle kernel NULL pointer dereference (address 0000000000000000)
>swapper[1]: Oops 11012296146944 [1]
>Modules linked in:
>
>Pid: 1, CPU 0, comm: swapper
>psr : 00001210085a6010 ifs : 8000000000000001 ip : [<a000000100075431>]
>Not
>tainted
>ip is at sd_degenerate+0x71/0xc0
>unat: 0000000000000000 pfs : 400000000000048a rsc : 0000000000000007
>rnat: a000000100041ee0 bsps: 0000000000000000 pr : 0000000000009941
>ldrs: 0000000000000000 ccv : 0000000000000001 fpsr: 0009804c8a70433f
>csd : 0000000000000000 ssd : 0000000000000000
>b0 : a000000100075510 b6 : a000000100072b60 b7 : a000000100072b60
>f6 : 1003e0000000000000000 f7 : 1003e000000000000000e
>f8 : 1003e0000000000000200 f9 : 000000000000000000000
>f10 : 000000000000000000000 f11 : 000000000000000000000
>r1 : a000000101143620 r2 : 0000000000000099 r3 : e0000000011154a8
>r8 : 0000000000000000 r9 : 000000000000004d r10 : e000000001115484
>r11 : 0000000000000000 r12 : e000000102f0fcf0 r13 : e000000102f08000
>r14 : 0000000000000004 r15 : 0000000000000040 r16 : ffffffffffff5430
>r17 : 0000000000000030 r18 : 0000000000000030 r19 : 00000000000000fd
>r20 : e0000000011054a8 r21 : 0000000000000000 r22 :
>
>
>Best Regards,
>
>Akio Takebe
_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel
|
Home