[Xen-devel] git commit 9fd67b4ed0714ab718f1f9bd14c344af336a6df7 (x86-64: Give vvars their own page) breaks Xen PV guests (64-bit).

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

Hey Andy,

I just started testing linus/master and found out that I get this bootup error:

mapping kernel into physical memory
about to get started...
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 3.0.0-rc1-00169-gae7bd11 (konrad@phenom) (gcc 
version 4.4.4 20100503 (Red Hat 4.4.4-2) (GCC) ) #1 SMP PREEMPT Mon Jul 25 
10:55:02 EDT 2011
[    0.000000] Command line: console=hvc0 debug     earlyprintk=xenboot
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] released 0 pages of unused memory
[    0.000000] Set 0 page(s) to 1-1 mapping.
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000080000000 (usable)
[    0.000000]  Xen: 0000000100000000 - 0000000100800000 (usable)
[    0.000000] bootconsole [xenboot0] enabled
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) 
==> (reserved)
[    0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x100800 max_arch_pfn = 0x400000000
[    0.000000] last_pfn = 0x80000 max_arch_pfn = 0x400000000
[    0.000000] initial memory mapped : 0 - 100e2000
[    0.000000] Base memory trampoline at [ffff88000009b000] 9b000 size 20480
[    0.000000] init_memory_mapping: 0000000000000000-0000000080000000
[    0.000000]  0000000000 - 0080000000 page 4k
[    0.000000] kernel direct mapping tables up to 80000000 @ 7fbfd000-80000000
[    0.000000] xen: setting RW the range 7ff76000 - 80000000
[    0.000000] init_memory_mapping: 0000000100000000-0000000100800000
[    0.000000]  0100000000 - 0100800000 page 4k
[    0.000000] kernel direct mapping tables up to 100800000 @ 7f3f3000-7fbfd000
[    0.000000] xen: setting RW the range 7f3f8000 - 7fbfd000
[    0.000000] RAMDISK: 01b6f000 - 100e2000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000100800000
[    0.000000] Initmem setup node 0 0000000000000000-0000000100800000
[    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x00100800
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[3] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x000000a0
[    0.000000]     0: 0x00000100 -> 0x00080000
[    0.000000]     0: 0x00100000 -> 0x00100800
[    0.000000] On node 0 totalpages: 526224
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 5 pages reserved
[    0.000000]   DMA zone: 3923 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 14280 pages used for memmap
[    0.000000]   DMA32 zone: 505912 pages, LIFO batch:31
[    0.000000]   Normal zone: 28 pages used for memmap
[    0.000000]   Normal zone: 2020 pages, LIFO batch:0
(XEN) mm.c:940:d10 Error getting mfn 1888 (pfn 1e3e48) from L1 entry 
8000000001888465 for l1e_owner=10, pg_owner=10
(XEN) mm.c:5049:d10 ptwr_emulate: could not get_page_from_l1e()
[    0.000000] BUG: unable to handle kernel NULL pointer dereference at         
  (null)
[    0.000000] IP: [<ffffffff8103a930>] xen_set_pte+0x20/0xe0
[    0.000000] PGD 0 
[    0.000000] Oops: 0003 [#1] PREEMPT SMP 
[    0.000000] CPU 0 
[    0.000000] Modules linked in:
[    0.000000] 
[    0.000000] Pid: 0, comm: swapper Not tainted 3.0.0-rc1-00169-gae7bd11 #1  
[    0.000000] RIP: e030:[<ffffffff8103a930>]  [<ffffffff8103a930>] 
xen_set_pte+0x20/0xe0
[    0.000000] RSP: e02b:ffffffff81801df8  EFLAGS: 00010097
[    0.000000] RAX: 0000000000000000 RBX: ffff88000193dff8 RCX: ffffffffff5ff000
[    0.000000] RDX: 0000000010000001 RSI: 8000000001888465 RDI: ffff88000193dff8
[    0.000000] RBP: ffffffff81801e18 R08: 0000000000000000 R09: 0000000000007ff0
[    0.000000] R10: aaaaaaaaaaaaaaaa R11: aaaaaaaaaaaaaaaa R12: 8000000001888465
[    0.000000] R13: 000000000e573000 R14: 0000000080000000 R15: 0000000000000000
[    0.000000] FS:  0000000000000000(0000) GS:ffffffff81889000(0000) 
knlGS:0000000000000000
[    0.000000] CS:  e033 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.000000] CR2: 0000000000000000 CR3: 0000000001803000 CR4: 0000000000000660
[    0.000000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.000000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    0.000000] Process swapper (pid: 0, threadinfo ffffffff81800000, task 
ffffffff8180b020)
[    0.000000] Stack:
[    0.000000]  ffffffffff5ff000 8000000001888465 ffffffffff5ff000 
8000000001888465
[    0.000000]  ffffffff81801e38 ffffffff8106db53 0000000000000800 
8000000001888465
[    0.000000]  ffffffff81801e48 ffffffff8106dbc0 ffffffff81801e58 
ffffffff810720f6
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff8106db53>] set_pte_vaddr_pud+0x43/0x60
[    0.000000]  [<ffffffff8106dbc0>] set_pte_vaddr+0x50/0x70
[    0.000000]  [<ffffffff810720f6>] __native_set_fixmap+0x26/0x30
[    0.000000]  [<ffffffff810387e1>] xen_set_fixmap+0xa1/0x160
[    0.000000]  [<ffffffff818a3fa4>] map_vsyscall+0x50/0x55
[    0.000000]  [<ffffffff818a355a>] setup_arch+0xab1/0xb5d
[    0.000000]  [<ffffffff8103aa3f>] ? 
__raw_callee_save_xen_restore_fl+0x11/0x1e
[    0.000000]  [<ffffffff815a8fc5>] ? printk+0x3c/0x3e
[    0.000000]  [<ffffffff8189da0c>] start_kernel+0xd8/0x3c7
[    0.000000]  [<ffffffff8189d346>] x86_64_start_reservations+0x131/0x135
[    0.000000]  [<ffffffff818a096f>] xen_start_kernel+0x5cf/0x5d6
[    0.000000] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 48 83 ec 20 
48 89 5d f0 4c 89 65 f8 48 89 fb 49 89 f4 e8 55 ab 02 00 83 f8 01 74 10 <4c> 89 
23 48 8b 5d f0 4c 8b 65 f8 c9 c3 0f 1f 00 ff 14 25 80 5a 
[    0.000000] RIP  [<ffffffff8103a930>] xen_set_pte+0x20/0xe0
[    0.000000]  RSP <ffffffff81801df8>
[    0.000000] CR2: 0000000000000000
[    0.000000] ---[ end trace a7919e7f17c0a725 ]---
[    0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
[    0.000000] Pid: 0, comm: swapper Tainted: G      D     
3.0.0-rc1-00169-gae7bd11 #1
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff815a8e72>] panic+0x96/0x1ad
[    0.000000]  [<ffffffff8108e9a1>] do_exit+0x7e1/0x960
[    0.000000]  [<ffffffff8108ac4a>] ? kmsg_dump+0xca/0x110
[    0.000000]  [<ffffffff815ad4cb>] oops_end+0xab/0xf0
[    0.000000]  [<ffffffff8106e343>] no_context+0xf3/0x260
[    0.000000]  [<ffffffff8106e5d5>] __bad_area_nosemaphore+0x125/0x1e0
[    0.000000]  [<ffffffff8103ab8e>] ? xen_restore_fl+0x3e/0x80
[    0.000000]  [<ffffffff8106e69e>] bad_area_nosemaphore+0xe/0x10
[    0.000000]  [<ffffffff815af426>] do_page_fault+0x306/0x4e0
[    0.000000]  [<ffffffff818bedde>] ? memblock_find_region+0x45/0x7b
[    0.000000]  [<ffffffff818bedde>] ? memblock_find_region+0x45/0x7b
[    0.000000]  [<ffffffff818bf406>] ? memblock_add_region+0x7f/0x3ef
[    0.000000]  [<ffffffff818bf101>] ? memblock_init+0x79/0xbf
[    0.000000]  [<ffffffff8103ab8e>] ? xen_restore_fl+0x3e/0x80
[    0.000000]  [<ffffffff815ac885>] page_fault+0x25/0x30
[    0.000000]  [<ffffffff8103a930>] ? xen_set_pte+0x20/0xe0
[    0.000000]  [<ffffffff8103a92b>] ? xen_set_pte+0x1b/0xe0
[    0.000000]  [<ffffffff8106db53>] set_pte_vaddr_pud+0x43/0x60
[    0.000000]  [<ffffffff8106dbc0>] set_pte_vaddr+0x50/0x70
[    0.000000]  [<ffffffff810720f6>] __native_set_fixmap+0x26/0x30
[    0.000000]  [<ffffffff810387e1>] xen_set_fixmap+0xa1/0x160
[    0.000000]  [<ffffffff818a3fa4>] map_vsyscall+0x50/0x55
[    0.000000]  [<ffffffff818a355a>] setup_arch+0xab1/0xb5d
[    0.000000]  [<ffffffff8103aa3f>] ? 
__raw_callee_save_xen_restore_fl+0x11/0x1e
[    0.000000]  [<ffffffff815a8fc5>] ? printk+0x3c/0x3e
[    0.000000]  [<ffffffff8189da0c>] start_kernel+0xd8/0x3c7
[    0.000000]  [<ffffffff8189d346>] x86_64_start_reservations+0x131/0x135
[    0.000000]  [<ffffffff818a096f>] xen_start_kernel+0x5cf/0x5d6


Using git bisect (see attached bisection log) I've narrowed it down
to this commit:

commit 9fd67b4ed0714ab718f1f9bd14c344af336a6df7
Author: Andy Lutomirski <luto@xxxxxxx>
Date:   Sun Jun 5 13:50:19 2011 -0400

    x86-64: Give vvars their own page

    Move vvars out of the vsyscall page into their own page and mark
    it NX.

Please see attached .config file

The guest config is as follow:

kernel="/home/konrad/ssd/xtt/dist/common/vmlinuz"
ramdisk="/home/konrad/ssd/xtt/dist/common/initramfs.cpio.gz"
extra="console=hvc0 debug     earlyprintk=xenboot"
memory=2048
vcpus=4
name="latest"
on_crash="preserve"
vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
vfb = [ 'vnc=1, vnclisten=0.0.0.0,vncunused=1']

And I am using Xen 4.1.1 hypervisor. This Wiki: 
http://wiki.xensource.com/xenwiki/XenParavirtOps
has details on how to compile Xen, pvops, etc.

bisect-log
Description: Text document

.config
Description: Text document

early-devs
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel