[Xen-devel] RE: [PATCH] mem_sharing: fix race condition of nominate and unshare

[MaoXiaoyun <tinnycloud@xxxxxxxxxxx>]

Another failure on BUG() in mem_sharing_alloc_page()
 
 memset(&req, 0, sizeof(req));
 if(must_succeed)
    {
        /* We do not support 'must_succeed' any more. External operations such
         * as grant table mappings may fail with OOM condition!
         */
        BUG();===================>bug here
    }
    else
    {
        /* All foreign attempts to unshare pages should be handled through
         * 'must_succeed' case. */
        ASSERT(v->domain->domain_id == d->domain_id);
        vcpu_pause_nosync(v);
        req.flags |= MEM_EVENT_FLAG_VCPU_PAUSED;
 &nbs p;  }
       
log below:

(XEN) Xen BUG at mem_sharing.c:347
(XEN) ----[ Xen-4.0.0  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82c4801c0344>] mem_sharing_unshare_page+0x5d4/0x6e0
(XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: ffff83030913eb78   rcx: 0000000000000000
(XEN) rdx: ffff82f6094d78e0   rsi: 0000000000000001   rdi: ffff82c48035f650
(XEN) rbp: ffff83048d950000   rsp: ffff82c48035f5e8   r8:  0000000000000000
(XEN) r9:  ffffffffffffffff   r10: 0000000000000001   r11: 0000000000000002
(XEN) r12: 000000000000ecbf   r13: ffff82c48035f628   r14: ffff83033f4fd760
(XEN) r15: ffff82f6025540c0   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 0000000339044000   cr2: ffff8801589fac78
(X EN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff82c48035f5e8:
(XEN)    ffff880159dcf000 ffff83033f4fd770 0000000000000000 ffff83030913eb60
(XEN)    0000000000235012 ffff8300bf554000 0000000100000009 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000d000000bf ffff83059b097000 000000000000ecbf
(XEN)    000000000012aa06 ffff82c48035f724 ffff8304f20d2a00 ffff8800b742bbc0
(XEN)    ffff83048d950000 ffff82c48010bfa9 ffff8304fca169b0 000000000053ea75
(XEN)    0000000000000000 ffff8304fca169b0 0000000000000000 ffff82c48035f6f8
(XEN)    0000000000000001 ffff8304fca169b0 ffff83023fe60000 ffff8304fca169b0
(XEN)    000002fd8035ff28 ffff8800b742bbc0 ffff880159dcf000 00003d3600000002
(XEN)    ffffffffffff003c 000000048cb88000 ffff82f60a7d4ea0 0000000d8010a650
(XEN)    0000000000000100 ffff83023fe60000 ffff83057cdbd8c0 ffffffffffffffea
(XEN)    ffff8800b742bb70 0000000000000000 ffff8800b742bbf0 ffff8800b742bbc0
(XEN)    0000000000000001 ffff82c48010da9b 0000000000000202 ffff82c48035fec8
(XEN)    ffff82c48035f7c8 00000000801880af ffff83023fe60010 0000000000000000
(XEN)    ffff82c400000001 ffff82c48035ff28 0000000000000008 ffff8800b742bbc0
(XEN)    ffff880159dcf000 0000000000000000 0000000000000000 00020000000002fd
(XEN)    000000000053ea75 ffff83021bdc47e8 ffff830310a20000 00000003370af718
(XEN)    0000000000000000 0000000000000000 001a000000000096 000000000053ea75
(XEN)    ffff83023fe514b0 ffff830310a20000 ffff880159d51000 0000000000000000
(X EN)    0000000000000000 00020000000005b6 0000000000509bb5 ffff83031884fdb0
(XEN) Xen call trace:
(XEN)    [<ffff82c4801c0344>] mem_sharing_unshare_page+0x5d4/0x6e0
(XEN)    [<ffff82c48010bfa9>] gnttab_map_grant_ref+0xbf9/0xe30
(XEN)    [<ffff82c48010da9b>] do_grant_table_op+0x14b/0x1080
(XEN)    [<ffff82c48015dc6b>] get_page_type+0xb/0x20
(XEN)    [<ffff82c48015df6d>] get_page_from_l1e+0x2ed/0x4c0
(XEN)    [<ffff82c480159ce9>] is_iomem_page+0x9/0x70
(XEN)    [<ffff82c48015b999>] put_page_from_l1e+0x59/0x150
(XEN)    [<ffff82c48015f6c5>] ptwr_emulated_update+0x2e5/0x420
(XEN)    [<ffff82c48015f946>] ptwr_emulated_write+0x86/0x90
(XEN)    [<ffff82c480159cb5>] ptwr_emulated_read+0x15/0x40
(XEN)    [<ffff8 2c48017511d>] x86_emulate+0x7ad/0x115d0
(XEN)    [<ffff82c48010fb44>] do_xen_version+0xb4/0x480
(XEN)    [<ffff82c4801447da>] __find_next_bit+0x6a/0x70
(XEN)    [<ffff82c48011c519>] cpumask_raise_softirq+0x89/0xa0
(XEN)    [<ffff82c4801872fc>] reprogram_hpet_evt_channel+0x8c/0x110
(XEN)    [<ffff82c4801880af>] handle_hpet_broadcast+0x16f/0x1d0
(XEN)    [<ffff82c4801447da>] __find_next_bit+0x6a/0x70
(XEN)    [<ffff82c480187622>] hpet_legacy_irq_tick+0x42/0x50
(XEN)    [<ffff82c48011c519>] cpumask_raise_softirq+0x89/0xa0
(XEN)    [<ffff82c4801478e2>] update_runstate_area+0x102/0x110
(XEN)    [<ffff82c480118351>] csched_vcpu_wake+0x101/0x1b0
(XEN)    [<ffff82c4801447da>] __find_next_bit+0x6a/0x70
(XEN)     [<ffff82c48015a1d8>] get_page+0x28/0xf0
(XEN)    [<ffff82c48015ed72>] do_update_descriptor+0x1d2/0x210
(XEN)    [<ffff82c480113d7e>] do_multicall+0x14e/0x340
(XEN)    [<ffff82c4801e3169>] syscall_enter+0xa9/0xae
(XEN)   
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Xen BUG at mem_sharing.c:347
(XEN) ****************************************
(XEN)
(XEN) Manual reset required ('noreboot' specified)
 
> Date: Sat, 15 Jan 2011 01:00:27 +0800
> Subject: Re: [PATCH] mem_sharing: fix race condition of nominate and unshare
> From: juihaochiang@xxxxxxxxx
> To: tinnycloud@xxxxxxxxxxx
> CC: xen-devel@xxxxxxxxxxxxxxxxxxx; tim.deegan@xxxxxxxxxx
>
> Hi, all:
>
> Is that possible that the domain is dying?
> In mem_sharing_gfn_account(): could you try the following?
>
> d = get_domain_by_id(gfn->domain);
> if (!d) printk("null domain %x\n", gfn->domain); /* add this line to
> see which domain id you see */
> BUG_ON(!d);
>
> When this domain id printed out, could you check if the printed domain
> id is dying?
> If the domain is dying, then the question seems to be:
> "Given a domain id from the gfn_info, how do we know the domain is
> dying? or we have stored a wrong information inside the hash list?"
>
> 2011/1/14 MaoXiaoyun <tinnycloud@ho tmail.com>:
> > Hi Tim:
> >
> >      Thanks for the patch, xen panic on more stressed test. ( 12HVMS, each
> > of them reboot every 30minutes).
> >      Please refer to below log.
> >
> > blktap_sysfs_create: adding attributes for dev ffff8801044bc400
> > blktap_sysfs_create: adding attributes for dev ffff8801044bc200
> > __ratelimit: 4 callbacks suppressed
> > (XEN) Xen BUG at mem_sharing.c:454
> > (XEN) ----[ Xen-4.0.0  x86_64  debug=n  Not tainted ]----
> > (XEN) CPU:    0
> > (XEN) RIP:    e008:[<ffff82c4801bf52c>] mem_sharing_gfn_account+0x5c/0x70
> > (XEN) RFLAGS: 0000000000010246   CONTEXT: hypervisor
> > (XEN) rax: 0000000000000000   rbx: 0000000000000001   rcx: 0000000000000000
> > (X EN) rdx: 0000000000000000   rsi: 000000000000005f   rdi: 000000000000005f
> > (XEN) rbp: ffff8305894f0fc0   rsp: ffff82c48035fc48   r8:  ffff82f600000000
> > (XEN) r9:  00007fffcdbd0fb8   r10: ffff82c4801f8c70   r11: 0000000000000282
> > (XEN) r12: ffff82c48035fe28   r13: ffff8303192a3bf0   r14: ffff82f60b966700
> > (XEN) r15: 0000000000000006   cr0: 0000000080050033   cr4: 00000000000026f0
> > (XEN) cr3: 000000032ea58000   cr2: ffff880119c2e668
> > (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> > (XEN) Xen stack trace from rsp=ffff82c48035fc48:
> > (XEN)    00000000fffffff7 ffff82c4801bf8c0 0000000000553b86 ffff8305894f0fc0
> > (XEN)    ffff8302f4d12cf0 0000000000553b86 ffff 82f603e28580 ffff82c48035fe38
> > (XEN)    ffff83023fe60000 ffff82c48035fe28 0000000000305000 0000000000000006
> > (XEN)    0000000000000006 ffff82c4801c0724 ffff82c4801447da 0000000000553b86
> > (XEN)    000000000001a938 00000000006ee000 00000000006ee000 ffff82c4801457fd
> > (XEN)    0000000000000096 0000000000000001 ffff82c48035fd30 0000000000000080
> > (XEN)    ffff82c480376980 ffff82c480251080 0000000000000292 ffff82c48011c519
> > (XEN)    ffff82c48035fe28 0000000000000080 0000000000000000 ffff8302ef312fa0
> > (XEN)    ffff8300b4aee000 ffff82c48025f080 ffff82c480251080 ffff82c480118351
> > (XEN)    0000000000000080 0000000000000000 ffff8300b4aef708 00000de9e9529c40
> > (XEN)    ffff8300b4aee000 0000000000000292 ffff8305cf9f09b8 0000000000000001
> > (X EN)    0000000000000001 0000000000000000 00000000002159e6 fffffffffffffff3
> > (XEN)    00000000006ee000 ffff82c48035fe28 0000000000305000 0000000000000006
> > (XEN)    0000000000000006 ffff82c480104373 ffff8305cf9f09c0 ffff82c4801a0b63
> > (XEN)    00000000159e6070 ffff8305cf9f0000 0000000000000007 ffff83023fe60180
> > (XEN)    0000000600000039 0000000000000000 00007fae14b30003 000000000054fdad
> > (XEN)    0000000000553b86 ffffffffff600429 000000004d2f26e8 0000000000088742
> > (XEN)    0000000000000000 00007fae14b30070 00007fae14b30000 00007fffcdbd0f50
> > (XEN)    00007fae14b30078 0000000000430e98 00007fffcdbd0fb8 0000000000cd39c8
> > (XEN)    0005aeb700000007 00007fae15bd2ab0 0000000000000000 0000000000000246
> > (XEN) Xen call trace:
> > (XEN) & nbsp;  [<ffff82c4801bf52c>] mem_sharing_gfn_account+0x5c/0x70
> > (XEN)    [<ffff82c4801bf8c0>] mem_sharing_share_pages+0x170/0x310
> > (XEN)    [<ffff82c4801c0724>] mem_sharing_domctl+0xe4/0x130
> > (XEN)    [<ffff82c4801447da>] __find_next_bit+0x6a/0x70
> > (XEN)    [<ffff82c4801457fd>] arch_do_domctl+0xdad/0x1f90
> > (XEN)    [<ffff82c48011c519>] cpumask_raise_softirq+0x89/0xa0
> > (XEN)    [<ffff82c480118351>] csched_vcpu_wake+0x101/0x1b0
> > (XEN)    [<ffff82c480104373>] do_domctl+0x163/0x1000
> > (XEN)    [<ffff82c4801a0b63>] hvm_set_callback_irq_level+0xe3/0x110
> > (XEN)    [<ffff82c4801e3169>] syscall_enter+0xa9/0xae
> > (XEN)
> > (XEN)
> > (XEN) ****************** **********************
> > (XEN) Panic on CPU 0:
> > (XEN) Xen BUG at mem_sharing.c:454
> > (XEN) ****************************************
> > (XEN)
> > (XEN) Manual reset required ('noreboot' specified)
> >
>
> Bests,
> Jui-Hao

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel