WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 		  
Home Products Support Community News
 
   
 

xen-devel

[Top] [All Lists]
<prev [Date] next>
[Advanced]
 <prev [Thread] next>

RE: [Xen-devel] 3.1/2 live migration panic

from [Dan Magenheimer] [Permanent Link][Original]
To: "John Levon" <levon@xxxxxxxxxxxxxxxxx>, "Tim Deegan" <Tim.Deegan@xxxxxxxxxx>
Subject: RE: [Xen-devel] 3.1/2 live migration panic
From: "Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>
Date: Fri, 1 Feb 2008 14:19:40 -0700
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Fri, 01 Feb 2008 13:20:24 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <20080123191530.GA25831@xxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: Oracle Corporation
Reply-to: "dan.magenheimer@xxxxxxxxxx" <dan.magenheimer@xxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AchlGCaIm4mkJsfoSTqq836Dv3Z4WQ== 
Any progress on this one?  We may be seeing it too (on 3.1.3 near final),
at least the call trace looks very similar to one of the trace's that
John previously posted on this thread.

In our case, the problem occurred on an xm create after heavy usage
for >24 hours.  64-bit Xen, 32-bit dom0, AMD x86_64 x 8, if that helps.

Thanks,
Dan

(XEN) Xen call trace:
(XEN)    [<ffff828c8016a02f>] shadow_set_l1e+0x2f/0x1b0
(XEN)    [<ffff828c8016e5d8>] sh_page_fault__shadow_4_guest_4+0x8e8/0xec0
(XEN)    [<ffff828c80169699>] sh_make_shadow+0x479/0x4b0
(XEN)    [<ffff828c8016d459>] sh_update_cr3__shadow_4_guest_4+0x409/0x510
(XEN)    [<ffff828c80166f85>] shadow_update_paging_modes+0x95/0xd0
(XEN)    [<ffff828c8015906f>] svm_cr_access+0xecf/0xf50
(XEN)    [<ffff828c8015509c>] get_effective_addr_modrm64+0x13c/0x3d0
(XEN)    [<ffff828c8014b1d0>] hvm_io_assist+0xe30/0xe60
(XEN)    [<ffff828c80146297>] hvm_do_resume+0x27/0x150
(XEN)    [<ffff828c80151ff6>] vlapic_has_interrupt+0x26/0x60
(XEN)    [<ffff828c801595c8>] svm_vmexit_handler+0x4d8/0x15f0
(XEN)    [<ffff828c80114676>] vcpu_periodic_timer_work+0x16/0x80
(XEN)    [<ffff828c80151f46>] vlapic_get_ppr+0x26/0xb0
(XEN)    [<ffff828c8014b4d4>] is_isa_irq_masked+0x34/0x90
(XEN)    [<ffff828c80151ff6>] vlapic_has_interrupt+0x26/0x60
(XEN)    [<ffff828c8014b5ac>] cpu_has_pending_irq+0x2c/0x60
(XEN)    [<ffff828c8015b08a>] svm_stgi_label+0x8/0xe

(more crash dump data if needed)

> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]On Behalf Of John Levon
> Sent: Wednesday, January 23, 2008 12:16 PM
> To: Tim Deegan
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-devel] 3.1/2 live migration panic
> 
> 
> On Tue, Jan 22, 2008 at 09:45:41AM +0000, Tim Deegan wrote:
> 
> > Argh.  Well, here's more debugging, since you seem to hit 
> the _l1e case
> > more often.  This patch includes the previous two as well.
> 
> See below. I also saw "Can't see the l1e" version as well.
> 
> cheers
> john
> 
> (XEN) sh error: shadow_get_and_create_l1e(): Can't see the 
> l2e, even with TLB flushPagetable walk from ffff81c0ffc06928:
> (XEN)  L4[0x103] = 00000001d2f4d063 000000000007dd4e
> (XEN)  L3[0x103] = 00000001d2f4d063 000000000007dd4e
> (XEN)  L2[0x1fe] = 00000001f73ca067 000000000007dc91
> (XEN)  L1[0x006] = 0000000000000000 ffffffffffffffff
> (XEN) Pagetable walk from ffffff01a4a6e8f0:
> (XEN)  L4[0x1fe] = 00000001f73ca067 000000000007dc91
> (XEN)  L3[0x006] = 0000000000000000 ffffffffffffffff
> (XEN) v->arch.shadow_table[0] == 0x1d2f4d
> (XEN) CR3 = 0x1d2f4d000
> (XEN) Xen WARN at multi.c:1910
> (XEN) ----[ Xen-3.1.2-xvm  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    2
> (XEN) RIP:    e008:[<ffff828c801b3ca0>] 
> shadow_get_and_create_l1e+0x147/0x46c
> (XEN) RFLAGS: 0000000000010286   CONTEXT: hypervisor
> (XEN) rax: ffff828c802035e4   rbx: ffff8300e3daa100   rcx: 
> 0000000000000008
> (XEN) rdx: ffff828c8027dbf2   rsi: 000000000000000a   rdi: 
> ffff828c802035e4
> (XEN) rbp: ffff8300e2e0fc38   rsp: ffff8300e2e0fb98   r8:  
> 00000000ffffffff
> (XEN) r9:  00000000ffffffff   r10: ffff828c8027dfdf   r11: 
> ffff828c8027dbe6
> (XEN) r12: ffffff01a4a6e8b8   r13: ffffff01a48594c0   r14: 
> ffffff01a4859480
> (XEN) r15: ffffff0146e28608   cr0: 000000008005003b   cr4: 
> 00000000000006f0
> (XEN) cr3: 00000001d2f4d000   cr2: ffff81c0ffc06928
> (XEN) ds: 004b   es: 004b   fs: 0000   gs: 01c3   ss: 0000   cs: e008
> (XEN) Xen stack trace from rsp=ffff8300e2e0fb98:
> (XEN)    00000020e2e0fbc0 ffff8300e2e0fbc0 ffff8300e2e0fbc8
> (XEN)    ffff828c8015ba1d ffffffffffffffff 0000000000000000
> (XEN)    ffff8300e2e0fc38 ffff8300e2e0fbf8 0000000000000008
> (XEN)    ffff81c0ffc06928 0000000000000008 0000000000000008
> (XEN)    0000000000000000 ffff81c0ffc06928 000000000015d83e
> (XEN)    ffff8300e3dab280 00000006e2eda100 ffff8300e2e0fda8
> (XEN)    ffff8300e2e0fdd8 ffff8300e2eca100 ffff8300e2e0fe58
> (XEN)    ffff828c801b5da0 000000fc00000000 0000000800000002
> (XEN)    0000000000000044 ffff8301c1e7cab8 00000000001c1e7c
> (XEN)    00000000001c60c1 ffff8300e2e0fd98 0000000000000006
> (XEN)    0000000100000006 000000008015b93f 00000001c60c1065
> (XEN)    0000000000000000 ffff8300e2e0fc98 ffff81ff80a5bab8
> (XEN)    0000000000000008 0000000000000000 ffff8300e2e0fd20
> (XEN)    00000006e2e0fd20 0000000100000000 ffff8300e2e0fd20
> (XEN)    ffff8300e2e0fd08 ffff828c8015b5f9 000000208021b300
> (XEN)    0000000000000000 0000000000000004 ffff8300e2e0fe90
> (XEN)    ffffff000414e4d0 0000000400000020 ffff8300e2e0fe8c
> (XEN)    ffffff000414e4cc ffff8300e2e0fd88 ffff828c801668a3
> (XEN)    ffff8300e2e0fd68 0000000000000000 0000000000000004
> (XEN)    ffff8300e2e0fe8c ffffff000414e4cc 000000008023f4c0
> (XEN) Xen call trace:
> (XEN)    [<ffff828c801b3ca0>] shadow_get_and_create_l1e+0x147/0x46c
> (XEN)    [<ffff828c801b5da0>] 
> sh_page_fault__shadow_4_guest_4+0x598/0xce7
> (XEN)    [<ffff828c8016234f>] paging_fault+0x3c/0x3e
> (XEN)    [<ffff828c801622f9>] fixup_page_fault+0x22b/0x245
> (XEN)    [<ffff828c80162391>] do_page_fault+0x40/0x15c
> (XEN)
> (XEN) ----[ Xen-3.1.2-xvm  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    2
> (XEN) RIP:    e008:[<ffff828c801b3cb3>] 
> shadow_get_and_create_l1e+0x15a/0x46c
> (XEN) RFLAGS: 0000000000010286   CONTEXT: hypervisor
> (XEN) rax: ffff81c0ffc06928   rbx: ffff8300e3daa100   rcx: 
> 0000000000000008
> (XEN) rdx: ffff828c8027dbf2   rsi: 000000000000000a   rdi: 
> ffff828c802035e4
> (XEN) rbp: ffff8300e2e0fc38   rsp: ffff8300e2e0fb98   r8:  
> 00000000ffffffff
> (XEN) r9:  00000000ffffffff   r10: ffff828c8027dfdf   r11: 
> ffff828c8027dbe6
> (XEN) r12: ffffff01a4a6e8b8   r13: ffffff01a48594c0   r14: 
> ffffff01a4859480
> (XEN) r15: ffffff0146e28608   cr0: 000000008005003b   cr4: 
> 00000000000006f0
> (XEN) cr3: 00000001d2f4d000   cr2: ffff81c0ffc06928
> (XEN) ds: 004b   es: 004b   fs: 0000   gs: 01c3   ss: 0000   cs: e008
> (XEN) Xen stack trace from rsp=ffff8300e2e0fb98:
> (XEN)    00000020e2e0fbc0 ffff8300e2e0fbc0 ffff8300e2e0fbc8
> (XEN)    ffff828c8015ba1d ffffffffffffffff 0000000000000000
> (XEN)    ffff8300e2e0fc38 ffff8300e2e0fbf8 0000000000000008
> (XEN)    ffff81c0ffc06928 0000000000000008 0000000000000008
> (XEN)    0000000000000000 ffff81c0ffc06928 000000000015d83e
> (XEN)    ffff8300e3dab280 00000006e2eda100 ffff8300e2e0fda8
> (XEN)    ffff8300e2e0fdd8 ffff8300e2eca100 ffff8300e2e0fe58
> (XEN)    ffff828c801b5da0 000000fc00000000 0000000800000002
> (XEN)    0000000000000044 ffff8301c1e7cab8 00000000001c1e7c
> (XEN)    00000000001c60c1 ffff8300e2e0fd98 0000000000000006
> (XEN)    0000000100000006 000000008015b93f 00000001c60c1065
> (XEN)    0000000000000000 ffff8300e2e0fc98 ffff81ff80a5bab8
> (XEN)    0000000000000008 0000000000000000 ffff8300e2e0fd20
> (XEN)    00000006e2e0fd20 0000000100000000 ffff8300e2e0fd20
> (XEN)    ffff8300e2e0fd08 ffff828c8015b5f9 000000208021b300
> (XEN)    0000000000000000 0000000000000004 ffff8300e2e0fe90
> (XEN)    ffffff000414e4d0 0000000400000020 ffff8300e2e0fe8c
> (XEN)    ffffff000414e4cc ffff8300e2e0fd88 ffff828c801668a3
> (XEN)    ffff8300e2e0fd68 0000000000000000 0000000000000004
> (XEN)    ffff8300e2e0fe8c ffffff000414e4cc 000000008023f4c0
> (XEN) Xen call trace:
> (XEN)    [<ffff828c801b3cb3>] shadow_get_and_create_l1e+0x15a/0x46c
> (XEN)    [<ffff828c801b5da0>] 
> sh_page_fault__shadow_4_guest_4+0x598/0xce7
> (XEN)    [<ffff828c8016234f>] paging_fault+0x3c/0x3e
> (XEN)    [<ffff828c801622f9>] fixup_page_fault+0x22b/0x245
> (XEN)    [<ffff828c80162391>] do_page_fault+0x40/0x15c
> (XEN)
> (XEN) Pagetable walk from ffff81c0ffc06928:
> (XEN)  L4[0x103] = 00000001d2f4d063 000000000007dd4e
> (XEN)  L3[0x103] = 00000001d2f4d063 000000000007dd4e
> (XEN)  L2[0x1fe] = 00000001f73ca067 000000000007dc91
> (XEN)  L1[0x006] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 2:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0000]
> (XEN) Faulting linear address: ffff81c0ffc06928
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
Previous by Date:  [Xen-devel] Updated dom0 for pvops: now boots on 32-bit!, Stephen C. Tweedie
Next by Date:  Re: [Xen-devel] 3.1/2 live migration panic, John Levon
Previous by Thread:  [Xen-devel] Updated dom0 for pvops: now boots on 32-bit!, Stephen C. Tweedie
Next by Thread:  Re: [Xen-devel] 3.1/2 live migration panic, John Levon
Indexes:  [Date] [Thread] [Top] [All Lists]
 
   

Copyright , Citrix Systems Inc. All rights reserved. Legal and Privacy