WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] trouble with xenserver and xfs (soft lockup - CPU#0 stuc

To: Yann Cezard <yann.cezard@xxxxxxxxxxx>
Subject: Re: [Xen-users] trouble with xenserver and xfs (soft lockup - CPU#0 stuck for 61s!)
From: Sebastian Gosenheimer <s.gosenheimer@xxxxxxxxx>
Date: Fri, 30 Apr 2010 14:45:01 +0200
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Fri, 30 Apr 2010 05:46:02 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4BD6A598.9010906@xxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <4BD572F2.5060405@xxxxxxxxx> <4BD6A598.9010906@xxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4
Am 27.04.2010 10:51, schrieb Yann Cezard:
> Sebastian Gosenheimer a écrit :
>> Hi everybody,
>>
>> i hope that i'm on the right mailing list. I'm having some trouble with
>> xfs and xen. We are running the newest xenserver version with ha on dell
>> servers and a dell equalogic.
>>
>> I set up a fileserver (debian lenny 2.6.29-xs5.5.0.17) with one ext3
>> partiton for the os and one xfs partition for the data. Two webserver
>> are using this fileserver with nfs.
>>
>> Now i run the second time into following problem when rsync was running
>> for backup:
>>
>> [212521.428003] BUG: soft lockup - CPU#0 stuck for 61s! [rsync:29921]
>> [212521.428003] Modules linked in: ipv6 nfsd nfs lockd nfs_acl
>> auth_rpcgss sunrpc xenfs xfs exportfs loop evdev xen_netfront pcspkr
>> ext3 jbd mbcache xen_blkfront thermal_sys
>> [212521.428003]
>> [212521.428003] Pid: 29921, comm: rsync Tainted: G      D
>> (2.6.29-xs5.5.0.17 #1)
>> [212521.428003] EIP: 0061:[<c02f2221>] EFLAGS: 00000287 CPU: 0
>> [212521.428003] EIP is at __write_lock_failed+0x9/0x1c
>> [212521.428003] EAX: e1875cbc EBX: c1351e1a ECX: e2407880 EDX: dd004c50
>> [212521.428003] ESI: 00000000 EDI: c536fd40 EBP: e1875cbc ESP: c1351dac
>> [212521.428003]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069
>> [212521.428003] CR0: 8005003b CR2: b806e348 CR3: 20d99000 CR4: 00002620
>> [212521.428003] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
>> [212521.428003] DR6: ffff0ff0 DR7: 00000400
>> [212521.428003] Call Trace:
>> [212521.428003]  [<c02f23a3>] ? _write_lock+0xe/0xf
>> [212521.428003]  [<e33f7409>] ? xfs_iget+0x328/0x44f [xfs]
>> [212521.428003]  [<e340e925>] ? xfs_lookup+0x69/0x97 [xfs]
>> [212521.428003]  [<e341638b>] ? xfs_vn_lookup+0x36/0x6e [xfs]
>> [212521.428003]  [<c0194ee1>] ? do_lookup+0xa6/0x116
>> [212521.428003]  [<c019584d>] ? __link_path_walk+0x524/0x631
>> [212521.428003]  [<c01057fc>] ? xen_force_evtchn_callback+0xc/0x10
>> [212521.428003]  [<c0195d5c>] ? path_walk+0x4f/0xa3
>> [212521.428003]  [<c0196aa0>] ? do_path_lookup+0x132/0x178
>> [212521.428003]  [<c019732e>] ? getname+0x5e/0xb0
>> [212521.428003]  [<c0197b05>] ? user_path_at+0x37/0x5f
>> [212521.428003]  [<c0198e80>] ? filldir64+0x0/0xc5
>> [212521.428003]  [<c019187f>] ? vfs_lstat_fd+0x12/0x3
>> [212455.928003]  [<c0191912>] ? sys_lstat64+0xf/0x23
>> [212455.928003]  [<c0199124>] ? vfs_readdir+0x7c/0x8c
>> [212455.928003]  [<c0198e80>] ? filldir64+0x0/0xc5
>> [212455.928003]  [<c01991cf>] ? sys_getdents64+0x9b/0xa5
>> [212455.928003]  [<c0107f23>] ? sysenter_past_esp+0x3c/0x62
>> [212455.928003]  [<c0107f5b>] ? sysenter_do_call+0x12/0x2f
>>
>> I have to force reboot to bring the vm back online. Are there any known
>> problems with xfs and xen? Because i know from another deployment that
>> they are having nearly the same issues with xfs. There are no problems
>> with the ext3 filesystem.
>>
>> Thank you for your help!
>>
>>
> Hi Sebastian,
>
> I'm having similar issues with two Lenny domU (amd64 and i386),
> both with XFS + quite heavy load (I should say normal load for
> what they do : database server and fileserver).
> They are running the standard Lenny Xen kernel (2.6.26-2-xen-*)
>
> After searching a little, I found this thread :
> http://developer.amazonwebservices.com/connect/thread.jspa?threadID=28968&start=0&tstart=0
>
> Not sure that it is connected to my (or your) problem, but the domUs
> I have problems with both have XFS log version 2, i have others domUs
> with XFS filesystems and quite heavy charge (mail servers basically),
> but they are still in Etch and XFS log version 1, no problem with them.
>
> So I think I will give the log version workaround a try.
>
> Perhaps could you give it a try to ?
>
> In any case, if you find something about that problem, I would be happy
> to have feedback from you.
>
> Cheers,
>

Hi Yann,

sorry for the late answer. I was messing arround the week with some 
other things.

I have now since four days up and running a new kernel on the fileserver 
(2.6.32.12-xs5.5.0.17). Since this time everything seems to be fine.

Right now i don't know if there is a xfs or nfs problem in the kernel 
2.6.26-2-xen-*. I will update all our machines to a newer kernel know 
which are using nfs and/or xfs.

 From my opionion the thread you sent isn't connected to my problem.

Did the workaround work for you or do you still have problems.

--sg


Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. 
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten 
haben, 
informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. 
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail sind nicht 
gestattet. 

This e-mail may contain confidential and/or privileged information. 
If you are not the intended recipient (or have received this e-mail in error) 
please notify 
the sender immediately and destroy this e-mail.  
Any unauthorized copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden. 



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>