WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-api

RE: [Xen-API] Frequent I/O error of HTTP server and memory error on XenS

To: 'Frank' <frank@xxxxxxxxx>, "xen-api@xxxxxxxxxxxxxxxxxxx" <xen-api@xxxxxxxxxxxxxxxxxxx>, "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-API] Frequent I/O error of HTTP server and memory error on XenServer5.6 FP1
From: Dave Scott <Dave.Scott@xxxxxxxxxxxxx>
Date: Thu, 13 Jan 2011 22:41:39 +0000
Accept-language: en-US
Acceptlanguage: en-US
Cc:
Delivery-date: Thu, 13 Jan 2011 14:41:53 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4D2F7922.7050906@xxxxxxxxx>
List-help: <mailto:xen-api-request@lists.xensource.com?subject=help>
List-id: Discussion of API issues surrounding Xen <xen-api.lists.xensource.com>
List-post: <mailto:xen-api@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-api>, <mailto:xen-api-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-api>, <mailto:xen-api-request@lists.xensource.com?subject=unsubscribe>
References: <4D2F7922.7050906@xxxxxxxxx>
Sender: xen-api-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Acuzb1ktmDp0A7tyS8yCHXZ7F+rVMwAAiE0A
Thread-topic: [Xen-API] Frequent I/O error of HTTP server and memory error on XenServer5.6 FP1
Hi,

My best guess is that the "Thread.create: Cannot allocate memory" means that 
there's insufficient virtual address space in the process to allocate a 10MiB 
thread stack. I suspect this is being caused by there being too many threads 
(approx 300 is the max)

Since xapi currently uses a thread per connection model there are probably too 
many open connections -- could you describe the software which is using the 
apache xmlrpc client? Is it possible that someone is opening connections but 
not closing them quickly enough?

If you install gdb and attach it to the running (child -- the big one) xapi 
process it'll tell you how many threads are present and allow you to examine 
the stacks of each one ('thread apply all bt') -- it should be easy to spot 
where the threads are blocked (e.g. in a network recv or perhaps on a 
pthread_mutex)

At some point in future we'll have to switch to a different threading model and 
make the connection handling more robust. In the meantime we have to be a bit 
careful about connection cleanup :/

Thanks,
Dave

> -----Original Message-----
> From: xen-api-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-api-
> bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Frank
> Sent: 13 January 2011 22:14
> To: xen-api@xxxxxxxxxxxxxxxxxxx; xen-users@xxxxxxxxxxxxxxxxxxx
> Subject: [Xen-API] Frequent I/O error of HTTP server and memory error
> on XenServer5.6 FP1
> 
> Hi,
>      I am using XenServer5.6 FP1 GA release with openvswitch as network
> backend. When I create VM through xapi,  "apache.xmlrpc.XmlRpcException:
> I/O error" happens very frequently. At meanwhile my xencenter lost
> connection to hosts.
>      Another thing is when I tried to kill VMs from xencenter, it
> complains "Internal error:Sys_error("Thread.create: Cannot allocate
> memory")", see attached picture.
>      does anyone meet similar errors?
>      Thank you.

_______________________________________________
xen-api mailing list
xen-api@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/mailman/listinfo/xen-api