WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] unable to open xenstore issue

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] unable to open xenstore issue
From: AP Xen <apxeng@xxxxxxxxx>
Date: Thu, 3 Jun 2010 17:51:55 -0700
Delivery-date: Thu, 03 Jun 2010 17:52:50 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type:content-transfer-encoding; bh=ZFd31Xp1aZe+WgunYIPBK+dv2XWVJ0Ldg9biLF7lW7U=; b=a0JBYkI5oVhJVrcEOMOzff2QiN8IJYsiDsVtmvJ52zQi08yeh7O2lFclsHvbL4548x qVmv6PIr0wTRnS+u/NM9DbUsrTrk/b9zbfsiHjS763X/R3jLmDp8LM9ZcUU/3EY/VOBd a4oqFfHu7hxWzsTLupUYeaPKWKBozlMgFi3yc=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=ODLZrQGYJTFwUs8mdtVNlS8wfl9/FRe8nuPMWC04OjF6wxpnPIIUwnuV9CC91RqfiI 3qk3t/1Iq0Ltu9mpgg+Gh5hX3J/9BaQ/maUmfm+u1IloFiE6BU1z7ht6vKUUAamIykwP tEMD3VUdjISGmyLC6vITAJ94nWNo7tuDZTXSE=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Hi,

I am running SLES 10 SP2 with the following Xen packaged with it:
xen_major              : 3
xen_minor              : 2
xen_extra              : .0_16718_16-0.3

After a YEAR of uptime, my server is acting up.

xm commands are not working. xm list stays there for ever.
xm top returns
unable to open xenstore
:Connection refuse

xm dmesg shows the following output:
(XEN) Freed 100kB init memory.
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) instrlen.c:252:d303 Cannot read from address 77b36c4a (eip
77b36c4a, mode 8)
(XEN) sh error: sh_remove_write_access(): can't remove write access to
mfn 4d60a8: guest has 1 special-use mappings of it
(XEN) domain_crash called from common.c:1850
(XEN) Domain 1129 (vcpu#0) crashed on cpu#2:
(XEN) ----[ Xen-3.2.0_16718_16-0.3  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    2
(XEN) RIP:    0008:[<000000008fb5918f>]
(XEN) RFLAGS: 0000000000010286   CONTEXT: hvm
(XEN) rax: 00000000ffacc008   rbx: 0000000000000000   rcx: 0000000083a08200
(XEN) rdx: 0000000000000000   rsi: 000000008187eca0   rdi: 0000000000000001
(XEN) rbp: 000000008a0c0cf8   rsp: 000000008a0c0cdc   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 000000008001003b   cr4: 00000000000006b9
(XEN) cr3: 000000001f3450a0   cr2: 000000009746300f
(XEN) ds: 0023   es: 0023   fs: 0030   gs: 0000   ss: 0010   cs: 0008
(XEN) printk: 164 messages suppressed.
(XEN) grant_table.c:340:d0 Iomem mapping not permitted
ffffffffffffffff (domain 1202)
(XEN) grant_table.c:340:d0 Iomem mapping not permitted
ffffffffffffffff (domain 1202)
(XEN) grant_table.c:340:d0 Iomem mapping not permitted
ffffffffffffffff (domain 1202)
(XEN) grant_table.c:340:d0 Iomem mapping not permitted
ffffffffffffffff (domain 1202)
(XEN) grant_table.c:340:d0 Iomem mapping not permitted
ffffffffffffffff (domain 1202)
(XEN) grant_table.c:340:d0 Iomem mapping not permitted
ffffffffffffffff (domain 1311)
(XEN) grant_table.c:340:d0 Iomem mapping not permitted
ffffffffffffffff (domain 1311)
(XEN) grant_table.c:340:d0 Iomem mapping not permitted
ffffffffffffffff (domain 1311)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)
(XEN) instrlen.c:252:d1670 Cannot read from address 7fefd8215f4 (eip
7fefd8215f4, mode 8)

All my domains have crashed but I am not able to find info about the
nature of the crash other than for the above info.

The xend.log does not show errors. However, I see some errors in
xend-debug.log. For some of the domains I see the following:

Traceback (most recent call last):
  File "/usr/lib64/python2.4/site-packages/xen/web/httpserver.py",
line 140, in process
    resource = self.getResource()
  File "/usr/lib64/python2.4/site-packages/xen/web/httpserver.py",
line 172, in getResource
    return self.getServer().getResource(self)
  File "/usr/lib64/python2.4/site-packages/xen/web/httpserver.py",
line 351, in getResource
    return self.root.getRequestResource(req)
  File "/usr/lib64/python2.4/site-packages/xen/web/resource.py", line
39, in getRequestResource
    return findResource(self, req)
  File "/usr/lib64/python2.4/site-packages/xen/web/resource.py", line
26, in findResource
    next = resource.getPathResource(pathElement, request)
  File "/usr/lib64/python2.4/site-packages/xen/web/resource.py", line
49, in getPathResource
    val = self.getChild(path, request)
  File "/usr/lib64/python2.4/site-packages/xen/web/SrvDir.py", line
71, in getChild
    val = self.get(x)
  File "/usr/lib64/python2.4/site-packages/xen/xend/server/SrvDomainDir.py",
line 52, in get
    return self.domain(x)
  File "/usr/lib64/python2.4/site-packages/xen/xend/server/SrvDomainDir.py",
line 44, in domain
    dom = self.xd.domain_lookup(x)
  File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomain.py",
line 525, in domain_lookup
    raise XendInvalidDomain(str(domid))
XendInvalidDomain: <Fault 3: '549'>

Not sure why the domain was flagged as invalid. Doing a create on it
again went through.

There also a ton of these messages:
Traceback (most recent call last):
  File "/usr/lib64/python2.4/SocketServer.py", line 463, in
process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib64/python2.4/SocketServer.py", line 254, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib64/python2.4/site-packages/xen/util/xmlrpclib2.py",
line 105, in <lambda>
    (lambda x, y, z:
  File "/usr/lib64/python2.4/site-packages/xen/util/xmlrpclib2.py",
line 65, in __init__
    server)
  File "/usr/lib64/python2.4/SocketServer.py", line 521, in __init__
    self.handle()
  File "/usr/lib64/python2.4/BaseHTTPServer.py", line 316, in handle
    self.handle_one_request()
  File "/usr/lib64/python2.4/BaseHTTPServer.py", line 310, in handle_one_request
    method()
  File "/usr/lib64/python2.4/site-packages/xen/util/xmlrpclib2.py",
line 82, in do_POST
    self.send_response(200)
  File "/usr/lib64/python2.4/BaseHTTPServer.py", line 367, in send_response
    self.wfile.write("%s %d %s\r\n" %
  File "/usr/lib64/python2.4/socket.py", line 256, in write
    self.flush()
  File "/usr/lib64/python2.4/socket.py", line 243, in flush
    self._sock.sendall(buffer)
error: (32, 'Broken pipe')

At this point I was forced to reboot the server to get it back and
running. I am not seeing any of the errors as of yet but I am afraid
it will show up after heavy activity of creating and destroying VMs.
Is this is a known bug that has been fixed in upstream Xen?

Thanks,
Aravindh

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-devel] unable to open xenstore issue, AP Xen <=