WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] [PATCH] XML-RPC: Cope with large integers on x86-64 systems

To: xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] [PATCH] XML-RPC: Cope with large integers on x86-64 systems
From: Anthony Liguori <aliguori@xxxxxxxxxx>
Date: Mon, 27 Mar 2006 20:29:13 -0600
Cc: Ewan Mellor <ewan@xxxxxxxxxxxxx>
Delivery-date: Tue, 28 Mar 2006 02:30:54 +0000
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mail/News 1.5 (X11/20060309)
Attached patch fixes a problem in marshaling large integers. It seems to only occur on systems with very large memory. However, on those systems, it's a show-stopper. I see this as a temporary solution until we can sanitize the XML-RPC functions and get rid of all the S-Expression stuff.

I'll run xm-test tomorrow against it. Right now I've only done basic testing.

Regards,

Anthony Liguori
# HG changeset patch
# User anthony@xxxxxxxxxxxxxxxxxxxxx
# Node ID 8dc8c7bdcbac0e56c606fe5c9205b4d9742e60d7
# Parent  8b5a752167a17785b998a080a5c1ce1991b9379b
In some cases, Xend can return a very large integer.  This so far only happens
on 64bit systems where the sizeof the Python integer is > 32bit.  Presumably,
this is occuring because of the fact that we return PFNs as part of the
domain info so on a system with greater than 2G of memory we get into the
invalid integer range.

For now, the work-around is to overload the xmlrpclib Marshaller and instead
of throwing an exception when we get out-of-range integers, we marshal them
as strings.  In the future, we can handle this in a more elegant way by
introducing higher-level types.  We also won't have to deal with things like
PFNs once we clean up the XML-RPC interface.

Signed-off-by: Anthony Liguori <aliguori@xxxxxxxxxx>

diff -r 8b5a752167a1 -r 8dc8c7bdcbac tools/python/xen/util/xmlrpclib2.py
--- a/tools/python/xen/util/xmlrpclib2.py       Mon Mar 27 10:16:36 2006
+++ b/tools/python/xen/util/xmlrpclib2.py       Mon Mar 27 23:40:31 2006
@@ -21,9 +21,10 @@
 """
 
 from httplib import HTTPConnection, HTTP
-from xmlrpclib import Transport
+from xmlrpclib import Transport, Fault
 from SimpleXMLRPCServer import SimpleXMLRPCServer, SimpleXMLRPCRequestHandler
-import xmlrpclib, socket, os, traceback
+import xmlrpclib, socket, os, traceback, string
+from types import *
 
 # A new ServerProxy that also supports httpu urls.  An http URL comes in the
 # form:
@@ -31,6 +32,9 @@
 # httpu:///absolute/path/to/socket.sock
 #
 # It assumes that the RPC handler is /RPC2.  This probably needs to be improved
+
+MAXINT =  2L**31-1
+MININT = -2L**31
 
 class HTTPUnixConnection(HTTPConnection):
     def connect(self):
@@ -58,6 +62,67 @@
         xmlrpclib.ServerProxy.__init__(self, uri, transport, encoding,
                                        verbose, allow_none)
 
+class Marshaller(xmlrpclib.Marshaller):
+    def __init__(self, encoding=None, allow_none=0):
+        xmlrpclib.Marshaller.__init__(self, encoding, allow_none)
+        self.dispatch[IntType] = Marshaller.dump_int
+        self.dispatch[LongType] = Marshaller.dump_long
+        
+    def dump_int(self, value, write):
+        if value > MAXINT or value < MININT:
+            self.dispatch[StringType](self, str(value), write)
+        else:
+            xmlrpclib.Marshaller.dump_int(self, value, write)
+
+    def dump_long(self, value, write):
+        if value > MAXINT or value < MININT:
+            self.dispatch[StringType](self, str(value), write)
+        else:
+            xmlrpclib.Marshaller.dump_long(self, value, write)
+
+def dumps(params, methodname=None, methodresponse=None, encoding=None,
+          allow_none=0):
+    assert isinstance(params, TupleType) or isinstance(params, Fault),\
+           "argument must be tuple or Fault instance"
+
+    if isinstance(params, Fault):
+        methodresponse = 1
+    elif methodresponse and isinstance(params, TupleType):
+        assert len(params) == 1, "response tuple must be a singleton"
+
+    if not encoding:
+        encoding = "utf-8"
+
+    m = Marshaller(encoding, allow_none)
+
+    data = m.dumps(params)
+
+    if encoding != "utf-8":
+        xmlheader = "<?xml version='1.0' encoding='%s'?>\n" % str(encoding)
+    else:
+        xmlheader = "<?xml version='1.0'?>\n" # utf-8 is default
+
+    if methodname:
+        if not isinstance(methodname, StringType):
+            methodname = methodname.encode(encoding)
+        data = (
+            xmlheader,
+            "<methodCall>\n"
+            "<methodName>", methodname, "</methodName>\n",
+            data,
+            "</methodCall>\n"
+            )
+    elif methodresponse:
+        data = (
+            xmlheader,
+            "<methodResponse>\n",
+            data,
+            "</methodResponse>\n"
+            )
+    else:
+        return data
+    return string.join(data, "")
+
 # This is a base XML-RPC server for TCP.  It sets allow_reuse_address to
 # true, and has an improved marshaller that serializes unknown exceptions
 # with full traceback information.
@@ -74,13 +139,13 @@
                 response = self._dispatch(method, params)
 
             response = (response,)
-            response = xmlrpclib.dumps(response,
-                                       methodresponse=1,
-                                       allow_none=1)
+            response = dumps(response,
+                             methodresponse=1,
+                             allow_none=1)
         except xmlrpclib.Fault, fault:
-            response = xmlrpclib.dumps(fault)
+            response = dumps(fault)
         except:
-            response = xmlrpclib.dumps(
+            response = dumps(
                 xmlrpclib.Fault(1, traceback.format_exc())
                 )
 
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel