WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] [Patch] adjust the cpu-affinity to more than 64 cpus

To: James Song <jsong@xxxxxxxxxx>, Masaki Kanno <kanno.masaki@xxxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-devel] [Patch] adjust the cpu-affinity to more than 64 cpus
From: "Liu, Jinsong" <jinsong.liu@xxxxxxxxx>
Date: Wed, 31 Mar 2010 02:23:47 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc:
Delivery-date: Tue, 30 Mar 2010 11:24:57 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4BA3DA11020000200007A2B7@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <27928229.post@xxxxxxxxxxxxxxx> <4BA0AE00020000780003576C@xxxxxxxxxxxxxxxxxx> <27941020.post@xxxxxxxxxxxxxxx> <27941371.post@xxxxxxxxxxxxxxx> <B9CAC712593E1Ekanno.masaki@xxxxxxxxxxxxxx> <4BA3DA11020000200007A2B7@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcrHRAeBJFdRnXIMTkO27bfXwF3qyAI8btwA
Thread-topic: [Xen-devel] [Patch] adjust the cpu-affinity to more than 64 cpus
James and Masaki,
 
The issue Masaki said does exist, it appear under ia32pae.
A patch (c/s 21087) is to fix the bug.
 
Thanks,
Jinsong


From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of James Song
Sent: Friday, March 19, 2010 5:10 PM
To: Masaki Kanno; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] [Patch] adjust the cpu-affinity to more than 64 cpus

I update upstream code to c/s 20145. This patch works fine and I haven't meet this problem.
 
linux-2j72:/home # xm vcpu-pin  0 1 0
linux-2j72:/home # xm vcpu-list 0
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0     0   ---      23.4 0
Domain-0                             0     1     0   r--      19.2 0 
 
 
B.T.W, there are something wrong when compile c/s 21046!
 
-James

>>> Masaki Kanno <kanno.masaki@xxxxxxxxxxxxxx> 2010-3-19 11:14 >>>
Hi James,

I tested xm vcpu-pin command with xen-unstable changeset 21044.
The command failed as follows.  Also Time(s) values by xm vcpu-list
command were strange.  The values were same in all VCPUs.

# xm vcpu-list
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0     1   r--     116.5 any cpu
Domain-0                             0     1     1   r--     116.5 any cpu
# xm vcpu-pin Domain-0 0 0
Error: Cannot pin vcpu: 0 to cpu: [0] - (22, 'Invalid argument')
Usage: xm vcpu-pin <Domain> <VCPU|all> <CPUs|all>

Set which CPUs a VCPU can use.
# xm vcpu-list
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0     1   r--     117.0 any cpu
Domain-0                             0     1     1   r--     117.0 any cpu


I reverted changeset 21044 and 21040, and then I tested xm vcpu-pin
command again.  The command succeeded as follows.

# xm vcpu-list
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0     0   r--      60.8 any cpu
Domain-0                             0     1     1   -b-      42.8 any cpu
# xm vcpu-pin Domain-0 0 0
# xm vcpu-list
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0     0   r--      61.6 0
Domain-0                             0     1     1   -b-      43.2 any cpu

Best regards,
Kan

Wed, 17 Mar 2010 20:41:22 -0700 (PDT), "James (song wei)" wrote:

>
>Keir, could you take a look at this issue.
>New Patch for this issue:
>Singed-off-by: James (Song Wei) <jsong@xxxxxxxxxx>
>
>diff -r 8b269215464b tools/libxc/xc_domain.c
>--- a/tools/libxc/xc_domain.cWed Mar 10 14:01:32 2010 +0800
>+++ b/tools/libxc/xc_domain.cThu Mar 18 11:37:55 2010 +0800
>@@ -105,23 +105,28 @@
> int xc_vcpu_setaffinity(int xc_handle,
>                         uint32_t domid,
>                         int vcpu,
>-                        uint64_t cpumap)
>+                        uint64_t *cpumap, int cpusize)
> {
>     DECLARE_DOMCTL;
>     int ret = -1;
>-    uint8_t local[sizeof (cpumap)];
>+    uint8_t *local = malloc(cpusize);
>
>+    if(local == NULL)
>+    {
>+        PERROR("Could not alloc memory for Xen hypercall");
>+        goto out;
>+    }
>     domctl.cmd = XEN_DOMCTL_setvcpuaffinity;
>     domctl.domain = (domid_t)domid;
>     domctl.u.vcpuaffinity.vcpu    = vcpu;
>
>-    bitmap_64_to_byte(local, &cpumap, sizeof(cpumap) * 8);
>+    bitmap_64_to_byte(local, cpumap, cpusize * 8);
>
>     set_xen_guest_handle(domctl.u.vcpuaffinity.cpumap.bitmap, local);
>
>-    domctl.u.vcpuaffinity.cpumap.nr_cpus = sizeof(cpumap) * 8;
>+    domctl.u.vcpuaffinity.cpumap.nr_cpus = cpusize * 8;
>    
>-    if ( lock_pages(local, sizeof(local)) != 0 )
>+    if ( lock_pages(local, cpusize) != 0 )
>     {
>         PERROR("Could not lock memory for Xen hypercall");
>         goto out;
>@@ -129,9 +134,10 @@
>
>     ret = do_domctl(xc_handle, &domctl);
>
>-    unlock_pages(local, sizeof(local));
>+    unlock_pages(local, cpusize);
>
>  out:
>+    free(local);
>     return ret;
> }
>
>@@ -139,18 +145,25 @@
> int xc_vcpu_getaffinity(int xc_handle,
>                         uint32_t domid,
>                         int vcpu,
>-                        uint64_t *cpumap)
>+                        uint64_t *cpumap, int cpusize)
> {
>     DECLARE_DOMCTL;
>     int ret = -1;
>-    uint8_t local[sizeof (cpumap)];
>+    uint8_t * local = malloc(cpusize);
>+
>+    if(local == NULL)
>+    {
>+        PERROR("Could not alloc memory for Xen hypercall");
>+        goto out;
>+    }
>
>     domctl.cmd = XEN_DOMCTL_getvcpuaffinity;
>     domctl.domain = (domid_t)domid;
>     domctl.u.vcpuaffinity.vcpu = vcpu;
>
>+
>     set_xen_guest_handle(domctl.u.vcpuaffinity.cpumap.bitmap, local);
>-    domctl.u.vcpuaffinity.cpumap.nr_cpus = sizeof(cpumap) * 8;
>+    domctl.u.vcpuaffinity.cpumap.nr_cpus = cpusize * 8;
>    
>     if ( lock_pages(local, sizeof(local)) != 0 )
>     {
>@@ -161,8 +174,9 @@
>     ret = do_domctl(xc_handle, &domctl);
>
>     unlock_pages(local, sizeof (local));
>-    bitmap_byte_to_64(cpumap, local, sizeof(local) * 8);
>- out:
>+    bitmap_byte_to_64(cpumap, local, cpusize * 8);
>+out:
>+    free(local);
>     return ret;
> }
>
>diff -r 8b269215464b tools/libxc/xenctrl.h
>--- a/tools/libxc/xenctrl.hWed Mar 10 14:01:32 2010 +0800
>+++ b/tools/libxc/xenctrl.hThu Mar 18 11:37:55 2010 +0800
>@@ -310,11 +310,13 @@
> int xc_vcpu_setaffinity(int xc_handle,
>                         uint32_t domid,
>                         int vcpu,
>-                        uint64_t cpumap);
>+                        uint64_t *cpumap,
>+                        int cpusize);
> int xc_vcpu_getaffinity(int xc_handle,
>                         uint32_t domid,
>                         int vcpu,
>-                        uint64_t *cpumap);
>+                        uint64_t *cpumap,
>+                        int cpusize);
>
> /**
>  * This function will return information about one or more domains. It is
>diff -r 8b269215464b tools/python/xen/lowlevel/xc/xc.c
>--- a/tools/python/xen/lowlevel/xc/xc.cWed Mar 10 14:01:32 2010 +0800
>+++ b/tools/python/xen/lowlevel/xc/xc.cThu Mar 18 11:37:55 2010 +0800
>@@ -217,8 +217,12 @@
> {
>     uint32_t dom;
>     int vcpu = 0, i;
>-    uint64_t  cpumap = ~0ULL;
>+    uint64_t  *cpumap;
>     PyObject *cpulist = NULL;
>+    int nr_cpus, size;
>+    xc_physinfo_t info;
>+    xc_cpu_to_node_t map[1];
>+    uint64_t cpumap_size = sizeof(*cpumap);
>
>     static char *kwd_list[] = { "domid", "vcpu", "cpumap", NULL };
>
>@@ -226,26 +230,38 @@
>                                       &dom, &vcpu, &cpulist) )
>         return NULL;
>
>+    set_xen_guest_handle(info.cpu_to_node, map);
>+    info.max_cpu_id = 1;
>+    if ( xc_physinfo(self->xc_handle, &info) != 0 )
>+        return pyxc_error_to_exception();
>+ 
>+    nr_cpus = info.nr_cpus;
>+
>+    size = (nr_cpus + cpumap_size * 8 - 1)/ (cpumap_size * 8);
>+    cpumap = malloc(cpumap_size * size);
>+    if(cpumap == NULL)
>+        return pyxc_error_to_exception();
>+
>     if ( (cpulist != NULL) && PyList_Check(cpulist) )
>     {
>-        cpumap = 0ULL;
>+        for ( i = 0; i < size; i++)
>+        {
>+            cpumap[i] = 0ULL;
>+        }
>         for ( i = 0; i < PyList_Size(cpulist); i++ )
>         {
>             long cpu = PyInt_AsLong(PyList_GetItem(cpulist, i));
>-            if ( cpu >= 64 )
>-            {
>-                errno = EINVAL;
>-                PyErr_SetFromErrno(xc_error_obj);
>-                return NULL;
>-            }
>-            cpumap |= (uint64_t)1 << cpu;
>+            cpumap[cpu / (cpumap_size * 8)] |= (uint64_t)1 << (cpu %
>(cpumap_size * 8));
>         }
>     }
>  
>-    if ( xc_vcpu_setaffinity(self->xc_handle, dom, vcpu, cpumap) != 0 )
>+    if ( xc_vcpu_setaffinity(self->xc_handle, dom, vcpu, cpumap, size *
>cpumap_size) != 0 )
>+    {
>+        free(cpumap);
>         return pyxc_error_to_exception();
>-   
>+    }
>     Py_INCREF(zero);
>+    free(cpumap);
>     return zero;
> }
>
>@@ -365,7 +381,11 @@
>     uint32_t dom, vcpu = 0;
>     xc_vcpuinfo_t info;
>     int rc, i;
>-    uint64_t cpumap;
>+    uint64_t *cpumap;
>+    int nr_cpus, size;
>+    xc_physinfo_t pinfo = { 0 };
>+    xc_cpu_to_node_t map[1];
>+    uint64_t cpumap_size = sizeof(*cpumap);
>
>     static char *kwd_list[] = { "domid", "vcpu", NULL };
>    
>@@ -373,12 +393,25 @@
>                                       &dom, &vcpu) )
>         return NULL;
>
>+    set_xen_guest_handle(pinfo.cpu_to_node, map);
>+    pinfo.max_cpu_id = 1;
>+    if ( xc_physinfo(self->xc_handle, &pinfo) != 0 )
>+        return pyxc_error_to_exception();
>+    nr_cpus = pinfo.nr_cpus;
>     rc = xc_vcpu_getinfo(self->xc_handle, dom, vcpu, &info);
>     if ( rc < 0 )
>         return pyxc_error_to_exception();
>-    rc = xc_vcpu_getaffinity(self->xc_handle, dom, vcpu, &cpumap);
>+    size = (nr_cpus + cpumap_size * 8 - 1)/ (cpumap_size * 8);
>+
>+    if((cpumap = malloc(cpumap_size * size)) == NULL)
>+        return pyxc_error_to_exception();
>+
>+    rc = xc_vcpu_getaffinity(self->xc_handle, dom, vcpu, cpumap,
>cpumap_size * size);
>     if ( rc < 0 )
>+    {
>+        free(cpumap);
>         return pyxc_error_to_exception();
>+    }
>
>     info_dict = Py_BuildValue("{s:i,s:i,s:i,s:L,s:i}",
>                               "online",   info.online,
>@@ -386,19 +419,19 @@
>                               "running",  info.running,
>                               "cpu_time", info.cpu_time,
>                               "cpu",      info.cpu);
>-
>     cpulist = PyList_New(0);
>-    for ( i = 0; cpumap != 0; i++ )
>+    for ( i = 0; i < nr_cpus; i++ )
>     {
>-        if ( cpumap & 1 ) {
>+        if (*(cpumap + i / (cpumap_size * 8)) & 1 ) {
>             PyObject *pyint = PyInt_FromLong(i);
>             PyList_Append(cpulist, pyint);
>             Py_DECREF(pyint);
>         }
>-        cpumap >>= 1;
>+        cpumap[i / (cpumap_size * 8)] >>= 1;
>     }
>     PyDict_SetItemString(info_dict, "cpumap", cpulist);
>     Py_DECREF(cpulist);
>+    free(cpumap);
>     return info_dict;
> }
>
http://old.nabble.com/file/p27941371/adjust_vcpuaffinity_more_cpu.patch
>adjust_vcpuaffinity_more_cpu.patch
>
>
>
>--
>View this message in context: http://old.nabble.com/-Patch--adjust-the-cpu-
>affinity-to-more-than-64-cpus-tp27928229p27941371.html
>Sent from the Xen - Dev mailing list archive at Nabble.com.
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@xxxxxxxxxxxxxxxxxxx
>http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel