WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] [Patch] support of lock profiling in Xen

To: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] [Patch] support of lock profiling in Xen
From: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Date: Thu, 08 Oct 2009 09:48:04 +0200
Delivery-date: Thu, 08 Oct 2009 00:48:40 -0700
Dkim-signature: v=1; a=rsa-sha256; c=simple/simple; d=ts.fujitsu.com; i=juergen.gross@xxxxxxxxxxxxxx; q=dns/txt; s=s1536b; t=1254988148; x=1286524148; h=from:sender:reply-to:subject:date:message-id:to:cc: mime-version:content-transfer-encoding:content-id: content-description:resent-date:resent-from:resent-sender: resent-to:resent-cc:resent-message-id:in-reply-to: references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:list-owner:list-archive; z=From:=20Juergen=20Gross=20<juergen.gross@xxxxxxxxxxxxxx> |Subject:=20[Patch]=20support=20of=20lock=20profiling=20i n=20Xen|Date:=20Thu,=2008=20Oct=202009=2009:48:04=20+0200 |Message-ID:=20<4ACD9934.4070908@xxxxxxxxxxxxxx>|To:=20"x en-devel@xxxxxxxxxxxxxxxxxxx"=20<xen-devel@xxxxxxxxxxxxxx e.com>|MIME-Version:=201.0; bh=eumOwo6sgGoX69IkRAHC2csFOIzoBCqtikbRAg/+7V4=; b=HFrgfVv2143KlYj++Uxjv3C1QrFBc9zNui4W4udAG4EPKkBhPCGtSYsw 2aYzudBc12WFJPEL/ogH1+NXRl/fH1MF31yYd95mp0LssosbyWGUccP5a S8URoDeP8E/I9jxPmxk6cwLtjbSLvu/pJsI8Fne0BWhWhYRy0Ca5eM5kh uWcigckYAcYttmyBKRlMAL9Soos2AvHii1hSah6TDGfEYgMZhLcZT4LIY aFmCijF8VZ3oVkLeifwRXiFDs7R7U;
Domainkey-signature: s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:Subject:X-Enigmail-Version: Content-Type; b=ROq0bnK8Ouz1UgzTgiCDfZJWL+bNQFqNNnTnPPjVCKSYxoJIq1Nd/LWe prCxjxkn+Rvj0dZcKqCkxr0RR73Aj2lKRu14uZk0YeG16HypWo2U0H+1F xxzjjIoKJiDJjBQReQ4TXJnR6aRJhduTf05puLrpMNPKbzaXSAH4f/4UF I66xzAY+xx7zU55I0/Abbk4uyEVQgK68UncVSM/2260iAsxwnXRG7qMeZ pe4q06MLAomZPGRBo0xm0NXBgrYZL;
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: Fujitsu Technology Solutions
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla-Thunderbird 2.0.0.22 (X11/20090707)
Hi,

attached is a patch to support lock profiling in the Xen hypervisor.
The feature is switchable at compile time (like software performance counters)
in Rules.mk. If lock profiling is not activated, code and data structures are
the same as without the patch. With activated lock profiling some statistical
data is added to each spinlock_t structure and locking/unlocking keeps track
for each lock of:
- number of successful locks
- cumulated time of lock held
- number of locking calls leading to wait cycles
- cumulated waiting time

The profile data may be obtained either via hotkey from the console or with
the new tool xenlockprof.

To be able to find all locks for printing out the profile data, a linked list
with the locks is built up. Locks defined via DEFINE_SPINLOCK are added to
this list automatically, locks in dynamically allocated data structures have
to be added (and removed) explicitly to the queue to be able to print the
profile data.
Adding is done via spin_lock_init_prof (there are two additional calls with
one or two index parameters to be able to distinguish locks e.g. per domid and
vcpu-nr), removing is done via spin_lock_destroy_prof (may be called for a
completely zeroed structure as well).

I'm not completely satisfied with the solution for the dynamically initialized
locks, but I had no better idea in the first run.
Another enhancement would be to expand the profiling to rw-locks as well, but
this would have required a rewrite of the lock routines using try_lock like
for spinlocks. I could do this if the lock profiling is accepted.

Comments welcome :-)


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 636 47950
Fujitsu Technolgy Solutions               e-mail: juergen.gross@xxxxxxxxxxxxxx
Otto-Hahn-Ring 6                        Internet: ts.fujitsu.com
D-81739 Muenchen                 Company details: ts.fujitsu.com/imprint.html
Support of lock profiling in Xen

Signed-off-by: juergen.gross@xxxxxxxxxxxxxx

diff -r 729567f615c1 tools/libxc/xc_misc.c
--- a/tools/libxc/xc_misc.c     Wed Oct 07 16:29:03 2009 +0100
+++ b/tools/libxc/xc_misc.c     Thu Oct 08 09:18:52 2009 +0200
@@ -121,6 +121,30 @@
     return rc;
 }
 
+int xc_lockprof_control(int xc_handle,
+                        uint32_t opcode,
+                        uint32_t *n_elems,
+                        uint64_t *time,
+                        xc_lockprof_data_t *data)
+{
+    int rc;
+    DECLARE_SYSCTL;
+
+    sysctl.cmd = XEN_SYSCTL_lockprof_op;
+    sysctl.u.lockprof_op.cmd = opcode;
+    sysctl.u.lockprof_op.max_elem = n_elems ? *n_elems : 0;
+    set_xen_guest_handle(sysctl.u.lockprof_op.data, data);
+
+    rc = do_sysctl(xc_handle, &sysctl);
+
+    if (n_elems)
+        *n_elems = sysctl.u.lockprof_op.nr_elem;
+    if (time)
+        *time = sysctl.u.lockprof_op.time;
+
+    return rc;
+}
+
 int xc_getcpuinfo(int xc_handle, int max_cpus,
                   xc_cpuinfo_t *info, int *nr_cpus)
 {
diff -r 729567f615c1 tools/libxc/xenctrl.h
--- a/tools/libxc/xenctrl.h     Wed Oct 07 16:29:03 2009 +0100
+++ b/tools/libxc/xenctrl.h     Thu Oct 08 09:18:52 2009 +0200
@@ -700,6 +700,14 @@
                      int *nbr_desc,
                      int *nbr_val);
 
+typedef xen_sysctl_lockprof_data_t xc_lockprof_data_t;
+/* IMPORTANT: The caller is responsible for mlock()'ing the @data array. */
+int xc_lockprof_control(int xc_handle,
+                        uint32_t opcode,
+                        uint32_t *n_elems,
+                        uint64_t *time,
+                        xc_lockprof_data_t *data);
+
 /**
  * Memory maps a range within one domain to a local address range.  Mappings
  * should be unmapped with munmap and should follow the same rules as mmap
diff -r 729567f615c1 tools/misc/Makefile
--- a/tools/misc/Makefile       Wed Oct 07 16:29:03 2009 +0100
+++ b/tools/misc/Makefile       Thu Oct 08 09:18:52 2009 +0200
@@ -10,7 +10,7 @@
 
 HDRS     = $(wildcard *.h)
 
-TARGETS-y := xenperf xenpm xen-tmem-list-parse gtraceview gtracestat
+TARGETS-y := xenperf xenpm xen-tmem-list-parse gtraceview gtracestat 
xenlockprof
 TARGETS-$(CONFIG_X86) += xen-detect xen-hvmctx
 TARGETS := $(TARGETS-y)
 
@@ -22,7 +22,7 @@
 INSTALL_BIN-$(CONFIG_X86) += xen-detect
 INSTALL_BIN := $(INSTALL_BIN-y)
 
-INSTALL_SBIN-y := xm xen-bugtool xen-python-path xend xenperf xsview xenpm 
xen-tmem-list-parse gtraceview gtracestat
+INSTALL_SBIN-y := xm xen-bugtool xen-python-path xend xenperf xsview xenpm 
xen-tmem-list-parse gtraceview gtracestat xenlockprof
 INSTALL_SBIN-$(CONFIG_X86) += xen-hvmctx
 INSTALL_SBIN := $(INSTALL_SBIN-y)
 
@@ -49,7 +49,7 @@
 %.o: %.c $(HDRS) Makefile
        $(CC) -c $(CFLAGS) -o $@ $<
 
-xen-hvmctx xenperf xenpm gtracestat: %: %.o Makefile
+xen-hvmctx xenperf xenpm gtracestat xenlockprof: %: %.o Makefile
        $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) $(LDFLAGS_libxenctrl)
 
 gtraceview: %: %.o Makefile
diff -r 729567f615c1 tools/misc/xenlockprof.c
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/tools/misc/xenlockprof.c  Thu Oct 08 09:18:52 2009 +0200
@@ -0,0 +1,129 @@
+/* -*-  Mode:C; c-basic-offset:4; tab-width:4 -*-
+ ****************************************************************************
+ * (C) 2009 - Juergen Gross - Fujitsu Technology Solutions
+ ****************************************************************************
+ *
+ *        File: xenlockprof.c
+ *      Author: Juergen Gross (juergen.gross@xxxxxxxxxxxxxx)
+ *        Date: Oct 2009
+ * 
+ * Description: 
+ */
+
+#include <xenctrl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/mman.h>
+#include <errno.h>
+#include <string.h>
+
+static int lock_pages(void *addr, size_t len)
+{
+    int e = 0;
+#ifndef __sun__
+    e = mlock(addr, len);
+#endif
+    return (e);
+}
+
+static void unlock_pages(void *addr, size_t len)
+{
+#ifndef __sun__
+    munlock(addr, len);
+#endif
+}
+
+int main(int argc, char *argv[])
+{
+    int                xc_handle;
+    uint32_t           i, j, n;
+    uint64_t           time;
+    double             l, b, sl, sb;
+    char               name[60];
+    xc_lockprof_data_t *data;
+
+    if ((argc > 2) || ((argc == 2) && (strcmp(argv[1], "-r") != 0)))
+    {
+        printf("%s: [-r]\n", argv[0]);
+        printf("no args: print lock profile data\n");
+        printf("    -r : reset profile data\n");
+        return 1;
+    }
+
+    if ((xc_handle = xc_interface_open()) == -1)
+    {
+        fprintf(stderr, "Error opening xc interface: %d (%s)\n",
+                errno, strerror(errno));
+        return 1;
+    }
+
+    if (argc > 1)
+    {
+        if (xc_lockprof_control(xc_handle, XEN_SYSCTL_LOCKPROF_reset, NULL,
+                                NULL, NULL) != 0)
+        {
+            fprintf(stderr, "Error reseting profile data: %d (%s)\n",
+                    errno, strerror(errno));
+            return 1;
+        }
+        return 1;
+    }
+
+    n = 0;
+    if (xc_lockprof_control(xc_handle, XEN_SYSCTL_LOCKPROF_query, &n,
+                            NULL, NULL) != 0)
+    {
+        fprintf(stderr, "Error getting number of profile records: %d (%s)\n",
+                errno, strerror(errno));
+        return 1;
+    }
+
+    n += 32;    /* just to be sure */
+    data = malloc(sizeof(*data) * n);
+    if ((data == NULL) || (lock_pages(data, sizeof(*data) * n) != 0))
+    {
+        fprintf(stderr, "Could not alloc or lock buffers: %d (%s)\n",
+                errno, strerror(errno));
+        return 1;
+    }
+
+    i = n;
+    if ( xc_lockprof_control(xc_handle, XEN_SYSCTL_LOCKPROF_query, &i,
+                             &time, data) != 0)
+    {
+        fprintf(stderr, "Error getting profile records: %d (%s)\n",
+                errno, strerror(errno));
+        return 1;
+    }
+
+    unlock_pages(data, sizeof(*data) * n);
+
+    if (i > n)
+    {
+        printf("data incomplete, %d records are missing!\n\n", i - n);
+        i = n;
+    }
+    sl = 0;
+    sb = 0;
+    for (j = 0; j < i; j++)
+    {
+        if (data[j].idx2 >= 0)
+            sprintf(name, data[j].name, data[j].idx1, data[j].idx2);
+        else if (data[j].idx1 >= 0)
+            sprintf(name, data[j].name, data[j].idx1);
+        else
+            sprintf(name, data[j].name);
+        l = (double)(data[j].lock_time) / 1E+09;
+        b = (double)(data[j].block_time) / 1E+09;
+        sl += l;
+        sb += b;
+        printf("%-50s: lock:%12ld(%20.9fs), block:%12ld(%20.9fs)\n",
+            name, data[j].lock_cnt, l, data[j].block_cnt, b);
+    }
+    l = (double)time / 1E+09;
+    printf("total profiling time: %20.9fs\n", l);
+    printf("total locked time:    %20.9fs\n", sl);
+    printf("total blocked time:   %20.9fs\n", sb);
+
+    return 0;
+}
diff -r 729567f615c1 xen/Rules.mk
--- a/xen/Rules.mk      Wed Oct 07 16:29:03 2009 +0100
+++ b/xen/Rules.mk      Thu Oct 08 09:18:52 2009 +0200
@@ -6,6 +6,7 @@
 verbose       ?= n
 perfc         ?= n
 perfc_arrays  ?= n
+lock_profile  ?= n
 crash_debug   ?= n
 frame_pointer ?= n
 
@@ -49,6 +50,7 @@
 CFLAGS-$(crash_debug)   += -DCRASH_DEBUG
 CFLAGS-$(perfc)         += -DPERF_COUNTERS
 CFLAGS-$(perfc_arrays)  += -DPERF_ARRAYS
+CFLAGS-$(lock_profile)  += -DLOCK_PROFILE
 CFLAGS-$(frame_pointer) += -fno-omit-frame-pointer -DCONFIG_FRAME_POINTER
 
 ifneq ($(max_phys_cpus),)
diff -r 729567f615c1 xen/arch/x86/xen.lds.S
--- a/xen/arch/x86/xen.lds.S    Wed Oct 07 16:29:03 2009 +0100
+++ b/xen/arch/x86/xen.lds.S    Thu Oct 08 09:18:52 2009 +0200
@@ -62,6 +62,13 @@
        *(.data.read_mostly)
   } :text
 
+#ifdef LOCK_PROFILE
+  . = ALIGN(32);
+  __lock_profile_start = .;
+  .lockprofile.data : { *(.lockprofile.data) } :text
+  __lock_profile_end = .;
+#endif
+
   . = ALIGN(4096);             /* Init code and data */
   __init_begin = .;
   .init.text : {
diff -r 729567f615c1 xen/common/keyhandler.c
--- a/xen/common/keyhandler.c   Wed Oct 07 16:29:03 2009 +0100
+++ b/xen/common/keyhandler.c   Thu Oct 08 09:18:52 2009 +0200
@@ -347,6 +347,20 @@
 };
 #endif
 
+#ifdef LOCK_PROFILE
+extern void spinlock_profile_printall(unsigned char key);
+static struct keyhandler spinlock_printall_keyhandler = {
+    .diagnostic = 1,
+    .u.fn = spinlock_profile_printall,
+    .desc = "print lock profile info"
+};
+extern void spinlock_profile_reset(unsigned char key);
+static struct keyhandler spinlock_reset_keyhandler = {
+    .u.fn = spinlock_profile_reset,
+    .desc = "reset lock profile info"
+};
+#endif
+
 static void run_all_nonirq_keyhandlers(unsigned long unused)
 {
     /* Fire all the non-IRQ-context diagnostic keyhandlers */
@@ -428,6 +442,12 @@
     register_keyhandler('p', &perfc_printall_keyhandler);
     register_keyhandler('P', &perfc_reset_keyhandler);
 #endif
+
+#ifdef LOCK_PROFILE
+    register_keyhandler('l', &spinlock_printall_keyhandler);
+    register_keyhandler('L', &spinlock_reset_keyhandler);
+#endif
+
 }
 
 /*
diff -r 729567f615c1 xen/common/spinlock.c
--- a/xen/common/spinlock.c     Wed Oct 07 16:29:03 2009 +0100
+++ b/xen/common/spinlock.c     Thu Oct 08 09:18:52 2009 +0200
@@ -1,7 +1,11 @@
+#include <xen/lib.h>
 #include <xen/config.h>
 #include <xen/irq.h>
 #include <xen/smp.h>
+#include <xen/time.h>
 #include <xen/spinlock.h>
+#include <xen/guest_access.h>
+#include <public/sysctl.h>
 #include <asm/processor.h>
 
 #ifndef NDEBUG
@@ -41,56 +45,97 @@
 
 #endif
 
+#ifdef LOCK_PROFILE
+
+#define LOCK_PROFILE_REL                                          \
+    lock->profile.time_hold += NOW() - lock->profile.time_locked; \
+    lock->profile.lock_cnt++
+#define LOCK_PROFILE_VAR    s_time_t block = 0
+#define LOCK_PROFILE_BLOCK  block = block ? : NOW();
+#define LOCK_PROFILE_GOT                                          \
+    lock->profile.time_locked = NOW();                            \
+    if (block)                                                    \
+    {                                                             \
+        lock->profile.time_block += lock->profile.time_locked - block; \
+        lock->profile.block_cnt++;                                \
+    }
+
+#else
+
+#define LOCK_PROFILE_REL
+#define LOCK_PROFILE_VAR
+#define LOCK_PROFILE_BLOCK
+#define LOCK_PROFILE_GOT
+
+#endif
+
 void _spin_lock(spinlock_t *lock)
 {
+    LOCK_PROFILE_VAR;
+
     check_lock(&lock->debug);
     while ( unlikely(!_raw_spin_trylock(&lock->raw)) )
+    {
+        LOCK_PROFILE_BLOCK;
         while ( likely(_raw_spin_is_locked(&lock->raw)) )
             cpu_relax();
+    }
+    LOCK_PROFILE_GOT;
 }
 
 void _spin_lock_irq(spinlock_t *lock)
 {
+    LOCK_PROFILE_VAR;
+
     ASSERT(local_irq_is_enabled());
     local_irq_disable();
     check_lock(&lock->debug);
     while ( unlikely(!_raw_spin_trylock(&lock->raw)) )
     {
+        LOCK_PROFILE_BLOCK;
         local_irq_enable();
         while ( likely(_raw_spin_is_locked(&lock->raw)) )
             cpu_relax();
         local_irq_disable();
     }
+    LOCK_PROFILE_GOT;
 }
 
 unsigned long _spin_lock_irqsave(spinlock_t *lock)
 {
     unsigned long flags;
+    LOCK_PROFILE_VAR;
+
     local_irq_save(flags);
     check_lock(&lock->debug);
     while ( unlikely(!_raw_spin_trylock(&lock->raw)) )
     {
+        LOCK_PROFILE_BLOCK;
         local_irq_restore(flags);
         while ( likely(_raw_spin_is_locked(&lock->raw)) )
             cpu_relax();
         local_irq_save(flags);
     }
+    LOCK_PROFILE_GOT;
     return flags;
 }
 
 void _spin_unlock(spinlock_t *lock)
 {
+    LOCK_PROFILE_REL;
     _raw_spin_unlock(&lock->raw);
 }
 
 void _spin_unlock_irq(spinlock_t *lock)
 {
+    LOCK_PROFILE_REL;
     _raw_spin_unlock(&lock->raw);
     local_irq_enable();
 }
 
 void _spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)
 {
+    LOCK_PROFILE_REL;
     _raw_spin_unlock(&lock->raw);
     local_irq_restore(flags);
 }
@@ -104,13 +149,32 @@
 int _spin_trylock(spinlock_t *lock)
 {
     check_lock(&lock->debug);
+#ifndef LOCK_PROFILE
     return _raw_spin_trylock(&lock->raw);
+#else
+    if (!_raw_spin_trylock(&lock->raw)) return 0;
+    lock->profile.time_locked = NOW();
+    return 1;
+#endif
 }
 
 void _spin_barrier(spinlock_t *lock)
 {
+#ifdef LOCK_PROFILE
+    s_time_t block = NOW();
+    u64      loop = 0;
+
+    check_lock(&lock->debug);
+    do { mb(); loop++;} while ( _raw_spin_is_locked(&lock->raw) );
+    if (loop > 1)
+    {
+        lock->profile.time_block += NOW() - block;
+        lock->profile.block_cnt++;
+    }
+#else
     check_lock(&lock->debug);
     do { mb(); } while ( _raw_spin_is_locked(&lock->raw) );
+#endif
     mb();
 }
 
@@ -248,3 +312,139 @@
     check_lock(&lock->debug);
     return _raw_rw_is_write_locked(&lock->raw);
 }
+
+#ifdef LOCK_PROFILE
+extern struct lock_profile *__lock_profile_start;
+extern struct lock_profile *__lock_profile_end;
+
+static s_time_t lock_profile_start = 0;
+static struct lock_profile *lock_profile_anc = NULL;
+static int lock_profile_n = 0;
+static spinlock_t lock_profile_lock = SPIN_LOCK_UNLOCKED;
+
+void spinlock_profile_printall(unsigned char key)
+{
+    s_time_t now = NOW();
+    s_time_t diff;
+    struct lock_profile *q;
+
+    diff = now - lock_profile_start;
+    printk("Xen lock profile info SHOW  (now = %08X:%08X, "
+        "total = %08X:%08X)\n", (u32)(now>>32), (u32)now,
+        (u32)(diff>>32), (u32)diff);
+    spin_lock(&lock_profile_lock);
+    for (q = lock_profile_anc; q != NULL; q = q->next)
+    {
+        if (q->idx1 == -1) printk(q->name);
+        else if (q->idx2 == -1) printk(q->name, q->idx1);
+        else printk(q->name, q->idx1, q->idx2);
+        printk("\n  lock:%12ld(%08X:%08X), block:%12ld(%08X:%08X)\n",
+            q->lock_cnt, (u32)(q->time_hold >> 32), (u32)q->time_hold,
+            q->block_cnt, (u32)(q->time_block >> 32), (u32)q->time_block);
+    }
+    spin_unlock(&lock_profile_lock);
+    return;
+}
+
+void spinlock_profile_reset(unsigned char key)
+{
+    s_time_t now = NOW();
+    struct lock_profile *q;
+
+    if ( key != '\0' )
+        printk("Xen lock profile info RESET (now = %08X:%08X)\n",
+            (u32)(now>>32), (u32)now);
+    lock_profile_start = now;
+    spin_lock(&lock_profile_lock);
+    for (q = lock_profile_anc; q != NULL; q = q->next)
+    {
+        q->lock_cnt =0;
+        q->block_cnt = 0;
+        q->time_hold = 0;
+        q->time_block = 0;
+    }
+    spin_unlock(&lock_profile_lock);
+    return;
+}
+
+void spinlock_profile_add(struct lock_profile *p)
+{
+    spin_lock(&lock_profile_lock);
+    lock_profile_n++;
+    p->prev = NULL;
+    if (lock_profile_anc != NULL) lock_profile_anc->prev = p;
+    p->next = lock_profile_anc;
+    lock_profile_anc = p;
+    spin_unlock(&lock_profile_lock);
+    return;
+}
+
+void spinlock_profile_rm(struct lock_profile *p)
+{
+    if (p->name == NULL) return;
+    spin_lock(&lock_profile_lock);
+    if (p->next != NULL) p->next->prev = p->prev;
+    if (p->prev != NULL) p->prev->next = p->next;
+    else lock_profile_anc = p->next;
+    lock_profile_n--;
+    spin_unlock(&lock_profile_lock);
+    return;
+}
+
+/* Dom0 control of lock profiling */
+int spinlock_profile_control(xen_sysctl_lockprof_op_t *pc)
+{
+    int rc;
+    int i;
+    struct lock_profile *q;
+    xen_sysctl_lockprof_data_t elem;
+
+    rc = 0;
+    switch (pc->cmd)
+    {
+    case XEN_SYSCTL_LOCKPROF_reset:
+        spinlock_profile_reset('\0');
+        break;
+    case XEN_SYSCTL_LOCKPROF_query:
+        spin_lock(&lock_profile_lock);
+        q = lock_profile_anc;
+        for (i = 0; i < pc->max_elem; i++)
+        {
+            if (q == NULL) break;
+            safe_strcpy(elem.name, q->name);
+            elem.idx1 = q->idx1;
+            elem.idx2 = q->idx2;
+            elem.lock_cnt = q->lock_cnt;
+            elem.block_cnt = q->block_cnt;
+            elem.lock_time = q->time_hold;
+            elem.block_time = q->time_block;
+            if (copy_to_guest_offset(pc->data, i, &elem, 1))
+            {
+                rc = -EFAULT;
+                break;
+            }
+            q = q->next;
+        }
+        pc->nr_elem = lock_profile_n;
+        pc->time = NOW() - lock_profile_start;
+        spin_unlock(&lock_profile_lock);
+        break;
+    default:
+        rc = -EINVAL;
+        break;
+    }
+    return rc;
+}
+
+static int __init spinlock_profile_init(void)
+{
+    struct lock_profile **data;
+
+    for (data = &__lock_profile_start; data < &__lock_profile_end; data++)
+    {
+        spinlock_profile_add(*data);
+    }
+    return 0;
+}
+__initcall(spinlock_profile_init);
+#endif
diff -r 729567f615c1 xen/common/sysctl.c
--- a/xen/common/sysctl.c       Wed Oct 07 16:29:03 2009 +0100
+++ b/xen/common/sysctl.c       Thu Oct 08 09:18:52 2009 +0200
@@ -29,6 +29,9 @@
 
 extern long arch_do_sysctl(
     struct xen_sysctl *op, XEN_GUEST_HANDLE(xen_sysctl_t) u_sysctl);
+#ifdef LOCK_PROFILE
+extern int spinlock_profile_control(xen_sysctl_lockprof_op_t *pc);
+#endif
 
 long do_sysctl(XEN_GUEST_HANDLE(xen_sysctl_t) u_sysctl)
 {
@@ -144,6 +147,15 @@
     break;
 #endif
 
+#ifdef LOCK_PROFILE
+    case XEN_SYSCTL_lockprof_op:
+    {
+        ret = spinlock_profile_control(&op->u.lockprof_op);
+        if ( copy_to_guest(u_sysctl, op, 1) )
+            ret = -EFAULT;
+    }
+    break;
+#endif
     case XEN_SYSCTL_debug_keys:
     {
         char c;
diff -r 729567f615c1 xen/include/public/sysctl.h
--- a/xen/include/public/sysctl.h       Wed Oct 07 16:29:03 2009 +0100
+++ b/xen/include/public/sysctl.h       Thu Oct 08 09:18:52 2009 +0200
@@ -454,6 +454,34 @@
 
 #define PG_OFFLINE_OWNER_SHIFT 16
 
+#define XEN_SYSCTL_lockprof_op       11
+/* Sub-operations: */
+#define XEN_SYSCTL_LOCKPROF_reset 1   /* Reset all profile data to zero. */
+#define XEN_SYSCTL_LOCKPROF_query 2   /* Get lock profile information. */
+struct xen_sysctl_lockprof_data {
+    char     name[40];     /* lock name (may include up to 2 %d specifiers) */
+    int32_t  idx1;         /* index 1 (e.g. domain id) or -1 */
+    int32_t  idx2;         /* index 2 (e.g. vcpu nr) or -1 */
+    uint64_t lock_cnt;     /* # of locking succeeded */
+    uint64_t block_cnt;    /* # of wait for lock */
+    uint64_t lock_time;    /* nsecs lock held */
+    uint64_t block_time;   /* nsecs waited for lock */
+};
+typedef struct xen_sysctl_lockprof_data xen_sysctl_lockprof_data_t;
+DEFINE_XEN_GUEST_HANDLE(xen_sysctl_lockprof_data_t);
+struct xen_sysctl_lockprof_op {
+    /* IN variables. */
+    uint32_t       cmd;               /* XEN_SYSCTL_LOCKPROF_??? */
+    uint32_t       max_elem;          /* size of output buffer */
+    /* OUT variables (query only). */
+    uint32_t       nr_elem;           /* number of elements available */
+    uint64_t       time;              /* nsecs of profile measurement */
+    /* profile information (or NULL) */
+    XEN_GUEST_HANDLE_64(xen_sysctl_lockprof_data_t) data;
+};
+typedef struct xen_sysctl_lockprof_op xen_sysctl_lockprof_op_t;
+DEFINE_XEN_GUEST_HANDLE(xen_sysctl_lockprof_op_t);
+
 struct xen_sysctl {
     uint32_t cmd;
     uint32_t interface_version; /* XEN_SYSCTL_INTERFACE_VERSION */
@@ -471,6 +499,7 @@
         struct xen_sysctl_cpu_hotplug       cpu_hotplug;
         struct xen_sysctl_pm_op             pm_op;
         struct xen_sysctl_page_offline_op   page_offline;
+        struct xen_sysctl_lockprof_op       lockprof_op;
         uint8_t                             pad[128];
     } u;
 };
diff -r 729567f615c1 xen/include/xen/spinlock.h
--- a/xen/include/xen/spinlock.h        Wed Oct 07 16:29:03 2009 +0100
+++ b/xen/include/xen/spinlock.h        Thu Oct 08 09:18:52 2009 +0200
@@ -19,17 +19,63 @@
 #define spin_debug_disable() ((void)0)
 #endif
 
+#ifdef LOCK_PROFILE
+/*
+    lock profiling on: ALL locks are subject to profiling, data is available
+    only for locks declared or inited with profiling support
+*/
+struct lock_profile {
+    struct lock_profile *next;   /* forward link in profile queue */
+    struct lock_profile *prev;   /* backward link in profile queue */
+    char *name;                  /* lock name, may contain %d for idx */
+    u64  lock_cnt;               /* # of complete locking ops */
+    u64  block_cnt;              /* # of complete wait for lock */
+    s64  time_locked;            /* system time of last locking */
+    s64  time_hold;              /* cumulated lock time */
+    s64  time_block;             /* cumulated wait time */
+    int  idx1;                   /* index to print (e.g. domain id) or -1 */
+    int  idx2;                   /* index to print (e.g. vcpu nr) or -1 */
+};
+
+#define _LOCK_PROFILE(name, i1, i2) { 0, 0, name, 0, 0, 0, 0, 0, i1, i2 }
+#define _LOCK_PROFILE_ANC(name) ;                                           \
+    static struct lock_profile * __lock_profile_##name                      \
+    __attribute_used__ __attribute__ ((__section__(".lockprofile.data"))) = \
+    &name.profile
+
+void spinlock_profile_add(struct lock_profile *p);
+void spinlock_profile_rm(struct lock_profile *p);
+
+#else
+struct lock_profile { };
+#define _LOCK_PROFILE(name, i1, i2) { }
+#define _LOCK_PROFILE_ANC(name)
+#define spinlock_profile_add(x) ((void)0)
+#define spinlock_profile_rm(x) ((void)0)
+#endif
+
 typedef struct {
     raw_spinlock_t raw;
     u16 recurse_cpu:12;
     u16 recurse_cnt:4;
     struct lock_debug debug;
+    struct lock_profile profile;
 } spinlock_t;
 
 
-#define SPIN_LOCK_UNLOCKED { _RAW_SPIN_LOCK_UNLOCKED, 0xfffu, 0, _LOCK_DEBUG }
-#define DEFINE_SPINLOCK(l) spinlock_t l = SPIN_LOCK_UNLOCKED
+#define SPIN_LOCK_UNLOCKED { _RAW_SPIN_LOCK_UNLOCKED, 0xfffu, 0, _LOCK_DEBUG, \
+                             _LOCK_PROFILE(NULL, -1, -1) }
+#define SPIN_LOCK_UNLOCKED_PROF(n, i1, i2) { _RAW_SPIN_LOCK_UNLOCKED, 0xfffu, \
+                             0, _LOCK_DEBUG, _LOCK_PROFILE(n, i1, i2) }
+#define DEFINE_SPINLOCK(l) spinlock_t l = SPIN_LOCK_UNLOCKED_PROF(#l, -1, -1) \
+                             _LOCK_PROFILE_ANC(l)
 #define spin_lock_init(l) (*(l) = (spinlock_t)SPIN_LOCK_UNLOCKED)
+#define spin_lock_init_prof2(l, n, i1, i2) {                                  \
+    *(l) = (spinlock_t)SPIN_LOCK_UNLOCKED_PROF(n, i1, i2);                    \
+    spinlock_profile_add(&(l)->profile);  }
+#define spin_lock_init_prof1(l, n, i1)   spin_lock_init_prof2(l, n, i1, -1)
+#define spin_lock_init_prof(l, n)        spin_lock_init_prof2(l, n, -1, -1)
+#define spin_lock_destroy_prof(l)        spinlock_profile_rm(&(l)->profile)
 
 typedef struct {
     raw_rwlock_t raw;
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel