WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] [PATCH][XM-TEST ] Add feature of test layering in groups for

To: xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] [PATCH][XM-TEST ] Add feature of test layering in groups for xm-test
From: Woody Marvel <marvel@xxxxxxxxxx>
Date: Fri, 10 Mar 2006 16:00:38 -0800
Delivery-date: Sat, 11 Mar 2006 00:00:32 +0000
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization: IBM LTC
Reply-to: marvel@xxxxxxxxxx
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Add feature to layer xm-test tests, grouping of tests by casename and
execution of tests by user specified lists.

Signed-off-by: Woody Marvel <marvel@xxxxxxxxxx>

-- 
Sincerely,      Woody Marvel
IBM Linux Technology Center
Open Source Virtualization
email: marvel@xxxxxxxxxx
503-578-3833   Beaverton, OR 97006
**********************************
# HG changeset patch
# User root@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Node ID a53dc86beec72390c9e4a5a1466204639831e085
# Parent  d4e433d615b04f4bbb61f240a3813a8fdb50d1f2
HG Patch


Add feature to layer xm-test tests, grouping of tests by casename and
execution of tests by user specified lists.

Signed-off-by: Woody Marvel <marvel@xxxxxxxxxx>

diff -r d4e433d615b0 -r a53dc86beec7 tools/xm-test/README
--- a/tools/xm-test/README      Fri Mar 10 16:32:04 2006
+++ b/tools/xm-test/README      Fri Mar 10 23:56:10 2006
@@ -120,8 +120,9 @@
    # ./runtest.sh <logfile>
 
 This will run all tests, as well as generate and submit a report at
-the end.  All output files will begin with "<logfile>."  If you wish to
-prevent submission of a report, add "-d" to the command line like this:
+the end. All output files will begin with "<logfile>."
+If you wish to prevent submission of a report, add "-d" to the 
+command line like this:
 
    # ./runtest.sh -d <logfile>
 
@@ -131,15 +132,19 @@
 
    # ./runtest.sh -s <logfile>
 
-For people needing a quick test run instead the full suite, a quick
-mode has been added that will attempt to run a representative subset
-of tests.  This is not a substitute for the whole suite, but will
-verify that some of the major functions of xen and xm are working:
-
-   # ./runtest.sh -q <logfile>
-
-Because of the current structure of the reporting software, submission
-of quick test run results is not supported.
+Group test sets are supported in xm-test. This is form of layering of 
+tests groups/cases/tests. In the framework directory "grouptest",
+files exist for group processing. The user can add groups, casenames
+and test lists as required. Default group run is "groutest/default".
+
+   # ./runtest.sh -g <groupname> <logfile>
+
+* NOTE: There is a subset of tests in group mode that is added
+and will attempt to run, not a substitute for the full test suite,
+a subset of tests.
+   # ./runtest.sh -g quick <logfile>
+
+
 
 It may be desirable to run a specific test group.  This can be
 accomplished by doing the following:
diff -r d4e433d615b0 -r a53dc86beec7 tools/xm-test/runtest.sh
--- a/tools/xm-test/runtest.sh  Fri Mar 10 16:32:04 2006
+++ b/tools/xm-test/runtest.sh  Fri Mar 10 23:56:10 2006
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/sh 
 
 ##
 ## Test driver script
@@ -11,7 +11,7 @@
     echo "  Where opts are:"
     echo "  -d          : do not submit a report for this run"
     echo "  -b          : do not ask any questions (batch mode)"
-    echo "  -q          : run a quick test set"
+    echo "  -g          : run a group test set"
     echo "  -e <email>  : set email address for report"
     echo "  -s <report> : just submit report <report>"
     echo "  -h | --help : show this help"
@@ -92,11 +92,12 @@
     echo "Running sanity checks..."
     make -C tests/_sanity check 2>&1 | grep REASON
     if [ $? -eq 0 ]; then
-       echo "Sanity checks failed"
-       exit 1
-    fi
-
-}
+        echo "Sanity checks failed"
+        exit 1
+    fi
+
+}
+
 
 # Get contact info if needed
 get_contact_info() {
@@ -128,26 +129,21 @@
 
 # Run the tests
 run_tests() {
-    output=$1
-    echo Running real tests...
-    TEST_VERBOSE=1 make -k check > $output 2>&1
-}
-
-run_tests_quick() {
-
-    output=$1
-
-    create_tests="01_create_basic_pos.test 07_create_mem64_pos.test 
10_create_fastdestroy.test 14_create_blockroot_pos.test"
-    unpause_tests="01_unpause_basic_pos.test"
-    memset_tests="01_memset_basic_pos.test 03_memset_random_pos.test"
-    help_tests="06_help_allcmds.test"
-    testgroups="create unpause memset help"
-
-    echo "*** Quick test" > $output
-    for group in $testgroups; do
-       eval $(echo list=\$${group}_tests)
-       echo "*** Running tests [$list] from $group"
-       (cd tests/$group && TEST_VERBOSE=1 make -k check TESTS="$list") >> 
$output 2>&1
+    groupentered=$1
+    output=$2
+
+    exec <  grouptest/$groupentered
+    while read casename testlist; do
+       echo Running $casename tests...
+       echo "*** case $casename from group $groupentered" >> $output
+       if [ -z "$testlist" ]; then
+          echo "*** Running tests for case $casename" >> $output
+          (cd tests/$casename && TEST_VERBOSE=1 make -k check) >> $output 2>&1
+       else
+          echo "*** Running tests $testlist from case $casename" >> $output
+          (cd tests/$casename && TEST_VERBOSE=1 make -k check 
TESTS="$testlist") >> $output 2>&1
+       fi
+
     done
 
 }
@@ -195,6 +191,7 @@
 report=yes
 batch=no
 run=yes
+GROUPENTERED=default
 
 # Resolve options
 while [ $# -gt 0 ]
@@ -213,8 +210,13 @@
          echo $1 > contact_info
          echo "(Email set to $1)"
          ;;
-      -q)
-         run=quick
+      -g)
+         shift
+          GROUPENTERED=$1
+          if [ ! -f grouptest/$GROUPENTERED ]; then
+             echo "No file for group $GROUPENTERED"
+             exit 1
+          fi
          ;;
       -s)
          run=no
@@ -265,18 +267,15 @@
 if [ "$run" != "no" ]; then
     runnable_tests
     make_environment_report $OSREPORTTEMP $PROGREPORTTEMP
-    if [ "$run" = "yes" ]; then
-       run_tests $OUTPUT
-    else
-       run_tests_quick $OUTPUT
-    fi
+    run_tests $GROUPENTERED $OUTPUT
     make_text_reports $PASSFAIL $FAILURES $OUTPUT $TXTREPORT
     make_result_report $OUTPUT $RESULTREPORTTEMP
     cat $OSREPORTTEMP $PROGREPORTTEMP $RESULTREPORTTEMP > $XMLREPORT
     rm $OSREPORTTEMP $PROGREPORTTEMP $RESULTREPORTTEMP
-fi
-
-if [ "$report" = "yes" ] && [ "$run" = "yes" ]; then
+
+fi
+
+if [ "$report" = "yes" ]; then
     if [ ! -f "$XMLREPORT" ]; then
        echo "No such file: $XMLREPORT"
        exit 1
diff -r d4e433d615b0 -r a53dc86beec7 tools/xm-test/grouptest/create
--- /dev/null   Fri Mar 10 16:32:04 2006
+++ b/tools/xm-test/grouptest/create    Fri Mar 10 23:56:10 2006
@@ -0,0 +1,1 @@
+create
diff -r d4e433d615b0 -r a53dc86beec7 tools/xm-test/grouptest/default
--- /dev/null   Fri Mar 10 16:32:04 2006
+++ b/tools/xm-test/grouptest/default   Fri Mar 10 23:56:10 2006
@@ -0,0 +1,30 @@
+block-create
+block-destroy
+block-list
+console
+create
+destroy
+dmesg
+domid
+domname
+enforce_dom0_cpus
+help
+info
+list
+memmax
+memset
+migrate
+network
+network-attach
+pause
+reboot
+restore
+save
+sedf
+shutdown
+sysrq
+unpause
+vcpu-disable
+vcpu-pin
+vtpm
+
diff -r d4e433d615b0 -r a53dc86beec7 tools/xm-test/grouptest/quick
--- /dev/null   Fri Mar 10 16:32:04 2006
+++ b/tools/xm-test/grouptest/quick     Fri Mar 10 23:56:10 2006
@@ -0,0 +1,4 @@
+create 01_create_basic_pos.test 07_create_mem64_pos.test 
10_create_fastdestroy.test 14_create_blockroot_pos.test
+unpause 01_unpause_basic_pos.test
+memset 01_memset_basic_pos.test 03_memset_random_pos.test
+help 06_help_allcmds.test
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel