WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-cim

[Xen-cim] Cimtester vs. Sblim-testsuite

To: <xen-cim@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-cim] Cimtester vs. Sblim-testsuite
From: "Szymanski, Lukasz K" <Lukasz.Szymanski@xxxxxxxxxx>
Date: Mon, 23 Apr 2007 17:11:34 -0400
Delivery-date: Mon, 23 Apr 2007 14:10:06 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-cim-request@lists.xensource.com?subject=help>
List-id: xen-cim mailing list <xen-cim.lists.xensource.com>
List-post: <mailto:xen-cim@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-cim>, <mailto:xen-cim-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-cim>, <mailto:xen-cim-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-cim-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AceF6/ii1ITQv0hKSteOMOEuh6MHdA==
Thread-topic: Cimtester vs. Sblim-testsuite

During last Thursday's call Jim and I talked about choosing the test suite which will be least burdensome and most flexible.  I spent some time evaluating sblim-test in the last few days in order to give both cimtester and sblim-test a fair shake. It's a somewhat close race.

I am attaching an excel spreadsheet with the details of my findings but the short story is this:

Cimtester has one input file, sblim test has one input class per test. 
Advantage: cimtester

Cimtester supports exec query from within the test.  Sblim-test does not.
Advantage: cimtester

Cimtester supports runnig external tests from within the test.  Sblim-test does not.
Advantage: cimtester

Cimtester has one output file which is not always very informative.  Sblim-test has one, nicely formatted output file per class

Advantage: sblim-test

Cimtester only catches exceptions, sblim-test will catch and interpret wbemcli errors.
Advantage: sblim-test

Sblim test has an additional set of tests to shake out consistency.
Advantage: sblim-test (see spreadsheets for more details)

I have a rudimentary script which writes the cimtester input xml file.  Sblim-test would require some more effort of that nature, but if the IBM folks are using sblim-test and have some scripts already, life might be a lot easier.  Cimtester requires some interpretation on the output end, because the errors are not as informative or well formatted.  Sblim-test has a whole suite to test consistency but it would require more work and I am not sure if the value is worth the effort. Two big strikes against sblim test are lack of support for exec query and external tests.  I think that tips the scales in favor of cimtester, but I am open to discussion.  Please let me know what you think.

Luke

<<cim_sblim_diff.xls>>

Attachment: cim_sblim_diff.xls
Description: cim_sblim_diff.xls

_______________________________________________
Xen-cim mailing list
Xen-cim@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-cim
<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-cim] Cimtester vs. Sblim-testsuite, Szymanski, Lukasz K <=