Click here to Skip to main content
15,867,835 members
Articles / Programming Languages / Visual Basic
Article

The best Unit Test Framework (UTF) in town

Rate me:
Please Sign up or sign in to vote.
4.36/5 (14 votes)
26 Jan 20045 min read 113.5K   261   43   26
Unit Test Framework that support security context and multi threading stress

Sample screenshot

Introduction

This is not the only free Unit Test in town, but this one has some important advantages over other ones:

  • This Unit Test can check your code against different security context
  • It can test your code under multi-thread stress
  • It has an advanced GUI includes agents and performance control
  • It saves the testing result for farther analysis.

The idea behind this Unit Test Framework (UTF) is derived from Nunit but the implementation is different.  If you already invested in Nunit you can still use your Nunit test with this UTF.  All you have to do is:

  • Mark your test with the UTF attributes.
  • Call your [Setup] method from the constructor.

Currently the UTF support 2 method signatures:

  • object MethodName()
  • void MethodName()

In order to mark your code as testing code you should put the assembly under the UTF execution folder or sub folder.

You must mark your assembly by the following attribute:

C#
using WiseMobility.UnitTest.Attributes;
[assembly: UTAssemblyBehavior
("Permissions Checks", TextColor="yellow", Enable=true)]

At the class level you must use the following attribute:

C#
[UTTypeBehavior ("Reflection Manip", TextColor="DarkBlue", Enable=true)]

At the method level you must use the following attribute:

C#
[UTMethodBehavior ("OS Version", ExpectedResult=EnmExpectedResult.eSuccess,
TextColor="Green",
Enable=true)]

or

C#
[UTMethodBehavior ("Fail as expected", ExpectedResult=EnmExpectedResult.eFail,
ExpectedException=typeof(DivideByZeroException), TextColor="SlateBlue")]

You can mark the method success expectation and if you expect failure you should mark the expected exception.  For more information see the following specification.

Other optional attributes are:

  • Performace tracing attributes (this attribute an be attached to assembly/class/method):
C#
[assembly: UTPerfMon (EnmPerformanceType.eCPU_Percent)]
[assembly: UTPerfMon (EnmPerformanceType.eMemory_Percent)]
  • Security context attributes (this attribute an be attached to assembly/class/method):
C#
[assembly: UTSecurityPermissionAttribute (enmSecPermissionOptions.eReflection,
Comment="Some Comments")]
  • You can mark any class or method under the assembly for ignoring the higher level of security context attribute:
C#
[UTSecurityPermissionAttribute (enmSecPermissionOptions.eReflection,
Behavior=enmLimitIgnore.eIgnore)]

For more details simply go through the following UTF specification.  The specification will give you general view but you more then invited to research the code.

Currently the report is not implemented, but you can use any report mechanism on top of the UTP database (SQL server) to generate your own report.

If you have question or have developed some improvement that you want to share with other you can reach me at the following mail: bnayanet@hotmail.com

Enjoy J

Conceptual and Design Goals

Introduction

  • Unit testing is the main method that the developer conducts in order to ensure quality code shipping.
  • The tests should be valid under different security privilege, and under multi threaded stress.
  • The Unit Test Framework tool should increase the development process productivity by helping to develop more robust code.
  • The idea behind it is to find most of the bugs before the programmer release his code to the QA department or to his colleagues.

For more information of why using unit test framework see the following article:

The benefits of automated unit testing

Conceptual

The Unit Test Framework constructs a framework for running additional test and assesses the result.  The programmer is responsible for writing the tests. The Unit Test Framework is responsibility for assessing the results.

Sample screenshot

Motivation

The motivation of using the unit test framework is:

  • Construct consistent methods of testing code under different contexts (i.e. under restricted permission or multi threading stress), and to identify problems as soon as possible in order to reduce the development process cost and increase productivity.
  • Ensure the code quality (the benefit of robust code will affect the entire development cycle form the development stage through QA to deployment).
  • Evaluate code state after it has been change (either because of design changes or bugs fixes). Sometimes changing code in one component causes others to fail.  In this case it’s in our interest to identify the exact failure point as fast as possible (by running all the tests that relate to the project we can get the project back to a good state).
  • Tracing the project state at any time either by the programmer or any of his superiors (for example it could be interesting to check the state before deployment).

Scope

The scope of the Unit Test Framework is testing code non related to user interfaces.

Design Goals

The design goals of the Unit Test Framework specification are:

  • Unify the way that programmers test their code before release.
  • Test the code under multi-threading stress.
  • Test the code under different security permission restrictions.
  • Assess the code results (success / fail).
  • Assess the code performance (execution time).
  • Having a way of assessing code on demand (even long after its release).
  • Having both immediate assessment and assessing reports.
    • Immediate assessment will assess a single test execution.
    • Assessment report will assess any history test executions.
  • Support for saving the execution results.

Logical

Declarative Test Expectation

Any of the programmer's tests must be tagged by some declarative test expectation (i.e. what does the programmer expect from the test).  This expectation is used by the Unit Test Framework assessment logic.

Such declarations will contains:

  • Whether the programmer expects the code to succeed or fail.
  • If he’s expecting failure he must specify the exception type that he’s expecting to be thrown.
  • Under which permission restriction should the code be assessed, meaning that the code should be assessed under full trust and under partial trust while each of the permission restriction is denied.

Assessing Test Execution

Test execution is assessed as success under the followings cases:

  • The test execution hasn’t thrown any exception and it it declared as success by the programmer.
  • The test execution has thrown an exception that matches the exception type that the programmer expects to be thrown by the test (as specified in the declaration) and the test declared as expected to fail.

The execution is assessed as failed under the following cases:

  • The test execution has thrown an exception while the declaration expected success.
  • The test execution has thrown an exception which does not match the declared expected exception type and the declared expected failure.
  • The test execution has succeeded while the declaration expects failure.

Multi threading Stress

Using asynchronous execution allows the Unit Test Framework to assess the execution under a stressed multi-threaded environment by executing multiple instance of the test simultaneously.

Storing The Assessments Results

The results are stored in a database repository either by demand or automatically.

  • Storing by demand will occur by responding to the actor save command.
  • Automatic storage will be performed if the memory utilization is raised above some limit percent in order to avoid out of memory exceptions.

Threads Controls

Because of the asynchronous nature of the Unit Test Framework a singleton thread manager handles the followings tasks:

  • Creating a new thread.
  • Aborting all test execution threads.
  • Suspending all test execution threads.
  • Resuming all test execution threads.

Performance Counters

Performance counter manager will wrap the .NET performance counters in order to expose the relevant counters in an optimized way.

.Net Expert

Sample screenshot

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


Written By
Chief Technology Officer Wise Mobility
Israel Israel
C.T.O and co-founder at wise-mobility l.t.d
www.wisemobility.com
http://bnaya.blogspot.com/

Comments and Discussions

 
QuestionNUnitASP? Pin
jasongb18-Nov-04 3:53
jasongb18-Nov-04 3:53 
AnswerRe: NUnitASP? Pin
Bnaya Eshet19-Nov-04 3:37
Bnaya Eshet19-Nov-04 3:37 
GeneralGood job Pin
Steven Campbell13-Apr-04 7:54
Steven Campbell13-Apr-04 7:54 
GeneralRe: Good job Pin
Jonathan de Halleux13-Apr-04 7:58
Jonathan de Halleux13-Apr-04 7:58 
Steven Campbell wrote:
They are all too simplistic, and do not provide the structural support that programmers need to be able to write and use unit tests effectively

What do you think of this one : http://mbunit.tigris.org[^] ?


Jonathan de Halleux - www.dotnetwiki.org -
MbUnit - QuickGraph - NCollection
GeneralRe: Good job Pin
Steven Campbell13-Apr-04 11:46
Steven Campbell13-Apr-04 11:46 
GeneralRe: Good job Pin
Jonathan de Halleux13-Apr-04 22:40
Jonathan de Halleux13-Apr-04 22:40 
GeneralRe: Good job Pin
Bnaya Eshet14-Apr-04 9:21
Bnaya Eshet14-Apr-04 9:21 
QuestionSave and Report not working? Pin
vplsek11-Mar-04 5:50
vplsek11-Mar-04 5:50 
AnswerRe: Save and Report not working? Pin
Bnaya Eshet11-Mar-04 6:23
Bnaya Eshet11-Mar-04 6:23 
QuestionHow it work??? Pin
PabloAbrahamArgentina6-Feb-04 4:06
PabloAbrahamArgentina6-Feb-04 4:06 
Answer[assembly: UTAssemblyBehavior (...)] Pin
Bnaya Eshet7-Feb-04 7:23
Bnaya Eshet7-Feb-04 7:23 
GeneralSome thoughts Pin
Marc Clifton29-Jan-04 4:08
mvaMarc Clifton29-Jan-04 4:08 
GeneralRe: Multi Threaded Stress testing Pin
Bnaya Eshet29-Jan-04 20:04
Bnaya Eshet29-Jan-04 20:04 
GeneralAdded featues Pin
Bnaya Eshet28-Jan-04 11:10
Bnaya Eshet28-Jan-04 11:10 
GeneralMemory Tracking Pin
Jonathan de Halleux28-Jan-04 4:55
Jonathan de Halleux28-Jan-04 4:55 
GeneralRe: Memory Tracking Pin
Bnaya Eshet28-Jan-04 11:02
Bnaya Eshet28-Jan-04 11:02 
GeneralFix article format Pin
Jonathan de Halleux28-Jan-04 4:54
Jonathan de Halleux28-Jan-04 4:54 
GeneralRe: Fix article format Pin
Bnaya Eshet28-Jan-04 11:06
Bnaya Eshet28-Jan-04 11:06 
QuestionHow do I use it? Pin
Mark Focas27-Jan-04 11:53
Mark Focas27-Jan-04 11:53 
AnswerRe: How do I use it? Pin
Bnaya Eshet27-Jan-04 20:53
Bnaya Eshet27-Jan-04 20:53 
GeneralRe: How do I use it? Pin
Bnaya Eshet27-Jan-04 23:27
Bnaya Eshet27-Jan-04 23:27 
GeneralToo much typing Pin
Thomas Eyde27-Jan-04 8:01
Thomas Eyde27-Jan-04 8:01 
GeneralRe: Too much typing Pin
Bnaya Eshet27-Jan-04 20:50
Bnaya Eshet27-Jan-04 20:50 
GeneralRe: Too much typing Pin
Thomas Eyde28-Jan-04 9:48
Thomas Eyde28-Jan-04 9:48 
GeneralWhere are the source files and demo Pin
Antonio Barros27-Jan-04 1:47
professionalAntonio Barros27-Jan-04 1:47 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.