Click here to Skip to main content
Click here to Skip to main content

The best Unit Test Framework (UTF) in town

, 26 Jan 2004
Rate this:
Please Sign up or sign in to vote.
Unit Test Framework that support security context and multi threading stress

Sample screenshot

Introduction

This is not the only free Unit Test in town, but this one has some important advantages over other ones:

  • This Unit Test can check your code against different security context
  • It can test your code under multi-thread stress
  • It has an advanced GUI includes agents and performance control
  • It saves the testing result for farther analysis.

The idea behind this Unit Test Framework (UTF) is derived from Nunit but the implementation is different.  If you already invested in Nunit you can still use your Nunit test with this UTF.  All you have to do is:

  • Mark your test with the UTF attributes.
  • Call your [Setup] method from the constructor.

Currently the UTF support 2 method signatures:

  • object MethodName()
  • void MethodName()

In order to mark your code as testing code you should put the assembly under the UTF execution folder or sub folder.

You must mark your assembly by the following attribute:

using WiseMobility.UnitTest.Attributes;
[assembly: UTAssemblyBehavior
("Permissions Checks", TextColor="yellow", Enable=true)]

At the class level you must use the following attribute:

[UTTypeBehavior ("Reflection Manip", TextColor="DarkBlue", Enable=true)]

At the method level you must use the following attribute:

[UTMethodBehavior ("OS Version", ExpectedResult=EnmExpectedResult.eSuccess,
TextColor="Green",
Enable=true)]

or

[UTMethodBehavior ("Fail as expected", ExpectedResult=EnmExpectedResult.eFail,
ExpectedException=typeof(DivideByZeroException), TextColor="SlateBlue")]

You can mark the method success expectation and if you expect failure you should mark the expected exception.  For more information see the following specification.

Other optional attributes are:

  • Performace tracing attributes (this attribute an be attached to assembly/class/method):
[assembly: UTPerfMon (EnmPerformanceType.eCPU_Percent)]
[assembly: UTPerfMon (EnmPerformanceType.eMemory_Percent)]
  • Security context attributes (this attribute an be attached to assembly/class/method):
[assembly: UTSecurityPermissionAttribute (enmSecPermissionOptions.eReflection,
Comment="Some Comments")]
  • You can mark any class or method under the assembly for ignoring the higher level of security context attribute:
[UTSecurityPermissionAttribute (enmSecPermissionOptions.eReflection,
Behavior=enmLimitIgnore.eIgnore)]

For more details simply go through the following UTF specification.  The specification will give you general view but you more then invited to research the code.

Currently the report is not implemented, but you can use any report mechanism on top of the UTP database (SQL server) to generate your own report.

If you have question or have developed some improvement that you want to share with other you can reach me at the following mail: bnayanet@hotmail.com

Enjoy J

Conceptual and Design Goals

Introduction

  • Unit testing is the main method that the developer conducts in order to ensure quality code shipping.
  • The tests should be valid under different security privilege, and under multi threaded stress.
  • The Unit Test Framework tool should increase the development process productivity by helping to develop more robust code.
  • The idea behind it is to find most of the bugs before the programmer release his code to the QA department or to his colleagues.

For more information of why using unit test framework see the following article:

The benefits of automated unit testing

Conceptual

The Unit Test Framework constructs a framework for running additional test and assesses the result.  The programmer is responsible for writing the tests. The Unit Test Framework is responsibility for assessing the results.

Sample screenshot

Motivation

The motivation of using the unit test framework is:

  • Construct consistent methods of testing code under different contexts (i.e. under restricted permission or multi threading stress), and to identify problems as soon as possible in order to reduce the development process cost and increase productivity.
  • Ensure the code quality (the benefit of robust code will affect the entire development cycle form the development stage through QA to deployment).
  • Evaluate code state after it has been change (either because of design changes or bugs fixes). Sometimes changing code in one component causes others to fail.  In this case it’s in our interest to identify the exact failure point as fast as possible (by running all the tests that relate to the project we can get the project back to a good state).
  • Tracing the project state at any time either by the programmer or any of his superiors (for example it could be interesting to check the state before deployment).

Scope

The scope of the Unit Test Framework is testing code non related to user interfaces.

Design Goals

The design goals of the Unit Test Framework specification are:

  • Unify the way that programmers test their code before release.
  • Test the code under multi-threading stress.
  • Test the code under different security permission restrictions.
  • Assess the code results (success / fail).
  • Assess the code performance (execution time).
  • Having a way of assessing code on demand (even long after its release).
  • Having both immediate assessment and assessing reports.
    • Immediate assessment will assess a single test execution.
    • Assessment report will assess any history test executions.
  • Support for saving the execution results.

Logical

Declarative Test Expectation

Any of the programmer's tests must be tagged by some declarative test expectation (i.e. what does the programmer expect from the test).  This expectation is used by the Unit Test Framework assessment logic.

Such declarations will contains:

  • Whether the programmer expects the code to succeed or fail.
  • If he’s expecting failure he must specify the exception type that he’s expecting to be thrown.
  • Under which permission restriction should the code be assessed, meaning that the code should be assessed under full trust and under partial trust while each of the permission restriction is denied.

Assessing Test Execution

Test execution is assessed as success under the followings cases:

  • The test execution hasn’t thrown any exception and it it declared as success by the programmer.
  • The test execution has thrown an exception that matches the exception type that the programmer expects to be thrown by the test (as specified in the declaration) and the test declared as expected to fail.

The execution is assessed as failed under the following cases:

  • The test execution has thrown an exception while the declaration expected success.
  • The test execution has thrown an exception which does not match the declared expected exception type and the declared expected failure.
  • The test execution has succeeded while the declaration expects failure.

Multi threading Stress

Using asynchronous execution allows the Unit Test Framework to assess the execution under a stressed multi-threaded environment by executing multiple instance of the test simultaneously.

Storing The Assessments Results

The results are stored in a database repository either by demand or automatically.

  • Storing by demand will occur by responding to the actor save command.
  • Automatic storage will be performed if the memory utilization is raised above some limit percent in order to avoid out of memory exceptions.

Threads Controls

Because of the asynchronous nature of the Unit Test Framework a singleton thread manager handles the followings tasks:

  • Creating a new thread.
  • Aborting all test execution threads.
  • Suspending all test execution threads.
  • Resuming all test execution threads.

Performance Counters

Performance counter manager will wrap the .NET performance counters in order to expose the relevant counters in an optimized way.

.Net Expert

Sample screenshot

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

About the Author

Bnaya Eshet
Chief Technology Officer Wise Mobility
Israel Israel
C.T.O and co-founder at wise-mobility l.t.d
www.wisemobility.com
http://bnaya.blogspot.com/

Comments and Discussions

 
QuestionNUnitASP? Pinsussjasongb18-Nov-04 3:53 
AnswerRe: NUnitASP? PinmemberBnaya Eshet19-Nov-04 3:37 
GeneralGood job PinmemberSteven Campbell13-Apr-04 7:54 
GeneralRe: Good job PinmemberJonathan de Halleux13-Apr-04 7:58 
GeneralRe: Good job PinmemberSteven Campbell13-Apr-04 11:46 
GeneralRe: Good job PinmemberJonathan de Halleux13-Apr-04 22:40 
GeneralRe: Good job PinmemberBnaya Eshet14-Apr-04 9:21 
QuestionSave and Report not working? Pinmembervplsek11-Mar-04 5:50 
AnswerRe: Save and Report not working? PinmemberBnaya Eshet11-Mar-04 6:23 
QuestionHow it work??? PinmemberPabloAbrahamArgentina6-Feb-04 4:06 
Answer[assembly: UTAssemblyBehavior (...)] PinmemberBnaya Eshet7-Feb-04 7:23 
GeneralSome thoughts PineditorMarc Clifton29-Jan-04 4:08 
GeneralRe: Multi Threaded Stress testing PinmemberBnaya Eshet29-Jan-04 20:04 
GeneralAdded featues PinmemberBnaya Eshet28-Jan-04 11:10 
GeneralMemory Tracking PinmemberJonathan de Halleux28-Jan-04 4:55 
GeneralRe: Memory Tracking PinmemberBnaya Eshet28-Jan-04 11:02 
GeneralFix article format PinmemberJonathan de Halleux28-Jan-04 4:54 
GeneralRe: Fix article format PinmemberBnaya Eshet28-Jan-04 11:06 
QuestionHow do I use it? PinmemberMark Focas27-Jan-04 11:53 
AnswerRe: How do I use it? PinmemberBnaya Eshet27-Jan-04 20:53 
GeneralRe: How do I use it? PinmemberBnaya Eshet27-Jan-04 23:27 
GeneralToo much typing PinmemberThomas Eyde27-Jan-04 8:01 
GeneralRe: Too much typing PinmemberBnaya Eshet27-Jan-04 20:50 
GeneralRe: Too much typing PinmemberThomas Eyde28-Jan-04 9:48 
GeneralWhere are the source files and demo PinmemberAntonio Barros27-Jan-04 1:47 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140709.1 | Last Updated 27 Jan 2004
Article Copyright 2004 by Bnaya Eshet
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid