This article presents a unique tool to run repeatable performance tests in an application developed by programmers. The NTime tool is very similar to NUnit tool - another unit testing tool, so the users of NUnit will find an almost similar GUI and functionality.
This is the first release of the tool that performs repeatable tasks to help managers, architects, developers and testers test an application against its performance. This version may not be very consistent and it may report errors under different situations. However, the tool was tested under many situations.
This section provides information for the application management team that wishes to use NTime tool to keep applications working at the desired performance level.
Program managers are welcome to specify application performance boundaries at the beginning of the application development process. They may also note which modules and functions of the application need to work and at which time frames. The performance specifications written by them may be helpful for developers when designing the application performance. Further, the NTime tool tests their performance specifications against real, developed applications. The tests performed by NTime tool may be reproduced at different time periods to see whether the application is giving the expected performance. But the main advantage of using this tool is that it enables managers to create configuration test files for developers and testers to give them a starting point on how to optimize the application.
Developers use NTime tool to test whether their application works in accordance with the performance specifications designed by managers or by themselves. They need to run the NTime tool when they aren't sure whether their rewritten code still works well. The NTime tool can also be made to remain open on the desktop even when the programmers develop, compile or debug their applications - every application that is built will be reloaded in NTime automatically and shown with the updated tests.
Testers are actively in the performance test stage. They run repeatable tests after every application's alpha release. Testers will review hundreds or thousands of tests and register them to the bug tracing applications.
NTime architecture is similar to that of NUnit tool. This section shows the features found in NTime tool.
Simultaneous compile and test time
NTime tool may still be running even when the compilation process of the application that we want to test is going on. After the application has been built, NTime will reflect all the changes, you just need to run the tests and wait for the performance test results whether they are accepted or rejected.
Actually, NTime's timings are based on high frequency timers that rely on different onboard chipsets' architecture. This difference generates more or less precision in the performed tests. The disadvantages, while profiling the tests, include system processes and CPU load. Even after closing the heavy loaded applications, there may be other bad things that can occur - like, NTime needs some CPU and threads to perform complex tests and therefore you can indicate about 10 microseconds overhead, so if you want to test high frequency running functions, then you need to write inside of the test code a "
for" loop statement i.e. 100 loops, and then test the result for the specified time multiplied by 100.
NTime tool is supported by two documentation files. The NTime Framework SDK will be your starting point to test your applications against performance tests by adding .NET attributes into your code.
NTime framework SDK
NTime Framework SDK is the often used SDK documentation. This help file shows you the available .NET attributes that you can use in your testing applications. Many attributes are similar to that of NUnit attributes, so if you've used NUnit tool before, then working with NTime will be quite easy.
NTime GUI SDK
After the installation of NTime, you will find the NTime source documentation. This will help you to add some functionality to NTime, but remember every change that you make to this tool is only for your private and non-commercial use. Read License.rtf file for further information about NTime licensing.
This NTime version is a GUI only application. In future, I plan to develop a console application to provide batch script usability. For further improvements, you may send comments to firstname.lastname@example.org with your suggestions to help me make the application work better. Bugs found in this release will be fixed in the next version, so the earlier found errors will be corrected as soon as possible. The source code of NTime will be soon available on the sourceforge.net website, so you can track the latest versions.
Using the code
Below, you can see a sample code which uses all the attributes available with NTime framework. See the NTime framework SDK documentation for more information:
public class TestClass
public void GlobalSetUp()
public void GlobalTearDown()
public void LocalSetUp()
public void LocalTearDown()
[TimerHitCountTest(300, Threads = 3,
Unit = TimePeriod.Second)]
public void WebRequest()
else if(System.Threading.Thread.CurrentThread.Name ==
[TimerDurationTest(20, Unit = TimePeriod.Millisecond)]
public void GameScreenFlicking()
// we want to calc game AI, 3d engine and
// other stuff to keep it running at 50Hz
// vertical screen sync.
[TimerDurationTest(10, Unit = TimePeriod.Millisecond)]
[TimerIgnore("This test is disabled.")]
public void SuperSortAlgorithm()
// code here will not be profiled until TimerIgnore
// flag was removed.
[TimerCounterTest("Process", "% Processor Time",
Threads = 1, InstanceName = "NTimeGUI",
MinimumValue = 0, MaximumValue = 50)]
public void ProcessorUsage()
// we want to see whether our application is
// optimized to work with minimal CPU usage.
for(int i = 0; i < 150; i++)
Points of interest
This tool contains some .NET technologies used inside the code, including application domains and .NET remoting - these technologies were introduced to allow simultaneous compile and run test functionality, so that the assemblies may be loaded into separate application domains which run shadowed assemblies, and so the original ones may be overwritten by the compiler. Also, the assemblies can be unloaded from the application domain anytime.
The second thing is the file monitoring which tracks file changes and refreshes the project test tree to reflect changes in the assemblies when a new DLL is deployed.
- Bug fixes:
- Tested methods were launched five times to average time, but the
LocalSetup method was launched only once. Now it is launching both during every test method call.
- Added named threads to detect which thread is executing the test method. Look at the example method '
- Bug fixes:
- Tested assemblies could not read the application configuration settings.
- NTimeSetup.exe installer failed when the source files were being selected for installation.
- New option dialog box allows you to specify the application configuration settings file.
- NTime.Framework.dll was promoted to GAC to allow for better shared reference to it.
- Updated NTime application with bug fixes and an additional console mode.
- Added source code and documentation download separately.
- First release of NTime tool.