Click here to Skip to main content
Click here to Skip to main content

How to Test Console Applications

, 14 Feb 2007
Rate this:
Please Sign up or sign in to vote.
This article demonstates a very simple way to automate testing of your console application

Introduction

All UI tools frequently have convenient command lines to perform automation. When you are working on a small application, you usually don't need a user interface at all.
Automated testing is very expensive, but when it goes around console applications, there is a way to avoid spending much time and effort on implementing automation. Welcome to the world of console testing.

Definitions

Solution, project and assembly are used in Visual Studio/.NET meaning.

  • Tested assembly/project – project referenced by testing assembly and needs to be unit tested
  • Testing assembly/project – project that performs unit testing of tested assembly

General Recommendation for Testing in Visual Studio

  • Keep all your output in the same folder. If you have two executable projects that are definitely not suitable for keeping together, probably it's time to split the solution.
  • If you have test data, keep it either in the resource of testing assembly or in a dedicated file copied to the output folder
  • If you have a dedicated file, the evident method to copy it is to use post-build event. For example:

    xcopy "$(SolutionDir)TestData\SampleExcelTable.xls" $(SolutionDir)bin\Debug /r /y 

Using the Code

So all you need is the usual test fixture either in Visual Studio Team Edition or NUnit.
I usually name those fixtures as ProgramTests and keep them in the same testing project as other unit tests.

There are some class members used:

  • TextWriter m_normalOutput; used to swap console output

Other two used to accumulate all messages retrieved during application execution are:

  • StringWriter m_testingConsole;
  • StringBuilder m_testingSB;

The standard test fixture methods used to replace output and snip error code are as follows:

  • TestFixtureSetUp – Set working folder and replace output with mock streams. Mock streams are usual string builders and can be used to analyze output. For example, if you know that your application should output message "in progress" you can check it after execution
  • TestFixtureTearDown – Restore normal output after working out whole test fixture
  • SetUp – Clears string builder before running each test
  • TearDown – Prints cached in string builder content to normal console. This is not a must, but it's frequently needed in development.
namespace MyConsoleApp.Tests
{
    /// <span class="code-SummaryComment"><summary></span>
    /// Testing application BuildAdvantage.Console.exe
    /// <span class="code-SummaryComment"></summary></span>
    [TestFixture]
    public class ProgramTests
    {

        TextWriter m_normalOutput;
        StringWriter m_testingConsole;
        StringBuilder m_testingSB;

        [TestFixtureSetUp]
        public void TestFixtureSetUp()
        {
            // Set current folder to testing folder
            string assemblyCodeBase = 
                System.Reflection.Assembly.GetExecutingAssembly().CodeBase;

            // Get directory name
            string dirName = Path.GetDirectoryName(assemblyCodeBase);

            // remove URL-prefix if it exists
            if (dirName.StartsWith("file:\\"))
                dirName = dirName.Substring(6);

            // set current folder
            Environment.CurrentDirectory = dirName;

            // Initialize string builder to replace console
            m_testingSB = new StringBuilder();
            m_testingConsole = new StringWriter(m_testingSB);

            // swap normal output console with testing console - to reuse 
            // it later
            m_normalOutput = System.Console.Out;
            System.Console.SetOut(m_testingConsole);
        }

        [TestFixtureTearDown]
        public void TestFixtureTearDown()
        {
            // set normal output stream to the console
            System.Console.SetOut(m_normalOutput);
        }

        [SetUp]
        public void SetUp()
        {
            // clear string builder
            m_testingSB.Remove(0, m_testingSB.Length);
        }

        [TearDown]
        public void TearDown()
        {
            // Verbose output in console
            m_normalOutput.Write(m_testingSB.ToString());
        }

Private method StartConsoleApplication is used to start a process in "unit tests". It waits until the console tested application that is running returns an error code of executed tested application. All output is saved in local member testingSB.

/// <span class="code-SummaryComment"><summary></span>
/// Starts the console application.
/// <span class="code-SummaryComment"></summary></span>
/// <span class="code-SummaryComment"><param name="arguments">The arguments for console application. </span>
/// Specify empty string to run with no arguments</param />
/// <span class="code-SummaryComment"><returns>exit code</returns></span>
private int StartConsoleApplication(string arguments)
{
    // Initialize process here
    Process proc = new Process();
    proc.StartInfo.FileName = "MyConsoleApp.exe";
    // add arguments as whole string
    proc.StartInfo.Arguments = arguments;

    // use it to start from testing environment
    proc.StartInfo.UseShellExecute = false;

    // redirect outputs to have it in testing console
    proc.StartInfo.RedirectStandardOutput = true;
    proc.StartInfo.RedirectStandardError = true;

    // set working directory
    proc.StartInfo.WorkingDirectory = Environment.CurrentDirectory;
    
    // start and wait for exit
    proc.Start();
    proc.WaitForExit();
    
    // get output to testing console.
    System.Console.WriteLine(proc.StandardOutput.ReadToEnd());
    System.Console.Write(proc.StandardError.ReadToEnd());

    // return exit code
    return proc.ExitCode;
}

The simplest test example is as follows:

[Test]
public void ShowCmdHelpIfNoArguments()
{
    // Check exit is normal
    Assert.AreEqual(0, StartConsoleApplication(""));

    // Check that help information shown correctly.
    Assert.IsTrue(m_testingSB.ToString().Contains(
    Program.COMMAND_LINE_HELP));
}

The first line checks that the application has worked out correctly and returned zero error code. The latter checks that console output contains command line help. It's invariant behavior for any pure console application. Empty quote specifies no arguments, otherwise you can put arguments as in usual command prompt: StartConsoleApplication("argument /switcher options").

WARNING: Arguments separated by space will be recognized as different, quote itself (") is neglected. But the current framework can be easily extended to accept multiple-word arguments.

Negative testing can be done like this: In this framework you have an opportunity to check if output messages are correct by using m_testingSB.ToString().Contains() construction.

[Test]
public void StartWithNonExistingPath()
{
    // Throw file not found. Exit code is 1
    Assert.AreEqual(1, StartConsoleApplication(
        "NonExistingMetricsTable.xls"));

    Assert.IsTrue(m_testingSB.ToString().Contains("File not found"));

    // Error code in console
    Assert.IsTrue(m_testingSB.ToString().Contains("11")); 
 
    // File name
    Assert.IsTrue(m_testingSB.ToString().Contains("NonExistingMetricsTable.xls")); 
}

Final Notes

Please pay attention to the following:

  • You don't need to reference the tested project to perform testing like this: You just use command line to run the executable.
  • It's acceptable both for .NET 1.1 and .NET 2.0 (Only disadvantage of .NET 1.1: you cannot reference the tested executable assembly to get its resources likewise Program.COMMAND_LINE_HELP).
  • You can use the code snippet attached to this article.
  • ProgramTest is quite slow and definitely will spoil your statistics of unit testing timing, but you can still keep it separately from other unit tests.
  • It's good to meet the same recommendation as for unit tests: keep program tests independent from the environment, relative path, order of running and so on.
  • Such program test easily can be involved in continuous integration.
  • Future development of this project plans to implement timeout for test to not let whole build process to halt.

History

  • 15th February, 2007: Initial post

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

atriplex

United States United States
No Biography provided

Comments and Discussions

 
GeneralMy vote of 5 PinmemberNikos1722-Feb-13 1:16 
SuggestionAnother option PinmemberVasil Trifonov18-Nov-12 22:18 
GeneralSource code missing PinmemberTBermudez15-Feb-07 4:50 
The link to the source code is broken.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140827.1 | Last Updated 15 Feb 2007
Article Copyright 2007 by atriplex
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid