Test Driven Development

Test Driven Development (TDD) can be a very effective method to develop reliable and maintainable software. However, I have witnessed instances where the development process and results were from ideal because the tenets of TDD were not fully understood. I will provide a brief overview of TDD, which will include a description of the concepts, development process and potential benefits associated with TDD.

Concepts

Rapid Feedback During Development

The most basic goal of TDD is to provide the developer with the shortest development cycle possible. This is based on the concept that it is simpler and less expensive to find and fix defects, the closer you are to the point where the defect was introduced. This seems reasonable if you consider that you have all of the context and details for the change you just made floating around in your head, these extra details are forgotten over time.

Manage the Risk of Change

This rapid cycle of constant feedback that informs you of the quality level of each change. This process works best when you have a unit-test framework for your development environment; unit-test frameworks are an entirely different topic. For now, let's assume that it is easy to write and run all of the tests that you develop during a TDD session. Each test should be small, and only verify a tiny port of the code being developed. This is why it is important for it to be simple to create and run new tests.

Reduce Waste, Maximize Value

We want lots of tests, but no more than it takes to verify the code. As you are developing with this instant feedback cycle, you are able to focus on solving the problem at hand, and your array of tests that you are building the code upon, provides feedback on the overall system if you make a mistake. The result of your implementation should be a testable piece of logic that is minimal and correct. Hopefully building the feature with only statements that are required, and eliminating wasteful code that is often put in place for a cool future addition.

Red. Green. Refactor.

"Red. Green. Refactor." is the mantra of a developer working by TDD. If "Red. Green. Refactor." is not mentioned when a person describes TDD, they are most likely not describing it accurately. Simply put:

• Red: A test is written for a small non-existent feature, then it is run and inevitably fails.
• A set of tests that fail is called "Red"
• Green: The feature is implemented - Rerun the test and it passes.
• A set of tests that pass is called "Green"
• Refactor: Inspect the code, can it be improved?
• Is all of the functionality implemented?
• Can the implementation be simplified, especially duplicated code?

Keep the mantra in mind; it will help you focus on the process and the goals of TDD.

Let's go into a bit more detail with an example to demonstrate the details that are often glossed over. We'll walk through building a function to convert a temperature from Celsius to Fahrenheit. This should only take two or three iterations to get a complete function with the correct functionality. The detail of the process I demonstrate below is a bit exaggerated, however, the process itself scales very well far all types of development with a unit-test framework.

This is the starting point of the function, which will compile.

C/C++

float celsius_to_fahrenheit(float temperature) 
{ 
  return 0; 
}

The Approach

We will use with the simplest conversion that we know about Celcius, the freezing point of water, which is zero. This has the equivalent value of thirty-two in fahrenheit. Let's write a test that will verify this fact. I will use some imaginary verification MACROs to verify the code.

C/C++

void TestCelsiusAtZero() 
{ 
  ASSERT_EQUAL(32, celsius_to_fahrenheit(0) ); 
}

Now we initiate the tests. 

• TestCelsiusAtZero():          Fail

This is good, because now we have verified that we have written a test that fails. Yes, it is possible to write a test that never fails, which provides no value, and adds to our maintenance overhead. We have just achieved RED in our TDD development cycle. The next step is to add the feature code that will allow this test to pass. Keep in mind, we want simple.  Simple code is easy to understand and easy to maintain.

C/C++

float celsius_to_fahrenheit(float temperature) 
{ 
  return 32; 
}

Run the tests:

• TestCelsiusAtZero():          Pass

You might say, "Well that's cheating!"

Well is it? When we run our single test, it indicates we have done the right thing. TestCelsiusAtZero() is only verifying one facet of our function. This one facet is correct, for the moment. This means that we have reached the next step, GREEN.

It's time to analyze our solution, or REFACTOR. Did we add all of the functionality that is required to create a correct solution? Obviously not, Fahrenheit has other temperatures that 32°. The next test will verify a conversion of the boiling point of water, 100°C.

C/C++

void TestCelsiusAt100() 
{
  ASSERT_EQUAL(212, celsius_to_fahrenheit(100) ); 
}

This time there are two tests that are run.

• TestCelsiusAtZero():           Pass
• TestCelsiusAt100():            Fail

RED

With no changes to the implementation, we still expect the first function to pass and we have now verified our new test fails properly. It's time to add the implementation details to the conversion function to support our conversion from 100°C without breaking our first test.

C/C++

float celsius_to_fahrenheit(float temperature) 
{ 
  return (temperature == 0) ? 32 : 212; 
}

Run the tests:

• TestCelsiusAtZero():           Pass
• TestCelsiusAt100():            Pass

GREEN 

REFACTOR

Yes this is exaggerated, but hopefully you see the point. Let's select one more temperature, the average temperature of the human body, 37°C. Implement the test:

C/C++

void TestCelsiusAtHumanBodyTemp() 
{ 
  ASSERT_EQUAL(98.6f, celsius_to_fahrenheit(37.0f) ); 
}

Run the tests:

• TestCelsiusAtZero():           Pass
• TestCelsiusAt100():            Pass
• TestCelsiusAtHumanBodyTemp():  Fail

RED

Add the implementation for this test:

C/C++

float celsius_to_fahrenheit(float temperature) 
{ 
  return (temperature * 5.0f / 9.0f) + 32.0f; 
}

Run the tests:

• TestCelsiusAtZero():           Pass
• TestCelsiusAt100():            Pass
• TestCelsiusAtHumanBodyTemp():  Pass

GREEN

REFACTOR

Upon inspection this time, it appears that we have all of the functionality to complete the implementation of this function and meet the requirements. Can this function be further simplified? Possibly, by reducing 5.0 / 9.0 into a decimal. However, I believe that the fraction 5/9 is clearer. Therefore I will choose to leave it as it is, and declare done for this function.

Benefits

By default, the code is written to be testable and more maintainable. The code also contains unit-tests from the very beginning of development. This will help eliminate the undefined amount of debugging time that is usually required at the end of a project. As each change is added to the code, continue to add a test before making the change. This will ensure as much code as possible is covered by a test and continue to add value to your codebase.

Creating tests helps focus on smaller steps to develop and verify each part of code used to develop a feature. This increased focus can improve the productivity for the developer. Single paths through the code are considered for the addition of new tests and changes to the code. This leads to exceptional and error cases can be handled in a verifiable and useful manner. Finally, no more code than is necessary is developed. Code for potential "cool" features  in the future is left out because it may not be verifiable, or they would require more tests for something that is not required. All of these factors help contribute to a leaner and more correct codebase.

The tests become a sandbox and playground for new developers learning the project. They can make a change, run the tests, and see how the different parts are interconnected by what breaks. Undo the change, and poke into another spot. This is a much more fun and interactive approach to learning. Especially when compared to tediously reading through the code in your head. Alternatively, veteran developers of the project can experiment with their changes, and verify their hypothesis to determine if a change they are considering is the best choice or not.

Serendipity

An unexpected benefit I have experience many times is the early use of the objects and APIs by developing the tests. I have found it very helpful to be able to use the interfaces that I am developing as I design them. I have gotten mid-way through the development of an object and thought "This interface is sh*t!" What appeared to be perfectly reasonable as a header file on paper and design diagrams was actually a very cumbersome and clumsy object to use. Developing the tests gave me the chance to experience and discover this before I had completed my implementation.

Similarly, I have discovered errors in assumptions for the behavior of a feature-set critical to the system. This was for a public command interface where the command variables could be set or get one at a time. However, I discovered a set of parameters that were required to be set in a specific order, because even if the entire set of parameters would result in a valid configuration, the system could be commanded into a state where the configuration sequence would have to be started over if they were sent in the wrong order. Since I discovered this early enough in the development, I was able to raise this issue, and the team made the appropriate design changes to account for this issue. Had this been discovered in qualification testing, it would have been much more difficult to design and implement the change. Not to mention how much more time it probably would have required compared to discovery of this issue early in the schedule.

One last benefit I have experienced is the development of small, modular, and reusable components. The Test Driven Development process focuses on small tasks and incremental steps. This has helped me develop function and object interfaces that are more cohesive. They perform one task, and they do it very well. This lets me create a small collection of interoperable functions and components that I can use to compose more complex objects and functions that remain cohesive. Yet when I inspect their logic and tests, they still feel simple and easy to maintain. Basically, I have become much better at managing complexity with the use of Test Driven Development.

Drawbacks

Test Driven Development cannot be easily applied to all types of development. One example is User Interface testing. Full functional testing may be required of the application before many useful tasks can be verified. TDD therefore cannot be brought into the development early enough to benefit the entire project. However, it is important to note, no matter where you start in the development process, once pragmatic tests can be written, TDD can be applied to help guide additional changes.

The tests that are developed are part of the maintenance effort required by the project. This is also the case for any other type of development that creates tests. If the tests are not maintained, the value they provide is lost. It is just as important to write small maintainable tests, as it is to write small and maintainable production code. The simplest way to ensure the tests are maintained, is to make running the unit-tests part of the build process. The system will not create the output binary unless all of the tests pass. Continuous Integration is an excellent process to help manage this task in a pragmatic way.

Unit Test Process in General

Most of other drawbacks are shared with other processes that are based upon large sets of automated regression tests. It does not matter whether these are unit-tests or higher level component tests. The tests must be maintained. That is why it is important to write maintainable tests.

Management support becomes essential because of the previous drawback. A project management team that does not understand the benefits of the process may view the unit-tests as a waste of time that could be spent writing code. The entire organization that has direct input to the codebase must understand, believe in, and follow the process. Otherwise, the test-set will slowly fall into disrepair with incomplete patches of code that are vulnerable to risk when changes are made.

Misunderstandings

There are two common misinterpretations that I would like to bring to your attention to help you identify if you are moving down this path. This will help you self correct and maximize the potential benefit from following TDD practices.

Write All of Your Tests First

The fast feedback concept is lost when this is the statement that is emphasized: "Write the tests first". This has been misinterpreted as "write all of your tests upfront, and then write your code." Value can still be derived from a process like this, because much thought will be put into writing and compiling the tests, which hopefully carries over to the implementation, and the code will still be testable. However, I believe this makes the task of developing a solution much more difficult. Another layer of indirection has been created before the code is developed. The developed code must fit in this testing mold.

There is one thing that I have noticed about this interpretation of TDD that can make it successful in the short-term. That is the use of mock objects. Mock objects are a testing tool that allows behavioral verification of a unit, verifying that certain functions are called with the correct values, specified number of times and order. In this case, it is not as difficult to imagine what the functional implementation should be to test it, because you are thinking in terms of the behavior already as you develop the test.

Behavior Driven Tests are somewhat fragile, in that they use internal knowledge of the implementation to verify it is doing the correct action. If you use the same interface, return the same results, but use a different implementation, a Behavior Driven Test may possibly fail, where as a Data Driven Test will continue to function properly. In this case, the same data unit test could verify two different implementations of an object, while a separate test suite would need to be created for each implementation with behavior tests.

Yeah, But TDD Won't Find Bugs At Integration Testing

This misunderstanding has to do with unit testing in general, just as much as TDD itself. The comment is most often heard from a developer or manager that has not yet seen the value TDD can provide, let alone experienced it first-hand. The very first thing I think everyone should understand when they work at the unit test level is that the unit test is for the developer. It is written and maintained by the developer, and it is intended to give the developer near instant feedback on changes they make to the system.

The second thing to understand is a unit test does not find bugs. It is written to detect a bug that the developer has already found or imagined will exist. Integration testing is an entirely different level of testing. While developers are making changes to properly integrate their software, they can still use the unit tests to perform regression testing, however, bugs will still pop up. The developer should write a test to detect this defect before they make the changes to fix the problem. A developer or software tester found the defect in integration. Now a unit test will detect the defect exists before the next integration test cycle starts. Again, the unit tests and TDD are for the developers.

Conclusion

I discovered Test Driven Development out of frustration about four years ago when I was searching a brick-and-mortar bookstore for a better way to write software. I played around with it, read some books by Martin Fowler and Kent Beck, and I have been using TDD successfully ever since. When you try to explain TDD to someone else that has not seen the need for a better way than they are already used to, your efforts may fall on deaf ears. However, I have found sometimes the best way to convey the value of something, is to simply demonstrate it.

Test Driven Development is about three things:

1. Rapid Feedback: Red, Green, Refactor.
2. Manage the Risk of Change: Make sure each change adds value to your code
3. Reduce Waste, Maximize Value: Eliminate code that does not provide value, only write code that is necessary

This is in contrast to the developer that chooses to make an enormous number of changes over 3 weeks. Then, one day you hear them say in a status meeting "I'm going to start to try and get it to compile tomorrow."

Which method of implementation do you think has the greatest chance of success?