Unit Test, TDD, and some limitations

I heard of unit test and TDD for the first time since 2001 when I was studying Extreme Programming and reading those XP books. I endorsed unit tests ever since, with DUnit. At the beginning I had thought that unit test was originated from Delphi. Before I heard of unit test, I had been doing a lot tests during development, and I built some little console apps or desktop apps as test harnesses that tested small portions of the applications in order to make sure they were all working fine. Often these small portions were implemented in a Unit. Yes, each Delphi source file is called Unit. At the end of a project I developed, around 20% to 30% of codes were written for testing, and thrown away then, since running these little tests manually was costly, and the integrated application itself could then become a fairly good test harness.

TDD has been becoming popular since mid 2000s, partly because Windows shops started having MSTest along with Visual Studio Unit Testing Framework since VS 2005.

TDD is not a standalone software development methodology, but could be a part of Extreme Programming or Agile, or any methodology. To me, TDD is more of an attitude toward software development, rather than a methodology or practice. Software is the work of wetware, our brain, which has limited power in remembering complex structures, long procedures and vast amount of information. Classic waterfall, Extreme Programming and Agile etc. are all helping our brain to handle the complexity. With TDD, we build little components, and test them to ensure they are working, and then we move on, and the tests we created have then become guardian angels against breaking changes.

TDD is certainly not a silver bullet for Quality Assurance, and has some shortcomings, as documented in various software engineering text books. It is unpleasant if not dangerous that TDD or unit tests become an “official” procedure for any modification during maintenance.

After fixing some vulnerabilities in codes, for example, resource leaks, do you think you could write a unit test case against such a modification?

My answer is NO. Why?

In Extreme Programming or Agile, during refactoring, you should not alter business logic, and may not even care about business logic. Thus you don’t know what to assert against the business logic, nor you have anything to assert against resource leaks. If you have already a test case against the function that generate resource leaks, good luck, since you could run the test case after fixing resource leaks. If you don’t have a test case, you might need to write one after studying the business logic of the function. However, for some legacy codes, writing a test case might be very costly if not entirely impossible, because of dirty designs, for example, strong and rich couplings. Writing a test case might require cascaded refactoring and mocking, eventually destroy the purpose and the benefits of unit tests.

We need to balance the risk of breaking changes, and the risk of not fixing vulnerabilities.

Nevertheless, we still need to test against the fixes, however, not necessarily through unit tests, or other test activities defined by TDD.