 |
|
 |
Looking forward to more in-depth article. I'd be interested in how you'd use either TestDriven.NET or NUnit to test presentation layers (GUI) in VB.NET application. This drives us crazy, but we feel it gives us a better product. We also feel that we need to do GUI testing better (ie, faster). Any insights would be appreciated.
Steve
|
|
|
|
 |
|
 |
Hi Steve,
I'm completely in agreement that we need better presentation-layer testing.
I am actually cooking up a few articles describing NUnitASP refinements including automatically launching an instance of the Cassini web server from the base class.
One of our projects right now could benefit from presentation layer functional testing. Unfortunately, the presentation layer technology always seems to be ahead of the testing tools. Just as the stock .NET controls are almost testable, now we have Ajax to contend with. If we ever get JavaScript unit testing frameworks integrated, I'm sure the technology will have moved on again.
ASP.NET also makes it difficult to unit test for two reasons: 1) ViewState and single form tag restriction; 2) nested control naming conventions. Both of these features mean it is difficult to change much about your layout without affecting the unit tests. You'd have to scrap ASP.NET for a more testable framework to really solve those problems I think. Look at Rails, for example. Because you control GET/POST variable names and no ViewState, it's easy to make functional calls into the web controller layer. Then all you need to do is a visual and functional inspection of the presentation itself to ensure that it correctly calls the controller. That is a much more manageable way to go, but you lose the component-oriented value found in .NET. Of the two choices, I know components is the right business answer but sometimes Rails-like frameworks sure seem refreshingly simple.
Ben
|
|
|
|
 |
|
 |
I can't begin to imagine the issues with testing in ASP.NET; we don't use ASP, just WinForms, tied to an Access or SQL Server database.
We've looked at NUnitForms some time ago, but came away with the feeling that it was "not ready for prime time". We're planning on looking at it again in the near future.
We're trying to use MVC pattern where ever we can in our presentation layer and this seems to make testing the GUI a little bit easier, but it still is a pain. Still have to wrestle with Public/Private scoping, raising/capturing events, etc.
Steve
|
|
|
|
 |
|
 |
sgclark wrote: I'd be interested in how you'd use either TestDriven.NET or NUnit to test presentation layers (GUI) in VB.NET application. This drives us crazy, but we feel it gives us a better product. We also feel that we need to do GUI testing better (ie, faster). Any insights would be appreciated.
dnrTV (Dot Net Rocks) has a show on the subject of unit testing GUIs: http://www.dnrtv.com/default.aspx?showID=14[^]
It is a follow on from show's 10 and 11, so you might need to watch them too in order to pick up the foundation.
|
|
|
|
 |
|
 |
Thanks for the tip, Colin. I'll check those out.
|
|
|
|
 |
|
 |
When I read the subtitle "Learn about unit testing from an expert", I really expected something a little less superficial. OK, a LOT less superficial. No examples of tests beyond a simple one, no examples of Init/Teardown, and an assertion that unit testing implementation is somehow bad, when the reality is that implementation errors are probably at the core of most program flaws.
In short, it didn't tell me ANYTHING beyond what I could find out within 1-2 clicks from the Homepage for Testdriven.Net or nUnit.
|
|
|
|
 |
|
 |
Hi Robert,
Thanks for the comments. I can tell you're eager to learn about unit testing and that's great, but you can also be more tactful. It takes time to write (and read) articles, so we have to be careful to give each other the benefit of the doubt.
1. This article is one in a series and you are probably looking for the next article. As I said to the other user who made a similar comment, it's very difficult to talk about unit testing without diving into a specific programming methodology, which I intentionally avoided in this short article. Some programmers want to know just how to use the unit test runner and do not wish to hear about any philosophies related to unit testing. At my company, there is a major difference between knowing how to use a tool (this article) and knowing when to use a tool (the article you want).
2. Did you need examples of setup/teardown beyond the description? You are the first programmer I have encountered who has asked for more than the description I provided in this article. I'm happy to add examples, just let me know what is confusing. Articles should not be longer than they need to be and there is just not that much to say about setting up a unit testing framework in VS2005.
3. I would like to hear more from you on "an assertion that unit testing implementation is somehow bad" because I am not sure what you mean. If this dives into a methodology or philosophy discussion, I will save it for the next article
4. You may have missed the last part of this article where I listed several tips on what makes a good unit test, independent of any methodology. If these are available on Testdriven.Net or nUnit sites, please send links because I think they are not.
Let me know if you have any questions.
Ben
|
|
|
|
 |
|
 |
Ben Allfree wrote: Did you need examples of setup/teardown beyond the description? You are the first programmer I have encountered who has asked for more than the description I provided in this article.
Everyone learns differently. Some people need the examples to back up the description, especially when the subject area is new to them.
|
|
|
|
 |
|
 |
Ben,
As a general rule, I always try to remain tactful in these posts, and apologize for my tone, but I personally felt the "lead-in" to the article set expectations that were not met. You are completely correct that I should give you the benefit of the doubt, as you have stepped up to the plate and that is generally to be commended. However, as an avid browser of this and other such sites, I truly find it frustrating when articles appear that promise big and deliver small, and perhaps I am wrongly labelling this article as such.
I'm really not trying to pick a fight, but when I dove into an article from an "expert", I guess I expected something more. I've only been implementing TDD for about 6-7 months now, so my hope was that I would learn some expert tips to enhance my usage. Now that I know this article was the beginning of a series, I look forward to reading more.
Point #2: My point was that examples of Init & Teardown, however simple, would have illustrated the purpose of these more than simply stating that "If you have setup code that should be run", or somesuch. Recursive definitions don't shed enough light on the topic, IMO. Even a statement like "such as initializing the database to a known starting point" (or something like that) would help clarify their usage without adding complexity. See Peter Provost's article on here as an example of the depth I'm discussing. Again, I look forward to more depth on these in a later article.
Point #3: What I inferred from your comment "Unit tests should validate and enforce business rules, not specific implementation" and "Therefore, implementation tests enforce implementation requirements, which is generally a Bad Idea." was that writing unit tests for the low-level implementation was bad. If there's a flaw in my understanding of those points, I'm willing to be corrected.
Point #4: You are correct that your suggestions at the end are not to be found on the TDD software sites. Perhaps I was unclear in my comments about the tutorial, as that was what I was referring to.
It's my opinion that your suggestions tend to fly in the face of prevailing usage of these tools, as you also mentioned. I truly welcome more discussion of the different mindsets there, as it certainly is an interesting change from the conventional wisdom. Your approach seems to advocate testing at a higher level than what is commonly thought of as a "unit". While these "high-level" tests are of course important, HOW you arrive at the high-level result is equally important, IMO. Your reasoning regarding implementation changes is correct to a point, but prudent usage of unit testing would require that these rewritten implementations also have rewritten test harnesses that serve to validate the new logic.
It has been my experience that implementation tests are INVALUABLE when performing regression tests, as things like OS changes, database changes, etc. can and will break a tested app all too frequently.
In summary, I apologize for the tone of my initial post, and hope that my comments can be viewed as helpful suggestions.
|
|
|
|
 |
|
 |
Thanks for the clarification Robert! I could tell you were an avid reader, nobody else would take the time to post.
I agree about recursive definitions. I'll think about how to improve that statement as well as how to provide some clear examples. Database initialization is a great topic. You will like the upcoming TDD article , I think.
#3 is a bold statement that could use more explanation. TDD helps to avoid tests that favor implementation over business rules, but many programmers think in code, thus their tests reflect that thought model. That is the Bad Idea of which I speak.
You and I may not initially agree on testing approaches, but I will try to make my case and explain why I disagree that implementation tests are invaluable for system integration (regression, db changes, etc as you mentioned).
The problems caused by implementation tests can best be seen in two areas.
The first area is comprised of teams that write unit tests after the code is written. Their tests almost always enforce what the code does instead of what it should do. Or more importantly, the tests focus on the current solution rather than the problem intended to be solved. Teams usually come to hate unit tests because the tests begin to prevent progress over time. They enforce implementation decisions that were made before the problem was fully understood, or by a less experienced programmer, or perhaps decisions just made in a hurry for a quick fix.
The second area where you can see the harm of implementation tests is more theoretical, but still clear. First, go read the famous Fred Brooks article about No Silver Bullets. In it, he speaks about two types of complexity: essential and accidental. He uses the terms to differentiate between complexity that is inherent in the problem to be solved versus complexity rooted in the current tools and technology used to solve the problems. If you agree with his analysis as I do, you will classify the complexity you see into those two buckets, or maybe even subdivide from there. Essential complexity is what you want to capture in testing. Accidental complexity is what you want to *cover through* testing. So while I am still advocating 100% code coverage with tests, I am advocating that assertions and test methods should reflect the essential patterns and complexity of the business problem rather than being concerned about testing every class directly. In the perfect test harness, you can refactor and otherwise change your implementation without touching your tests.
Letting go of implementation tests is a very uncomfortable proposition for many developers. If your tests cover the business problem, it will naturally test the necessary aspects of implementation. For example, testing that you can connect to the database does nothing to advance your solution that wouldn't also be covered by a business-oriented test such as "customers who update their email address are sent a confirmation message to the new address".
Your tests should be runnable wherever you deploy your application. If you can get that working, system integration is not so difficult.
Ben
|
|
|
|
 |
|
 |
All great points. Our approaches seem to be converging the more we look at this.
My approach (let's call it "quasi-Agile" just to see who's paying attention) is to not only design from the top down, but to implement in this fashion as well. Consequently, a "slice" of the design may start with a stubbed-out Customer object, followed by stubbing out the methods surfaced during analysis. These stubbed (actually nMocked these days) implementations (my usage, not yours) would get Unit Tests designed at that point, so we're in agreement about the focus on the high-level testing. Successively drilling down into the design to the "nuts and bolts" then occurs in the context of the high-level design, not the reverse as is so often the case.
I also agree 100% about the misuse of Unit testing. Too many new converts to TDD do in fact write their tests to prove their code works, rather than using them as a "goal" for their code to satisfy. For me, it HAS to be used in a true TDD (emphasis of Test-DRIVEN) fashion, where the tests are designed as "use case validators" as you suggest.
BTW - Brooks is, and has been for quite a while, a constant reference for me.
Looking forward to the rest of the series.
|
|
|
|
 |
|
 |
There are definitely occasions where I want to test/enforce implementation. For example, right now I'm fighting with some serialization logic and once I get it pinned down I don't want anybody to mess with it.
Another example is a situation where I need to test that a given token returns the correct media stream from a web service. I guess one could argue that tokens are business concepts because they describe how things are delivered to users or customers, but that's a stretch.
So there are situations where there's no choice but to test implementation code. Often I find that just being mindful of the difference can influence how a test is written.
|
|
|
|
 |
|
 |
Dear Ben,
I think that article about unit testing should be more specific, and it'll be better if you describe unit testing, not used tools (especially commercial ). For example, why i should prefer TestDriven.NET for $95, instead of NUnit, which is free ?
I like VS integration, but in this case it's not required.
So, waiting for more in-depth unit testing article, with good examples of what can be done using this technology.
I'm using it, and have a good understanding of what can be done, but not newbies - they won't get this information from this article
Sorry,
Stanislav
|
|
|
|
 |
|
 |
Hey Stanislav,
Thanks for the great points, I will update the article.
To answer your specific concerns:
The version of TestDriven.NET intended for use in this article is the "personal" edition, which is free. It is much more convenient than NUnit in my opinion because it is integrated with Visual Studio. NUnit does notice when you recompile, so aside from switching windows it is nearly as convenient.
As for unit testing techniques and methodologies, I understand exactly what you mean. I am writing a very in-depth article on test-driven development that makes use of this unit testing foundation. It's hard to talk much about unit testing without touching on at least one programming methodology and I wanted this article to stay methodology-free
Let me know if you have any questions.
Ben
|
|
|
|
 |
|
 |
Thanks !
So, waiting for programming methodology article (agile ?)
I think that in-depth test-driven development article can be split into several parts, at least 3. Just because it's simple to read and to understand, when you see just a main points, step-by step.
I have experience in phpUnit, so can add some info to your article, if you need to include php things, just drop me a line.
Best regards,
Stanislav
|
|
|
|
 |
|