Click here to Skip to main content
15,917,645 members
Please Sign up or sign in to vote.
5.00/5 (3 votes)
See more:
I've noted with interest that there seems to be a lack of open source tools for .NET test coverage analysis. I'm pretty comfortable with .NET, but off the top of my head, I can't figure out how someone could do it without making their own version of the runtime or tampering with the IL emitted when code is compiled (which sounds like either great resume material, insanity, or possibly both).

I'm just kind of curious, but I'm not sure even how this sort of thing is typically done.
Posted

Easier than tampering with the IL is tampering with the source code. I used to do formal unit testing and I don't know if it's a formal term or not but we called it "instrumenting a file" and it's a bit of tedium that automated processes are made for. The trick is to modify the source without actually changing the control flow of the code. Take the following function:
private void MyMethod()<br />{<br />    int x = SomeFunction();<br />    int y = SomeOtherFunction();<br /><br />    if (x > y)<br />    {<br />        // do something<br />    }<br />    else<br />    {<br />        // do something else<br />    }<br />}
There are 5 bits of code you need to ensure are executed, so for each place insert a boolean:
static bool flag1 = false;<br />static bool flag2 = false;<br />static bool flag3 = false;<br />static bool flag4 = false;<br />static bool flag5 = false;<br /><br />private void MyMethod()<br />{<br />    flag1 = true; // verify that the method even gets called<br /><br />    int x = SomeFunction();<br /><br />    flag2 = true; // if SomeFunction() throws an unhandled exception this won't be executed<br /><br />    int y = SomeOtherFunction();<br /><br />    flag3 = true; // same thing with SomeOtherFunction()<br /><br />    if (x > y)<br />    {<br />        flag4 = true; // verify the true case<br />        // do something<br />    }<br />    else<br />    {<br />        flag5 = true; // verify the false case<br />        // do something else<br />    }<br />}
Then you can run your test cases and check the value of your bools to make sure that all of your code was executed. Note that the control flow of the code hasn't changed, we've just added stuff to verify that all bits are executed. The difficulty comes in when you start taking into account the endless variations of source code. For example:
private void MyMethod()<br />{<br />    int x = SomeFunction();<br />    int y = SomeOtherFunction();<br /><br />    if (x > y)<br />    {<br />        // do something<br />    }<br />}
looks easier, but you still need to insert the else to the if when you instrument that method to prove that the if statement can be failed. If the if can't be failed then it's unnecessary (dead code) and shouldn't be there.


The biggest PITA to instrument (and test) is full MCDC coverage (google it). Take the following sample:
private void MyMethod(bool a, bool b)<br />{<br />    if (a || b)<br />    {<br />        // do something<br />    }<br />}
There's a multi-conditional if statement there, and you need to prove that both conditions are necessary. Why? Well, suppose hypothetically that a is always true. Then the statement will always evaluate to true and there's no point in checking either a or b. The same thing happens if b is always true, so you need to instrument that in such a way to prove that both a and b can be false independently of each other. To verify full coverage that one statement turns into something like this:
static bool flag1 = false;<br />static bool flag2 = false;<br />static bool flag3 = false;<br />static bool flag4 = false;<br /><br />private void MyMethod(bool a, bool b)<br />{<br />    flag1 = true;<br /><br />    if (a && !b)<br />    {<br />        flag2 = true;<br />        // do something<br />    }<br />    else<br />    {<br />        // a is definitely false <br />        if (b)<br />        {<br />            flag3 = true;<br />            // do something<br />        }<br />        else<br />        {<br />            flag4 = true;<br />            // don't forget the implicit else<br />        }<br />    }<br />}
and that 's just a simple multi-conditioned statement! Try figuring out how to break this down:
if (a || ((b && c) || d) || (x > 2))<br />{<br />    // we're all guilty of this at least once :)<br />}
Hopefully you either don't care about such things or you can afford to buy a tool to do this for you. Trust me, it will drive you crazy.




 
Share this answer
 
The .NET Framework has a profiling API available for just such a thing. Here's a nice introductory article.[^].

 
Share this answer
 


CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900