Click here to Skip to main content
Rate this: bad
good
Please Sign up or sign in to vote.
See more: .NETCF .NET
I've noted with interest that there seems to be a lack of open source tools for .NET test coverage analysis. I'm pretty comfortable with .NET, but off the top of my head, I can't figure out how someone could do it without making their own version of the runtime or tampering with the IL emitted when code is compiled (which sounds like either great resume material, insanity, or possibly both).

I'm just kind of curious, but I'm not sure even how this sort of thing is typically done.
Posted 23-Jul-09 16:15pm
gantww1.3K
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

Easier than tampering with the IL is tampering with the source code. I used to do formal unit testing and I don't know if it's a formal term or not but we called it "instrumenting a file" and it's a bit of tedium that automated processes are made for. The trick is to modify the source without actually changing the control flow of the code. Take the following function:
private void MyMethod()
{
    int x = SomeFunction();
    int y = SomeOtherFunction();

    if (x > y)
    {
        // do something
    }
    else
    {
        // do something else
    }
}
There are 5 bits of code you need to ensure are executed, so for each place insert a boolean:
static bool flag1 = false;
static bool flag2 = false;
static bool flag3 = false;
static bool flag4 = false;
static bool flag5 = false;

private void MyMethod()
{
    flag1 = true; // verify that the method even gets called

    int x = SomeFunction();

    flag2 = true; // if SomeFunction() throws an unhandled exception this won't be executed

    int y = SomeOtherFunction();

    flag3 = true; // same thing with SomeOtherFunction()

    if (x > y)
    {
        flag4 = true; // verify the true case
        // do something
    }
    else
    {
        flag5 = true; // verify the false case
        // do something else
    }
}
Then you can run your test cases and check the value of your bools to make sure that all of your code was executed. Note that the control flow of the code hasn't changed, we've just added stuff to verify that all bits are executed. The difficulty comes in when you start taking into account the endless variations of source code. For example:
private void MyMethod()
{
    int x = SomeFunction();
    int y = SomeOtherFunction();

    if (x > y)
    {
        // do something
    }
}
looks easier, but you still need to insert the else to the if when you instrument that method to prove that the if statement can be failed. If the if can't be failed then it's unnecessary (dead code) and shouldn't be there.


The biggest PITA to instrument (and test) is full MCDC coverage (google it). Take the following sample:
private void MyMethod(bool a, bool b)
{
    if (a || b)
    {
        // do something
    }
}
There's a multi-conditional if statement there, and you need to prove that both conditions are necessary. Why? Well, suppose hypothetically that a is always true. Then the statement will always evaluate to true and there's no point in checking either a or b. The same thing happens if b is always true, so you need to instrument that in such a way to prove that both a and b can be false independently of each other. To verify full coverage that one statement turns into something like this:
static bool flag1 = false;
static bool flag2 = false;
static bool flag3 = false;
static bool flag4 = false;

private void MyMethod(bool a, bool b)
{
    flag1 = true;

    if (a && !b)
    {
        flag2 = true;
        // do something
    }
    else
    {
        // a is definitely false 
        if (b)
        {
            flag3 = true;
            // do something
        }
        else
        {
            flag4 = true;
            // don't forget the implicit else
        }
    }
}
and that 's just a simple multi-conditioned statement! Try figuring out how to break this down:
if (a || ((b && c) || d) || (x > 2))
{
    // we're all guilty of this at least once :)
}
Hopefully you either don't care about such things or you can afford to buy a tool to do this for you. Trust me, it will drive you crazy.




  Permalink  
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 2

The .NET Framework has a profiling API available for just such a thing. Here's a nice introductory article.[^].

  Permalink  
  Print Answers RSS
0 Marcin Kozub 265
1 Richard MacCutchan 239
2 OriginalGriff 208
3 Sergey Alexandrovich Kryukov 185
4 Praneet Nadkar 167
0 OriginalGriff 7,913
1 Sergey Alexandrovich Kryukov 7,232
2 DamithSL 5,604
3 Manas Bhardwaj 4,986
4 Maciej Los 4,865


Advertise | Privacy | Mobile
Web02 | 2.8.1411023.1 | Last Updated 23 Jul 2009
Copyright © CodeProject, 1999-2014
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100