|
|
Introduction
Test Driven Development is the
practice of (1) writing tests, (2) writing code that passes those tests, and (3)
then refactoring. This concept is becoming very popular in the .NET community
due to the quality assurance that it adds. While it is easy to test public
methods, the natural question emerges "How do I test protected and private
methods?"
This article will:
- Summarize some key points of the "Should-you-even-test private methods
debate"?
- Make the case that it is still useful to at least know how to test private
and protected methods - regardless of which side of the debate you're on.
- Provide the theory and downloadable code sample to demonstrate these testing
techniques.
Background Theory
Should You Test Private Methods?
A Google
search will show you that there's a lot of debate about using private
methods, let alone testing them. The table below summarizes some of the common
views of the pro and con for both issues.
|
Pro |
Con |
| Use private methods |
- Encapsulation - Private methods provide encapsulation, which makes
the code easier for the end client to use
- Refactoring - It is easier to refactor private methods because they
are never directly called by external clients, therefore changing the signature
won't break any method calls.
- Validation - Unlike public methods that must validate all input
because they are called by external clients, private methods are called safely
within the class and don't require the same rigor of validation - the inputs
should have been already validated in the public methods.
- Test Scope - Exposing every method as public greatly increases the
test scope. Private methods are used only how the developer intended them,
whereas public methods need to be tested for every possible case which required
a broader test scope.
|
- Didn't Refactor - If a class is complicated enough to merit private
members, then it should be refactored.
- Hides Functionality - Private methods (if designed correctly) provide
useful features that clients may want access to, so any private method worth
testing should really be public.
|
| Test private methods |
- Test Control - Private methods can contain complex logic and it
increases test control to be able to directly access the method and test it,
instead of needing to indirectly accessing it through a public method.
- Principle - Unit Testing is about testing the smallest functional
piece of code; private methods are functional pieces of code, therefore on
principle private methods should be testable.
|
- Already Covered - Only the public interface should be tested. Private
methods should already have thorough test coverage from being called by the
public methods that are tested.
- Brittle Code - If you refactor the code, and juggle around the
private methods, and if you had tests linked to those private methods, then you
need to juggle around your tests too.
|
There are bright and experienced people on both sides of the issue. So while
I have no intention or expectation of ending the "should I test private method"
debate, there is still value for both sides to know how to test them. Even if
you think that private methods should not be tested:
- Your opinion will be more influential if you can at least show that you are
able to test them, but choose not to (i.e. you're not saying "Don't test private
methods" simply because you don't know how).
- Having the option of testing non-public methods lets you see what really
works best for your team over time.
- As long as there are still some valid situations, it's worth having a
convenient way to test them.
Good Criteria and Inadequate Techniques
Andrew Hunt and David Thomas explain in their book, Pragmatic
Unit Testing in C# with NUnit, that good unit tests are ATRIP:
- Automatic
- Thorough
- Repeatable
- Independent
- Professional
There are three additional criteria that any testing overhead for
private/protected methods should meet:
- Transparency - Don't alter the System Under Test (SUT), such as
adding wrapper methods in the production code.
- Scope - Be able to run in both Debug and Release mode.
- Simplicity - Have minimal overhead, and hence be easy to change and
simple enough to introduce minimal risk.
Keeping these criteria in mind, there are several strategies that fall
short:
| Strategy |
Problem |
| Don't have any private methods. |
|
Use the directives #if DEBUG ... #endif
to wrap a public method which in turns wraps the private method. The unit tests
can now indirectly access that private method through the public wrapper. (This
is a method that I myself have used many times, and found it to be tedious and
non-object oriented). |
- This only works in Debug mode.
- This is procedural and not object-oriented. We would need to wrap the
individual methods in both the production code and unit tests.
- This alters the SUT by adding the public method
wrappers.
|
Use the [Conditional("DEBUG")] attribute
on public methods that wrap the private methods. |
- This only works in Debug mode.
|
| Create internal methods to access the private method;
then have a public test class elsewhere in the assembly that wraps those
internal methods with public ones. |
- This alters the release code by adding the internal hooks, ultimately making
the private methods accessible in production.
- This requires a lot of extra coding, and is hence
brittle.
|
Testing Protected Methods
A protected method is visible only to derived classes, therefore it is not
immediately available to a test suite. For example, suppose we wanted to test
the method from ClassLibrary1.MyObject: protected string MyProtectedMethod(string strInput, int i32Value)
{
return this.Name + ": " + strInput + ", " +
i32Value.ToString();
}
The book Pragmatic
Unit Testing in C# with NUnit explains one solution: make a derived class
MyObjectTester that inherits class MyObject, and then
create a public method TestMyProtectedMethod that wraps the
protected one. For example: public new string TestMyProtectedMethod(string strInput, int i32Value)
{
return base.MyProtectedMethod(strInput,
i32Value);
}
This approach is simple, yet meets all the criteria:
| Criteria |
Fulfillment |
| Transparency |
By using inheritance and putting the
MyObjectTester class in the UnitTests assembly, it
doesn't add any new code to the production assembly. |
| Scope |
Nothing in this approach depends on Debug-only
techniques. |
| Simplicity |
Although this approach requires a new derived class,
and an additional public wrapper method for each protected method, it is
object-oriented and type safe. |
Testing Private Methods
Testing private methods is a little more involved; but we can still do it
using System.Reflection.
You can use Reflection to dynamically access methods of a type, including both
instance and static private methods. Note that accessing private methods does
require the ReflectionPermission,
but that is not a problem for Unit Tests running on a development machine or
build server.
Suppose we wanted to test the private method MyPrivateMethod
from ClassLibrary1.MyObject:
private string MyPrivateMethod(string strInput, DateTime dt, double
dbl)
{
return this.Name + ": " + strInput + ", " +
dt.ToString() + ", " + dbl.ToString();
}
One solution is to create a UnitTestUtilities project with a helper class to
call the test method via reflection. For example, the download solution has the
following methods in UnitTestUtilities.Helper:
public static object RunStaticMethod(System.Type t, string strMethod,
object [] aobjParams)
{
BindingFlags eFlags =
BindingFlags.Static | BindingFlags.Public |
BindingFlags.NonPublic;
return RunMethod(t, strMethod,
null, aobjParams, eFlags);
}
public static object RunInstanceMethod(System.Type t, string strMethod,
object objInstance, object [] aobjParams)
{
BindingFlags eFlags = BindingFlags.Instance | BindingFlags.Public |
BindingFlags.NonPublic;
return RunMethod(t, strMethod,
objInstance, aobjParams, eFlags);
}
private static object RunMethod(System.Type t, string
strMethod, object objInstance, object [] aobjParams, BindingFlags eFlags)
{
MethodInfo m;
try
{
m = t.GetMethod(strMethod, eFlags);
if (m == null)
{
throw new ArgumentException("There is no method '" +
strMethod + "' for type '" + t.ToString() + "'.");
}
object objRet = m.Invoke(objInstance, aobjParams);
return objRet;
}
catch
{
throw;
}
}
Private method RunMethod takes in the necessary parameters that
Reflection needs to invoke a method, and then returns the value. It has two
public methods that wrap this: RunStaticMethod and
RunInstanceMethod for static and instance methods respectively.
Walking through RunMethod, it first gets the MethodInfo
from a type. Because we expect this to only be called for existing
methods, a null method triggers an Exception. Once we have the
MethodInfo, we can invoke the method given the instantiated object
(null for static methods) and the parameter array.
We could use this Utility in an NUnit test like so:
[Test] public void TestPrivateInstanceMethod()
{
string strExpected = "MyName: Hello, 5/24/2004
12:00:00 AM, 2.1";
ClassLibrary1.MyObject objInstance
= new MyObject("MyName");
object obj =
UnitTestUtilities.Helper.RunInstanceMethod(
typeof(ClassLibrary1.MyObject), "MyPrivateMethod",
objInstance, new object[3] {"Hello",
new DateTime(2004,05,24), 2.1});
string strActual = Convert.ToString(obj);
Assert.AreEqual(strExpected,strActual);
}
| Criteria |
Fulfillment |
| Transparency |
The only extra code we created -
UnitTestUtilities, is not shipped in production. |
| Scope |
Nothing in this approach depends on Debug-only
techniques. |
| Simplicity |
This approach can call any method with a single call.
Once you have the UnitTestUtilities, the only complication is
creating the correct parameters (method name, data types, etc...) for
RunInstanceMethod or RunStaticMethod. Because the
method is being dynamically called, the parameters aren't checked at compile
time. |
Conclusion
While there is a debate on whether or not to test private methods, at least
we have the ability to do so. We can test protected methods using inheritance to
create a derived TesterClass that wraps the base protected methods
with public ones. We can test private methods using Reflection, which can be
abstracted to a UnitTestUtility helper class. Both of these
techniques can help to improve test coverage.
| You must Sign In to use this message board. |
|
| | Msgs 1 to 25 of 26 (Total in Forum: 26) (Refresh) | FirstPrevNext |
|
|
 |
|
|
 |
|
|
 |
|
|
 |
|
|
I complied and started using this code, it's wonderful. I ran into one snag, which was quickly fixed by making an addition to your code. I often test for expected exceptions, most notably ArgumentNull and ArgumentOutOfRange. However since the Helper class uses Reflection any thrown exceptions within in the non-public methods get wrapped in a System.Reflection.TargetInvocationException and therefore my tests failed because the expected exception type was not met.
I suggest changing RunMethod to the following to make testing for Expected Exceptions a bit easier. You could apply this try-catch logic to all of your tests that expect exceptions, but it seems more efficient to let the Helper class handle it, and avoid the repetition.
private static object RunMethod(System.Type t, string strMethod, object objInstance, object[] aobjParams, BindingFlags eFlags) { MethodInfo m; try { m = t.GetMethod(strMethod, eFlags); if (m == null) { throw new ArgumentException("There is no method '" + strMethod + "' for type '" + t.ToString() + "'."); }
object objRet = m.Invoke(objInstance, aobjParams); return objRet; } catch(System.Reflection.TargetInvocationException tie){ if(tie.InnerException != null){ throw tie.InnerException; }else { throw tie; } } catch { throw; } }
Thanks for the artcicle. JB
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
I also found that I needed a change to work with non-public methods that have output or reference parameters. Changing the object[] parameter to ref param makes both possible.
public static object RunStaticMethod(System.Type t, string strMethod, ref object[] aobjParams) { BindingFlags eFlags = BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic; return RunMethod(t, strMethod, null, ref aobjParams, eFlags); }
public static object RunInstanceMethod(System.Type t, string strMethod, object objInstance, ref object[] aobjParams) { BindingFlags eFlags = BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic; return RunMethod(t, strMethod, objInstance, ref aobjParams, eFlags); }
private static object RunMethod(System.Type t, string strMethod, object objInstance, ref object[] aobjParams, BindingFlags eFlags) { MethodInfo m; try { m = t.GetMethod(strMethod, eFlags); if (m == null) { throw new ArgumentException("There is no method '" + strMethod + "' for type '" + t.ToString() + "'."); } object objRet = m.Invoke(objInstance, aobjParams); return objRet; } catch(System.Reflection.TargetInvocationException tie){ if(tie.InnerException != null){ throw tie.InnerException; }else { throw tie; } } catch { throw; } }
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Great article, very usefull!
But i did have a problem when I called a private function with different signatures, I got a System.Reflection.AmbiguousMatchException That is why i made another method to get the method using its name + signature.
When the System.Reflection.AmbiguousMatchException is thrown I try again now looping through the methods to find the method with the right signature. That is handled by the method GetMethodUsingSignature.
public static object RunPrivateMethod(System.Type t, string methodName, object objInstance, ref object [] aobjParams) { MethodInfo m = null; BindingFlags bindingFlags = BindingFlags.Instance | BindingFlags.NonPublic; try { m = t.GetMethod(methodName, bindingFlags); } catch (System.Reflection.AmbiguousMatchException ex) { //Console.WriteLine("AmbiguousMatchException for method: " + methodName + "\tTry again using GetMethodUsingSignature" ); m = GetMethodUsingSignature(t, methodName, aobjParams, bindingFlags); } if (m == null) { throw new ArgumentException("There is no method '" + methodName +"' for type '" + t.ToString() + "'."); } object objRet = m.Invoke(objInstance, aobjParams); return objRet; }
private static MethodInfo GetMethodUsingSignature(System.Type type, string methodName, object[] aobjParams, BindingFlags bindingFlags) { MethodInfo m = null; MethodInfo[] myArrayMethodInfo = type.GetMethods(bindingFlags); for(int i=0;i<myArrayMethodInfo.Length;i++) { MethodInfo myMethodInfo = (MethodInfo) myArrayMethodInfo[i]; if (myMethodInfo.Name == methodName) { if (myMethodInfo.GetParameters().Length == aobjParams.Length) { int counter = 0; ParameterInfo [] methodSignarure = myMethodInfo.GetParameters(); foreach (object parameter in aobjParams) { if (parameter.GetType() != methodSignarure[counter].ParameterType & (methodSignarure[counter].ParameterType.IsByRef & parameter.GetType() != methodSignarure[counter].ParameterType.GetElementType())) { break; } counter++; } m = myMethodInfo; break; } } } return m; }
-- modified at 7:28 Sunday 23rd April, 2006
|
| Sign In·View Thread·PermaLink | 5.00/5 (1 vote) |
|
|
|
 |
|
|
I think there can be more transparent approach: If we have an utility that generates a wrapper assembly (simillar to wrapper assembly for COM) for another assembly where all items became public (that uses reflection) and for tests we just use these (proxy) classes and methods.
This approach is also good if you want to have all tests in separate assembly. In such way you can transperently test internal classes.
If somebody knows such utility I'd like to use it.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
As far as I know the new Visual Studio 2005 Team Edition generates wrappers for private methods / classes. BTW: Internal classes can now be tested by using the 'InternalsVisibleTo' attribute in the AssemblyInfo class.
Michael
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
You are right. But i am still using .Net 1.1. Anyway my approach is the nearest to approach in VS 2005. So using this strategy will be good when you will migrate to 2005.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
I've actually developed such a tool and was surprised to find this thread when I was doing some more searches before I post my implementation.
http://www.futureware.biz/downloads/rpg20060122.zip
I haven't wrote any external documentation for it yet, but the XML documentation should be thorough enough. You should find the following in the zip file:
1. A console application which has the generator (rpg.exe). 2. A sample class library which has the internal classes. 3. A sample unit test assembly which has the generates proxies for these classes and some test cases.
All what you need to do to generate proxies for all internal classes in an assembly is:
rpg.exe myassembly.dll
This will generate files like MyFirstProxy.cs, MySecondProxy.cs (for classes MyFirst and MySecond).
Looking forward for your feedback.
Regards, Victor
Victor Boctor http://www.futureware.biz[^]
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Thanks for the great article!
Is there a similar way to test private classes?
I am writing a library which contains some private helper classes to hold internal data structures. It does not make sense to expose them publicly, but nevertheless, they should be tested. The approach you describe fails when you want to pass 'typeof(ClassLibrary1.MyObject)' into RunInstanceMethod, because I cannot access the private type in my test project. So far I am stuck with #if DEBUG making the classes public.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Yes, there is!
Type privateType = Type.GetType("TopNamespace.SubNameSpace.ContainingClass+NestedClass,MyAssembly");
object obj = UnitTestUtilities.Helper.RunInstanceMethod( privateType, "MyPrivateMethod", objInstance, new object[3] {"Hello", new DateTime(2004,05,24), 2.1});
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
That's helpful, but you don't show to get objInstance if you're dealing with a private class.
I added another method to the Helper class that will create that object instance for you based on the privateType:
public static object GetObjectInstance(System.Type t) { BindingFlags eFlags = BindingFlags.DeclaredOnly | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.CreateInstance; return t.InvokeMember(null, eFlags, null, null, null); }
Now you can call the RunInstanceMethod.
FWIW
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
I'm wondering if there is any way to extend this to being able to call a private constructor? What I'm trying to do is test a singleton class which needs to be reset between tests because otherwise the class may have some state information left over from the previous test.
Previously I would have just added a #if #endif block so that the test could use a public constructor.
So far my attempts at calling a private constructor look like this:
public static object RunConstructor(System.Type type, params object[] parameters) { BindingFlags bindingAttr = BindingFlags.CreateInstance | BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic;
int len = parameters.Length; Type[] types = new Type[len]; ParameterModifier[] modifiers = new ParameterModifier[len]; for (int i=0; i { types[i] = parameters[i].GetType(); modifiers[i] = new ParameterModifier(0); } ConstructorInfo constructor = type.GetConstructor(bindingAttr, null, types, modifiers); return constructor.Invoke(bindingAttr, null, parameters, null); }
Any ideas?
Cada uno es artifice de su ventura WDevs.com - Open Source Code Hosting, Blogs, FTP, Mail and Forums
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Good point. Offhand I'm not sure, although I'll keep my eyes open about it.
Tim
Tim Stall MCAD.Net http://timstall.dotnetdevelopersjournal.com/
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
This does it for me (just change BindingFlags.Static to BindingFlags.Instance):
Type type = typeof(TypeWithPrivateCtor); Type[] ctorParams = new Type[0]; ConstructorInfo info = type.GetConstructor(BindingFlags.CreateInstance | BindingFlags.Public | BindingFlags.Instance | BindingFlags.NonPublic, null, ctorParams, null); object obj = info.Invoke(null);
Cheers! Nick
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
But instead of all that faffing about with inherited classes and reflection what I do is create the private method, then create a debug block like this:
class MyClass { private int MyPrivateMethod(string someParam) { // Do Stuff } #if DEBUG public int TEST_MyPrivateMethod(string someParam) { return MyPrivateMethod(someParam); } #endif }
Then in my unit tests I can call the TEST_... method and for other code I know that any method beginning TEST_... won't be available in the release build, so I just don't use it. Visual Studio is also quite good at helping as it will grey out the text in the DEBUG block if I've got the current build set to DEBUG.
It is also possible to create your own blocks and have a special unit test build.
Also, I ensure that the test method is named exactly the same as the original method but with a "TEST_" prefix on it. Ever since I started taking the Cradle-to-grave naming concept seriously I cannot believe how much time I've saved by not having to think about what such and such a method or variable is called.
You mention in the article: "This only works in Debug mode." Is that really such a big deal? If it is, then there is nothing to stop you defining a UNITTEST block and having a release build with the additional tests in it.
You also wrote: "This is procedural and not object-oriented. We would need to wrap the individual methods in both the production code and unit tests." I don't see why you are wrapping the methods in the production code. The compiler will strip out the wrappers in production code as they won't be used in the production code. I also don't see how it "is procedural" And: "This alters the SUT by adding the public method wrappers." I don't see this as a problem. No more of a problem that inheriting and putting the wrapper in the derived class, or creating a wrapper using reflection.
Cada uno es artifice de su ventura WDevs.com - Open Source Code Hosting, Blogs, FTP, Mail and Forums
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Hey Colin, Thanks for the feedback. Keep in mind that I initially started using #if DEBUG … #endif. It does work, and is certainly better than no test at all.
> You mention in the article: "This only works in Debug mode." Is that really such a big deal? Why not take the approach that lets you run tests on the Release build - the same code shipped to the client? In your sample, you’ve done all the work to create a test for MyPrivateMethod, and while you can run it in debug mode guaranteeing some degree of success (if it passes in Debug, it will probably pass in Release), you can’t run it in Release.
> I don't see why you are wrapping the methods in the production code. > And: "This alters the SUT by adding the public method wrappers." The gist of my point is that you need to physically alter the source code file by (1) wrapping creating public TEST_xxx methods, and (2) wrapping these methods in #if DEBUG … #endif (or something similar like “UNITTEST). These alterations do not add any feature to the client; they are used only to assist in testing.
> I also don't see how it "is procedural" It encloses #if DEBUG … #endif directives around each method/group of methods instead of applying them at the object level. Perhaps it would be clearer to think of it as “not object oriented”.
Tim
Tim Stall MCAD.Net http://timstall.dotnetdevelopersjournal.com/
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
TimStall wrote: if it passes in Debug, it will probably pass in Release
Ah... Okay - I take your point, they may be some subtle difference between debug and release that isn't obvious or immediately apparent.
I orignally wrote: I don't see why you are wrapping the methods in the production code
TimStall responded: The gist of my point is that you need to physically alter the source code file by (1) wrapping creating public TEST_xxx methods, and (2) wrapping these methods in #if DEBUG … #endif (or something similar like "UNITTEST"). These alterations do not add any feature to the client; they are used only to assist in testing.
I guess point 1 will be mitigated in C# 2.0 and the introduction of partial classes - no longer need to modify the original source file as these wrappers can be added in a secondary source file.
As for point 2: I see that they don't add anything to the client, but why is the client caring about unit tests?
TimStall wrote: Perhaps it would be clearer to think of it as “not object oriented”.
I think this is a point that will sink in over the course of the next few days, but for the moment I cannot see a problem creating a #if UNITTEST block next to a method to expose it publicly in a unit testing build (i.e. Effectively a debug or release type build with unit testing wrappers in it). The wrapper is simply has an identical signature (save for the public exposure and the "TEST_" prefix) and does nothing but call the private method.
Anyway - It might make more sense when I get a chance to play with these ideas over the weekend.
Cada uno es artifice de su ventura WDevs.com - Open Source Code Hosting, Blogs, FTP, Mail and Forums
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
> I cannot see a problem creating a #if UNITTEST block next to a method to expose it publicly
While I agree that it works, one reason that I switched is because it requires writing a bunch of extra test code. The approach I outlined requires only a single utility method once - not a new wrapper for each private method.
Tim Stall MCAD.Net http://timstall.dotnetdevelopersjournal.com/
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
TimStall wrote: While I agree that it works, one reason that I switched is because it requires writing a bunch of extra test code.
Okay - I'm sold 
One change I made to your helper class was to change the Run...Method signature to:public static object RunStaticMethod(System.Type t, string strMethod, params object[] aobjParams) Adding the params means that the call to this method is less clunky. For example you had object obj = UnitTestUtilities.Helper.RunInstanceMethod( typeof(ClassLibrary1.MyObject), "MyPrivateMethod", objInstance, new object[3] {"Hello", new DateTime(2004,05,24), 2.1}); which can now be object obj = UnitTestUtilities.Helper.RunInstanceMethod( typeof(ClassLibrary1.MyObject), "MyPrivateMethod", objInstance, "Hello", new DateTime(2004,05,24), 2.1);
Finally, I have a request. May I use this code in an open source project that I am working on? For details of the project see: SqlDocuTest[^]
Cada uno es artifice de su ventura WDevs.com - Open Source Code Hosting, Blogs, FTP, Mail and Forums
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
| | |