Testing Entity Framework Applications, Part 2: Typemock

This is the second of a three part series that deals with the issue of faking test data in the context of a legacy app that was built with Microsoft's Entity Framework (EF) on top of an MS SQL Server database – a scenario that can be found very often. Please read the first part for a description of the sample application, a discussion of some general aspects of unit testing in a database context, and of some more specific aspects of the here discussed EF/MSSQL combination.

Lately, I wondered how you would ‘mock’ the data layer of a legacy application when this data layer is made up of an MS Entity Framework (EF) model in team with a MS SQL Server database. The question originally came up in the context of how you could enable higher-level integration tests (automated UI tests, to be exact) for a legacy application that uses this EF/MSSQL combo as its data store mechanism – a not so uncommon scenario.

The question sparked my interest, so I decided to dive into it a bit deeper. What I've found out is, in short, that it's not very easy and straightforward to do it – but it can be done. The two strategies that are best suited to fit the bill involve using either the (commercial) Typemock Isolator tool or the (free) NDbUnit framework. This post will present the Typemock approach...

When it comes to mocking and faking, Typemock is something like the swiss army knife in a tester's toolbelt. It's functionally comparable to MS Moles, in that there aren't any limits in what it can do (it's really Mocking the Unmockable). You can easily write all kinds of test doubles for components that you otherwise could not fake: Static classes, Framework classes, Sealed classes, Non-virtual members and so on... The big difference to Moles however is that Typemock has a well-crafted API and generally is so much easier to handle, understand, learn, and remember that this alone would justify the price difference.

Preparing the Test Data

As I have shown in the first part, the PersonRepository expects an instance of EF's ObjectContext class in its c'tor. This project-specific class is generated by the EF designer and contains, among (too) many other things, the conceptual entity model which is derived from the underlying database. (It's called SchoolEntities in our case.)

This marks the point where we can plug in the Typemock Isolator tool to isolate the system from the underlying database, faking the ObjectContext and thus having the opportunity to feed the PersonRepository class with self-provided data on a per-test basis as required.

The test fixture's Setup code will look like this, when using this approach:

[TestFixture, TestsOn(typeof(PersonRepository))] 
[Description("Uses Typemock Isolator 2010 
to fake the data for the applications EF data model. " + 
             "Trial version can be downloaded from here: 
              http://www.typemock.com/typemock-isolator-product3/")] 
public class PersonRepositoryFixture 
{ 
    #region Fields 

    private SchoolEntities _schoolContext; 
    private PersonRepository _personRepository; 

    #endregion // Fields 

    #region Setup/TearDown 

    [SetUp] 
    public void SetUp() 
    { 
        _schoolContext = Isolate.Fake.Instance<SchoolEntities>(); 
        _personRepository = new PersonRepository(_schoolContext); 
    } 

    ...

Note that we've altered only one single line of code here, compared to the 'original' version from the previous part: The _schoolContext field is initialized with a Fake instance of the SchoolEntities class instead of the 'real' object (see here for a short clarification of the sometimes confusing terminology around mocks, fakes, stubs and the like).

What does it mean? Well, it means that our _schoolContext field actually references a true instance of the SchoolEntities class, and not some sort of proxy object. You can call all methods and properties on it, they will do nothing by default and will return the respective default value for value types, or another Fake instance for reference types - unless we explicitly tell Typemock to return some custom, pre-defined values for us (or call some custom code on our behalf). And that's exactly what we need here to isolate the system under test from its EF data model (and its database, consequently).

To accomplish this, we first define a helper class to hold the test data that we will use (and also mimic other required database properties like e.g. referential constraints). It will act as a kind of 'in-memory database':

static class FakeDataHolder 
{ 
    #region Fields 

    // The xxxData collections serve as an internal cache. They represent the database, 
    // are filled during construction  and not altered thereafter. 
    private static readonly ReadOnlyCollection<Person> PersonData; 
    private static readonly ReadOnlyCollection<OfficeAssignment> OfficeAssignmentData; 

    // The xxxList collections mirror the xxxData collections. 
    // They represent a working copy of the data, 
    // which is exposed through properties to the outside and 
    // can be reset via the 'Reset()' method. 
    private static readonly List<Person> 
    PersonList = new List<Person>(); 
    private static readonly List<OfficeAssignment> 
    OfficeAssignmentList = new List<OfficeAssignment>(); 

    #endregion // Fields 

    #region Properties 

    public static IQueryable<Person> Persons 
    { 
        get { return PersonList.AsQueryable(); } 
    } 

    public static IQueryable<OfficeAssignment> OfficeAssignments 
    { 
        get { return OfficeAssignmentList.AsQueryable(); } 
    } 

    #endregion // Properties 

    #region Construction 

    static FakeDataHolder() 
    {  
        PersonData = new List<Person> 
            { 
                new Person {PersonID = 1, 
                FirstName = "Kim", LastName = "Abercrombie"}, 
                new Person {PersonID = 2, 
                FirstName = "Gytis", LastName = "Barzdukas"}, 
                new Person {PersonID = 3, 
                FirstName = "Peggy", LastName = "Justice"}, 
                new Person {PersonID = 4, 
                FirstName = "Fadi", LastName = "Fakhouri"}, 
                new Person {PersonID = 5, 
                FirstName = "Roger", LastName = "Harui"}, 
                new Person {PersonID = 6, 
                FirstName = "Yan", LastName = "Li"}, 
                new Person {PersonID = 7, 
                FirstName = "Laura", LastName = "Norman"}, 
                new Person {PersonID = 8, 
                FirstName = "Nino", LastName = "Olivotto"}, 
                new Person {PersonID = 9, 
                FirstName = "Wayne", LastName = "Tang"}, 
                new Person {PersonID = 10, 
                FirstName = "Meredith", LastName = "Alonso"}, 
                new Person {PersonID = 11, 
                FirstName = "Sophia", LastName = "Lopez"}, 
                new Person {PersonID = 12, 
                FirstName = "Meredith", LastName = "Browning"}, 
                new Person {PersonID = 13, 
                FirstName = "Arturo", LastName = "Anand"}, 
                new Person {PersonID = 14, 
                FirstName = "Alexandra", LastName = "Walker"}, 
                new Person {PersonID = 15, 
                FirstName = "Carson", LastName = "Powell"}, 
                new Person {PersonID = 16, 
                FirstName = "Damien", LastName = "Jai"}, 
                new Person {PersonID = 17, 
                FirstName = "Robyn", LastName = "Carlson"}, 
                new Person {PersonID = 18, 
                FirstName = "Roger", LastName = "Zheng"}, 
                new Person {PersonID = 19, 
                FirstName = "Carson", LastName = "Bryant"}, 
                new Person {PersonID = 20, 
                FirstName = "Robyn", LastName = "Suarez"}, 
                new Person {PersonID = 21, 
                FirstName = "Roger", LastName = "Holt"}, 
                new Person {PersonID = 22, 
                FirstName = "Carson", LastName = "Alexander"}, 
                new Person {PersonID = 23, 
                FirstName = "Isaiah", LastName = "Morgan"}, 
                new Person {PersonID = 24, 
                FirstName = "Randall", LastName = "Martin"}, 
                new Person {PersonID = 25, 
                FirstName = "Candace", LastName = "Kapoor"}, 
                new Person {PersonID = 26, 
                FirstName = "Cody", LastName = "Rogers"}, 
                new Person {PersonID = 27, 
                FirstName = "Stacy", LastName = "Serrano"}, 
                new Person {PersonID = 28, 
                FirstName = "Anthony", LastName = "White"}, 
                new Person {PersonID = 29, 
                FirstName = "Rachel", LastName = "Griffin"}, 
                new Person {PersonID = 30, 
                FirstName = "Alicia", LastName = "Shan"}, 
                new Person {PersonID = 31, 
                FirstName = "Jasmine", LastName = "Stewart"}, 
                new Person {PersonID = 32, 
                FirstName = "Kristen", LastName = "Xu"}, 
                new Person {PersonID = 33, 
                FirstName = "Erica", LastName = "Gao"}, 
                new Person {PersonID = 34, 
                FirstName = "Roger", LastName = "Van Houten"} 
            } 
            .AsReadOnly();  

        OfficeAssignmentData = new List<OfficeAssignment> 
            { 
                new OfficeAssignment 
                { InstructorID = 18, Location = "143 Smith" } 
            } 
            .AsReadOnly(); 
    } 

    #endregion // Construction 

    #region Operations 

    public static void Reset() 
    { 
        PersonList.Clear(); 
        PersonList.AddRange(PersonData); 

        OfficeAssignmentList.Clear(); 
        OfficeAssignmentList.AddRange(OfficeAssignmentData); 
    } 

    public static void AddPerson(Person person) 
    { 
        PersonList.Add(person); 
    } 

    public static void DeletePerson(Person person) 
    { 
        if (OfficeAssignmentList.Any
        (@a => @a.InstructorID == person.PersonID)) 
        { 
            throw new InvalidOperationException
            ("FK_OfficeAssignment_Person"); 
        } 

        PersonList.Remove(person); 
    } 

    #endregion // Operations 

} // class FakedataHolder

The second step then would be to make our test fixture's _schoolContext field return these values (or call the appropriate FakeDataHolder methods, respectively). For this purpose, we can declare a helper method like this in the PersonRepositoryFixture class:

private void FakeData() 
{ 
    FakeDataHolder.Reset();  

    Isolate.WhenCalled(() => _schoolContext.People)  
           .WillReturnCollectionValuesOf(FakeDataHolder.Persons); 
    Isolate.WhenCalled(() => _schoolContext.People.AddObject(null)) 
           .DoInstead(@ctx => 
           FakeDataHolder.AddPerson(@ctx.Parameters[0] as Person)); 
    Isolate.WhenCalled(() => _schoolContext.People.DeleteObject(null)) 
           .DoInstead(@ctx => 
           FakeDataHolder.DeletePerson(@ctx.Parameters[0] as Person)); 
}

I won't go into the details of Typemock's API syntax here (you may consult the Online Documentation for this, if you are interested), but generally I like it because it's very easy to read and understand without oversimplifying things. Or, in other words: I don't have to explain much here, the code itself tells it all...

Executing Some Basic Tests

Here are some of the possible tests that can be written with the above preparations in place:

[Test, Isolated, MultipleAsserts, TestsOn("PersonRepository.GetNameList")] 
public void GetNameList_ListOrdering_ReturnsTheExpectedFullNames() 
{ 
    FakeData(); 

    List<string> names = 
        _personRepository.GetNameList(NameOrdering.List); 

    Assert.Count(34, names); 
    Assert.AreEqual("Abercrombie, Kim", names.First()); 
    Assert.AreEqual("Zheng, Roger", names.Last()); 
} 

[Test, MultipleAsserts, TestsOn("PersonRepository.GetNameList")] 
public void GetNameList_NormalOrdering_ReturnsTheExpectedFullNames() 
{ 
    FakeData(); 

    List<string> names = 
        _personRepository.GetNameList(NameOrdering.Normal); 

    Assert.Count(34, names); 
    Assert.AreEqual("Alexandra Walker", names.First()); 
    Assert.AreEqual("Yan Li", names.Last()); 
} 

[Test, Isolated, TestsOn("PersonRepository.AddPerson")] 
public void AddPerson_CalledOnce_IncreasesCountByOne() 
{ 
    FakeData(); 

    int count = _personRepository.Count; 

    _personRepository.AddPerson(new Person 
    { FirstName = "Thomas", LastName = "Weller" }); 

    Assert.AreEqual(count + 1, _personRepository.Count); 
} 

[Test, Isolated, TestsOn("PersonRepository.RemovePerson")] 
public void RemovePerson_CalledOnce_DecreasesCountByOne() 
{ 
    FakeData(); 

    int count = _personRepository.Count; 

    _personRepository.RemovePerson(new Person { PersonID = 33 }); 

    Assert.AreEqual(count - 1, _personRepository.Count); 
} 

[Test, Isolated, TestsOn("PersonRepository.RemovePerson")] 
public void RemovePerson_ForWhomAnOfficeAssignmentExists_Throws() 
{ 
    FakeData(); 

    Assert.Throws<RepositoryException>(() => 
        _personRepository.RemovePerson(new Person { PersonID = 18 })); 
}

I kept these tests deliberately simple to underpin the fact that Typemock-based tests don't need to be much different from non-faked tests, and also to make them comparable to the other options that are discussed here (the NDbUnit approach and the non-faked version, running your tests directly against a real instance of Microsoft SQL Server, namely).

But Wait, What If Things are Becoming Somewhat More Complex?

Admittedly (and intentionally), the above tests cover only some very simple test cases. This may be enough for some of your test scenarios: the 'normal' use cases, which may make up something well above 95% of what your application effectively is doing in production ("it just works"), but is easily covered with a few simple tests. The more interesting and important bits are the exceptional situations; the corner cases, where something unexpected happens. This is what consumes most of the time and effort in software development: To make an application robust and stable, and make it react in a predictable, user-friendly way even under exceptional circumstances.

In such a situation (i.e., if you want to simulate some sort of exceptional condition), chances are that you will have to dive deep into the production code - or even some 3^rd party code, if it is available - to find out what you will have to return from your fake. A (still very simple) test of this kind could be this, for example:

[Test, Isolated, MultipleAsserts, 
TestsOn("PersonRepository.GetCourseMembers")] 
[Row(null, typeof(ArgumentNullException))] 
[Row("", typeof(ArgumentException))] 
[Row("NotExistingCourse", typeof(ArgumentException))] 
public void GetCourseMembers_WhenGivenVariousInvalidValues_Throws
     (string courseTitle, Type expectedInnerExceptionType) 
{ 
    // Generally return false from the expression 
    // '_schoolContext.Courses.Any(...)' 
    Isolate.WhenCalled(() => _schoolContext.Courses.Any(null)) 
           .WillReturn(false); 

    var exception = Assert.Throws<RepositoryException>(() => 
                                _personRepository.GetCourseMembers(courseTitle)); 
    Assert.IsInstanceOfType(expectedInnerExceptionType, exception.InnerException); 
}

This looks quite trivial, but it took me some minutes to understand that it's the Any() extension method that I had to fake here, and I also had to slightly refactor the FakeDataHolder() helper class to make this work. All in all, not too big a deal, you may say. True, but keep in mind that small things like that can easily add up to a huge amount of additional work (that your manager might not be willing to pay for)...

The other situation in which using Typemock becomes somewhat more laborious is when you have to perform a lot of preparation work of the above described kind to simulate a regular scenario, which you would otherwise (when running against a real database) just take for granted, without thinking much about it. Look at this test, for example:

[Test, Isolated, MultipleAsserts, TestsOn("PersonRepository.GetCourseMembers")] 
public void GetCourseMembers_WhenGivenAnExistingCourse_ReturnsListOfStudents() 
{ 
    // Avoid the ArgumentNullException 
    Isolate.WhenCalled(() => _schoolContext.Courses.Any(null)) 
           .WillReturn(true); 
    // Always return true from the expression 'Person.StudentGrades.Any(...)' 
    var fakePerson = Isolate.Fake.Instance<Person>(); 
    Isolate.WhenCalled(() => fakePerson.StudentGrades.Any(null)) 
           .WillReturn(true); 
    Isolate.Swap.AllInstances<Person>() 
           .With(fakePerson); 
    // Return the expected list of persons 
    Isolate.WhenCalled(() => _schoolContext.People) 
           .WillReturnCollectionValuesOf(new List<Person> 
                                            { 
                                                new Person {PersonID = 9, 
                                                FirstName = "Wayne", 
                                                LastName = "Tang"}, 
                                                new Person {PersonID = 10, 
                                                FirstName = "Meredith", 
                                                LastName = "Alonso"}, 
                                                new Person {PersonID = 11, 
                                                FirstName = "Sophia", 
                                                LastName = "Lopez"}, 
                                                new Person {PersonID = 12, 
                                                FirstName = "Meredith", 
                                                LastName = "Browning"}, 
                                                new Person {PersonID = 14, 
                                                FirstName = "Alexandra", 
                                                LastName = "Walker"}, 
                                                new Person {PersonID = 22, 
                                                FirstName = "Carson", 
                                                LastName = "Alexander"} 
                                            } 
                                            .AsQueryable()); 

    List<Person> persons = 
    _personRepository.GetCourseMembers("Macroeconomics"); 

    Assert.Count(6, persons); 
    Assert.ForAll( 
        persons, 
        @p => new[] { 9, 10, 11, 12, 14, 22 }.Contains(@p.PersonID), 
        "Person has none of the expected IDs."); 
}

The test asserts that, when asking for the members of a specific course, we actually are provided with the expected list, if we have set up the underlying ObjectContext fake to return the correct data. (One word about the Isolate.Swap.AllInstances() expression here: It instructs Typemock to replace all future instances of the Person class with the here defined fakePerson object, whenever the code under test should create a new Person instance.) See how the 'Arrange' part of the test blows up and also requires quite some intimate knowledge about the code under test, in order to provide the required return values (I've heard this once being called Wishful mocking...)?

Sometimes, especially when dealing with a legacy codebase that you cannot change, this is the only option at all to put such code under test, but it easily can mean that you will have to do a lot of work, if you will have to account for all kinds of database relations, triggers, check constraints, stored procedures and stuff like that. How much, will largely depend on the complexity of the underlying database (and of course also on the complexity of the code under test)...

Conclusion

Typemock is a powerful and flexible tool to handle the here described EF/MSSQL scenario. It's blazingly fast, fully isolates your system under test from any external data source, and lets you quickly and easily adapt/refactor your test code if you need to. All your test data are defined directly in the test code, so chances are high that you get immediate feedback in form of a compiler error when you have changed your database schema (and consequently the EF conceptual model) for some reason.

The downsides are more practical in nature: As I illustrated above, it can be a lot of work to fake certain scenarios with Typemock, so be prepared that it may not always be straightforward and easy to author tests with the help of the tool (but there will be a rich reward for your efforts...). Also, Typemock is a commercial tool,which means that it does not come for free in financial terms. You will have to make the explicit decision at some point whether it's worth its price in your specific situation or not.

Whereas this part of the article series has shown how the Typemock Isolator tool can be used to deal with a scenario where an application sits on top of an MS Entity Framework class model, the next part will accomplish the same with the NDbUnit framework.

The Sample Solution

A sample solution (VS 2010) with the code from this article series is available via my Bitbucket account from here (Bitbucket is a hosting site for Mercurial repositories. The repositories may also be accessed with the Git and Subversion SCMs - consult the documentation for details. In addition, it is possible to download the solution simply as a zipped archive – via the 'get source' button on the very right.). The solution contains some more tests against the PersonRepository class, which are not shown here. Also, it contains database scripts to create and fill the School sample database.

To compile and run, the solution expects the Gallio/MbUnit framework to be installed (which is free and can be downloaded from here), the NDbUnit framework (which is also free and can be downloaded from here), and the Typemock Isolator tool. Moreover, you will need an instance of the Microsoft SQL Server DBMS, and you will have to adapt the connection strings in the test projects App.config files accordingly.