Click here to Skip to main content
14,265,429 members

Typemock vs. Google Mock: A Closer Look

Rate this:
4.50 (2 votes)
Please Sign up or sign in to vote.
4.50 (2 votes)
26 Nov 2018CPOL
Typemock vs. Google Mock: A Closer Look

Introduction

Writing tests for C++ can be complicated, especially when you are responsible for maintaining legacy code or working third-party APIs. Fortunately, the C++ marketplace is always expanding, and you have several testing frameworks to choose from. Which one is the best? In this post, we'll consider Typemock vs. Google Mock.

We'll use Typemock's Isolator++ and Google Mock, the C++ framework that is bundled with Google Test, to write a test function for a small project. As we implement the tests, we'll examine the difference in how the frameworks approach the same problem.

A Unit Test Project

I cited Martin Fowler's definition of mocking in an earlier post. Fowler creates example code for a warehouse project that accepts orders and sends emails.

Fowler defines an order object that interacts with a warehouse and mail service to fill orders and notify clients. He illustrates different approaches for mocking the mail service and warehouse so the order can be tested.

This GitHub project contains Fowler's classes implemented in C++ with tests written in Google Mock. Let's use those classes as a starting point, with some small changes, for our comparison.

We're only going to look at test methods since both frameworks operate well within test runners like Google Test and Microsoft's Unit Testing Framework. Also, to keep the code simple and easy to read, we'll use raw pointers and values.

First, we have the mail service. Orders use it to send a message indicating whether it fills a request.

class MailService
{
public:
    virtual void send(std::string message) = 0;
};

Next, we have the warehouse. It contains inventories of different products. Orders query it with a quantity and a product name. If the product exists and there is sufficient quantity, the order removes that amount.

class Warehouse
{
public:
    virtual bool hasInventory(int quantity, std::string product) const = 0;
    virtual void remove(int quantity, std::string product) = 0;
};

Finally, we have the order. Orders fill themselves by querying the warehouse. Then it sends a message that reflects the result.

class Order
{
public:
    Order(int quantity, std::string product)
    {
        this->quantity = quantity;
        this->product = product;
    }

    void setMailService(MailService* mailService)
    {
        this->mailService = mailService;
    }

    bool fill(Warehouse &warehouse)
    {
        if (warehouse.hasInventory(quantity, product))
        {
            warehouse.remove(quantity, product);
            this->mailService->send("Order filled.");
            return true;
        }
        else
        {
            this->mailService->send("Order not filled.");
            return false;
        }
    }

private:
    std::string product;
    int quantity;
    MailService* mailService;
};

Let's look at how we write tests for this code with Isolator and Google Mock.

Creating Mocks

Creating mocks is where the differences between Google Mock and Isolator++ are most apparent. Google Mock is designed to resemble declarative libraries, such as JMock and EasyMock. Isolator++ is designed for isolating classes and writing tests in the arrange-act-assert test model.

To create a mock with Google Mock, we have to derive a class and then mock the methods we need.

The mail service is the simplest of the two classes we need.

class MockMailService : public MailService
{
public:
    MockMailService()
    {
    }
    MOCK_METHOD1(send, void(std::string));
};

Mail service's one-and-only method is virtual. As we'll see later, mocking a non-virtual method with Google Mock is more complicated.

Mocking a class with Isolator++ requires less code. We declare a mock variable in our test class using template syntax. We'll see how to add the mock method below.

MailService* mailservice = FAKE<MailService>();

Let's declare our Google Mock warehouse before we move on to writing tests.

class MockWarehouse : public Warehouse
{
public:
   MOCK_CONST_METHOD2(hasInventory, bool(int, std::string));
   MOCK_METHOD2(remove, void(int, std::string));
};

Now let's create a test!

Writing Tests

Test methods created with Google Mock and Isolator++ seem similar at first, but the philosophical difference between the two frameworks are still clear.

First, let's take a look at a Google Mock test.

TEST_METHOD(Fill)
{
    MockWarehouse warehouse;
    MockMailService* mailService = new MockMailService();

    Order order(50, "Talisker");
    order.setMailService(mailService);

    EXPECT_CALL(warehouse, hasInventory(50, "Talisker"))
        .Times(1)
        .WillOnce(Return(true));
    EXPECT_CALL(warehouse, remove(50, "Talisker"))
        .Times(1);

    EXPECT_CALL(*mailService, send("Order filled.")).Times(1);
    ASSERT_TRUE(order.fill(warehouse));
}

We start by creating a mock warehouse and a mock mail service.

Then we create an instance of the order class, passing in the quantity and product name. Next, we give the order object the mail service.

Initialization is complete, so now we can set expectations.

First, we set two expectations for the warehouse: one is the check for enough Taliskers and the second is to remove them.

Then, we set the call to the mail service.

We tell Google Mock to expect each call to our mocks once and only once.

Finally, we check for success filling the order.

Let's take a look at Isolator++'s implementation of the same test.

TEST_METHOD(Fill) 
{
  MailService* mailService = FAKE<MailService>();

  Warehouse* warehouse = FAKE<Warehouse>();

  Order order(50, "Talisker");
  order.setMailService(mailService);

  WHEN_CALLED(warehouse->hasInventory(EQ(50), "Talisker")).Return(true);
  WHEN_CALLED(warehouse->remove(EQ(50), "Talisker")).Ignore();
  WHEN_CALLED(mailService->send("Order Filled")).Ignore();

  Assert::IsTrue(order.fill(*warehouse));

  int count = TIMES_CALLED(warehouse->hasInventory(EQ(50), "Talisker"));

  Assert::AreEqual(1, count);

  count = TIMES_CALLED(warehouse->remove(EQ(50), "Talisker"));
  Assert::AreEqual(1, count);

  count = TIMES_CALLED(mailService->send("Order Filled"));

  Assert::AreEqual(1, count);

  ISOLATOR_CLEANUP();
}

First, we start by creating a mock warehouse and a mock mail service, just as we did for Google Mock. However, we use the FAKE template instead of declaring a derived class.

Then, we create an instance of the order class, passing the quantity and product name. We set the mail service too, just like in the Google Mock test initialization.

First, we set the mocks for the warehouse. This looks different from the Google Mock implementation. We are not specifying how many times each call should be made.

Then we set the mock method for the mail service. This completes our arrangements.

Then, we fill the order and check the result. That is our one-and-only action.

Finally, we check to be sure each method was called with the correct parameters and only once. These checks are our assertions.

Typemock Vs. Google Mock

While these two tests verify the same behavior, they look very different.

First, Isolator++ doesn't require derived classes. So, to be blunt, there's less work to do. With the warehouse and email service classes, this means saving a few minutes. The difference with production code could be hours.

But less code means less effort up front and less maintenance down the road. With Google Mock, when you change a mocked class, you have to reflect the changes in both the mock class definition and the tests. Imagine maintaining a large number of tests for a more extensive set of classes.

With Isolator, only tests require changes when a class changes. Often, the only test changes are related to new behaviors.

There are also critical differences in the declaration of our mocks.

With Google Mock, we declare an instance of the derived class.

MockWarehouse warehouse;

With Isolator++, we declare the mock with the FAKE template.

Warehouse* warehouse = FAKE<Warehouse>();

Rather than relying on a well-named class, the object's type tells us what it is.

In the Google Mock test, we used a syntax familiar to Java developers for setting expectations:

EXPECT_CALL(warehouse, hasInventory(50, "Talisker"))
        .Times(1)
        .WillOnce(Return(true));

In a single declaration, we specify the call, how many times we expect the test to call it, and the return value.

With Isolator++, the mock is part of the arrangements, while the checks for whether or not the test called it and how often is part of the assertions.

WHEN_CALLED(warehouse->hasInventory(EQ(50), IS("Talisker"))).Return(true);
ASSERT_EQ(1, TIMES_CALLED(warehouse->hasInventory(EQ(50), IS("Talisker"))));

This clear delineation between the parts of a test makes Isolator++ tests more natural to read and maintain.

Mocking Non-Virtual Functions

Now, let's imagine a scenario that's less ideal.

What if our mail service is a concrete class?

class MailService
{
public:
    void send(std::string message);
};

Our Isolator++ tests don't have to change. The procedure for mocking a concrete class with non-virtual methods is the same as for a virtual one. We declare an instance of the class, initialize it with a FAKE, and mock the methods we need for the test.

Google Mock can't mock a non-virtual function. To mock this class, we have to modify not just our test code, but the code under test too. We have to implement dependency injection via a template that accepts duck typing.

First, we create a mock object for mail service which is almost identical to the first, except it is not derived from MailService.

class MockMailService
{
public:
    MockMailService()
    {
    }
    MOCK_METHOD1(send, void(std::string));
};

Then, we have to make the order class a template so we can pass it either a mock or a concrete mail service.

template <class Service>
class Order
{
public:

    Order(Service* service, int quantity, std::string product)
    {
        this->quantity = quantity;
        this->product = product;
        this->mailService = service;
    }

    bool fill(Warehouse &warehouse)
    {
        if (warehouse.hasInventory(quantity, product))
        {
            warehouse.remove(quantity, product);
            this->mailService->send("Order filled.");
            return true;
        }
        else
        {
            this->mailService->send("Order not filled.");
            return false;
        }
    }

private:
    std::string product;
    int quantity;
    Service* mailService;
};

Now we declare a different type of order depending on whether we are writing code for production or test.

For production, we use the actual mail service.

MailService = new MailService();
Order<MailService> order(mailService, 50, "Talisker");

But for tests, we use the mock class.

MockMailService mockMailService = new MockMailService();
Order<MockMailService> order(mockMailService, 50, "Talisker");

It's easy, and a good idea, to write in-house code with interfaces that lend themselves to testing. Unfortunately, we often need to test with classes we don't control.

The Verdict

So, how do the two mocking frameworks stack up against each other?

Ease of Use

Isolator++ gets a small edge when it comes to writing simple tests. With Google Mock, we had to create a new class that inherited the class we wanted to mock. With Isolator++, we wrote a single line of code. But the difference between the two frameworks was more apparent when it came to mocking a non-virtual function. Since Google Mock can't mock a concrete function, it required some extra work. Isolator++ uses the same syntax regardless of whether a function is concrete or virtual.

Maintainability

Code changes and tests need to change with it. Isolator++ makes it easier to keep your tests up to date since all your test code lives in your test classes. Google Mock requires a derived class for every object, which means updating class definitions every time a mocked object changes.

Learning Curve

Which framework you find easier to use may depend on where you're coming from. Google Mock is inspired by several popular Java testing frameworks, and if you're familiar with them, you should feel right at home. Isolator++ resembles Isolator, Typemock's .NET testing framework. If you've used Isolator before, then Isolator++ will be easy to master.

However, how you write tests might be a factor too. Isolator++ makes using the Arrange-Act-Assert test pattern much more comfortable to implement than with Google Mock.

Gmock is available as part of the open-source Google Test framework. You can download it here.

Isolator++ is a commercial product from Typemock. It's available for both Windows and Linux. Download a trial copy today.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Hila Berger
Israel Israel
No Biography provided

Comments and Discussions

 
QuestionEasier Syntax Pin
Eli Lopian9-Dec-18 4:37
memberEli Lopian9-Dec-18 4:37 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Article
Posted 26 Nov 2018

Tagged as

Stats

4.9K views
3 bookmarked