RMI for C++

• Download source (including demo project) - 171 Kb

Please Note

This article is quite old now. The code samples still compile with current versions of RCF, but for up to date information, please refer to this article:

RCF - Interprocess Communication for C++

Introduction

The officially sanctioned way of making distributed function calls between C++ programs is to use CORBA, but for many applications, this is overkill. The CORBA specifications allow distributed function calls to be made between code written in any number of languages, and to make it all work, specialized tools need to be integrated into the build process, in order to translate object definitions written in CORBA's IDL to whichever native language is being used (C++, Java, etc.).

However, if we assume that the server and client are both written in the same language, let us assume C++, since it is possible to do away with these complexities. In particular, instead of elaborate definitions of interfaces and marshalling specifications, we can simply defer to C++.

Instead of separate IDL files with object interfaces, we specify the interfaces directly in C++ source code, using the preprocessor, and to marshal arguments across process boundaries, we use the native C++ serialization framework provided in the latest release of the Boost library.

A Simple Example

As an example, a simple echo server looks like this:

#include <RCF/RCF.hpp> 


RCF_BEGIN(I_Echo, "I_Echo")
  RCF_METHOD_R1(std::string, echo, const std::string &);
RCF_END(I_Echo);

class Echo
{
public:
  std::string echo(const std::string &msg) { return msg; }
};

int main()
{
  int port = 50001;
  RCF::RcfServer server(port);
  server.bind<I_Echo, Echo>();
  server.start();
  return 0;
}

And the client:

#include <RCF/RCF.hpp>


RCF_BEGIN(I_Echo, "I_Echo")
  RCF_METHOD_R1(std::string, echo, const std::string &);
RCF_END(I_Echo);

int main()
{
  std::cout << RcfClient<I_Echo>("localhost", 
                               50001).echo("my message");
  return 0;
}

The Boost.Serialization library is used to serialize parameters and return values. It handles standard types and containers automatically, and is easily extended to user defined classes. It also allows us to serialize pointers, with proper handling of polymorphic pointers and multiple pointers to single objects.

Basic Usage

There are three basic steps to using this framework:

1. Use the RCF_xxx macros to define interfaces.
2. Use the RcfServer class to expose objects that implement the interface.
3. Use the RcfClient<> classes to invoke methods on the objects exposed by the server.

The interface definition macros are used as follows:

RCF_BEGIN( type, type_id )
  // ...

  RCF_METHOD_xx( return_type, name, ....):
  // ...

RCF_END( type )

type is the identifier for the interface, type_id is a string giving a runtime description of the interface. The RCF_METHOD_xx macros define the member functions, and are named according to the number of arguments and whether the return value is void or not. So, for a function func accepting two strings and returning an integer, we write:

RCF_METHOD_R2(int, func, std::string, std::string);

and if the function has a void return type, we would instead write:

RCF_METHOD_V2(void, func, std::string, std::string);

Dispatch IDs for each function are generated automatically; the first member function is numbered 0, the next one 1, and so on. So, the order in which the functions appear in the definition is important, unlike in CORBA, where dispatch IDs are based on the function name. The dispatch IDs are generated using templates and not any preprocessor __LINE__ trickery, so the interface does not change if blank lines are inserted. The maximum number of member functions that can appear between RCF_BEGIN() and RCF_END() is at the moment limited to 25, but this limit is arbitrary.

The purpose of the RCF_xxx macros is to define the class RcfClient<type>. This class serves as a client stub, from the user's point of view, but also has facilities that allow the framework to use it as a server stub. These macros can be used in any namespace, not just the global namespace.

Once we have defined an interface using the RCF_xxx macros, we can start a server and bind the interface to concrete objects:

{
  // create the server and tell it which port to listen on

  RCF::RcfServer server(port);

  // Interface is the identifer of the interface we're exporting,

  // Object is a type that implements that interface

    
  // one object for each client

  server.bind<Interface, Object>(); 

  // ... or one object shared by all clients

  Object object;
  server.bind<Interface>(object); 

  // tell the server to start listening for connections

  server.start();

  // ...


  // the server will shut down automatically as it goes out of scope

}

The objects are statically bound to the corresponding interface; there is no need for the object to derive from an interface class as is the case for traditional dynamic polymorphism. Instead, the compiler resolves the interface at compile time, which is not only more efficient, but also allows more flexible semantics.

The server can handle multiple simultaneous clients, even in single threaded mode, and can be stopped at any time. The lifetime of objects exposed by the server is determined by the number of current connections to the given object; once there are no more live connections to the object, a timeout is set, and when it expires, the object is deleted.

To make a client call, we instantiate the corresponding RcfClient<> template and pass the server IP and port number to the constructor. When the first remote method is called, the client then attempts to connect to the server, queries for the given object, invokes the requested member function of the remote object, and then returns the remote return value.

// define the interface

RCF_BEGIN(Interface, "Interface")
  RCF_METHOD_R2(int, add, int, int);
RCF_END(Interface);

// ...


{
  std::string ip = "localhost";
  int port = 50001;
  RcfClient<Interface> client(ip, port);
  
  // connect and call the add function

  int sum = client.add(1,1);
  
  // connection closed as we exit scope

}

Should any exceptions arise on the server side while invoking the requested object, an exception of type RCF::RemoteException will be propagated back to the client and thrown. Should any exceptions arise anywhere else on the server side, e.g., while serializing arguments, then the server will forcibly close the connection, and the client will throw an exception.

RCF will automatically handle a range of parameter types, including C++ primitive types (int, double, etc.), std::string, STL containers, and pointers and references to any of the previously mentioned types. Polymorphic pointers and references, and multiple pointers to single objects are correctly handled as well. Smart pointers are also supported (boost::shared_ptr, std::auto_ptr), and are the safest way of passing polymorphic parameters.

In CORBA, one can tag a parameter as in, out, or inout, depending on which direction(s) one wants the parameter to be marshaled. In RCF, the marshaling directions are deduced from the parameter type, according to the following conventions:

Value: in
Pointer: in
Const reference: in
Nonconst reference: inout

Nonconst reference to pointer: out

To use user-defined types as parameters or return values, some additional serialization code is needed. What that code is depends on which serialization protocols are being used; by default Boost.Serialization is used, and an example of passing a user-defined type would look like the following:

struct MyStruct
{
  int a;
  int b;
  int c;
  double d;
  std::string s;
  std::map <std::string, std::vector<std::string> > m;
  
  template<typename Archive>
  void serialize(Archive &archive, unsigned int version)
  {
    ar & a & b & c & d & s & m;
  }
  
};

RCF_BEGIN(MyInterface, "MyInterface")
  RCF_METHOD_R1(MyStruct, myfunc, const MyStruct &);
RCF_END(MyInterface);

Details

The server and client classes use BSD-style sockets to implement the networking, over TCP, and the whole framework has been compiled and tested on Linux, Solaris (x86 and SPARC) and Win32, using Visual C++ 7.1, Codewarrior 9.0, Borland C++ 5.5, and GCC 3.2. Building RCF requires v. 1.32.0 or later of the Boost library, although the only parts of Boost that need to be built are Boost.Serialization, and, for multithreaded builds, Boost.Threads. Multithreaded builds are enabled by defining RCF_USE_BOOST_THREADS before including any RCF headers.

To use RCF in your own application, you'll need to include the src/RCF.cpp file among the sources of the application, and link to the necessary libraries from Boost, along with OS-specific socket libraries (on Windows that would be ws2_32.lib, on Linux libnsl, etc.).

I've included a demo project for Visual Studio .NET 2003, which includes everything needed to compile, link, and run a server/client pair, with the exception of the Boost library, which needs to be downloaded and unzipped, but no building is needed.

Performance, as measured in requests/second, is highly dependent on the serialization protocol, and also on the compiler being used. Before turning to Boost.Serialization, I used a serialization framework of my own, with which I could clock around 3000 minimal requests/sec. using Visual C++ 7.1, and 3300 requests/sec. with Codewarrior 9.0, on a loopback connection on a 1400Mhz, 384Mb PC running Windows XP. GCC 3.2, on the other hand, was far slower. Using Boost.Serialization, however, I've been nowhere near these numbers; on average, it's around five times slower.

Conclusion

RMI is a well known concept in Java circles, what I've done here is to do something similar in C++, without all the complications of CORBA. If you like it, please tell me, if you don't, well, please tell someone else.... Jokes aside, any and all feedback is appreciated, all I ask is that if you grade the article, and do so with a low grade, then please leave an explanatory comment!

History

• 8 Feb 2005 - First release.
• 10 Mar 2005
  • Now includes a custom serialization framework, so you no longer have to use Boost's. Both serialization frameworks are supported though, use the project-wide RCF_NO_BOOST_SERIALIZATION and RCF_NO_SF_SERIALIZATION defines to control which ones are used. Default behaviour is to compile both.
  • Default client timeout changed to 10s.
  • Server can be configured to only accept clients from certain IP numbers.
  • Server can be configured to listen only on a specific network interface, such as 127.0.0.1.
  • Client stubs themselves are now properly serializable.
• 4 April 2005

  More bugfixes, including:

  • Much-improved network performance (thanks to Jean-Yves Tremblay for finding the bug).
  • Shortened exception messages in release builds.
  • Client stubs automatically reset their connections when exceptions are thrown (eg for timeouts).
  • Finer-grained exception classes.
• 11 July 2005
  • Stripped CVS folders from distribution.
  • Added user-definable callback functions to be called when RcfServer has started.
• 16 Aug 2005
  • Added facilities for server-bound objects to query the IP address of the client that is currently invoking them. To see how it works, open the file RCF/test/Test_ClientInfo.cpp in the download. Just place a call to RCF::getCurrentSessionInfo().getClientInfo().getAddress(), and you'll receive a string containing the IP address of the client that is invoking the method.
• 23 Sep 2005
  • Initialization and deinitialization of the framework can now be done explicitly, be defining the project-wide preprocessor symbol RCF_NO_AUTO_INIT_DEINIT, and then calling RCF::init() and RCF::deinit() at appropriate times. This is mainly useful for DLL builds, so that the DLL can be loaded without automatically initializing Winsock.
• 19 Oct 2005
  • Compatible with Boost 1.33.0.
  • Added enum serialization to the built-in serialization engine, through the SF_SERIALIZE_ENUM macro. For an example of its use, see test/Test_Serialization.cpp.
  • Added a license.
• 30 Jan 2006
  • Miscellaneous bugfixes.
  • The built-in maximum message size limit has been changed to 50 Kb. Look in src/RCF/Connection.cpp, line 374, if you need to change this.
  • I'll only be making sporadic maintenance releases of this version of RCF from now on. You can find the next generation of RCF here.