RCF - Interprocess Communication for C++

• Download RCF 04 - 637.64 KB
• Download RCF 09c - 392.34 KB

Introduction

RCF (Remote Call Framework) is a C++ framework I've written to provide an easy and consistent way of implementing interprocess calling functionality in C++ programs. It is based on the concept of strongly typed client/server interfaces, a concept familiar to users of IDL-based middlewares such as CORBA and DCOM. However, by catering only to C++, RCF can harness C++-specific language features that allow the formulation of interprocess calls in a relatively simple and uncluttered manner.

This is the second generation of the RCF library, the first of which was presented in a previous article here on CodeProject.

An example is worth a thousand words, to paraphrase an old Chinese saying... This article is framed as a series of examples, covering the salient features of RCF.

Basics

We'll begin with the canonical echo server and client examples that seem to feature in all networking and IPC demonstrations. We want to expose and then call a function that accepts a string, and which returns the same string. Using RCF, the server code becomes:

#include <RCF/Idl.hpp>
#include <RCF/RcfServer.hpp>
#include <RCF/TcpEndpoint.hpp>

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(std::string, echo, const std::string &)
RCF_END(I_Echo)

class Echo
{
public:
    std::string echo(const std::string &s)
    {
        return s;
    }
};

int main()
{
    Echo echo;
    RCF::RcfServer server(RCF::TcpEndpoint(50001));
    server.bind<I_Echo>(echo);
    server.startInThisThread();
    return 0;
}

... and the client code becomes:

#include <RCF/Idl.hpp>

#include <RCF/TcpEndpoint.hpp>

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(std::string, echo, const std::string &)
RCF_END(I_Echo)

int main()
{
    RcfClient<I_Echo> echoClient(RCF::TcpEndpoint("localhost", 50001));
    std::string s = echoClient.echo(RCF::Twoway, "what's up");
    return 0;
}

I_Echo is an RCF interface defined by the RCF_BEGIN/RCF_METHOD/RCF_END macros. These macros correspond to what in CORBA would be an IDL definition, but in this case, the interface definition is placed inline in C++ source code and requires no separate compilation step, as is the case for IDL interfaces. The interface is simply included in the source code of the server and the client, and compiled along with the rest of the program.

The RCF::Twoway argument in the client stub invocation is a flag that tells RCF to make a two-way client call; the client sends a request, waits for a response, and if none is received within a configurable duration of time, an exception is thrown. The other option is to use RCF::Oneway; a request is sent, but the server does not send a response, and the client stub call will return control to the user immediately. If the directional argument is omitted, RCF::Twoway is used by default.

The client stub is not synchronized in any way, and should only be accessed by a single thread at a time. The server, though, is inherently multi-threaded although in the example above, it has been written as a single-threaded process. RcfServer::startInThisThread() hijacks the calling thread and converts it to a worker thread of the server.

In multi-threaded builds, one can also call RcfServer::start() and the threads needed to drive the server will then automatically be created. Multi-threaded builds require the Boost.Threads library and the definition of the preprocessor symbol RCF_USE_BOOST_THREADS.

We can also rewrite our client/server pair to use the UDP protocol instead. This time, we'll let the server and the client both reside in the same process, but in different threads:

#include <RCF/Idl.hpp>
#include <RCF/RcfServer.hpp>
#include <RCF/UdpEndpoint.hpp>

#ifndef RCF_USE_BOOST_THREADS
#error Need to build with RCF_USE_BOOST_THREADS
#endif

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(std::string, echo, const std::string &)
RCF_END(I_Echo)

class Echo
{
public:
    std::string echo(const std::string &s)
    {
        return s;
    }
};

int main()
{
    Echo echo;
    RCF::RcfServer server(RCF::UdpEndpoint(50001));
    server.bind<I_Echo>(echo);
    server.start();
    RcfClient<I_Echo> echoClient(RCF::UdpEndpoint("127.0.0.1", 50001));
    std::string s = echoClient.echo(RCF::Twoway, "what's up");
    server.stop(); // would happen anyway as server object goes out of scope

    return 0;
}

The distinguishing feature of UDP, with respect to TCP, is that UDP is stateless. There is no guarantee of the order in which packets that are sent will arrive, or if they will even arrive at all. On a local connection, like in the example above, using two-way semantics will usually work since the packets are not being subjected to the vagaries of a real network. In general, however, one-way semantics would be needed.

Interfaces

The interface definition macros function exactly like they did in the previous generation of RCF. The RCF_BEGIN() macro commences an interface definition with the given name and runtime description, and RCF_END() ends the interface definition. In between, one can have up to 25 RCF_METHOD_xx() macros to define the member methods of the interface.

The last two letters in the RCF_METHOD_xx() macros indicate the number of arguments and whether the return type is void. For instance, RCF_METHOD_V3 is used to define a method with three parameters and a void return value, while RCF_METHOD_R2 defines a method with two arguments and a non-void return value.

The net result of these macros is to define the RcfClient<type> class, where type is the name of the interface. This class is then used directly by the client as a client stub, and indirectly by the server as a server stub. RCF interfaces can be defined in any namespace:

namespace A
{
    namespace B
    {
        RCF_BEGIN(I_X, "I_X")
        RCF_METHOD_V0(void, func1)
        RCF_METHOD_R5(int, func2, int, int, int, int, int)
        RCF_METHOD_R0(std::auto_ptr<std::string>, func3)
        RCF_METHOD_V2(void, func4,
           const boost::shared_ptr<std::string> &,
           boost::shared_ptr<std::string> &)
        // ..

        RCF_END(I_X)
    }
}

int main()
{
    A::B::RcfClient<A::B::I_X> client;
    // or

    A::B::I_X::RcfClient client;
    // ...

}

Server Bindings

On the server side, the interface needs to be bound to a concrete implementation. This is done through the templated RcfServer::bind() functions. There are several variations that accommodate various memory management idioms. The following calls all accomplish pretty much the same thing, the binding of an Echo object to the I_Echo interface:

{
    // bind to an object...
    Echo echo;
    server.bind<I_Echo>(echo);

    // or to a std::auto_ptr<>...
    std::auto_ptr<Echo> echoAutoPtr(new Echo());
    server.bind<I_Echo>(echoAutoPtr);

    // or to a boost::shared_ptr<>...
    boost::shared_ptr<Echo> echoPtr(new Echo());
    server.bind<I_Echo>(echoPtr);

    // or to a boost::weak_ptr<>...
    boost::weak_ptr<Echo> echoWeakPtr(echoPtr);
    server.bind<I_Echo>(echoWeakPtr);
}

By default, the binding is available to clients through the name of the interface. The server can also expose several objects through the same interface, but in that case it needs to explicitly set the binding names:

{
    RcfServer server(endpoint);

    // bind first object
    Echo echo1;
    server.bind<I_Echo>(echo1, "Echo1");

    // bind second object
    Echo echo2;
    server.bind<I_Echo>(echo2, "Echo2");

    server.start();

    RcfClient<I_Echo> echoClient(endpoint);

    echoClient.getClientStub().setServerBindingName("Echo1");
    std::cout << echoClient.echo("this was echoed by the echo1 object");

    echoClient.getClientStub().setServerBindingName("Echo2");
    std::cout << echoClient.echo("this was echoed by the echo2 object");
}

Marshaling

RCF follows C++ conventions when it comes to determining in which directions arguments are marshaled. In particular, all arguments to an interface method are in parameters, all non-const reference arguments are in-out parameters, and the return value is an out parameter. There are currently no provisions for overriding these conventions along the lines of the in/out/inout qualifiers in IDL definitions.

Not everything in C++ can be safely marshaled, and this places some limitations on the types of arguments to interface methods. To wit: pointers and references are allowed as arguments; references to pointers are not allowed as arguments; pointers and references are not allowed as return values.

This means that if an interface method needs to return a pointer -- for instance, a polymorphic pointer -- then the return type needs to be a smart pointer such as std::auto_ptr<> or boost::shared_ptr<>. Alternatively, one of the arguments could be a non-const reference to a smart pointer.

Serialization

The echo example only serialized a std::string object, but it is possible to send almost any C++ class or structure within the limits of the serialization system in use. RCF has its own serialization framework, prosaically named Serialization Framework (SF), but it also supports the Boost.Serialization framework, part of the Boost library.

In general, it is necessary to include serialization code for the classes that are being marshaled. If you have a std::vector<> argument in an interface, you will need to include <SF/vector.hpp> or <boost/serialization/vector.hpp> (or both), and similarly for other STL and Boost classes.

To use your own classes as arguments in RCF interfaces, you will need to define custom serialization functions. In most cases, it's quite simple:

#include <boost/serialization/string.hpp>

#include <boost/serialization/map.hpp>
#include <boost/serialization/vector.hpp>

#include <SF/string.hpp>

#include <SF/map.hpp>
#include <SF/vector.hpp>

struct X
{
    int myInteger;
    std::string myString;
    std::map<
        std::string,
        std::map<int,std::vector<int> > > myMap;
};

// this serialization function
// will work as is with both SF and B.S.

template<typename Archive>

void serialize(Archive &ar, X &x)
{
    ar & x.myInteger & x.myString & x.myMap;
}

// if you need to distinguish between SF and B.S. serialization,
// specialize the SF serialization function:
//void serialize(SF::Archive &ar, X &x)
//{
//    ar & myInteger & myString & myMap;
//}

Serialization can quickly become a complex affair, especially when dealing with pointers of polymorphic objects and cycles of pointers. Both SF and Boost.Serialization handle these situations, but in different ways, and catering to both may require writing separate serialization code for each serialization system. The following is an example of sending polymorphic objects, using both SF and Boost.Serialization:

class Base
{
    // some members here
    // ...
};

typedef boost::shared_ptr<Base> BasePtr;

class Derived1 : public Base
{
    // some members here
    // ...
};

class Derived2 : public Base
{
    // some members here
    // ...
};

template<typename Archive>

void serialize(Archive &ar, Base &base, const unsigned int)
{
    // ...
}

template<typename Archive>
void serialize(Archive &ar, Derived1 &derived1, const unsigned int)
{
    // valid for both SF and B.S.
    serializeParent<Base>(derived1);

    // ...
}

template<typename Archive>
void serialize(Archive &ar, Derived2 &derived2, const unsigned int)
{
    // valid for both SF and B.S.
    serializeParent<Base>(derived1);

    // ...
}

// Boost type registration, needed on both server and client
BOOST_CLASS_EXPORT_GUID(Derived1, "Derived1")
BOOST_CLASS_EXPORT_GUID(Derived2, "Derived2")

RCF_BEGIN(I_PolymorphicArgTest, "")
    RCF_METHOD_R1(std::string, typeOf, BasePtr)
RCF_END(I_PolymorphicArgTest)

class PolymorphicArgTest
{
public:
    std::string typeOf(BasePtr basePtr)
    {
        return typeid(*basePtr).name();
    }
};

{
    // SF type registration, needed on both server and client
    SF::registerType<Derived1>("Derived1");
    SF::registerType<Derived2>("Derived2");

    RcfClient<I_PolymorphicArgTest> client(endpoint);

    // SF serialization (default)
    client.getClientStub().setSerializationProtocol(RCF::SfBinary);
    std::string typeBase = client.typeOf( BasePtr(new Base()) );
    std::string typeDerived1 = client.typeOf( BasePtr(new Derived1()) );
    std::string typeDerived2 = client.typeOf( BasePtr(new Derived2()) );

    // Boost serialization
    client.getClientStub().setSerializationProtocol(RCF::BsBinary);
    typeDerived2 = client.typeOf( BasePtr(new Derived2()) );
}

Inheritance

Multiple inheritance is now supported for RCF interfaces. Interfaces may inherit not only from each other, but also from standard C++ classes. Methods in an interface are identified by the combination of their dispatch ID and the name of the interface that they belong to. This information suffices for the server to map an incoming client call to the correct function on the server binding. The example below demonstrates interface inheritance:

RCF_BEGIN(I_A, "I_A")
    RCF_METHOD_V0(void, func1)
RCF_END(I_Base)

RCF_BEGIN(I_B, "I_B")
    RCF_METHOD_V0(void, func2)
RCF_END(I_Base)

// derives from I_A
RCF_BEGIN_INHERITED(I_C, "I_C", I_A)
    RCF_METHOD_V0(void, func3)
RCF_END(I_Base)

// derives from I_A and I_B
RCF_BEGIN_INHERITED_2(I_D, "I_D", I_A, I_B)
    RCF_METHOD_V0(void, func4)
RCF_END(I_Base)

class I_E
{
public:
    virtual void func5() = 0;
};

// derives from abstract base class I_E
RCF_BEGIN_INHERITED(I_F, "I_F", I_E)
    RCF_METHOD_V0(void, func5)
RCF_END(I_Base)

{
    RcfClient<I_C> clientC(endpoint);
    clientC.func3();
    clientC.func1();

    RcfClient<I_D> clientD(endpoint);
    clientD.func4();
    clientD.func2();
    clientD.func1();

    RcfClient<I_F> clientF(endpoint);
    I_E &e = clientF;
    e.func5();
}

Filters

RCF supports compression and encryption of messages out of the box, by way of a filter concept. Filters are applied to both server- and client-side, and can be applied either to the transport layer -- for instance, applying an SSL filter to a stream-oriented transport like TCP -- or to individual message payloads such as, for instance, compression of messages destined for a packet-oriented transport like UDP. The first case will be referred to as a transport filter, and the second as a payload filter.

Transport Filters

The process of installing a transport filter on a server-client conversation is initiated by the client. The client queries the server to ascertain if the server supports a given filter. If the server does, the filter is installed on both ends of the transport and the communication resumes.

The procedure of querying the server and installing the filter is handled automatically by the client stub. The user only needs to call ClientStub::setTransportFilters() and immediately before the next remote call is made, the client stub will make a call of its own to query the server and install the filter.

Transport filters can be two-way, in the sense that a single read or write operation may well result in multiple read and write requests being made in the downstream direction. A typical example of that is an SSL encryption filter that may, at any moment, need to initiate a handshake operation involving multiple downstream read and write requests.

Payload Filters

Payload filters do not require any particular steps on behalf of either the server or the client. If a client stub has been imbued with a sequence of payload filters via ClientStub::setPayloadFilters(), then the payload of the message will be passed through the given filters. The payload will be prefixed with enough data to enable the server to decode which filters have been used, thereby allowing the server to decode the payload. Should the server not recognize a filter, an exception is passed back to the client.

Payload filters must be one-way, i.e. a read operation can only result in one or more read operations in the downstream direction, and similarly for write operations. RCF comes pre-supplied with several filters: two for compression, based on Zlib, and one for SSL encryption, based on OpenSSL. These filters can also be chained to each other to create sequences of filters.

The OpenSSL encryption filter can only be used as a transport filter, since the process of SSL encryption and decryption requires a two-way conversation, as mentioned above. On the other hand, both the stateless and the stateful compression filters can be used either as transport or payload filters. Over non-stream-oriented transports such as UDP, it will only be meaningful to use the stateless compression filter. However, over transports like TCP, either or both can be used.

An example:

{
    // compression of payload
    RcfClient<I_X> client(endpoint);
    client.setPayloadFilters( RCF::FilterPtr(
        new RCF::ZlibStatelessCompressionFilter() ) );

    // encryption of transport
    std::string certFile = "client.pem";
    std::string certFilePwd = "client_password";

    client.setTransportFilters( RCF::FilterPtr(
        new RCF::OpenSslEncryptionFilter(
            RCF::SslClient, certFile, certFilePwd)) ) );

    // multiple chained transport filters
    //       - compression followed by encryption
    std::vector<RCF::FilterPtr> transportFilters;

    transportFilters.push_back( RCF::FilterPtr(
        new RCF::ZlibStatefulCompressionFilter()));

    transportFilters.push_back( RCF::FilterPtr(
        new RCF::OpenSslEncryptionFilter(
            RCF::SslClient, certFile, certFilePwd)) ) );

    client.setTransportFilters(transportFilters);

    // multiple chained payload filters - double compression
    std::vector<RCF::FilterPtr> payloadFilters;

    payloadFilters.push_back( RCF::FilterPtr(
        new RCF::ZlibStatefulCompressionFilter()));

    payloadFilters.push_back( RCF::FilterPtr(
        new RCF::ZlibStatefulCompressionFilter()));

    client.setPayloadFilters(payloadFilters);
}

{
    std::string certFile = "server.pem";
    std::string certFilePwd = "server_password";

    // FilterService service enables the server to load the filters it needs
    RCF::FilterServicePtr filterServicePtr( new RCF::FilterService );

    filterServicePtr->addFilterFactory( RCF::FilterFactoryPtr(
        new RCF::ZlibStatelessCompressionFilterFactory) );

    filterServicePtr->addFilterFactory( RCF::FilterFactoryPtr(
        new RCF::ZlibStatefulCompressionFilterFactory) );

    filterServicePtr->addFilterFactory( RCF::FilterFactoryPtr(
        new RCF::OpenSslEncryptionFilterFactory(certFile, certFilePwd)) );

    RCF::RcfServer server(endpoint);
    server.addService(filterServicePtr);
    server.start();
}

Remote Object Creation

The RcfServer class allows users to expose single instances of a class to remote clients, but it makes no provisions for remote clients to create any objects on the server. This functionality is rather the purview of the ObjectFactoryService service. Once the ObjectFactoryService service has been installed into a server, it becomes possible for clients, via the I_ObjectFactory interface, to create as many objects as the service is configured to allow.

The ObjectFactoryService service also implements a garbage collection policy, whereby objects that are no longer in use (i.e. have no active client sessions and have not been accessed for a configurable duration of time) are eventually deleted. Be warned, however, that there are no security provisions built into this service at present. The objects that are created can, in principle, be accessed by any other client of the server.

{
    // allow max 50 objects to be created
    unsigned int numberOfTokens = 50;

    // delete objects after 60 s, when no clients are connected to them
    unsigned int objectTimeoutS = 60;

    // create object factory service
    RCF::ObjectFactoryServicePtr objectFactoryServicePtr(
        new RCF::ObjectFactoryService(numberOfTokens, objectTimeoutS) );

    // allow clients to create and access Echo objects, through I_Echo
    objectFactoryServicePtr->bind<I_Echo, Echo>();

    RCF::RcfServer server(endpoint);
    server.addService(objectFactoryServicePtr);
    server.start();
}

{
    RcfClient<I_Echo> client1(endpoint);
    bool ok = RCF::createRemoteObject<I_Echo>(client1);
    // client can now be used to access a newly created object on the server.


    RcfClient<I_Echo> client2(endpoint);
    client2.getClientStub().setToken( client1.getClientStub().getToken() );
    // client1 and client2 will now be accessing the same object

}

Publish/Subscribe

The publish/subscribe pattern is a well-known idiom of distributed programming. One process plays the role of a publisher and sends regular packets of information to a list of subscribers. The subscribers call in to the publisher and request subscriptions to the data that the publisher is publishing.

For packet-oriented transports such as UDP, it is relatively straightforward to build this functionality on top of existing RCF primitives. For stream-oriented transports, TCP in particular, RCF provides some extra features to enable publish/subscribe functionality. The connection that a subscriber calls in on is reversed and then subsequently used by the publisher for publishing.

This functionality is encompassed by the dual PublishingService and SubscriptionService services. The following example describes how these services are used:

RCF_BEGIN(I_Notify, "I_Notify")
    RCF_METHOD_V1(void, func1, int)
    RCF_METHOD_V2(void, func2, std::string, std::string)
RCF_END(I_Notify)

{
    RCF::RcfServer publishingServer(endpoint);
    RCF::PublishingServicePtr publishingServicePtr(
        new RCF::PublishingService() );
    publishingServer.addService(publishingServicePtr);
    publishingServer.start();

    // start accepting subscription requests for I_Notify
    publishingServicePtr->beginPublish<I_Notify>();

    // call func1() on all subscribers
    publishingServicePtr->publish<I_Notify>().func1(1);

    // call func2(std::string, std::string) on all subscribers
    publishingServicePtr->publish<I_Notify>().func2("one", "two");

    // stop publishing I_Notify and disconnect all subscribers
    publishingServicePtr->endPublish<I_Notify>();

    publishingServer.stop();
}

{
    RCF::RcfServer subscribingServer(endpoint);

    RCF::SubscriptionServicePtr subscriptionServicePtr(
        new RCF::SubscriptionService() );

    subscribingServer.addService( subscriptionServicePtr );
    subscribingServer.start();

    Notify notify;
    subscriptionServicePtr->beginSubscribe<I_Notify>(
        notify,
        RCF::TcpEndpoint(ip, port));

    // notify.func1() and notify.func2()
    // will now be remotely invoked by the publisher
    // ...
    subscriptionServicePtr->endSubscribe<I_Notify>(notify);
}

Extensibility

Transports

A (well-deserved) criticism levelled at the previous version of RCF concerned its overly close relationship with the TCP protocol. RCF now sports a transport-agnostic design and, for starters, supports both the TCP and UDP protocols. More importantly, the architecture is such that it shouldn't be too difficult for third parties to write and customize their own client and server transports. On the client side, the I_ClientTransport base class provides the hooks for making client calls. On the server side, it is instead necessary to write a service.

RcfServer Services

The RcfServer class accommodates third party extensions through a service concept. Services are notified, among other things, when the server starts and stops. Also, services can be dynamically added and removed from the server while the server is running. A typical service may bind an object to an interface on the server and may also request the server to spawn a thread (or spawn its own threads) to do some kind of periodic activity.

Server transports are implemented as services. It is thus possible to have several transports serving a single RcfServer object. Some of the services available are:

• ObjectFactoryService: Allows clients to create objects on the server
• FilterService: Allows the server to dynamically load filters in response to client requests
• PublishingService: Enables publishing functionality on a server
• SubscriptionService: Enables subscription functionality on a server

I_Service is the base class of all services and is documented in the reference documentation. To see how I_Service can be implemented, see the source code for the services mentioned above.

Portability

Compilers

RCF 0.4 has been tested on Visual C++ 7.1, Visual C++ 8.0, gcc 3.x, Borland C++ 5.6, and Metrowerks CodeWarrior 9.2. Additionally, RCF 0.9c supports Visual C++ 6.0 and gcc 2.95.

Platforms

RCF's TCP server implementation is now based on Win32 I/O completion ports, a technology limited to Windows 2000 and later. The TCP client implementation and the UDP server/client implementations are based on BSD sockets, and should be widely portable. On non-Windows platforms, and optionally on Windows as well, RCF leverages the asio library to implement TCP servers.

Building

In general, to build an application using RCF, you need to include the file src/RCF/RCF.cpp among the sources of your application. You will also need the header files of the Boost library; any of the more recent versions should be OK. If you plan on using Boost.Serialization, you will need version 1.31.0 or later.

There are several preprocessor symbols that can be used to control which parts of RCF are compiled. These symbols need to be defined globally, i.e. they should be placed in the project settings and not defined/undefined in source code.

• RCF_USE_BOOST_THREADS: Utilize the Boost threading library for mutex and thread generation capabilities. If not defined, then RCF will not be threadsafe.
• RCF_USE_BOOST_READ_WRITE_MUTEX: Utilize the read/write mutex found in version 1.32.0 of Boost. If not defined, a simple replacement will be used. Only effective if RCF_USE_BOOST_THREADS is defined.
• RCF_USE_ZLIB: Compile in support for Zlib compression.
• RCF_USE_OPENSSL: Compile in support for OpenSSL encryption.
• RCF_USE_BOOST_SERIALIZATION: Compile in support for the Boost.Serialization library.
• RCF_USE_SF_SERIALIZATION: Compile in support for RCF's built-in serialization framework. Automatically defined if neither RCF_USE_BOOST_SERIALIZATION nor RCF_USE_SF_SERIALIZATION is defined.
• RCF_NO_AUTO_INIT_DEINIT: Disables RCF's automatic initialization/deinitialization facility. If defined, the user will have to explicitly call RCF::init() and RCF::deinit() at appropriate times. In particular, this is necessary when compiling RCF into a DLL in order to avoid premature initialization.

Summing up, all third party build dependencies (Boost.Threads, Boost.Serialization, Zlib, OpenSSL) are optional. Instructions for building these libraries are beyond the scope of this article, but if you're having trouble building the Boost libraries, an option is to bypass Boost's build utility, bjam, and instead simply compile the corresponding CPP files into your application.

To use the Boost.Threads library, for instance, you can just include the CPP files in boost_root/libs/thread/src in your project. Then define one of the symbols in boost_root/boost/thread/detail/config.hpp, whichever one is appropriate (probably BOOST_THREAD_USE_LIB).

As for OpenSSL, the usual export/import restrictions apply. I won't bore anyone by quoting them here...

Testing

There is a comprehensive set of tests in the /test directory of the download, all of which should compile and run without failures. The tests can be built and run automatically using the Boost.Build utility or built and run by hand. The tests exercise more functionality than I've mentioned here and are written in part to be useful as a source of information for users.

Feedback

The current and eventual shape and scope of this library is very dependent on the feedback I get from its users. So, please let the comments fly! All viewpoints are welcome. Either post them here or email them to me at jlindrud -at- hotmail -dot- com.

And, of course, a big thanks to all those who weighed in on the previous incarnation of RCF! Your comments made a difference.

History

• 2005-12-23 - Version 0.1 released
• 2006-04-06 - Version 0.2 released

  RCF now compiles and runs on Linux and Solaris, as well as on Windows. Servers and clients can be distributed across multiple platforms, and still communicate seamlessly.

  The asio networking library is a prerequisite for using RCF on non-Windows platforms. Download asio, make sure that the asio headers are available to your compiler, and make sure the preprocessor symbol RCF_USE_ASIO is defined when compiling RCF. asio requires version 1.33.0 or later of Boost and you'll probably want to define BOOST_DATE_TIME_NO_LIB when using it in order to avoid unnecessary Boost dependencies.

  RCF 0.2 has been compiled and tested on the following compilers:

  • gcc 3.2 (MinGW on Windows)
  • gcc 3.3 (Linux)
  • gcc 3.4 (Solaris)
  • Borland C++ 5.6
  • Metrowerks CodeWarrior 9.2
  • Microsoft Visual C++ 7.1
  • Microsoft Visual C++ 8.0

  Various bugs reported on this forum have also been fixed.

  Apologies to anyone who's been waiting for this release... I originally planned to release it several months ago, but a busy work schedule (and some stubborn test cases) intervened. Sorry!

• 2006-07-30 - Version 0.3 released
  • RCF support for asio has now been advanced to version 0.3.7 from 0.3.5, by David Bergman. Thanks, David!
  • Support for asio 0.3.5 has been dropped
  • The preprocessor symbol RCF_USE_ASIO is now named RCF_USE_BOOST_ASIO
  • Miscellaneous issues reported on this forum have also been fixed
• 2006-09-19 - Version 0.4 released
  • 64 bit compatibility: RCF now compiles and runs on 64 bit Solaris, Linux and Windows platforms
  • Fast serialization of std::vector<T>, where T is primitive (char, int, etc.)
  • For those using RCF to communicate between 32 and 64 bit systems, you can avoid serialization errors caused by size discrepancies between 32 and 64 bit systems, by not using types like long or std::size_t in RCF interfaces. Use one of the boost typedefs instead (boost::int32_t, boost::uint32_t, boost::int64_t, boost::uint64_t).
  • Finally, thanks to Sören Freudiger at the Technical University of Braunschweig, for lending me an account on their 64-bit Linux machines!
• 2007-07-11 - Version 0.9c released
  • RCF 0.9c, a pre-release version of RCF 1.0, is now available from the download section of Google Code. In development for over a year, RCF 0.9c constitutes a major restructuring and upgrade of RCF 0.4. It has been thoroughly put through its paces as the networking backend of a major commercial ECM platform.
  • Applications using RCF 0.4 should have relatively few difficulties upgrading to RCF 0.9c. If you have any questions on porting your applications to 0.9c, feel free to contact me either on this forum or via email. I will be happy to help you out.
  • Among RCF 0.9c features are:
    • Zero-copy, zero-heap allocation core for fast and scalable performance
    • SSPI filters, for transparent Kerberos and NTLM authentication and encryption on Windows platforms
    • OpenSSL filter, for transparent SSL authentication and encryption
    • Server-side multithreading
    • Server-side session objects
    • Built-in runtime versioning, for backward and forward runtime compatibility
    • Robust publish/subscribe functionality
    • Support for legacy compilers, in particular Visual C++ 6, Borland C++ Builder 6 and gcc 2.95
    • Support for 64-bit compilers
• 2007-08-23 - Article content updated
• 2008-04-28 - Version 0.9d-P1 released
  • RCF 0.9d-P1 is a preview of RCF-0.9d. It has been fully tested on Windows, against the Visual C++ range of compilers (6.0, 7.1, 8.0, 9.0). The RCF 0.9d release will include full support for Linux and Solaris, and extensive documentation in the form of a User Guide.
  • Get it from the download page of the Google Code site
  • RCF 0.9d features include:
    • Win32 named pipe transport implementations (RCF::Win32NamedPipeEndpoint)
    • Boost.Thread no longer required
    • UDP multicasting and broadcasting
    • SF serialization for boost::tuple, boost::variant and boost::any
    • Support for exporting RCF from a DLL
    • Compatible with latest Boost (1.35.0) and Boost.Asio (0.3.8+) versions
    • Guaranteed wire-compatible with RCF 0.9c
• 2008-07-20 - Article content updated
• 2008-07-20 - Version 0.9d released
  • RCF 0.9d is now available for download
  • As part of the 0.9d release, the RCF User Guide is now available online.
  • As always, questions and comments are welcome!
• 2008-10-22 - Version 1.0 released
  • RCF 1.0 is now available for download
  • RCF 1.0 features include:
  • Support for more compilers (Intel C++ 9 and 10.1 for Windows, gcc 4.3)
  • Support for more platforms (FreeBSD and OS X)
  • Compatible with Boost 1.36.0 and 1.37.0
  • Supports use of UNIX domain sockets as transport (RCF::UnixLocalEndpoint)
  • Updates to the RCF User Guide
  • Thanks to Duane Murphy for providing the original OS X port.
  • If you come across any bugs, please either report them to me via email, or enter them into the issue tracker over at Google Code.

Licenses

Please note: The code for each download is entirely separate. The licenses for the article and code are as follows:

• The article is licensed under The Code Project Open License (CPOL)
• RCF 0.4 is licensed under the MIT license
• RCF 0.9c and successors are licensed under the GPL license

You must Sign In to use this message board.

FAQ  Noise Tolerance Very HighHighMediumLowVery Low  Search Messages    Layout Expand All MessagesThread ViewNo Javascript (slow)No Javascript Preview    Per page 102550

Msgs 1 to 25 of 598 (Total in Forum: 598) (Refresh) FirstPrevNext

Possible memory leak when disconnected during a remote call? [modified] etomacruz 19:52 26 Nov '08   Hi Jarl, I'm doing a file streaming performance test on RCF with 50 simultaneous clients. Each of this client is a separate process that sends a 1MB file to the server repeatedly 50x. I'm using RCF::ByteBuffer as an argument to hold the 100KB per file chunk sent to the server. On some occasions in some client processes when sending a file chunk to the server it is interrupted by a temporary disconnection with error code #31 (Client read operation failed) based on the RCF::Exception handler. I handle this problem by just repeating file streaming routine from the start and closing the open file handle in the server to avoid resource leak in the server, thus, successfully sending the file to the server even if temporary disconnections occur. My question is will such remote call that sends a chunk of data to the server during a disconnected state or disconnected while the data chunk is sent cause a memory leak in the RCF server? Because I observed that the RCF server process does not return to it's normal memory usage when such errors occur, and the more file streaming interruptions occur the greater the memory usage (leak) becomes. If yes, is there a remedy for this? Regards, Jun modified on Thursday, November 27, 2008 1:45 AM Sign In·View Thread·PermaLink 1.00/5 (1 vote) Re: Possible memory leak when disconnected during a remote call? Jarl Lindrud 2:43 27 Nov '08   Hi Jun, There shouldn't be any leaks in either client or server, regardless of whether any exceptions are thrown. If I understand correctly, you are getting a RCF::Exception thrown on the client side, with error code 31. There should be an operating system error code inside the exception as well, can you see what it is? When the client encounters an error, it closes its end of the connection, and in response the server will then close its own end of the connection. If you want to make sure this is happening, put a breakpoint in the RcfSession destructor, and check that it gets executed right after the exception is thrown on the client. Once the RcfSession is destroyed, all the resources held by that connection are freed. If you have a self contained sample program demonstrating the problem, that I can run, I'll be happy to do so. Regards, Jarl. Sign In·View Thread·PermaLink Re: Possible memory leak when disconnected during a remote call? etomacruz 18:01 27 Nov '08   Hello Jarl, Here's the detailed error log: Error code: 31 Error message: Client read operation failed. Subsystem: 1 Subsystem name: Operating system Subsystem error: 10054 What do u think is the cause of that error? It only occurs when the RCF server is very busy attending to 50 simultaneous client file streaming calls (each client has different file being transferred). I have tried to debug it through logging and found out that the exception error occurs in the remote call that sends/receives a file chunk to the server. The client uses dynamic binding so as I understand that the server object is not immediately destroyed when the client has temporary disconnection when the server is busy, which I take advantage of so that the client can easily repeat the sending of file to the server and avoid having corrupted files in the server. I set the cleanupThreshold = 0.01 and have maximum token of 200 so the server should clean up everytime the unused server objects reached to 2. It will be difficult to pull out the bits and pieces of code from the program to have a runnable program to demonstrate the problem so I'll just put here some code snippets: // Server side RCF_BEGIN(ISITE, "SiteInterface") RCF_METHOD_R2(short, OpenFile, std::wstring, bool); // OpenFile(wstrFileName, bMode) RCF_METHOD_R0(RCF::ByteBuffer, ReadFile); RCF_METHOD_R1(short, WriteFile, RCF::ByteBuffer); // WriteFile(FileChunk) RCF_METHOD_R0(short, CloseFile) RCF_METHOD_R0(short, TestConnection) RCF_END(ISITE); class SiteImpl { public: SiteImpl(); ~SiteImpl(); short OpenFile(std::wstring wstrFileName, bool bMode); // Opens the local file in the server for reading (true)/writing short WriteFile(RCF::ByteBuffer FileChunk); // Writes the file chunk to the previously opened file RCF::ByteBuffer ReadFile(); // Returns the read file chunk short CloseFile(); // Closes the file after all file chunks have been read/written short TestConnection(); // Returns 0; }; //Client side class ClientImpl { RcfClient<ISITE> m_IServerPtr; public: ClientImpl(); //Get RCF connection here ~ClientImpl(); short TestConnection() { short sError = -1; short sTries = 5; do { sError = -1; sTries--; sError = m_IServerPtr.TestConnection(); if (sError != 0) Sleep(2000); while ((sError != 0) && (sTries > 0)); return sError; } short SendFileToServer(std::wstring wstrFileName) { int iLocalFile = 0; short sError = MNI_E_FAIL, sAttempts = RCF_CONNECTION_ATTEMPTS; bool bExceptionErr = false; do { bExceptionErr = false; sAttempts--; sError = MNI_E_FAIL; std::wstring wstrFilePath = LOCAL_DIR + L"\\" + wstrFileName; iLocalFile = _wopen(wstrFilePath.c_str(), _O_RDONLY | _O_BINARY ); if (iLocalFile == -1) { DoLog(L"Client", L"SendFileToServer", L"Unable to open local file", LL_ERROR); continue; } char szFileChunk[MAX_RCF_CHUNK_SIZE + 1]; int nBytesRead = 0; TestConnection(); try { if (sError = m_IServerPtr.OpenFile(wstrFileName, MODE_READ) != MNI_S_OK) { _close(iLocalFile); DoLog(L"Client", L"SendFileToServer", L"Failed to create file in the server", LL_ERROR); continue; } } catch (const RCF::Exception& e) { std::wstring wstrErrInfo = GetExceptionInfo(e); _close(iLocalFile); bExceptionErr = true; DoLog(L"Client", L"SendFileToServer", L"RCF exception error on ISITE::OpenFile", LL_ERROR, wstrErrInfo); continue; } try { do { nBytesRead = 0; memset(szFileChunk, 0, MAX_RCF_CHUNK_SIZE + 1); nBytesRead = _read(iLocalFile, szFileChunk, MAX_RCF_CHUNK_SIZE); if (nBytesRead > 0) { RCF::ByteBuffer byteBuffer(szFileChunk, strlen(szFileChunk)); sError = m_IServerPtr.WriteFile(byteBuffer); //Exception always occur here when server is busy if (sError != MNI_S_OK) break; } } while (nBytesRead > 0); _close(iLocalFile); m_IServerPtr.CloseFile(); } catch (const RCF::Exception& e) { std::wstring wstrErrInfo = GetExceptionInfo(e); _close(iLocalFile); bExceptionErr = true; DoLog(L"Client", L"SendFileToServer", L"RCF exception error on file streaming", LL_ERROR, wstrErrInfo); continue; } if (sError != MNI_S_OK) { DoLog(L"Client", L"SendFileToServer", L"Server file write error", LL_ERROR); continue; } sError = MNI_S_OK; } while ((bExceptionErr || (sError != MNI_S_OK)) && (sAttempts > 0)); if (bExceptionErr || (sError != MNI_S_OK)) { DoLog(L"Client", L"SendFileToServer", L"Failed to send file to server queue", LL_ERROR); return MNI_E_FAILED_FILE_STREAMING; } return MNI_S_OK; } }; Regards, Jun Sign In·View Thread·PermaLink Re: Possible memory leak when disconnected during a remote call? Jarl Lindrud 21:33 27 Nov '08   Thanks for that. The issue here is probably with how the ObjectFactoryService is being used. When you create the ObjectFactoryService, what do you pass in as objectTimeoutS? If it is e.g. 30 seconds, then an object will only be cleaned up if it has been untouched for at least 30 seconds. What I would suggest, though, is that when your clients are done using their SiteImpl object, have them explicitly delete it, by calling m_IServerPtr.getClientStub().deleteRemoteObject(); You could also check that when all your clients have finished, if you wait for objectTimeoutS seconds, you should see that all the SiteImpl objects get deleted on the server. Once that happens, memory usage should be back to approximately what it was before the clients started. Regards, Jarl. Sign In·View Thread·PermaLink Re: Possible memory leak when disconnected during a remote call? etomacruz 22:57 27 Nov '08   I set the objectTimeoutS to 60 sec. Ok, I'll try to have the server objects deleted explicitly by the clients and see what happens. Thanks, Jarl! Sign In·View Thread·PermaLink Server Singleton Object suiram40 10:34 29 Oct '08   Hi, I am posting this again. It seems that all remote objects by default are 'singletons'. RcfClient* p = new RcfClient( RCF::TcpEndpoint(Gip, 50001)); // instance 1 RcfClient* q = new RcfClient( RCF::TcpEndpoint(Gip, 50001)); // instance 2 std::string pname = "p-name"; std::string qname; p->SetName(name); q->GetName(qname); qname is 'p-name' never been set. My question is how I can avoid this. Thank you. Marius C. Sign In·View Thread·PermaLink Re: Server Singleton Object Jarl Lindrud 19:03 29 Oct '08   Hi Marius, By default, all clients will access the same object (the one that you bound by calling RcfServer::bind<>()). If you want each client to have its own object, you can create session objects instead; have a look at the section on session binding in the User Guide. Regards, Jarl. Sign In·View Thread·PermaLink Re: Server Singleton Object suiram40 19:18 29 Oct '08   Thank you. Problem solved. Sign In·View Thread·PermaLink Linux Builds suiram40 19:24 28 Oct '08   I wanted to build the server under Linux distro(s) and I cannot figure out following error: -server.cpp: undefined reference to 'boost::system::get_system_category(); when defining RCF_USE_BOOST_ASIO. Also forcing the RCF_USE_BOOST_ASIO under Windows OS comes out with -cannot open file 'libboost_system-vcXX-mt-s-1_36.lib' Thank you. Marius C @ zalsoft.com Sign In·View Thread·PermaLink Re: Linux Builds Jarl Lindrud 20:03 28 Oct '08