Plugin System – an alternative to GetProcAddress and interfaces

• Download source code and binaries - 125 Kb

Introduction

Plugins are the common way for extending applications. They are usually implemented as DLLs. The host application locates the plugins (either by looking in a predefined folder, or by some sort of registry setting or configuration file) then loads them one by one with LoadLibrary. The plugins are then integrated into the host application which extends it with new functionality.

This article will show how to create a host EXE with multiple plugin DLLs. We'll see how to seamlessly expose any of the host's classes, functions and data as an API to the plugins. There will be some technical challenges that we are going to solve along the way.

We'll use a simple example. The host application host.exe is an image viewer. It implements a plugin framework for adding support for different image file formats (24-bit BMP and 24-bit TGA in this example). The plugins will be DLLs and will have extension .IMP (IMage Parser) to separate them from regular DLLs. Note however, that this article is about plugins, not about parsing images. The provided parsers are very basic and for demonstration purpose only.

There are many articles describing how to implement a simple plugin framework. See [1], [2] for examples. They usually focus on two approaches:

1. The plugin implements a standard (and usually small) set of functions. The host knows the names of the functions and can find the address using GetProcAddress. This doesn't scale well. As the number of functions grows the maintenance gets harder and harder. You can only do so much if you have to manually bind every function by name.
2. The function returned by GetProcAddress is used to pass an interface pointer to the plugin or to obtain an interface pointer from the plugin. The rest of the communications between the host and the plugin is done through that interface. Here's how you do it:

The interface way

Interfaces are base classes where all member functions are public and pure virtual, and there are no data members. For example:

// IImageParser is the interface that all image parsers
// must implement
class IImageParser
{
public:
    // parses the image file and reads it into a HBITMAP
    virtual HBITMAP ParseFile( const char *fname )=0;
    // returns true if the file type is supported
    virtual bool SupportsType( const char *type ) const=0;
};

The actual image parsers inherit from the interface class and implement the pure virtual functions. The BMP plugin can look like this:

// CBMPParser implements the IImageParser interface
class CBMPParser: public IImageParser
{
public:
    virtual HBITMAP ParseFile( const char *fname );
    virtual bool SupportsType( const char *type ) const;

private:
    HBITMAP CreateBitmap( int width, int height, void **data );
};

static CBMPParser g_BMPParser;
// The host calls this function to get access to the
// image parser
extern "C" __declspec(dllexport) IImageParser *GetParser( void )
{
    return &g_BMPParser;
}

The host will use LoadLibrary to load BmpParser.imp, then use GetProcAddress("GetParser") to find the address of the GetParser function, then call it to get the IImageParser pointer.

The host keeps a list of all registered parsers. It adds the pointers returned by GetParser to that list.

When the host needs to parse a BMP file it will call SupportsType(".BMP") for each parser. If SupportsType returns true, the host will call ParseFile with the full file name and will draw the HBITMAP.

For complete sources see the Interface folder in the download file.

The base class doesn't really have to be pure interface. Technically the constraint here is that all members have to be accessible through the object's pointer. So you can have:

• pure virtual member functions (they are accessed indirectly through the virtual table)
• data members (they are accessed through the object's pointer directly)
• inline member functions (they are not technically accessed through the pointer, but their code is instantiated a second time in the plugin)
• That leaves non-inline and static member functions. The plugin cannot access such functions from the host and the host cannot access such functions from the plugin. Unfortunately in a large application such functions can be the majority of the code.

For example, all image parsers need the CreateBitmap function. It makes sense for it to be declared in the base class and implemented on the host side. Otherwise each parser DLL will have a copy of that function.

Another limitation of this approach is that you cannot expose any global data or global functions from the host to the plugins.

So how can we improve this?

Split the host into DLL and EXE

Take a look at the USER32 module. It has two parts – user32.dll and user32.lib. The real code and data is in the DLL, and the LIB just provides placeholder functions that call into the DLL. The best part is that all this happens automatically. You link with user32.lib and automatically gain access to all functionality in user32.dll.

MFC goes a step further – it exposes whole classes that you can use directly or inherit. They do not have the limitations of the pure interface classes we discussed above.

We can do the same thing. Any base functionality you want to provide to the plugins can be put in a single DLL. Use the /IMPLIB linker option to create the corresponding LIB file. The plugins can then link with that library, and all exported functionality will be available to them. You can split the code between the DLL and the EXE any way you wish. In the extreme case shown in the sources the EXE only contains a one line WinMain function whose only job is to start the DLL.

Any global data, functions, classes, or member functions you wish to export must be marked as __declspec(dllexport) when compiling the DLL and as __declspec(dllimport) when compiling the plugins. A common trick is to use a macro:

#ifdef COMPILE_HOST
// when the host is compiling
#define HOSTAPI __declspec(dllexport)
#else
// when the plugins are compiling
#define HOSTAPI __declspec(dllimport)
#endif

Add COMPILE_HOST to the defines of the DLL project, but not to the plugin projects.

On the host DLL side:

// CImageParser is the base class that all image parsers
// must inherit
class CImageParser
{
public:
    // adds the parser to the parsers list
    HOSTAPI CImageParser( void );
    // parses the image file and reads it into a HBITMAP
    virtual HBITMAP ParseFile( const char *fname )=0;
    // returns true if the file type is supported
    virtual bool SupportsType( const char *type ) const=0;

protected:
    HOSTAPI HBITMAP CreateBitmap( int width, int height,
        void **data );
};

Now the base class is not constrained of being just an interface. We are able to add more of the base functionality there. CreateBitmap will be shared between all parsers.

This time instead of the host calling a function to get the parser and add it to the list, that part is taken over by the constructor of CImageParser. When the parser object is created its constructor will automatically update the list. The host doesn't need to use GetProcAddress to see what parser is in each DLL anymore.

On the plugin side:

// CBMPParser inherits from CImageParser
class CBMPParser: public CImageParser
{
public:
    virtual HBITMAP ParseFile( const char *fname );
    virtual bool SupportsType( const char *type ) const;
};
static CBMPParser g_BMPParser;

When g_BMPParser is created, its constructor CBMPParser() will be called. That constructor (implemented on the plugin side) will call the constructor of the base class CImageParser() (implemented on the host side). That's possible because the base constructor is marked as HOSTAPI.

For complete sources see the DLL+EXE folder in the download file.

Wait, it gets even better:

Combine the host DLL and the host EXE

Usually an import library is created only when making DLLs. It is a little known trick that import library can be created even for EXEs. In Visual C++ 6 the /IMPLIB option is not available directly for EXEs as it is for DLLs. You have to add it manually to the edit box at the bottom of the Link properties. In Visual Studio 2003 it is available in the Linker\Advanced section, you just have to set its value to $(IntDir)/Host.lib.

So there you go. You have a host EXE, a number of plugin DLLs, and you can share any function, class or global data in the host with all plugins. There is no need to use GetProcAddress at all, ever, since the plugins can register themselves with the host's data structures.

For complete sources see the EXE folder in the download file.

The DEF file

As the host application grows bigger you will want to split it into separate static libraries. And then you are going to hit a problem.

Let's say the constructor of CImageParser is in one of the libraries and not in the main project. There is no code in the main project that refers to that function (obviously, only plugins will need to call it from their own constructors). The linker, being smart, will decide there is no use for such functions and will remove it from the EXE.

So how do you trick the linker into adding the constructor to the EXE? This is a perfect task for a DEF file. A DEF file is a text file listing all symbols that the DLL or EXE will export. The linker will be forced to include them into the output, even if no code refers to them. A DEF file can look like this:

EXPORTS
    // the C++ decorated name for the CImageParser constructor
    ??0CImageParser@@QAE@XZ

    // the C++ decorated name for CImageParser::CreateBitmap
    ?CreateBitmap@CImageParser@@IAEPAUHBITMAP__@@HHPAPAX@Z

To give a DEF file to the linker in VC6 you have to manually add the option /DEF:<filename> to the command line. In VS2003 you can do that in the Linker\Input section.

How do you create the DEF file? You can do it manually by listing all symbols you want to export, or you can do it automatically:

defmaker – create DEF files automatically

defmaker is a simple tool that scans LIB files, finds all symbols that are exported by the libraries, and adds them to a DEF file.

// defmaker - creates a DEF file from a list of libraries.
// The output DEF file will contain all _declspec(dllexport)
// symbols from the libraries.
// /def:<def file> must be added to the linker options
// for the DLL/EXE.
//
// Parameters:
//   defmaker <output.def> <library1.lib> <library2.lib> ...
//
// Part of the Plugin System tutorial
//
/////////////////////////////////////////////////////////////

#pragma warning( disable: 4786 ) // Identifier was truncated
                  // to 255 characters in the debug info

#include <stdio.h>
#include <windows.h>
#include <string>
#include <set>
#include <Dbghelp.h>

struct StrNCmp
{
    bool operator()(const std::string &s1,
                    const std::string &s2) const
    {
        return stricmp(s1.c_str(),s2.c_str())<0;
    }
};
std::set<std::string,StrNCmp> g_Names;

static const char *EXPORT_TAG[]=
{
    "/EXPORT:", // VC6 SP5, VC7.1, VC8.0
    "-export:", // VC6 SP6
};

static bool CmpTag( const char *data )
{
    for (int i=0;i<sizeof(EXPORT_TAG)/
             sizeof(EXPORT_TAG[0]);i++)
        if (strnicmp(EXPORT_TAG[i],data,
                 strlen(EXPORT_TAG[i]))==0)
            return true;
    return false;
}

static bool ParseLIB( const char *fname )
{
    int len=strlen(EXPORT_TAG[0]);

    bool err=true;
    // create a memory mapping of the LIB file
    HANDLE hFile=CreateFile(fname,GENERIC_READ,
         FILE_SHARE_READ,NULL,OPEN_EXISTING,
         FILE_ATTRIBUTE_NORMAL|FILE_FLAG_RANDOM_ACCESS,0);
    if (hFile!=INVALID_HANDLE_VALUE) {
        HANDLE hFileMap=CreateFileMapping(hFile,NULL,
             PAGE_READONLY,0,0,0);
        if (hFileMap!=INVALID_HANDLE_VALUE) {
            const char *data=
                  (const char *)MapViewOfFile(hFileMap,
                  FILE_MAP_READ,0,0,0);
            if (data) {
                err=false;
                // search for the EXPORT_TAG and
                // extract the symbols
                int size=GetFileSize(hFile,NULL);
                for (int i=0;i<size-len;i++)
                    if (CmpTag(data+i)) {
                        i+=len;
                        const char *text=data+i;
                        while (data[i]!=0 && data[i]!=' '
                               && data[i]!='/' && i<size)
                            i++;
                        std::string name(text,data+i);
                        // add the symbols to a sorted set
                        g_Names.insert(name);
                    }
                UnmapViewOfFile(data);
            }
            CloseHandle(hFileMap);
        }
        CloseHandle(hFile);
    }
    return !err;
}

int main( int argc, char *argv[] )
{
    if (argc<3) {
        printf("defmaker: Not enough command line parameters.\n");
        printf("Usage: defmaker <def file> <libfiles>\n");
        return 1;
    }
    for (int i=2;i<argc;i++) {
        printf("!defmaker: Parsing library %s.\n",argv[i]);
        if (!ParseLIB(argv[i])) {
            printf("defmaker: Failed to parse library %s.\n",
                  argv[i]);
            return 1;
        }
    }
    FILE *def=fopen(argv[1],"wt");
    if (!def) {
        printf("defmaker: Failed to open %s for writing.\n",
               argv[1]);
        return 1;
    }
    fprintf(def,"EXPORTS\n");
    for (std::set<std::string,StrNCmp>::iterator it=
             g_Names.begin();it!=g_Names.end();++it) {
        std::string name=*it;
        int len=name.size();
        if (len>5 && name[len-5]==',')
            name[len-5]=' '; // converts ",DATA" to " DATA"
        fprintf(def,"\t%s\n",name.c_str());
    }
    fclose(def);
    printf("defmaker: File %s was created successfully.\n",
           argv[1]);
    return 0;
}

You use it like this:

defmaker <output.def> <library1.lib> <library2.lib> ...

For our example the command line is:

defmaker "$(IntDir)\host.def" "ImageParser\$(IntDir)\ImageParser.lib"

In VC6 you add this to the Pre-link step tab of the linker options. In VS2003 you do this in Build Events\Pre-Link Event in the project's options. It is going to be executed just before the linking step. Defmaker will produce the host.def file, which will then be used by the linker.

Defmaker locates the symbols by searching for the "/EXPORT:" tag in the LIB file. (Note: for some unknown reason only in VC6 service pack 6 the tag has changed to "-export:", so defmaker searches for both). The decorated C++ name of the symbol is found immediately after the tag. If the symbol refers to data instead of code it will be followed by the text ",DATA". The DEF file format requires data symbols to be marked with "<space>DATA" instead. Defmaker will convert one to the other. Probably it will be better to parse the LIB file following the official file format specs, but I have found that searching for the tags to be 100% successful.

Another use of defmaker is not related to plugins or DLLs. Sometimes you need to force the linker to include a global object even though there are no references to it. A common example is a factory system where each factory is a global object that registers itself in a list (like CImageParser does above). But if your factory object is in a static library and not in the main project the linker may decide to remove it. With defmaker you can mark the object with __declspec(dllexport) and it will be added to the EXE file.

Tip: Add the path to defmaker.exe to Visual Studio's Executable files settings. You will be able to use it from any project.

Conclusion

We've seen here how to create a plugin system without relying on GetProcAddress or trying to squeeze all functionality through interfaces. To expose any symbol from the host to the plugins just mark it with HOSTAPI. The rest is automatic. You have direct control which symbols to export and which not to.

You would write code in the plugin just as easily as writing code in a monolithic application or a static library. You can have access to base classes, global functions, and global data no matter if you write a plugin or a simple application. It is still a very good idea to have a clear separation between host functionality and plugin functionality, but it should be based on your own architecture and not dictated by technical limitations.

A word of caution - with great power comes great responsibility. You have the power to share as much or as little of the host's internals with the plugins. A key to a well designed plugin framework as with any design is finding the balance - in this case providing a simple yet powerful API. You need to export enough functionality to aid the plugin developers, yet hide the features that are likely to change in future versions or will needlessly compromise the stability of the host.

The source code

The source zip file contains four folders:

• Interface - a plugin system using GetProcAddress and interfaces.
• DLL+EXE - a plugin system using separate EXE and DLL for the host.
• EXE - the final plugin system using a single EXE for the host.
• defmaker - the sources for the defmaker tool. A compiled version of defmaker.exe is included in the root folder. It is required for compiling the EXE version of the host.

The sources contain project files for Visual C++ 6 and Visual Studio 2003. For Visual Studio 2005 you can open either project and convert it to the latest format.

Links:

[1] Plug-In framework using DLLs by Mohit Khanna
[2] ATL COM Based Addin / Plugin Framework With Dynamic Toolbars and Menus by thomas_tom99