Click here to Skip to main content
15,867,141 members
Articles / Programming Languages / C++
Article

How to do run-time (or explicit) linking of C++ plug-in components and objects

Rate me:
Please Sign up or sign in to vote.
4.43/5 (14 votes)
2 Jan 2001CPOL 303K   1.9K   107   73
Extending the functionality of your programs using explicit linking
  • Download demo project - 59 Kb
  • Introduction

    How can we create a program that will be able to work with objects that do not even exist at the time when our executable is conceived? Would it not be nice if we could design a program, which functionality we would be able to extend without altering the original executable’s source (and even without recompiling): an application where we can plug-in additional functionality on the fly.

    This is possible if we use the proposed development scheme of working with plug-in objects. What we need is a framework that works for predefined base objects, but that is able to load DLLs containing (new) child objects that extend, alter or specialize the behaviour. It is even possible to manipulate different generations of these objects: They co-exist peacefully together.

    One of the difficulties of working with DLLs and allocating and de-allocating (and thus exchanging) memory is the way the heap manager works. Unless one uses GlobalAlloc, it is imperative that all the memory allocated in a particular DLL, is also deleted or de-allocated by the same DLL (read 'the same heap manager' instead of 'the same DLL'). If one does not follow this rule, there is the distinct possibility that one will try to delete memory that is unknown by the heap manager of the ‘called’ DLL. These bugs are fatal and very difficult to find!

    Let say you created an object 'A' inside the DLL called 'A-DLL'. You pass this object to the executable ('Bogus.exe') or another DLL called 'B-DLL' and you try to de-allocate the memory associated with Object 'A'. The heap manager of 'Bogus.exe' or 'B-DLL' does not know about the existence of the memory associated with object 'A' and could thus delete invalid memory. If you would use the GlobalAlloc function and sorts, you do not use the heap managers, but instead memory is directly obtained from the operating system. This approach certainly works, but IMHO is very ugly and hardly C++.

    What is proposed here is a scheme for C++ Objects.

    It is better to enforce the rule "de-allocate where you allocated it". That's where the DLLProxy Class comes into the picture. Only one instance of this DLLProxy exists per DLL. And this instance is responsible for allocation and de-allocation of our (plug-in) objects in the DLLs. In fact, these objects, handle the memory management for objects created inside a DLL.

    When the application is designed, one must of course know what kind of functionality is going to be extended in the future: communication protocols, MFC views or documents, windows, different languages for code generation, support future hardware equipment or new algorithms… Almost everything is possible if the function class interface is properly defined.

    The behaviour will be altered by the virtual function mechanism later on. In the example, you can see that the function DoSomething() is called, but what actually happens inside this function is determined by the implementation in the Plug_Object_Child class. The Plug_Object_Child class is defined inside the Plug-in DLL and was allocated by a particular DLLProxyChild Object. This DLLProxyChild Object is a specialization of the DLLProxy class because it has to know which Plug_Object(_Child) objects to allocate and de-allocate!

    Explicit linking is required, because the plug-in enabled executable is not even aware of the existence of the particular plug-in DLLs at compile and link time.

    Sample Image

    What kind of objects?

    When we design our application, we must of course know what kind of functionality we are going to extend in the future. Do we want to extend communication protocols, MFC views or documents, windows, different languages for code generation, support future hardware equipment or new algorithms… you name it. Almost everything is possible.

    We then create a virtual base class for our to-be-extended functionality. Our application will always work with this class definition. The behavior will be altered by the virtual function mechanism later on. In our example, we will call DoSomething(), but what actually happens inside this function is determined by the implementation in the Plug_Object_Child class. The Plug_Object_Child class is defined inside a DLL and was allocated by a particular DLLProxyChild Object. This DLLProxyChild Object is a specialization of the DLLProxy class because it has to know which Plug_Object(_Child) objects to allocate and deallocate!

    This is a simple explanation of how it works. Sounds simple for anyone familiar with DLLs and the virtual function mechanism, does it not? Of course we need EXPLICIT linking, because we do not know whether the DLLs with extended functionality exist (or how many there exist in the future) when building our plug-in enabled application.

    The Procedure

    What we do is quite simple.

    1. First, one searches a particular directory for the existence of DLLs, then their usability is determined: the DLLs must contain the necessary exported functions to obtain a DLLProxy Object. (In other words, it is a check whether the DLLs are really plug-in DLLs)
    2. Next, the DLLs are mapped in the executable’s address space (Load the library) and
    3. the function pointers to the required functions are stored. (Obtained by calling GetProcAddress)
    4. These FARPROC function pointers can be caste to the correct function prototype, so that these functions can be called to obtain the DLLProxy object.
    5. This DLLProxy Object is used to create and delete Plug_Object instances.
    6. In the end, when it is certain that no more DLLProxy objects exist, the DLLs can be mapped out (FreeLibrary).
    7. The DLLRTLoader (DLL Run-time Loader) class automated this functionality.
    8. However, do not forget to define GetDLLProxy function in every Plug-in DLL. It is a function with the following prototype:
      typedef DLLProxy* (WINAPI *GETDLLPROXY)(void);

      It is exported through the DLLs .def file when building the DLL project.

    The Classes and their Interfaces

    // Our DLL Run-Time  Loader looks like this:
    // This class is used for searching out the DLLs we can use in a particular directory.
    // We can map in or map out the DLLs dependent on whether we need them or not
    // Functions obtained from the DLLs can be called through this class.
    class DLLRTLoader  
    {
    public:
    	DLLRTLoader();
    	virtual ~DLLRTLoader();
    
    	bool LookForDlls(const char* searchdirectory, bool recursive = true);
    	void FlushUnusedDLLs(void);
    
    	// necessary func names
    	void AddFuncName(const char* theFN);
    	FileFinder* GetFileFinder(void);
    	DLLFileEntry* SearchDLLFileEntry(const char* completepath);
    	DynamicSortedArray<DLLFileEntry*>* GetMappedDLLList(void); // TOO list not elements
    	DynamicSortedArray<DLLFileEntry*>* GetDLLList(void); // TOO list not elements
    
    	unsigned  GetNumberOfUsableDlls(void);
    	unsigned  GetNumberOfMappedDLLs(void);
    
    	void Debug(ostream& theStream);
    protected:
    	virtual void CheckForDLLFuncs(void);
    	FileFinder myFileFinder;
    	DynamicSortedArray<DLLFileEntry*> myDLLs;
    	DynamicArray<char*> myFuncNames;
    };

    // This class is used to implement some kind of reference counting mechanism on the usage
    // of the DLLs. If an object is created from a particular DLL, its DLL usage reference count will be 
    // incremented and that is done by manipulating these objects
    class DLLFileEntry : public FileEntry
    {
    public:
    	DLLFileEntry(const char* path);
    	DLLFileEntry(const FileEntry&);
    	virtual ~DLLFileEntry();
    
    	inline void IncDllUsage(void) // increment Usage
    	{
    		_ASSERT(myMapped_In == true);
    		myRefCount++;
    	};
    
    	inline void DecDllUsage(void) // decrement Usage
    	{
    		_ASSERT(myMapped_In == true);
    		_ASSERT(myRefCount > 0);
    		myRefCount--;
    	};
    
    	inline HINSTANCE GetModuleHandle(void) // returns unsafe lib handle
    	{
    		return myLibHandle;
    	}
    
    	unsigned  GetRefCount(void);
    	void SetRefCount(unsigned  ref);
    
    	void SetProxy(DLLProxy* theProxy);
    	DLLProxy* GetProxy(void);
    
    	bool GetDllOK(void);
    
    	bool IsMappedIn(void); // returns whether mapped in or not
    	virtual  MapIn(void); // only allowed if not mapped in and usage count == 0
    	virtual bool CheckAndStoreFuncs(void); // checks and stores the function names
    	virtual  MapOut(void); // only allowed if mapped in and usage count == 0
    	virtual FARPROC GetFuncAddress(const char* funcname); 
    	// Get the FARPROC function address
    	void SetFuncNameList(DynamicArray<char*>* theFuncNames);
    
    	void Debug(ostream& theStream);
    protected:
    	DLLFileEntry(const DLLFileEntry&); // do not allow copy
    	DLLFileEntry& operator=(const DLLFileEntry&); // do not allow copy
    
    	void SetDllOK(bool value);
    
    	unsigned  myRefCount;
    	bool myMapped_In;
    	HINSTANCE myLibHandle;
    	Tree<FARPROC> myFuncAddresses;
    	DynamicArray<char*>* myFuncNames;
    private:
    	bool myDLLisOK;
    	DLLProxy* myProxy;
    };

    // this class will be used to implement the Object Creation/Destruction
    // Allocation and De-allocation inside the DLL
    // since memory allocated inside a DLL must also be deleted by the same DLL
    class CLASS_DECL_DLL DLLProxy
    {
    public:
    	// construction
    	DLLProxy();
    	virtual ~DLLProxy();
    
    	// creation and deletion
    	virtual Plug_Object* CreateObject(void);
    	virtual void DeleteObject(ProxyInterface* theObject);
    
    	// operations
    	// for reference counting
    	void SetDLLFE(DLLFileEntry* theDLLFE);
    	void DecDllUsage(void); // decrement Usage
    	void IncDllUsage(void); // increment Usage
    	char* GetDLLRelativePath(void); // get relative path to DLL for the proxy
    
    	// return valid HANDLE to loaded module or NULL
    	HMODULE GetSafeModuleHandle(void);
    	// return root path to module or NULL
    	static char* GetRootModulePath(void);
    	static void SetRootModulePath(const char* modulerootpath);
    
    protected:
    	// do not allow copy
    	DLLProxy(const DLLProxy&);
    	DLLProxy& operator=(const DLLProxy&);
    
    	DLLFileEntry* myDLLFE;
    
    	static char* myModuleRootPath;
    };

    // This class is a mix-in class and just brings in the DLLProxy member into an 
    // (existing) plug-in class
    class CLASS_DECL_DLL ProxyInterface  
    {
    public:
    	ProxyInterface();
    	virtual ~ProxyInterface();
    
    	// theDLL Proxy Access Functions
    	DLLProxy* GetProxy(void);
    	void SetProxy(DLLProxy* theProxy);
    
    protected:
    	// the DLLProxy pointer is not owned by the ProxyInterface
    	// therefore it is not allocated or de-allocated here
    	DLLProxy* myProxy;
    };

    // Plug-in Base Class
    // Has all the common functionality
    // This base class is virtual
    // pointers to it cannot be deleted
    // since the constructor and destructors are protected
    class CLASS_DECL_DLL Plug_Object : public ProxyInterface  
    {
    public:
    	// virtual functions specifying the interface of the Plug-in Objects
    	virtual void DoSomething(void) = 0;
    protected:
    	Plug_Object(DLLProxy* theProxy)
    {
    	myProxy = theProxy;
    }
    	virtual ~Plug_Object();
    };

    // Implementation of Create and Delete inside a DLLProxy
    // JUST for DEMONSTRATION purposes
    // will NEVER be called!
    Plug_Object* DLLProxy::CreateObject(void)
    {
    	_ASSERT(myDLLFE != NULL);
    	
    	// this line of code will be DIFFERENT for EVERY PLUG_OBJECT and DLLPROXY
    	// here a Plug_Object specialization object will be allocated, created
    	// and initialized
    	Plug_Object* toReturn = NULL;
    	if (toReturn != NULL)
    	{
    		// Do Some Reference Counting
    		myDLLFE->IncDllUsage();
    		toReturn->SetProxy(this);
    	}
    	else
    	{
    		_ASSERT(NULL);
    	}
    	return toReturn;
    }

    void DLLProxy::DeleteObject(ProxyInterface* theObject)
    {
    	if (theObject != NULL && theObject->GetProxy() == this)
    	{
    		delete theObject;
    		theObject = NULL;
    		if (myDLLFE != NULL)
    		{
    		// Do Some Reference Counting
    			myDLLFE->DecDllUsage();
    		}
    	}
    	else
    	{
    		_ASSERT(NULL);
    	}
    }

    Create a new DLL Project or in other words what does a Plug-in DLL contain?

    One Global pointer to a DLLProxy object

    DLLProxyChild1 theProxy;
    DLLProxy* theProxy = &theProxy;

    1 exported function (put the name in the .DEF file) =

    extern "C" DLLProxy* GetDLLProxy(void)
    {
    	return theProxy;
    }

    Definitions of the specific specialization Plug_Object class and the specific specialization DLLProxy class:

    class DLLProxyChild1 : public DLLProxy  
    {
    public:
    	DLLProxyChild1 ();
    	virtual ~ DLLProxyChild1 ();
    
    	Plug_Object* CreateObject(void);
    	void DeleteObject(ProxyInterface* theObject);
    };

    // Plug-in Class
    // Has all the common functionality
    class CLASS_DECL_DLL Plug_Object_Child1 : public Plug_Object  
    {
    public:
    	Plug_Object_Child1(DLLProxy* theProxy);
    	virtual ~Plug_Object_Child1();
    
    	// ACTUALLY do Something!!
    	void DoSomething(void);
    protected:
    };

    // Implementation of Create and Delete inside a DLLProxyChild1
    Plug_Object* DLLProxyChild1::CreateObject(void)
    {
    	_ASSERT(myDLLFE != NULL);
    	
    	// this line of code will be DIFFERENT for EVERY PLUG_OBJECT and DLLPROXY
    	Plug_Object_Child1* toReturn = new Plug_Object_Child1;
    	if (toReturn != NULL)
    	{
    		// Do Some Reference Counting
    		myDLLFE->IncDllUsage();
    		toReturn->SetProxy(this);
    	}
    	else
    	{
    		_ASSERT(NULL);
    	}
    	return toReturn;
    }

    void DLLProxyChild1::DeleteObject(ProxyInterface* theObject)
    {
    	if (theObject != NULL && theObject->GetProxy() == this)
    	{
    		delete theObject;
    		theObject = NULL;
    		if (myDLLFE != NULL)
    		{
    		// Do Some Reference Counting
    			myDLLFE->DecDllUsage();
    		}
    	}
    	else
    	{
    		_ASSERT(NULL);
    	}
    }

    // shared memory!
    #pragma data_seg( ".GLOBALS")
    int nProcessCount = 0;
    int nThreadCount = 0;
    #pragma data_seg()
    
    // remember! For every Process
    DLLProxyChild1 theProxy;
    DLLProxy* theProxy = &theProxy;
    
    extern "C" BOOL APIENTRY
    DllMain	(	HANDLE hModule, 
                DWORD  ul_reason_for_call, 
                LPVOID lpReserved
    		)
    {
    	// Remove this if you use lpReserved
    	UNREFERENCED_PARAMETER(lpReserved);
    
    	switch( ul_reason_for_call ) 
    	{
    case DLL_PROCESS_ATTACH:
    		{
    			nProcessCount++;
    			break;
    		}
    case DLL_THREAD_ATTACH:
    		{
    			nThreadCount++;
    			break;
    		}
    case DLL_THREAD_DETACH:
    		{
    			nThreadCount--;
    			break;
    		}
    case DLL_PROCESS_DETACH:
    		{
    			nProcessCount--;
    			break;
    		}
    default:
    		{
    			break;
    		}
        }
        return TRUE;
    }

    Serialization of Plug-in Objects

    Concerning serialization of Plug-in Objects, we can distinguish between projects or programs using MFC and projects without MFC.

    A. With MFC

    To Save

    1. Define a Serialize(CArchive& theArchive) function for every Plug_Object.
    2. As usual, call the Serialize function of the CDocument.
    3. The Plug_Objects store themselves to file through their Serialize function

    To Load

    1. Before the Serialize function of the CDocument is performed, all the necessary run-able code must be present in memory. All the DLLs, present in the DLL root path are recursively mapped in.
    2. Load the file
    3. Flush the unused DLLs.

    B. Without MFC

    Remember we have the DLL root path and we also have the relative path of the DLLs to the DLL root path. So write a function in every Plug_Object that does this:

    // This function will reconstruct the object from a BYTE stream and return NULL if not successful, 
    // Otherwise it will return the pointer for the next object to serialize. 
    // So if a particular Plug_Object needs 5 bytes to DeSerialize itself, the Original BYTE pointer 
    // address + 5 is returned  (which you can pass to the next DeSerialize…)
    BYTE*  DeSerialize(const BYTE* pFileData);
    
    // This function constructs the BYTE stream to save and returns it, and of course also
    // sets the size of the BYTE stream so you know how many bytes to write to your file/pipe/…
    BYTE* Serialize( unsigned int& serialization_byte_size );

    The serialisation scheme then looks like this

    To save:

    While (still_objects_to_save)
    {
    	Save relative plug-in DLL path size in 4 bytes (so you know which plug-in code to load when loading the file)
    	Save the relative plug-in DLL path itself (to identify the correct object)
    	Serialize() the object and save it
    }

    To load:

    While (still_data_to_read)
    {
    	Read byte stream
    	4 bytes (for the size) -> relative path string
    	Load Plug-in DLL if not already loaded (with relative path)
    	Construct object from Plug-in DLL
    	Call DeSerialize() on Plug-in object (in fact this is a 2 phase construction!)
        // Repeat the above steps until the stream is finished (the entire file is read)
    }

    Some more info...

    More information about the difference between a similar scheme, which I discovered some time ago in MSDN, and COM can be found in the MSDN Article "From CPP to COM" by Markus Horstmann, where COM is presented as a superior (?) solution.

    It has been pointed out to me that there is a more general solution to be found on Dynamic C++ Classes as "a lightweight mechanism to update code in a running program" at http://actcomm.dartmouth.edu/dynamic/

    See the sample project for a demonstration of its usage. I hope all things all clear. If they are not: try stepping through the debugger, that sometimes helps. If something is not clear in the above explanation, let me know and I will try to clarify things!

    Updates

    Now works with VC++ 6

    License

    This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


    Written By
    Technical Lead rtos.be
    Belgium Belgium
    Gert Boddaert is an experienced embedded software architect and driver developer who worked for companies such as Agfa Gevaert, KBC, Xircom, Intel, Niko, (Thomson) Technicolor, Punch Powertrain, Fifthplay, Cisco and Barco. For more obscure details, please take a look at his LinkedIn profile. Anyway, he started out as a Commercial Engineer – option “Management Informatics”, but was converted to the code-for-food religion by sheer luck. After writing higher level software for a few years, he descended to the lower levels of software, and eventually landed on the bottom of embedded hell… and apparently still likes it down there.

    His favourite motto: “Think hard, experiment and prototype, think again, write (easy and maintainable) code”,

    favourite quote: “If you think it’s expensive to hire a professional to do the job, wait until you hire an amateur.” – by Red Adair,

    I can be contacted for real-time embedded software development projects via http://www.rtos.be and http://www.rtos.eu

    Comments and Discussions

     
    GeneralBase class defined in DLL Pin
    boffboy7-Oct-01 0:03
    boffboy7-Oct-01 0:03 
    GeneralRe: Base class defined in DLL Pin
    Gert Boddaert8-Oct-01 22:34
    Gert Boddaert8-Oct-01 22:34 
    GeneralIrritating Serialization Bug Pin
    Joel Holdsworth30-Sep-01 9:46
    Joel Holdsworth30-Sep-01 9:46 
    GeneralRe: Irritating Serialization Bug Pin
    Tom Morris5-Nov-01 11:05
    Tom Morris5-Nov-01 11:05 
    GeneralCallback functions Pin
    Mark Verlinden24-Sep-01 2:55
    Mark Verlinden24-Sep-01 2:55 
    GeneralRe: Callback functions Pin
    Gert Boddaert25-Sep-01 22:05
    Gert Boddaert25-Sep-01 22:05 
    GeneralI have a problem... Pin
    Joel Holdsworth20-Sep-01 10:36
    Joel Holdsworth20-Sep-01 10:36 
    GeneralRe: I have a problem... Pin
    Gert Boddaert22-Sep-01 3:50
    Gert Boddaert22-Sep-01 3:50 
    A couple of solutions are possible:

    - virtualize the base classes to the maximum, all the plug-ins and the main program have to agree on is the way they talk to each other through the function interfaces.
    - Create a common DLL (load-time linked) which is used by the plug-ins and the main program.

    cheers,

    Gert.

    --------------------------------------------------
    If my messages appear curt, I apologize.
    I try to be brief to save your time as well as mine.
    --------------------------------------------------

    GeneralThread synchronisation of Plug-in objects Pin
    Fred Olusina24-May-01 6:27
    Fred Olusina24-May-01 6:27 
    GeneralRe: Thread synchronisation of Plug-in objects Pin
    Gert Boddaert28-May-01 1:19
    Gert Boddaert28-May-01 1:19 
    QuestionHow to implement multiple functions Pin
    4-Jan-01 3:22
    suss4-Jan-01 3:22 
    GeneralAlready done. Pin
    Member 11825589-Aug-00 19:28
    Member 11825589-Aug-00 19:28 
    GeneralPoint worth mentioning here Pin
    Phil Kovacs21-Jun-00 9:42
    sussPhil Kovacs21-Jun-00 9:42 
    GeneralRe: Point taken but already mentioned Pin
    Gert Boddaert21-Jun-00 21:14
    Gert Boddaert21-Jun-00 21:14 
    GeneralRe: Point worth mentioning here Pin
    Daniel Lohmann5-Jan-01 5:33
    Daniel Lohmann5-Jan-01 5:33 
    Generaldll driving an atl control Pin
    SoftZ21-Jun-00 7:01
    SoftZ21-Jun-00 7:01 
    GeneralRe: dll driving an atl control Pin
    Gert Boddaert21-Jun-00 21:21
    Gert Boddaert21-Jun-00 21:21 
    QuestionOne DLL - Many functions? Pin
    Kevin Stratton2-Jun-00 5:18
    Kevin Stratton2-Jun-00 5:18 
    AnswerRe: One DLL - Many functions? Pin
    Kevin Stratton2-Jun-00 8:57
    Kevin Stratton2-Jun-00 8:57 
    GeneralSerialize Plugin Objects Pin
    Gert Boddaert5-Jun-00 4:34
    Gert Boddaert5-Jun-00 4:34 
    AnswerRe: One DLL - Many functions? Pin
    4-Jan-01 3:16
    suss4-Jan-01 3:16 
    QuestionMultiple Implementation of same Plugin-Feature ? Pin
    daniel schmid28-May-00 23:18
    daniel schmid28-May-00 23:18 
    AnswerRe: Multiple Implementation of same Plugin-Feature ? Pin
    Gert Boddaert29-May-00 21:59
    Gert Boddaert29-May-00 21:59 
    GeneralPlug in With MFC Pin
    Paul Bedford & John Davis4-May-00 9:06
    sussPaul Bedford & John Davis4-May-00 9:06 
    GeneralRe: Plug in With MFC Pin
    Gert Boddaert4-May-00 22:47
    Gert Boddaert4-May-00 22:47 

    General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

    Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.