Click here to Skip to main content
Click here to Skip to main content

Understanding Custom Marshaling Part 1

, 18 Aug 2006
Rate this:
Please Sign up or sign in to vote.
Learn the fundamental principles of COM custom marshaling by code examples.

Introduction

COM Marshaling is a subject most developers can and do take for granted. Indeed the Microsoft COM engineers have done such a marvelous job of concealing the internals of marshaling that we are left totally unaware of the precision that take place with each cross-apartment method call.

This article is part one of a multi-part series of articles in which I plan to expound on the various methods of achieving COM Custom Marshaling. In this first installment, I will touch on the concept of a proxy and that of a marshal data packet. To this end, I have prepared basic sample implementation codes which we will walk through thoroughly. It is my hope that as we study this subject in-depth, we will come to realize and appreciate how marshaling works.

This series of articles is advanced in nature and I do expect the reader to possess prior knowledge on the following topics :

  1. COM development in general.
  2. Apartments (MTA and STA in particular).
  3. Interface marshaling in general (including the concepts of proxies and stubs).

Armed with the above-mentioned knowledge, we will now proceed to explore the world of custom marshaling. Let us start by going through some background on COM marshaling in general.

A Primer On COM Marshaling

Custom Marshaling.

The term custom marshaling is a misnomer. It is in fact a generic architecture that stipulates the rules and requirements of marshaling in general. What we know of as standard marshaling is actually a specific instance of custom marshaling . Any other forms of marshaling will be treated as various manifestations of custom marshaling. Each of these manifestations is transparent to the COM sub-system. For convenience of discussion, we will refer to standard marshaling as just that even though we know that it is Microsoft's specific implementation of custom marshaling. We will also regard all private, non-Microsoft marshaling as custom marshaling.

Marshaling is indelibly associated with apartments. We know that an apartment is a logical container inside an application for COM objects which share the same thread access rules (i.e. regulations governing how the methods and properties of an object are invoked from threads within and without the apartment in which the object belongs). We also know that all COM objects live inside exactly one apartment.

In order that an object be used in an apartment other than its own, the object's interfaces must first be exported from its original apartment and then imported into the target (client) apartment. The resultant interface pointer which is used by the client apartment, however, will not be the original object itself. It is instead something known as a proxy. When COM performs the exporting and importing of interface pointers, it does so using a conjugate pair of procedures known as marshaling and unmarshaling. For the sake of convenience, we will refer to all types of client apartments (as mentioned above) as simply client apartments.

When a method of the imported interface is invoked, the proxy must somehow pass control to the original object (in its original apartment), get it to execute the method and then return control back to the proxy. In this way, it is ensured that a method is always executed in the correct apartment. When control is passed to the original object, the proxy's thread must halt until control is returned to the proxy.

This passing of control from one apartment to another is also known as method remoting. Method remoting is how all cross-thread, cross-process and cross-machine communications occur in COM.

Custom marshaling is essentially the generic mechanism that lets a COM object control the way it communicates with its proxy in a client apartment. The client apartment can either be in the same process as the COM object or in another process or even in a process in another machine. When we implement custom marshaling, what we eventually wind up doing is to implement a custom proxy.

We mentioned earlier that when a method of an imported interface is invoked, the proxy must somehow pass control to the original object. Under custom marshaling, the protocol for this is totally within the domain of the object and its proxy. COM plays no part in the communication process. However, when a proxy is created, COM gives the object a one time chance to pass something to the proxy. This is done to help the object establish its protocol with its proxy. This something is known as a marshal data packet. We shall explore these in greater detail in the coming sections below.

A term which will be covered in a future article but is worth mentioning at this time is that of a stub. Stubs are not always required although they can serve to simplify object-to-proxy communications. As will be made clear later on, stubs will not be relevant in the sample codes that we present in this article.

The IMarshal interface.

To support custom marshaling, a COM object must implement the IMarshal interface. If it does not, COM deems that standard marshaling is to be used whenever this object's interfaces are to be marshaled. Hence the first point about creating custom marshaling is to ensure that your COM object implements the IMarshal interface.

The second point about creating custom marshaling is that a proxy object must be defined. This proxy is also a COM object and it must also implement the IMarshal interface albeit not all of the IMarshal methods need to be non-trivial.

Note also that once an object has declared that it will implement custom marshaling, it must custom marshal all of its interfaces. The IMarshal interface consists of 6 methods. We will study these more in-depth when we explore our sample codes. For now, a short examination of how COM interacts with the IMarshal methods would be appropriate as much of what will be expounded on very soon afterwards will depend on a good understanding of this interface.

When called upon to perform marshaling, COM first queries the object for its IMarshal interface. Once COM discovers that it supports IMarshal, it will call its IMarshal::GetUnmarshalClass() method. This method returns to COM the CLSID of the object that will be used for the unmarshaling process. In other words, COM is asking for the CLSID of the proxy that will be created when unmarshaling takes place. Having a CLSID implies that this proxy is a COM object.

Later, when unmarshaling is to occur, COM will use this CLSID to create the proxy object within the importing client apartment. Sensibilities dictate that such a COM object should be housed within a DLL and that it should support both the Single-Threaded and Multi-Threaded Apartments. These requirements will ensure that there will be no need for a proxy to the proxy that has just been created !

Still within the context of marshaling, COM will next call the IMarshal::GetMarshalSizeMax() method to determine the size of its marshal data packet. We will start to go deep into marshal data packets in the next section. The IMarshal::MarshalInterface() method of the object will then be invoked. From this method, COM expects to obtain the marshal data packet from the object. This marshal data packet can then either be passed to client code (for later unmarshaling) or be kept by COM itself inside a table in memory (to be retrieved multiple-times later by client code for unmarshaling).

Now, when unmarshaling takes place, a proxy COM object must first be created. Then the marshal data packet must be passed to it via its IMarshal::UnmarshalInterface() method. This is how a proxy becomes initialized. Being initialized could mean that a proxy now has what it takes to communicate with its original object, or that a proxy constructs itself as a clone of the original object. Whichever is the case, once a proxy has been initialized, COM will no longer play any part in object-to-proxy communications.

The Marshal Data Packet

From a high-level point of view, marshaling and unmarshaling is the collective act of transforming an interface pointer from a source apartment into a series of bytes and then transporting this series of bytes to a client apartment which will reverse-transform the bytes back into an interface pointer usable by the client apartment.

The transformation of anything into a series of bytes is known as serialization. The series of bytes obtained from serialization is more commonly referred to as a stream. The serialization of an interface pointer is better known as marshaling the interface pointer. The stream obtained from marshaling is also known as a marshal data packet. The marshal data packet stream object is always referenced by a pointer to an IStream interface.

The format of a marshal data packet used for custom marshaling is as follows :

CustomMarshaledObjectReference

The above diagram is a slightly modified version of the same taken from Don Box's book Essential COM (page 245).

The first part of the marshal data packet is a 4-byte signature which is hardcoded to the characters "MEOW" which is an acronym for Microsoft Extended Object Wire. This is followed by a single byte flags field. I personally do not know the purpose of this field. Next comes a 16-byte GUID field which is filled with the IID of the interface which is being marshaled. After that comes another 16-byte GUID field which is filled with the CLSID of the custom proxy which will be created in the client apartment.

We then have a 4 byte data the purpose of which is also unknown to me at this time. I resorted to calling this 4-byte field "Reserved" rather than speculate its actual intended purpose. The next field "Byte Count" is important as it indicates the length (in bytes) of the custom data which will follow.

This last section of the marshal data packet (i.e. "Custom Marshal Data") is particularly important. It is created by the original COM object and will be passed to a proxy during the proxy's initialization. To COM, this custom marshal data is essentially an opaque array of bytes that can contain anything as long as a proxy knows how to use it to establish communication with the original COM object or to re-create the object within the context of the client apartment. This array of bytes is the something that gets passed between an object and its proxy. This is the essence of custom marshaling.

We will revisit this structure again later when we go through our sample codes.

Note that unmarshaling may not always occur after an object has been marshaled. There is no rule stating this necessity. However because a marshal data packet potentially represents an object, its presence may warrant the need to add a reference count to the original object. This has to do with something known as strong and weak connections. We shall discuss this in a later article.

Basic Example

In this section, we present a simple example of marshaling known as "Marshal-by-Value". It is a perfect illustratory example for custom marshaling. The premise behind "Marshal-by-Value" is to create a proxy that is a clone of the original object. This concept applies to immutable objects (i.e. objects whose properties which will not change once they have been initialized).

Because immutable objects will never change the values of their properties, it makes no difference whether a proxy is a reference to the original object or is an exact copy of the object itself. This being the case, when a proxy to the object is required by a client apartment, we may as well clone the object and deliver it to the apartment.

And because a "Marshal-by-Value" proxy need not communicate with its original object, this example is simple enough for me to illustrate in-depth the proxy creation process. Proxy creation is a very important first step in understanding custom marshaling. Furthermore, since method calls will be made directly from the client code to the proxy, there will be no need for any marshaling of method arguments. This will be covered in a future article of this series.

The complete source codes for the basic example implementation are included in the zip file. Once unzipped, it will be stored in the following folder :

<main folder>\BasicSample01

where <main folder> is wherever you have copied the zip file to.

Inside BasicSample01, there are 3 additional folders : Interfaces, Implementations and Clients. Let us now examine the projects contained within these folders.

The BasicSample01Interfaces Solution

Inside the Interfaces folder, we have a Visual Studio .NET project folder named BasicSample01Interfaces in which the Visual Studio .NET solution project BasicSample01Interfaces.sln is found.

This BasicSample01Interfaces.sln project will not contain any useful implementation code. Its purpose is simply to allow us to automate (via ATL Wizards) the creation and maintenance of two COM interfaces named IImmutable and IImmutableObjectFactory.

Listed below is a fragment of code taken from BasicSample01Interfaces.idl showing the IImmutable and the IImmutableObjectFactory interfaces :

[ 
 object, 
 uuid(BF0DC81A-46FB-4300-88E5-2B8EEB2CEEA1), 
 dual, 
 nonextensible, 
 helpstring("IImmutable Interface"), 
 pointer_default(unique) 
] 
interface IImmutable : IDispatch  
{  
 [propget, id(1), helpstring("property LongValue")] 
          HRESULT LongValue([out, retval] LONG* pVal); 
}; 
 
[ 
 object, 
 uuid(CCAB57EA-A497-44C0-B0A4-E781B7F47AA9), 
 dual, 
 nonextensible, 
 helpstring("IImmutableObjectFactory Interface"), 
 pointer_default(unique) 
] 
interface IImmutableObjectFactory : IDispatch 
{ 
 [id(1), helpstring("method CreateObject")] 
          HRESULT CreateObject 
          ( 
            [in] LONG InitializationValue, 
            [out,retval] IImmutable** ppIImmutableObjectReceiver 
          ); 
}; 

Note that the IImmutable interface definition stipulates the basic attributes of this interface in terms of its GUID, methods, argument types, etc.

Customizable attributes like the apartment type, packaging format (in-proc DLL or EXE server) and whether standard or custom marshaling will be used, are the decision of the eventual implementation code.

An object which implements the IImmutable interface must possess a readonly long property. This readonly property must also never change once set internally.

The IImmutableObjectFactory interface describes an object which serves as a factory of IImmutable objects. An IImmutableObjectFactory object must implement the CreateObject() method which serves as an entry point for the creation and initialization of an IImmutable object. This method takes a long parameter which is used as the value to be set for the long property of the IImmutable object to be created and returned.

The BasicSample01Interfaces.sln solution will specifically help to maintain the BasicSample01Interfaces.idl file, which will be referenced by the BasicSample01InterfacesImpl.sln project (will be described in the next sub-section), and the BasicSample01Interfaces.h header file, which will be referenced by the various client codes included in the source codes zip file.

The BasicSample01InterfacesImpl Solution

Next, we examine the BasicSample01InterfacesImpl.sln project the folder of which can be found in the Implementations folder.

The BasicSample01InterfacesImpl.sln project contains concrete classes which implement the IImmutable and IImmutableObjectFactory interfaces. These are the CImmutableObjectFactoryImpl and the CImmutableImpl C++ classes respectively.

Let us now examine these classes in greater detail.

The CImmutableObjectFactoryImpl class

The CImmutableObjectFactoryImpl class is implemented using ATL. It is an STA class but this fact is not important as we will not be performing any custom marshaling for this interface. If CImmutableObjectFactoryImpl should be instantiated inside an STA thread, no marshaling will be immediately required in order that it be usable by the current apartment.However, if it is instantiated inside an MTA thread, then marshaling will be immediately required but we will be relying on standard marshaling for this.

Besides these immediate needs, any other cross apartment marshaling for CImmutableObjectFactoryImpl's IImmutableObjectFactory interface will also use standard marshaling.

Listed below is the implementation code for the only method of the IImmutableObjectFactory interfce :

STDMETHOD(CreateObject) 
( 
  long InitializationValue, 
  IImmutable ** ppIImmutableObjectReceiver 
) 
{ 
  CComObject<CImmutableImpl>* pCImmutableImpl = NULL; 
      
  CComObject<CImmutableImpl>::CreateInstance(&pCImmutableImpl); 
       
  if (pCImmutableImpl) 
  { 
    CImmutableImpl* pBase 
      = dynamic_cast<CImmutableImpl*>(pCImmutableImpl); 
       
    if (pBase) 
    { 
      pBase -> SetLongValue(InitializationValue); 
    } 
       
    pCImmutableImpl -> QueryInterface 
    ( 
      IID_IImmutable, 
      (void**)ppIImmutableObjectReceiver 
    ); 
  } 
      
  return S_OK; 
} 

The CreateObject() method simply creates an instance of CImmutableImpl and then sets its long property value by calling its non-COM public function SetLongValue(). The long value used here is taken from CreateObject()'s first parameter.

CreateObject() then returns a pointer to the new CImmutableImpl instance's IImmutable interface via QueryInterface().

The CImmutableImpl class

The CImmutableImpl class is also implemented using ATL. Like CImmutableObjectFactoryImpl, it is designated as an STA object. This means that if it is instantiated inside an STA thread, marshaling will not be immediately required. However, if it is instantiated inside an MTA thread, marshaling will be immediately required. This would complicate matters a little. We will be demonstrating something similar to this in a more advanced sample in a future article. The current basic sample client code will not be touching on this. The cross-apartment marshaling we will be demonstrating in our basic sample will be across two STA apartments.

CImmutableImpl supports the IImmutable interface by defining a long member data m_lLongValue. This long value is set via the non-COM public function named SetLongValue(). This method is only callable from code within the BasicSample01InterfacesImpl.dll server itself and only CImmutableObjectFactoryImpl uses it to initialize a CImmutableImpl object's long value.

The heart and soul of CImmutableImpl, however, is in its implementation of the IMarshal interface. CImmutableImpl shows its support for the IMarshal interface in its class definition :

class ATL_NO_VTABLE CImmutableImpl : 
 public CComObjectRootEx<CComSingleThreadModel>,
 public CComCoClass<CImmutableImpl, &CLSID_ImmutableImpl>,
        ...
 public IMarshal
{
   ...
   ...
   ...
}

The IMarshal interface requires implementors to supply six methods. We shall examine CImmutableImpl's implementations of these functions below.

GetUnmarshalClass()

HRESULT GetUnmarshalClass( 
  REFIID riid, 
  void * pv, 
  DWORD dwDestContext, 
  void * pvDestContext, 
  DWORD mshlflags, 
  CLSID * pCid 
); 

The GetUnmarshalClass() method is fully documented in MSDN. It is the first IMarshal method that COM will invoke after determining that an object will be performing its own custom marshaling (through the discovery of the object's IMarshal interface). COM calls this method to determine the CLSID of the marshaling proxy object. It is the proxy that will perform the unmarshaling of the original object in the importing client apartment.

In our basic sample, because the proxy is an exact copy of the object itself, i.e., the CImmutableImpl object is its own proxy. Hence the following code can be found in CImmutableImpl::GetUnmarshalClass():

STDMETHOD(GetUnmarshalClass) 
( 
  REFIID riid, 
  void * pv, 
  DWORD dwDestContext, 
  void * pvDestContext, 
  DWORD mshlFlags, 
  CLSID* pCid 
) 
{ 
  // The class that will perform the unmarshaling will be 
  // this class itslf. 
  *pCid = GetObjectCLSID(); 
 
  return S_OK; 
} 

We simply return the CLSID of the CImmutableImpl class itself. This tells COM that once CImmutableImpl has performed the necessary marshaling process and has acquired a marshal data packet stream (these are accomplished via the IMarshal::MarshalInterface() method), COM should then create, in the target client apartment, another instance of the CImmutableImpl class which will serve as the proxy to the original object.

The following diagram illustrates what happens when GetUnmarshalClass() is called :

GetUnmarshalClass

GetMarshalSizeMax()

HRESULT GetMarshalSizeMax( 
  REFIID riid, 
  void * pv, 
  DWORD dwDestContext, 
  void * pvDestContext, 
  DWORD mshlflags, 
  ULONG * pSize 
); 

GetMarshalSizeMax() is the second IMarshal method that COM will invoke. The purpose of this method is to determine the size of the marshal data packet to be created for marshaling the IImmutable interface. This method works in pair with the MarshalInterface() method as we shall see later. Once COM has determined this size value, it will proceed to prepare the stream object (a container for bytes) which will be used to store the marshal data packet.

CImmutableImpl's implementation of GetMarshalSizeMax() is listed below :

STDMETHOD(GetMarshalSizeMax) 
( 
  REFIID riid,  
  void * pv, 
  DWORD dwDestContext, 
  void * pvDestContext, 
  DWORD mshlFlags, 
  ULONG* pSize 
) 
{ 
  HRESULT hr = S_OK; 
 
  *pSize = sizeof(CImmutableMarshaledObjectReferenceStruct); 
 
  return hr; 
}

Here we see that the size of the CImmutableMarshaledObjectReferenceStruct structue will be returned. This is because we will be storing a CImmutableMarshaledObjectReferenceStruct structure into the marshal data packet.

The CImmutableMarshaledObjectReferenceStruct structure is listed below :

typedef struct 
  CImmutableMarshaledObjectReferenceStructTag
{
  long lLongValue;
}
CImmutableMarshaledObjectReferenceStruct;

As mentioned earlier, to COM, the customized data contained inside the marshal data packet is essentially an opaque array of bytes that can contain anything as long as a proxy knows how to use it to establish communication with the original COM object or to re-create the object within the context of the client apartment. For our CImmutableImpl class, this customized array of bytes will contain the CImmutableMarshaledObjectReferenceStruct structue. As we shall see later on, this structure contains all the necessary information for our proxy to re-create the original object in the client apartment.

The following diagram illustrates what happens when GetMarshalSizeMax() is called :

GetMarshalSizeMax

MarshalInterface()

HRESULT MarshalInterface( 
  IStream * pStm, 
  REFIID riid, 
  void * pv, 
  DWORD dwDestContext, 
  void * pvDestContext, 
  DWORD mshlflags 
); 

The MarshalInterface() method is the next method to be called. This is a very important method as it is here that the marshal data packet will be created and end up being serialized and marshaled across to a target apartment. Whatever we place into the marshling data packet should contain information that will help the proxy to uniquely identify and establish connection with the original CImmutableImpl object (in order to invoke its properties and methods) or to re-create it in the context of the client apartment.

CImmutableImpl's MarshalInterface() method is listed below :

STDMETHOD(MarshalInterface)  
( 
  IStream* pStm, 
  REFIID riid, 
  void *pv, 
  DWORD dwDestContext, 
  void * pvDestContext,  
  DWORD mshlFlags 
) 
{       
  if (dwDestContext != MSHCTX_INPROC) 
  { 
    return E_FAIL; 
  } 
     
  CImmutableMarshaledObjectReferenceStruct 
    ImmutableMarshaledObjectReferenceStruct; 
     
  ImmutableMarshaledObjectReferenceStruct.lValue = m_lValue; 
     
  *pStm << ImmutableMarshaledObjectReferenceStruct; 
     
  return S_OK; 
} 

We stipulate that the marshaling context must be MSHCTX_INPROC (ie. the unmarshaling will be done in another apartment in the same process). The other marshaling contexts are not supported in our basic example.

We essentially create a CImmutableMarshaledObjectReferenceStruct structure and fill its only member data lValue with the current value in m_lValue. Once this is done, we will write the contents of our structure into the stream. We use a global stream support function (listed in marshalhelpers.h) to perform this.

The CImmutableMarshaledObjectReferenceStruct structure is used to store the current value of the m_lValue member data (which is returned when the IImmutable interface's LongValue property is requested by a client). This structure will be contained as raw bytes in the marshal data packet and will be passed to the IMarshal::UnmarshalInterface() method of the proxy which will reverse-transform these raw bytes into a CImmutableMarshaledObjectReferenceStruct structure. More on this later.

The following diagram illustrates what happens when MarshalInterface() is called :

MarshalInterface

UnmarshalInterface()

HRESULT UnmarshalInterface( 
  IStream * pStm, 
  REFIID riid, 
  void ** ppv 
); 

The UnmarshalInterface() method is to be implemented by a proxy. It is meant to be the congugal opposite of the MarshalInterface() method which is meant to be implemented by the original object. However, CImmutableImpl provides an implementation because it is its own proxy class. What happens before UnmarshalInterface() is invoked is that a new instance of CImmutableImpl (playing the role of a proxy) will be created inside the client apartment. Thereafter, this new instance's UnmarshalInterface() method will be called with an IStream pointer of the stream object which contains the marshal data packet created previously during the MarshalInterface() call.

What is expected from the UnmarshalInterface() method call is to return an interface pointer (whose IID is specified by the riid parameter) from the unmarshaled object (i.e. the proxy).

Let us examine CImmutableImpl's UnmarshalInterface() method :

STDMETHOD(UnmarshalInterface) 
( 
  IStream* pStm, 
  REFIID riid, 
  void ** ppv 
) 
{ 
  CImmutableMarshaledObjectReferenceStruct 
    ImmutableMarshaledObjectReferenceStruct; 
  
  *pStm >> 
    (CImmutableMarshaledObjectReferenceStruct&) 
      ImmutableMarshaledObjectReferenceStruct; 
 
  m_lValue = ImmutableMarshaledObjectReferenceStruct.lValue; 
  
  return QueryInterface(riid, ppv); 
} 

We create a new instance of the CImmutableMarshaledObjectReferenceStruct struct which is used to download all the contents of the input stream. Thereafter, we reconstruct CImmutableImpl by retrieving the lValue member of the structure and filling m_lValue with this value. This is exactly how the proxy becomes a "Marshal-By-Value" proxy. The original object is effectively re-created. And in our simple example, this completes the reconstruction process.

We then perform a QueryInterface() to retrieve the required interface pointer. The interface required is none other than that of IImmutable (i.e., IID_IImmtable).

The following diagram illustrates what happens when UnmarshalInterface() is called :

UnmarshalInterface

ReleaseMarshalData()

HRESULT ReleaseMarshalData( 
  IStream * pStm 
); 

This method is COM's way of indicating to an object (which has been marshaled) that a marshal object packet is being destroyed. The ocurrence of this is a very important event. However it is not significant in our basic example, as will be the case for most "Marshal-by-value" proxies. It is related to two concepts known as table marshaling and strong connections. In general only strong table-marshaled objects need to provide a non-trivial implementation of ReleaseMarshalData().This topic will be discussed again in a more advanced article in the future.

CImmutableImpl's version of ReleaseMarshalData() is listed below :

STDMETHOD (ReleaseMarshalData)(IStream *pStm) 
{ 
  return S_OK; 
} 

It is a trivial implementation indeed.

DisconnectObject()

HRESULT DisconnectObject( 
  DWORD dwReserved 
); 

This method applies only to objects which are housed within EXE COM servers. The typical situation in which this method is called on an object is when the EXE COM server is forcibly terminated while it still has at least one running object. In this situation, prior to shutting down, the EXE server must invoke the CoDisconnectObject() API on each of these running objects. For each object that implements IMarshal, CoDisconnectObject() will call the IMarshal::DisconnectObject() method so that each object that manages its own marshaling can take steps to notify its proxy that it is about to shut down.

After this is done, clients which hold proxies may continue to call the proxy's methods but the proxy must then return the HRESULT CO_E_OBJECTNOTCONNECTED.

This method is equally irrelevant to "Marshal-by-Value" proxies because a proxy is a clone of the original object and will not need to communicate with it. The above-described scenario do not apply for our basic example. Hence CImmutableImpl's version of DisconnectObjectI() is also trivial :

STDMETHOD (DisconnectObject)(DWORD dwReserved) 
{ 
  return S_OK; 
}  

The Client Solutions

I have prepared 3 separate client projects each of which illustrate a slightly different way to marshal and unmarshal our immutable object across apartments. These solutions are contained within the Clients folder. Let us examine these projects individually.

The VCConsoleClient01 Solution

The VCConsoleClient01.sln project provides a good basic example of demonstrating cross-apartment marshaling and unmarshaling with the lowest-level techniques available : i.e. via the CoMarshalInterface() and CoUnmarshalInterface() APIs and the creation of a stream object via CreateStreamOnHGlobal().

One of the highlights of the VCConsoleClient01 solution is our examination of the contents of the marshal data packet that will be created after we call the CoMarshalInterface() API.

VCConsoleClient01.sln is a console-based application. The following is a synopsis of this application :

  1. A thread is started in which an instance of the IImmutableObjectFactory object is created and from which an instance of the IImmutable object is gotten.
  2. The IImmutable object is then marshaled and a marshal data packet will be created for it.
  3. The marshal data packet is then transported across to another apartment where it is examined and the IImmutable interface contained inside is imported.
  4. After importing the IImmutable interface, we will use the proxy to call a method of the IImm<CODE>utable interface.

The source codes of the VCConsoleClient01 solution is simple enough for us to break it down into its constituent components and have them analyzed individually. These are summarized below :

The _tmain() function

The _tmain() function is the entry point of the application. It is listed below :

int _tmain(int argc, _TCHAR* argv[])
{
    ::CoInitialize(NULL);
    
    g_hInterfaceMarshaled = CreateEvent(NULL, TRUE, FALSE, NULL);
    
    Demonstrate_Cross_Apartment_Call_Via_Stream();
    
    if (g_hInterfaceMarshaled)
    {
      CloseHandle(g_hInterfaceMarshaled);
      g_hInterfaceMarshaled = NULL;
    }
 
    ::CoUninitialize();
    
  return
0;
}

Nothing sophisticated about the _tmain function. It essentially initializes the thread that it runs on to be an STA thread. It then creates an event handle g_hInterfaceMarshaled which will be signalled when the IImmutable object has been marshaled. _tmain than calls the Demonstrate_Cross_Apartment_Call_Via_Stream function. This function is where the real action starts.

The Demonstrate_Cross_Apartment_Call_Via_Stream() Function

The Demonstrate_Cross_Apartment_Call_Via_Stream() function is where the good stuff begins. This function works hand-in-hand with another named ThreadFunc_CustomMarshaledObject() to provide a complete demonstration of exporting and importing an interface pointer from one apartment to another.

Due to its length, we will not be printing a complete listing in this section. Instead we will provide a general synopsis of this function. Specific parts of this function will be examined closely in a later section when we discuss how it works together with ThreadFunc_CustomMarshaledObject().

Here below is a general synopsis :

  1. It creates a thread headed by the entry point function ThreadFunc_CustomMarshaledObject().
  2. ThreadFunc_CustomMarshaledObject() will create an IImmutable object (through a IImmutableObjectFactory object) and initialize its long value to some number. After creating the IImmutable object, it will create a marshal data packet for it and pass it (via an IStream interface pointer) to Demonstrate_Cross_Apartment_Call_Via_Stream().
  3. After receiving the IStream interface pointer to the marshal data packet, Demonstrate_Cross_Apartment_Call_Via_Stream() will cast it to a MarshalDataPacket structure for examination purposes.
  4. Next, the marshal data packet is unmarshaled and we will then obtain a proxy to the original IImmutable object. However, because the proxy is actually a "Marshal-By-Value" proxy, what we will obtain is actually a clone of the original IImmutable object.
  5. We will use the proxy to obtain the long value contained inside. We will note that the long value is indeed the one that was used to initialize the original IImmutable object.

The ThreadFunc_CustomMarshaledObject() Function

As mentioned previously, ThreadFunc_CustomMarshaledObject() works hand-in-hand with Demonstrate_Cross_Apartment_Call_Via_Stream().

Here below is a general synopsis of this function :

  1. ThreadFunc_CustomMarshaledObject() is the entry point function of a thread.
  2. It is initialized to be an STA thread.
  3. An instance of the IImmutableObjectFactory object is created and from this, an instance of a IImmutable object is obtained.
  4. The IImmutable object is then marshaled. The resultant marshal data packet is then transported to the Demonstrate_Cross_Apartment_Call_Via_Stream() function.
  5. ThreadFunc_CustomMarshaledObject() then enters a message loop which will not break until a WM_QUIT message is posted to this thread.
  6. The fact that this thread remains alive (via the message pump) means that the IImutable object will also remain alive for perusal by other apartments.

How do the two functions work together ?

In this section, we examine how the above two functions work together to demonstrate marshaling/unmarshaling. We will also see the IMarshal methods of the IImmutable object in action while marshaling and unmarshaling take place. We will describe what goes on chronologically starting from the Demonstrate_Cross_Apartment_Call_Via_Stream() function.

  1. Demonstrate_Cross_Apartment_Call_Via_Stream() starts by creating a new thread headed by the entry point function ThreadFunc_CustomMarshaledObject().

  2. It also defines an uninitialized pointer to an IStream interface named lpStream:

    // lpStream is an uninitialized pointer to a stream.
    LPSTREAM lpStream = NULL;
    
  3. We will pass the address of lpStream to the CreateThread() API as the parameter to the ThreadFunc_CustomMarshaledObject() entry point function :

      // We create a thread which is started by the function
      // ThreadFunc_CustomMarshaledObject(). We also pass the
      // uninitialized IStream pointer to this thread function
      // as a parameter.
      hThread = CreateThread
      (
        (LPSECURITY_ATTRIBUTES)NULL,
        (SIZE_T)0,
        (LPTHREAD_START_ROUTINE)ThreadFunc_CustomMarshaledObject,
        (LPVOID)(&lpStream),
        (DWORD)0,
        (LPDWORD)&dwThreadId
      );  
    

    This is done so that it will be initialized by ThreadFunc_CustomMarshaledObject() to point to the marshal data packet for the IImmutable object which will be created in that thread.

  4. After creating the thread, Demonstrate_Cross_Apartment_Call_Via_Stream() will wait for the g_hInterfaceMarshaled event handle to be set :

      // We wait for the thread to initialize.
      ThreadMsgWaitForSingleObject(g_hInterfaceMarshaled, INFINITE);
     
      ResetEvent(g_hInterfaceMarshaled);
    
  5. This event handle was first created inside _tmain(). It will be set inside ThreadFunc_CustomMarshaledObject() when the IImmutable object has been marshaled :

        // Signal the situation that the immutable object
        // has been marshaled by setting the g_hInterfaceMarshaled
        // event.
        if (g_hInterfaceMarshaled)
        {
          SetEvent(g_hInterfaceMarshaled);
        }
    

    When this event handle is set, it is a signal to the Demonstrate_Cross_Apartment_Call_Via_Stream() function that its lpStream now holds a marshal data packet and is ready for use.

  6. On the ThreadFunc_CustomMarshaledObject() side, the pointer to lpStream is represented by the local variable named ppStreamReceiver. It will be initialized to the value of the LPVOID parameter passed from Demonstrate_Cross_Apartment_Call_Via_Stream() which is actually the address of the IStream pointer lpStream:

    DWORD WINAPI ThreadFunc_CustomMarshaledObject(LPVOID lpvParameter)
    {
      MSG  msg;
      long  lLong = 0;
      LPSTREAM* ppStreamReceiver = (LPSTREAM*)lpvParameter;
      ...
      ...
      ...
    
  7. ThreadFunc_CustomMarshaledObject() will then proceed to create an IImmutableObjectFactory object and thereafter, to obtain an IImmutable object from it :

      // Create an instance of the COM object which has ProgID 
      // "BasicSample01InterfacesImpl.ImmutableObjectFactoryImpl" 
      // and manage it
      // via its IImmutableObjectFactory interface.
      // Now, because the resultant object (spIImmutableObjectFactory) 
      // is an STA object, and this thread is an STA thread, 
      // spIImmutableObjectFactory will live in the same STA 
      // as this thread.
      _CoCreateInstance
      (
        "BasicSample01InterfacesImpl.ImmutableObjectFactoryImpl", 
        spIImmutableObjectFactory
      );
        
      // Get the immutable object from spIImmutableObjectFactory.
      // Now because the immutable object is also an STA object, 
      // it will live in the same STA as this thread.
      spIImmutableObjectFactory -> CreateObject
      (
        101, 
        (IImmutable**)&spIImmutable
      );
    

    Note that IImmutable object's long property will be initialized to a value of 101.

  8. ThreadFunc_CustomMarshaledObject() will then create a stream object using the CreateStreamOnHGlobal() API :

        ::CreateStreamOnHGlobal
        (
          0,
          TRUE,
          ppStreamReceiver
        );
    

    The CreateStreamOnHGlobal() API is used to create a Stream Object which will reside in global memory. We set the first parameter to zero to indicate that we want CreateStreamOnHGlobal() to internally allocate a new shared memory block of size zero. The second parameter is set to TRUE so that when the returned stream object is Release()'d, the global memory will also be freed.

    The newly allocated stream object will be returned as an IStream pointer. CreateStreamOnHGlobal() expects a pointer to a LPSTREAM in its third parameter. For this, we pass ppStreamReceiver. This way, lpStream (from Demonstrate_Cross_Apartment_Call_Via_Stream()) is set to store the resultant pointer to the IStream interface of the newly created stream object.

  9. The CoMarshalInterface() API is then used to create a marshal data packet for the IImmutable object :

          ::CoMarshalInterface
          (
            *ppStreamReceiver,
            __uuidof(IImmutablePtr),
            (IUnknown*)pIUnknown,
            MSHCTX_INPROC,
            NULL,
            MSHLFLAGS_NORMAL
          );
    

    For the first parameter, CoMarshalInterface() expects a pointer to a stream object to be used during marshaling. For this, we pass the contents of ppStreamReceiver. This way, the lpStream (from Demonstrate_Cross_Apartment_Call_Via_Stream()) is used to store the marshal data packet of the IImmutable object. The use of the MSHCTX_INPROC as the destination context (parameter 4) indicates that the unmarshaling of the data in the stream will be done in another apartment of the same process.

    The use of the MSHLFLAGS_NORMAL flag (parameter 6) indicates that the marshal data packet can be unmarshaled just once, or not at all.

    During the course of running CoMarshalInterface(), COM will attempt to get the CLSID of the proxy for the IImmutable object and then to get the marshal data packet from it.

    The following sequence of function calls on the side of the IImmutable object will take place :

    CImmutableImpl::GetUnmarshalClass()
    CImmutableImpl::GetMarshalSizeMax()
    CImmutableImpl::MarshalInterface()

    The reader may want to put a breakpoint in these functions for observation purposes.

  10. When CImmutableImpl::GetUnmarshalClass() is called, we will indicate to CoMarshalInterface the CLSID of the object that will take charge of unmarshaling the interface pointers of the IImmutable object. This will be the CLSID of the CImmutableImpl object itself. Note that the parameters to GetUnmarshalClass will reflect the parameter values of CoMarshalInterface.

  11. When CImmutableImpl::GetMarshalSizeMax() is called, we will pass the size of the CImmutableMarshaledObjectReferenceStruct structure.

  12. When CImmutableImpl::MarshalInterface() is called, notice that we create a CImmutableMarshaledObjectReferenceStruct structure and then serialize it into the stream object whose IStream interface is passed as first parameter.

  13. After CoMarshalInterface() completes, we must reset the stream to the beginning. Otherwise a later call to the CoUnmarshalInterface() API using this same stream object will fail with error STG_E_READFAULT :

          // We will need to reset the stream to the beginning.
          // Otherwise a later call to the CoUnmarshalInterface() 
          // API will fail with error STG_E_READFAULT.
         (*ppStreamReceiver) -> Seek(li, STREAM_SEEK_SET, NULL);
    
  14. Demonstrate_Cross_Apartment_Call_Via_Stream() will then pass lpStream to the ExamineStream() function for a closer look at what the marshal data packet contains. More on ExamineStream() later.

  15. Demonstrate_Cross_Apartment_Call_Via_Stream() will then pass lpStream to the CoUnmarshalInterface() API to construct a proxy to the IImmutable interface from the stream :

        ::CoUnmarshalInterface
        (
          lpStream,
          __uuidof(IImmutablePtr),
          (void**)&spIImmutable
        );
    
  16. Note that at this point in time, the Marshal Data Packet is contained inside the stream. All COM needs is a new proxy object to which it passes the Marshal Data Packet (in the UnmarshalInterface method call). When CoUnmarshalInterface is called with our marshal data packet stream, COM will create a proxy to the IImmutable object created inside the ThreadFunc_CustomMarshaledObject thread.

  17. During the course of execution of the CoUnmarshalInterface API, COM will pass the marshal data packet stream to the proxy in order for it to be initialized. The following sequence of function calls will take place :

    CImmutableImpl() (constructor)
    CImmutableImpl::FinalConstruct() (part of proxy construction)
    CImmutableImpl::UnmarshalInterface() (the marshal data packet will be passed here).

    The reader is encouraged to place breakpoints inside these functions to see the sequence of CImmutableImpl function calls.

  18. Notice that in CImmutableImpl::UnmarshalInterface(), a reverse of the activities of CImmutableImpl::MarshalInterface() is performed. A proxy which is effectively a clone of the original IImmutable object is constructed.

  19. After the proxy to the IImmutable object has been created and initialized, Demonstrate_Cross_Apartment_Call_Via_Stream() will proceed to call its get_LongValue() method :

      // At this time, spIImmutable has been initialized to be a proxy
      // of the immutable object.
      if (spIImmutable)
      {   
        spIImmutable -> get_LongValue(&lLongValue);        
      }
    

    The value 101 will be returned, proving that the proxy is indeed a clone of the original object.

  20. Thereafter Demonstrate_Cross_Apartment_Call_Via_Stream will terminate the ThreadFunc_CustomMarshaledObject thread by posting a WM_QUIT message to it :

    PostThreadMessage(dwThreadId, WM_QUIT, 0, 0);
  21. When ThreadFunc_CustomMarshaledObject receives the WM_QUIT message, it will exit its message loop and head towards the end of the function. CoUninitialize is called and the IImmutableObjectFactory and IImmutable smart pointer objects will be destroyed, thereby Release()'ing them.

The ExamineStream() Function

The ExamineStream function uses standard IStream manipulation APIs to cast the data contained inside a stream into a MarshalDataPacket structure. This structure is listed below :

struct MarshalDataPacket
{
  char   Signature[4];
  char   chFlags;
  IID   iidMarshaledInterface;
  CLSID   clsidCustomProxy;
  unsigned long ul;
  unsigned long ulSize;
  unsigned char ucCustomMarshalDataBytes[1];
};

Notice that it is a complete representation of the marshal data packet illustrated in the "The Marshal Data Packet" section. The MarshalDataPacket struct is defined to help us cast the contents of a marshal data packet to a structure which will make its analysis much easier.

The ExamineStream function will allocate in memory a buffer which is the size of the stream object pointed to by the lpStream parameter. After that, its contents are copied into a MarshalDataPacket structure instance.

The diagram below shows a QuickWatch on the field values of this structure :

MarshaledDataPacket

The format of the marshal data packet is revealed. We can see how the raw bytes of the customized part of the marshal data is used to contain the value of the lLongValue field (value 101) of the CImmutableMarshaledObjectReferenceStruct structure.

The Other Client Solutions

I have included two other client solutions which are contained inside the VCConsoleClient02 and VCConsoleClient03 folders. We do not have to go through their code as they are each sufficiently similar to VCConsoleClient01. My intention is to demonstrate other methods of marshaling and unmarshaling interfaces across apartments.

The following is a summary of their special features :

VCConsoleClient02

Instead of CoMarshalInterface() and CoUnmarshalInterface(), CoMarshalInterThreadInterfaceInStream() and CoGetInterfaceAndReleaseStream() are used. Please refer to MSDN for details on these two APIs.

These two APIs are actually warppers for the elaborate work that we did in VCConsoleClient01 of constructing a stream object and then using it to create a marshal data packet for marshaling and unmarshaling.

VCConsoleClient03

VCConsoleClient03 presents a radically different way of managing marshal data packets and proxies. It uses the Global Interface Table (GIT) to store the marshal data packet of an object. Whenever a proxy to the object is required, it is first created in the client apartment, then the marshal data packet from the GIT is retrieved and used to initialize the proxy.

In this example, the reader will see the IMarshal::ReleaseMarshalData() method in action. This method is COM's way of indicating to an object (which has been marshaled) that a marshaled object packet is being destroyed. A marshaled object reference can be considered an additional reference to an object. Hence, a call to ReleaseMarshalData should signal an object to perform reference count state management. This is especially true of objects which have been table-marshaled (i.e. their marshal data packets are stored in a table somewhere, e.g. the Global Interface Table) However it is not significant in our basic example, as will be the case for most "Marshal-by-value" proxies. This is because a "Marshal-by-value" proxy is independent of its original object.

Conclusion

In this first part of a multi-part dissertation on custom marshaling, I have attempted to provide an in-depth analysis of a proxy and a marshal data packet. In part two, we will go deeper by studying more advanced examples of custom marshaling. For the first time, we will touch on the role of stubs in custom marshaling.

Acknowledgements And References

Essential COM by Don Box. Published by Addison-Wesley

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Lim Bio Liong
Web Developer
Singapore Singapore
Lim Bio Liong is a Specialist at a leading Software House in Singapore.
 
Bio has been in software development for over 10 years. He specialises in C/C++ programming and Windows software development.
 
Bio has also done device-driver development and enjoys low-level programming. Bio has recently picked up C# programming and has been researching in this area.

Comments and Discussions

 
QuestionSynchronization? PinmemberRené Greiner7-Feb-13 8:52 
GeneralMy vote of 5 Pinmembermanoj kumar choubey28-Feb-12 18:17 
GeneralThis was extremely helpful PinmemberNigel Stuke8-Sep-11 14:15 
QuestionAny update for Part 2 ? Pinmemberboytan25-Mar-11 20:18 
AnswerRe: Any update for Part 2 ? Pinmemberehaerim13-Jun-13 10:30 
GeneralThank you Bio... very nice! PinmemberPatrick Parks10-Apr-08 20:11 
GeneralIf i do not want to pass a Stream handle to the thread Pinmemberaslamshikoh22-Feb-08 0:24 
Questionpass interface pointer to exe server Pinmemberpearman198211-May-07 1:52 
AnswerRe: pass interface pointer to exe server PinmemberLim Bio Liong11-May-07 6:51 
GeneralRe: pass interface pointer to exe server [modified] Pinmemberpearman198211-May-07 7:48 
GeneralRe: pass interface pointer to exe server PinmemberLim Bio Liong13-May-07 0:09 
Hello pearman1982,
 
I believe the problem with your code is that you have not developed and registered a Proxy/Stub DLL for your interfaces ITest, IClientInterface and ICallback.
 
The need for a Proxy/Stub DLL arises when your interfaces are derived directly from IUnknown (viz IDispatch).
 
Even if your implementation codes are inside a DLL, you may still get a crash if ITest::CallbackMethod() is called from a thread separate from the one in which it was created. You need a Proxy/Stub DLL to perform proper marshaling.
 
For more information on how to create a proxy/stub DLL, please refer to my blog entry :
 
http://www.codeproject.com/script/profile/whos_who.asp?msg=1413206&id=24366#xx1413206xx[^]
 
I have also created a set of sample codes for you.
 
Please email me :
 
bio_lim_2004@yahoo.com
 
and I will email you my sample codes.
 
Best Regards,
Bio.
 

GeneralRe: pass interface pointer to exe server Pinmemberpearman198213-May-07 6:31 
GeneralRe: pass interface pointer to exe server PinmemberLim Bio Liong13-May-07 6:38 
GeneralRe: pass interface pointer to exe server Pinmemberpearman198213-May-07 8:18 
GeneralRe: pass interface pointer to exe server PinmemberLim Bio Liong13-May-07 18:06 
GeneralRe: pass interface pointer to exe server Pinmemberpearman198213-May-07 23:35 
GeneralRe: pass interface pointer to exe server PinmemberLim Bio Liong14-May-07 0:48 
GeneralRe: pass interface pointer to exe server Pinmemberpearman198214-May-07 0:57 
GeneralRe: pass interface pointer to exe server PinmemberLim Bio Liong14-May-07 3:23 
GeneralThoroughly impressed. PinmemberKirkMan30-Apr-07 14:05 
GeneralVery Nice PinmemberJeffrey Walton16-Dec-06 1:39 
GeneralA Great Article Atleast for Me [modified] Pinmembersunit54-Sep-06 21:01 
GeneralRe: A Great Article Atleast for Me PinmemberLim Bio Liong7-Sep-06 14:58 
GeneralRe: A Great Article Atleast for Me PinmemberThatsAlok4-Oct-06 7:19 
GeneralRe: A Great Article Atleast for Me PinmemberLim Bio Liong4-Oct-06 19:07 
GeneralRe: A Great Article Atleast for Me [modified*2] PinmemberThatsAlok4-Oct-06 19:36 
GeneralRe: A Great Article Atleast for Me PinmemberLim Bio Liong4-Oct-06 20:47 
GeneralRe: A Great Article Atleast for Me PinmemberThatsAlok4-Oct-06 7:21 
GeneralRe: A Great Article Atleast for Me PinmemberLim Bio Liong4-Oct-06 19:17 
NewsUpdated Article Source Codes PinmemberLim Bio Liong20-Aug-06 16:57 
GeneralRe: Updated Article Source Codes PinmemberJeffrey Walton16-Dec-06 1:37 
QuestionBenefits of Custom Marshaling over Standard Marshaling? PinmemberWong Shao Voon20-Aug-06 16:07 
AnswerRe: Benefits of Custom Marshaling over Standard Marshaling? [modified] PinmemberLim Bio Liong20-Aug-06 18:47 
GeneralRe: Benefits of Custom Marshaling over Standard Marshaling? Pinmembertrevorde stickman1-Jan-07 15:27 
AnswerRe: Benefits of Custom Marshaling over Standard Marshaling? PinmemberLim Bio Liong21-Aug-06 6:23 
GeneralRe: Benefits of Custom Marshaling over Standard Marshaling? PinmemberWong Shao Voon21-Aug-06 18:58 
GeneralRe: Benefits of Custom Marshaling over Standard Marshaling? PinmemberWong Shao Voon22-Aug-06 16:05 
GeneralRe: Benefits of Custom Marshaling over Standard Marshaling? PinmemberLim Bio Liong7-Sep-06 14:52 
GeneralRe: Benefits of Custom Marshaling over Standard Marshaling? Pinmemberehaerim26-Aug-06 6:24 
GeneralRe: Benefits of Custom Marshaling over Standard Marshaling? Pinmembersvsundar8131-Aug-06 5:05 
GeneralRe: Benefits of Custom Marshaling over Standard Marshaling? PinmemberLim Bio Liong7-Sep-06 14:56 
GeneralSample Client code cannot be built PinmemberWong Shao Voon20-Aug-06 15:57 
GeneralRe: Sample Client code cannot be built PinmemberLim Bio Liong20-Aug-06 16:50 
GeneralRe: Sample Client code cannot be built PinmemberLim Bio Liong20-Aug-06 18:22 
GeneralThe need for a message pump PinmemberWong Shao Voon20-Aug-06 19:03 
GeneralRe: The need for a message pump PinmemberWong Shao Voon20-Aug-06 19:37 
AnswerRe: The need for a message pump PinmemberLim Bio Liong20-Aug-06 20:53 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140827.1 | Last Updated 18 Aug 2006
Article Copyright 2006 by Lim Bio Liong
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid