Circular Reference-proof ATL Object Collections

• Download the implementation headers - 9 Kb
• Download ATL COM Demo - 23 Kb
• Download VB 6.0 demo - 3Kb

Introduction

A common problem with collections of COM objects occurs when each member of the collection needs to be able to refer back to the parent collection object. There are a number of articles on the net trying to solve this problem using weak links, but these have a serious reliability problem if the weakly linked object gets destroyed by the client. The problem becomes much more serious when you need to maintain collections of collections.

This article demonstrates a possible solution, which is extremely easy to implement. I have included the small test programs in the demo file, that really do nothing except confirm that all object reference counters return to zero at the right time. It includes an ATL COM exe project (in VC++ 2003) implementing the templates, and a small VB6.0 project to demonstrate the creation and deletion of objects in any order. It is easy to see that the reference counts return to zero when all objects are destroyed, simply by viewing the Windows Task Manager, and watching the executable unload itself.

Using the code

I have designed 2 files, which comprise the source download, to be included in any ATL project that requires this type of collection. The CollectionDefs.idl can be #included in your project's IDL, after the imports of oaidl.idl and ocidl.idl. Then include the CollectionHelpers.h file in any module headers that require the implementations of any of these interfaces.

From here on, I will demonstrate examples for the usage of these headers using the Department/Employee example. To use the templates, first define your Collection's Coclass and Interface. At this time, you should also at least declare the Coclass and Interface of your collections member's. We then start off implementing the Collection object.

Standard properties and methods that have been implemented in the Helpers module include Add and Remove methods, and get_Item, get__NewEnum, and get_Count properties. You can expose your own interfaces for any of these in your collection interface, and simply call on the helper class to implement these functions. An implementation of IEnumVARIANT has also been included, so the _NewEnum property returns a VB compliant collection.

// CollectionTest.idl
// ...
interface IDepartment : IDispatch{
  [propget, id(DISPID_VALUE), helpstring("property Item")] 
    HRESULT Item([in] VARIANT Index, [out, retval] IEmployee** pVal);
  [id(DISPID_NEWENUM), propget] HRESULT _NewEnum(
    [out, retval]IUnknown **ppUnk);
  [propget, id(1), helpstring("property Name")] 
    HRESULT Name([out, retval] BSTR* pVal);
  [propput, id(1), helpstring("property Name")] 
    HRESULT Name([in] BSTR newVal);
  [id(2), helpstring("method Add")] HRESULT Add([in] 
    IEmployee* pEmployee);
  [id(3), helpstring("method Remove")] HRESULT Remove([in] VARIANT Index);
  [propget, id(4), helpstring("property Count")] 
    HRESULT Count([out, retval] long* pVal);
};

Once the Coclass has been defined in your C++ header file, simply inherit it from CContainer. The CContainer class is templated, taking your collection's member's exposed interface and the type of STL container that should be used as template parameters. The first template argument is pretty much self-explanatory, (in our example, it is the IEmployee interface), but the second argument requires a little forethought, and must adhere to a couple of prerequisites. Our example uses a map class, to enable fast lookup of an employee based on name.

class ATL_NO_VTABLE CDepartment : 
  public CComObjectRootEx<CComSingleThreadModel>,
  public CComCoClass<CDepartment, &CLSID_Department>,
  public IDispatchImpl<IDepartment, &IID_IDepartment,
   &LIBID_CollectionTestLib, /*wMajor =*/ 1, /*wMinor =*/ 0>,
  public CContainer<IEmployee, std::map<CComBSTR, IMemberData*> >
{
  ...
};

There are 4 types of STL containers that have had implementation code written in the header. These are list, vector, set, and map containers. Each of the containers hold a reference to an implementation of an IMemberData interface, which will eventually be up to you to write. The map container has only been implemented to accept a CComBSTR type index.

The actual collection is stored in a member variable of the CContainer object, m_coll, which has a type of CSTLContainer<YourSTLContainerType>. This class is directly inherited from YourSTLContainerType, but overloads the find, append, insert, and erase methods that are common to many of the STL collections. It is therefore possible to operate on m_coll in your own methods in a consistent way, regardless of the collection type being used. It also has a Reference() method, which takes an STL iterator as a parameter, and returns a pointer to your IMemberObject.

In order to implement the standard collection type methods and properties in your collection, it is simply a matter of referring them to the implementation in the CContainer object.

STDMETHODIMP CDepartment::Remove(VARIANT Index) {
  return Container::Remove(Index);
}

STDMETHODIMP CDepartment::get_Count(long* pVal) {
  return Container::get_Count(pVal);
}

STDMETHODIMP CDepartment::get__NewEnum(IUnknown **ppUnk) { 
   return Container::get__NewEnum(ppUnk); 
 }
 
STDMETHODIMP CDepartment::Add(IEmployee* pEmployee) {
  CComQIPtr<IMember> pMember = pEmployee;
  if (!pMember)
    return E_INVALIDARG;
  return Container::Add(pEmployee);
}

STDMETHODIMP CDepartment::get_Item(VARIANT Index, IEmployee** pVal) {
  CComPtr<IMember> pMember;
  HRESULT hr = Container::get_Item(Index, &pMember);
  if (SUCCEEDED(hr))
    hr = pMember.QueryInterface(pVal);
  return hr;
}

Notice that the Remove, get_Count, and get_NewEnum call directly back to the CContainer's implementation (Container is simply an internal typedef for CContainer<typename IExposed, typename STLContainer>). The Add method has a couple of lines of code at the top, just to ensure that the object being passed does, in fact, implement IMember (which is a prerequisite). Otherwise, it just calls the CContainer method too. CContainer's get_Item method actually returns the IMember interface, which in turn is queried for the IEmployee interface to return to the client.

The next step is to implement your member objects. This takes a little more work, as this is where the circular referencing situation is handles. The way it works is that the implementation of your member objects, minus any parental references, is implemented in an entirely different Coclass. Your actual exposed objects simply reference this CMemberData class in order to set and retieve its state properties.

The way to implement this is to define the interface for your member object, and then copy and rename the class definition. This means going through and changing a few lines of the ATL generated code, but not a lot of work. You need to rename the constructor, of course, as well as theBEGIN_COM_MAP entry. As this class is not exposed to the client, you should remove its inheritance from ComCoClass, and change the DECLARE_REGISTRY entry to DECLARE_NO_REGISTRY. After that, inherit your new data class from CMemberData, and your exposed coclass from CMember. Once again, these classes are templated to allow the helpers to cross-reference each other.

The CMemberData class takes one template parameter, which is your exposed coclass's implementation. This allows a CMemberData object to create an instance of your coclass, insert a reference to itself, and return this newly created object to the client. This is the method used to avoid the circular reference. What is important here is that the CMemberData object does not contain any references to the parent collection. All upward references are held be the CMember implementation in your exposed object.

// CEmployeeData
class ATL_NO_VTABLE CEmployeeData : 
  public CComObjectRootEx<CComSingleThreadModel>,
  //public CComCoClass<CEmployee, &CLSID_Employe>,
  public IDispatchImpl<IEmployee, &IID_IEmployee, 
    &LIBID_CollectionTestLib, /*wMajor =*/ 1, /*wMinor =*/ 0>,
  public CMemberData<CEmployee>
{
public:
  CEmployeeData() { }

DECLARE_NO_REGISTRY ( )

BEGIN_COM_MAP(CEmployeeData)
  COM_INTERFACE_ENTRY(IEmployee)
  COM_INTERFACE_ENTRY(IDispatch)
  COM_INTERFACE_ENTRY(IMemberData)
END_COM_MAP()
//...
protected:  // Property Data
  CComBSTR  m_sName;
public:
  STDMETHOD(get_Name)(BSTR* pVal) { *pVal = m_sName.Copy(); return S_OK; }
  STDMETHOD(put_Name)(BSTR newVal) { m_sName = newVal; return S_OK; }
  STDMETHOD(get_Department)(IDepartment** pVal) { return E_NOTIMPL; }
};

Notice that this data class also implements your exposed interface. This is not really necessary, but makes life simpler to implement. In fact, it really does not need to implement any interfaces except IMemberData, which has, in fact been implemented by the CMemberData class you inherited from. What is does need to do is to be able to provide minimum state information to your exposed member class. However, by implementing the exposed interface, your exposed object simply passes all requests for state information directly to the data class through its m_pData member. The property that refers to the parent collection (get_Department) in this case simply returns E_NOTIMPL.

There are also a number of helper methods associated with the IMemberData class, to improve the efficiency or data retrieval from the STL collections. These methods allow your set or map based collections perform comparisons on either the objects themselves, or upon string keys obtained from the object. There are five methods involved, two to allow object comparisons, and three to allow key comparisons. By implementing one or both of these groups of methods, the STL is able to do fast lookups of your objects based on their contents.

The methods used for object comparison prototype's are:

HRESULT IMemberData::CanCompareObjects(void);
HRESULT IMemberData::CompareObject(IMemberData* pObject, short *pResult);

CanCompareObjects simply returns S_OK if the CompareObject method has been implemented, or S_FALSE if it has not. CompareObject, when implemented, works the same was as the library string comparison functions, returning a negative, zero, or positive result based on whether the object being compared is less than, equal to, or greater than the object being compared to respectively. Implementing these methods allow a set of IMemberObjects ot be stored and referenced efficiently.

Alternatively, it is possible to use a map collection to index your objects based] on a BSTR key. By implementing the key comparison methods, it is possible to efficiently add objects with an embedded key to collection, and then access those objects based on that key value. The prototypes for the key-access methods are:

HRESULT IMemberData::CanCompareKeys();
HRESULT IMemberData::CompareKey(BSTR Key, short *pResult);
HRESULT IMemberData::get_Key(BSTR *pKey);

These work basically the same way as the CompareObject methods, with the addition of the get_Key() method to extract the key from a complete object.

Finally, to complete your collection implementation, you need to derive your exposed coclass from CMember. This class takes 2 template parameters, being the Coclass of your IMemberData implemetaion, and the interface exposed by this coclass. If this interface is the same as the one implemented in your CMemberData derived class, then implementing the interface in this class is as simple as calling the same method in the data class.

HRESULT CExposedClass::MyMethod(Param1, Param2) { 
  return CDataClass::MyMethod(Param1, Param2);
}

The only difference to this setup is when you want to reference the parent collection. Remember that the data class can not have a reference to the parent, or else the whole system becomes invalid, and you will end up with your circular references. Consequently, the CMember implementation holds a full (not weak) reference to the collection, called m_pParent, which can be returned to the client application.

HRESULT CExposedClass::get_Parent(IMyCollection **ppVal) {
  return m_pParent->QueryInterface(ppVal);
}

Once all this is implemented, you have a reference-safe collection, where each of the members of the collection hold a secure link back to the parent. So long as the client holds a reference to any member of the collection, or to the collection object itself, or event to an enumerator on the collection, you can be guaranteed that all the objects will remain intact. As soon as the client releases all these references, the objects are correctly destroyed, and the resources are released back to your application.

How it works

Apart from the tricky little bits of code implementing the indexing on the STL containers, the idea behind this collection is pretty simple. The CContainer class has one member, which is the STL container itself. When this is populated, it is populated by CMemberData derived objects. These objects contain the state information required to correctly implement the objects you want to collect. These objects do not, however, reference back to the container, so the container's reference count remains at 1 for each reference held by the client. The MemberData objects' reference counts are also at 1, for the reference being held by the container object. They will not be destroyed until the container is destroyed.

If the client creates a new Member object, it also creates a MemberData object internally for holding its state information. This MemberData object is referenced by the CMember's m_pData member. When this object is added to the collection, the CContainer's add method actually puts the m_pData object into the collection, and then sets the m_pParent member of the CMember class to point to itself. At this point, the reference count of the Container and the MemberData objects are incremented to 2, while the CMember's reference count remains at 1. Releasing the CMember object in the client will then cause the object to be destroyed, while its state information remains intact in the collection. The reference count of the Container and MemberData objects are decremented back to 1 at this time.

When the client retrieves a member from the collection, the implementation actually create a new CMember instance, and then replace this new object's state data with the CMemberData object from the collection. It also sets the m_pParent reference in the new object.

If the client releases their hold on the Container object, whilst retaining a reference to a Member object, the Container will remain intact, due to the reference tom_pParent in the Member. Consequently, all of the MemberData objects in the collection remain intact, as these are referenced by the Container. Only when all client references to the Container, and any Members of the Container are released, will the Container get destroyed. The actual Member object are only ever reference by the client, so it has complete control over their life-span.

Implementation Summary

To recant, follow these 10 simple steps to implement a circular reference-proof collection.

1. Define your collection class and interfaces with the ATL wizard.
2. Define your collection member's class and interface with the ATL wizard.
3. Inherit your collection class from CContainer<IExposed, STLType>, and add a reference to the IContainer interface to the COM_MAP.
4. Copy and rename your member class defintion in the generated header file, remembering to remove the copied class's inheritance from CComCoClass, to DECLARE_NO_REGISTRY, and to change the Class name in the constructor and the COM_MAP declaration.
5. Derive your copied class from CMemberData<CMemberCoClass>. Add a reference to the IMember interface in this classe's COM_MAP.
6. Derive your original member class from CMember<CMemberDataClass, IExposed>, and add a reference to IMember to this class's COM_MAP.
7. Write your member's implementation code in yout copy of the original object (the one derived from CMemberData).
8. Reference the MemberData's properties and method's through the CMember's m_pData member.
9. Implement a (get_)Parent property or method by returning a reference generated to CMember's m_pParent member.
10. Implement your collection object by referencing the methods pre-defined in the CContainer class