XONOR pointers. eXclusive Ownership & Non Owning Reference pointers

• Download demo project - 180.25 KB
• Download source - 16.66 KB

Update

This article and the associated source code have been updated.

The scope of the system presented has been expanded, its operation simplified and the source code made more transparent.

A more complete description of the changes is given in the revision history.

Introduction

Here is a smart pointer system suitable for C++ application development in which:

• You write less object and reference management code than in Java or C#.
• You lose none of precise and deterministic control of object lifetimes that C++ provides.
• You do not suffer from memory leaks or dangling pointers.
• You do not suffer from unintended failure to release references
• Although there is a small memory and execution overhead, its operation is direct and without iterations.

It adheres to the principle that object lifetimes and memory use should be deterministically controlled by scope and the programmers explicit design and should not be compromised by unreleased secondary references as occurs with the practice of using boost::shared_ptr as a 'catch all' solution.

It is accompanied by an example project which demonstrates and tests many aspects of the smart pointer system. It is a Microsoft Visual Studio 2005 project using ATL/WTL. The smart pointer system itself is pure C++ but this implementation makes use of ATL collection classes to provide arrays and lists.

Background

There has been for many years a well established model of C++ programming in which all objects dynamically created with the new operator are assigned to a smart pointer which ensures that the object is deleted automatically at the appropriate time - which is when the smart pointer is reset or goes out of scope. This model is good for eliminating memory leaks and also eliminates the need to write code that calls the delete operator. These smart pointers can be described as exclusive ownership pointers. Examples are:

• std::auto_pointer
• boost::scoped_ptr

This works very well, introduces no complications and is a thoroughly recommended practice but it still leaves a large problem unsolved which send shivers up the spine of many developers and more importantly their bosses - the 'dangling pointer':

All of the programs I have written have also required me to use pointers which don't own the objects they point at, they just reference them as part of the operation of the program e.g. a pointer to the current focus. For these I have always used raw pointers because that is all there are. The problem is that these secondary non owning references can very easily become dangling pointers. Being just plain raw pointers they don't know when the object they point at has been deleted. Code has to be written to ensure that when an object is deleted, all pointers that reference it are set to NULL first. This code can be complex, repetitive, ugly, tedious to write and therefore very easily faulty.

There is no doubt that the highly skilled (and therefore expensive) man hours that are lost ensuring that pointers don't get left dangling or even worse trying to find out why an application crashes inexplicably have led to the popularity of languages that rely on garbage collection to manage object lifetimes such as Java and C#.

My feeling is that if programmers choose Java or C# rather than C++ because they are more convenient for certain tasks then that is fine but if they are frightened to touch C++ for fear of dangling pointers, then let's fix the problem in C++.

There are garbage collectors for C++ but really they turn it into another language. For one thing they will play havoc with many class libraries that encapsulate the releasing of resources in destructors.

The solution that seems to be gaining acceptance for C++ is to wrap all pointers as strong or shared pointers so that objects are only deleted when the last thing pointing at them is reset or goes out of scope. This provides the same guarantee that objects will stay alive as long as something points at them but instead of waiting for a garbage collector they are deleted promptly as soon as nothing points at them. This is the solution recommended by C++ Standard Library TR1 extension for memory handling which has adopted the boost::shared_ptr.
This worries me because it has two drawbacks that are not good for the health of C++ development:

1. One is widely acknowledged - the problem of cyclic references. If you use shared_ptr for all secondary references then when two objects directly or indirectly refer to each other they will never be released – each keeps the other alive forever. This is a classic permanent memory leak caused by using shared_ptr. The recommended solution is to identify the cyclic reference situation and break the cycle with a weak_ptr. The fact that a weak pointer is needed to break a cycle of strong references simply illustrates how inappropriate it is to use strong pointers for these roles in the first place. Cyclic references are not always easy to identify so this represents a considerable programming hazard that shared_ptr introduces.
2. The other problem is hardly ever mentioned although its equivalent in the world of garbage collection has now become an issue and has a name "failure to release secondary references". Using shared_ptr to wrap all pointers does eliminate the possibility of pointers dangling but it does so by extending the lifetime of the objects they point at. This is OK if it is what you want but if it was simply an oversight not to null the secondary reference then you will be retaining the object it points at in memory for no good reason – probably in conflict with your design intentions.

With both of these problems, neither the compiler or the run-time system have any way of knowing that you have made a mistake so you don't even get a warning – you will not know that your design intentions have been compromised until serious memory problems emerge further down the line.

What is being missed here is that the problem in C++ never was objects being deleted too early, it was secondary pointers continuing to point at them when they are not there because of an oversight of the programmer. So why are we trying to cover up the error of the programmer by making the objects live longer?

To put this simply:

• To extend the lifetime of an object (which was probably correctly designed) to appease a non nulled secondary pointer (which was an oversight) is plain wrong.
• Don't extend the life of the object to appease the erroneous pointer – tell the erroneous pointer that the object has been deleted.

I worry that the very action of making all pointers shared_ptrs will lead to sloppy programming and that the fact that this model specifically hides the consequences of this sloppiness is going to degrade the quality of C++ development.

While considering all of this I found myself thinking "all I want is that my secondary references automatically test as null if the object they were pointing at has been deleted".

The system presented here is the result of doing just one thing: having a smart pointer for non-owning reference pointers which includes a mechanism to automatically set them to NULL when the object they point at is destroyed. The mechanism necessarily has to have the owner of the object involved therefore a new type of owner pointer is required to work with it.
For this reason I have called it:

XONOR pointers = eXclusive Ownership & Non Owning Reference pointers

XONOR Pointers

A smart pointer system with a mechanism to eliminate both memory leaks and dangling pointers which respects the distinction between exclusive ownership pointers and non-owning reference pointers. A smart pointer system offers the opportunity for a much needed declarative distinction between the two - and here it is:

owner_ptr<T> is an exclusive ownership pointer to an object of type T.

It is the primary reference to a dynamically allocated object and the controller of its lifetime. Effectively you can think of it as being the object itself (it is as close as you can get to it!) .

ref_ptr<T> is a non owning reference to any object of type T whose lifetime is already controlled by an owner_ptr.

This is a weak pointer but unlike the second class TR1 weak_prt<T> it is a first class pointer that can be de-referenced and used in the same way as a raw pointer. You use it everywhere that you want to hold a reference to something that already exists. It is named simply ref_ptr to indicate that it is just an observing reference and the name is brief to encourage its frequent and almost casual use.

With this distinction made, it is possible to construct a system which handles memory correctly with no anomalies and no new programming hazards.

In addition a shared ownership pointer is provided for genuine cases of shared ownership (not for holding secondary references!!)

strong_ptr<T> a shared ownership pointer.

Sometimes you really do want a shared ownership pointer, for instance to control the life of a resource or service with widely distributed use. strong_ptr is provided for this.

Also a strong_ptr survives the storage mechanisms of STL collections that read elements out by making temporary copies. With owner_ptr<T> (or any exclusive ownership pointer) this 'copying out' process has the undesirable effect of causing the pointee to be deleted.

Building a Complete Integrated Solution

The aim of this system is to provide C++ programmers and their managers with reasonable confidence that their code will not generate memory leaks or suffer from dangling pointers. Therefore it must embrace as much as possible of what they are likely to do. Ideally it should be possible to code without declaring or using any raw pointers. This ideal cannot be fully achieved because most APIs work with raw pointers but what can be achieved is that the use of raw pointers is considerably reduced and made explicit so that any problems they cause can be quickly found. With this aim, the following has been added:

new_ptr<T>

Sometimes some work needs to be done on a new object before assigning it to a permanent owner - this may even include deciding what is to be the permanent owner or owners: new_ptr<T> can temporarily hold an object created with the new operator before assigning it to an owner_ptr or strong_ptr. This is useful for initialising a new object before adding it to a collection.

If the new object is not assigned to an owner, new_ptr will delete it when it goes out of scope.

Also new_ptr can be used as the return value of an object creation function. It has the advantage that if the return value is not assigned to an owner then the object will be deleted automatically when the function returns.

Once a new_ptr has been assigned to an owner, a ref_ptr can be taken from it which will point at the new owner.

You cannot take a secondary reference (ref_ptr) from a new_ptr until it has been assigned to an owner. If you attempt to, the secondary reference ( ref_ptr) will simply hold NULL even though the new_ptr has a value.

The same initialisation work can be done using owner_ptr or strong_ptr as both can be returned from functions, however new_ptr<T> has three advantages:

1. It provides a declarative indication within a function or code block that its purpose is the initialisation of a new object before assigning it to an owner.
2. It allows creation functions to be created that work for both owner_ptr and strong_ptr (conversion between owner_ptr and strong_ptr simply cannot be allowed).
3. It allows a slightly more concise and tidier initialisation sequence in many cases as shown in the code samples below:

Using owner_ptr<T>:

//Adding a CRectangleBox to an array of VisualObjects (Common base class)
//and returning the object created.
//Using owner_ptr<CRectangleBox> to hold created object
ref_ptr<CRectangleBox> AddRectangleBox(CPoint point, CSize size)
{
owner_ptr<CRectangleBox> newRectangleBox(new CRectangleBox);
//object created and assigned to owner_ptr
//take reference BEFORE assigning to owner
ref_ptr<CRectangleBox> rBox=newRectangleBox;
//we have to declare a ref_ptr to hold a reference the new object because
//because once newRectangleBox has been assigned to the array, it will loose
//ownership and be set to NULL
m_VisualObjects.Add(newRectangleBox);//assign to owner
//newRectangleBox is now EMPTY so we must use the reference rBox
//to complete the initialisation and as the return value.
rBox->m_Point=point;
rBox->m_Size=size;
return rBox;
}

Using new_ptr<T>:

//Adding a CRectangleBox to an array of VisualObjects (Common base class)
//and returning the object created.
//Using new_ptr<CRectangleBox> to hold created object
ref_ptr<CRectangleBox> AddRectangleBox(CPoint point, CSize size)
{
new_ptr<CRectangleBox> newRectangleBox(new CRectangleBox);
//object created and assigned to new_ptr
//with new_ptr you CANNOT take a reference
//until it has been assigned to an owner
m_VisualObjects.Add(newRectangleBox);//assign to owner
//we can continue to use newRectangleBox (the new_ptr) to complete
//the initialisation and as the return value.
newRectangleBox->m_Point=point;
newRectangleBox->m_Size=size;
return newRectangleBox;
}

The difference is that with owner_ptr (as with std::auto_ptr) assignment is necessarily destructive. This means that the assignee takes ownership and the assigner is set to NULL - this has to be the case because two pointers can't both be exclusive owners. new_ptr behaves differently; instead of setting to NULL when it is assigned to an owner, it becomes a passive non-owning observer (for the rest of its short lifetime). References can then be taken which will point to the new owner.

It is not necessary to use new_ptr<T> but its use is recommended.

fast_ptr<T>

This is a variant of ref_ptr<T>. ref_ptr has a bit of execution overhead each time you deference it - it checks that the object is still there first (that is what it is for). If you have a code block that makes extensive use of a ref_ptr, de-referencing it many times, then it is a bit obsessive and wasteful of execution time to check that the object exists each time. fast_ptr behaves slightly differently - it doesn't check that the object exists at all when it is de-referenced, instead it locks the object into existence when the fast_ptr is initialised and unlocks when it goes out of scope. This doesn't mean it keeps the object alive, it means that any attempt to delete the object while the fast_ptr is using it will cause an exception to be thrown.

Use fast_ptr only within code blocks within which you can be reasonably confident that nothing is going to try and delete the object it is referencing. A fast_ptr can never be null. It must be initialised on construction with a valid non null reference and cannot be reset. The only way to use it is as follows:

if(rObject)//Test that rObject is valid
{
fast_ptr frObject=rObject; //Construct the fast_ptr from rObject
frObject->DoThis();//Use it.....
frObject->DoThat();
//etc.....
}

It is not necessary to use fast_ptr<T> but its use can speed up the execution of critical code blocks.

enable_ref_ptr_to_this<T> class and its ref_ptr_to_this() method:

Sometimes it is necessary to take a reference to the this pointer from within a class. enable_ref_to_this<T> is an add-in base class which you can add to the inheritance list of the class that you are working with. It provides ref_to_this() method which returns a ref_ptr wrapping the this pointer.

enable_self_delete<T> class and its self_delete() method

Some objects delete themselves: for instance modeless dialogs. enable_self_delete<T> is an add-in base class which you can add to the inheritance list of the class that you are working with. It must be initialised by an owner_ptr, either on construction or by calling its set_owner() method. The object should delete itself by calling the self_delete() method. The self_delete method ensures that the owner_ptr is nulled as the object is deleted. In this case the owner_ptr must not be stored in a dynamic array or any kind of container that moves its elements around in memory.

referencable(Type, value variable name) is a macro for declaring objects by value so that a ref_ptr can be taken from them. Sometimes we have a class member declared by value because there is no reason to declare a pointer and create the object with the new operator but we still want to be able to take a safe ref_ptr reference from it.

ref_ptr_to(value variable name) is a macro for taking a ref_ptr from an object declared by value using the referencable macro.

Collections of Xonor Pointers

Some care is needed with storing owner_ptr<T> in collections. It seems that all collections make temporary copies of their elements as they are added and owner_ptr<T> is OK with this (due to its destructive copy constructor). This means that you can use owner_ptr<T> in ATL collections such as CAtlArray and CAtlList.

For ATL the optional XonorAtlColls.h defines owner_ptr_array<T> and owner_ptr_list<T> as follows:

template <class T> class owner_ptr_array : public CAtlArray<owner_ptr<T> >
{
};
template <class T> class owner_ptr_list : public CAtlList<owner_ptr<T> >
{
};

Some collections (in particular STL (Standard Template Library) collections) also make temporary copies of their elements as they are read out and while performing various internal operations. owner_ptr<T> is NOT OK with this - it causes the elements involved to delete their objects. You cannot directly use any kind of exclusive ownership pointer in STL collections - they may become unintentionally deleted. For STL collections you must use strong_ptr<T> for elements that own their object. strong_ptr<T> as its name suggests is very robust and will survive just about anything. Of course if you do this you lose the declarative guarantee of exclusive ownership but if you are careful not to share these elements with other strong_ptr<T>s they will still behave as exclusive owners.

For STL the optional XonorSTLColls.h defines owner_ptr_array<T> and owner_ptr_list<T> as follows:

template <class T> class owner_ptr_array : public vector<strong_ptr<T> >
{
};

template <class T> class owner_ptr_list : public list<strong_ptr<T> >
{
};

If you really want to have an STL collection that holds owner_ptr<T>s (because you want a declarative assurance that they will not be shared), you can use owner_ptr<T, false> (which doesn't delete its pointee when it goes out of scope) for the collection elements and override the collection class to handle every occurrence that involves removing elements from the collection so that reset() is called on the owner_ptr. This is a bit of work but results in a solidly engineered generic collection which does exactly what you want.

There is no problem whatsoever with storing ref_ptr in any kind of collection and fast_ptr just doesn't belong in collections.

Rules of Use

The following operations are the same for all smart pointers in this system except that fast_ptr cannot be initialised as NULL:

dereference operator ->	opObject->AMethod()	All pointers
initialise as NULL	owner_ptr<CObject> opObject; owner_ptr<CObject> opObject(NULL); owner_ptr<CObject> opObject=NULL;	All pointers except fast_ptr
test for non NULL	if(opObject) if(opObject!=NULL)	All pointers except fast_ptr
test for NULL	if(!opObject) if(NULL==opObject)	All pointers except fast_ptr

Comparison tests if two pointers point at the same object. In the case of comparing two owner_ptrs or an owner_ptr with a strong_ptr it will always return false. Two owner_ptrs and also an owner_ptr and a strong_ptr cannot point at (and therefore own) the same object therefore by definition the comparison must return false.
new_ptr does not support comparison.

comparison	if(opObject==rCurrentSelection)	ref_ptr with ref_ptr	Tests that both pointers point at the same object
		ref_ptr with fast_ptr
		ref_ptr with owner_ptr
		ref_ptr with strong_ptr
		fast_ptr with strong_ptr
		fast_ptr with owner_ptr
		strong_ptr with strong_ptr
		strong_ptr with owner_ptr	Always returns false
		owner_ptr with owner_ptr
		new_ptr	Does not support comparison

The rules of assignment and construction are what really determines the shape of this system because they determine how each kind of pointer gets to hold a non null reference.

assignment and construction	new_ptr	an uncast raw pointer returned by the new operator
	owner_ptr	an uncast raw pointer returned by the new operator
		a new_ptr which then becomes a passive observer from which references can be taken
		another owner_ptr which will loose ownership and be reset to NULL
	ref_ptr & fast_ptr (fast_ptr construction only)	a new_ptr that has been assigned to an owner
		an owner_ptr
		a ref_ptr
		a fast_ptr
		a strong_ptr
	strong_ptr	an uncast raw pointer returned by the new operator
		a new_ptr which then becomes a passive observer from which references can be taken
		another strong_ptr
		a ref_ptr. Assigns NULL if not a reference to another strong_ptr

The = operator of owner_ptr applied to another owner_ptr performs destructive assignment in the same way as std:auto_ptr. It transfers ownership and leaves the source pointer empty. This operation is NOT intended to be used in application code but has to be there so that owner_ptr can be returned from functions. Destructive assignment is anti-intuitive (you would expect = to make a copy and leave the source unchanged) therefore it is not helpful for it to appear in application code. Instead, use one of the two dot methods which make it clear what is being done:

• .make_copy(owner_ptr<T>) makes a value copy of the object owned by the other owner_ptr
• .steal_object(owner_ptr<T>) takes the other owner_ptrs object and makes it its own. The other owner_ptr is left as NULL. This is the same effect as the = operator would have.

Similarly owner_ptr has a public destructive copy constructor which is needed so it can be used in collections. Although it is public, you should not use it in application code - in this case it is hard to see why you should want to!

Polymorphism

Polymorphism is fully supported. Automatic implicit downcasts from derived to base class are provided for all assignments.

With a polymorphic system it is normally important that the base class destructor is declared as virtual - this ensures that the delete operator always deletes an object of the derived class that it was created with using the new operator - otherwise only the base class portion will be deleted leaving a memory leak (class slicing). It is very easy to forget to do this!
With this smart pointer system this is unnecessary - the smart pointer system knows which derived class was used to create the object and ensures that an object of the same class is deleted. It is very important NOT to cast an object explicitly before or during its assignment to an owner.

An explicit up_cast<T, U>() function is provided to cast all types from base to derived type where this is necessary. Otherwise there should be no explicit casting.

What Can Still Bite You

The points of vulnerability with this system are its interfaces with raw pointers.

1. owner_ptr, strong_ptr and new_ptr are initialised by a raw pointer returned by the new operator. The pointer returned by the new operator should be immediately assigned to one of these smart pointers without being changed in any way and then should never be used again.

   e.g.

   owner_ptr<U> op=new T; //Correct and safe

   It is not hard to see that the following would break the system:

   •  T* pT=new T; //First bad move -
       // Don't even give a variable name to the pointer returned
       // by the new operator.
     owner_ptr<U> op1= pT;
     //Now we have two owner_ptrs thinking that they own
     //the same object - disastrous
     owner_ptr<U> op2= pT;
     

     but this one is not so obvious

     //op will eventually delete an object of type U even though
     //the object created was type T
     owner_ptr<U> op=(U*)new T; 

     It may look like the cast (U*) is harmless or even helpful but it conceals the original type T from the initialisation of the owner_ptr. This line of code will prevent the anti class slicing mechanism from working resulting in memory leaks. The owner_ptr has its own implicit casting operator but it will only act correctly if it is assigned with the original uncast pointer returned by the new operator.

   To avoid such problems:

   • Never even name a variable to hold the pointer returned by the new operator - always assign it directly to a xonor pointer.

     Never cast the pointer returned by the new operator - not even during the assignment to a Xonor pointer.

   If you need to carry out initialisation on a new object before giving it an owner then assign the pointer returned by the new operator directly to a new_ptr which can hold it temporarily before it is assigned to an owner.

2. Many APIs take raw pointers as parameters. In general this is fine - you can pass in the raw pointer of any Xonor pointer by using the dot method .get_pointer(). There are two things an API can do with these pointers which would break the system:
   1. Delete the pointer you have passed - the smart pointer system will know nothing of this and will try to delete it again later
   2. Store the pointer you have passed - the stored pointer will not know when you delete the object and could dangle.

   Most APIs will not do these things and if they do, it will usually be made clear in the documentation.

3. Many APIs have return values which are raw pointers - here you should take a great deal of care. You need to know if you are being given a secondary non owning reference pointer (which it will be in most cases) or if it is a pointer that you have to take ownership of (if this is the case then the documentation should say so quite clearly).

   If you have to own the object then assign the return value directly to an owner_ptr, strong_ptr or new_ptr as if it was the return value from the new operator.

   If (as in most cases) it is a non owning reference pointer then use it and store it as a raw pointer. I'll say that again - USE IT AND STORE IT AS A RAW POINTER. It is futile to try and hold it with a Xonor pointer because we know nothing about its lifetime.
   Handle this raw pointer with care - this is something between you and the API that you are calling.

4. Xonor pointers can throw two types of exceptions xonor_ptr_exception and xonor_ptr_fatal_exception. Both types of exception are only thrown because of coding errors which you should correct. No external factors, not even memory or resource allocation will throw xonor_ptr_exceptions. It is not really a good idea to handle either type and continue execution but if you insist on covering up mistakes in this way then you can handle a xonor_ptr_exception and continue execution but NEVER handle a xonor_ptr_fatal_exception and continue execution - these fatal exceptions are typically thrown in constructors and destructors and therefore damage has already been done. If a xonor_ptr_fatal_exception occurs you have made a serious coding error and you must correct it.
5. The destructive copy constructor and assignment operator of owner_ptr applied to another owner_ptr has already been mentioned. Misunderstanding this won't provoke memory leaks or dangling pointers but it may surprise you.
6. The enable_self_delete add-in base class is strictly speaking not part of the Xonor pointers system, it is a utility that works with it and it has the capacity to break it. enable_self_delete uses the address of an owner_ptr so it is essential that the owner_ptr must not move. This means that it must not be stored in a dynamic array. Passing an owner_ptr stored in a dynamic array to enable_self_delete can result in the worst type of memory corruption that we have gone to so much trouble to avoid. So far I haven't found a way of detecting and preventing this so it is best to regard enable_self_delete as potentially dangerous but sometimes very useful. Taking the address of an owner_ptr is in general prevented but enable_self_delete has a special permission to do this but is unable to detect when it is inappropriate.

How It Works

For the most part these smart pointers are similar to others, implemented as template classes with a great deal of operator overriding and making use of destructors to either delete the pointee or reduce its reference count. What is different is the integration of several smart pointers with different roles to work together.

All smart pointers hold a pointer to an object of type T and also a pointer to a reference controller.

The reference_controller holds:

• A weak reference count
• A strong reference count
• A pointer to the original object and of the same type as the original object
  and also has a virtual function that is called to delete objects.

A reference controller is created whenever two or more xonor_ptrs point at the same object. It is also created on construction and assignment from the new operator when the stored type is a base class of the type created because its virtual function is required for the anti class slicing mechanism.

The strong reference count indicates the number of xonor_ptrs owning the object - this can be:

• 1 - owner_ptr or single strong_ptr
• many - multiple strong_ptrs
• or 0 - the object has been deleted

The strong reference count controls the object lifetime - when it goes to zero the object is deleted.

The weak reference count indicates the number of xonor_ptrs referencing the object -This can be:

• 1 or many - there is still a reference to this object
• or 0 - nothing references the object anymore

The weak reference count controls the lifetime of itself. When the count goes to zero, it is deleted.

The reference controller associated with a XonorPtr may not be specialised to the same type as the XonorPtr itself. This is because the reference controller must hold (and delete) a pointer of the same type as the original object created. The XonorPtr doesn't know the type specialisation of its reference controller. It only knows it as a unspecialised base class - for this reason it calls a virtual function of the reference controller when it is time to delete the object.

It does appear that there is some redundancy in holding the pointer to the created object. There is a copy in each XonorPtr and also in the reference controller. I think the redundancy is real, one copy should be enough if we could have a system that casts it appropriately depending on where it is used. I think that this could be done but my efforts, although partly successful, produced too much obfuscation which I particularly wanted to avoid. So for now the redundancy remains in the interest of having solid transparent guarantees of correct operation.

Now let's look at the "user forgets to null a reference" scenario in the simplest way:

 owner_ptr<T> opT=new T; //line 1 - create object and assign to owner
ref_ptr<T> rT=opT; //line 2 - take a secondary reference from the owner
r->DoSomething(); //line 3 - use the secondary reference to call a method
opT=NULL; //line 4 - reset the owner and delete the object
if(rT!=NULL) //line 5 - test that the secondary reference is still valid
rT->DoSomething(); //line 6 - use the secondary reference to call a method

In the first line, the internal pointer of opT is assigned by new T - no reference controller is created.

In the second line, the internal pointer of rT is assigned by the internal pointer of opT and a reference controller is created with a strong count of 1 and a weak count of 2.

In the third line, rT checks that the strong count is non zero and DoSomething() is called.

In the fourth line, opT is reset. The object is destroyed and its internal pointer is set to NULL. Also the strong count is set to zero and the weak count is reduced to 1.

In the fifth line, rT is tested for being non-null and returns false because the strong count is zero. The pointer knows now that the object is gone and therefore resets itself and releases the reference controller reducing its reference count to zero which causes the reference controller to be deleted.

The sixth line does not get called.

If the test for non null had been omitted, then the dereference operator would have made the same check on the strong count and would have thrown an exception.

The key functionality is that the reference_controler can outlive the object it is associated with and continues to exists so that any remaining secondary references are able to see that the object no longer exists.

strong_ptr has the same interaction with the WeakCount but additionally increments the StrongCount on taking a reference and decrements it when it goes out of scope or is reset.

To summarize:

• When the StrongCount decrements to zero, the object pointed at is deleted. The life of the object is over.
• An owner_ptr can simply delete the object. If it has a reference_controler, its StrongCount will be set to zero. The reference_controler itself will only be deleted if the owner_ptr was the last thing pointing at it.
• When the WeakCount decrements to zero, the reference_controler itself is deleted. There are no more secondary references that need to know about it.

There are two further complications:

If an object is held by a strong_ptr then its weak count is held as a negative number. This is to make it possible to assign strong_ptr from a ref_ptr if the object it references has shared ownership strong_ptr(s). This is correct and useful. A negative weak count indicates that it references an object with shared ownership and therefore this can be done. Otherwise the object is exclusively owned and this cannot be done - the strong_ptr will be simply assigned NULL.

fast_ptr uses a negative StrongCount to lock the object against destruction. If the StrongCount is negative when an attempt is made to delete the object (for instance its primary reference goes out of scope) then an exception is thrown.

A side effect is that if a fast_ptr is taken from a strong_ptr then the StrongCount is already in use with a positive value which produces a conflict. The solution is that the fast_ptr abandons the idea of locking and increments the StrongCount thus acting as another strong_ptr.This isn't really a problem because already the object is no longer directly controlled by scope and as fast_ptr normally has a short lifetime it is unlikely to unnaturally extend the life of the object. In any case if the fast_ptr is the last reference left keeping a shared object alive when it goes out of scope, it will throw an exception.

One thing that concerned me most was the allocation of reference_controlers.
Initially I had them being allocated directly from the heap using the new operator and then I realised two undesirable consequences:

• Frequent heap allocation of small objects is costly.
• Any reference_controler which remain in memory due to live references to deleted objects will tend to cause fragmentation making new memory allocations more complicated.

The solution to this was to create pool from which the reference_controlers are allocated. A pool of 100 is created on startup and if that is exhausted then it is extended by a further block which is 10% larger up to a maximum of 100 blocks. This mildly exponential growth rate doesn't increase the granularity of block allocations too dramatically but allows a maximum of 13 million reference controllers.

The idea is that this:

is better than this:

Although the first has 100 slots reserved for reference controllers and the second only occupies the space needed by the 4 live reference_controlers, the second case is more problematic for future memory allocations.

The reference controller pool always knows where the next free slot is (easy to achieve because they are all the same size) so allocation and deallocation is very fast.

Because the reference controller pool is global, its methods for allocation and de-allocation are protected by a single critical section so that using Xonor pointers in more than one thread doesn't cause thread conflicts. As indicated below that does NOT mean that these smart pointers are thread safe - they are not!

Transparency of Source Code

I have completely revised the layout of the source code implementing these smart pointers to try and make their operation as transparent as possible.

Each smart pointer class has a small number of private or protected methods that carry out specific actions, each with a verb like name. These are laid out as I normally lay out code - a new line for everything, indentation and lots of white space.

Having different types of smart pointers interacting with each other means that there are a large number of public methods representing the possible conversions between them - mostly constructors and overriding of the assignment operator. These are laid out as one-liners making a simple sequence of calls to the verb like private/protected methods. This makes it easier to get an overview of how they work, check them for consistency and verify their correctness.

I have done this very specifically in the hope that it will encourage programmers to try it and consider adopting it. To trust this system, it is not enough to agree that it is a good idea, you also have to be able to see that it has been done right!

Overhead and Performance

Yes - these is an overhead with this system:

• In general Xonor pointers occupy up to three times more space than if raw pointers were used.
• Assignment of one Xonor pointer to another involves the execution of several lines of code.
• Dereferencing of ref_ptrs involves an extra step of checking the strong ref count.

Mitigating this:

• Dereferencing of an owner_ptr or strong_ptr only involves a non null check of its own internal pointer.
• Dereferencing of a fast_ptr is instantaneous - as if it is a raw pointer.
• None of the execution overhead involves any iterations or walking of collections.
• A reference controller pool avoids continual construction and destruction of reference controllers on the heap.

Using the Source Code

Copy all of the XonorPtrs header files (All the header files that begin with `Xonor' found in the XonorPtrs folder) into an include directory and include the following in the file you want to build:

#include <XonorPtrs.h>
#include <XonorPtrs.hpp>
using namespace XonorPtrs;

If you compile more than one *.cpp file, then you should only include XonorPtrs.hpp in one of them.

To use it, just start declaring your pointers as XonorPtrs and code as normal. If you stay with it, you can enjoy the luxury of not having track down and NULL multitudes of secondary references.

Caveat

This system has been designed so that normal object orientated programming is more comfortable and less hazardous and I have tried to embrace as much as possible of the kind of scenarios that I have found in my programming experience.

• If you try strange and unconventional constructions then it may not work - I have tried to prohibit unusual use by generating compiler errors, but I doubt that this is watertight.
• It is not 'thread safe'. It does not provide a mechanism for the smart pointers to be simultaneously accessible to two threads.

One of the problems with developing a smart pointer system is that if it does not work properly, it will cause the very problems it has been designed to solve. What I present here works and has not failed any test I have put it through but the code implementing it is sufficiently lengthy and complex to have some chance of containing errors or oversights that have not been detected. I hope that by making the code transparent I have made it easier to spot and correct any mistakes and that this has reduced the risks involved with adopting it.

The Example Program

The example program is a simple diagram builder/editor. You can create rounded rectangles and ellipses, you can move and size them graphically, you can join them together with lines fill them with text and select an individual font and text colour for each box.

It has been written specifically to demonstrate and test the use of Xonor pointers. In most cases I have tried to come up with plausible applications of the functionality of Xonor pointers.

There are two classic polymorphic subsystems:

• Visual Objects which is very concretely object orientated
• Mouse Modes which is a more abstract use of polymorphism but still totally practical

These demonstrate and test the use of new_ptr, owner_ptr, owner_ptr_array, ref_ptr, fast_ptr, enable_ref_ptr_to_this and its ref_ptr_to_this method
and the referencable and ref_to macros.

There is a modeless dialog which demonstrates and tests the use of:

• enable_self_delete and its self_delete method

There is a font pool which demonstrates and tests the use of:

• strong_ptr

It is a WTL application - you will have to install the WTL library - free from SourceForge.

Summary

It does seem to work quite reliably and most importantly it solves rather than hides the problem of the programmer forgetting to null secondary references by nulling them automatically. I have adopted it for everything new that I do and I have retrospectively implemented it in much of the library code that I have written. None of this has given me any problems so I feel I can recommend it to others with the argument that it puts C++ ahead of C# and Java in terms of both coding comfort and memory and pointer safety without compromising how C++ works.

I am not an expert on smart pointers. I just needed to have a memory protection system that I could work with that doesn't undermine the integrity of good C++ programming. If someone who is an expert can re-write this in a more elegant or efficient way or that more comprehensively assures or verifies its integrity I will be delighted.

Revision History

• 14^th May, 2008: First submitted
• 11^th June, 2008: Source code and article updated
• Changes to code:
• Abandoned trying to optimise away the pointer held in the reference controller resulting in more straightforward operation.
• owner_ptr given destructive copy constructor and assignment operator following the example of std::auto_ptr but explicit use in application code not recommended.
• Reference controller pool given mildly exponential growth rate.
• Code layout redesigned to make it more readable, transparent and verifiable
• Changes to article sections:
• XONOR pointers
• Building a complete integrated solution
• Collections of Xonor pointers
• Rules of use
• What can still bite you
• How it works
• Transparency of source code
• Using the source code
• 27^th June, 2008: Source code updated to resolve build issues and conformance with Visual Studio 2008