Windows subclassing and hooking with C++ classes

• Download source files - 9.47 Kb
• Download demo project - 107 Kb

Introduction

The first time I saw the CWnd class documentation I found it great. A very good wrapper for W32 API into the C++ context. But as time goes on, certain choice MFC developers did in the past becomes no more suitable for what the enhancement of the language had been. In particular, there is no possibility to subclass a Windows “window” more than once by a CWnd object. The problem has been solved in a variety of ways (I remember Paul Di Lascia, from Microsoft). I propose this solution, based essentially on STL collections and a class designed to capture a window procedure and doing its own dispatching.

All you have to do is derive from CWndSubclasser class and override the SubWndProc function. You can instantiate on the heap as many instances you want and associate each instance to a window. Various subclassers can be associated to a same window. Windows are identified by their HWND, hence, they are not required to be CWnd windows.

Furthermore, with the same technique used for window subclassing I also created classes to capture the WH_CALLWNDPROC (they may be useful if you have to capture a particular message independently of the window it is directed) and the WH_FOREGROUNDIDLE (useful to manage idle time processing for example in a library, and you cannot necessarily have access to the CWinApp object) hooks.

These classes are independent. You don't need to hook to subclass or to subclass to hook.

Class derivation and subclassing

The two concepts must not be confused.

Class derivation happens in OOP languages, and - essentially - consist in a definition of a class based on other base classes, where certain functions (probably virtual) are replaced. This happens inside the definition of a class.

Windows subclassing happens when a window procedure is replaced by another that may (or not, or may sometime) call the original one. This happens outside and independently of the definition of the "class" (or ... what defines the original window procedure). In this sense, class derivation is "static" (done by the compiler), while window subclassing is "dynamic" (done at runtime).

In MFC all windows are based on the same window procedure (AfxWndProc) that, once detected the window (HWND) a message is referred, dispatch that message to a virtual function of a CWnd object (OnWndMsg) that parses the associated message map and calls the required handler (if any) or calls Default, that – in turns – pass the original message to DefWndProc or to an original window procedure eventually existing if the window was not created by MFC.

When doing this, MFC is itself doing a “subclassing”, but it stores the original procedure in a single member variable. Hence only one (or none, if you get the CWnd frown an existing non-MFC window) subclassing is possible.

The idea of “subclassing” is essentially the same that comes with Win32: you replace a window procedure with another and – while processing messages in the new procedure – decide when and how to call the previous one (the default behavior).

The need of subclassing happens when you have to make a particular task over a particular message for a variety of different windows.

Each different window may be – itself – a CWnd derived object, but if you have to trap some messages (for example to customize menu behaviour or appearance in a same way for all your windows) you have to re-implement the same handlers for all the CWnd classes. That’s where subclassing may be useful: You create another object that intercepts the window procedures, and associate an instance for each of the window.

This object defines what to do with the messages and calls the original window procedure when needed.

In this implementation, however, I didn’t want to use one window procedure for each subclass, but rely on a virtual function of a specific object. So I provide a global internal window procedure that replaces the old one (if it is AfxWndProc , it means we are subclassing an MFC window … without MFC knowing that) and dispatch the messages iterating with a recursion (I’ll be clearer later) through an HWND associated list of “subclassers”.

In fact, “subclassers” are stored in reverse order on an std::lst and list are stored in std::map associated to HWND . The very first time a CWndSubclasser object is associated to a window, the window is subclassed (in W32 sense). All subsequent CWndSubclassers eventually associated to that window, don’t subclass it again, but simply chain into the list associated to the window.

When a message comes to the window procedure, it identifies the list and calls a virtual function (WndProcSub) on the first object in the list (the last associated to that HWND).

Its up to you to call – in your processing – the Default() member function that recursively calls that virtual function on the next object (or the previous window procedure, if the list is ended). Thus, wherever you place the Default() call (at the beginning or at the and of your override) you – in fact – affect the order of processing. Exactly like calling DefWndProc in Win32.

Hooking

To “hook”, more simply, I just provide a static function that dispatch to an internal list of object. Such objects are derived from CHookIdle or from CWndProcHook as needed.

History and dependency

During the deployment of the subclasser, I found that a problem arise with subclasser destruction.

In particular, DefWndProc (that is mostly used by every Windows window) often process messages generating other messages. This leads to the window procedure (whatever it is) to be called recursively. This means that we cannot destroy a subclasser – for example – on the WM_NCDESTROY message, because its virtual function may be still invocated (with data pushed on the stack) from a previous (but not yet returned) WM_SYSCOMMAND (the click on the “close” button on the caption bar).

So, the following rule must be applied:

1. Always construct subclassers object on the heap, and don’t associate their deletion to a CWnd destruction.
2. Delete the object only after verifying that no more recursions are in progress.

A very simple way to do this is using smart pointers (uh-oh ... the header included here is more recent than the one posted in that article ... it will be better to post an update!): in fact, in the provided window procedure (is in WndSubclasser.cpp), every time a subclasser needs to be called, a smart pointer is defined on the stack and initialized to the subclasser instance.

If you also, after creating the instance on the heap, refer it with a GE_::Safe::PtrStrong, you can be sure that the subclasser will never be destroyed until “something” (you or the window procedure) still needs it.

When you don’t need your subclasser anymore, just set your referring smart pointer to NULL (or destroy the smart pointer). The object will be deleted only after all recursions have returned (because every recursion destroys it’s own originated smart pointer on return).

Now, being these object designed to be handled by smart pointers, I do it in the safest way, by deriving the subclasser base from Safe::ObjStrong. The provided sample does exactly what is described.

// ../utility/WndSubclasser.h"
    ...            
    class CWndSubclasser;
    typedef Safe::PtrStrong<CWndSubclasser> PWndSubclasser;
    ...
            
// Appwnd.h
...
class CAppWnd :
    public CFrameWnd
{
protected:
    PHookIdle1 _pHookIdle1;
    GE_::Utils::PWndSubclasser _pS1, _pS2;
public:
    CAppWnd(void);
    virtual ~CAppWnd(void);
    DECLARE_MESSAGE_MAP()
    afx_msg int OnCreate(LPCREATESTRUCT lpCreateStruct);
    afx_msg void OnPaint();
};
...


//AppWnd.cpp
...
int CAppWnd::OnCreate(LPCREATESTRUCT lpCreateStruct)
{
    if (CFrameWnd::OnCreate(lpCreateStruct) == -1)
        return -1;
    
    CApp::OutConsole("Creating CApWnd: not yet subclassed\n");

    _pS1 = new CSub1;
    _pS1->Subclass(*this);
    _pS2 = new CSub2;
    _pS2->Subclass(*this);
    _pHookIdle1.New();

    return 0;
}
...

Using the code

I didn't specialize the smart pointers for these objects, thus, they assume by default that "dynamic_cast" is possible. For this reason, you must enable RTTI on your project.

If you don't want (or you cannot), typedef the smart pointers to your derived classes specifying the second template parameter to be GE_::Safe::FStaticCast (the defaut is FDynamicCast) and remove all the references to the debug class GE_::Mfc::STrace. (it's just for debugging). I - however - suggest to let RTTI enabled.

Sample code

The provided sample shows how a CFrameWnd (it does nothing but painting a piece of static text) is subclassed twice, using two objects derived from CWndSubclasser. Also, an idle processor and a hook are instantiated. You can easily redo the same things any number of time. In the subclassers, trapped messages display rows of text in an associated console.

Also, the “About” dialog is subclassed with another instance of the same object used to subclass the mainframe.

Note the way it is colored and the behavior of the cursor, and note how the code that does that has been written only once onto a specific object. (I didn’t recolor the static text control and handle the cursor shape over the button just to make the difference evident)

Description

The key point to understand is in the CWndSubclasser Subclasss function and the Destroy function.

bool CWndSubclasser::Subclass(HWND hWnd)
{
    if(!hWnd || _hWnd) return false;
    _hWnd = hWnd;
    _bCleared = false;
    TLstpWndSubClasser& lst = g_map[hWnd];
    lst.push_front(this);

    WNDPROC& oldPrc = g_prcmap[hWnd];
    if(!oldPrc) //not yet subclassed
    {
        oldPrc = (WNDPROC) GetWindowLongPtr(hWnd, GWL_WNDPROC);
        SetWindowLongPtr(hWnd, GWL_WNDPROC, (LONG_PTR)SubWindowProc);
    }
    
    return true;
}

bool CWndSubclasser::Clear()
{
    if(!_hWnd) return false;
    _bCleared = true;

    TLstpWndSubClasser& lst = g_map[_hWnd];
    lst.remove(this);
    
    if(lst.size() == 0)
    {
        g_map.erase(_hWnd);
        SetWindowLongPtr(_hWnd, GWL_WNDPROC, (LONG_PTR)g_prcmap[_hWnd]);
        g_prcmap.erase(_hWnd);
    }

    return true;
}

The Clear function it is called also in the destructor (but you can call it an unlimited number of times) and the Subclass function must be called by you on an existent window.

The g_xxx variables are std::maps defined as globals in an unnamed namespace.

In this implementation Subclass means push the subclasser object into a HWND associated dedicated list, and set the HWND window procedure to the one provided in WndSubclasser.cpp.

The window procedure is defined in these terms:

    struct 
    SWndSubclasserParams {
        TLstpWndSubClasser::iterator
        i; TLstpWndSubClasser::iterator
        end; HWND
        hWnd; UINT uMsg; WPARAM wParam; LPARAM lParam; LPVOID
        prevtoken; };
    LRESULT

    CALLBACK SubWindowProc( HWND
    hwnd, //      handle to window UINT
    uMsg, //      message identifier WPARAM
    wParam, //  first message parameter LPARAM
    lParam //   second message parameter )
    {
        TMapHwndLstSubClasser::iterator
        mapI = g_map.find(hwnd); ASSERT(mapI
        != g_map.end()); 
        static CCriticalSection ccs; 
        CSingleLock    lock(&ccs, true);
        SWndSubclasserParams prms;
        TLstpWndSubClasser& lst = mapI->second;
        prms.i = lst.begin();
        prms.end = lst.end();
        prms.hWnd = hwnd; prms.uMsg = uMsg; 
        prms.wParam = wParam; prms.lParam = lParam;
        prms.prevtoken = NULL;

        return CWndSubclasser::Call(&prms);
    }

Note: all this code is into an unnamed namespace. You will never use it directly.

Basically, a SWndSubclasserParams internal structure is allocated on the stack and filled in.

It is then passed as an address in the token parameter of the static Call function, that does the real job.

LRESULT CWndSubclasser::Call(LPVOID token)
{
    SWndSubclasserParams* pPrms = (SWndSubclasserParams*)token;
    
    if(pPrms->i == pPrms->end) 
        //finished the list: just call the original window procedure. 
        //This will go into AfxWndProc and hence in 
        //CWnd::OnWndMsg and into the message map.
        return CallWindowProc(
           g_prcmap[pPrms->hWnd], 
           pPrms->hWnd, 
           pPrms->uMsg, 
           pPrms->wParam, 
           pPrms->lParam
        );
    
    CWndSubclasser* pWS = *pPrms->i;
    PWndSubclasser pKeep(pWS);
    pPrms->i++; // the nextime Call is called, 
                   // will process the next in list
    pPrms->prevtoken = pWS->_token; //save for later
    pWS->_token = token; //remember for Defauklt calling
    LRESULT r=0;
    if(pPrms->uMsg == WM_NCDESTROY)
        pWS->_bCleared = true; //will not destroy yet
    
    if(pWS->_bCleared) r = pWS->Default(); //skip a declared 
                                                 // as "destroyed"
    else r = pWS->WndProcSub(
        pPrms->uMsg, 
        pPrms->wParam, 
        pPrms->lParam); //just call the subproc
    
    pWS->_token = pPrms->prevtoken; //restore previous token 
                                 //(now can return also from recursion)
    return r;
}

The iterator in CWndSubclasser is intended to point to the “next to process” subclasser.

If we are at the end of the list, we just call the ex window procedure.

Otherwise

1. We retrieve the pointed object
2. We increment the reference counting (PWndSubclasser pKeep(pWs) will live until return)
3. We increment the iterator (for future recursions)
4. We put the “token” into the object we’re just calling, after saving its old value.
5. We call (apart some particular cases) the WndProcSub virtual function.

It is expected that, depending on your needs, WndProcSub calls Default that, in turn, calls Call again (but the iterator has been incremented, hence the next subclasser will be referred).

This trick allows you to define as many subclassers you want: all you need to do is override the WndProcSub and, depending on the message:

• Call Default() and then do some processing (the subclasser acts as an add-on to the “default” action taken by previous subclasser and default window procedure)
• Do your process and don’t call Default (you replace the functionality with yours)
• Call Default() after your process.

Other objects

With the same identical technique, I also defined CHookIdle and CWndProcHook.

The difference is, they are general Windows hooks, and are not associated to a particular window.

CHookIdle chains in a list and a static hook procedure installed on WH_FOREGROUNDIDLE, calls the OnIdle virtual function. (It’s up to you in your override to call Default(). Note: the base, just calls Default)

CWndProcHook chains in another list, and a static hook procedure installed on WH_CALLWNDPROC calls OnHook.

CWndSubclasser and CWndProcHook

There is certain overlap in the functionality of the two objects, but they are not the same.

I think it is important to note the differences:

• CWndSubclasser when instantiated does nothing until it is associated to a specific HWND. From then on, it responds only for the HWND it has been associated. CWndProcHook respond when Windows is about to call a window procedure. It does not refer to a particular window, but can “spy” everything.
• CWndSubclasser relies on an “alternative” window procedure. The original one is called by Default(). It’s up to you to decide “if” and “when” (or “where”) to call it. CWndProcHook relies on a Windows hook. It can take actions, but it is not a real override of a window procedure. The window procedure is always called. Default() is only to define "if" and "when" to process different instances of CWndProcHook you may have created and hence, chained.

Other stuff

I included in the package also SmartPtr.h (they are used in the classes) and also another very simple class: GE_::Mfc::STrace. It can be used on RTTI enabled projects to do tracing into functions, tacking the recursion level.

Its usage is simple: declare a variable on the stack, and call the Trace function. The typical case is: GE_::Mfc::STrace trc; trc.Trace(typeid(*this),this,"<<yourtext>>"); and in the output window of the debugger you will see:

• a number of dots as the number of Trace recursion in progress
• the runtime type name of the object
• the address of the object
• the text you provide

It is used in debug versions of the objects I just described in this article.