Subclassing 101

Introduction

Note: All of the code in this article is for use with the PowerBasic compiler (see http://powerbasic.com). It should be easy to convert it to other languages though.

Every window has its own class (i.e., edit, static, etc.). A class is a specific type of window with specific features. The edit class allows typing in text. The button class actually is multiple controls, such as a button, checkbox, optionbutton or frame.

Dialogs are a unique window class predefined by windows. Controls are also predefined by windows. You can create your own custom window class as well.

Every window class at some point must be registered with windows. When a class is registered, one parameter passed is the address to the window procedure for the class. The predefined window classes all have their own window procedure which exist within the operating system DLLs. When your applications must send messages to a predefined window class, windows will send it to the window procedure in the operating system DLL which contains that class.

Every window procedure looks like this:

FUNCTION WindowProc(BYVAL hWnd AS LONG, BYVAL Msg AS LONG, BYVAL wParam AS
LONG, BYVAL lParam AS LONG) AS LONG
    SELECT CASE Msg
        CASE %WM_CREATE
        CASE %WM_DESTROY
        CASE ELSE
    END SELECT
FUNCTION=DefWindowProc(hWnd,Msg,wParam,lParam)
END FUNCTION

By default, window messages are processed by the DefWindowProc function. This function processes all messages in a default way. The window procedure may also include processing of any message to customize how the class will look and act. Also the window procedure will likely have custom messages (WM_USER + SomeNumber) that handle custom features of the window class. For example, the edit control has custom messages which start with the prefix EM_.

When you, the programmer, creates (registers) your own window class, you have access to the window procedure for that class in your application. The problem with the predefined window classes (i.e., controls, dialogs), is that you don't have direct access to the window procedure for those classes. The code for their window procedures exists in the operating system DLLs and not in your application.

This is where subclassing comes in !

Every window has the ability to have its internal window procedure address to be changed. When a window (i.e. control) is created, it stores the address to the original window procedure for the class in its data structure (instance data). Windows allows you to change the address to another address so the window uses a different window procedure. The problem with this is that if you change the window procedure address from the original, you lose all the original's features.

Subclassing is a way of rerouting messages to a window procedure.

When you use the GetWindowLong function with the GWL_WNDPROC flag, you get the original window procedure address for the original window class the window is based on. Now you have the original window procedure address.

By using the SetWindowLong function, you can change the window procedure address for the specific window to another one, in this case a custom one written by you. SetWindowLong also returns the previous value so you don't have to use GetWindowLong to get the original value.

Lastly, windows has an interesting function called CallWindowProc which allows you to send messages to the original window procedure by passing its address.

Now, let's subclass a window:

FUNCTION SubClassWindow(BYVAL hWnd&, BYVAL NewAddress&) AS LONG
    FUNCTION=SetWindowLong(hWnd&, %GWL_WNDPROC, NewAddress&)
END FUNCTION

GLOBAL gOrigAddress&

SUB StartSubClass(BYVAL hWnd&)
    gOrigAddress&=SubClassWindow(hWnd&, CODEPTR(MyWindowProc))
END SUB

FUNCTION MyWindowProc(BYVAL hWnd AS LONG, BYVAL Msg AS LONG, BYVAL wParam AS
LONG, BYVAL lParam AS LONG) AS LONG
    SELECT CASE Msg
        ' define all custom message processing below
        CASE %SOME_CUSTOM_MESSAGE_OF_MYOWN
        CASE %WM_CREATE
        CASE %WM_DESTROY
        CASE ELSE
    END SELECT
    FUNCTION=CallWindowProc(gOrigAddress&,hWnd,Msg,wParam,lParam)
END FUNCTION

By saving the original window procedure address for the windows original class, you can now send all unprocessed messages in your custom subclass window procedure to the original window procedure so the window has the attributes of the original class. You can also preprocess any messages before the original window does so as to add new features.

It should be pointed out that this is only a very simple explanation of subclassing. There are a number of factors that should be taken into consideration when actually writing your own subclassing code. One important one is to restore the original window procedure address in the WM_DESTROY message like this:

CASE %WM_DESTROY
    SetWindowLong hWnd&, %GWL_WNDPROC, gOrigAddress&

This is important because a window class can be subclassed multiple times (yes this does work), even by external programs (i.e. skinning utilities), so it is important to always restore the original window procedure address when the control is about to be destroyed.

Another consideration is what messages your subclass window procedure will get and how to process them. A subclass window procedure will never get the WM_CREATE message or WM_NCCREATE message, so don't put any code there. This is because the window class first has to be created before it can be subclassed, so these messages have already occurred and won't occur again (there is a way around this limitation by using a technique called superclassing, which is different than subclassing).

It is also important to sometimes process a message by a call to the original window procedure before your code does something. This is easily done by doing the following:

CASE %WM_NCHITTEST
        ReturnValue& = allWindowProc(gOrigAddress&,hWnd,Msg,wParam,lParam)
        ' you can do something now and even change the value 
        ' ReturnValue& to change how Windows
        ' responds to this message
        FUNCTION=ReturnValue&
        EXIT FUNCTION

Notice that the return value of CallWindowProc is stored and returned (plus you must exit the window procedure function now). The return value of a message is important in Windows and has meaning to the operating system.

It should also be pointed out that when subclassing (it is slightly different when superclassing), you don't have control of the instance data storage of the window class (known as window bytes of the class defined in the cbWndExtra member of the WNDCLASSEX structure passed in the RegisterClassEx function when the window class was registered by the operating system). If you only subclass one window, then you can use Global variables to store instance data (meaning data unique to a single instance of that window type). If you subclass multiple controls of the same window class, you may want to actually use some kind of instance based data storage, such as window properties (not the same as properties in an object oriented programming language) using the SetProp and GetProp API functions. While window properties are not as fast (speed of execution) as window bytes, they are plenty fast enough for most instances.

Lastly, be careful in how you process the messages in a subclass window procedure. Make sure you don't mess up the window classes existing message processing. It is best to read the Windows API documentation about any messages you plan to process to make sure you handle the message properly. In some cases, rather than modify a message, you can use the following technique which won't conflict with the normal timing of a window classes message processing.

CASE %WM_SIZE
   PostMessage hWnd, %Some_Custom_Message_Of_My_Own,0,0
CASE %Some_Custom_Message_Of_My_Own
    ' now do something you want to do when the window has been resized
    ' but now you know it has finished everything it needs to do associated with
    ' resizing including repainting

By posting a message, it will allow the current message to finish, plus most of any pending messages the current message may generate before your custom code (in a custom message) gets to be executed, before your custom message code gets executed.

If you add a number of custom messages for your subclass window procedure, it is usually best to define those messages using some kind of specific prefix. Custom messages are normally defined like this:

%SomeNewMessage    =    %WM_USER + 100

The problem is that some window classes my use this range for defining its own messages (beyond the standard WM_ standard messages). It may be better to define a custom message like this:

%SomeNewMessage    =    %WM_APP + 100

This will put the message in a range, unlikely touched by the original window class. Also it is good habit to define a prefix for your custom messages so they stand out from normal window messages like this:

%MYPREFIX_SomeNewMessage = %WM_APP + 100

Let's say I subclass an Edit class window to make my own masked edit control. I could create a custom message for passing a mask value to the window class like this:

%ME_SETMASK        =    %WM_APP+100

The %ME_ stands for Masked Edit. Windows itself (the SDK) uses 2 to 3 character prefixes plus an underscore character for defining most message prefixes. I find it best to follow this standard. It produces very readable code.

The best way to learn how to write your own subclass procedures is to simply experiment with it is a test program, rather than an actual application. Fine tune your code first and then move the code to an applications source code. Be prepared for a few GPFs when you make mistakes. While it may not happen often, it can happen, especially if you mess up the timing of a class in its own message processing. No harm done though. Windows will just terminate the program. I do find it best though to reboot the computer when it GPF's because of your subclassing code errors, since you can introduce instabilities into Windows in rare instances. Don't be afraid to experiment with subclassing though despite this. Even some simple subclassing procedures can add interesting effects on the original window class, just by modifying the return value of CallWindowProc.

Now you are ready to start experimenting with subclassing to create new features in the existing window controls.

I should point out that some programming languages are better suited to writing this kind of code. The techniques above probably remind you of the "old days" when people wrote Windows applications in languages like C, rather than C++ or the new .NET languages. The advent of object oriented programming languages also shields programmers from even dealing with things like window procedures and message processing. If you want the ease of coding in BASIC, the power of C, fast executables and all the constructs necessary to write this kind of code, I would strongly suggest you use the Powerbasic compiler. You can compile your code to a DLL which can then be used in most other programming languages such as C or Visual Basic.

You will be amazed at how much you can extend Windows using tricks like subclassing.

History

• 27^th June, 2010: Initial post