Selection Overlay DLL
5.00/5 (7 votes)
API which shows a Selection Overlay and notifies the caller when it's resizing and notifies the final rectangle.
Introduction
This is an API which shows a Selection Overlay and notifies the caller when it's resizing and notifies the final rectangle.
Background
I'm a programmer for the Royal Dutch Navies. A few weeks back a colleague asked me if I knew a way to show a Selection Overlay (such as a transparent square as Windows Explorer uses in its listview when you select a file). The source provided with this example is the result of the above need.
Using the code
To be able to use the code, you need to include SelectionOverlay.h and link to SelectionOverlay.lib or late load the calls. Besides this u'll need to include windows.h before u include the h.
In VB or C#, you can just use the wrappers. (For convenience purposes, I've left a compiled compressed version of the DLL in the VB Test directory.)
You fill the structure to call with with the necessary information, like this example taken from test.cpp, where filling of the Alpha Bitmap has been overriden (optional):
...
OurInstance = new CallBackTest();
...
CallBackTest::CallBackTest()
{
BorderWidth = 2;
CheckWidth = 1 + BorderWidth;
ColorBorder = CreateAlphaColor(RGB(0, 0, 200), 200);
ColorBackground = CreateAlphaColor(RGB(75, 128, 255), 50);
ColorBackground2 = CreateAlphaColor(RGB(100, 128, 255), 75);
ColorBackground3 = CreateAlphaColor(RGB(0, 64, 128), 63);
}
bool CallBackTest::OnFillAlpha(void* Tag, HBITMAP BitmapDIBHandle)
{
bool Result = false;
BITMAP Bitmap = {0};
GetObject(BitmapDIBHandle, sizeof(BITMAP), &Bitmap);
if ((Bitmap.bmWidthBytes * Bitmap.bmHeight) >= sizeof(long))
{
long* Pixels = (long*) Bitmap.bmBits;
if (Pixels)
{
Result = true;
long BorderRight = Bitmap.bmWidth - CheckWidth;
long BorderBottom = Bitmap.bmHeight - CheckWidth;
for (long Y = 0; Y < Bitmap.bmHeight; Y++)
{
for (long X = 0; X < Bitmap.bmWidth; X++)
{
long Index = ((Y * Bitmap.bmWidth) + X);
if ((X < BorderWidth) || (X > BorderRight) ||
(Y < BorderWidth) || (Y > BorderBottom))
{
Pixels[Index] = ColorBorder;
}
else if (((Y % 8) == 1) || ((X % 8) == 1))
{
Pixels[Index] = ColorBackground3;
}
else if ((X & 2 & Y) != 2)
{
Pixels[Index] = ColorBackground;
}
else
{
Pixels[Index] = ColorBackground2;
}
}
}
}
}
return Result;
}
bool CallBackTest::OnFillRGB(void* Tag, HDC hDC, int Width, int Height)
{
//no need to be implemented, but this way u can place a breakpoint
return false;
}
void CallBackTest::OnMove(void* Tag, RECT Overlay)
{
//no need to be implemented, but this way u can place a breakpoint
int c =2;
}
void CallBackTest::OnEnd(void* Tag, RECT Overlay)
{
//just a breakpoint moment here atm, but implementation required
int c =2;
}
//... Start of the code on a mouse down
SELECTIONOVERLAYDATA_CLASS Temp = { 0 };
//... Needed even if u implement FillAlpha, in case user fails we fill it
Temp.AlphaLevel = 50;
Temp.BorderAlphaLevel = 200;
Temp.BorderWidth = 1;
//... Needed even if u implement FillAlpha, in case user fails we fill it
Temp.OverlayColor = RGB(75, 128, 255);
Temp.BorderColor = RGB(0, 0, 200);
Temp.Parent = hWnd;
Temp.Destination = (ISelectionOverlay*)OurInstance;
switch (message)
{
case WM_LBUTTONDOWN:
Temp.ButtonLead = LBUTTON;
break;
case WM_MBUTTONDOWN:
Temp.ButtonLead = MBUTTON;
break;
case WM_RBUTTONDOWN:
Temp.ButtonLead = RBUTTON;
break;
case WM_XBUTTONDOWN:
switch ((wParam & 0x0000FFFF))
{
case MK_XBUTTON1:
Temp.ButtonLead = XBUTTON1;
break;
case MK_XBUTTON2:
Temp.ButtonLead = XBUTTON2;
break;
}
break;
}
SelectionOverlay(Temp); After this, the overlay will ask the caller if it wants to draw the bitmap itself (or on failure of drawing will ask the caller if it wants to draw the dc) then the overlay will notify the caller with Move data (created when the overlay has the need to resize) and at final the overlay will call the caller with End data (created when the mouse button goes up and the final rectangle is known).
I will skip the SelectionOverlay.h in this document. If you want to see how the six different structures look, you can read this in the h. The difference in the structures is present to supply the widest range I could.
I will explain each parameter present in the structures to explain their use:
Parent- The parent of the overlay window (notice it's our parent, with creation but the overlay is no WS_CHILD), Mandatory.OverlayColor- Color of the overlay (RGB), Mandatory.BorderColor- Color of the border of the overlay (RGB), Mandatory.BorderWidth- Width of the border in pixels, Mandatory.AlphaLevel- Alphalevel to blend the overlay with, Mandatory.BorderAlphaLevel- Alphalevel to blend the border of the overlay with, Mandatory.Tag- Value which can be passed on in our most basic structures, value will be returned on the notifications to the listening window.ButtonLead- Tells which button we monitor:LBUTTON,MBUTTON,RBUTTON,XBUTTON1, orXBUTTON2, if not suppliedLBUTTONwill be defaulted.Listener- The window addressed to listen to our notifications ofWM_ONMOVEandWM_ONEND, if not supplied Parent will be used instead.ClientRect- Supplies the screen rectangle the overlay must stay between (Used in the 3 RECT structures), Mandatory.Destination- Interface to tall back to (used in theCLASSstructure), Mandatory.Dummy- What it says, unused, used to align it with out base structureSELECTIONOVERLAYDATA(Used in the CB structures).Move- CallbackOnMove(Used in the CB structures).EndCall- CallbackOnEnd(Used in the CB structures), Mandatory.FillAlpha- CallbackOnFillAlpha(Used in the CB structures).FillRgb- CallbackOnFillRgb(Used in the CB structures).
Concluding:
The RECT structure with their corresponding calls, are ment for the cases u want to use other screen coords for your rectangle then the clientrect of the Parent. The CB structures with their corresponding calls supply the possibility to be called back on a standard call method. The
CLASS structures with their corresponding calls supply the possibility to be called back on a class interface (where
OnMove, OnFillAlpha and OnFillRgb has an implementation and OnEnd has not (is Pure))
Rules:
Not much actually, though the API demands that the supplied mouse button is in down state on calling. And each mandatory value need to be supplied.Points of Interest
Wrappers
I've supplied two wrappers, one for Microsoft Visual Basic 6.0 and one for Microsoft C# (v4.0). They both have support for normal and ClientRect and overriding the Filling functions.Why VB? Pretty convenient reason actually, VB tends to crash on events which are not generated by it's own threads. This aided in getting the call close to threadsafe. If VB would stay alive, likely I would supply a bigger purpose then C/C++ alone. As result as long as the trigger for calling the api is coming forth out of the same thread as the window comes which we call back to, we would come back in the correct thread. Which means that we would not have a reason to Invoke in C#, or any corresponding technique in an other language. Last of course only valid if the trigger and it's result are only used upon windows in the same thread. Thou be warned, the filling of a bitmap/dc comes forth out of it's own thread, though VB6 as the VB test example shows, can handle it in this way (thanx to cSuperclass)
For the window procedure hook in VB6 I used:
Fastest, safest subclasser, no module!
Size:
These days most things aren't about size anymore, though i prefer still to keep things as small as I could make them. To accomplish a smaller size i used a library coming out of an old MSDN(Mine version comes from MSDN October 2001 (The last MSDN with full information for Microsoft Visual Studio 6.0)) called LIBCTINY.LIB, it's technique is very well explained in it's document:
MSDN October 2001, Under The Hood: Reduce EXE and DLL Size with LIBCTINY.LIB. Further more i've compressed the release version of the DLL even more with help of a splendid compressor tool, called UPX
32Bit:
As I've no 64 bit machine nor has access to one, I've only designed this API to work with 32 bit. LIBCTINY itself, I expect not to work with 64 bit, but also calls asSetWindowLong should be replaced in use with a 64 bit machine, if anyone is willing to convert this
API to 64 bit, I would love to add it to this project.Late loading:
I wanted to narrow down the dependencies of this API, to get the widest support i could reach in 32 bit Windows. All calls to the kernel are linked, all calls to User32 and GDI32 are late loaded (making use ofloadlibrary and getprocadress). As result the
DLL has four DLLs it depends on. Kernel, NTDLL, User32 and GDI32 (also one of the results of using LIBCTINY).How it works::
Threads, keeping them alive and cross bridges:
Because I wanted the call to be asynchronous (and with that not blocking the caller after calling us), I needed a second thread. A thread where I could receive data from the low level mouse hook. Problem with a low level mouse hook is that you've only a certain amount of time to process, if u take to long, you'll be kicked out of the hook chain. To make sure i keep the overlay alive i needed to move all drawing to another thread.
So in short we've three threads: the caller, the mousehook, our overlay. The caller is kept alive on it's own, so we had no need to keep the thing alive, though our own 2 threads we did had the need for. I used a technique i hadn't used in ages, coming forth out of the beginning of programming windows in plain C.
To keep the thread alive i used
GetMessage. As most will see, I only check it's result on 0 and don't translate or dispatch. Why
I don't use it is simple, i don't want to process errors, just go on and pray
was my thought. I don't translate or dispatch cause my input is a mousehook and my own generated messages.
Now i still needed to make a bridge between the threads. The solution for this also came back to old times,
SendNotifyMessage, Which comes back with result from the call if it's in your own thread, but comes back without waiting for a result if it adds an message to a window procedure in another thread. In order to be able to
receive I needed windows to notify to.
I used the HWND_MESSAGE on parent to get message windows.
CreateWindowEx(WS_EX_NOPARENTNOTIFY, "STATIC", NULL, 0, -1, -1, 0, 0, HWND_MESSAGE, NULL, 0, 0);Which are light weighted in comparison with normal windows. I didn't needed to show them anyways.
Colors:
I wanted the user to be able to supply RGB and the alphalevel for that parts of the overlay, in order to allow that, i needed to convert the RGB to alphacolor. After reading Vorlath - Project V: Advanced Alpha BlendingI knew what the alphacolor actually was, and what i needed to do to create a source alpha. I took the 100% proven method which did not use dividing, LIBCTINY does not supply it, and with this solution I could keep using LIBCTINY.
The overlay:
The overlay is a plain old static window with these specifics: WS_EX_NOPARENTNOTIFY,
WS_EX_LAYERED, WS_EX_TRANSPARENT, WS_POPUP, and
WS_VISIBLE.
WS_EX_NOPARENTNOTIFY: we don't want our parent to get notifications coming forth from our window.WS_EX_LAYERED: We need this to be able to alphablend.WS_EX_TRANSPARENT: We don't want to obscure the windows we're above. With this flag we're assured all beneath us has done it's drawing when we receiveWM_PAINT.WS_POPUP: our window acts as popup.WS_VISIBLE: we want to be seen.
The alpha blending:
We create a 32 bit bitmap which we fill with our alphablended colors. If we cannot create a 32 bit bitmap, we fill the dc of the screen with 2 fillrect commands. After this
Updatelayeredwindow is called with correct flags. After being done all created will be destroyed again.
Caching:
To make sure i would get as quick as I could think off, I decided to cache as much as I could, and to use casting for data which aligned correctly, but actually request another structure;
(POINT*)&crPos, (SIZE*)&Info.bmiHeader.biWidth
are both examples of this.
In Short:
We create necessary colors and brushes. We create a thread to listen to the low level mouse hook. We create a thread to update a layered window, which also notifies the caller with Move and End information.
To keep our own threads alive we use a window and
GetMessage.
Beneath u'll see how the examples from the source (for C# and VB) use the dll, both override FillAlpha, done this so everyone can read how to coop with the pointer in each language, thou ofcourse the drawing of the dll is still present so u don't need to override the filling, it's optional.
C# Example:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using SelectionOverlayWrap;
namespace SelectionOverlayTest
{
/// <summary>
/// We want to get OnMove and OnEnd notifications, so implement both
/// </summary>
public partial class Form1: Form, ISelectionOverlayOnEnd, ISelectionOverlayOnMove, ISelectionOverlayOnFillAlpha, ISelectionOverlayOnFillRGB
{
[StructLayout(LayoutKind.Sequential)]
private struct BITMAP
{
internal int bmType;
internal int bmWidth;
internal int bmHeight;
internal int bmWidthBytes;
internal Int16 bmPlanes;
internal Int16 bmBitsPixel;
internal IntPtr bmBits;
}
[DllImport("gdi32.dll", EntryPoint = "GetObjectW")]
private static extern int GetObject(IntPtr hObject, int nCount, ref BITMAP lpObject);
private static int BorderWidth = 2;
private static int CheckWidth = 1 + BorderWidth;
private static int ColorBorder = SelectionOverlay.CreateAlphaColor(SelectionOverlay.RGB(0, 0, 200), 200);
private static int ColorBackground = SelectionOverlay.CreateAlphaColor(SelectionOverlay.RGB(75, 128, 255), 50);
private static int ColorBackground2 = SelectionOverlay.CreateAlphaColor(SelectionOverlay.RGB(100, 128, 255), 75);
private static int ColorBackground3 = SelectionOverlay.CreateAlphaColor(SelectionOverlay.RGB(0, 64, 128), 63);
/// <summary>
/// Will be called on OnMove notification
/// </summary>
/// <param name="Tag">ignored</param>
/// <param name="Rectangle">The square of the overlay</param>
public void OnMove(object Tag, Rectangle Rectangle)
{
button1.Text = Rectangle.Left.ToString() + ", " +
Rectangle.Top.ToString() + ", " +
Rectangle.Right.ToString() + ", " +
Rectangle.Bottom.ToString();
}
/// <summary>
/// Will be called on OnEnd notification
/// </summary>
/// <param name="Tag">ignored</param>
/// <param name="Rectangle">The square of the overlay</param>
public void OnEnd(object Tag, Rectangle Rectangle)
{
button2.Text = Rectangle.Left.ToString() + ", " +
Rectangle.Top.ToString() + ", " +
Rectangle.Right.ToString() + ", " +
Rectangle.Bottom.ToString();
}
public bool OnFillRGB(object Tag, IntPtr hDC, int Width, int Height)
{
return false;
}
public bool OnFillAlpha(object Tag, IntPtr BitmapDIBHandle)
{
bool Result = false;
if (BitmapDIBHandle != IntPtr.Zero)
{
BITMAP Retriever = new BITMAP();
GetObject(BitmapDIBHandle, Marshal.SizeOf(Retriever), ref Retriever);
int lSize = ((Retriever.bmWidthBytes * Retriever.bmHeight)/4);
if ((lSize >= 4) && (Retriever.bmBits != IntPtr.Zero))
{
int[] lPixels = new int[lSize];
try
{
Marshal.Copy(Retriever.bmBits, lPixels, 0, lSize);
Result = true;
}
catch { }
if (Result)
{
long BorderRight = Retriever.bmWidth - CheckWidth;
long BorderBottom = Retriever.bmHeight - CheckWidth;
for (long Y = 0; Y < Retriever.bmHeight; Y++)
{
for (long X = 0; X < Retriever.bmWidth; X++)
{
long Index = ((Y * Retriever.bmWidth) + X);
if ((X < BorderWidth) || (X > BorderRight) ||
(Y < BorderWidth) || (Y > BorderBottom))
{
lPixels[Index] = ColorBorder;
}
else if (((Y % 8) == 1) || ((X % 8) == 1))
{
lPixels[Index] = ColorBackground3;
}
else if ((X & 2 & Y) != 2)
{
lPixels[Index] = ColorBackground;
}
else
{
lPixels[Index] = ColorBackground2;
}
}
}
Result = false;
try
{
Marshal.Copy(lPixels, 0, Retriever.bmBits, lSize);
Result = true;
}
catch { }
}
}
}
return Result;
}
/// <summary>
/// Starts us
/// </summary>
public Form1()
{
InitializeComponent();
}
/// <summary>
/// On mousedown checks which button was fired and if it's an supported one
/// it opens a selection overlay for that button
/// </summary>
/// <param name="sender">Whisperer</param>
/// <param name="e">Mouse data</param>
private void Form1_MouseDown(object sender, MouseEventArgs e)
{
bool NoGo = false;
VKKeys ToUse = VKKeys.LBUTTON;
if ((e.Button & System.Windows.Forms.MouseButtons.Left) == System.Windows.Forms.MouseButtons.Left)
{
ToUse = VKKeys.LBUTTON;
}
else if ((e.Button & System.Windows.Forms.MouseButtons.Right) == System.Windows.Forms.MouseButtons.Right)
{
ToUse = VKKeys.RBUTTON;
}
else if ((e.Button & System.Windows.Forms.MouseButtons.Middle) == System.Windows.Forms.MouseButtons.Middle)
{
ToUse = VKKeys.MBUTTON;
}
else if ((e.Button & System.Windows.Forms.MouseButtons.XButton1) == System.Windows.Forms.MouseButtons.XButton1)
{
ToUse = VKKeys.XBUTTON1;
}
else if ((e.Button & System.Windows.Forms.MouseButtons.XButton2) == System.Windows.Forms.MouseButtons.XButton2)
{
ToUse = VKKeys.XBUTTON2;
}
else
{
NoGo = true;
}
if (!NoGo)
{
SelectionOverlay.Show(Color.FromArgb(50, 75, 128, 255),
Color.FromArgb(200, 0, 0, 200),
1,
this,
this.Handle,
ToUse);
}
}
}
} VB Example:
Option Explicit
'Only need to implement the functions u want to use (IOnEnd is mandatory though)
Implements IOnEnd
Implements IOnMove
Implements IOnFillAlpha
Implements IOnFillRGB
Dim blaat As Double
Dim Client As New VBRect
Private Type BITMAP
bmType As Long
bmWidth As Long
bmHeight As Long
bmWidthBytes As Long
bmPlanes As Integer
bmBitsPixel As Integer
bmBits As Long
End Type
Dim BorderWidth As Long
Dim CheckWidth As Long
Dim ColorBorder As Long
Dim ColorBackground As Long
Dim ColorBackground2 As Long
Dim ColorBackground3 As Long
Private Declare Function GetObject Lib "gdi32.dll" Alias "GetObjectA" (ByVal hObject As Long, ByVal nCount As Long, ByVal lpObject As Long) As Long
Private Declare Sub RtlMoveMemory Lib "kernel32.dll" (ByRef Destination As Any, ByRef Source As Any, ByVal Length As Long)
Private Sub Form_Load()
BorderWidth = 2
CheckWidth = 1 + BorderWidth
ColorBorder = CreateAlphaColor(RGB(0, 0, 200), 200)
ColorBackground = CreateAlphaColor(RGB(75, 128, 255), 50)
ColorBackground2 = CreateAlphaColor(RGB(100, 128, 255), 75)
ColorBackground3 = CreateAlphaColor(RGB(0, 64, 128), 63)
End Sub
Private Sub Form_MouseDown(Button As Integer, Shift As Integer, X As Single, Y As Single)
ShowOverlay Me.hWnd, 50, 200, 1, RGB(75, 128, 255), RGB(0, 0, 200), Me
' Client.Left = 25
' Client.Top = 25
' Client.Right = 400
' Client.Bottom = 300
' ShowOverlay Me.hWnd, 50, 200, 1, RGB(75, 128, 255), RGB(0, 0, 200), Me, , Client
End Sub
Private Sub IOnEnd_OnEnd(ByVal Tag As Object, ByVal Left As Long, ByVal Top As Long, ByVal Right As Long, ByVal Bottom As Long)
Print Left & ", " & Top & ", " & Right & ", " & Bottom
End Sub
Private Function IOnFillAlpha_OnFillAlpha(ByVal Tag As Object, ByVal hBitmapHandle As Long) As Boolean
Dim bResult As Boolean
Dim oBitmap As BITMAP
Dim lSize As Long
bResult = False
Call GetObject(hBitmapHandle, Len(oBitmap), VarPtr(oBitmap))
lSize = oBitmap.bmWidth * oBitmap.bmHeight
If ((lSize >= 4) And (oBitmap.bmBits <> 0)) Then
bResult = True
Dim lPixels() As Long
Dim BorderRight As Long
Dim BorderBottom As Long
Dim X As Long
Dim Y As Long
Dim LoopHeight As Long
Dim LoopWidth As Long
Dim Index As Long
LoopHeight = oBitmap.bmHeight - 1
LoopWidth = oBitmap.bmWidth - 1
BorderRight = oBitmap.bmWidth - CheckWidth
BorderBottom = oBitmap.bmHeight - CheckWidth
ReDim lPixels(lSize)
RtlMoveMemory lPixels(0), ByVal oBitmap.bmBits, lSize * 4
For Y = 0 To LoopHeight
For X = 0 To LoopWidth
Index = (Y * oBitmap.bmWidth) + X
If ((X < BorderWidth) Or (X > BorderRight) Or _
(Y < BorderWidth) Or (Y > BorderBottom)) Then
lPixels(Index) = ColorBorder
ElseIf (((Y Mod 8) = 1) Or ((X Mod 8) = 1)) Then
lPixels(Index) = ColorBackground3
ElseIf ((X And 2 And Y) <> 2) Then
lPixels(Index) = ColorBackground
Else
lPixels(Index) = ColorBackground2
End If
Next X, Y
RtlMoveMemory ByVal oBitmap.bmBits, lPixels(0), lSize * 4
End If
IOnFillAlpha_OnFillAlpha = bResult
End Function
Public Function IOnFillRGB_OnFillRGB(ByVal Tag As Object, ByVal hDC As Long, ByVal Width As Long, ByVal Height As Long) As Boolean
IOnFillRGB_OnFillRGB = False
End Function
Private Sub IOnMove_OnMove(ByVal Tag As Object, ByVal Left As Long, ByVal Top As Long, ByVal Right As Long, ByVal Bottom As Long)
Command1.Caption = Left & ", " & Top & ", " & Right & ", " & Bottom
End Sub
History
rev 9: Expanded the DLL with callback to overwrite filling of the bitmap/dc, updated the C# and VB wrapper, adjusted test.cpp from the C++ test.exe, adjusted this page. Added CreateAlphaColor to the DLL. Restyled VB Wrapper to have only 1 SelectionOverlay function, rect has become an optional.
rev 7: Bugfix in the cpp. _stdcall was missing on the two callbacks used in the class variant of the method. Somehow this bug slipped through, cause for a reason i don't understand, calling a _cdecl from out debug with an _stdcall prototype did not crash, while release ofcourse does. A well found, solved.
rev 5: Adjusted some grammar. And a missing hint about windows.h