ClearType over Remote Desktop in Windows XP

• Download source - 92.18 KB

Introduction

A few of my favorite things are:

• ClearType font smoothing in Windows XP
• Working from home using Remote Desktop
• Not running Vista

Unfortunately, when using Remote Desktop Protocol (RDP) to connect to your Windows XP machine, font smoothing using ClearType is disabled. My guess is that back in 2001 when XP was released, Microsoft made the decision that ClearType over RDP would be prohibitively slow.

In 2007, however, multi-megabit broadband is much more common and the decision to not allow users to enable ClearType seems a tad bit dated (which Microsoft itself seems to realize... Windows Vista supports ClearType over RDP).

So, armed with both WinDBG and my assumption that patching a few choice bytes in the kernel would rectify this problem, I set off to enable ClearType over RDP in Windows XP.

Some Background

I wish I could describe the exact methodology I used to find the code path that does the RDP check but it was really just a lot of trial and error. Also, the fact that I wrote this code over a year ago means I don't even remember all the dirty details myself. Memories of fruitless days spelunking through disassembled kernel code are memories that are quickly repressed, it seems.

So, despite not really having any instructional value, I wanted to put this utility out there. It's made long days of Remote Desktop work a little easier on my eyes and I know the other ClearType aficionados I've shared it with are equally relieved that they finally have a solution to the problem.

Overview

There are two parts to the program:

1. A kernel driver that patches a few bytes in win32k.sys
2. A user-mode application that loads the driver and tells it to go ahead and do the patch

The kernel portion is easy. Basically, in win32k.sys, there's a code path that looks like this (courtesy of WinDBG's disassembler):

bf811387 66393550399abf  cmp     word ptr [win32k!gProtocolType (bf9a3950)],si
bf81138e 0f85c2feffff    jne     win32k!LFONTOBJ::ppfeMapFont+0x77 (bf811256)

I found in my debugging that the gProtocolType variable was different depending on whether you were at the console or accessing the machine via Remote Desktop. In a console session, the jne branch was not taken; over RDP it was.

The simplest change (and the one that produced the desired results) was to just skip that jne instruction. Replacing it with nops meant that the branch wouldn't be followed - the non-RDP code-path would always be the one taken.

The kernel portion of this program (RdpClearType.sys) simply does this nop patching.

However, there is one caveat: it appears that each logon session gets its own copy of win32k.sys mapped into its address space (in a region called "Session Space"). When I installed the driver and either set it to start on boot or started it manually using the Service Control Manager (SCM), it didn't work. My guess is that, in both cases, because the driver was being loaded by the SCM (which wasn't running in my logon session) the effects weren't visible from my session.

My solution was to create a user-mode loader application that broke the process down into two steps:

1. Load and start the driver through the normal SCM interfaces (CreateService and StartService). My driver's DriverEntry code would be called on one of the SCM's threads and execute in its logon session. This is why my DriverEntry function doesn't do the actual patching...it simply creates a "device" that is visible to user-mode called \DosDevices\RdpClearType.
2. After the driver has been loaded and started, the user-mode code opens the \DosDevices\RdpClearType device and calls DeviceIoControl. While servicing this call, the driver's code is executing on my thread running in my logon session. Doing the patching here affects win32k.sys in my Session Space.
3. Finally, a call to the Windows API function ValidateRect causes the entire screen to refresh. All the fonts are now rendered using ClearType.

Just to keep things easy, I've embedded the RdpClearType.sys driver inside of RdpClearType.exe as a resource. When you run RdpClearType.exe, the SYS file is extracted to your temp directory and loaded from there.

Usage

It couldn't be easier...just run RdpClearType.exe on the machine you're connecting to. You can even safely run it when you're Remote Desktop'ed in (which is extra-fun because you get to see the transformation happen right before your eyes!).

Neither the patch nor the driver persists across reboots. You'll need to run RdpClearType.exe each time you reboot.

Compatibility

I've been using this for about a year on Windows XP Professional SP2. I'm sure a Service Pack will invalidate the hard-coded address I'm patching and I don't do any sort of memory scanning or symbol lookup to find the location. There's just a simple check to make sure the existing bytes are what I expect them to be.

If a Service Pack or an update breaks it, I'll do what I can to update the code in a timely manner. Of course, if you have the Windows Driver Development Kit (DDK), Visual Studio, WinDBG and a lot of patience, you can find the new address yourself and rebuild the program. (Fun!)

History

• 2007-10-13: Initial release
• 2007-10-22: Added another possible patch address. The patch will now work with win32k.sys versions 5.1.2600.3099 and 5.1.2600.2770. A big thanks to Rob deMontarnal for his help in debugging the compatibility issue in the initial release!
• 2007-11-06: Added patch addresses for win32k.sys versions 5.1.2600.3180 and 5.1.2600.3115
• 2007-11-09: Added patch address for win32k.sys version 5.1.2600.2180