Click here to Skip to main content
Click here to Skip to main content

ClearType over Remote Desktop in Windows XP

, 6 Nov 2007
Rate this:
Please Sign up or sign in to vote.
A kernel patch that will enable ClearType over RDP in Windows XP SP2

Introduction

A few of my favorite things are:

  • ClearType font smoothing in Windows XP
  • Working from home using Remote Desktop
  • Not running Vista

Unfortunately, when using Remote Desktop Protocol (RDP) to connect to your Windows XP machine, font smoothing using ClearType is disabled. My guess is that back in 2001 when XP was released, Microsoft made the decision that ClearType over RDP would be prohibitively slow.

In 2007, however, multi-megabit broadband is much more common and the decision to not allow users to enable ClearType seems a tad bit dated (which Microsoft itself seems to realize... Windows Vista supports ClearType over RDP).

So, armed with both WinDBG and my assumption that patching a few choice bytes in the kernel would rectify this problem, I set off to enable ClearType over RDP in Windows XP.

Some Background

I wish I could describe the exact methodology I used to find the code path that does the RDP check but it was really just a lot of trial and error. Also, the fact that I wrote this code over a year ago means I don't even remember all the dirty details myself. Memories of fruitless days spelunking through disassembled kernel code are memories that are quickly repressed, it seems.

So, despite not really having any instructional value, I wanted to put this utility out there. It's made long days of Remote Desktop work a little easier on my eyes and I know the other ClearType aficionados I've shared it with are equally relieved that they finally have a solution to the problem.

Overview

There are two parts to the program:

  1. A kernel driver that patches a few bytes in win32k.sys
  2. A user-mode application that loads the driver and tells it to go ahead and do the patch

The kernel portion is easy. Basically, in win32k.sys, there's a code path that looks like this (courtesy of WinDBG's disassembler):

bf811387 66393550399abf  cmp     word ptr [win32k!gProtocolType (bf9a3950)],si
bf81138e 0f85c2feffff    jne     win32k!LFONTOBJ::ppfeMapFont+0x77 (bf811256)

I found in my debugging that the gProtocolType variable was different depending on whether you were at the console or accessing the machine via Remote Desktop. In a console session, the jne branch was not taken; over RDP it was.

The simplest change (and the one that produced the desired results) was to just skip that jne instruction. Replacing it with nops meant that the branch wouldn't be followed - the non-RDP code-path would always be the one taken.

The kernel portion of this program (RdpClearType.sys) simply does this nop patching.

However, there is one caveat: it appears that each logon session gets its own copy of win32k.sys mapped into its address space (in a region called "Session Space"). When I installed the driver and either set it to start on boot or started it manually using the Service Control Manager (SCM), it didn't work. My guess is that, in both cases, because the driver was being loaded by the SCM (which wasn't running in my logon session) the effects weren't visible from my session.

My solution was to create a user-mode loader application that broke the process down into two steps:

  1. Load and start the driver through the normal SCM interfaces (CreateService and StartService). My driver's DriverEntry code would be called on one of the SCM's threads and execute in its logon session. This is why my DriverEntry function doesn't do the actual patching...it simply creates a "device" that is visible to user-mode called \DosDevices\RdpClearType.
  2. After the driver has been loaded and started, the user-mode code opens the \DosDevices\RdpClearType device and calls DeviceIoControl. While servicing this call, the driver's code is executing on my thread running in my logon session. Doing the patching here affects win32k.sys in my Session Space.
  3. Finally, a call to the Windows API function ValidateRect causes the entire screen to refresh. All the fonts are now rendered using ClearType.

Just to keep things easy, I've embedded the RdpClearType.sys driver inside of RdpClearType.exe as a resource. When you run RdpClearType.exe, the SYS file is extracted to your temp directory and loaded from there.

Usage

It couldn't be easier...just run RdpClearType.exe on the machine you're connecting to. You can even safely run it when you're Remote Desktop'ed in (which is extra-fun because you get to see the transformation happen right before your eyes!).

Neither the patch nor the driver persists across reboots. You'll need to run RdpClearType.exe each time you reboot.

Compatibility

I've been using this for about a year on Windows XP Professional SP2. I'm sure a Service Pack will invalidate the hard-coded address I'm patching and I don't do any sort of memory scanning or symbol lookup to find the location. There's just a simple check to make sure the existing bytes are what I expect them to be.

If a Service Pack or an update breaks it, I'll do what I can to update the code in a timely manner. Of course, if you have the Windows Driver Development Kit (DDK), Visual Studio, WinDBG and a lot of patience, you can find the new address yourself and rebuild the program. (Fun!)

History

  • 2007-10-13: Initial release
  • 2007-10-22: Added another possible patch address. The patch will now work with win32k.sys versions 5.1.2600.3099 and 5.1.2600.2770. A big thanks to Rob deMontarnal for his help in debugging the compatibility issue in the initial release!
  • 2007-11-06: Added patch addresses for win32k.sys versions 5.1.2600.3180 and 5.1.2600.3115
  • 2007-11-09: Added patch address for win32k.sys version 5.1.2600.2180

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Dan Farino
Web Developer
United States United States
No Biography provided

Comments and Discussions

 
GeneralRe: XP SP3 support RDP ClearType natively PinmemberJasonBunting30-May-08 8:04 
GeneralRe: XP SP3 support RDP ClearType natively PinmemberRandy8711-Aug-08 5:58 
GeneralRe: XP SP3 support RDP ClearType natively PinmemberMember 425585817-Oct-08 7:24 
GeneralRe: XP SP3 support RDP ClearType natively Pinmemberbgs26422-Dec-08 0:19 
GeneralRe: XP SP3 support RDP ClearType natively Pinmembertechnowannabe5-Jan-09 7:22 
GeneralRe: XP SP3 support RDP ClearType natively PinmemberR Cheung18-Oct-08 22:24 
QuestionDoes SP3 now enable clear type font smoothing? PinmemberPete121516-May-08 15:10 
AnswerRe: Does SP3 now enable clear type font smoothing? Pinmemberyomoma217-May-08 3:57 
GeneralRe: Does SP3 now enable clear type font smoothing? PinmemberPete121518-May-08 13:04 
GeneralRe: Does SP3 now enable clear type font smoothing? Pinmemberyomoma219-May-08 10:43 
GeneralNot working for xp sp3 5.1.2600.5512 Pinmembersendonline5-May-08 10:03 
GeneralRe: Not working for xp sp3 5.1.2600.5512 Pinmemberyomoma26-May-08 10:47 
GeneralRe: Not working for xp sp3 5.1.2600.5512 PinmemberJasonBunting9-May-08 9:08 
QuestionAny chance of an update? PinmemberDave Abrahams23-Apr-08 15:17 
AnswerRe: Any chance of an update? PinmemberDave Abrahams23-Apr-08 15:19 
GeneralDoesn't work with 5.1.2600.3335 PinmemberPete12159-Apr-08 14:48 
GeneralRe: Doesn't work with 5.1.2600.3335 [modified] PinmemberKhan_alpha10-Apr-08 3:31 
GeneralRe: Doesn't work with 5.1.2600.3335 PinmemberR Cheung10-Apr-08 7:57 
GeneralRe: Doesn't work with 5.1.2600.3335 Pinmemberchuccaville14-Apr-08 3:02 
GeneralRe: Doesn't work with 5.1.2600.3335 Pinmembervtowel14-Apr-08 20:39 
GeneralRe: Doesn't work with 5.1.2600.3335 PinmemberKhan_alpha15-Apr-08 8:51 
GeneralRe: Doesn't work with 5.1.2600.3335 PinmemberDan Farino16-Apr-08 14:59 
GeneralRe: Doesn't work with 5.1.2600.3335 PinmemberKhan_alpha17-Apr-08 3:44 
GeneralRe: Doesn't work with 5.1.2600.3335 PinmemberR Cheung17-Apr-08 20:18 
GeneralRe: Doesn't work with 5.1.2600.5508 Pinmemberyomoma220-Apr-08 15:23 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140827.1 | Last Updated 6 Nov 2007
Article Copyright 2007 by Dan Farino
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid