Click here to Skip to main content
13,705,376 members
Click here to Skip to main content
Add your own
alternative version

Stats

24.9K views
522 downloads
62 bookmarked
Posted 28 Oct 2015
Licenced CPOL

HotPatching : (VERY) Deep Inside

, 28 Aug 2016
Rate this:
Please Sign up or sign in to vote.
A ready to be used hotpatching library with four methods!

Introduction

I have created a game that is so addictive that I cannot afford to let the user quit it in order to update it.
I am already facing several lawsuits regarding this issue and, unless you vote for me now, I won't be able to pay my lawyer.

Not that I care much of course. He is not interested in any money because he will not quit playing my game. But in order to satisfy all of them, I must implement HotPatching techniques in Windows.

And I also have a customer that demands hotpatching, but does not like DLLs. Hey, is hotpatching from yourself even possible? Well, I guess yes. 

Now with cleaned up library (one .h file!) and no administrator needs!

What is hotpatching?

It's the technique that you update a process without first quitting. To accomplish that, you patch the function calls at runtime. In the most common scenario, you have an executable and some dlls which contain functions.

Patching would occur if your executable would download another DLL from your server and load it. This DLL then would replace the old functions with the patched ones at runtime.
    
If you are still here, I will present even more crazy scenarios later on. Keep reading.

The four methods

There are four methods of hotpatching:

  • Loading a DLL to your address space that patches your EXE's functions. This is the safest method.
    • Advantages: Safest, Easiest. Work is done in single address space.
    • Disadvantages: You need an extra DLL.
  • Patching your EXE from an updated EXE via COM Interop and some proxy functions.
    • Advantages: You don't need an additional DLL 
    • Disadvantages: Work is done by proxying calls, different address spaces.
  • Patching your EXE from an updated EXE via Shared Memory and some proxy functions.
    • Advantages: You don't need an additional DLL 
    • Disadvantages: Work is done by proxying calls, different address spaces.
  • Patching your EXE by loading an updated EXE in the same address space (!!!) as your EXE.
    • Advantages: All work is done within a single EXE and within a single address space.
    • Disadvantages: Hackish (?) method.

I know you think I am crazy. No I am not - I just like to play with windows internals a lot.

The three projects

The Visual Studio 2015 solution included here contains three items:
    

  •     The HotPatching library in a form of hotpatch.h, along with my update USM and XML libraries.
  •     The patching DLL and,
  •     The testing executable which also serves as a self-preparing for hotpatching, self-patching COM component, self-patching USM component and self-patching EXE component.

The projects expose all four hot patching methods in both x86 and x64.

Preparing the Compiler and the Linker

 In order for a function to be eligible for hotpatching, three things must be valid:
        

  •    The function must not be inline.
  •    The size of the function must be at least two bytes.
  •    The function parameter list must not be optimized out by the compiler (example follows) and,
  •    There must be enough padding BEFORE the function entry to store the jump to the patch.

        
You can force VC++ compiler to never inline a function by using __declspec(noinline)

To accomplish the rest of the padding requirements, you need to compile with /HOTPATCH (only necessary in x86) and link with /FUNCTIONPADMIN:5 in x86 and /FUNCTIONPADMIN:12 in x64. I will explain later what these bytes are used for. In older Visual Studio editions the linker ignores the /FUNCTIONPADMIN parameter so it does not put the required padding before the function in x64 builds.

What to patch?

In order to patch, one must know where is the function to be patched. Some might say to export a set of patchable functions from the executable, so the patcher can search for them in the export table and find their addresses. This would work but a) you have to export all your patchable functions manually and b) what if you forget to export a function that needs patching?
    
My solution is to take advantage of the PDB debugging information and the DIA SDK in order to export ALL the functions that your application has, save their addresses and then include that information in the original executable as a resource. In order for this to work you need to compile with /Zi and link with /DEBUG, even in Release mode, so the PDB is generated.

Post-Build configuration

Once the Test Executable has been built, a post-build event is called. The HOTPATCH::AutoPrepareExecutable() moves self to another file, copies self back to old file, then calls HOTPATCH::PrepareExecutable() which:
        

  • Loads the DIA SDK and the PDB file
  • Loads the executable in order to be able to get the functions' virtual addresses. The addresses of the functions are then saved, along with the load address of the executable, to an XML string.

HOTPATCH::PrepareExecutable also accepts a list of compilands. If you pass an empty initializer list, all compilands are included. To include, say, only main.obj, pass {L"main.obj"}.
            
Then, PostBuildPatch calls HOTPATCH::PatchExecutable() which simply uses BeginUpdateResource(), UpdateResource() and EndUpdateResource() to save the XML string inside the Test Executable.
    
If you have more than one patchable modules, then you would repeat this process for all of them. For simplicity, my tool only loads an EXE file. You would of course start an executable which would load the module you want to process. 

x86 function patching

The patch is implemented by:

  •     Writing to the 5 bytes before the function's entry point (where the linker has created the space with /FUNCTIONMINPAD) a relative 32-bit JMP instruction which jumps to the new function's entry point.
  •     Replacing the first two bytes of the function's entry point with 0xF9 0xEB, this is the assembly for "JMP $ - 5" (that's why the function needs to have at least 2 bytes available), atomically.

        
The JMP address must be relative to the current EIP, so we subtract the new function's entry point from the old function's entry point - 5.

To find the old function's entry point we assume that it's distance from the module loading address will be always the same. When the Post-Build Patching tool saved the function RVAs, it also saved the current module loading address. Now that we might have a new module loading address, we can recalculate the function RVA with simple subtraction.

x64 function patching

The patch is implemented by: 

  • Writing to the 12 bytes before the function's entry point (where the linker has created the space with /FUNCTIONMINPAD) the following:
    • MOV RAX,XXXXXXXXXXXXXXXX , where XXXXXXXXXXXXXXXX is the absolute jump address of the new entry point.
    • JMP RAX
  • Replacing the first two bytes of the function's entry point with 0xFw 0xEB, this is the assembly for "JMP $ - 12", atomically.

Microsoft says that only 6 bytes are needed for x64 hotpatching, but I don't understand how. If you use a relative JMP like x86, this can only take a 32-bit parameter and, in case that the new entry is far away than the old entry (over 31 bits since the 32th bit is sign-extended) you can't jump. Therefore I 've decided to patch through JMP RAX which can jump to an unconditional 64-bit absolute address.

The RAX register is considered volatile and it is not used for function parameter passing. Therefore it's a safe bet to use it for hotpatching. If you want further safety, you can use the JMP [address] where you can store the address to jump in the memory, at the expense of more padding bytes needed. 

Visual Studio optimizations / behaviours

If a function in your program is only called once, there is a chance that it won't be patchable. Consider this example:    

// Actual prototype
void foo(int x);

If this function is only called once, say, foo(5), then the compiler could optimize the function and, instead of passing 5 to the stack or to a register, it hardcodes the value inside the function, therefore translating the actual function to this:

// Compiled prototype
void foo()
{
int x = 5;
}
 
This means that the patch function which you will provide  won't have the same signature as the patched one, it will search for the x parameter in the stack/registers and boom.
 
Another thing. When a proxy is used to call a function, enough stack must be present for the callee because it seems that VS requires some extra bytes of stack to do work.

Using the library from your executable

 Nothing to be done. Everything is done by the Patch DLL, so the executable can be patch-unaware. 
 You only need to load the DLL and call some exported function that patches your executable:   

HINSTANCE hL = LoadLibrary(L"..\\dllpatch\\dllpatch.dll");
HRESULT(__stdcall *patch)() = (HRESULT(__stdcall *)())GetProcAddress(hL,"Patch");
if (patch)
    patch();

Using the library from the Patch DLL

HOTPATCH hp;
hp.ApplyPatchFor(GetModuleHandle(0),L"FOO::PatchableFunction1",PatchableFunction1);

And that's all. You just specify the name of the function (the same name saved with the DIA SDK) and 
the replacement, which must have the same function signature (or boom).

Without the DLL

OK, now it's going to be a crazy bet. Is it even possible to patch yourself from your own executable? 
Let's say that you have an APP.EXE that downloads a new APP.EXE from your server. But you don't have a patch DLL and you are bored to write one. Could you patch yourself with yourself?
    
Of course, as you know, you cannot LoadLibrary an EXE file, for even if it is relocatable, the CRT is not the same. Even if you load the EXE and get a pointer with GetProcAddress, calling the function is likely to get you a crash since no proper initialization has occurred.

Actually, you CAN LoadLibrary an EXE and you CAN prepare it for execution in the very same address space of YOUR EXE, but this is the 4th method and it will be explained in another article. For now, let's stay in more conventional methods.
    
Therefore you have to work with interprocess mechanisms, i.e this crazy sequence of operations:
    

  • Register the downloaded app as a COM server
  • Query the COM server for the patching names
  • Patch the functions yourself

But because patching functions are not anymore in your own address space, you must patch with a "proxy" which will take the arguments from the current call, save them all, and pass it then to the COM server via Interop. But because parameters have also to be passed, this new "proxy" must be dynamic in memory (i.e. a new one for each patch) and not a static function in your code. Therefore, a some(?)-assembly-stuff has to be created dynamically (with VirtualAlloc() and VirtualProtect()), then construct a COMCALL object in that memory with placement new which is readable,writable and executable, and then this memory should save all registers (8 in x86, 16 in x64) in some unique per patch memory area, then save also the IDispatch* COM server interface, then pass it via IDispatch::Invoke() to the remote server.

Registering/Unregistering yourself as COM Server

This is easy, you call HOTPATCH::PrepareExecutableForCOMPatching with a temporary CLSID and PROGID and then  HOTPATCH::RegisterCOMServer and HOTPATCH::Unregister do the job by writing or deleting some keys under HKEY_CURRENT_USER. You don't need anymore admin because the hotpatching interface is not a new interface (therefore, it does not need registration) it is merely an IDispatch.

Registering the Patch Names

The COM Server implements an IClassFactory and an IDispatch which does the work with 2 additional member functions.
    
The program calls HOTPATCH::AckGetPatchNames to get the function names to be patched. This function internally calls with COM Interop the IHotPatch::GetNames of the COM Server. It returns a BSTR with space-separated function names that the COM server is ready to patch. Now that the library is implemented with plain IDispatch, there are no direct calls, but rather indirect calls to IDispatch::Invoke().

Patching a function by installing a Proxy

When HOTPATCH::ApplyCOMPatchFor is called, it places a patch to the function, replacing it, not with the target function (which resides in another address space now) but with a dynamically created buffer which is also executable:

#pragma pack(push,1)

struct COMCALL
    {
    unsigned char jmp1 = 0xE9;
    unsigned char jmp2 = 0x27;
    unsigned char jmp3 = 0x01;
    unsigned char jmp4 = 0x00;
    unsigned char jmp5 = 0x00; // JMP $ + 300

    void* HPPointer = 0;
    void* HPClass = 0;
#ifdef _WIN64
    char data[179];
#else
    char data[187];
#endif
    char name[100];

    // Push pop to stack
#ifdef _WIN64
    struct MOVER
        {
        unsigned char pushreg = 0;
        unsigned char poprax = 0x58;
        unsigned short movrax = 0xA348;
        unsigned long long addr = 0;
        };
#else
    struct MOVER
        {
        unsigned char pushreg = 0x66;
        unsigned char popeax = 0x58;
        unsigned char moveax = 0xA3;
        unsigned long addr = 0;
        };
#endif

    MOVER m1[8]; // base registers

#ifdef _WIN64
    struct MOVER2
        {
        unsigned char pushregp = 0x41;
        unsigned char pushreg = 0;
        unsigned char poprax = 0x58;
        unsigned short movrax = 0xA348;
        unsigned long long addr = 0;
        };

    MOVER2 m2[8]; // r8-r15 registers
#endif

    // Call the COMPatchGeneric
#ifdef _WIN64

#ifndef XCXCALL
    unsigned short movd1 = 0xB848;
    unsigned long long regaddr = 0;
    unsigned short movd2x = 0xA348;
    unsigned long long movd2 = 0;
#else
    unsigned short movrcx = 0xB948;
    unsigned long long regaddr = 0;
    unsigned short subrsp100_1 = 0x8148;
    unsigned short subrsp100_2 = 0x00EC;
    unsigned short subrsp100_3 = 0x0001;
    unsigned char subrsp100_4 = 0;
#endif // XCXCALL
    unsigned short movrax = 0xB848;
    unsigned long long calladdr = 0;
    unsigned short callrax = 0xD0FF;
#ifdef XCXCALL
    // Restore the stack
    unsigned short addrsp100_1 = 0x8148;
    unsigned short addrsp100_2 = 0x00C4;
    unsigned short addrsp100_3 = 0x0001;
    unsigned char addrsp100_4 = 0;
#endif
    unsigned char ret = 0xC3;
#else
    // Call the COMPatchGeneric
#ifndef XCXCALL
    unsigned short movd1 = 0x05C7;
    unsigned long movd2 = 0;
    unsigned long regaddr = 0;
#else
    unsigned char movecx = 0xB9;
    unsigned long regaddr = 0;
    // Give the callee some stack to work with
    unsigned short subesp100_1 = 0xEC81;
    unsigned short subesp100_2 = 0x0100;
    unsigned short subesp100_3 = 0x0000;
#endif

    unsigned char moveax = 0xB8;
    unsigned long calladdr = 0;
    unsigned short calleax = 0xD0FF;
#ifdef XCXCALL
    // Restore the stack
    unsigned short addesp100_1 = 0xC481;
    unsigned short addesp100_2 = 0x0100;
    unsigned short addesp100_3 = 0x0000;
#endif
    unsigned char ret = 0xC3;
#endif // WIN64

    COMCALL(IHotPatch*dispp,HOTPATCH* hpx,size_t targetcall,const wchar_t* fname)
        {
        HPPointer = dispp;
        HPClass = hpx;
        calladdr = targetcall;
        regaddr = (size_t)this;
#ifndef XCXCALL
        movd2 = (size_t)&COMCALPTR;
#endif
        for (int i = 0; i < 8; i++)
            {
            m1[i].pushreg = (unsigned char)(0x50 + i);
            m1[i].addr = (size_t)(data + i * sizeof(size_t));
            }
#ifdef _WIN64
        for (int i = 0; i < 8; i++)
            {
            m2[i].pushreg = (unsigned char)(0x50 + i);
            m2[i].addr = (unsigned long long)(data + (i + 8) * 8);
            }
#endif
        size_t le = wcslen(fname);
        if (le > 50)
            le = 50;
        memcpy(name,fname,le * 2);
        }
    };

#pragma pack(pop)

The first 5 bytes are a JMP $ + 300, which leaves some space to save data. In the data[] and name[] the current values of the registers are set, by executing the "MOVER" structure 8 times (i.e. PUSH EXX, POP EAX,MOV [address],EAX), where EXX and address are set by the constructor of the COMCALL32 structure at runtime.

Finally, there is a sequence in which the address of the IHotPatch* pointer is saved to ECX, and then MOV EAX, COMPatchGeneric, CALL EAX, RET are encoded.

Some stack is added and removed before the call to COMPatchGeneric / USMPatchGeneric

COMPatchGeneric is another, this time C++ function, which reads the structure address from ECX/RCX, then reads all the required data from the COMCALL32 structure, creates the appropriate BSTR and SAFEARRAY COM variables in order to pass them to the COM server and, finally, passes them through IHotPatch::Call.

For x64 it is similar, except that 16 registers have to be saved (r9-r15 as well). The proxy bytes in the COMCALL structure change their size as well.

Without COM

You thought I 'm done? Hahahah. Not so fast. I plan to use my USM to do the same interprocess mechanism but without COM. Less registry and, most important, less buggy COM proxying. Very much direct stuff.

The mechanism for using shared memory is similar to the COM mode.  You create a shared memory area which is initially used to request the patch names. Then you run the patcher manually and make it (say, through a command line option) to also open the same shared memory area and return you the names. This is implemented in HOTPATCH::StartUSMServer.

Instead of passing stuff to COM, the patcher uses the mutual class (found in USM) to "call" the remote function. This method has the benefit that it does not need registration, but it needs extra work in sending/receiving data from/to the patcher.

Without USM

You still think I 'm done? You are far beyond my expectations. I ALWAYS search for more crazy ways to do crazy things.

And now, I have the craziest method. I have my EXE, and an updater EXE (which happens to be my same EXE). And I load the EXE as a DLL in the SAME address memory and I patch it as if it where a dll.

The mechanisms for that method will be explained in my article about using an EXE as a DLL.

ToDo    

I have to also find a way to exclude stuff from the DIA SDK. I 've put some string searches like std::, ATL:: etc to exclude microsoft stuff. If anyone knows a solution to include ONLY the functions found in YOUR source code, let me know.

Also, the COM method does not work in x64. If anyone got a clue, let me know.

A few last words...    

As I said, my clients are now using this library so they never quit the game they love that much. However my girlfriend is also stuck at the PC's screen and because the game never quits since patching does not require a restart, she won't cook for me anymore.

Therefore, in case you or your company find this library useful, do tell your boss about me. He might hire me and then I could afford to buy some food :)

GOOD LUCK.  

History

28 - 08 - 2016: Enhanced library, no more admin for COM, cleaned interface.
01 - 11 - 2015: Added another method by patching an EXE using the EXE as DLL - Article on it follows.    

 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Michael Chourdakis
Engineer
Greece Greece
I'm working in C++, PHP , Java, Windows, iOS and Android.

I 've a PhD in Digital Signal Processing and Artificial Intelligence and I specialize in Pro Audio and AI applications.

My home page: http://www.michaelchourdakis.com

You may also be interested in...

Pro

Comments and Discussions

 
GeneralMy vote of 5 Pin
docNyie9-Sep-16 4:14
memberdocNyie9-Sep-16 4:14 
GeneralMy vote of 4 Pin
CarelAgain21-May-16 21:17
memberCarelAgain21-May-16 21:17 
SuggestionHow to patch in x64 with 6 bytes and IA64 with 0 bytes Pin
Jakob Bohm23-Nov-15 9:43
memberJakob Bohm23-Nov-15 9:43 
GeneralRe: How to patch in x64 with 6 bytes and IA64 with 0 bytes Pin
Michael Chourdakis26-Nov-15 1:17
memberMichael Chourdakis26-Nov-15 1:17 
AnswerRe: How to patch in x64 with 6 bytes and IA64 with 0 bytes Pin
Jakob Bohm26-Nov-15 4:25
memberJakob Bohm26-Nov-15 4:25 
GeneralMy vote of 3 Pin
den2k885-Nov-15 4:29
professionalden2k885-Nov-15 4:29 
GeneralRe: My vote of 3 Pin
Michael Chourdakis5-Nov-15 4:57
memberMichael Chourdakis5-Nov-15 4:57 
GeneralRe: My vote of 3 Pin
john morrison leon7-Nov-15 9:40
memberjohn morrison leon7-Nov-15 9:40 
GeneralRe: My vote of 3 Pin
Michael Chourdakis18-Nov-15 7:04
memberMichael Chourdakis18-Nov-15 7:04 
GeneralMy vote of 5 Pin
john morrison leon18-Nov-15 12:16
memberjohn morrison leon18-Nov-15 12:16 
GeneralRe: My vote of 5 Pin
Michael Chourdakis2-Dec-15 4:18
memberMichael Chourdakis2-Dec-15 4:18 
GeneralMy vote of 5 Pin
R. Hoffmann1-Nov-15 22:52
professionalR. Hoffmann1-Nov-15 22:52 
GeneralRe: My vote of 5 Pin
Michael Chourdakis1-Nov-15 23:13
memberMichael Chourdakis1-Nov-15 23:13 
Question5 here. Pin
2374129-Oct-15 15:18
member2374129-Oct-15 15:18 
QuestionAwesome! Pin
koothkeeper29-Oct-15 6:24
professionalkoothkeeper29-Oct-15 6:24 
GeneralMy vote of 5 Pin
koothkeeper29-Oct-15 6:24
professionalkoothkeeper29-Oct-15 6:24 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web04-2016 | 2.8.180920.1 | Last Updated 28 Aug 2016
Article Copyright 2015 by Michael Chourdakis
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid