Click here to Skip to main content
15,904,494 members
Articles / Desktop Programming / Win32

Windows File Monitoring System Using Windows API Hooking

Rate me:
Please Sign up or sign in to vote.
4.25/5 (18 votes)
29 Oct 2008CPOL6 min read 115.9K   77   13
A Windows file monitoring system.


This Windows file monitoring system aims at providing security to files in a Windows environment. I was in a need to design an application which monitors file open, close, and save operations on Windows and restricts the user from accessing a subset of file types until this utility is installed. This was achieved by hooking Windows file related APIs and then preprocessing file open, save, and close operations in Windows as per the requirement. The preprocessing may be encryption of files, corrupting the file's header section, etc. If this utility is installed on the system, then the file, initially in the encrypted format, would first be decrypted before being opened, and then would again be encrypted when the file is closed or saved. This would help prevent the file from being accessed anywhere else where this utility is not installed, thereby providing security to the file.

I found a few good applications like AvaFind, FileMon, etc. performing file monitoring functionality. But, these applications use system drivers to achieve their goals. Because of the complexities involved in writing driver programs, I tried to achieve the same through Win32 user level programming. I have achieved this by hooking the CreateProcess(), OpenProcess(), CreateFile(), CloseHandle(), and WriteFile() functions of kernel32.dll.

Out of the various ways available for hooking Windows APIs, I chose the method of "Hooking by altering the Import Address Table (IAT)" of all the processes running on the system. The DLL containing the code to change the IAT is injected into the address space of the target process (the process in which the DLL is to be injected), by creating a remote thread in the address space of that process using the Windows API CreateRemoteThread().

Injecting DLLs by remote threads is well documented in Jeffrey Richter's article "Load Your 32-bit DLL into Another Process's Address Space Using INJLIB". Injecting DLLs has by far been the most widely used concept, with the greatest advantage of getting hold over the process, once your DLL is within its address space. But, this method has a disadvantage of the DLL not being injected into the target process in case kernel32.dll does not get loaded at its well known preferred load address. Hence, the method of injecting a DLL is based on the sole assumption that the load addresses of Kernel32.dll remain same for both the processes.

Design and Implementation

Coming back to the original problem...

The first step I performed was to create HookAPI.dll, which contains the code to hook Windows APIs, and then this DLL is injected in all the running processes on the system. Once injected into the target process, HookAPI.dll changes the IAT of the process and all of its loaded modules. HookAPI.dll contains a function called GetIAList() that traverses the IAT of the process in which it is injected. It uses EnumProcessModules() to get the list of all the modules of the process in which it is injected. Thereafter, it checks which function is to be hooked, and replaces its address in the IAT by the address of the wrapper function provided for that API. GetNewAddress() is the function which traverses a list specifying the functions to be hooked and returns the address of the wrapper function provided for the function to be hooked.

Here are the logical steps of a replacing cycle:

  • Locate the import section from the IAT of each process loaded by the process DLL modules as well as the process itself.
  • Find the IMAGE_IMPORT_DESCRIPTOR chunk of the DLL that exports that function.
  • Locate the IMAGE_THUNK_DATA which holds the original address of the imported function.
  • Replace the function address with the address of the wrapper function.
void GetIAList()
    HMODULE hMods[1024];
    TCHAR szLibFile[MAX_PATH];
    HANDLE hProcess = GetCurrentProcess();
    HMODULE hModule = GetModuleHandle(TEXT("HookAPI.dll"));
    DWORD cbNeeded = 0;
    multimap <CString,void*> m_mapOld;
    multimap <CString,void*> :: iterator m_AcIter;
    PROC pfnNewAddress = NULL;
    unsigned int i = 0;
    PROC* ppfn = NULL;
    CString  str;
    ULONG ulSize = 0;
    if( EnumProcessModules(hProcess, hMods, sizeof(hMods), &cbNeeded))
        for ( i = 0; i < (cbNeeded / sizeof(HMODULE)); i++ )
            TCHAR szModName[MAX_PATH];
            // Get the full path to the module's file.
            if ( GetModuleFileNameEx( hProcess, hMods[i], 
                                      szModName, sizeof(szModName)))
                // We must skip the IAT of HookAPI.dll
                // from being modified as it contains
                // the wrapper functions for Windows AOIs being hooked.
                if(_tcscmp(szModName,szLibFile) == 0)
            // Get the address of the module's import section
            ImageDirectoryEntryToData(hMods[i], TRUE, 
                    IMAGE_DIRECTORY_ENTRY_IMPORT, &ulSize);
            if(NULL != pImportDesc)
                while (pImportDesc->Name)
                    PSTR pszModName = (PSTR)((PBYTE) hMods[i] + pImportDesc->Name);
                    CString  strModName = pszModName;
                    // Get caller's IAT 
                    PIMAGE_THUNK_DATA pThunk = (PIMAGE_THUNK_DATA)
                                ( (PBYTE) hMods[i] + pImportDesc->FirstThunk );
                    while (pThunk->u1.AddressOfData)
                        // Get the address of the function address
                        ppfn = (PROC*) &pThunk->u1.AddressOfData;
                        // Store the dll name and address of the function from the IAT
                        // into a map in the form ("KERNEL32.dll:<address of CreateFile>",
                        // <address that contains the address of CreateFile in IAT>).
                        // The map contains the entries in the form
                        // ("KERNEL32.dll:0x110023",0x707462)
                        // ("KERNEL32.dll:0x110045",0x707234)
                        // ("KERNEL32.dll:0x110074",0x402462)
                        // ...
                        m_mapOld.insert( func_Pair( str, ppfn ) );
    // Traverse the map to hook the appropriate function.
    for(m_AcIter = m_mapOld.begin() ; m_AcIter != m_mapOld.end() ; m_AcIter++)
        // pfnNewAddress = GetNewAddress(m_AcIter -> first);
        // m_AcIter -> first is a string that gives all the informatino 
        // about the Windows API to be hooked like the name of the DLL 
        // in which the function is actually present, its address in that DLL 
        // and its address in IAT. 
        // GetNewAddress should be implemented in such a way that it should return 
        // the address of the wrapper function for Windows API implemented in HookAPI.dll
        if(pfnNewAddress != NULL)
            PROC* pfnOldAddress = (PROC*)m_AcIter -> second;
            MEMORY_BASIC_INFORMATION mbi = {0};
            VirtualQuery( pfnOldAddress, &mbi, sizeof(MEMORY_BASIC_INFORMATION) );
            VirtualProtect( mbi.BaseAddress,mbi.RegionSize,PAGE_EXECUTE_READWRITE,
            // Replace the origional address of API with the address of corresponding 
            // wrapper function 
            *pfnOldAddress = *pfnNewAddress;

            DWORD dwOldProtect = 0;
            VirtualProtect( mbi.BaseAddress, mbi.RegionSize, mbi.Protect, &dwOldProtect );

Once this DLL is designed, the next job is to create Injector.exe which injects the hooking DLL, HookAPI.dll, in all the running processes, by using CreateRemoteThread(), as shown below. InjectIntoExistingProcesses() is the function which injects the DLL in all the running processes. It uses EnumProcesses() to get the list of all the running processes. Here, pszLibFile contains the path of the DLL which has to be injected, which is nothing but a path to HookAPI.dll.

// Get the path of DLL to be injected
_tcscpy(_tcsrchr(pszLibFile,TEXT('\\')) + 1,TEXT("HookAPI.dll"));

VirtualAllocEx() is used to allocate memory in the address space of the remote process where the DLL is to be loaded. WriteProcessMemory() is used to write the DLL path in the allocated memory space. GetProcAddress() gives the address of the LoadLibrary() API (assuming that the load address of kernel32.dll is same for all the processes), and then CreateRemoteThread() finally creates a thread in the remote process and loads the DLL in the address space of the remote process.

void InjectIntoExistingProcesses(PCWSTR pszLibFile)
    PWSTR pszLibFileRemote = NULL;
    int cch = 1+lstrlenW(pszLibFile);
    int cb = (cch + sizeof(WCHAR))*sizeof(WCHAR);

    DWORD aProcesses[1024], cbNeeded, cProcesses;    
    EnumProcesses( aProcesses, sizeof(aProcesses), &cbNeeded); 
    cProcesses = cbNeeded / sizeof(DWORD);

    // Get process handle for each running process. 
    for (int i = 0; i < cProcesses; i++)
          HANDLE hRunningProcess = 
            OpenProcess( PROCESS_ALL_ACCESS,FALSE, aProcesses[i] );
          pszLibFileRemote = (PWSTR)VirtualAllocEx(hCurrentProcess,NULL,cb,
          SIZE_T nBytes = 0;
          pszLibFileRemote,(PVOID) pszLibFile,cb,&nBytes );
          if (pszLibFileRemote != NULL)
          if (hRunningProcess != NULL)
             CloseHandle(hRunningProcess );

When HookAPI.dll hooks the CreateProcess(), OpenProcess(), CreateFile(), CloseHandle(), and WriteFile() functions of all the running processes, we get control in our wrapper functions for almost all the file operations done on the system. CreateProcess() and OpenProcess() are hooked to capture the creation of any new process through a running process, and then HookAPI.dll is again injected into the newly created process via the wrapper function provided for CreateProcess() and OpenProcess(), respectively. This is done so that the DLL gets injected in each newly created process, before any new process starts. The CreateFile(), CloseHandle(), and WriteFile() functions are hooked in order to sniff the file open, close, and write operations, respectively, in any running process as well as in newly created processes. The wrapper functions provided for the hooked functions such as CreateFile(), CloseHandle(), and WriteFile() may then be modified for preprocessing the file operations as per the requirement.

Issues with the first run of this utility

Initially, all the files on the system are neither in encrypted form nor in decrypted form. This utility requires that all the concerned files (files belonging to a particular subset of file types) on the system should be in encrypted form before they are opened. Hence, this encryption should be done at the time of installation of this utility. One way to do this, in case a particular noise pattern is defined for corrupting the file headers, is to check whether this noise pattern exists in the file. If yes, remove the noise, i.e., decrypt the file and proceed in the fashion mentioned above. If no, it shows that this is the first run and no preprocessing is needed. Another approach is to find all the files on the system belonging to a particular subset of file types the utility is intended to process, and encrypt them all at the time of installing this utility.



This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Written By
Software Developer
India India
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

QuestionMy Vote of 3 Pin
Raghav Gaur13-Jan-16 23:53
Raghav Gaur13-Jan-16 23:53 
GeneralMy vote of 1 Pin
Doorman66629-Jun-13 2:17
Doorman66629-Jun-13 2:17 
GeneralMy vote of 1 Pin
Christian Brüggemann29-Feb-12 20:51
Christian Brüggemann29-Feb-12 20:51 
QuestionProblem Pin
lilyNaz20-Sep-11 18:35
lilyNaz20-Sep-11 18:35 
GeneralHelp required with FileWatcher class Pin
Member 373086912-May-11 0:59
Member 373086912-May-11 0:59 
Question[My vote of 2] Souce code? Pin
tommygr18-Jun-09 8:41
tommygr18-Jun-09 8:41 
GeneralSource Code Pin
Sachin Gorade22-May-09 5:25
Sachin Gorade22-May-09 5:25 
GeneralHelp Pin
Joe Rozario19-Apr-09 20:21
Joe Rozario19-Apr-09 20:21 
GeneralQuestion Pin
Khayralla17-Feb-09 8:37
Khayralla17-Feb-09 8:37 
GeneralYes source is missing Pin
binidoggy2-Feb-09 13:06
binidoggy2-Feb-09 13:06 
QuestionSource code? Pin
Jeova Almeida20-Nov-08 8:18
Jeova Almeida20-Nov-08 8:18 
AnswerRe: Source code? Pin
cute_friend707723-May-09 0:17
cute_friend707723-May-09 0:17 
QuestionVista? Terminal Services? Pin
Jeova Almeida20-Nov-08 3:08
Jeova Almeida20-Nov-08 3:08 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.