Click here to Skip to main content
12,699,492 members (29,753 online)
Click here to Skip to main content
Add your own
alternative version


8 bookmarked

Host-Guest communication: Bird’s Eye View

, 9 Jun 2010 CPOL
Rate this:
Please Sign up or sign in to vote.
This article describes way of secure and undetectable communication between software inside Virtual Machie and software running on the host OS

The Problem

I like virtual machines. I use them very often to run suspicions soft, to run different OS or to isolate programs from each other. One day I wanted to write a program which must be run inside VM and must communicate with software which is placed outside VM. I started to study possible ways to do this and find the following methods:

  1. Network connection. It is the most obvious way.
  2. Via files in shared folders.
  3. There is a special API for VMWare.

But I was not satisfied with any of these methods:
The 1st and the 2nd methods could be easily detected by rootkits and anti-viruses. Of course, if you know what to search for, you can detect many other things, but network activity and file I/O are the most obvious things to check.
The 3rd method is VM-dependent.
Also the 2nd and the 3rd methods require guest additions to be installed, however this was not critical for me.

And I wanted something which is very hard or even impossible to detect and which is not tied to any specific VM-product. I hadn’t found ready-made solutions and decided to do it myself.

How This Problem Could be Solved

So my goal was to create an undetectable communication engine between host and guest, which is not tied to any concrete VM. Also I add that I develop for Windows, and I will talk about implementation for this family of operating systems. But I think that this idea will work for other OSes as well.

How can this goal be achieved? The first thing which came into my mind was the idea that I can read and write VM’s memory. I can just reserve a region of memory inside the virtual machine and perform communication via this region. Inside the guest, everything is rather simple – I just need to reserve memory region. The most difficult thing is to arrange access to this memory region from host and make proper synchronization. And if you want to successfully implement such method, you should handle several technical issues which I describe in this article.

What Should You Know to Successfully Solve the Problem?

There are several things which you should know to understand my solution to the problem. Here is the list:

  • Knowledge of C/C++
  • General knowledge of WinAPI and DLL injection
  • How does virtual memory work
  • Understanding of multi-threaded programming and synchronization

There is plenty of information on the internet on the above mentioned topics. In addition, I can recommend the book “Windows via C/C++” by Jeffrey Richter and Christophe Nasarre.

When you become familiar with all these things, we can move to the first technical issue:

1. Where Can You Find VM’s Memory and How Can You Read it from the Host OS?

From the processes point of view, structure of many desktop VM solutions (VMWare workstation, VirtualBox, VirtualPC, Parallels Desktop) is as follows:

  • There are one or more GUI and service processes
  • For each running virtual machine, there is exactly one process which performs VM execution and stores its physical memory. This process is called “vmware-vmx.exe” for VMWare, “VirtualBox.exe” for Virtual Box, “Virtual PC.exe” for VirtualPC and “prl_vm_app.exe” for Parallels Workstation. In this article, I will call this process as “VM execution process”.

For example, here is a screenshot of task manager with 2 running VMWare instances:

vmware.exe” and “vmware-tray.exe” are service processes which are responsible for GUI and some other auxiliary things. And two “vmware-vmx.exe” instances are processes which actually perform execution of virtual machines.

And here is another obstacle…

2. How Can You Find Memory Region of Interest Inside the VM Execution Process?

Of course, VM physical memory is stored inside one process, but its internal structure is completely unknown. That’s why our program inside VM should not only reserve memory for communication, but it should also mark reserved memory with some signature.
Here I want to emphasize one important aspect of virtual memory. Virtual memory of each process is divided into pages (in 32bit Windows page size is 4096 bytes). And if pages have consecutive virtual addresses, it is not guaranteed that they lie consecutively in the physical memory. Moreover we do not know how VM execution process arranges physical memory of virtual memory of virtual machine. The following picture demonstrates the problem:

So we need to mark EACH page involved in communication with distinctive signature.
Another issue is the fact that guest OS can move pages or swap them into disk. We do not want this to happen, because in such case all our efforts in searching of pages location will be in vain. That’s why we need to lock page with help of VirtualLock function before we start the communication.

So our program inside the VM should do the following:

  1. Reserve memory region which will be used for communication.
  2. Lock this memory region.
  3. Write some distinctive signature into each page of the region.

And program in the host should scan entire virtual memory of VM execution process to find pages marked with the signatures.

Here we come to the final question…

3. How Can We Read and Write Target Memory Region and How Can We Synchronize Guest and Host Read-write Operations?

The answer to the first part of the question is rather simple. There are two common ways to do it:

  • We can use ReadProcessMemory and WriteProcessMemory WinAPI functions.
  • We can inject our own code into target process, so our injected code will be able to directly read and write memory.

Of course, if you implement everything I mentioned above, you’ll be able to arrange some kind of shared memory between host and guest. But it is not enough. You also need to implement some kind of synchronization primitives.

My method of synchronization is based on the assumption that each VM actually executes (not interpret!) code inside it (at least non-privileged instructions). This assumption is valid for the above mentioned VMs (VMWare, VirtualBox, …). Another idea is that some CPU instructions executes atomically even on multi-CPU cores. In particular, there is standard C function “InterlockedCompareExchange” which is based on “lock xchg” instruction. And “lock xchg” performs atomic compare exchange of CPU register and memory cell. And this “lock xchg” instruction is executed natively by virtual machine.

I did not write assembler code, but just used InterlockedCompareExchange function to create a simple mutex:

bool LockMutex(DWORD* mutexAddr)
    DWORD result;
    DWORD startTime = GetTickCount();
    result = InterlockedCompareExchange((LONG*)mutexAddr, 
    DWORD curTime = GetTickCount();
    while (result != MUTEX_STATE_FREE && (curTime - startTime < MAX_WAIT_TIME))
        if (sleepTime > startTime + maxWaitTime - curTime)
            sleepTime = startTime + maxWaitTime - curTime;
        result = InterlockedCompareExchange((LONG*)mutexAddr, 
        curTime = GetTickCount();
    if (result == MUTEX_STATE_FREE)
        return true;
    return false;
bool UnlockMutex(DWORD* mutexAddr)
    return InterlockedCompareExchange((LONG*)mutexAddr, 

A very important aspect of such mutex implementation is that the code which locks and unlocks the mutex must be executed in the address space of virtual machine. That’s why you have to use DLL injection to access the memory of virtual machine.

Scheme of Communication Engine

Combining all the above, we obtain the following scheme of communication engine:

  1. Program inside guest OS should do the following:
    1. Allocate and lock memory region
    2. Put special signature into each page which will be used for communication
    3. Waiting for connection from host
    4. Use InterlockedCompareExchange-based mutexes for synchronization
  2. Program inside host OS should do the following:
    1. Find target VM process
    2. Inject code which will perform actual communication into target VM process
    3. Injected code should periodically scan VM memory for signatures of pages which will be involved in communication
    4. When these pages will be found, communication with guest could be performed, and InterlockedCompareExchange-based mutexes should be used for synchronization.

What’s Next

In this article, I described the main ideas behind the communication engine. In addition, I’m going to write several more articles which will cover implementation details with source code of the solution.

To be continued…


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Alexey Kurakin
Russian Federation Russian Federation
My name is Alexey.

I am PhD student at MIPT in the area of Computer Vision. In 2009, I am obtained MS degree in the area of Data Mining and Machine Learning. Besides my scientific interests, I am fond of creating complicated software and solve difficult technical problems.

I am experienced C# developed. Also I have skills in Matlab, C++, Assembler and some other programming languages.

You may also be interested in...

Comments and Discussions

GeneralVery informative! Pin
S.H.Bouwhuis21-Jun-10 23:40
memberS.H.Bouwhuis21-Jun-10 23:40 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.170118.1 | Last Updated 9 Jun 2010
Article Copyright 2010 by Alexey Kurakin
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid