Kernel-Mode Debugging in a VM using Visual Studio 2012

Introduction

This article is about how to setup Kernel-Mode debugging in a VM using Visual Studio 2012. This article will be helpful for people new to driver development that wish to create, build, deploy, and debug a simple "HelloKernel". project.

Background

The setup process can be divided to three parts:

Setting up the Virtual Machine
Setting up Visual Studio 2012
Setting up your HelloKernel Project

Each part will be explained step by step.

How to Setup the Virtual Machine

The Virtual Machine setup was tested on VMWare 9.0 running Windows 7 x64 OS.

Create a new VM, use default settings (tested on VmWare, Win7 x64)
Add a serial port to the VM, follow these steps:

Right click VM -> Settings -> Add -> Serial Port -> Output to named pipe
Named pipe = \\.\pipe\com_2
First Combobox = The end is the server
Second Combobox = The other end is an application
Connect at power on = Checked
Finish
Yield CPU on poll = Checked

In the VM, in an elevated Command Prompt window, enter the following commands:

bcdedit /debug on
bcdedit /dbgsettings serial debugport:2 baudrate:115200

In Windows, search for "Manage advanced sharing settings":

-Set "Turn on file and printer sharing"
-Set "Turn off password protected sharing"

In Windows, search for "Computer Management":

-Local Users and Groups -> Users:
-Administrator -> Right Click -> Properties -> Uncheck Account is disabled
-Administrator -> Right Click -> Set Password -> Set your password

Reboot the VM

How to Setup Visual Studio 2012

Install Visual Studio 2012 and WDK 8.
On the main toolbar, press Driver-> Test -> Configure Computers -> Add New Computer:

Computer name = VM name
Select Provision computer and choose debugger settings
Next
Connection Type = Serial
Pipe = Checked
Reconnect = Checked
Port = \\.\pipe\com_2
Target Port = com2
Next -> Finish

(Don't worry if Status is Undetermined)

You may be prompt for Username\Password, enter Administrator\YourPassword
Important! Currently Visual Studio 2012 has a weird defect that causes the steps above to fail if your Windows username has spaces in it, The active username must have no spaces!

How to Setup Your HelloKernel Project

Use the following tutorial to create your HelloKernel project: http://msdn.microsoft.com/en-us/library/windows/hardware/hh439665(v=vs.85).aspx
In order to debug your project do the following: -Press F5 -When VS12 changes to DebugMode, on the main toolbar press Debug -> Break All -In the "Debugger Immediate Window" write the following commands:

-bu HelloKernel!DriverEntry (tells the debugger to break on DriverEntry)
-g (Tells the debugger to continue).

Your code should break on DriverEntry, and you can start debugging

Summary

I hope this article will help people new to driver development getting started quickly.

It may seem straight forward, but I must say I encountered many problems trying to setup my environment, with nearly no useful and complete help found on the web.