Click here to Skip to main content
Click here to Skip to main content
Go to top

Kernel-Mode Debugging in a VM using Visual Studio 2012

, 31 Aug 2013
Rate this:
Please Sign up or sign in to vote.
This tip is about how to setup Kernel-Mode Debugging in a VM using Visual Studio 2012.

Introduction

This tip is about how to setup Kernel-Mode debugging in a VM using Visual Studio 2012. This tip will be helpful for people new to driver development that wish to create, build, deploy, and debug a simple "HelloKernel" project.

Background

The setup process can be divided to three parts:

  • Setting up the Virtual Machine
  • Setting up Visual Studio 2012
  • Setting up your HelloKernel project

Each part will be explained step by step.

How to Setup the Virtual Machine

The Virtual Machine setup was tested on VMWare 9.0 running Windows 7 x64 OS.

  • Create a new VM, use default settings (tested on VmWare, Win7 x64)
  • Add a serial port to the VM, follow these steps:
    • Right click VM -> Settings -> Add -> Serial Port -> Output to named pipe
    • Named pipe = \\.\pipe\com_2
    • First Combobox = The end is the server
    • Second Combobox = The other end is an application
    • Connect at power on = Checked
    • Finish
    • Yield CPU on poll = Checked
  • In the VM, in an elevated Command Prompt window, enter the following commands:
    • bcdedit /debug on
    • bcdedit /dbgsettings serial debugport:2 baudrate:115200
  • In Windows, search for "Manage advanced sharing settings":
    • -Set "Turn on file and printer sharing"
    • -Set "Turn off password protected sharing"
  • In Windows, search for "Computer Management":
    • -Local Users and Groups -> Users:
    • -Administrator -> Right Click -> Properties -> Uncheck Account is disabled
    • -Administrator -> Right Click -> Set Password -> Set your password
  • Reboot the VM

How to Setup Visual Studio 2012

  • Install Visual Studio 2012 and WDK 8.
  • On the main toolbar, press Driver-> Test -> Configure Computers -> Add New Computer:
    • Computer name = VM name
    • Select Provision computer and choose debugger settings
    • Next
    • Connection Type = Serial
    • Pipe = Checked
    • Reconnect = Checked
    • Port = \\.\pipe\com_2
    • Target Port = com2
    • Next -> Finish

(Don't worry if Status is Undetermined.)

  • You may be prompted for Username\Password, enter Administrator\YourPassword
  • Important! Currently Visual Studio 2012 has a weird defect that causes the steps above to fail if your Windows username has spaces in it, the active username must have no spaces!

How to Setup Your HelloKernel Project

  • Use this tutorial to create your HelloKernel project
  • In order to debug your project, do the following:
    • OPTION I:
      • Press F6 (Important! A newly configured VM may requires one successful build before debugging)
      • Press F5
      • When VS12 changes to DebugMode, on the main toolbar press Debug -> Break All
      • In the "Debugger Immediate Window", write the following commands:
        • bu HelloKernel!DriverEntry (Tells the debugger to break on DriverEntry)
        • g (Tells the debugger to continue)
      • OPTION II
    • OPTION II:
      • Open two instances of VS, one will be the Deployer and the second the Debugger
      • [In Debugger] On the main toolbar press Debug -> Attach to Process
      • [In Debugger] Select "Windows Kernel Mode Debugger" and attach to the VM
      • [In Debugger] When VS12 changes to DebugMode, on the main toolbar press Debug -> Break All
      • [In Debugger] In the "Debugger Immediate Window", write the following commands:
        • [In Debugger] bu HelloKernel!DriverEntry (Tells the debugger to break on DriverEntry)
        • [In Debugger] g (Tells the debugger to continue)
      • [In Deployer] Press F6
  • 3. Your code should break on DriverEntry, and you can start debugging
  • *Tip I - Be sure your solution configuration is valid (Win7 x86 Debug\Win7 x64 Release ...)
  • *Tip II - Redeploying a driver to the VM after it had one successful deployment may cause problems (Such as the VM wanting to reboot). A workaround will be to return to the "Configured" snapshot before each deployment.

Summary

I hope this tip will help people new to driver development in getting started quickly.

It may seem straight forward, but I must say I encountered many problems trying to setup my environment, with nearly no useful and complete help found on the web.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Michael Engstler
Team Leader
Israel Israel
No Biography provided

Comments and Discussions

 
SuggestionVisual Studio 2013 / WDK 8.1 additional setup requirements Pinmemberabch13-May-14 12:35 
QuestionI would like to add you to skype PinmemberMember 1027823116-Sep-13 15:12 
QuestionKernel mode debugging PinmemberAster Veigas30-Aug-13 22:49 
AnswerRe: Kernel mode debugging PinmemberMichael Engstler30-Aug-13 23:14 
GeneralRe: Kernel mode debugging PinmemberAster Veigas31-Aug-13 0:03 
GeneralRe: Kernel mode debugging PinmemberAster Veigas31-Aug-13 0:12 
GeneralRe: Kernel mode debugging PinmemberAster Veigas31-Aug-13 0:15 
GeneralRe: Kernel mode debugging PinmemberMichael Engstler31-Aug-13 3:43 
GeneralRe: Kernel mode debugging PinmemberAster Veigas2-Sep-13 1:50 
GeneralRe: Kernel mode debugging PinmemberMichael Engstler2-Sep-13 10:07 
QuestionSecond select is not necessary PinmemberMember 161419123-Aug-13 0:05 
GeneralMy vote of 4 PinmemberMathew D28-Apr-13 3:25 
GeneralRe: My vote of 4 PinmemberAster Veigas2-Sep-13 1:51 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140916.1 | Last Updated 31 Aug 2013
Article Copyright 2013 by Michael Engstler
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid