Click here to Skip to main content
Click here to Skip to main content
Go to top

Celerity: Sensory Overload

, 23 Oct 2012
Rate this:
Please Sign up or sign in to vote.
Sensor-controlled XNA Tunnel Game with VR Head-Tracking
This is an old version of the currently published article.

Please note

This article is an entry in our AppInnovation Contest. Articles in this sub-section are not required to be full articles so care should be taken when voting.


This competition submission is a game called Celerity: Sensory Overload. It combines the classic Tunnel Game genre with sensor control and most notably a novel take on 3D.


The main inspirations for this game are: 

How could I resist?   

Time was tight as I only started after the details of the competition went up. I invited a couple of talented friends to help me out; @Dave_Panic (3D trickery guru) and @BreadPuncher (game theorist & graphic designer).

I was thinking about doing something along the lines of gyro-based steering and/or perspective and Dave suggested doing the head tracking effect with the webcam. Marrying this concept with a classic tunnel game we had the foundation of an innovative, sensor-tastic game. You know how gamers subconsciously tend to lean and move their head around when playing some games even though it makes no difference? Now it will! 

VR Head Tracking With Just A Webcam 

In Johnny Lee's video, above, he is able to detect the position of the user's head in 3D space through infra red (IR) LEDs and an IR camera. 

The effect is fantastic but requires a special IR sensor and also that the user wears an IR-emitting device. I needed to create this effect using only sensors on an Intel Ultrabook, and no other equipment. Thankfully the Ultrabook has a webcam, so we can achieve this in a round-about way.

The application processes each frame from the webcam and run it through a Computer Vision (CV) image-processing library, EMGUCV. This returns a rectangle within a box, representing the relative position of the face within the view of the camera. As the user moves their face, the rectangle will move around, giving me an X/Y offset of the user's face. The X & Y of the 3D world's view can be skewed in relation to the user's own physical position. Note that we need to flip the image horizontally as the webcam is looking in the opposite direction to us.  

This effect can be taken even further, as the rectangle representing the user's face inherently has a size. This gets larger as they move towards the webcam, so we can also determine the relative Z position, too.  

It's helpful if the user starts with their head roughly central, which we assist with as part of the intro menu UX. There the user can see if they're vaguely "calibrated" before starting. 

Designing The UI    

I wanted to make use of Metro design principles in the user interface, as although this is a desktop game and not a Metro app, I value consistency and feel the UI will be more intuitive if it shares conventions with the operating system, Windows 8. 

Here are the layout mocks for the UI. For the final thing, imagine the animated tunnel effect behind the UI, only faded out somewhat. 


Building The App   

I wanted to do the app as an XNA app, as it's perfectly fast enough and nice and easy to code for. I don't know C++ yet and there's not enough time to both learn it and complete the game. Unfortunately XNA is an awkward combination with Windows 8, as it's not supported in Visual Studio 2012, at least at the time of writing. My thanks goes to Ibrahim's article which helped me initially create the project. Steps 1 & 2 are essentially from there.     

Step 1: Project Pains  

Step 1 was to make a project in VS2012 running on the XNA Framework. This was achieved by initially creating the application as a class library, importing all the XNA DLLs, and then converting to an x86 windows application. Manually create the following:  

The launcher class:  

static class Program
    static void Main(string[] args)
        using (var game = new CelerityGame())

The game class:

public partial class CelerityGame : Game
    GraphicsDeviceManager graphics;
    SpriteBatch spriteBatch;
    public CelerityGame()
         graphics = new GraphicsDeviceManager(this);
         Content.RootDirectory = "Content";

The Initialize method:

protected override void Initialize()

The LoadContent method:

protected override void LoadContent()
    spriteBatch = new SpriteBatch(GraphicsDevice);

The Update method:

protected override void Update(GameTime gameTime)

Step 2: Nearly Content 

Although there's no proper "Content" project you can still use the ContentManager class directly to load resources. For example: 

// "Content" is a folder in the root of my game project
Texture2D t = Texture2D.FromStream(graphics, TitleContainer.OpenStream(@"Content\example.png"))   

Step 3: SpriteFont Workaround  

Fonts are a bit more tricky. The issue is that whilst you have the class, you don't have SpriteFonts as project items in VS2012 so it requires a workaround. My solution is to simply create a Windows app in VS2010,  insert a SpriteFont there, build the project, and then copy the .XNB file from the bin directory over as a resource.  

It can then be loaded like this: 

// I have a file called "SegoeUILight56.xnb" in a subfolder Fonts within the main Content folder
// Note that unlike with Images, the code doesn't refer to "Content" or ".xnb", as these are assumed
SpriteFont f = contentManager.Load<SpriteFont>(@"Fonts\SegoeUILight56");  

Step 4: The Sound of Music 

Audio content files are similar to SpriteFonts. I used VS2010 to import an .MP3 content file. When it builds it both converts it to .WMA and produces an .XNB file. Copy both these files from the bin directory as before, and load like this:  

Song s = contentManager.Load<Song>(@"Audio\mySongName"); 

Play your song as normal:


Step 5: Webcam Woes (and EMGUCV)

In this project I'm using EMGUCV version 2.4.0. I was using 2.4.2. but was able to get a much smaller DLL profile with 2.4.0. whilst still keeping everything I needed. It was a cinch to perform a basic capture from a webcam thanks to their simple tutorials, however when I attempted something more ambitious I ran into two related problems. 

  • EMGUCV gives me a System.Drawing.Bitmap rather than a Texture2D
  • The performance became disastrous 

I found a number of examples of converters online and tried several but still had the performance issues. I suspected it was an issue with running it on a single thread, but when searching the net I found many people warning against using threading in XNA. Eventually I ignored their advice and tried a combination of this method of image conversion and my first ever use of Parallel.Invoke(). All of a sudden worked a treat without me ever having to quite get to the bottom of it! Don't you just love it when that happens? 

Here's my naïve but incredibly effective solution: 

Parallel.Invoke(() => QueryCamera(elapsedMilliseconds));  

Simply by calling my existing method like this instead of directly from my Update method it was insta-fixed. Sweet!

Yes, okay I ought to read what that's actually doing but first I have a game to make.  Smile | <img src=   

One last note on the head tracking; over time the appearance of the resultant rectangle is a bit jittery z-axis wise. This is because it often catches two rectangles per face, but sometimes just either one. This makes for frequent changes in size. I fixed this by taking a recent sample history and taking a rolling average to smooth it out.    

Step 6: Raster To Go Faster 

In the hope of building a responsive UI quickly I was keen to try a library Sacha Barber mentioned, XNAML. In theory this would allow me to build the UI layer in WPF, with its masterful handling of  whilst keeping the tunnel rendering layer in XNA.

With a little fiddling I got the library working, a simple test game canvas beneath a button on top. Unfortunately XNA layer was running at a fixed 96dpi whereas the  WPF layer adjusted to my operating system setting, 120. So close but so far.

This means I'm building the UI purely in bitmaps. Given the extremities of possible screen resolutions for a Windows 8 app (1024 wide -> 2560+ wide) making it responsive and still look good is quite a lot of work. Unfortunately this involves lots of rasterising vector assets, calculating coordinates, and of course trial & error. 

If anyone knows a way of getting XNA & WPF to play nicely, full screen and for different resolutions etc. then please do let me know. 

Step 7: Alpha Mares 

To be continued. (How a default setting made all our graphics look screwy and the simple fix) 

Step 8: Dark At The End Of The Tunnel  

Dave has been beavering away on a custom 3D engine for the tunnel. Here are a few words from him explaining how it was done... 

The game is going to feature a continuous tunnel, filled with obstacles, that the player must negotiate. In order to achieve this, first we’re going to need a tunnel. I decided to break down the problem into 2 classes: one to deal with the section of the tunnel we can actually see, TunnelSection, and one to deal with the tunnel as a whole, Tunnel

TunnelSection will have the following responsibilities:

  1. Construct the vertices of this piece of the Tunnel
  2. Construct the texture coordinates for each vertex.
  3. Draw the tunnel. 

Tunnel will:

  1. Maintain the position of the TunnelSections (we’re going to be moving the tunnel rather than the player in order to prevent coordinates growing too large).
  2. Define the actual shape or curvature of the tunnel as a whole. 

Today we’re going to look at creating the basic tunnel shape. Here’s the basic outline for the TunnelSection class. 

A tunnel made from triangles has the following properties: 

public float Radius { get; set; } //the radius of the tunnel walls
public int NumSegments { get; set; } //the number of segments in the wall. 5 would make a pentagonal tunnel.
public int TunnelLengthInCells { get; set; } //number of rings of vertices in the tunnel section
private float cellSize; //distance between the rings of vertices<span style="font-size: 14.28px; "> </span>

Here's how we create the vertices: 

private void ConstructVertices()
       int numVertex = NumSegments * TunnelLengthInCells;
       vertices = new VertexPositionColorTexture[numVertex];
       float sectionAngle = 2 * (float)Math.PI / NumSegments;
       int vertexCounter = 0;
       for (int i = 0; i < TunnelLengthInCells; i++)
              for (int j = 0; j < NumSegments; j++)
                     Matrix rotationMatrix = Matrix.CreateRotationZ(j * sectionAngle);
                     vertices[vertexCounter].Position = Vector3.Transform(new Vector3(0.0f, this.Radius, 0.0f), rotationMatrix);
                     vertices[vertexCounter].Position.Z = -cellSize * i;
}<span style="font-size: 14.28px; "> </span>

What we’re doing here is first we’re creating a new vertex with an x and z coordinate of zero and a y coordinate equal to our desired radius. Then we’re rotating that point around the origin by the appropriate angle and moving on to the next point. When we’ve done a full circle, we repeat the process but move out point further away by the distance defined by cellSize

Since we’d like to keep the square sections that make up the tunnel looking like squares we need to work out what the distance between 2 points in the ring of vertices would be. We can do this by creating a point at (0, radius, 0) rotating around the z axis appropriate angle (2 * (float)Math.PI / NumSegments) then measuring the distance between the 2 points. As follows:  

private float CalculateSectionSize()
       Vector3 point1 = new Vector3(0.0f, this.Radius, 0.0f);
       Vector3 point2 = Vector3.Transform(point1, Matrix.CreateRotationZ(2 * (float)Math.PI / NumSegments));
       return Vector3.Distance(point1, point2);
}<span style="font-size: 14.28px; "> </span>

Now we have our vertices we’re going to need to fill our index buffer. The index buffer tells the GPU which vertices to use in which triangle. It’s a list that points to the index of each vertex in the vertex array. 

private void ConstructIndices()
       int indexCount = TunnelLengthInCells * NumSegments * 6;
       indices = new short[indexCount];
       int indexCounter = 0;
       for (int i = 0; i < vertices.GetUpperBound(0) - NumSegments; i += NumSegments)
              for (int j = 0; j < NumSegments; j++)
                     //triangle 1
                     indices[indexCounter] = (short)(i + j);
                     indices[indexCounter + 1] = (short)(i + j + NumSegments);
                     indices[indexCounter + 2] = (short)(i + j + 1);
                     if (j == NumSegments - 1) indices[indexCounter + 2] = (short)i;
                     //triangle 2
                     if (j < NumSegments - 1)
                           indices[indexCounter + 3] = (short)(i + j + 1);
                           indices[indexCounter + 4] = (short)(i + j + NumSegments);
                           indices[indexCounter + 5] = (short)(i + j + NumSegments + 1);
                           indices[indexCounter + 3] = (short)(i + j + NumSegments);
                           indices[indexCounter + 4] = (short)(i);
                           indices[indexCounter + 5] = (short)(i + j + 1);
                     indexCounter += 6;
}<span style="font-size: 14.28px; "> </span> 

Here we’re looping though the vertices in the same order we created them wiring up the triangles as we go. The trick is to get them all winding clockwise so backface culling will not make the triangles invisible which requires a bit of mental visualisation to work out which vertices you want to wire into your triangle based on where we are in the triangle. Also, you’ll notice there’s a special case where we get to the end of ring of vertices. If this special case were not present, we would end up creating triangles that corkscrew along the tunnel and end up leaving a one triangle gap at the start and end of the tunnel section.

Here is a TunnelSection:


And here are several TunnelSections sewn together to form a Tunnel, with a nice curve for good measure: 


Step 9: Gyromancy (Steering)    

To be continued.    

Step 10: Collision Course 

To be continued. 

Step 11: Spit & Polish 

To be continued. 

Step 12: Release 

To be continued. 

Intended Sensor-Specific Features    

I'm deliberately keeping the app as straight-forward as possible with the initial release to ensure it gets done. I'd prefer to do something focused well, than have lots of features rushed in. Here is the basic "V1" sensor feature set:   

  • Webcam-based VR Head Tracking  
  • Gyro-based Steering 

"If I have time" Potential Features  

Here are some more features I'll love to stick in either according to time once the basic app is done or possibly as a later update:  

  • Multi-touch thumb weapon controls - why dodge an obstacle when you can blast it, right? This will be two areas of the screen which independently respond to thumb taps. Maybe even different gestures for different weapons, depending how viable a thumb gesture is when twirling an Ultrabook.  
  • Subtle contrast adjustment according to the ambient light sensor. I want to keep the look of the game similar, but the greys on white can become full black on white, for example. 
  • Shake-to-escape emergency random teleport feature, a bit like "Hyperspace" in Asteroids-style games.    
  • Alternate controls, e.g. gamepad, keyboard, in case peoples' arms get tired swinging it around (this thing is light, right?)  
  • A number of additional gameplay mechanics (e.g. power-ups, alt game-modes etc.) 
  • Some proper sound design, but this might require bringing someone else in who has experience in this area. Volunteers welcome!      
  • Original music score (I'd love to do this myself, but I'm busy coding at the moment) 

Coming Soon     

The UI is essentially done, and the basic tunnel has been made. The next step is movement, tying in the sensors for steering and tying in the head tracking data to the camera. 

Also I'm currently battling with deployment, as VS2012 really doesn't like XNA. And I mean even less than I thought it did! 

Feel free to bookmark this article or follow my Twitter feed as I'll keep updating them with the progress. 


This app is intended as a submission in the Desktop Games category. 


  • V1 First Draft    
  • V2 Minor corrections & progress updates 
  • V3 Fixed typo and minor updates 
  • V4 Added tunnel progress & how-to  



This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Adam David Hill
Software Developer
United Kingdom United Kingdom


Musician turned Software Engineer (turned professional around 6 years ago). Mainly interested in games & mobility.

Sometimes I do real work, too.


My articles:


My open source software:

Follow on   Twitter   Google+   LinkedIn

Comments and Discussions

Discussions posted for the Published version of this article. Posting a message here will take you to the publicly available article in order to continue your conversation in public.
QuestionBravo Adam!!!! PinmemberAbhishek Nandy6-Dec-12 19:36 
AnswerRe: Bravo Adam!!!! PinmemberAdam David Hill6-Dec-12 21:40 
GeneralMy vote of 5 PinmemberFlorian Rappl6-Dec-12 7:33 
GeneralRe: My vote of 5 PinmemberAdam David Hill6-Dec-12 22:01 
QuestionThis is such a cool Ultrabook idea PinprotectorPete O'Hanlon19-Nov-12 20:58 
AnswerRe: This is such a cool Ultrabook idea PinmemberAdam David Hill29-Nov-12 12:22 
GeneralRe: This is such a cool Ultrabook idea PinprotectorPete O'Hanlon29-Nov-12 12:27 
GeneralRe: This is such a cool Ultrabook idea PinprotectorPete O'Hanlon29-Nov-12 22:31 
QuestionMy Vote of 5 PinmemberAbhishek Nandy22-Oct-12 7:56 
GeneralMy vote of 5 PinmemberAbhishek Nandy22-Oct-12 7:55 
GeneralMy vote of 5 PinmemberShintu Dhang11-Oct-12 6:43 
GeneralRe: My vote of 5 PinmemberAdam David Hill11-Oct-12 6:59 
GeneralRe: My vote of 5 PinmemberShintu Dhang11-Oct-12 7:03 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140916.1 | Last Updated 23 Oct 2012
Article Copyright 2012 by Adam David Hill
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid