Microsoft Surface Intro

Introduction

This is a short introduction to Microsoft's Surface technology.

What is Surface?

The best way to quickly get an impression of the actual thing would be to watch this short video. I guess, formally, Surface is a combination of hardware that has some new input qualities and software APIs that come along with it to support them. The parody (also a recommended watch) describes it as 'a computer that is a big ass table’. For those of you who haven’t watched the movie, Surface is a table that has a computer screen on its surface. You can interact with it by touching the screen, which is far from new, but you can do it in multiple spots simultaneously (multiple users, multiple fingers of one user...), and you can place objects on it and have the Surface recognize them and interact with them.

Microsoft identifies four key concepts in Surface:

• Direct interaction – no mouse or keyboard
• Multi-touch contact – more than one interaction spot
• Multi-user experience – the table has no specific orientation; you may have several people stand around it and interact
• Object recognition

It seems Microsoft is also aiming towards wireless technologies, enabling data transfer between devices placed on the Surface.

What is it for?

The industry always thinks of new ways to interact with computers. While some of these ways are clearly very cool, their usefulness is not always that evident.

Microsoft hopes to have Surface integrated into our life in places computers don’t go today, using its unique attributes. They hope to have it in social occasions where people would use it to interact with each other (casinos, waiting rooms, conferences etc.). They would also like to use the object recognition capabilities to have the computer provide extra information regarding physical objects (in stores and such). Surface also seems to be naturally suited for manipulating visual information such as maps and photos (for military situation rooms?).

Home use of Surface doesn’t seem to be close at the moment with a price tag of $12,500 (developer’s edition goes for $15,000), and I cannot imagine anyone programming on it (unless programming for it). Maybe an architect would find it useful for work purposes. I would be disappointed if it would turn out to be a bit fancier touch screen kiosk. Time will tell.

Innovation

Some of the concepts behind Surface are out there since the eighties. There are even a few commercial products available. (Wikipedia quotes Steven Spielberg on Minority Report commentary saying that the concept of the input device used by Tom Cruise in the film came from consultation with Microsoft during the making of the movie.) Microsoft has an edge since it has a strong stand in the world of programming, and used it to make developing for Surface as easy as it could. Microsoft has also proven that it can make a reasonable hardware job when properly motivated.

Technology

Under the hood, Surface is a PC, running Vista, hidden in a table. The table projects the output on its surface, and has five infra-red cameras looking up to the surface and filming whatever is going on there. The computer also has a wired Ethernet 10/100, wireless 802.11 b/g, and Bluetooth 2.0.

Object recognition

Surface physical objects recognition is based on physical tags with patterns printed on them. These patterns represent a GUID and an orientation. The Surface plumbing parses the data and tells you what the GUID of the object placed on the table is and its orientation. Tags can be printed by developing parties, and they only need to be infra-red visible so they can even be transparent.

Surface Programming API

Microsoft introduced Surface's API during PDC2008. The movie is available here if you are willing to deal with the technical difficulties and its length (about an hour and a quarter). Should you choose to watch the movie, read no further, it is all taken from there.

If you like picture recognition, or manipulation, or DSP, you may get everything the cameras see and do whatever with it (I wouldn’t know where to start):

Alternatively, you can have the API do its magic and tell you where fingers and objects are touching the screen, and in what orientation etc.

WPF API

If you like having the higher levels, you would be glad to learn that Microsoft introduced a set of WPF controls that are very similar to controls we already work with. The new controls can function in multi-touch scenarios and are configurable to produce a 'real world' feel.

Surface controls

Standard WPF controls are designed to work in an environment that has only one source of input. This means that we are safe to assume that button release over a mouse should fire the 'button click' event. For Surface, the button might be pressed by more than one user at a time, so the event is supposed to be fired only once all fingers are off of it.

In other cases, a control might want to respond to an occasion where it receives more than one input at the same time (as with the 'enlarge picture' you have probably seen if you have watched any Surface video or Minority Report).

Some times, controls need to be costumed to produce 'real world' behavior. For instance, a hand movement in an increasing speed where the hand lifts off from the table at the end of the movement would be the real world for 'throwing this picture to the other side' (try it on your desk). This is relevant with the mouse too (and supported in some applications), but not nearly as critical as it is for Surface.

To support all this stuff, Microsoft introduced a library of WPF controls, where Button is replaced with SurfaceButton etc. Theoretically, all you need to do in order to migrate an application into Surface is import the namespace into your XAML and initiate a few 'Find and Replace' actions on your WPF. That said, Surface applications are still fundamentally different from other applications as they should be designed 'lying on their back', accessible from all sides, and supporting physical objects. If you wrap your head around this, you might find that you need some extra tools. These tools are provided in the form of Scatter control and TagVisualizer.

Scatter control

Scatter control is the Surface version for ItemsControl. It has an ItemsSource property. It can use an ItemTemplate etc. The Scatter control scatters its items on the Surface. It also provides them with the ability to be dragged (or tossed) around on the Surface, change size, and so forth. You can use it to determine some 'real world' attributes such as friction etc.

TagVisualizer

The TagVisualizer class is the way you attach a visual representation to a physical object, or rather to a tag. As mentioned, tags are actually small labels representing a GUID and a direction. The lower level APIs read and parse the information and places the appropriate visualization.

Using the code

As you can see, there is no code. I really wanted to produce some nifty XAML for this article, but unfortunately, Microsoft has not provided the developers community with a Surface emulation. Such emulation (and access to the Surface developers network) is only available for companies who joined the program (and purchased a development unit).

This week, Microsoft announced its intention to widen the Surface community and allow access to Surface resources to all Microsoft partners. So far, I have not been able to join in (they seem to be having some technical problems – could it be that the servers are overcrowded?).

A look ahead

Microsoft is investigating MMI (Man-Machine Interface) in various directions. They are currently working on an XBox answer to Nintendo WII based on cameras watching the user and recognizing movements (something done in the movies industry for a very long time now). They have Surface, and they probably have a few other surprises up their sleeves and they are not alone out there.

As mentioned earlier, the usages are not always very clear at first sight, but the concepts of multi-touch and objects recognition are probably here to stay. It is nice to know that even if you are not quite there yet, your WPF investment can be leveraged for this new stuff. It is also nice to know that some people out there are working on new and exciting stuff.

Summary

If you have survived thus far, I am glad this exciting technology caught your attention as it did mine. You might want to watch the PDC 2008 video, and maybe check in on the Surface website. I suspect we would be hearing a lot about Surface soon, and PDC 2009 would probably be a good place to look for some more.

The folks here at The Code Project have opened a category for Surface and I hope we will get to see some real Surface-WPF code soon.