SensiGator is an entry in the Education category of the 2013 Intel App Innovation Contest. It incorporates the Bing Maps WPF control DLL and taps into many of the Lenovo ThinkPad tablet's built-in sensors to create a unique map exploration experience. (The name "SensiGator" is a portmanteau combining the words "Sensor" and "Navigator".)
According to Microsoft, the Bing Maps WPF control combines the power of Windows Presentation Foundation and Bing Maps to provide an enhanced mapping experience. SensiGator takes this experience one step further by allowing a user to navigate the world map using information from the tablet's built-in sensors.
This creates a navigation experience similar to gliding in an aircraft, which includes a semi-transparent "head-up" display showing real-time location and map navigation data. SensiGator taps into the tablet's Inclinometer, Gyrometer, Compass, Orientation, and Geolocator sensor data to provide this real-time navigation experience.
Purpose and Intended Audience
SensiGator is intended to promote the learning of basic navigation and geography concepts through an immersive computer experience that is appropriate for modern education. The familiar world globe has been present in homes and classrooms for many years, encouraging youngsters to explore their planet and instill in them a sense of awe regarding the planet we all inhabit.
As tablet computers become increasingly prevalent in schools for students of all ages, simple apps like SensiGator that promote exploration and implicit learning will play an expanding role in the evolution of education. Furthermore, tablet technologies can reduce the need, and associated cost, of many of the traditional learning tools found in the classroom: hard-bound books, certain art supplies, and yes, the ubiquitous world globe.
About Bing Maps
According to Wikipedia, the spherical geometry of Earth was established by Greek astronomers in the third century BC, and the earliest globe appeared from that period (http://en.wikipedia.org/wiki/Celestial_globe#Celestial). Our human fascination with the immense planet we inhabit has persisted since then, and the advent of modern technology now allows us to create more engaging and immersive learning experiences for a generation raised on video games.
However, unlike a three-dimensional globe, the Bing Maps world map is a Mercator projection, a two-dimensional representation of our three-dimensional planet. According to Microsoft, Bing Maps provides a world map that users can directly manipulate to pan and zoom; in the case of SensiGator, this is accomplished through combinations of buttons, "touch" and tablet orientation.
Microsoft states that in order to make this interaction as fast and responsive as possible, they chose to pre-render the map at many different levels of detail, and to cut each map into tiles for quick retrieval and display. Microsoft refers to this as the Bing Maps Tile System. Furthermore, they state that although the Mercator projection significantly distorts scale and area near the poles, it has two important properties that outweigh the scale distortion:
- It’s a conformal projection, which means that it preserves the shape of relatively small objects. This is especially important when showing aerial imagery, because it is desirable to avoid distorting the shape of buildings (i.e., square buildings should appear square, not rectangular).
- It’s a cylindrical projection, which means that north and south are always straight up and down, and west and east are always straight left and right.
How To Use SensiGator
SensiGator is intended to operate with the tablet positioned face-up in either landscape (preferred) or portrait orientation. It has eight control buttons on the left side of the screen, and collapsible "head-up" display panels on the right. The buttons and display panels are semitransparent. The control buttons perform the following functions:
- Zoom In (magnifier image with plus sign) - Emulates a keyboard's "+" key and causes the map's Zoom Level to increment by 0.5 units*.
- Zoom Out (magnifier image with minus sign) - Emulates a keyboard's "-" key and causes the map's Zoom Level to decrement by 0.5 units*.
- Up Arrow - Emulates a keyboard's up arrow key and causes the map view to move in a northward direction (i.e., increase in latitude).
- Down Arrow - Emulates a keyboard's down up arrow key and causes the map view to move in a southward direction (i.e., decrease in latitude).
- Left Arrow - Emulates a keyboard's left arrow key and causes the map to move in a westward direction (i.e., decrease in longitude).
- Right Arrow - Emulates a keyboard's right arrow key and causes the map to move in a eastward direction (i.e., increase in longitude).
- Home - Repositions the map view to match the user's actual location, based on latitude and longitude coordinates from the tablet's Geolocator data. The Zoom Level is also set to 18 when this button is pressed.
- Information - Opens a dialog box with About and Help information on the app.
- (*See this link for information on Zoom Levels.)
The collapsible "head-up" display panels on the right side of the screen present the following information to the user:
- Map Location - Displays the current Latitude, Longitude and Zoom Level of the map view. This data changes in real-time as the user navigates the map using combinations of sensor, touch and button control.
- My Location - This panel contains an animated compass that shows the user's position relative to magnetic north (i.e., the compass will always attempt to point toward magnetic north using data from the tablet's compass sensor). The user's angular displacement relative to magnetic north is also shown numerically on the display (in degrees), along with his or her latitude and longitude coordinates.
- Sensor Nav - This panel displays information in response to sensor-based navigation. The map view will change in response to pitch and roll data from the tablet's inclinometer. More on this to follow.
Pitch and roll sensor readings (in degrees) are translated into x- and y-axis values that vary depending on whether the tablet is in portrait or landscape orientation. When the roll and/or pitch inclinometer readings exceed a tilt threshold of roughly ±7 degrees, the map will automatically start navigating in the direction of tilt (including diagonal translations).
The panel includes a graphical element that shows an arrow pointing in the direction of travel. When the tablet is at rest, the graphic changes to a sleepy face. There are also x- and y-axis level indicators framing the graphical element that reflect the magnitude of tablet tilting in either direction.
The Sensor Nav panel contains a sensitivity slider that can be used to increase or decrease the speed of map travel when navigating by sensor data. A non-linear equation was derived to control the translation speed at different map Zoom Levels. The sensitivity factor used in this equation changes in response to the slider control's position.
Zooming in and out of the map can also be accomplished using sensor data. Z-axis data from the tablet's Gyrometer (expressed in degrees-per-second) is used to initiate zoom actions. Rapid left-to-right tablet movements will increment the map's Zoom Level by 0.5 units (i.e., zoom in). Rapid right-to-left tablet movements will decrement the map's Zoom Level by 0.5 units (i.e., zoom out). Zooming in and out only occurs when the tablet's angular velocity exceeds ±150°/s in the z-axis.
A button is located in the upper right-hand side of the screen that can be used to show or hide the "head-up" display panels at any time. All button and sensor-based navigation is fully functional regardless of the display panel's visibility.
Other Input Modalities
Other input modalities and features can be realized when a USB keyboard and mouse are connected to the Lenovo ThinkPad tablet. When using a keyboard, all of the built-in Bing Map navigation control features are accessible: arrow keys to move up/down/left/right, "+" and "-" keys to zoom in and out, etc.
With a USB mouse connected, the user can navigate north/south/east/west by holding down the left mouse button while moving the mouse; zooming in and out is accomplished by scrolling the mouse wheel. While these are features that are built into the Bing Maps control, the SensiGator app incorporates additional mouse-based functionality by allowing the user to add pushpins to the map through double-clicking the left button. This provides a convenient way to mark locations on the map.
SensiGator also responds to the tablet's touchscreen, allowing Bing Map's built-in touch navigation control to navigate the map. However, some of Bing Map's touch capability has been inhibited to prevent undesirable behavior when multi-touch operations are attempted (more on this to follow).
Development Approach and Environment
As most developers who are into competition coding already know, Intel recently concluded Phase 2 of the Perceptual Computing Challenge and are currently judging submissions. I have four entries in that particular contest, and one of them utilized the Bing Maps SDK for gesture and voice controlled navigation. This is how I got my first exposure to using the Bing Maps SDK in a multimodal control application.
However, I resisted the temptation to re-enter any of these PerC projects in the AIC. I believe the spirit of these types of competitions is to develop something new and innovative that attempts to exploit those features of the target platform that differentiate it from other product offerings. (Had the interactive gesture camera been built into the tablets and AIOs that were awarded, that might be a different story.)
Nearly all of my development work entails the interaction of software with external hardware devices, but in this competition I wanted to deliver an app that would stand alone and not require any external interfaces. The sensor hub built into the Lenovo tablet seemed to satisfy my desire to interface with hardware elements, so I opted to submit project proposals that utilized these features.
SensiGator was developed from the ground up as a C#/WPF app, leveraging some of the knowledge gained - but very little work performed - as part of the PerC challenge. This development effort represents my first work using Visual Studio 2012 targeting .NET Framework 4.5.
As an added challenge, I did all of the software development and debugging on the Lenovo tablet itself, starting by downloading and installing Visual Studio 2012 Express on the tablet. Operation and installation testing was performed on both the Lenovo tablet and an Ultrabook computer that was awarded to me in last year's AIC.
(A special note of thanks to Intel Corp. is in order here, as they have been very generous in the support they offer to developers in these competitions.)
Bing Maps SDK
The process of registering a credential key and using the Bing Maps WPF SDK is well documented and straightforward, so I won't repeat it here. Please visit Microsoft's website for more information.
The rules of the AIC mandated that submissions must be Windows 8 desktop applications, so I opted to use the Bing Maps WPF SDK instead of Bing Maps for Windows Store Apps. This presented some challenges, mainly because the WPF SDK exhibits some quirky behavior when used in a Windows 8 desktop app. This behavior is especially evident when attempting multi-touch operations like zoom-in and zoom-out using pinch gestures. This problem is discussed in a MSDN forum.
Microsoft's response (noted in the forum link above) is as follows: "Our development team is aware of this issue. At this time, there are no plans for a next version of the WPF control. Note that if you are targeting Windows 8 you should use the Bing Maps Native control for Windows 8 which works great on all Windows 8 devices and has a lot more features than the WPF control."
To avoid a potentially negative user experience, I decided to handle the map's
TouchDown event to override Bing Map's default behavior. This essentially inhibits multi-touch and tap-zoom controls, but results in more predictable app behavior. The ability to reposition the map through touch-swipes is unaffected by this override.
In order to access the accelerometer, compass, gyrometer, and other sensors that comprise Intel's version of the Sensor Hub, I created a separate class library named "
SensorHub" and modified its .csproj file to target the Windows 8 platform. This process is described in detail here.
A good article and sample project that demonstrates how to access WinRT sensor information was published by Philip Schäfer on November 12th, 2012. I highly recommend studying this code in order to better understand the process of accessing sensor data from the Lenovo tablet. Here's a link to the website.
(Note: The two referenced sources noted above appear to have been merged and reposted as a CodeProject article on January 15, 2013 by a different author.)
The SensiGator app references the SensorHub.dll class library, instantiates a
SensorClass object for accessing tablet sensor data, and spawns three continuous threads for data processing, user interface interactions (using
Dispatcher.BeginInvoke), and acquiring Geolocator information. The data processing and UI threads update every 50 milliseconds, while the GeoThread updates every 5 seconds.
On the topic of geographical location data, I originally wanted to instantiate a
GeoLocator object at the main application level and subscribe to an event handler for location-change events, but due to some complications, I ultimately decided to use
GetGeopositionAsync() and wrap it in a
public method within my
SensorClass. This is why
Geolocation information is polled in a thread as opposed to being event-driven, but the overhead is minimal as the user's discernible geographic location is not changing rapidly with respect to the 5 second polling rate.
For the zoom and navigation buttons, I wanted to replicate the action of a keyboard so I used the Windows Input Simulator DLL, a C#
SendInput wrapper that simulates keyboard and mouse actions. This project is licensed under the Microsoft Public License (Ms-PL) and is available here.
SensiGator is installed by running a single executable, Setup.exe, which installs all the distinct application files, support files, and dependencies required to run SensiGator.
(Note: Some virus checking apps may warn the user or attempt to run the Setup installer in a sandbox. If this happens, disregard the warnings and allow the Setup app to run normally.)
About the Author
My background spans over 20 years of R&D and product development experience. I hold MS, BS and AS degrees in Software Engineering, Computer Science and Electronics Engineering, respectively. My day-to-day professional activities include a mix of both software and electronics design, with an emphasis on machine and device control applications.
I am also the founder of Human-Machine Technologies, Inc., a research and development NPO dedicated to improving the human experience through engineered solutions that integrate custom software and electronics with low-cost, commercially available NUI and BCI sensor technologies (http://www.human-machinetechnologies.com/).