This Guide contains information about the APIs that application developers need to use when they are developing apps targeted for Ultrabook™ devices. It covers information for developing apps that run on Windows* 7, Windows* 8 Desktop, Windows* Modern UI, as well as apps that run in web browsers.
This guide also provides descriptions of common user expectations for gestures and provides some guidelines for developing satisfying touch interfaces.
Now that there are many touch-capable Ultrabook™ devices available, software developers want to create software that is designed for touch input. For an app to be fully touch-enabled, it must meet three criteria:
Whether developers are writing a new app or want to update an existing app to include touch support, this guide will help them in their journey.
For more information on writing touch-enabled apps, refer to the MSDN article, Touch Interaction Design.
Consumers have a variety of touch-capable devices to choose from. Developers have a challenging task when attempting to develop apps that feel natural on each device. In general, touch-enabled apps should be designed to run on any of their targeted devices while taking full advantage of the touch capabilities of each. Ultrabook devices represent the high-end of touch-capability and should be the design point for an app’s touch interface.
Ultrabook devices with touchscreens have been available since 2011:
The rest of this Developer Guide assumes that the target platform is a touch-capable Ultrabook system. Software designed for this device can be adapted to other touch-enabled devices that run the same OS.
Designing apps today requires careful considerations of what environment customers most often use as well as which environment an app is best suited for. Whether an app is targeted for Windows 8 Desktop or as a Windows Store app, the developer needs to understand which interfaces are applicable.
If an app is targeted for the Windows 8 UI (also known as a Windows Store app), then it must use the WinRT APIs. If an app is to run in the Windows 8 Desktop environment, there are more choices: the legacy APIs from Windows 7 and the new Windows 8 APIs for touch. These interfaces will be discussed further in the sections below. Other options exist for developing web apps. Touch interfaces available for web browsers are also discussed below.
There are varying levels of interpretation of touch input. Pointer events are the most basic because they represent individual points of touch contact. Gesture and Manipulation events are built upon that foundation. Gesture events provide an easy way to capture simple tap-and-hold gestures. Manipulation events are for touch interactions that use physical gestures to emulate physical manipulation of UI elements. Manipulation events provide a more natural experience when the user interacts with UI elements on the screen. The available touch interfaces have varying levels of support for these three levels of interpretation.
Read more about Gestures, Manipulations and Interactions on MSDN
A pointer event is a single, unique input or “contact” from an input device such as a mouse, stylus, single finger, or multiple fingers. When a contact is made, the system creates a pointer when it is first detected and then it is destroyed when the pointer leaves the detection range or is canceled. In the case of multi-touch input, each contact is a unique pointer. Table 1 shows the interfaces for retrieving basic pointer events that are available to Ultrabook devices running Windows 7 and Windows 8.
Table 1. Pointer APIs available for Ultrabook Devices
Windows* 7 (Desktop)
WM_TOUCH
Windows 8 only (Desktop)
WM_POINTER
Windows Modern UI
PointerPoint
Touch Interfaces available for Windows 7 and Windows 8:
Refer to Guidelines for common user interactions on MSDN.
The WM_TOUCH message can be used to indicate that one or more pointers, such as a finger or pen, have made contact on the screen.
Sample Code:
Guidance:
The WM_POINTER messages are part of the Direct Manipulation APIs and are specific to the Windows 8 Desktop. This interface can be used to capture individual touch pointers as well as Gestures and Manipulations. The WM_POINTER messages will be discussed further in the section on Manipulation and Gesture Interactions.
Reference on MSDN: Direct Manipulation APIs
The PointerPoint class is part of the Windows Runtime environment and is compatible only with Windows Store apps. It provides basic properties for the input pointer associated with a single mouse, stylus, or touch contact. MSDN has sample code that can help developers get started working with the PointerPoint interface.
Sample Code on MSDN: Input: XAML user input events sample
Gesture events are used to handle static-finger interactions such as tapping and press-and-hold. Double-tap and right-tap are derived from these basic gestures:
Press and Hold
Static Gesture
A single contact is detected and does not move. Press and hold causes detailed information or teaching visuals to be displayed without a commitment to an action.
Tap
One finger touches the screen and lifts up immediately.
Turn
Manipulation Gesture
Two or more fingers touch the screen and move in a clockwise or counter-clockwise direction.
Slide
One or more fingers touch the screen and move in the same direction (also called Panning)
Swipe
One or more fingers touch the screen and move a short distance in the same direction.
Pinch
Two or more fingers touch the screen and move closer together or farther apart.
Stretch
Two or more fingers touch the screen and move further apart.
Windows 7
Windows 8 (Desktop)
WM_TOUCH + IManipulationProcessor
WM_GESTURE + GESTUREINFO structure
References on MSDN:
Press and Hold to learn
Causes detailed information or teaching visuals.
Tap for primary action
Invokes a primary action, for example launching an application or executing a command.
Slide to pan
Used primarily for panning interactions but can also be used for moving, drawing, or writing. Can also be used to target small, densely packed elements by scrubbing (sliding the finger over related object such as radio buttons).
Swipe to select, command, and move
Sliding the finger a short distance, perpendicular to the panning direction, selects objects in a list or grid.
Pinch and stretch to zoom
Not only used for resizing, this interaction also enables jumping to the beginning, end, or anywhere within the content with Semantic Zoom. A SemanticZoom control results in a zoomed out view for showing groups of items and quick ways to go back to them.
Turn to rotate
Rotating with two or more fingers causes an object to rotate.
Swipe from edge for app commands
App commands are revealed by swiping from the bottom or top edge of the screen.
Swipe from edge for system commends
Swiping from the right edge of the screen shows the “charms” that are used for system commands. Swiping from the left edge results in cycling through currently running apps and sliding from the top edge toward the bottom of the screen closes the app. Sliding from the top edge down and to the left or right edge snaps the current app to that side of the screen.
The IManipulationProcessor interface can be used in conjunction with the WM_TOUCH API to provide a way to add translation, rotation, scaling, and inertia to UI objects. This combination provides functionality similar to the gesture recognizing features of WM_POINTER. Once the Manipulation Processor is enabled, manipulation starts as soon as a touch gesture is initiated.
IManipulationProcessor
WM_GESTURE messages have a structure called GESTUREINFO that is available for the interpretation of gestures and manipulations. The MSDN web page for WM_GESTURE shows an example of how to obtain gesture-specific information using the GESTUREINFO structure
Note that WM_GESTURE has limitations, such as the maximum number of simultaneous touch inputs is only two and it does not support simultaneous gestures. For apps that require more capability but still need to support Windows 7 desktop, use the WM_TOUCH interface and either write a custom gesture recognizer, as detailed in the section Custom Gesture Recognition below, or use the Manipulation Processor interface with WM_TOUCH.
Sample Code on Intel Developer Zone (WM_GESTURE API + GESTUREINFO: Sample Application: Touch for Desktop
For more information on writing touch-enabled apps, refer to the MSDN article: Touch Interaction Design.
Applications targeted only for the Windows 8 Desktop can use the Direct Manipulation APIs (WM_POINTER messages). The pointer messages are passed to an internal Interaction Context object that performs recognition on the manipulation without the need to implement a custom gesture recognizer. There is a callback infrastructure where all interactions involving tracked contacts are managed.
Direct Manipulation is designed to handle both manipulation and gesture interactions and supports two models for processing input:
Gestures can be captured by initializing Direct Manipulation and preparing the system for input processing.
Refer to the Quickstart: Direct Manipulation on MSDN for an outline of the API calls required to accomplish typical tasks when working with Direct Manipulation.
The GestureRecognizer API is used to handle pointer input to process manipulation and gesture events. Each object returned by the PointerPoint method is used to feed pointer data to the GestureRecognizer. The gesture recognizer listens for and handles the pointer input and processes the static gesture events. For an example of how to create a GestureRecognizer object and then enable manipulation gesture events on that object see the MSDN GestureRecognizer web page (referenced below.)
GestureRecognizer
Sample Code on Intel Developer Zone: Sample Application: Touch for Windows* Store.
When possible, use the built-in gesture recognizers (see Table 3). If the provided gesture and manipulation interfaces do not provide the functionality that is needed, or maybe the app needs to disambiguate between taps and gestures more rapidly, it may be necessary to write custom gesture recognition software. If this is the case, customers expect an intuitive experience involving direct interaction with the UI elements in the app. It is best to base custom interactions on the standard controls to keep user actions consistent and discoverable. Custom interactions should only be used if there is a clear, well-defined requirement and basic interactions don't support the app’s desired functionality. See Table 4 for the list of common and expected interactions and consequences for touch interactions.
Code Sample on Intel Developer Zone (WM_TOUCH with custom gesture recognition): Touch for Windows Desktop
Touch input is also available to apps running in web browsers, with varying degrees of support depending on the browser. Since web browser capabilities change rapidly, it is generally better to detect supported features instead of specific browsers. Feature detection has proven to be a more effective technique once the determination has been made if it is Internet Explorer (IE) 10, a browser built on Webkit*, or a different browser that requires support. Feature detection requires less maintenance for the following reasons:
WebKit powers Apple Safari* and Google Chrome*, and soon Opera will move their browser over to use it. Internet Explorer 10 does not use WebKit; however, both WebKit and IE 10 are built on top of the Document Object Model (DOM) Level 3 Core Specification. To review the standards associated with touch events, refer to the standard’s Touch Events Version 1, dated January 2013.
References:
IE 10 has its own touch interfaces that must be called for processing touch events. Use the navigator object with the userAgent property to determine if the browser supports the desired features. The following example indicates that the browser is Internet Explorer.
userAgent
Usage:
<script type="text/JavaScript"> If (navigator.userAgent.indexOf(“MSIE”)>0) { // Run custom code for Internet Explorer. } </script>
Use the hasFeature method to determine if specific features are supported in the browser. For example, here is how to determine if a browser supports touch events (this works for IE 10 as well):
hasFeature
var touchSupported = document.implementation.hasFeature("touch-events","3.0");
Where “touch-events” is the feature that we are checking for and “3.0” is the DOM specification level that we are interested in. An app can then listen for the following touch events: touchstart, touchend, touchmove, and touchcancel.
touchstart
touchend
touchmove
touchcancel
Reference: hasFeature method
To process touch events using a WebKit-based browser (Chrome, Safari, etc.), simply set up the following three events to cover the main input states:
canvas.addEventListener( ‘touchstart’, onTouchStart, false ); canvas.addEventListener( ‘touchmove’, onTouchMove, false); canvas.addEventListener( ‘touchend’, onTouchEnd, false);
For Internet Explorer, reference the MSPointer event instead:
MSPointer
canvas.addEventListener( ‘MSPointerDown’, onTouchStart, false ); canvas.addEventListener( ‘MSPointerMove’, onTouchMove, false); canvas.addEventListener( ‘MSPointerUp’, onTouchEnd, false);
Similarly, there are gesture event listeners as well. They are gestureStart, gestureChange, and gestureEnd for the non IE 10 browsers.
gestureStart
gestureChange
gestureEnd
Download sample code handling DOM pointer events on MSDN: Input DOM pointer event handling sample.
While IE 10 does not use WebKit, it is built on top of the DOM Level 3 Events, HTML5, and Progress Event standards. This section provides information about IE 10 and how it interacts on Windows 7.
Standards:
Internet Explorer 10 on Windows 7 handles touch and pen input as simulated mouse input for the following Document Object Model (DOM) events:
IE 10 on Windows 7 will not fire any of the following DOM Gesture events:
MSGESTURE
NO
Yes
MSPOINTER
YES
getCurrentPoint
getIntermediatePoints
For more information on developing touch-enabled web apps for IE 10 (MSDN): Internet Explorer 10 Guide for Developers
Sample code on MSDN: Input: Manipulations and gestures (JavaScript)
Whether an app is a native app or a web app, the developer will want to add a check for hardware touch capability so that the app can configure its UI appropriately. Use the following methods to test for touch capability.
Apps targeting Windows 7 or for Windows 8 Desktop can call GetSystemMetrics with SM_DIGITIZER as the argument. The following code snippet is part of a Touch sample that can be downloaded from the Intel Developer Zone: Touch for Windows Desktop
GetSystemMetrics
void SDKSample::DeviceCaps::Touch::TouchGetSettings_Click(Platform::Object^ sender, Windows::UI::Xaml::RoutedEventArgs^ e) { Button^ b = safe_cast<Button^>(sender); if (b != nullptr) { TouchCapabilities^ pTouchCapabilities = ref new TouchCapabilities(); Platform::String^ Buffer; Buffer = "There is " + (pTouchCapabilities->TouchPresent != 0 ? "a" : "no") + " digitizer presentn"; Buffer += "The digitizer supports " + pTouchCapabilities->Contacts.ToString() + " contactsn"; TouchOutputTextBlock->Text = Buffer; } }
For Internet Explorer, use the msMaxTouchPoints property described as follows:
msMaxTouchPoints
If (navigator.msMaxTouchPoints) {…}
If (navigator.msMaxTouchPoints >1) {…}
Var touchPoints = navigator.msMaxTouchPoints;
For Chrome and Safari, use the following (same as above but replace msMaxTouchPoints with maxTouchPoints):
maxTouchPoints
var result = navigator.maxTouchPoints;
It can be somewhat tricky to test for touch devices generically from web apps. While some functions work well on some browsers, others indicate that touch is present when it is not, i.e., if the browser itself supports touch, it may report that touch is available even if the device is not touch-capable.
Note that MaxTouchPoints will return 0 in IE 10 (Desktop) running on Windows 7.
Apps designed for Ultrabook devices may need to process gestures such as taps, pans, zooms, etc. Apps that are touch-enabled may do little with the raw pointer data except to pass it to gesture detection.
New applications should be designed with the expectation that touch will be the primary input method. Mouse and stylus support require no additional work; however, software developers should consider several other factors when designing touch-optimized apps.
// Check for Touch support // Get the Touch Capabilities by calling GetSystemMetrics BYTE digitizerStatus = (BYTE)GetSystemMetrics(SM_DIGITIZER); // Hardware touch capability (0x80); readiness (0x40) if ((digitizerStatus & (0x80 + 0x40)) != 0) //Stack Ready + MultiTouch { RegisterTouchWindow(m_pWindow->GetHWnd(), TWF_WANTPALM); }
Note that GetSystemMetrics can be used to find out what is the maximum number of touch points that are available:
BYTE nInputs = (BYTE)GetSystemMetrics(SM_MAXIMUMTOUCHES);
Determine touch capabilities for Windows Store apps by using the TouchCapabilities class. The following code snippet can be found in the code sample on MSDN that demonstrates its use: Input: Device capabilities sample.
TouchCapabilities
Precision
Human Anatomy
Object state
Rich interaction
Software developers should supply appropriate visual feedback during interactions so that users can recognize, learn, and adapt to how their interactions are interpreted by both the app and by the OS. Visual feedback is important for users to let them know if their interactions are successful, so they can improve their sense of control. It can help reduce errors and help users understand the system and input device.
App developers, who want to develop touch-enabled apps, whether their apps will run natively or are targeted as web apps, need to have a clear understanding of which APIs are available to them. This guide covered the interfaces available for the following environments: Windows 7, Windows 8 Desktop, Windows Modern UI, as well as apps running in web browsers. While Gestures and Manipulations are possible in Windows 7, app developers may find that the Windows 8 APIs (those targeted for the Desktop and/or for the Modern UI) offer the best options for automatic gesture recognition.
App developers who are writing touch-enabled web apps need to check their code for IE 10 since IE 10 has its own interface that must be used to process touch, gestures, and manipulation. Other Webkit-based browsers are also based on the DOM Level 3 standards and have touch and gesture event support.
This guide also covered descriptions of common Gesture and Manipulation interactions and provided some guidelines for developing touch-enabled apps.
This article was written by Gael Hofemeier. Gael is an Evangelist Application Engineer at Intel Corporation. Her focus is providing technical content that developers writing software for Intel® Architecture need. In addition to writing content, she also moderates the Business Client Forum on the Intel Developer Zone.
See Gael's Blog Author Page
Intel, the Intel logo, Ultrabook, and Core are trademarks of Intel Corporation in the US and/or other countries.
Copyright © 2013 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.
