Touch Screen Explorer with Pen Flicks

• Download executable PenFlicks - 158 KB
• Download PenFlicks source - 452 KB
• Video demo in Ultra Mobile PC (Samsung Q1)/ Watch live(Avg. quality, taken by a phone)

Introduction

Pen Flicks is the latest methodology in Microsoft Windows Vista™ which is designed specifically for a Ultra Mobile/Tablet PC. They are gestures you can make with your tablet pen to quickly navigate and perform shortcuts.

This article describes a Touch Screen Windows Explorer which will leverage this new technology and will compare it with the traditional Windows Explorer. Here is a screenshot:

The Goal

The goal is to create an efficient Touch Screen Windows Explorer which can be used in the Tablet PC / Ultra Mobile PC environment, where mouse and keyboards are barely available.

• Big display for ease of use pertaining to small screens in Ultra Mobile PC
• Browse feature with a pen or fingertips in Touch Screen/Ultra Mobile PC
• Navigation using the forward, backward Pen Flicks, instead of mouse point and click
• Scroll the Explorer using Up and Down Pen Flicks without traditional scrollbars

Background

Pen Flicks are quick, linear pen movements associated with scrolling actions and commands. Pen Flicks allow quick pen access to navigation and editing actions.

Pen Flicks in action: standard up and down Flicks.

There are two categories of Pen Flicks: navigational and editing. Navigational Pen Flicks include drag up, drag down, move back, and move forward. Editing Pen Flicks include copy, paste, undo, and delete. For example, you can drag up or down on a page, move back or forward in a browser window, or paste an item into a document, all with a flick of your pen.

Pen Flicks settings can be found in the Control Panel of Windows Vista. You can see the details inside Pens and Input Devices. The feature works only with the Tablet PC / Ultra Mobile PC with Microsoft Vista. Click here to see the screenshot.

Windows Vista includes a set of eight basic Pen Flicks. See image below.

• Scroll up and down
• Navigate forward and back
• Copy
• Paste
• Delete
• Undo

These is the default set for Pen Flicks, and can be customized using the Pen Flicks API. If you don't override the feature, these functions will be called.

You can customize Pen Flicks to perform other functions, which increases your efficiency while making pen use feel more natural. To make pen training easier, Windows Vista includes a tutorial that presents the essentials of using a tablet pen to perform these shortcuts.

How the application talks with Pen Flicks

The Windows Vista operating system uses two approaches to interact with the application when a pen flick occurs, and they are in the following order:

1. Windows Vista sends a WM_TABLET_FLICK message. The FLICK_DATA structure and FLICK_POINT structure, along with the message, provides all the information about the pen flick, like screen coordinates, action for the pen flick, the direction, and the state of modifier keys such as Ctrl and Shift etc.

2. Windows Vista sends follow-up notifications WM_APPCOMMAND, WM_VSCROLL, or WM_KEYDOWN etc., depending on which action is associated with the pen flick.

A client application can process the WM_TABLET_FLICK message using the CWnd::WindowProc function, or can offer a response to the pen flick using the fact that Vista will send the action in the form of an APPCOMMAND with the backup keystroke. In this example, we will use the latter to interact with the Pen Flicks.

WM_TABLET_FLICK Message details (unmanaged):

WM_TABLET_FLICK
    WPARAM wParam
    LPARAM lParam

Parameters

• wParam: A FLICK_DATA structure that contains information about the pen flick that occurred.
• lParam: The FLICK_POINT structure that specifies where the pen flick occurred.

The FLICK_DATA structure contains information about a pen flick.

typedef struct FLICK_DATA
{
    FLICKACTION_COMMANDCODE iFlickActionCommandCode:5;
    FLICKDIRECTION iFlickDirection:3;
    BOOL fControlModifier:1;
    BOOL fMenuModifier:1;
    BOOL fAltGRModifier:1;
    BOOL fWinModifier:1;
    BOOL fShiftModifier:1;
    INT  iReserved:2;
    BOOL fOnInkingSurface:1;
    INT  iActionArgument:16;
}FLICK_DATA;

The FLICK_POINT structure provides information about a pen flick.

typedef struct FLICK_POINT
{
    INT x:16;
    INT y:16;
}FLICK_POINT;

The following is a list of application commands that can be assigned to flicks, with the backup keystroke message that might be sent.

Using the code

In this example, I am using the APPCOMMAND_BROWSER_BACKWARD and APPCOMMAND_BROWSER_FORWARD commands to browse up and down a level in the touch screen explorer, by calling Alt + Back Arrow keys for backward and Alt + Forward Arrow keys for the forward command.

// the user control

public TouchScreenExplorer()
{
    InitializeComponent();
}

protected override System.Windows.Forms.CreateParams CreateParams
{
    get
    {
        CreateParams Cp = base.CreateParams;
        Cp.Style = Cp.Style & ~0x200000;
        return Cp;
    }
}

public void ResetFolder()
{
    listBox1.Items.Clear();
    imglpreview.Images.Clear();  

}

public void SetFolder(ListView myView, ImageList imglp)
{
    explorerListItems = myView;
   listBox1.Items.Clear();
   imglpreview.Images.Clear();
    //imglpreview = myImageList;

    int i = 0;    
    ImageCount = 0;
        
    
    for (int j = 0; j < myView.Items.Count; j++)   
    {
        try
        {
            imglpreview.Images.Add(imglp.Images[j] );
            listBox1.Items.Add(new Controls.Development.
            ImageListBoxItem(myView.Items[j].Text, j));
        }
        catch(Exception e1)
        {
            MessageBox.Show(e1.Message + i);   
        }

    }
}

The default Up and Down Flicks can be directly used to scroll up and down in the Touch Screen Windows Explorer. To navigate a level up and down in the Explorer tree, I am explicitly checking if the calling Pen Flick is causing Alt + Left arrow for backward action or Alt + Right arrow for forward action, which is again caused by the default flick left and right action. Here is a portion of the code, see attached file for details:

public void NextImage()
{
    if (listBox1.SelectedIndex < listBox1.Items.Count-1)
    {
        listBox1.SelectedIndex = listBox1.SelectedIndex +1;
        ImageChangedEvent(this,EventArgs.Empty);  
    }
}

public void PreviousImage()
{
    if (listBox1.SelectedIndex >0)
    {
        listBox1.SelectedIndex = listBox1.SelectedIndex - 1;
        ImageChangedEvent(this,EventArgs.Empty);  
    }
}

private void listBox1_MouseUp(object sender, MouseEventArgs e)
{
    if (e.Button == MouseButtons.Right ) 
    GoBackEvent(this, EventArgs.Empty);

}

private void listBox1_KeyDown(object sender, KeyEventArgs e)
{
    if (e.Alt && (e.KeyCode == Keys.Left) )
        GoBackEvent(this, EventArgs.Empty);

    if (e.KeyCode == Keys.Back)
        GoBackEvent(this, EventArgs.Empty);

    if (e.Alt && (e.KeyCode == Keys.Right))
    {
        ResetFolder(); 
        GoForwardEvent(this, EventArgs.Empty);
    }
    if (e.KeyCode == Keys.Enter)
    {
        ResetFolder();
        GoForwardEvent(this, EventArgs.Empty);
    }

}

Three screens of the Touch Screen Explorer tree

A backward Flick in the second screen will display the first screen, and a forward Flick the third screen.

About the Explorer tree, I have made a Touch Screen Explorer control based on a Windows Explorer control from here and here. The Explorer control DLL can be found along with the source code. Here is a piece of code which shows the TreeNode Selected event of the traditional Explorer tree.

private void expTree1_ExpTreeNodeSelected (string pathName, CShItem CSI)
{
    ArrayList dirList = new ArrayList();
    ArrayList fileList = new ArrayList();
    int TotalItems;
    LastSelectedCSI = CSI;
    if (CSI.DisplayName.Equals(CShItem.strMyComputer)) {
        dirList = CSI.GetDirectories(true );
        // avoid re-query since only has dirs

    }
    else {
        dirList = CSI.GetDirectories(true);
        fileList = CSI.GetFiles();
    }
            
    SetUpComboBox(CSI);
    TotalItems = (dirList.Count + fileList.Count);
    if ((dirList.Count > 0))
    {
        
        sbr1.Text = "Location: " + pathName + ", "
         + dirList.Count + " Directorie(s) and " + 
         fileList.Count + " File(s)";

        ArrayList combList = new ArrayList(TotalItems);
        combList.AddRange(dirList);
        combList.AddRange(fileList);
        lv1.BeginUpdate();
        lv1.Items.Clear();
        imglpreviewForm.Images.Clear();

        foreach (CShItem item in combList) 
        {
            ListViewItem lvi = new ListViewItem
            (item.DisplayName);

           
            if (item.IsFolder)
            {
                CShItem localitem = item;
                imglpreviewForm.Images.Add(SystemImageListManager
                .GetIcon(localitem.IconIndexNormal, false));
                lvi.ImageIndex = SystemImageListManager
                .GetIconIndex(ref localitem, false, false);
                lvi.Tag = item;
                lv1.Items.Add(lvi);
            }
            
        }

        lv1.EndUpdate();
        myPhotos1.SetFolder(lv1, imglpreviewForm);
        myPhotos1.CaptionText = CSI.DisplayName; 
    }
    else {
        lv1.Items.Clear();

    }
}

In action

Thoughts

The Touch Screen Windows Explorer uses the latest technology, Pen Flicks, and gives more mobility to the user. It makes a Tablet PC/Ultra Mobile PC efficient by giving it the independence of keyboard and mouse. How different would have been the interface if there were never a keyboard or mouse ? What if scrollbars were never invented? These are some of the questions to get you started on interface interaction, when you develop your application for Tablet PC/Ultra Mobile application. Scrollbars are out, Point & click is out, Keyboard and mouse are out, Pen Flicks is in, Touch Screen in.

References

• Watch the Ink show about Pen Flick/ Download
• Flick Gestures, Flick API Reference, Pen Flicks
• What's new in Windows Vista for Tablet PC
• Windows Explorer in a user control - Quartz
• VB.NET Version Explorer Tree - Jim Parsells

And thanks

For coming so far! I hope you find this useful, and take care.

• Apr 15, 2007: First published
• May 02, 2007: Added demo video
• May 31, 2007: Content revised