Original Virtual PiliKeyboard with Prompt Feature. Samsung R-Series laptop keyboard.

• Download demo - 24.44 KB
• Download source - 41.02 KB 

Russian version of the article you can find here: 

HowToDoIt 

Introduction

This article shows that it is really easy to write a relatively nice and friendly keyboard application. Here we can see the usage of WinApi as .NET methods. Sure, reading this article you may think: “Hey, guy! We have a lot of virtual keyboard applications!”. Of course, you are right, but… but the other keyboards have not embedded prompt features that will show you next possible letters after you pressed a key. It looks like that:

It is the result of some portion of statistical approach. Exactly, I mean Markov chain.

Background

I like to read books and sites’ content lying on the back with the mouse in my hand. Imagine a situation when you are looking at an interesting film and suddenly hear the “QIP” “autch-autch” sound. You need to write some callback like: “YES, I AM SLEEPING!!!”. But to do that, you need to get your feet on. I do not really know how it sounds but it was a key reason to write the application I am presenting in this article.

Using the Code

The project is written using a free and really cool development software “SharpDevelop 3.0.0.3800”. Truly speaking I have done my best to write intuitive and welcome code with enough (I think so portion of comments. I have not applied refactoring in C-style like:

• Before refactoring: getSortedIndexesBubblesort1D
• After: gtStdIdxBblrt1D

Hope you will feel yourself free reading my code.

Let us go through the source code… The main problems I faced were:

• Drawing interface without any kind of unwanted shifts between buttons
• Creating of Markov chain matrix
• Desire to lose the focus of our keyboard application

Drawing the Interface

What about the first, it was not really a problem. “Time beats the pain!”. Some time and I just took a ruler and measure my own buttons. There were not so many measurements such as we have only several types of buttons for “Samsung R40” laptop. They are listed in the next enumeration:

enum KeyType
{
    EMPTY,
    REGULAR,// like A...Z
    LITTLE,// like F1-F12
    TAB,
    SHIFTL,
    SHIFTR,
    CAPSLOCK,
    BACKSPACE,
    SLASH,// like \
    ENTER,
    SPACEBAR,
    ALTL,
    ALTR,
    CTRLL,
    CTRLR
};

The next step was the creation of the buttons. It was really difficult to me . After that, I could load images for free.

// loading of the keys' images
Image imgTmp = null;
Image imgTab = Image.FromFile(pathTab);
Image imgAltL = Image.FromFile(pathAltL);
Image imgAltR = Image.FromFile(pathAltR);
Image imgCtrlL = Image.FromFile(pathCtrlL);
Image imgCtrlR = Image.FromFile(pathCtrlR);
Image imgSlash = Image.FromFile(pathSlash);
Image imgEnter = Image.FromFile(pathEnter);
Image imgShiftL = Image.FromFile(pathShiftL);
Image imgShiftR = Image.FromFile(pathShiftR);
Image imgLittle = Image.FromFile(pathLittle);
Image imgRegular = Image.FromFile(pathRegular);
Image imgSpacebar = Image.FromFile(pathSpacebar);
Image imgCapslock = Image.FromFile(pathCapslock);
Image imgBackspace = Image.FromFile(pathBackspace);

The algorithm to draw a keyboard is as follows:

• Draw each key image in a cycle
• Draw text to each key in the same cycle

The code for this part (DrawKeyboard(…) method) is of a large size so I will not represent it here, but you can always analyze it from the attached source code.

By the way, DrawKeyboard(…) method uses 2D array keyboardStructure that is made of KeyType. The content of this array is:

PiliKeyboard.MainForm.KeyType[][] keyboardStructure =
{
new KeyType[] {
		KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE,
		KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE,
		KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE,
		KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE,
		KeyType.LITTLE },

new KeyType[] {
		KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.BACKSPACE, KeyType.REGULAR },

new KeyType[] {
		KeyType.TAB, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.SLASH, KeyType.REGULAR },

new KeyType[] {
		KeyType.CAPSLOCK, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.ENTER, KeyType.REGULAR },

new KeyType[] {
		KeyType.SHIFTL, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR,
		KeyType.REGULAR, KeyType.REGULAR, KeyType.REGULAR, KeyType.SHIFTR,
		KeyType.REGULAR },

new KeyType[] {
		KeyType.REGULAR, KeyType.CTRLL, KeyType.REGULAR, KeyType.ALTL,
		KeyType.SPACEBAR, KeyType.ALTR, KeyType.REGULAR, KeyType.CTRLR,
		KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE},

new KeyType[] {
		KeyType.LITTLE, KeyType.LITTLE, KeyType.LITTLE}
};

Here I was “sliding” through the keyboard left-to-right and up-to-down filling this 2D array.

Creating Markov Chain Matrix

This part also was not difficult for now. Maybe because of the fact I have developed it many years ago when I was writing a diploma at the university.

The idea is trivial. The first step is to find out how many times each letter happens after given. It is the routine of getProbabilityMatrix(…) method. In the result, we will get the matrix of relative values (frequencies).

private static float[,] getProbabilityMatrix
	(string sDictionaryPath, string sInterestSymbols)
{
    long symbolsNum = 0;

    float[,] probabilityMatrix = 
	new float[sInterestSymbols.Length, sInterestSymbols.Length];

    probabilityMatrix = new float[sInterestSymbols.Length, sInterestSymbols.Length];

    // reading all the dictionary into the buffer
    string sDict = "";

    using (TextReader tr = new StreamReader(sDictionaryPath, Encoding.GetEncoding(1251)))
    {
        sDict = tr.ReadToEnd();
    }

    // making all the symbols from dictionary as lower cased
    sDict.ToUpper();

    for (int s = 0; s < sInterestSymbols.Length; s++)
    {
        for (int i = 0; i < sDict.Length - 1; i++)
        {
            if ((sDict[i].ToString().ToUpper() == 
			sInterestSymbols[s].ToString().ToUpper()))
            {
                for (int j = 0; j < sInterestSymbols.Length; j++)
                {
                    if (sDict[i + 1].ToString().ToUpper() == 
				sInterestSymbols[j].ToString().ToUpper())
                    {
                        probabilityMatrix[s, j] += 1;
                    }
                }
                symbolsNum++;
            }
        }
    }

    return probabilityMatrix;
} 

The next task is to sort these data to get something like: “After letter ‘a’ we have letters ‘f’, ‘r’, ‘m’ and so on”. This entire job is fulfilled in getSequenceMatrix(…) method.

//method to get sequential matrix
public static string[,] getSequenceMatrix(string sDictionaryPath, string sInterestSymbols)
{
    if (!File.Exists(sDictionaryPath))
        return null;

    float[,] probabilityMaxtrix = 
		new float[sInterestSymbols.Length, sInterestSymbols.Length];
    string[,] sequenceMatrix = 
		new string[sInterestSymbols.Length, sInterestSymbols.Length];

    float[] dataTmp = new float[sInterestSymbols.Length];
    int[] indxsTmp = new int[sInterestSymbols.Length];

    probabilityMaxtrix = getProbabilityMatrix(sDictionaryPath, sInterestSymbols);

    for (int r = 0; r < sInterestSymbols.Length; r++)
    {
        for (int i = 0; i < sInterestSymbols.Length; i++)
        {
            dataTmp[i] = probabilityMaxtrix[r, i];
        }

        getSortedIndexesBubblesort1D(ref dataTmp, ref indxsTmp);

        for (int c = 0; c < sInterestSymbols.Length; c++)
        {
            sequenceMatrix[r, c] = sInterestSymbols
		[indxsTmp[sInterestSymbols.Length - c - 1]].ToString().ToUpper();
        }
    }

    return sequenceMatrix;
}

Using these methods, we can see that the first input parameter is path to the dictionary. Let’s consider these dictionaries. What is the ideal vocabulary for the final user? It should be not little but big enough set of frequently used words. In this case, we will get the higher accuracy of the next letters prompting. Anytime, we can add word to our “database” simply by opening the “*.txt” file from “./Dictionaries/” folder. These files may be user specific and include some kind of abbreviations if needed. By the end, to control how many letters will be highlighted managed by numOfSuppKeysToShow integer variable. In my case, this number equals to “2” but you can set your value.

“Focusing” and “unfocusing” Problems

The meaning of this problem is in fact that while you are clicking the keyboard application, it catches the focus and SendKey() command will be sent to it in turn. To resolve this unpleasant situation, I decided to get the handle of the target window (window to where symbols should be typed) while mouse is moving under the unfocused virtual PiliKeyboard application and it works fine.

// method to get handle of the target window while the PiliKeyboard inactive
private void MainForm_MouseMove(object sender, MouseEventArgs e)
{
    // when we move cursor under the PiliKeyboard 
    // we are getting pointer to the target window
    IntPtr IntPtrTmp = CWinApi.GetForegroundWindow();

    if ((IntPtrTmp != hWndSource) && (IntPtrTmp.ToInt32() != 0))
    {
        hWndTarget = IntPtrTmp;
    }

    // updating input language var
    lCurrentLanguage = GetKeyboardLayoutLang();

    switch (lCurrentLanguage)
    {
        case LANG.UK:
            sCurrentLanguage = UK;
            break;
        case LANG.RU:
            sCurrentLanguage = RU;
            break;
        case LANG.EN:
            sCurrentLanguage = EN;
            break;
    }
}

Customization

The idea of customization lies in the fact that the user can simply change his own buttons style just by creating new or modifying existing set of buttons. One restriction! The sizes for the buttons should be the same as the original. An example of such a modified and a little crazy interface:

Finally

I would like to tell you that this application was made in a limited portion of time and it is not fully tested. So, it may be a bit buggy. I will appreciate your assistance and advice.

P.S. To change keyboard system layout language, just do right button mouse click.

Points of Interest

In the future version, it would be great to implement dropped-out list of possible words depending on the previously typed letters. In this case, we will get an increase of typing speed that may be commensurable to the “normal-way” input.

History

• 11^th November, 2011: Initial version