This article discusses how to create and use a web browser (IE only) compatible ink or drawing control using the Microsoft Tablet PC SDK version 1.7. A sample pre-built control is provided for use and demonstration. Also, please read the disclaimer at the end in case you have any issues with this article.
Having seen Microsoft's Tablet PC ink capabilities in action, I found it sad they didn't provide web or ASP.NET support of their ink libraries. That is until recently — with the release of the Tablet PC SDK version 1.7, Microsoft provides a way to create an ink user control that can be embedded in a webpage and interact with the web server via web scripting, etc.
There are several articles on the web (and a sample in the SDK) that talk about this new feature and how to build such a control. However, I found the documentation/articles to be lacking in some respect due to confusing presentation of how to deploy such an application. Furthermore, no complete source code (other than the sample in the SDK) is provided for evaluation. Only minor snippets are shown in the articles.
Thus, this article skips over the part on creating the control (you can refer to the reference websites below for that information) and instead focuses on:
- the deployment part using a sample demo control project, and
- possible (creative?) uses plus pros and cons for such a control.
The sample control is also pre-built with common drawing and inking features included so that you can use it as is, saving you time from having to write one up from scratch. Or you can also extend the control with additional properties & methods or modify the existing methods.
Code Magazine article: Ink on the Web
Mobile Ink Jots 3: Ink on the Web
Microsoft Tablet PC - Creating a Redistributable Setup
Creating Interactive Hand-Written Web Pages on Tablet PC
Code Magazine article: Introduction to Tablet PC Development
Code Magazine article: Ink Recognition and Ink Analysis
Ink Control Overview
We'll begin the article from the point after creation of the user control. First, some brief points to mention:
- The control can be implemented with interaction between itself and the external environment by (1) placing code for interaction as components/controls within the control or (2) placing code for interaction as public methods that are a part of the control that can be called from the external environment. In terms of making the control web usable, option two provides better customization and extendability, and better resizing of the control. Thus, the sample control project uses option two.
Using the Control
The following is required to run or execute the control on a browser client:
- .NET Framework 1.1 Runtime (or SDK) or greater.
- Internet Explorer (IE) 5.5 or greater
- Microsoft Ink Library version 1.7* or Windows XP Tablet PC w/ SP2 installed
- Mouse or other pointing device. A Windows XP Tablet PC system is recommended.
* You can get the Microsoft Ink Library version 1.7 by installing a custom Microsoft Ink version 1.7 library installer or from installing the Microsoft Tablet PC SDK version 1.7. Windows XP Tablet PC with Service Pack 2 also includes the library by default. Sadly, deploying the control over the web does not allow loading the reference libraries along with the control, unlike a windows form application where the Ink library can reside (without installation) in the same directory as the application. For developing & modifying the control, you will need to use the SDK (even if you have Windows XP Tablet PC w/ SP2).
Embedding & Loading the Control
To embed & load the control in a browser, you simply insert a special block of code utilizing the HTML object tag into a webpage. The tag should be similar to the following format (it worked for me, while some examples in the other articles didn't).
<object id="NameForObject" height="aValue" width="aValue"
With this code, the object will run upon the browser loading the webpage. You can't do much with it at this point, so you can go to the next step.
Interacting with the Control on the Client Side
The following code listing shows the available methods in the demo control. Not all are available for use over the web due to security restrictions, unless you add the website as a fully trusted site in Internet Explorer. NOTE: Not all methods were fully tested and some may not behave as you would expect.
public bool AntiAlias
public bool FitToCurve
public bool IgnorePressure
public void SetPenColor(string usrColor)
public void SetBackgroundColor(string usrColor)
public int PenTipStyle
public float PenWidth
public float PenHeight
public int Transparency
private bool SetToInkMode()
private bool SetToSelectMode()
private bool SetToDeleteMode()
public void ClearInk()
public bool CopyAsSerialInk()
public bool CopyAsBitmap()
public bool CutAsSerialInk()
public bool CutAsBitmap()
public bool Paste()
public void Shear(float x, float y)
public void ScaleInk(float x, float y)
public void Rotate(float degrees, int x, int y)
public void MoveInk(float x, float y)
public string InkToTextBestGuess()
public string InkToTextResults()
public string ReturnInkAsBase64Gif()
public byte ReturnInkAsGif()
public string ReturnInkAsBase64SerializedInk()
public byte ReturnInkAsSerializedInk()
public bool LoadInk(byte inkData)
public bool LoadInkFromBase64(string b64InkData)
public bool LoadBackgroundImage(string filePath)
protected void Dispose()
Interacting with the Control on the Server Side
First, implement the technique to retrieve the control's data and store it in a hidden form field.
<input type="button" value="Save as GIF" name="SaveCmd" onclick="SaveAsGif()">
<input type=hidden name="SavedInkData" value="empty">
Then, implement the technique to send the data to the server as a Base 64 encoded string. The demo project includes a simple server processing sample that just decodes & outputs the GIF image to the browser. In practice, you could further manipulate the image, store to a database, write to file, or send somewhere via email.
Notes on Deploying the Demo Application and About the Source Files
The demo application was created in the simplest fashion, and is not a standard Visual Studio .NET web application. To deploy the demo, you need only copy it to your web server. No special Visual Studio .NET or IIS virtual directory setup is required. The server side script is available in ASP.NET or PHP, which means you can deploy this application to IIS with ASP.NET (or PHP) or any web server that supports PHP. Only the browser clients need to support the .NET Framework.
For any ASP.NET configuration files and settings that are needed in your case, you may use your existing configuration files or create them yourself by manually creating them or using Visual Studio .NET to generate them.
The source is a Visual Studio .NET 2003 C# control project.
NOTE: The control works fine when used in a web application like the one described here but there may be problems when using it in a Windows Form/Desktop application. I tried dropping the control into a Windows forms project and just compiling it and I got errors (or maybe I didn't add the control to the application correctly). So if you plan to use it in such projects, you may have to tweak the control to get it to work.
IIS Deployment Issues
You may encounter some problems when deploying to IIS, particularly IIS 6 on Windows 2003 Server. Here are some tips on resolving such problems. Try them in the specified order:
- Verify in IIS, that your application has "Script only" execute permissions under your Virtual Directory, Home Directory, or Directory settings. If you have it set to "Scripts and Executables" you will not be able to download the web ink control DLL. If you must use "Scripts and Executables" because you are using CGI, you may need to rethink how you plan to deploy this application. This method solved my IIS problem. Click here for more info.
- Verify that you are not using URLSCAN (from Microsoft) or something similar on IIS. If you are using such a tool, you must configure it to allow downloads of DLL files. Click here for Using URLScan on IIS.
- Try lowering your IIS server security settings, if necessary. Probably not a good idea though, and I don't think you will really need to do this.
- When all else fails & you really like using this control & you are not required to use IIS/ASP/ASP.NET, consider hosting the application on a non-IIS web server like Apache.
Apache Deployment Issues
Alessandro Torrisi pointed out a potential issue on Apache (whether running server on Windows or Linux, to be safe) where the ink control DLL was not loaded into the application. According to Alessandro, to fix this, just add "application/x-msdownload dll" in the "/etc/apache2/mime.types" file and restart apache.
Points of Interest
Summary of Pros & Cons
I think such a control has potential in some applications. The negative aspects are that the control runs in a Microsoft client environment only, it requires downloading & installing the Microsoft Ink library, and text recognition is available only for Tablet PCs. These aspects might deter people from using & deploying the control. However, there are some positive aspects of this control too, particularly in the possible uses that the control can provide as well as the option of using any server side platform (it need not be Microsoft). In summary, while the control may not be feasible for general public & corporate use, it may prove useful in certain specialized applications. Plus it's a fun thing to toy around with in your spare time.
Possible (Creative?) Use Cases
- Inking or drawing on the web, as an alternative to using a Java applet or Macromedia Shockwave/Flash implementation. A good example of a Shockwave version can be seen at Tutor.com's Live Homework Help information site. I used to be a Tutor.com tutor and they do have a very nifty web whiteboard interface.
- Provide ink & drawing support (via the control on the client side) to ASP.NET applications or web services running on the Mono open source .NET platform (and Microsoft .NET too).
- Provide drawing capabilities to non-Microsoft and legacy web server platforms:
- Java, J2EE, EJB, servlets, JSP
- PHP (sample provided in demo project), Perl, Python, Ruby
- Classic ASP, ColdFusion
- Standard HTML/DHTML web pages only, Microsoft HTA applications
- Interact/integrate with Macromedia Shockwave/Flash plugin or Java applet for more capabilities?
- Provide ink recognition processing on the server side using the control on web browser clients & a Tablet PC as the web or application server (even better if the clients were running on Tablet PC too of course).
- Use the control as inkable form fields:
- Each instance of the control replaces a textbox in a form & on submit, the server processes the ink data along with the other form data.
- Or use the control like a Tablet PC Input Panel & on click, the ink is converted to text & can then be copied & pasted to the appropriate fields. NOTE: this option is only available on Tablet PC systems.
- Support digital signatures over the web (as GIF image or MS Ink format). Though this might present some legality, security, & privacy issues.
- Implementing a non-realtime digital whiteboard for conferencing--after drawing on the board, a user submits the data to the server for intermediate storage. Then either periodically/automatically by the application or manually performed by receiving users, the users can get an updated snapshot of the shared board (or the submitted user's board) loaded into their boards from the server. This is just my thoughts & hasn't been tested. It may be tricky to implement, if possible at all. But if it is possible, this is a neat thing to do.
- Use this control along with AJAX to make nifty dynamic web applications?
- Customized ink applications using the control
- Extend control to return ink in bitmap, JPEG, and other formats
- Finally, can you provide other ideas?
Updated Features and Information - as of Oct. 27, 2007
- Added new methods to the ink control to load ink from base 64 encoded data and to load a background image (primarily for Windows forms applications only).
- Added client side code for backup & restore of ink functionality. Backup feature only backs up current ink and overwrites previous backup. Backup uses the same hidden form field
SavedInkData, so when you "Save" or send ink to server, current backup gets overwritten as well. You can modify it so that backup uses a different form field instead, etc. Restore simply loads ink from the current backup. NOTE: You may have to click "Restore" twice to get it to load the backup, that's what happens on my system for some reason.
- For debugging purposes, I added a textbox with the same name as hidden form field "
SavedInkData" so that you can see and modify the base 64 encoded version of ink data. By default, this new field is commented out. You can switch to this one instead of the hidden form field for debugging.
- I added a new web server script that takes a supplied image URL, downloads it (to the server), converts it to Base 64 text, and returns that to the browser/web client for use by the web ink control. Unfortunately, this new script is currently available for ASP.NET only, no PHP.
- I added a new client side feature of using
XmlHttpRequest to pass a URL of an image to the server and get back its Base 64 encoded version so that you can load it into the web ink control. Unfortunately, I couldn't get it to load in the ink control. In any case, this is a good sample of how you can use
XmlHttpRequest to fetch data from server to load into your ink control, etc. I tried to use fetch from image URL, but you can fetch from database or file on server, etc. Also to note, I made use of my custom
XmlHttpRequest library for this (but you can use
XmlHttpRequest natively, for those of you who are proficient enough with it). See my other Code Project article titled "Web/HTTP Automation Libraries" for more information.
- See also my comment/forum post to this article for additional updates on web ink, web ink recognition, and IE and Firefox support.
- Scrolling causes problems with accessing form elements & selecting text, etc. This might be a browser or .NET rendering problem. Therefore, if you use a lot of form controls or have a lot of content below or above the control, etc., it is recommended that the webpage be viewed in full screen browser mode OR in portrait orientation (for Tablet PCs only) to minimize scrolling. Otherwise, consider having less controls or content on the page.
- Not all implemented control methods are supported when running the control over the web & some don't necessarily work as intended, so be aware of that.
- Restore feature may require you to click twice to load ink from backup.
- Load Ink from URL feature currently not working with ink control. Kept in demo project as an example of how to attempt this.
- This project might not work correctly with recent versions of IE and Windows, since original posting of this article.
- 12/07/2014 - add functionality and code snippet to export GIF image locally to an image element via data URI and the base64 encoded value of the ink GIF. Also cloned a version of the webpage as a Microsoft HTA file.
- 06/27/2006 - Initial Release. Some future enhancements may include additional output formats & built-in web request/response methods for input/output of data between control and a server.
- 07/04/2006 - Updated article & download files to clarify some things.
- 11/11/2006 - Updated article with information on deployment issues with IIS.
- 10/27/2007 - Updated article and code for issues and new features.
NOTE: This article and accompanying code are not entirely original. Rather, this article is intended to clarify and elaborate on the subject where the original articles seem to be lacking in detail. Additionally, source code (a compilation of code snippets from the original articles integrated together with my own code and extra functionality) and a demo application are provided for you to use freely. The original articles only provided code snippets, no complete code that you could use off the bat. Last, this article is intended to suggest and foster creative use of such an ink control beyond the standard Microsoft convention/platform.