ForestPad - a method for storing and retrieving textual information

Introduction

ForestPad is a method for storing and retrieving textual information and consists of three applications:

A PocketPC application (screenshot)
A Windows Desktop application (screenshot)
A Web Service application

Also included are two Setup projects. One for the PocketPC application that calls custom code which launches the ActiveSync application and one for a Windows Desktop version.

Before I go any further, I would like to pay homage to a wonderful program that I used for many years before deciding to write a replacement. The program is TreePad and it was written by Henk Hagedoorn . I would also like to thank CodeProject and all of the contributing developers for their hard work in creating one of the best resources on the web for information about .NET and C#.

Either ForestPadDesktop or ForestPadCE can be used independently. They become more powerful when used in combination with the ForestPadService which enables the PocketPC and Desktop applications to communicate over the web and syncronize data.

Currently, the Syncronize feature only checks to see which ForestPad document is newer. This means that data can be lost if you edit the same ForestPad document on two clients and then syncronize from each of them. Instead, you have to work in a "disconnected" mode. For instance, if you are editing data on the PocketPC in the field, you must syncronize the data to the ForestPadService, then syncronize before editing the same document in ForestPadDesktop.

This simple method works fine for me but could be extended in a variety of ways. You could add a check-in / check-out system similar to source control, you could syncronize data at the node level, or any other method that suites your needs. If you have a situation where you are always connected to the internet, you could even make the syncoronization automatic. If you only use one of the clients, you could still use the ForestPadService for the purpose of backup. The ForestPadService does not rely on a database so you can install it on any hosting account that supports the .NET Framework.

Background

I decided to write ForestPad as a way of storing all of the textual information that I need to recall. Here is the way that I use it:

I use it to store lyrics. When practicing guitar, I can quickly flip through the tabulature for the songs. (Ctrl-K) puts the cursor in the search box just like FireFox allowing me to quickly search for an item in the ForestPad document simply by paging using multiple presses of the enter key.

I store C# code fragments in it. This is allowed because the underlying file format is XML with CDATA sections. The only thing that currently can't be stored is an XML fragment that contains CDATA tags.

At work, I store IP addresses, urls, contacts, interesting programming articles, emails addresses, todo lists, the text of emails, you name it. If it is text and worth storing and retrieving, it goes into ForestPad. I also use it as a portable internet favorites application. To import a url, you just drag it from your browser onto the ForestPad Desktop application. Currently, links are not clickable as I implemented the TextBox control rather than the RichTextBox control. This decision was made because the .NET CF does not provide the RichTextBox yet (although I understand it is possible to access something like it through un-managed code.) You can also drag text from other programs to the TreeView, the main TextBox and the graphical buttons. In ForestPadDesktop, you can also select a section of text or a whole node and email the text if you have entered an SMTP server and a From address in the Settings section.

ForestPad design decisions

One of the things that always bothered me about TreePad was not a fault of TreePad but rather one of my own disorganization. I would constantly bury information so deep in a hierarchy that I would "lose" it. Also, there is a root node and for some inexplicable reason, it bothered me from a graphical perspective. I decided to follow a strict paradigm that both removed the root node and limited the depth of the hierarchy. In each ForestPad document, there are multiple forests which can contain multiple trees, branches, and leaves. This limited hierarchy, in my opinion, also makes the PocketPC version easier to use. Also, I decided to auto-name the TreeView node's Text property by displaying the text in the node up to the first carriage return. This allowed for quicker entry on the PocketPC as it is not necessary to name the node and made it easy to build an outline without minimizing the InputPanel.

The file format of TreePad seemed strange to me but one must take into account that it was designed around 1995. Each node had a number representing its level in the tree and was terminated using the following string:

<end node> 5P9i0s8y19Z

I would assume that Mr. Hagedoorn thought the sequence 5P9i0s8y19Z would be unlikely to appear in the text of a node and I think that was probably right, as I never had a problem with it during the time that I used TreePad.

A TreePad file with a root node and one sub-node looked like this:

<Treepad version 2.7>
dt=Text
<node>
name
0
<end node> 5P9i0s8y19Z
dt=Text
<node>
name
1
text
<end node> 5P9i0s8y19Z

For some interesting code that deals with parsing another file format, check out the TreePadConverter class. I have included a TreePad example file so that you can see how it works. The only limitation that I know of is that the TreePad file must only be 5 levels deep (a root node and four levels) so that it can be mapped into the forest, tree, branch, leaf format. The root node from the TreePad file will not be retained. If you have not used a Stack before and are curious about one of its many uses, it will be of special interest. (Note: The TreePadConverter has only been tested with "TreePad version 2.7" files)

I wanted to store ForestPad files using XML so that I could import the data into other progams and so that the file would retain its visual heirarchy when opened in UltraEdit .

So, in contrast to TreePad, the file format of ForestPad looks like this:

<?xml version="1.0" encoding="utf-8"?>
<forestPad
    guid="6c9325de-dfbe-4878-9d91-1a9f1a7696b0"
    created="5/14/2004 1:05:10 AM"
    updated="5/14/2004 1:07:41 AM">
<forest 
    name="A forest node"
    guid="b441a196-7468-47c8-a010-7ff83429a37b"
    created="01/01/2003 1:00:00 AM"
    updated="5/14/2004 1:06:15 AM">
    <data>
    <![CDATA[A forest node
        This is the text of the forest node.]]>
    </data>
    <tree
        name="A tree node"
        guid="768eae66-e9df-4999-b950-01fa9be1a5cf"
        created="5/14/2004 1:05:38 AM"
        updated="5/14/2004 1:06:11 AM">
        <data>
        <![CDATA[A tree node
            This is the text of the tree node.]]>
        </data>
        <branch
            name="A branch node"
            guid="be4b0993-d4e4-4249-8aa5-fa9c940ae2be"
            created="5/14/2004 1:06:00 AM"
            updated="5/14/2004 1:06:24 AM">
            <data>
            <![CDATA[A branch node
                This is the text of the branch node.]]></data>
                <leaf
                name="A leaf node"
                guid="9c76ff4e-3ae2-450e-b1d2-232b687214aa"
                created="5/14/2004 1:06:26 AM"
                updated="5/14/2004 1:06:38 AM">
                <data>
                <![CDATA[A leaf node
                    This is the text of the leaf node.]]>
                </data>
            </leaf>
        </branch>
    </tree>
</forest>
</forestPad>

Each item in the document contains a Guid , a Created date, and an Updated date. Text is contained within a CDATA section in the "data" node. Additional elements or attributes could be added to the ForestPad file format to allow futher functionality.

You can add nodes to the TreeView by clicking on one of the iconic buttons. ForestPad will add the node relative to the currently selected item. Nodes can be rearranged in ForestPadDesktop but currently can't be promoted or demoted. To delete a node, right-click on it and select Delete, or in ForestPadCE, hold the stylus on the node until the context menu appears.

Documents are stored in \My Documents\ForestPad on both platforms.

Summary of design goals

Overall, my design goals involved creating similar but even simpler program than TreePad with some fundamental differences.

XML-based file format
Limited node heirarchy
Auto-naming of nodes
Drag and drop of textual information
PocketPC and Desktop versions
Email text snippets from within ForestPadDesktop
Syncronization (and backup) of the data using a web service
Experimentation with a different style of user interface

About the code

ForestPadDesktop and ForestPadCE share some code. It is located in the ForestPadUtilities project. You will notice that there are several classes named xxxxxDesktop and xxxxxCE. The changes between these class are minimal and the code could have been combined using compiler switches. The disadvantage of this method is that there is duplicate code. The advantage is that it makes the build process smoother. Hopefully, Microsoft provides some obvious ommisions from the Compact Framework in the next release, such as the Guid class. Having to compile using the /unsafe directive is at odds with the concept of managed code.

If you haven't used the TreeView before, check out the ForestTreeNode class and the PopulateTreeView method in either ForestDesktop.cs or ForestCE.cs where you can see a ForestTreeNode object placed in a Tag attached to each TreeView node. This Tag contains a pointer to the corresponding node in the XmlDocument and provides a mechanism for keeping the TreeView and XmlDocument in sync.

public enum ForestType
{
    Forest = 0,
    Tree = 1,
    Branch = 2,
    Leaf = 3
}

namespace ForestPadUtilities
{
    /// <summary>
    /// Summary description for ForestTreeNode.
    /// </summary>
    public class ForestTreeNode
    {
        public ForestType NodeType;
        public string NodeName;
        public XmlNode NodePointer;
        public string NodeGuid;
        public string NodeCreated;
        public string NodeUpdated;

        public ForestTreeNode()
        {
        }
    }
}

Currently, you must download the source code to install the ForestPadService. At some point, I may build an installer.

In order to perform a full build of the code, you will have to modify the BuildCab.bat file in \ForestPadCE\BuildCabs\ to reflect the location of the files on your drive. The simplified command line is below:

cabwiz.exe ForestPadCE_PPC.inf /dest \ForestPad\ForestPadCE_Setup /err
    Logfile.log /cpu ARMV4 ARM SH3 MIPS X86 WCE420X86

You will also need to modify ForestPadCE_PPC.inf and update the paths. Read the CodeProject article Developing and Deploying Pocket PC Setup Applications for more information.

Build first in Debug mode and then in Release mode. I replaced the CAB files in the ForestPadCE_Setup project with empty text files to save space. They will be replaced with the actual CAB files during the first Release build.

There is a virtual directory to setup for this solution. Point a virtual directory to the ForestPadService directory. Give the ASPNET process full permissions to the ForestPadService\DATA folder. Then visit http://localhost/ForestPadService/Login.aspx and provide the Username: admin and Password: admin

Edit the admin user and create a new password. Then, in either ForestPadCE or ForestPadDesktop, in the Settings section, put http://localhost/ForestPadService/ForestPadService.asmx as the web service url, admin as your username and your new password. Save the changes, create a document, enter some text, and choose Syncronize.

Your data will transfer from the client to the ForestPadService.

Security

Security in ForestPad is minimal at the moment. It was not a major concern of mine when designing the program. Passwords for the ForestPad web service are encrypted within each application but currently the data stored on the ForestPad web service is not encrypted and the password is transferred to the web service in plain text. Adding encryption to the transaction would be a fairly trivial task. Also, it would be easy to store the ForestPad documents in a database, or to encrypt the XML.

You could also encrypt the information on the ForestPadCE and ForestPadDesktop clients. I chose not to do this as I wanted to be able to open the documents in a text editor. If you keep sensitive information, I would suggest making sure that both your PocketPC and your Desktop machine are password protected. For now, unless you modify the ForestPadService to include encryption, you could be putting yourself at risk by storing documents with it.

In the future, I will be releasing a version of the source at ForestPad.com that could be used to store secure information remotely.

Known open issues

ForestPadCE

Settings form displays instead of the main form sometimes. It seems to happen after you have opened and closed the settings form, then minimized the application and maximized it using the icon in the Start menu.
Problem with icon not showing up in 'Recent Programs" menu

ForestPadDesktop

Wierd resizing issue in the Desktop version -- on my system it grows vertically by 19 pixels on each launch. On other systems, acts unpredictably.

ForestPadService

Currently, ForestPad syncronizes using LastUpdated date. This means that data can be lost if the data is edited in two locations (e.g. ForestPadCE / ForestPadDesktop) and then synced from both locations. Data from the file that was edited longest ago would be lost.

Possibilities

This code could be used as the basis for many different types of projects.

For instance:

You could add a read-only attribute to the ForestPad User, and setup an identical Username and Password for everyone in an office to allow them all to receive ForestPad documents that you create. You could take the idea even further and allow the ForestPad User to store both their user specific documents and still receive the read-only syndicated documents. Or, modify the ForestPadService to redirect all the documents that a group of users create directly to you as a sort of data collection mechanism.

Add an option to ForestPadCE to store ForestPad documents on a removable memory card.

A search option could be added that would search through all of the ForestPad documents rather than just the one that is currently open. You could also extend the existing search to support regular expressions.

You could build a web-based viewer or editor for ForestPad documents.

Or, you could implement an option to transfer a ForestPad document between two PocketPC's using IR.