Gmail Agent API v0.5 / Mail Notifier & Address Importer

• Download source - 76 Kb
• Download setup - 175 Kb

What is it?

There are two distinct components here: an open source Gmail API written for the .NET framework, and a proof of concept Windows application built on top of that API that provides basic remote Gmail functions.

I developed these tools in the hopes of encouraging others to create interesting Gmail services. Admittedly, this project may not have a very long shelf life, as Sergey has intimated possible mail forwarding and RSS support, not to mention Gmail's recent listing of upcoming features (Gmail login required) that estimates a slew of features that are listed as “working on it” or “we'll try”. The address book import is currently listed as “sometime soon” but it's actually available now in the Contacts window. As Sergey mentioned, an enterprise version of Gmail would be well received, and I have no doubt that there would an API to go along with that (I recently integrated a Google Search Appliance, and can attest to its extensibility). Whether or not Google is interested in pursuing such features for the public side remains to be seen. Nonetheless, I hope to keep this project going, and wouldn't mind joining the Gmail team — there are lots of features I'd like to see implemented in Gmail.

About the Gmail Agent Applet

I'm sure most of you are more interested in the applet, so here are the features:

• Multiple account support
• Balloon notification of new messages with message preview
• Address book view and import (from tab-delimited text files)

The system requirements are:

• Windows 2000, or higher
• Microsoft .NET Framework v1.1

I haven't tried this with Mono — I doubt it works, but if it does, please let me know. If you are interested in POP access to Gmail, check out Pop Goes the Gmail (also a .NET project).

This is a proof of concept application, and there are plenty of idiosyncrasies. It works great for me, but your results may vary. I welcome anyone who wants to contribute to polishing this app.

About the Gmail Agent API

The goal of the API is to provide an extensible foundation for interfacing with Gmail. The objects in this namespace should be abstracted enough to be able to be adapted to any future changes Gmail makes.

Read the documentation for Gmail Agent API 0.5 to see what's available.

The main workhorse of this class is GmailAdapter. It is responsible for communicating with Gmail and maintaining the login and session information through the duration of the application. The GmailSession object holds all the state information for a single Gmail account, including a GmailThreadCollection of GmailThread objects. The GmailContact object represents a single Gmail address book entry. Again, GmailAdapter provides the methods to fill a GmailContact with information from Gmail.

Connection Overview

One oddity with the API is that it uses TLS instead of SSL for the encryption layer. For reasons unknown, the SSL provider was extremely intermittent and often failed to establish a secure link so I manually set the ServicePointManager to use TLS 1.0. This seems to be a common problem among .NET developers, and if anyone has a stable solution, I'd love to hear about it.

Here is a bare-bones example of how to establish a connection with Gmail using the API:

// init new adapter
GmailAdapter gmail = new GmailAdapter();

// create new session and assign username and password
GmailSession myAccount = new GmailSession();
myAccount.Username = "googler";
myAccount.Password = "showmethemoney";

// login and retrieve mailbox info
GmailAdapter.RequestResponseType loginResult = gmail.Refresh(myAccount);

// display mailbox info
if(loginResult == GmailAdapter.RequestResponseType.Success) {

    // show new inbox count
    Console.WriteLine("New Threads: " + myAccount.DefaultSearchCounts["Inbox"]);

    // if new threads exist, show the subject of the first one
    if(myAccount.UnreadThreads.Count > 0) {
        GmailThread newThread = (GmailThread)myAccount.UnreadThreads[0];
        Console.WriteLine("Latest thread subject: " + newThread.SubjectHtml);
    }
}

About the Gmail engine and protocol

You've probably noticed that Gmail's interface is extremely fast when compared to other web-based email systems like Yahoo! Mail and Hotmail. This is a result of Gmail's placement of the UI engine on the client-side as a JavaScript module. Whenever you log in to Gmail, a copy of the UI engine is loaded into one of the HTML page frames and remains there for the duration of your session (credit has to be given to Oddpost for being the first ones who perfected this idea). Subsequent actions from the Gmail interface are then routed through the Gmail UI engine in your browser, which in turn makes HTTP requests (via the XmlHttpRequest object) to the Gmail server, interprets the DataPack (more on this later), and updates the UI dynamically. In contrast, Hotmail and Yahoo! Mail follow traditional web application models and reload the entire UI after almost every action.

The item most relevant to this project is what I refer to as the “DataPack”, a base HTML file that contains only JavaScript array declarations that the UI engine parses and then uses to determine what to update. The advantages of this should be immediately obvious: reduced traffic load, and increased functionality — especially for developers who no longer have to resort to crude “screen scraping” techniques to interface with web applications. Although the ideal situation for external developers would be an XML-based DataPack, the JavaScript version is sufficient (and I suspect it was chosen for performance reasons as well).

The DataPack format consists of individual “DataItems”, or JavaScript arrays wrapped in a envelope function. An example:

D(["ts",0,50,106,0,"Inbox","fd36721220",154]);

The function D() references a runtime evaluator within the Gmail engine, which then interprets the attached array parameters. The "ts" element indicates that this is a threadlist summary item, and the subsequent elements denote start index, threads per page, estimated total, threadlist title, threadlist timestamp, and total threads. This is the same format that is applied to all array parameters sent through the DataPack:

[<DataItem_name>(,<custom_attribute>)]

The mappings to all the DataItems can be found in the engine code source (/gmail?view=page&name=js). For instance, qu contains quota information, while ct contains category (a.k.a. labels) definitions. Read through that file if you really want to get everything you can out of Gmail.

Determining the right URL to retrieve the DataPack is pretty straightforward, as most requests will return the same basic information, such as quota, category count, and inbox count. The main thing that changes is the threadlist summary, which depends on what page you're looking at. All the main folders — inbox, starred, trash, spam, etc. — are all really just pre-defined searches within Gmail. For example, the inbox DataPack URL is:

/gmail?search=inbox&view=tl&start=0&init=1&zx=

The search query for all unread threads is:

/gmail?search=query&q=is%3Aunread&view=tl&start=0&init=1&zx=

The main parameters are search= and q=, which define what set of threads the user is requesting. The zx= parameter is a proxy cache defeater, and I've omitted it here for brevity. See GmailAdapter.MakeUniqueUrl() for more information.

Gmail exploits another advantage of the DataPack model to increase efficiency by allowing for an empty document. This is employed by the 2-minute auto-refresh request. The inbox URL adds a few more parameters:

/gmail?view=tl&search=inbox&start=0&tlt=fd8dfa2e31&fp=c155594240dcc7cb&auto=1&zx=

The tlt= parameter is the thread list timestamp, which is treated like a checksum in determining the state of the client versus the mailbox state on the server. If the client timestamp is older than the one on the server, then a full DataPack is sent. Otherwise, Gmail sends an essentially empty DataPack.

Feedback

Feel free to contact me with any questions. Or, you can just leave a comment over here.