Hotmail Exposed

• Download source - 4.13 Kb

Introduction

Hotmail is probably the most popular e-mailing facility found on the web today. Millions of people around the world use it for everyday communication with friends and relatives. The main reason for it’s popularity is the fact that you can use the service absolutely free of charge. Users are able to read their e-mails daily using a web interface and there is a client program available, called Outlook Express, which most people will only use at home. Only these two methods are officially available to read mail from your Hotmail account. This is strange because Outlook Express is not the safest client to use for reading e-mails, especially the spam-sensitive Hotmail accounts. Secondly, web-mail is generally considered to be an annoyance, that is only there to offer access to your account from locations such as schools and public Internet access points.

No longer! This article will enable you to build your own client, using a sure and solid way to communicate with Hotmail in the same way as Outlook does. It will be shown how the protocol can be used to your own advantage, it isn’t at all hard to understand either.

To build a working client we will need to know about the way Outlook communicates with Hotmail. The protocol that is being used is called HTTPMail. Even when completely unfamiliar with this protocol, it is not hard to understand. It looks like an XML structure in which some elements are placed that you want to find. The server will respond with the same kind of structure with the queried elements filled with information. I will not start explaining how the protocol looks, if you want to find out, it’s available from SourceForge.

Now that the manner of communication is clear, there are some other things that need clarification. For instance; how do we authenticate ourselves, and how the heck are we going to implement such authentication? Usually this will mean calculating hashes and keeping passwords in a safe place. At first sight, this could seem like an enormous task to implement. It turned out to be not so difficult at all.

After a review of the SourceForge article, a few things are noticeably interesting. The authentication is done via HTTP headers and is described in an RFC (2617). It’s even called HTTP authentication, there have got to be some classes in the .NET framework that are able to do that! To parse the response in a safe manner, we will make use of XPath, it should be up to the job and there are classes available for that too!

Building the client

To build the client, two components need to be created:

• A proxy that is able to ‘speak’ HTTPMail and is able to do HTTP authentication as described in the RFC.
• The actual client that accesses Hotmail via the proxy and uses XPath for parsing of the responses.

Instead of building a custom proxy, another method could be used. Microsoft ships a component called XMLHTTP that is able to make XML based HTTP requests. The use of this class presented several problems:

1. A request for a limited amount of properties, such as the msgfolderroot, returns all the possible responses for that request. So a query to obtain only the msgfolderroot would return information about all the mailboxes. This isn’t fatal of course but it does show that the class could use some work. The proxy class that is built in this document does not show this problem.
2. The component generates weird looking HTTP packets. For instance, the query is placed in the HTTP headers, it should be sent as a data section. This will probably result in erroneous server responses, and may be causing problem 1.
3. Most importantly; the component will not work for some e-mail addresses, while the only configuration that can be made is the username and password. This problem is the biggest show-stopper when using XMLHTTP.

Before reading the rest of this article, I assume that you have basic knowledge of the following subjects:

• HTTPMail
• XPath
• Basic C#
• Cookies

Please note that the code is fully documented in the source files.

Hotmail proxy

The proxy will be responsible for making HTTP requests to the Hotmail servers. This will require various things to be implemented:

• Make requests using the PROPFIND method, instead of the usual GET and POST.
• Handle redirection.
• Handle cookies.
• Handle HTTP authentication.
• Send XML data to the remote host, which is basically the same as sending a string.

To send the requests and receive the responses, the class will make use of the HttpWebRequest and HttpWebResponse classes. Cookies will be stored in a CookieContainer. And most importantly authentication can be done by adding a NetworkCredential object to the HttpWebRequest. This will fully implement HTTP authentication, how easy is that! All classes reside in the System.Net namespace and are part of the .NET framework. This means: easy use and documented code!

Let’s start with building the class framework.

public class HotmailProxy
{
    private CookieContainer ccContainer;

    public HotmailProxy ()
    {
        ccContainer = new CookieContainer();
    }
}

Ok, not too interesting. Now let’s get on with building the really important part, building the method that actually sends the client’s request to a remote host. It will implement all the proxy’s requirements that are stated above!

The method will need to accommodate the request as a byte array, the destination, and the necessary credentials for authentication (which can be null if cookies are set after logging in):

private string SendRequestTo(byte[] requestBytes, 
           Uri destination, NetworkCredential credential)

The first thing that needs to be done is build the HttpWebRequest object. The object will need to have a specific UserAgent header, otherwise Hotmail will decline the request (pfff, how conceited). Also it will have a few other headers initialized.

Implementation of the authentication process is done by adding the NetworkCredential object to the request, that object was quite difficult to build, read about it further on.

{
    HttpWebRequest webRequest = 
       (HttpWebRequest)WebRequest.Create(destination);
    webRequest.Method = "PROPFIND";
    webRequest.Accept = "*/*";
    webRequest.AllowAutoRedirect = false;
    webRequest.UserAgent = "Mozilla/4.0 (compatible;" + 
       " MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)";
    webRequest.CookieContainer = new CookieContainer();
    webRequest.ContentLength = requestBytes.Length;
    webRequest.ContentType = "text/xml";
    webRequest.Credentials = credential;            
    webRequest.CookieContainer.Add(ccContainer.GetCookies(destination));

Now the request has been build, we will write the request to the requested host and try to get a response:

    try
    {
        Stream reqStream = webRequest.GetRequestStream();
        reqStream.Write(requestBytes,0,requestBytes.Length);
        reqStream.Close();
        HttpWebResponse webResponse = 
           (HttpWebResponse)webRequest.GetResponse();

After verification that a response was received, cookies and redirection can be handled. Otherwise an exception will be thrown (build your own MailException of some sort for this). Cookies should be compared to the stored cookies by matching names. Redirection is done by checking the HTTP status code, and recursively calling this method.

        if (webRequest.HaveResponse)
            {
                // First handle cookies

                foreach(Cookie retCookie in webResponse.Cookies)
                {
                bool cookieFound = false;
                foreach(Cookie oldCookie in 
                   ccContainer.GetCookies(destination))
                {
                    if (retCookie.Name.Equals(oldCookie.Name))
                    {
                        oldCookie.Value = retCookie.Value;
                        cookieFound = true;
                    }
                }
                if (!cookieFound)
                    ccContainer.Add(retCookie);
            }                
            // Next is redirection

            if ((webResponse.StatusCode == HttpStatusCode.Found) || 
            (webResponse.StatusCode == HttpStatusCode.Redirect) ||
            (webResponse.StatusCode == HttpStatusCode.Moved) ||
            (webResponse.StatusCode == HttpStatusCode.MovedPerm..y))
            {                
                // Get new location and call recursively

                WebHeaderCollection headers = webResponse.Headers;
                return SendRequestTo(requestBytes, 
                    new Uri(headers["location"]),credential);    
            }

Now that redirection has been handled and all cookies are set, the response stream can be read to receive the final server’s response. This finishes the method.

        // Read response

            StreamReader stream = new 
               StreamReader(webResponse.GetResponseStream());
            string responseString = stream.ReadToEnd();
            stream.Close();
            return responseString;
        }
        throw new Exception("No response received from host.");
    }
    catch(WebException e)
    {
        throw new Exception("Exception occured while sending request.",e);
    }
}

To complete the class, a public interface will need to be provided, that calls the SendRequestTo method. The request is an XML string, so the method will need to translate that string into a byte array. The following code is pretty basic. Check the input, build the byte array and send away!

public string SendRequest(string request, 
      Uri destination, NetworkCredential credential)
{
    if(request == null || request.Trim().Length == 0)
        throw new ArgumentNullException("request");
    else if (destination == null)
        throw new ArgumentNullException("destination");
    else
    {
        byte[] xmlBytes = Encoding.ASCII.GetBytes(request);
        return SendRequestTo(xmlBytes,destination, credential);
    }
}

Because authentication isn’t done in all the requests, the following method has been made available to make further requests from a selected host after logging in.

public string SendRequest(string request, Uri destination)
{
    return SendRequest(request,destination,null);
}

Hoera! This completes our proxy, it is now able to do all the things we started out to do! It is able to send requests to Hotmail servers across the world.

Hotmail client

Now for implementing a class that provides access to Hotmail. The following example will show how to connect to Hotmail using the proxy, get the msgfolderroot and request some information about mailboxes. The msgfolderroot can be queried for information such as mailbox names, message count and unread count. This will allow clients to determine how many messages are new, without having to download all the messages (which would be a stupid method to implement). Using this example and the webpage provided at the top of this article, it will be easy to implement the rest of an e-mail client (it was for me!).

First let’s begin again with the class framework:

public class HotmailClient
{
    private HotmailProxy hHttp = null;
    
    public HotmailClient()
    {}    
}

Connect()

Now for the only public method, Connect(). This method will connect to Hotmail using HTTP authentication, it will parse the response to obtain the URL of the msgfolderroot. Next this URL will be used to determine some mailbox information.

First of all, the proxy class en credentials needed for authentication are built:

public void Connect(string username, string password)
{
    hHttp = new HotmailHttp();
    NetworkCredential credentials = 
        new NetworkCredential(username,password,null);

Pffew, that was hard, one complete line of code. Easy enough as you can see! The next job will be to build the XML query for the msgfolderroot. (XMLHTTP will return all kinds of nonsense about the inbox, advertisements and others, while as you can see only the msgfolderroot is requested).

    string query = "<?xml version='1.0'?>" +
    "<D:propfind xmlns:D='DAV:' " + 
    "xmlns:h='http://schemas.microsoft.com/hotmail/' " + 
    "xmlns:hm='urn:schemas:httpmail:'>" + 
        "<D:prop>"                + 
            "<hm:msgfolderroot/>"    +
        "</D:prop>"                + 
    "</D:propfind>";

The query and the credentials can be used to get a response from the Hotmail gateway located at http://services.msn.com/svcs/hotmail/httpmail.asp. This will cause several redirections and cause the HTTP authentication to take place. All can be done with just a single line of code.

    try
    {
        string hotmailEntryPoint = 
          "http://services.msn.com/svcs/hotmail/httpmail.asp";
        string response = hHttp.SendRequest(query, 
          new Uri(hotmailEntryPoint),credentials);
        // Verify response

        if (response == null || response.Trim().Length == 0)
            throw new Exception();

The Hotmail server will respond with the URL of the msgfolderroot. This URL is placed in the XML based response, so it can be found using Xpath. Another method has been built to do just that.

        // Parse the response, further verifying it.

        Uri folderRootUri = ParseConnectResponse(response);

With the now obtained URL, information about all the mailboxes on the server can be retrieved.

    // Obtain available folders using folderRootUrl

        RetrieveMailboxes(folderRootUri);
    }
    catch(Exception e)
    {
        // Something went wrong.

        throw new MailException
           ("Exception occured while connecting to remote host.",e);
    }
}

This completes our first method. As you can see, it calls two other methods which we will now construct, these two contain the interesting parts, parsing the response! But first an important helper method needs to be build, without it all XPath queries fail.

CreateNamespaceManager (XmlNameTable table)

To obtain the URL of the folder root, we will use XPath. The problem lies in the fact that XPath will not return results for nodes that are declared in a namespace. It is able to do so, but information about namespaces need to be stated with the XPath query. This can be done by constructing an XmlNamespaceManager that knows about the namespaces in the Hotmail responses. As these namespaces appear to be really constant, a special method can be built to construct the necessary object.

private XmlNamespaceManager CreateNamespaceManager(XmlNameTable table)
{
    XmlNamespaceManager m = new XmlNamespaceManager(table);
    m.AddNamespace("hm","urn:schemas:httpmail:");
    m.AddNamespace("D","DAV:");
    m.AddNamespace("m","urn:schemas:mailheader:");
    m.AddNamespace("c","urn:schemas:contacts:");
    m.AddNamespace("h","http://schemas.microsoft.com/hotmail/");
    return m;
}

ParseConnectResponse(string response)

To parse the response, there are a two things that need to be done; place the returned response in an XmlDocument and parse the document using XPath to obtain the URL of the msgfolderroot.

private Uri ParseConnectResponse(string response)
{
    try
    {
        // Load response into XmlDocument

        XmlDocument dom = new XmlDocument();
        dom.LoadXml(response);
        // Query XmlDocument for msgfolderroot node.

        string xpath = "//hm:msgfolderroot";
        XmlNamespaceManager context = CreateNamespaceManager(dom.NameTable);
        XmlNode folderRoot = dom.SelectSingleNode(xpath,context);
        // Verify node

        if (folderRoot == null)
            throw new Exception("Node '" + xpath + "' not found.");
        // Get node text and verify,

        string folderRootUrl = folderRoot.InnerText;
        if ((folderRootUrl == null) || (folderRootUrl.Trim().Length == 0))
            throw new Exception("No url found in node '" + xpath + "'.");
        try
        {
            // Return the uri, this may result in a 

            // UriFormatException

            return new Uri(folderRootUrl);
        }
        catch
        {
            throw new MailException("Url found in node '" 
                 + xpath + "' is invalid:" + folderRootUrl);
        }
    }
    catch(Exception e)
    {
        // Something went wrong.

        throw new Exception
          ("Error occured while parsing connect response.",e);
    }
}

The way this method works is pretty basic now. The correct query to make can be determined using the webpage stated in the introduction, it’s also possible to use a network analyzer for this purpose. Probably you will need to use both. To parse the response, the XmlNamespaceManager is built, and XPath is used.

RetrieveMailboxes(Uri folderRootUrl)

The last method in this example will retrieve information about all mailboxes on the server. Basically it does the same thing as the first two methods combined, so you should already be able to build it yourself. The method is shown here for completion sake. It will build a query and send it to the URL of the msgfolderroot now determined.

private void RetrieveMailboxes(Uri folderRootUrl)
{
    try
    {
        // Build the needed query

        string query =    "<?xml version='1.0'?>" + 
        "<D:propfind xmlns:D='DAV:' " +
        "xmlns:hm='urn:schemas:httpmail:'>" + 
        "<D:prop>"                + 
            "<D:displayname/>"    +
            "<hm:special/>"        +
            "<hm:unreadcount/>" +
            "<D:visiblecount/>" +
        "</D:prop>"                + 
        "</D:propfind>";
        // Get a response from server No Credentials are used!

        string response = hHttp.SendRequest(query,folderRootUrl);
        // Verify response

        if (response == null || response.Trim().Length == 0)
            throw new ApplicationException
                 ("No response received from host.");    
        // Load response into XmlDocument

        XmlDocument dom = new XmlDocument();
        dom.LoadXml(response);
        // Query XmlDocument for all mailboxes using XPath

        string xpath = "//D:response";
        XmlNamespaceManager context = 
               CreateNamespaceManager(dom.NameTable);
        XmlNodeList mailBoxNodes = dom.SelectNodes(xpath,context);
        // Parse each node found

        foreach(XmlNode mailBoxNode in mailBoxNodes)
        {        
            try
            {
                // Direct mapping using XPath, should not 

                // result in any errors

                // as long as Hotmail keeps it protocol 

                // the same.

                string type = mailBoxNode.SelectSingleNode
                    ("descendant::hm:special",context).InnerText;
                string nameUrl = mailBoxNode.SelectSingleNode
                    ("descendant::D:href",context).InnerText;
                int visibleCount = Int32.Parse
                    (mailBoxNode.SelectSingleNode
                    ("descendant::D:visiblecount",
                    context).InnerText);
                int unreadCount = Int32.Parse
                    (mailBoxNode.SelectSingleNode
                    ("descendant::hm:unreadcount",
                    context).InnerText);

                Console.WriteLine("MailBox found: " + 
                   type + "\r\n" + "\turl: " + 
                   nameUrl + "\r\n" + "\tVisible: " + 
                   visibleCount + "\r\n" + "\tUnread: " +                 
                unreadCount + "\r\n");
            }
            catch(Exception e)
            {
                Console.WriteLine
                 ("Exception occured while " + 
                 "obtaining mailbox info: " + e.Message);
            }
        }
    }
    catch(Exception e)
    {
        // Something went wrong.

        throw new ApplicationException("Error occured" +
           " while retrieving available mailboxes.",e);
    }
}

Conclusion

As you can see, accessing Hotmail from C# can be as easy as any other type of mail server. The code works without any strange loops or use of classes that function only partly. With this code it should be easy to build your own e-mail client that is able to access Hotmail with the same type of interface as you would build for IMAP or POP. Happy coding!