Click here to Skip to main content
13,189,506 members (59,512 online)
Click here to Skip to main content
Add your own
alternative version

Tagged as

Stats

17.6K views
20 bookmarked
Posted 12 Jul 2010

Consuming Google (Reader) with .NET: Part 2 - Google Reader using .NET

, 12 Jul 2010
Rate this:
Please Sign up or sign in to vote.
An implementation of Google Reader in .NET

Introduction

This article starts where the previous one (Consuming Google (Reader) with .NET: Part 1 - Authentication) stopped.
This is an implementation of Google Reader in .NET.

Before getting started, it's important to know that there is a site with plenty of information on the Google Reader API: http://code.google.com/p/pyrfeed/wiki/GoogleReaderAPI. But what we'll also be doing is using Fiddler to reverse engineer the Google Reader API.

Reverse Engineering

When you visit Google Reader via your browser, you're basically always sending GET and POST requests. Using Fiddler, you can then view the content of those requests.

Let's say we want to subscribe to http://sandrinodimattia.net/Blog. When you're logged in to Google Reader, you can press the Add Subscription button, enter the URL and press Add. If you do this while Fiddler is open, you'll see the following:

fiddlerreader.png

After pressing the Add button, you can see that the URL /reader/api/0/subscription/quickadd was visited and 2 fields were posted (quickadd and T). And for each available action in Google Reader, you can use Fiddler to view the URL and the post fields that are hidden underneath.

If you take a closer look at the screenshot, you see a field called T, this is a token. It identifies your session, but expires quickly. That's why you'll see that our code requests a new token for each new request.

Adding POST to GoogleSession

In the last article, we created the GoogleSession class. This class helps us in getting data from Google. But now that we have to make POST requests, we'll also need to send data to Google. That's why we'll add the following method to our GoogleSession class:

/// <summary>
/// Send a post request to Google.
/// </summary>
/// <param name="url"></param>
/// <param name="postFields"></param>
/// <returns></returns>
public void PostRequest(string url, params GoogleParameter[] postFields)
{ 
    // Format the parameters.
    string formattedParameters = string.Empty;
    foreach (var par in postFields.Where(o => o != null))
        formattedParameters += string.Format("{0}={1}&", par.Name, par.Value);
    formattedParameters = formattedParameters.TrimEnd('&');
    
    // Append a token.
    formattedParameters += String.Format("&{0}={1}", "T", GetToken());
     
    // Get the current post data and encode.
    ASCIIEncoding ascii = new ASCIIEncoding();
    byte[] encodedPostData = ascii.GetBytes(
        String.Format(formattedParameters));
        
    // Prepare request.
    HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
    request.Method = "POST";
    request.ContentType = "application/x-www-form-urlencoded";
    request.ContentLength = encodedPostData.Length;
     
    // Add the authentication header. 
    request.Headers.Add("Authorization", "GoogleLogin auth=" + auth);
    
    // Write parameters to the request.
    using (Stream newStream = request.GetRequestStream())
        newStream.Write(encodedPostData, 0, encodedPostData.Length);
        
    // Get the response and validate.
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    if (response.StatusCode != HttpStatusCode.OK)
        throw new LoginFailedException(
            response.StatusCode, response.StatusDescription);
}    

This function uses a URL and GoogleParameters to build a POST request. And the token we talked about is also automatically included in the post fields. Thanks to this function, we'll be able to send POST requests to Google Reader with ease.

The GoogleService Base Class

Before really getting started, we'll just go and create a base class we might want to re-use when implementing other Google services. It encapsulates the authentication mechanism and the GoogleSession class.

public abstract class GoogleService : IDisposable
{ 
    /// <summary>
    /// Current google session.
    /// </summary>
    protected GoogleSession session;
    
    /// <summary>
    /// Creating this class will automatically try to log in and create a session.
    /// That way for each service we create we don't need to worry 
    /// about the implementation of authentication and session.
    /// </summary>
    /// <param name="service"></param>
    /// <param name="username"></param>
    /// <param name="password"></param>
    /// <param name="source"></param>
    protected GoogleService
	(string service, string username, string password, string source)
    {
        // Get the Auth token.
        string auth = ClientLogin.GetAuthToken(service, username, password, source);
        // Create a new session using this token.
        this.session = new GoogleSession(auth);
    }
    
    /// <summary>
    /// Clean up the session.
    /// </summary>
    public void Dispose()
    {
        if (session != null)
            session.Dispose();
    }
}

Item Types

When reading from Google Reader, you'll be working with 2 different types of data. Pure XML and syndication items. Syndication items (SyndicationItem in .NET) are actually items that come from a feed. For example, if you visit this site you'll get a list of your read items in the shape of an atom feed: http://www.google.com/reader/atom/user/-/state/com.google/read. Each item in this feed is a SyndicationItem.

For both types, we'll be using some basic base classes:

public abstract class GoogleSyndicationItem
{ 
    /// <summary>
    /// Initialize the item.
    /// </summary>
    /// <param name="item"></param>
    internal GoogleSyndicationItem(SyndicationItem item)
    {
        if (item != null)
        {
            LoadItem(item);
        }
    }
    
    /// <summary>
    /// Load the item (to be implemented by inheriting classes).
    /// </summary>
    /// <param name="item"></param>
    protected abstract void LoadItem(SyndicationItem item);
    
    /// <summary>
    /// Get the text from a TextSyndicationContent.
    /// </summary>
    /// <param name="content"></param>
    /// <returns></returns>
    public string GetTextSyndicationContent(SyndicationContent content)
    {
        TextSyndicationContent txt = content as TextSyndicationContent;
        if (txt != null)
            return txt.Text;
        else
            return "";
    }
}

public abstract class GoogleXmlItem : SyndicationItem
{
    /// <summary>
    /// Initialize the item.
    /// </summary>
    /// <param name="item"></param>
    internal GoogleXmlItem(XElement item)
    {
        if (item != null)
        {
            LoadItem(item);
        }
    }
    
    /// <summary>
    /// Load the item (to be implemented by inheriting classes).
    /// </summary>
    /// <param name="item"></param>
    protected abstract void LoadItem(XElement item);
    
    /// <summary>
    /// Get a list of descendants.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="descendant"></param>
    /// <param name="attribute"></param>
    /// <param name="attributeValue"></param>
    /// <returns></returns>
    protected IEnumerable<XElement> GetDescendants
	(XElement item, string descendant, string attribute, string attributeValue)
    {
        return item.Descendants(descendant).Where(o => o.Attribute(attribute) != null && 
		o.Attribute(attribute).Value == attributeValue);
    }
    
    /// <summary>
    /// Get a descendant.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="descendant"></param>
    /// <param name="attribute"></param>
    /// <param name="attributeValue"></param>
    /// <returns></returns>
    protected XElement GetDescendant(XElement item, string descendant, 
	string attribute, string attributeValue)
    {
        return GetDescendants(item, descendant, attribute, attributeValue).First();
    }
    
    /// <summary>
    /// Get the value of a descendant.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="descendant"></param>
    /// <param name="attribute"></param>
    /// <param name="attributeValue"></param>
    /// <returns></returns>
    protected string GetDescendantValue
	(XElement item, string descendant, string attribute, string attributeValue)
    {
        var desc = GetDescendant(item, descendant, attribute, attributeValue);
        if (desc != null)
            return desc.Value;
        else
            return "";
    }
}   

And for almost every type of data in Google Reader, I've also created a class:

  • Feed (the URL to a feed with the title of that feed)
  • ReaderItem (you could say this is an article, a blog post, ...)
  • State (this is the state of an item in Google Reader, like read, starred, ...)
  • Subscription (a subscription in Google reader is a feed you subscribed to)

Here is the implementation for subscription:

public class Subscription : GoogleXmlItem
{ 
    /// <summary>
    /// Id of the subscription.
    /// </summary>
    public string Id { get; set; }
    
    /// <summary>
    /// Title of the feed.
    /// </summary>
    public string Title { get; set; }
    
    /// <summary>
    /// URL to the subscription.
    /// </summary>
    public string Url { get; set; }
    
    /// <summary>
    /// List of categories.
    /// </summary>
    public List<string> Categories { get; set; }
    
    /// <summary>
    /// Initialize the subscription.
    /// </summary>
    /// <param name="item"></param>
    internal Subscription(XElement item)
        : base(item)
    {
    }
    
    /// <summary>
    /// Load the subscription item.
    /// </summary>
    /// <param name="item"></param>
    protected override void LoadItem(XElement item)
    {
        // Initialize categories list.
        Categories = new List<string>();
        
        // Regular fields.
        Id = GetDescendantValue(item, "string", "name", "id");
        Title = GetDescendantValue(item, "string", "name", "title");
        
        // Parse the URL.
        if (Id.Contains('/'))
            Url = Id.Substring(Id.IndexOf('/') + 1, Id.Length - Id.IndexOf('/') - 1);
            
        // Get the categories.
        var catList = GetDescendant(item, "list", "name", "categories");
        if (catList != null && catList.HasElements)
        {
            var categories = GetDescendants(item, "string", "name", "label");
            Categories.AddRange(categories.Select(o => o.Value));
        }
    }
} 

URLs Everywhere

Like mentioned before, the Google Reader API is based on URLs and GET/POST requests. To organise this, we've also got a few classes regarding URLs:

  • ReaderUrl: A single class containing all the required URLs and paths
  • ReaderCommand: Enum representing common tasks (like adding a subscription)
  • ReaderCommandFormatter: Class containing extension methods for ReaderCommand to convert these enum values to actual Google Reader URLs
public static class ReaderUrl
{
    /// <summary>
    /// Base url for Atom services.
    /// </summary>
    public const string AtomUrl = "https://www.google.com/reader/atom/";
    
    /// <summary>
    /// Base url for API actions.
    /// </summary>
    public const string ApiUrl = "https://www.google.com/reader/api/0/";
    
    /// <summary>
    /// Feed url to be combined with the desired feed.
    /// </summary>
    public const string FeedUrl = AtomUrl + "feed/";
    
    /// <summary>
    /// State path.
    /// </summary>
    public const string StatePath = "user/-/state/com.google/";
    
    /// <summary>
    /// State url to be combined with desired state. For example: starred
    /// </summary>
    public const string StateUrl = AtomUrl + StatePath;
    
    /// <summary>
    /// Label path.
    /// </summary>
    public const string LabelPath = "user/-/label/";
    
    /// <summary>
    /// Label url to be combined with the desired label.
    /// </summary>
    public const string LabelUrl = AtomUrl + LabelPath;
}

public enum ReaderCommand
{
    SubscriptionAdd,
    SubscriptionEdit,
    SubscriptionList,
    TagAdd,
    TagEdit,
    TagList,
    TagRename,
    TagDelete
}

public static class ReaderCommandFormatter
{
    /// <summary>
    /// Get the full url for a command.
    /// </summary>
    /// <param name="comm"></param>
    /// <returns></returns>
    public static string GetFullUrl(this ReaderCommand comm)
    {
        switch (comm)
        {
            case ReaderCommand.SubscriptionAdd:
                return GetFullApiUrl("subscription/quickadd");
            case ReaderCommand.SubscriptionEdit:
                return GetFullApiUrl("subscription/edit");
            case ReaderCommand.SubscriptionList:
                return GetFullApiUrl("subscription/list");
            case ReaderCommand.TagAdd:
                return GetFullApiUrl("edit-tag");
            case ReaderCommand.TagEdit:
                return GetFullApiUrl("edit-tag");
            case ReaderCommand.TagList:
                return GetFullApiUrl("tag/list");
            case ReaderCommand.TagRename:
                return GetFullApiUrl("rename-tag");
            case ReaderCommand.TagDelete:
                return GetFullApiUrl("disable-tag");
            default:
                return "";
        }
    }
    
    /// <summary>
    /// Get the full api url.
    /// </summary>
    /// <param name="append"></param>
    /// <returns></returns>
    private static string GetFullApiUrl(string append)
    {
        return String.Format("{0}{1}", ReaderUrl.ApiUrl, append);
    }
}

And Finally... ReaderService

Finally there's the implementation of the most common tasks in Google Reader:

public class ReaderService : GoogleService
{
    /// <summary>
    /// Current username.
    /// </summary>
    private string username;
    
    /// <summary>
    /// Initialize the Google reader.
    /// </summary>
    /// <param name="username"></param>
    /// <param name="password"></param>
    /// <param name="source"></param>
    public ReaderService(string username, string password, string source)
        : base("reader", username, password, source)
    {
        this.username = username;            
    }
    
    #region Feed
    /// <summary>
    /// Get the contents of a feed.
    /// </summary>
    /// <param name="feedUrl">
    /// Must be exact URL of the feed, 
    /// ex: http://sandrinodimattia.net/blog/syndication.axd
    /// </param>
    /// <param name="limit"></param>
    /// <returns></returns>
    public IEnumerable<ReaderItem> GetFeedContent(string feedUrl, int limit)
    {
        try
        {
            return GetItemsFromFeed(String.Format("{0}{1}", 
		ReaderUrl.FeedUrl, System.Uri.EscapeDataString(feedUrl)), limit);
        }
        catch (WebException wex)
        {
            HttpWebResponse rsp = wex.Response as HttpWebResponse;
            if (rsp != null && rsp.StatusCode == HttpStatusCode.NotFound)
                throw new FeedNotFoundException(feedUrl);
            else
                throw;
        }
    }
    #endregion
    #region Subscription
    /// <summary>
    /// Subscribe to a feed.
    /// </summary>
    /// <param name="feed"></param>
    public void AddSubscription(string feed)
    {
        PostRequest(ReaderCommand.SubscriptionAdd, 
		new GoogleParameter("quickadd",  feed));
    }
    
    /// <summary>
    /// Tag a subscription (remove it).
    /// </summary>
    /// <param name="feed"></param>
    /// <param name="folder"></param>
    public void TagSubscription(string feed, string folder)
    {
        PostRequest(ReaderCommand.SubscriptionEdit,
            new GoogleParameter("a", ReaderUrl.LabelPath + folder), 
            new GoogleParameter("s", "feed/" + feed), 
            new GoogleParameter("ac", "edit"));
    }
    
    /// <summary>
    /// Get a list of subscriptions.
    /// </summary>
    /// <returns></returns>
    public List<Subscription> GetSubscriptions()
    {
        // Get the XML for subscriptions.
        string xml = session.GetSource(ReaderCommand.SubscriptionList.GetFullUrl());
        // Get a list of subscriptions.
        return XElement.Parse(xml).Elements
	("list").Elements("object").Select(o => new Subscription(o)).ToList();
    }
    #endregion
    #region Tags
    /// <summary>
    /// Add tags to an item.
    /// </summary>
    /// <param name="feed"></param>
    /// <param name="folder"></param>
    public void AddTags(ReaderItem item, params string[] tags)
    {
        // Calculate the amount of parameters required.
        int arraySize = tags.Length + item.Tags.Count + 2;
        
        // Set all parameters.
        GoogleParameter[] parameters = new GoogleParameter[arraySize];
        parameters[0] = new GoogleParameter("s", "feed/" + item.Feed.Url);
        parameters[1] = new GoogleParameter("i", item.Id);
        
        int nextParam = 2;
        
        // Add parameters.
        for (int i = 0; i < item.Tags.Count; i++)
            parameters[nextParam++] = new GoogleParameter
		("a", ReaderUrl.LabelPath + item.Tags[i]);
        for (int i = 0; i < tags.Length; i++)
            parameters[nextParam++] = new GoogleParameter
		("a", ReaderUrl.LabelPath + tags[i]);
            
        // Send request.
        PostRequest(ReaderCommand.TagAdd, parameters);
    }
    
    /// <summary>
    /// Rename a tag.
    /// </summary>
    /// <param name="tag"></param>
    /// <param name="newName"></param>
    public void RenameTag(string tag, string newName)
    {
        PostRequest(ReaderCommand.TagRename,
            new GoogleParameter("s", ReaderUrl.LabelPath + tag),
            new GoogleParameter("t", tag),
            new GoogleParameter("dest", ReaderUrl.LabelPath + newName));
    }
    
    /// <summary>
    /// Remove tag (for all items).
    /// </summary>
    /// <param name="tag"></param>
    public void RemoveTag(string tag)
    {
        PostRequest(ReaderCommand.TagDelete,
            new GoogleParameter("s", ReaderUrl.LabelPath + tag),
            new GoogleParameter("t", tag));
    }
    
    /// <summary>
    /// Remove tag from a single item.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="tag"></param>
    public void RemoveTag(ReaderItem item, string tag)
    {
        PostRequest(ReaderCommand.TagEdit,
            new GoogleParameter("r", ReaderUrl.LabelPath + tag),
            new GoogleParameter("s", "feed/" + item.Feed.Url),
            new GoogleParameter("i", item.Id));
    }
    
    /// <summary>
    /// Get a list of tags.
    /// </summary>
    /// <returns></returns>
    public List<string> GetTags()
    {            
        string xml = session.GetSource(ReaderCommand.TagList.GetFullUrl());
        
        // Get the list of tags.
        var tagElements = from t in XElement.Parse(xml).Elements
				("list").Descendants("string")
                          where t.Attribute("name").Value == "id"
                          where t.Value.Contains("/label/")
                          select t;
                          
        // Create a list.
        List<string> tags = new List<string>();
        foreach (XElement element in tagElements)
        {
            string tag = element.Value.Substring(element.Value.LastIndexOf('/') + 1, 
                element.Value.Length - element.Value.LastIndexOf('/') - 1);
            tags.Add(tag);
        }
        
        // Done.
        return tags;
    }
    
    /// <summary>
    /// Get all items for a tag.
    /// </summary>
    /// <param name="tag"></param>
    /// <param name="limit"></param>
    /// <returns></returns>
    public IEnumerable<ReaderItem> GetTagItems(string tag, int limit)
    {
        return GetItemsFromFeed(String.Format("{0}{1}", 
	ReaderUrl.LabelPath, System.Uri.EscapeDataString(tag)), limit);
    }
    #endregion
    #region State
    /// <summary>
    /// Add state for an item.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="state"></param>
    public void AddState(ReaderItem item, State state)
    {
        PostRequest(ReaderCommand.TagEdit,
            new GoogleParameter("a", 
		ReaderUrl.StatePath + StateFormatter.ToString(state)),
            new GoogleParameter("i", item.Id),
            new GoogleParameter("s", "feed/" + item.Feed.Url));
    }
    
    /// <summary>
    /// Remove a state from an item.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="state"></param>
    public void RemoveState(ReaderItem item, State state)
    {
        PostRequest(ReaderCommand.TagEdit,
            new GoogleParameter("r", 
		ReaderUrl.StatePath + StateFormatter.ToString(state)),
            new GoogleParameter("i", item.Id),
            new GoogleParameter("s", "feed/" + item.Feed.Url));
    }
    
    /// <summary>
    /// Get the content of a state. 
    /// For example: Get all starred items.
    /// </summary>
    /// <param name="state"></param>
    /// <param name="limit"></param>
    /// <returns></returns>
    public IEnumerable<ReaderItem> GetStateItems(State state, int limit)
    {
        return GetItemsFromFeed(String.Format("{0}{1}", 
		ReaderUrl.StateUrl, StateFormatter.ToString(state)), limit); 
    }
    #endregion
    
    /// <summary>
    /// Post a request using a reader command.
    /// </summary>
    /// <param name="command"></param>
    /// <param name="postFields"></param>
    private void PostRequest(ReaderCommand command, params GoogleParameter[] postFields)
    {
        session.PostRequest(ReaderCommandFormatter.GetFullUrl(command), postFields);
    }
    
    /// <summary>
    /// Get items from a feed and convert them to a GoogleReaderItem.
    /// </summary>
    /// <param name="url"></param>
    /// <param name="limit"></param>
    /// <returns></returns>
    private IEnumerable<ReaderItem> GetItemsFromFeed(string url, int limit)
    {
        SyndicationFeed feed = session.GetFeed(url, 
		new GoogleParameter("n", limit.ToString()));
        return feed.Items.Select<SyndicationItem, ReaderItem>(o => new ReaderItem(o));
    }
}

And as you can see, the ReaderService does a few things:

  • Subscriptions (list, add)
  • Tags (add, rename, delete, ...)
  • States (add, remove, list)
  • Feed (list contents)

And it actually re-uses a few of the things we talked about:

  • GoogleSession to send post requests, get feeds, ...
  • ReaderCommand, ReaderCommandFormatter, ReaderUrl to do all the URL related stuff
  • GoogleParameter to set POST fields (fields we can find using Fiddler)

Putting It All Together

The console application was also updated with our ReaderService:

class Program
{
    static void Main(string[] args)
    {
        // Empty line.
        Console.WriteLine("");
        
        // Get username.
        Console.Write(" Enter your Google username: ");
        string username = Console.ReadLine();
        
        // Get password.
        Console.Write(" Enter your password: ");
        string password = Console.ReadLine();
        
        // Query.
        using (ReaderService rdr = new ReaderService
		(username, password, "Sandworks.Google.App"))
        {
            // Display.
            Console.WriteLine("");
            Console.WriteLine(" Last 5 articles from Sandrino's Blog: ");
            foreach (ReaderItem item in rdr.GetFeedContent
		("http://sandrinodimattia.net/blog/syndication.axd?format=rss", 5))
            {
                Console.WriteLine("  - " + item.Author + ": " + item.Title);
            }
        }
        
        // Pause.
        Console.ReadLine();
    }
}

There you go, now you've got everything you need to get started with Google Reader in .NET. The following article will be about creating a basic WPF application to have a simple desktop version of Google Reader.

Enjoy...

History

  • 12th July, 2010: Initial post

License

This article, along with any associated source code and files, is licensed under The Microsoft Public License (Ms-PL)

Share

About the Author

Sandrino Di Mattia
Technical Lead RealDolmen
Belgium Belgium
I'm a Technical Consultant at RealDolmen, one of the largest players on the Belgian IT market: http://www.realdolmen.com

All posts also appear on my blogs: http://blog.sandrinodimattia.net and http://blog.fabriccontroller.net

You may also be interested in...

Pro
Pro

Comments and Discussions

 
Questiongetting the list all entry feeds from a tag doesn't work Pin
André Ziegler18-Jun-13 20:25
memberAndré Ziegler18-Jun-13 20:25 
QuestionSubscribers number of a blog Pin
thePhargoth13-Mar-13 3:35
memberthePhargoth13-Mar-13 3:35 
GeneralThanks Pin
Gewgaw15-Aug-10 7:34
memberGewgaw15-Aug-10 7:34 
GeneralFail to mark item as read. Pin
wongchichong21-Jul-10 7:33
memberwongchichong21-Jul-10 7:33 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.171016.2 | Last Updated 12 Jul 2010
Article Copyright 2010 by Sandrino Di Mattia
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid