Unraveling the Netflix API - Part I - Netflix API Basics

• Download demo project - 8.42 KB
• Download source - 22.99 KB

Contents

• Introduction
• Background
• Obtaining Application Credentials
• Client Application Types
  • Non-Authenticated Requests
  • Signed Requests
  • Protected Requests
• Unraveling Signed Requests
  • REST
  • OAuth
  • Signing the Request
  • Making the Request
• Example Application
  • The OAuth Encapsulator
  • The Web Service Request
  • Running the Example Application
• Conclusion
• Glossary of Terms
• History

Introduction

I have been a Netflix customer for several years now and I just love the service. As a software developer, I was quite excited when I heard that they had opened up a Web Service API to access their service. Wasting no time, I headed over to http://developer.netflix.com to grab a piece of the action. However, my enthusiasm quickly waned as I started plowing through their documentation. Initially, I came across terms that were quite familiar: REST, XML, etc. But soon, I was confronted with some fundamental problems, such as:

• The documentation is quite extensive, but is only available in an online format. It would have been really handy to download a PDF to print out and browse offline at my leisure.
• They use some peculiar formatting that looks great in a browser, but when I printed it, pretty much all of the diagrams were lost or chopped to bits.
• The details are sometimes way oversimplified, or sometimes quite obtuse.

Through diligent effort searching for better explanations for their terminology, and any code that might demystify things for me, I finally was able to assemble some code the demonstrates the Netflix APIs - and it turned to be quite simple in the end. This is the first of a series of articles that describe the fundamentals of the Netflix APIs and demonstrate how .NET applications can access the Netflix services programmatically.

One last word about the Netflix documentation before we proceed. Please don't get me wrong. I don't mean to sound harsh about the documentation they produced. It's enough work just publishing a reliable and well designed API set, much less producing quality documentation. I give my utmost kudos to the Netflix team for the work they've done, and hope that my additional clarifications will be of benefit to everyone. Furthermore, my first encounter with the documentation was right after the API was released, so it's understandable that it was a bit rough. It has indeed improved subsequently; however, there are still areas remaining that these articles can help clarify.

And finally, you might want to skip ahead as you're reading to the Glossary of Terms at the end of this article. It lists some of the terminology that I'm using and how it relates to the terminology in the Netflix documentation.

Background

This article will introduce you to the technologies used by the Netflix APIs:

• REST
• OAuth

I realize that many of you CodeProject readers do not have access to the Netflix service. I hope that you will find the technical information in these articles of merit even if the practical application of the example code is of lesser benefit.

Obtaining Application Credentials

The first step in developing a Netflix client application is signing up for a developer account. There is a link on the Netflix Developer Network home page that leads to a form where you can sign up for a developer account. This form requires you to enter some basic information about yourself, including a name and password you would like to use for this account. After completing this form, click Register, and you will be presented with a confirmation page that informs you that you will be receiving an e-mail message with further instructions.

The e-mail message contains a link that takes you back to the Netflix account management site to confirm your developer account request. You will find a link on this page that will take you to another form where you register your application.

Note: You are only allowed to register one application with each developer account. If you wish to develop multiple applications, you must create a separate account for each application.

On the application registration form, you will fill out the basic information about your application. The most important field on this form is, "How did you hear about this API?". You must, of course, answer "Codeproject.com"! When completed, click Register Application and you will arrive at a page that confirms your application registration.

The Application Registration page contains several items of information, the most important of which are:

• Key - This is generally referred to as the "Consumer Key" and represents your application's identity when making Netflix API requests.
• Shared Secret - Also referred to as the "Consumer Secret", this is a secret token that is used when digitally signing your Netflix service requests. Without this token, no one can falsify a service request claiming to be you.

You may wish to print this page, or copy and paste the information into a file, for later reference. But don't worry, Netflix also sends you an e-mail message containing your consumer key and shared secret for archiving in your records.

Client Application Types

Now that you have your application credentials, you can run a quick test right away to verify that they are valid. There are three types of requests you can make:

1. Non-Authenticated Requests - These require nothing more than your consumer key.
2. Signed Requests - These require a digital signature, which will be described in this article.
3. Protected Requests- These are requests that access a subscriber's account, and will be described in a subsequent article.

Non-Authenticated Requests

A non-authenticated request can be constructed in a simple URL sent from a Web browser. This request requires only your Consumer Key and does not need signing. Per the Netflix API documentation, the only non-authenticated request currently supported is the auto-complete catalog search. The following example demonstrates this type of search, where YourConsumerKey is the consumer key you received for your application when you created a developer account.

http://api.netflix.com/catalog/titles/
         autocomplete?oauth_consumer_key=YourConsumerKey&term=Casper

You can copy and paste the preceding request example into your favorite browser, insert your own consumer key, and the results returned should be a list of movie titles associated with the term "Casper".

<?xml version="1.0" standalone="yes"?>

<autocomplete>
  <url_template>http://api.netflix.com/catalog/titles/
                    autocomplete?{-join|&|term}</url_template>
  <autocomplete_item>
    <title short="Casper"></title>
  </autocomplete_item>
  <autocomplete_item>
    <title short="Casper Van Dien"></title>

  </autocomplete_item>
  <autocomplete_item>
    <title short="Casper the Friendly Ghost: Casper's Birthday"></title>
  </autocomplete_item>
  <autocomplete_item>
    <title short="Casper Meets Wendy"></title>
  </autocomplete_item>
  <autocomplete_item>
    <title short="Casper: Scare School"></title>

  </autocomplete_item>
  <autocomplete_item>
    <title short="Casper: A Spirited Beginning"></title>
  </autocomplete_item>
  <autocomplete_item>
    <title short="Casper's Spookiest Tales"></title>
  </autocomplete_item>
  <autocomplete_item>
    <title short="Casper: Trick or Treat"></title>

  </autocomplete_item>
  <autocomplete_item>
    <title short="Casper & Wendy's Ghostly Adventures"></title>
  </autocomplete_item>
  <autocomplete_item>
    <title short="Best of Casper: Vol. 1"></title>
  </autocomplete_item>
</autocomplete>

Signed Requests

Signed requests are the basis of this article, and are described in detail in the following sections. With signed requests, you can access any of the Catalog Resources in the Netflix API set, such as:

• Search for a catalog title (movie, television series, etc.).
• Obtain details for a given catalog title.
• Search for people (actors, directors, etc.).

Protected Requests

Protected requests access subscriber account information, which requires an additional level of security. This will be described in a subsequent article in this series.

Unraveling Signed Requests

As I mentioned earlier, I was quite mystified by the Netflix documentation's explanation of the fundamentals of accessing the APIs, as described in their Authentication Overview topics. They tried, in my opinion, to weave too much technical information together in one big blob, which resulted in my spending more time trying to decipher their documentation conventions than actually understanding the application of the various technologies. I will not attempt to rewrite the Netflix API documentation in its entirety here, but rather to paraphrase their introductory overview in what, I hope, is a more logical and comprehensible manner.

REST

The Netflix API is base on REST, which stands for Representational State Transfer. Having been a SOAP (a.k.a., Web Services) developer for some time, I occasionally find myself having to describe REST to other SOAP developers. My short answer is that REST is a Web Services protocol where you simply put all of the information about the request into the HTTP request URI and query string parameters. I don't want to dwell on the differences between SOAP and REST in this article, but this comparison of the two is a brief way of describing how REST works. For those of you who are familiar with SOAP, or simply as a way of describing REST, here's my version of how the two protocols compare:

SOAP	REST
Action is contained in the SOAP header (WS-Transfer).	Action is specified by the HTTP method (GET, POST, PUT, DELETE).
Routing information is contained in the SOAP header (WS-Transfer).	Routing information is appended to the base URI.
Parameters are passed in the body.	Parameters are passed in the URI and query string. Because of this, the parameters must be URL encoded (also referred to as percent-encoding by those who may be conceptually challenged).
Errors are returned as SOAP faults.	Errors are returned as HTTP error codes. If you're really lucky, you might get a bit of detail about why the request was rejected, but usually, you get just an error code.
Security (authentication, message integrity, etc.), if implemented, is contained in the SOAP header (WS-Security).	With the Netflix APIs, security is implemented through OAuth.

OAuth

Briefly, OAuth provides a means by which services may be requested in a secure manner. There are two aspects to this security:

• Authentication - Providing information with the request that identifies you, and confirms that you are who you claim to be.
• Integrity - Providing a digital signature with the request that indicates if the request has been tampered with while it was enroute to the service provider.

Let's get right down to business and look at a REST service request using OAuth. First of all, let's look at the individual parameters for an example request:

HTTP Method: GET
Request URL: http://api.netflix.com/catalog/people
Query String:  max_results=10
              &oauth_consumer_key=YourConsumerKey
              &oauth_nonce=abcdefghijk
              &oauth_signature_method=HMAC-SHA1
              &oauth_timestamp=1234567890
              &oauth_version=1.0
              &term=frances

Note that the query string is wrapped in this example for clarity, and that YourConsumerKey is the key for your Netflix developer account. So what does all this mean?

• The first thing you need to make a service request is which HTTP method will be used (GET, POST, and so on).
• The next item is the URL, formally called the "service endpoint", to which the request will be directed.
• Lastly, a list of parameters that provides details about what information you wish to obtain, and some additional OAuth security parameters. Note that the query parameters are specified in alphabetical order, formally known as canonicalization. The important thing to observe here is that the two application-provided request parameters are max_results and term - everything else is for the OAuth security. Because of the canonicalization, the max_results and term parameters wind up at the beginning and end of the query string, respectively.

Now, let's take a look at the OAuth parameters individually.

OAuth Parameter	Description
oauth_consumer_key	This is the consumer key that was issued to you for your application. This is what identifies your application as an authorized Netflix service consumer.
oauth_nonce	A nonce is simply a unique identifier, somewhat like a GUID, that can be considered as having multiple purposes: It represents a unique value for each message so that any given message cannot be resent, thus preventing "replay" attacks. It can also represent a message identifier, somewhat equivalent to the <MessageID> in a SOAP WS-Addressing header. If Netflix supported asynchronous service requests, and included the nonce in the service response, this identifier could be used to correlate responses with their original requests.
oauth_signature_method	Describes the hash method used to generate the digital signature (to be described shortly). This is always HMAC-SHA1 for Netflix API requests.
oauth_timestamp	Indicates the date and time at which this request was created. This timestamp also prevents replay attacks, as previously described. Per the OAuth specification, the "timestamp is expressed in the number of seconds since January 1, 1970 00:00:00 GMT". In simple terms, it's the good old UNIX time format.
oauth_version	This is always version 1.0 for Netflix API requests.

Signing the Request

Now that we have the REST request and the OAuth security parameters prepared for the HTTP query string, all that's left is to add a digital signature and we're ready to create our Web request.

The signature is, as previously mentioned, a simple HMAC-SHA1 hash, which is readily available in the .NET System.Security.Cryptography namespace. The signature is generated by concatenating the following elements from the request into what the Netflix documentation calls a "base string", which is then fed to an instance of an HMAC-SHA1 hash generator class.

• The HTTP method.
• The base URL.
• The query string parameter list.
• The "Consumer Key" and "Consumer Secret" that were provided to you when you created your developer account.

Using the request example from the beginning of this section, this is the equivalent base string:

GET&
http%3A%2F%2Fapi.netflix.com2Fcatalog2Fpeople&
max_results%3010
%26oauth_consumer_key%3DYourConsumerKey
%26oauth_nonce%3Dabcdefghijk
%26oauth_signature_method%3DHMAC-SHA1
%26oauth_timestamp%3D1234567890
%26oauth_version%3D1.0
%26term%3Dfrances

Observe in this example that the base string is constructed by:

1. URL encoding each of the three parameters of the request as previously described.
2. Concatenating them into a single string in a specified order: HTTP method, URL, and query string (canonicalization again).

The signature (using HMAC-SHA1) is produced as follows:

1. The HMAC-SHA1 key is set to the URL-encoded Consumer Secret.
2. A "signature base" string is constructed by concatenating:
   1. The HTTP method ("GET", "POST", "PUT", "DELETE").
   2. The "normalized" URL, URL encoded. A normalized URL is one that is fully qualified, including the port designation (":80" for HTTP, ":443" for HTTPS).
   3. The "normalized" request parameters. This consists of all the query string parameters, sorted alphabetically and URL encoded.
3. The signature results from feeding the signature base to the HMAC-SHA1 hashing function.

The resulting HMAC-SHA1 signature value is then Base64-encoded and appended to the previously constructed URI and query string that comprises the service request.

Making the Request

After all of this preparation, you are ready at long last to send off your request, which is now quite simple. The request is sent over HTTP using a GET method, to the URL representing the service endpoint of interest, and the query parameters and digital signature are appended to the request. If everything is constructed correctly, you will receive an XML document in return that contains the information you requested.

Example Application

The example code accompanying this article is a simple application that demonstrates:

• An OAuth class that handles all of the details of generating a properly formed OAuth request with a digital signature.
• Making a synchronous HTTP request and receiving the results.

The OAuth Encapsulator

During my research on OAuth, I was fortunate to stumble across a nice bit of C# code that encapsulates all of the effort required for creating a properly formed OAuth request written by Eran Sandler (see the comments in the code for additional details). There are several public methods in this module, and I also added some tweaks of my own, and ultimately I found the following method to be the most appropriate for my purposes:

public string GenerateSignature(
    Uri url,                         // base URL for the request
    string consumerKey,              // consumer key provided by Netflix
                                     // for your application
    string consumerSecret,           // consumer secret provided
                                     // by Netflix for your application
    string token,                    // not used in this example
    string tokenSecret,              // not used in this example
    string httpMethod,               // HTTP method, e.g. "GET"
    string timeStamp,                // timestamp in UNIX time
    string nonce,                    // unique message identifier
    out string normalizedUrl,        // fully qualified URL (returned)
    out string normalizedRequestParameters) // sorted, URL-encoded,
                                            // query string parameters (returned)

The following code snippet demonstrates how the OAuth class is used to create a signed request:

OAuth.OAuthBase oauth = new OAuth.OAuthBase();

// inputs
Uri requestUrl = new Uri("http://api.netflix.com/catalog/titles/");
oauth.AddQueryParameter("term", "Casper");
oauth.AddQueryParameter("max_results", "10");

// outputs
string normalizedUrl;
string normalizedRequestParameters;

// generate request signature
string sig = oauth.GenerateSignature(requestUrl,
                 _myConsumerKey, _myConsumerSecret,
                 null, null,    // token , tokenSecret (not needed)
                 "GET", oauth.GenerateTimeStamp(), oauth.GenerateNonce(),
                 out normalizedUrl, out normalizedRequestParameters);
// construct request
string reqUrl = requestUrl + "?" +
                normalizedRequestParameters +
                "&oauth_signature=" + 
                oauth.UrlEncode(sig);

Note that the OAuthBase class takes care of all the URL encoding required for the query string parameters when it constructs the normalizedRequestParameters output parameter. However, the signature does require URL encoding before it is appended to the rest (ha ha, no pun intended) of the request.

Also note that I'm using the UrlEncode function provided by the OAuthBase class. This is very important! You may be tempted to use the more familiar URL encoder provided by the .NET HttpHelper.UrlEncode method; however, it returns the encoded characters in lowercase. Netflix requires the URL encoded characters to be in uppercase, which is what the OAuthBase class provides. There is at least one tale of woe in the Netflix Developer Forum where this minor detail was overlooked.

The Web Service Request

Whew! Now all we have to do is to fire off the request and (hopefully) get back a bucketful of data. For this example, I'm using a simple synchronous HTTP request using the WebRequest/WebResponse classes.

string results = "";
try
{
    WebRequest req = WebRequest.Create(reqUrl);
    WebResponse rsp = req.GetResponse();
    StreamReader sr = new StreamReader(rsp.GetResponseStream());
    results = sr.ReadToEnd();
    rsp.Close();
}
catch (Exception ex)
{
    txtRequest.Text ("Request failed: " + ex.Message);
    return;
}
// output the results of the results string to the application

Running the Example Application

The example code is an application that searches for Netflix titles. It requires three inputs: your consumer key, your consumer secret, and the term for which to search. Optionally, you can specify the maximum number of results to return, or choose zero to return the default of up to 25 items. The constructed REST request and the resulting response from the Netflix API call are displayed when Search is clicked.

The returned data is an XML document that contains information from the search request. It is fairly obvious what everything means; however, you can also find details in the Netflix API documentation if you need help in understanding it. In a subsequent article in this series, I will show you how to parse and use the data contained in this response.

Conclusion

So, there you have it. I hope this article will save you lots of time and confusion in writing your own Netflix client application. In future editions of this series, I'll demonstrate:

• How the Netflix data is structured and how to efficiently parse the requested information.
• How to obtain permission to access a subscriber's account, and how to view an account using Protected Requests.
• How to modify account information, such as making changes to a subscriber's movie queue, using Protected Requests.

I must also point out that the code example in this article, as in the subsequent articles, will be evolving over the course of this series. Please understand that each one of these examples is written to demonstrate a particular aspect of the Netflix API and therefore they are not intended to represent a complete library of the Netflix API functionality.

Glossary of Terms

One last item is that I want to point out some differences in the terminology that I'll be using in these articles and how it relates to the terminology found in the Netflix documentation. I prefer to use some specific, or perhaps more conventional, terminology, and so here is a cross reference for you to keep in mind as you read these articles.

Netflix Term	Article Term
User	Subscriber
Percent Encoding	URL Encoding
Parameter Ordering	Canonicalization

History

• September 2, 2009 - Original submission
• September 8, 2009 - Updated source code 

You must Sign In to use this message board.

FAQ    Noise Tolerance Very HighHighMediumLowVery Low   Layout Expand Posts & RepliesThread ViewNo JavascriptNo JS + Preview   Per page 102550

Msgs 1 to 9 of 9 (Total in Forum: 9) (Refresh) FirstPrevNext

The complete series NetDave 11:46 29 Sep '09   I should have mentioned this sooner. The complete series has been posted: Unraveling the Netflix API - Part I - Netflix API Basics[^] Unraveling the Netflix API - Part II - Accessing the Netflix Title Catalog[^] Unraveling the Netflix API - Part III - Accessing Subscriber Account Information[^] Unraveling the Netflix API - Part IV - Managing User Account Information[^] QRZ? de WAØTTN Sign In·View Thread·PermaLink Love it billoatman 11:08 29 Sep '09   Looking forward to the next one! -Bill Sign In·View Thread·PermaLink Title list gokul78 21:04 7 Sep '09   Hi Dave Thank You for the excellent series of articles. Is there a way to get a list of titles? List of Genres List of Ratings, etc? Thanks, Gokul Sign In·View Thread·PermaLink Re: Title list NetDave 7:27 8 Sep '09   Hi Gokul, yes, read the rest of the series and I think you'll find everything you're looking for. QRZ? de WAØTTN Sign In·View Thread·PermaLink Great Idea aspdotnetdev 14:48 7 Sep '09   Although I had the same idea, so I'm biased. 5's for all 4 articles, for the idea alone (I haven't read them fully yet, but it looks like you've done good work from a cursory overview). Visual Studio is an excellent GUIIDE. Sign In·View Thread·PermaLink Re: Great Idea NetDave 20:29 7 Sep '09   Thanks! I hope the articles live up to your expectations after you're read them QRZ? de WAØTTN Sign In·View Thread·PermaLink well done Donsw 7:45 7 Sep '09   Good article, I may sign up as a devleoper after reading this. cheers, Donsw My Recent Article : Backup of Data files - Full and Incremental Sign In·View Thread·PermaLink Re: well done NetDave 14:01 7 Sep '09   Thanks very much. I was hoping to end war and famine, but this was the best I could do. QRZ? de WAØTTN Sign In·View Thread·PermaLink Re: well done kiswa00 11:40 10 Sep '09   I'm thinking the same thing. Since I'm already a NetFlix member, it would be fun to play with the API. I'll be reading the other articles soon! Sign In·View Thread·PermaLink

Last Visit: 12:20 9 Feb '10     Last Update: 12:20 9 Feb '10 1