Everything About REST Web Services - What and How - Part 1

Introduction

REST stands for Representational State Transfer. A Yahoo or Google search on "REST web services" brings up tons of information. But if you are like what I was a few weeks ago – you will probably find all that information more annoying than helpful, because you want to create a REST web service in a hurry. We have, consciously or not, all become very used to tailor made instructions from the internet – and we love nothing more than a step by step guide to do something. Evil as much as it is, a "how-to" guide is all we love to dig out. In this particular web search, I was denied just that.

Soon I found out that writing a REST web service has no shortcuts – you have to first fully understand what REST is. I have a wonderful analogy – think of a C++ or JAVA or .NET newbie trying to find a spoon feeding article on "How to do object oriented design". Well, good luck! REST is just that – experts coin it as an "architectural style" – but I am writing this article in my own words, so I will steer clear of stuff that confused me in the first place. I think REST is where architecture and design merge, and the thin line between the two disappears.

Let us get straight to business. I will start with what a traditional web service developer who has no idea about REST needs to know. I will not start shooting jargons at the beginning, which is like attempting to create method from chaos. Instead, I will stay methodical the whole way.

Transitioning from Traditional Web Service to REST Design

Think of what a "traditional web service" is. It is an interface with exposed "methods." Clients know the methods' name, input and output and hence can call them.

Now imagine an interface that does not expose "methods". Instead, it exposes "objects". So when a client sees this interface, all it sees is one or more "objects". "An object" has no input and output – because "it does not do anything". It is a noun, not a verb. It is "a thing", not "an action".

For example, think of a traditional web service that provides the current weather conditions if you provide it with a city. It probably has a web method like GetWeatherInfo() which takes a city as input and provides weather data as output. It is easy for you so far to understand how a client will consume this web service.

Now imagine, in the place of the above web service, there is a new one that exposes cities as objects. So, when you look at it as a client, instead of GetWeatherInfo(), you see New York, Dallas, Los Angeles, London and so on. And these cities do not have any application specific methods hanging from them - they are apparently like inert gases - they themselves do not react.

You must be thinking – well, how does that help you, as a client, to get to the weather of Dallas? We will get to that in a few moments.

If all you get from a web service is a "set of objects", obviously you need a way to "act on them". The objects themselves have no methods for you to call, so you need a set of actions that you can apply onto these objects. In other words, you need to "apply a verb to the noun". If you see an object, say, an apple, which is "a noun", you can apply "a verb" like eat, to it. But not all verbs can be applied to all nouns. Like, you can drive a car, but cannot drive a television.

Thus, if a web service exposes only objects, and you are asked – well, let us now design a few standard actions or verbs that "all clients can apply to all objects they see", what verbs will you choose? Will you choose drive? No. Because then that action cannot be applied to most of the exposed objects. What are the verbs that "can be applied to all nouns?" I don't know about you, but GET comes to my mind first!

Yes, you can 'get' an apple, 'get' a car and even 'get' a city! From a programmatic standpoint, "to get" means "to retrieve information about".

GET has an opposite – PUT. "To put" means "to update information about". Wait a minute... are we sounding uncannily close to HTTP?

When you type a URL on your browser, the sequence of actions that happen underneath is ghoulishly close to what I just described! So your browser is supposed to display stuff about the URL you typed in, but where does it get that stuff from? If you treat that URL as "the object", now the browser needs to get information "on that object" from a remote server. Therefore, it sends a HTTP GET request.

HTTP is a "protocol". And it says: a client (in this example, the browser) can send a request to a server which is either a GET or a PUT or a POST or a DELETE (there are other verbs as well – HTTP 1.1 supports 8 verbs, but let us focus on these 4 for now). So if you pause for a moment and think that the entire cobweb called internet actually works based on these verbs, suddenly you will realize how strong and great these verbs are – suddenly your search for a set of universal actions will be over. The answer is right under your nose: the HTTP verbs. Virtually anything you do over the web is done by using one of these verbs – which is proof enough that these verbs should be "exhaustive". Which means if a web service that exposes only objects supports these four operations "on its exposed objects", any client should be able to do anything with that web service!

Congratulations, you have transitioned to a REST web service! A REST web service is one that exposes only objects, and supports a set of verbs on those objects. So returning to our weather example, your web service that exposes cities will also support GET operation on the cities. So when a client sends a request that reads like "GET LOS ANGELES", the response could be Los Angeles' weather data. The response could be any other information about Los Angeles as well – like its population or a set of images or its history. But your web service is probably not going to do all of them.

So choose a good domain name for your web service – say weatherinfo. Thus, a client request like

GET http://weatherinfo.com/45327 HTTP/1.1 

where 45327 is the city id for Los Angeles, is a valid request for your web service. Your web service could respond with:

1 HTTP/1.1 200 Ok
2 Date: Sat, 03 Nov 2007 07:35:58 GMT
3 Content-Type: text/xml
4 Content-length: 139
5
6 <City name="Los Angeles" datetime="2007-11-03 07:35:58 GMT" >
7 <Condition>Overcast</Condition>
8 <Temp>69.5</Temp>
9 <Humidity>80</Humidity>
10 </City> 

Line 1 is the initial line, lines 2 through 4 are the HTTP headers (there can be lots of headers, only 3 shown here), line 5 is the mandatory blank line separating header and body, and lines 6 through 10 constitute the "HTTP Body (or content)" – this part is the data that the response carries and can be in any format, not necessarily XML. In fact, the most commonly used format on the web is HTML – one that web servers use to send back data to browsers. Whatever it is, the "Content-type" header usually specifies it. But if you are writing a web service, XML is a better choice, but that is just me. If your web service does not return complex or composite data, the format does not need to be XML – it can be text/plain, in which case the body will just be a string of characters.

There is the input and output of your first REST web service!

REST is not Just About Web Services

HTTP creators had the same goals in mind – a universal and simple protocol for client server communication. So we are not really re-inventing the wheel – writing a REST "web service" is not that different than writing a "web server application". Isn't that how it should be? Because it is all web. But there are significant differences as well – not in terms of technology, but in terms of use. Let us deal with these differences for a while.

Firstly, when a browser (think client) issues the HTTP request:

GET http://www.yahoo.com HTTP/1.0 

The browser is expecting back "lots of data" – text, headlines, images and links – whatever you see on the Yahoo home page. All that data comes back to your browser (as HTTP body) in "HTML format". But when a client calls a "web service" to get some information, it is usually some specific information. Thus, the first difference between a web application and a web service is that the HTTP body or content in response to GET significantly varies in content and format.

Secondly, a browser hardly needs to issue any command other than GET, so most web applications are not coded to react to, say, PUT or DELETE. However, POST is handled by many servers, but that is mostly because POST is the most misused HTTP verb – more often than not, a web application requires clients to send a POST request not because POST is the right thing, but because POST can be flexibly used for any generic request that brings some incoming data and takes back some outgoing data.

However, a properly designed REST web service should handle GET, POST, PUT, DELETE and HEAD operations. We will go over the details of these operations in the section "Steps of Designing a REST Web Service", which will be in the second part of this series.

The point of this section is: "A web service differs from a website only in usage." Everything else is the same – in fact, the request/ response protocol can also be the same. The HTTP verbs are adequate to be used by any web service because they have successfully modeled all online communication. So why do we write a web service creating our own actions or verbs? Creating a web method like GetWeatherInfo() is nothing but creating your own verb. This style, understandably, was a natural extension of our desktop programming style – classes and interfaces have methods, so a web service must have methods as well. So why do we call it a web service?

These are not just questions that challenge a long standing and successful traditional way of doing things. These questions actually help us understand why SOAP was invented. Do you know why SOAP was invented? SOAP was invented to marry these different styles – the style of web and the style of service (where service = interface or class). A web request uses HTTP protocol, but somehow it has to convey the name of the action desired – GetWeatherInfo. Not only that, it has to carry in parameters, and carry out return data. All that is achieved by a complex concoction called SOAP.

But we had to invent that concoction only because we were too lazy to let go the traditional "method call way of thinking".

The Two Big Challenges of Thinking Like the Web

You might ask – it is probably not that easy to replace the traditional web method calls with the simple HTTP actions to achieve everything. You are right – it is not easy. But the effort spent is well worth it. Let us examine the two big challenges.

• First – effectively replacing all operations with a few simple verbs needs guts
• Second – a web like request/ response mechanism means "you have to embrace statelessness"

Let us take these up one by one. We understood how we can replace GetWeatherInfo. But how will we replace GetLoanApprovalDecision? Or in an e-commerce situation, how will we replace Buy?

Designing a REST web service, as you will learn, essentially consists of two steps: decide on what objects you will expose, and then, decide how will you react to GET, PUT, POST and DELETE on each of those objects? Those are the weapons you have in your arsenal. Are they limited or restrictive in any way? In a way, they are, because you cannot design a web method such as GetLoanApprovalDecision. But that limitation is good – it forces all of us to design web services in a standard way, enforcing some simple and powerful rules which are by no means inadequate. Again, it is like harnessing the power of object oriented methodology – given a long and complex C or VB6 program, can you convert it to an object oriented design and still do the same thing? Of course you can. If you can't, you need to go back to the classroom again; you just cannot say OOD is not good enough for this scenario. In my view, REST design is just like that – if you cannot possibly think of a way to model GetLoanApprovalDecision in a REST-like manner, go get a text book on REST. REST can do it, but you are not ready to think it out.

My next article on this topic (Part 2) will start with "Steps of Designing a REST Web Service" – in which I will try to go over the thought processes we need to have to be able to do that.

Moving on to the second challenge, statelessness: The essence of statelessness is that any call should not need to refer to any previous call. All calls are independent. The server does not need any knowledge of what happened during a previous call while processing a particular call. You want an example? What about a web service where a client has to call Login() first and then call the rest of the stuff? Is that stateless? No. Because while processing the second call, the web service has to remember that this user has already logged in using a previous call. The "logged-in state" is preserved in the web service. Client does not need to pass the credentials again with the second or third call. This is "not" statelessness, rather "statefulness". REST does not allow this because a truly extensible and scalable web service (or for that matter, website) should not be stateful - it should be stateless. State complicates matters so much that it is never worth it. Allowing state to creep in between calls does not let you process subsequent client requests on different machines. I will not go into the details of pros and cons of having state – this article is not about that. But the point is we often design our traditional web services with state built in. When we move to REST, we can no longer do that, as Roy Fielding has expressly prohibited state in REST. Who is Roy Fielding? He is a guy whose name is on the page of the HTTP specifications committee. He is the person who used the term Representational State Transfer for the first time.

In the next part of this series, I will take up the specifics of designing a REST web service, and touch upon its implementation as well. We will see why REST (arguments "for" REST), how to build a REST service and a REST client.