REST Architectural Style

REST Architectural Style

By Mark Volkmann, OCI Partner

November 2004

Introduction

REST is an architectural style, not a standard or API. It was conceived by Roy Fielding in his dissertation in 2000 (see references at end). However, existing standards including URLs, HTTP and XML can be used to implement REST applications. REST is used to build distributed applications such as Web apps and Web services. REST servers are any pieces of software that respond to REST requests. They can be implemented in many ways, including Java servlets. The Web is an example of REST architecture. The many kinds of software running on web servers that return resources are examples of REST servers. Web browsers are a kind of REST client, sending requests to URLs using HTTP.

This article provides an overview of the REST architecture and an example Java implementation.

Overview

REST stands for REpresentational State Transfer. The main ideas behind REST are:

  1. A software component (client) requests a "resource" from a service by supplying a resource identifier and possibly a desired media type.
  2. A "representation" of the resource is returned as a sequence of bytes along with metadata to describe it. The metadata is in the form of name-value pairs and can be represented using HTTP headers.
  3. Obtaining this representation causes the software component to "transfer" to a new "state".

Why Another Approach To Web Services?

Hasn't the W3C already defined everything needed for Web services? Well ... the W3C started simple, but continues to make Web services more complex. Here's a quote from Edd Dumbill of O'Reilly.


REST can be used as a replacement for SOAP, but not WSDL and UDDI. WSDL can be used to describe REST request and response messages. UDDI can be used to catalog and discover REST services. In fact, for distributed applications that don't require the complexity of CORBA, EJB and the like, REST can be used as a replacement for those as well.

REST Characteristics

Resources and Identifiers

A REST resource is a specific, retrievable thing, not an abstract concept. For example, instead of having a "car" resource with representations like "photo" and "sales report," those are the resources. A car photo from a specific view (front, side and rear) is a resource that may have a JPEG representation. A car sales report for a specific car model in a specific month/year is a resource that may have PDF and XML representations.

This quote by Jan Algermissen on the rest-discuss mailing list explains REST resources well when viewed in the context of using HTTP.
 

Resource identifiers are typically URLs. Below are examples of good URLs for the previous resource examples. There are two common styles: using query parameters and extra path information.

http://host:port/webapp/carPhoto?make=BMW&model=Z3&year=2001&view=front
http://host:port/webapp/carPhoto/BMW/Z3/2001/front

http://host:port/webapp/carSales_report?make=BMW&model=Z3&year=2001&salesYear=2004&salesMonth=4
http://host:port/webapp/carSales_report/BMW/Z3/2001/2004/4

An underlying goal is to make as many things as possible retrievable by an HTTP GET request. This enables browser-based testing by simply entering URLs.

Typical REST Choices

REST applications typically use

However, Roy Fielding's dissertation doesn't mandate these and in fact doesn't even mention XML.

RESTful?

"RESTful" is a term often used on the rest-discuss mailing list. When something is non-RESTful it violates the REST philosophy or something specific in Roy Fielding's dissertation. There is a sense that being RESTful is good and being non-RESTful is bad. Of course, specific applications may have well-justified reasons to use parts of REST and be non-RESTful in others.

An example of this is including links to related data in response messages. This is considered a RESTful thing to do. Clients simply issue GET requests to the link URLs to retrieve additional, related data. For some applications, this practice can make response messages prohibitively large. Applications may prefer to include the minimal amount of data needed in response messages that allows client code to construct those URLs themselves.

For example, a RESTful response message describing a music collection might look like the following.

<musicCollection owner="Mark Volkmann" xmlns:xlink="http://www.w3.org/1999/xlink">
   <artist xlink:href=
      "http://myhost:8080/music/collection/Mark+Volkmann/artist/Tori+Amos"/>
   <artist xlink:href=
      "http://myhost:8080/music/collection/Mark+Volkmann/artist/Sting"/>
</musicCollection>

A somewhat non-RESTful response message describing the same thing might look like the following.

<musicCollection owner="Mark Volkmann">
   <artist name="Tori Amos"></artist>
   <artist name="Sting"></artist>
</musicCollection>

Clients of a service returning this kind of non-RESTful response would have to know how to construct the URLs required to obtain detailed information about each artist such as their recordings.

Using HTTP With REST

The following is a summary of the HTTP methods used by REST applications. See RFC 2616 section 9 for more details.

CRUD is an acronym that stands for Create, Retrieve, Update, Delete. A "CRUD matrix" is often used to document requirements for each major type of data (resource) in an application.

Example

This example will demonstrate use of a simple Java framework for building REST applications. Use of Java and a framework like this is not required in order to implement REST servers or REST clients. The framework merely makes implementing them a bit easier.

The steps we will follow are:

  1. Determine the required resource types.
  2. Determine the required operations (HTTP methods) for each resource type.
  3. Assign URLs to each resource.
  4. Create Java Bean classes to model each resource type.
  5. Write schemas to describe request and response XML messages.
  6. Write a class to convert resource objects to and from XML.
  7. Create a servlet for each resource type to handle its requests.
  8. Create a deployment descriptor (web.xml) that registers servlets to use for each resource type.
  9. Build a WAR file that includes web.xml, servlets, REST.jar (contains framework classes).
  10. Write client code that uses HTTP framework class to invoke REST services.

Framework Classes

Our Java framework for building REST clients and servers contains the classes shown in the following diagram.

Framework Classes

Step 1: Resource Types

The resource types we need to support are:

Step 2: Required Operations

In this application, we want the ability to:

Step 3: Resource URLs

Our URLs conform to the following pattern:

http://host:port/web-app-name/resource-type[/resource-id]

For example,
http://localhost:8080/MusicCollection/artist/Mark+Volkmann/Sarah+McLachlan 
can be used with
GET to retrieve an XML description of the artist,
DELETE to delete the artist,
PUT to add a recording to the artist (content is an XML description of the recording) and
POST to perform other operations on the artist

Step 4: Resource Classes

We need a basic Java Bean class to model each resource type. The classes we'll create are MusicCollection, Artist and Recording. We'll put each of these classes in the com.ociweb.music.resource package.

Note that if we started with an XML Schema that described each resource, we could generate these classes using JAXB.

Step 5: Schema

We'll create an XML schema to describe the valid XML that can be used in our request and response messages. While REST doesn't require creating XML schemas, there are two reasons to do so. The first is to provide documentation of the expected message contents. The second is to support validation of the messages at runtime on the client-side, server-side or both.

Any XML schema language can be used. Popular choices include DTDs, XML Schema, RELAX NG and Schematron. We'll use the RELAX NG compact syntax.

How strict should we make the schemas? To provide backward compatibility when new capabilities are added to existing REST servers, it is useful to make them "loose-in, strict-out". "loose-in" means the server accepts request messages that are missing data used by later versions. "strict-out" means the responses produced by the server are strictly defined, so clients know exactly what to expect. To support this, servers must be written to recognize the server version clients want to use and generate response messages in the format advertised by each version. These practices allows older clients to continue using new versions of servers without modification. This is easier to implement when messages contain XML than when messages contain serialized Java objects (as used by Java RMI and EJB).

Many people prefer RELAX NG over the alternatives because of its simpler syntax. It can use data types defined by XML Schema. Two of these, xsd:boolean and xsd:int, are used below. A RELAX NG schema is composed of patterns. The start pattern defines what constitutes a valid root element. In our case there are three choices. As proof of how easy the RELAX NG syntax is to understand, I'll bet you can figure out what the following schema allows without any further explanation!

start = collection-element | artist-element | recording-element
 
collection-element = element collection {
  attribute owner {text}
}
 
artist-element = element artist {
  attribute owner {text}
  attribute name {text}
  attribute solo {xsd:boolean}
  attribute leadVocalGender {"F"|"I"|"M"}
}
 
recording-element = element recording {
  attribute artist {text}
  attribute title {text}
  attribute format {"A"|"C"|"D"}
  attribute year {xsd:int}

Step 6: XML Conversion

One of the easier Java APIs to use for working with XML is JDOM. We'll use it to write the code that converts resource objects to and from XML.

The framework class XMLAdapterBase provides methods that make JDOM even easier to use for our purposes. They simplify getting and setting attribute values of various data types on JDOM Element objects. XMLAdapterBase holds onto the "current element" so the methods that get and set attributes don't require a parameter to specify the element on which they are operating. This tidies up the code a bit when a large number of attributes are being processed.

Here are some example methods from our XMLAdapter class that extends XMLAdapterBase.

  /**
   * Converts a MusicCollection object to a JDOM Element.
   */
  public Element toElement(MusicCollection collection) {
    setCurrentElement(new Element("collection"));
    setAttribute("owner", collection.getOwner());
    return getCurrentElement();
  }
 
  /**
   * Converts a MusicCollection object to a JDOM Document.
   */
  public Document toDocument(MusicCollection collection) {
    return new Document(toElement(collection));
  }
 
  /**
   * Converts a JDOM Document to a MusicCollection object.
   */
  public MusicCollection collectionFromXML(Document doc) {
    setCurrentElement(doc.getRootElement());
    String owner = getStringAttribute("owner");
    return new MusicCollection(owner);
  }

It is not necessary to manually write the code to perform these conversions. Libraries like JAXB and Castor XML can automate this. However, when writing a REST client for a REST service that you didn't write, care must be taken to ensure that the XML used in the client exactly match what the REST service expects. This includes the order of child elements and the choice between using attributes or child elements to hold each atomic value. If the service provides an XML Schema then JAXB can be used to generate classes that know how to convert their objects to and from XML. Otherwise, manually writing conversion methods like those above will likely be required. Keep in mind though that using a tool to generate XML from objects will make it more difficult to support the "loose-in, strict-out" concept discussed earlier.

Step 7: Servlets

We need a servlet to handle requests for each resource type. The servlets we'll create are MusicCollectionServlet, ArtistServlet and RecordingServlet. We'll put each of these servlet classes in the com.ociweb.music.server package. These servlets extend com.ociweb.rest.RESTServlet which somewhat simplifies writing them.

As an example of what the servlets must do, methods from ArtistServlet appear below. In order to simply the code, instead of storing resource data in a database, we're just holding it in a static HashMap. There is a HashMap for each resource type and a HashMap that holds each of those HashMaps. The details behind this part of the application are not relevant to understanding REST.

The doPut method handles requests to add a new recording to an existing artist (when the root element of the XML in the request body is "recording") or modify an existing artist (when the root element is "artist"). Notice how the IO framework class is used to extract XML from the request body. Also notice how the getResourceURL method, from the RESTServlet base class, is used to construct the URL of the resource that has just been modified/created. This URL is written to the response body using the IO framework class.

  protected void doPut(HttpServletRequest req, HttpServletResponse res)
  throws IOException, ServletException {
    Document doc = IO.readXML(req);
    String rootName = doc.getRootElement().getName();
    XMLAdapter adapter = new XMLAdapter();       
    if ("artist".equals(rootName)) {
      // Modify an existing artist.
      Artist artist = adapter.artistFromXML(doc);
      Map map = getMap(req, "artist");
      map.put(artist.getName(), artist);
      String url = getResourceURL(req, RESOURCE_TYPE, artist.getName());
      IO.write(res, url);
    } else if ("recording".equals(rootName)) {
      // Add a new recording to an existing artist.
      Recording recording = adapter.recordingFromXML(doc);
      Map map = getMap(req, "recording");
      map.put(recording.getTitle(), recording);
      String url = getResourceURL(req, "recording", recording.getTitle());
      IO.write(res, url);
    } else {
      writeTextResponse(res, rootName + " cannot be PUT in an artist");
    }                
  }

The doGet method handles requests to retrieve an XML representation of an artist from a music collection. Notice how the XMLAdapter class is used to convert Artist objects to XML. Also notice how the IO framework class is used to write the XML into the response body.

 protected void doGet(
  HttpServletRequest req, HttpServletResponse res)
 throws IOException, ServletException {
  String artistName = getResourceId(req);
  Map map = getMap(req, RESOURCE_TYPE);
  XMLAdapter adapter = new XMLAdapter();
  if (artistName == null) {
    writeErrorResponse("artist name missing from URL");
  } else {
    Artist artist = (Artist) map.get(artistName);
    if (artist == null) {
      writeErrorResponse(res, "no artist named " + artistName + " found");
    } else {
      Document doc = adapter.toDocument(artist);
      IO.writeXML(res, doc);
    }
  }
}

The doDelete method handles requests to delete an artist from a music collection. When the resource id is "ALL," all artists are deleted. Otherwise a specific artist is deleted.

 protected void doDelete(
  HttpServletRequest req, HttpServletResponse res)
 throws IOException, ServletException {
  Map map = getMap(req, RESOURCE_TYPE);
  String artistName = getResourceId(req);
  if ("ALL".equals(artistName)) {
    map.clear();
  } else {
    map.remove(artistName);
  }
}

The doPost method handles requests for all other artist-related operations. The only operation currently supported is getRecordingCount. Our convention is to use the name of the root element of the XML in the request body to indicate the name of the operation to be performed. getRecordingCount is a method in the servlet. Another option would be use reflection to invoke the method corresponding to the operation name.

 protected void doPost(
  HttpServletRequest req, HttpServletResponse res)
 throws IOException, ServletException {
  String artistName = getResourceId(req);
  Document doc = IO.readXML(req);
  String operation = doc.getRootElement().getName();
 
  String content = "getRecordingCount".equals(operation) ?
    String.valueOf(getRecordingCount(req, artistName)) :
    "unsupported operation " + operation;
  IO.write(res, "text/plain", content);
}

Step 8: Deployment Descriptor

Java Web applications use the file web.xml to describe details about deployment. The only things we need to specify are

  1. a shorthand name for each servlet class
  2. a mapping from URL patterns to shorthand names

Here's our web.xml deployment descriptor file.

<?xml version="1.0"?>
<!DOCTYPE web-app
  PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.2//EN"
  "http://java.sun.com/j2ee/dtds/web-app_2_2.dtd">
<web-app>
 
  <servlet>
    <servlet-name>artist</servlet-name>
    <servlet-class>com.ociweb.music.server.ArtistServlet</servlet-class>
  </servlet>
  <servlet>
    <servlet-name>collection</servlet-name>
    <servlet-class>com.ociweb.music.server.MusicCollectionServlet</servlet-class>
  </servlet>
  <servlet>
    <servlet-name>recording</servlet-name>
    <servlet-class>com.ociweb.music.server.RecordingServlet</servlet-class>
  </servlet>
 
  <servlet-mapping>
    <servlet-name>artist</servlet-name>
    <url-pattern>/artist/*</url-pattern>
  </servlet-mapping>
  <servlet-mapping>
    <servlet-name>collection</servlet-name>
    <url-pattern>/collection/*</url-pattern>
  </servlet-mapping>
  <servlet-mapping>
    <servlet-name>recording</servlet-name>
    <url-pattern>/recording/*</url-pattern>
  </servlet-mapping>
 
</web-app>

Step 9: WAR

A Web Application Archive (WAR) contains all the files required to deploy a Java Web application. Ours is built using the following Ant target. It includes all JAR files in the lib directory of the project which contains jdom.jar and REST.jar. REST.jar contains our REST framework classes.

<target name="war" depends="compile" description="builds the war file">
  <war warfile="${war.file}" webxml="src/web/web.xml">
    <classes dir="build/classes"/>
    <lib dir="lib" includes="*.jar"/>
  </war>
</target>

It is deployed to a Tomcat server by copying it to Tomcat's webapps directory.

Step 10: Client

Now for the client code! The XMLAdapter framework class is used to convert resource objects to XML. The URLUtil framework class is used to construct the URLs to which HTTP requests are sent. The HTTP framework class is used to send all HTTP requests.

package com.ociweb.music.client;
 
import com.ociweb.music.resource.*;
import com.ociweb.rest.HTTP;
import com.ociweb.rest.URLUtil;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import org.jdom.Document;
 
public class Client {
  private static final String MARK = "Mark Volkmann";
  private static final String SARAH = "Sarah McLachlan";
  private static final String URL_PREFIX =
    "http://localhost:8080/MusicCollection";
 
  private static URLUtil urlUtil = new URLUtil(URL_PREFIX);
 
  public static void main(String[] args) throws IOException {
    XMLAdapter adapter = new XMLAdapter();
 
    // Clear out any objects previously created on the server.
    HTTP.delete(urlUtil.getURL("collection", "ALL"));
    HTTP.delete(urlUtil.getURL("artist", "ALL"));
    HTTP.delete(urlUtil.getURL("recording", "ALL"));
 
    // Create a music collection resource.
    MusicCollection collection = new MusicCollection(MARK);
    Document doc = adapter.toDocument(collection);
    String url = HTTP.put(urlUtil.getURL("collection", null), doc);
    System.out.println("collection url = " + url);
 
    // Create an artist resource,
    // adding it to the previously created music collection.
    Artist artist = new Artist(MARK, SARAH, true, Artist.FEMALE);
    doc = adapter.toDocument(artist);
    url = HTTP.put(new URL(url), doc);
    System.out.println("artist url = " + url);
 
    // Create a recording resource,
    // adding it to the previously created artist.
    Recording recording =
      new Recording(SARAH, "Afterglow", Recording.CD, 2003);
    doc = adapter.toDocument(recording);
    url = HTTP.put(new URL(url), doc);
    System.out.println("recording url = " + url);
 
    // Get XML representations of all the resources that have been created.
    System.out.println(HTTP.get(urlUtil.getURL("collection", MARK)));
    System.out.println(HTTP.get(urlUtil.getURL("artist", SARAH)));
    System.out.println(HTTP.get(urlUtil.getURL("recording", "Afterglow")));
 
    // Invoke a non-CRUD operation on an artist resource.
    System.out.println("recording count for " + SARAH + " is "
      + HTTP.post(urlUtil.getURL("artist", SARAH), "<getRecordingCount></getRecordingCount>"));
 
    // Invoke a non-CRUD operation on a music collection resource.
    System.out.println("artists in collection for " + MARK + " is "
      + HTTP.post(urlUtil.getURL("collection", MARK), "<getArtists></getArtists>"));
  }    
}

The output from this code follows.

collection url =
  http://localhost:8080/MusicCollection/collection/Mark+Volkmann
artist url =
  http://localhost:8080/MusicCollection/artist/Sarah+McLachlan
recording url =
  http://localhost:8080/MusicCollection/recording/Afterglow
 
<collection owner="Mark Volkmann" />
<artist owner="Mark Volkmann" name="Sarah McLachlan"
  solo="true" leadVocalGender="F" />
<recording artist="Sarah McLachlan" title="Afterglow"
  format="D" year="2003" />
 
recording count for Sarah McLachlan is 1
artists in collection for Mark Volkmann is
<artists>
  <artist owner="Mark Volkmann" name="Sarah McLachlan"
    solo="true" leadVocalGender="F" />
</artists>

Java and HTTP

So how does Java send HTTP requests and receive HTTP responses? The framework classes HTTP and IO hide these details. Here's what they do.

To send an HTTP request to a given URL in client code, we do the following. The variable method can be set to "PUT", "GET", "DELETE" or "POST".

URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod(method);
conn.setDoOutput(true);
if (content != null) {
  Writer writer = new OutputStreamWriter(conn.getOutputStream());
  writer.write(content);
  writer.flush();
}

To read the contents of an HTTP response in client code, we do the following.

BufferedReader reader =
  new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuffer sb = new StringBuffer();
while (true) {
  String line = reader.readLine();
  if (line == null) break;
  sb.append(line);
}
reader.close();
String response = sb.toString();

To read the contents of an HTTP request in server code, just replace the conn variable in the code above with the parameter that holds the HTTPServletRequest object.

To send an HTTP response from server code, we do the following. The variable res is an HttpServletResponse object. Common values for mimeType are text/plain, text/html and text/xml.

res.setContentType(mimeType);
PrintWriter writer = res.getWriter();
writer.write(content);
writer.flush();

REST vs. SOAP

Earlier we said that REST could be used as a replacement for SOAP. What are the pros and cons of doing this?

Pros

Cons

Comparison of SOAP and REST request messages

Here's the content of an example REST message that uses HTTP PUT to add a new artist to an existing music collection. The content doesn't need to specify the operation to be performed because the fact that HTTP PUT is being used does that.

<artist owner="Mark Volkmann" name="Sarah McLachlan" 
  solo="true" leadVocalGender="F"/>

Here's what an equivalent SOAP message might look like when SOAP encoding is used. The first element inside the SOAP body indicates the operation to be performed.

<soap:Envelope
  xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
  <soap:Body>
    <addArtist xmlns="some-namespace-uri">
      <artist owner="Mark Volkmann" name="Sarah McLachlan" 
        solo="true" leadVocalGender="F"/>
    </addArtist>
  </soap:Body>
</soap:Envelope>

The xs and xsi namespace prefixes are used to indicate datatypes of text inside XML elements in the SOAP body. Our example has no such text, but if it did and the datatype of that text was an arbitrary string, the element would have the attribute xsi:type="xs:string".

SOAP messages are somewhat smaller when literal encoding is used because the lines starting with xmlns:xs, xmlns:xsi and soap:encodingStyle can be removed.

Summary

REST is sufficient for a large percentage of distributed applications. It is simpler and more portable, in terms of the number of programming languages that can easily use it, than other approaches. However, marshalling and unmarshalling of data is typically left up to developers.

If REST is so great, why hasn't it taken off more? I believe these are the main reasons.

If you'd like a copy of the framework classes and the example application discussed in this article, request it by sending email to mark@ociweb.com. There are no usage restrictions.

References

Software Engineering Tech Trends (SETT) is a regular publication featuring emerging trends in software engineering.