Apache Thrift

Apache Thrift

By Andrew Prunicki, OCI Senior Software Engineer

June 2009

Introduction

Thrift is a framework for creating interoperable and scalable services. Thrift was originally developed at Facebook, and contributed to Apache in order to foster greater use. Thrift is released under the Apache 2.0 license.

Through a simple and straight-forward Interface Definition Language (IDL), Thrift allows you to define and create services that are both consumable by and serviceable by numerous languages. Using code generation, Thrift creates a set of files that can then be used to create clients and/or servers. In addition to interoperability, Thrift can be very efficient through a unique serialization mechanism that is efficient in both time and space.

The choice of programming language at Facebook is based on what language is best suited for the task at hand. While pragmatic, this flexibility resulted in some difficulties when these applications needed to call one another. After analysis, Facebook's engineers did not find anything currently existing that met their needs of interoperability, transport efficiency, and simplicity (amongst others). Out of this need, Facebook's engineers developed efficient protocols and a services infrastructure that became Thrift. Facebook now uses Thrift for their back-end services - the reason for which it was designed.

This article is structured in the following topics:

• Thrift Architecture
• Supported Protocols, Transports and Servers
• Creating a Thrift Service
• Comparing Thrift
• Conclusion

Thrift Architecture

Thrift includes a complete stack for creating clients and servers. The diagram below depicts the Thrift stack.

The top portion of the stack is generated code from your Thrift definition file. Thrift services result in generated client and processor code. These are represented by the brown boxes in the diagram. The data structures that are sent (other than built-in types) also result in generated code. These result in the red boxes. The protocol and transport are part of the Thrift runtime library. Therefore with Thrift, you can define a service, and are free to change the protocol and transport without re-generating your code.

Thrift also includes a server infrastructure to tie the protocols and transports together. There are blocking, non-blocking, single and multithreaded servers available.

The "Underlying I/O" portion of the stack differs based on the language in question. For Java and Python network I/O, the built-in libraries are leveraged by the Thrift library, while the C++ implementation uses its own custom implementation. 

Supported Protocols, Transports and Servers

Thrift allows you to choose independently between your protocol, transport and server. With Thrift being originally developed in C++, Thrift has the greatest variation among these in the C++ implementation.

Thrift supports both text and binary protocols. The binary protocols outperform the text protocols, but there are times when the text protocols may be useful (such as in debugging). Some of the protocols Thrift supports:

• TBinaryProtocol - A straight-forward binary format encoding numeric values as binary, rather than converting to text.
• TCompactProtocol - Very efficient, dense encoding of data (See details below).
• TDenseProtocol - Similar to TCompactProtocol but strips off the meta information from what is transmitted, and adds it back in at the receiver. TDenseProtocol is still experimental and not yet available in the Java implementation.
• TJSONProtocol - Uses JSON for encoding of data.
• TSimpleJSONProtocol - A write-only protocol using JSON. Suitable for parsing by scripting languages
• TDebugProtocol - Uses a human-readable text format to aid in debugging.

While the above protocols describe "what" is transmitted, Thrift's transports are the "how". Here are a number of transports that Thrift supports:

• TSocket - Uses blocking socket I/O for transport.
• TFramedTransport - Sends data in frames, where each frame is preceded by a length. This transport is required when using a non-blocking server.
• TFileTransport - This transport writes to a file. While this transport is not included with the Java implementation, it should be simple enough to implement.
• TMemoryTransport - Uses memory for I/O. The Java implementation uses a simple ByteArrayOutputStream internally.
• TZlibTransport - Performs compression using zlib. Used in conjunction with another transport. Not available in the Java implementation.

Lastly, Thrift provides a number of servers:

• TSimpleServer - A single-threaded server using std blocking io. Useful for testing.
• TThreadPoolServer - A multi-threaded server using std blocking io.
• TNonblockingServer - A multi-threaded server using non-blocking io (Java implementation uses NIO channels). TFramedTransport must be used with this server.

Thrift allows only one service per server. Although this is certainly a limitation, this can be accommodated through a work-around. By defining a composite service that extends all of the other services that a given server should process, a single server can thus accommodate multiple services. If this work-around is insufficient for your needs, you can always create multiple servers. This scenario would mean however you would be using more resources than necessary (ports, memory, etc.).

TCompactProtocol

Given that the TCompactProtocol is the most-efficient method in the Java implement of Thrift and the example used throughout this article, some further explanation of the protocol is warranted. This protocol writes numeric tags for each piece of data. The recipient is expected to properly match these tags with the data. If the data is not present, there simply is no tag/data pair.

For integers, the TCompactProtocol performs compression using Variable-Length Quantity (VLQ) encoding from the MIDI file format. VLQ is a relatively simple format that uses 7 of 8 bits out of each byte for information, with the 8th bit used as a continuation bit. VLQ's worst-case encoding is acceptable. For a 32bit int, it is 5 bytes. For 64 bit ints, it is 10 bytes. The diagram below shows how the decimal value 106903 (0x1A197) is represented in VLQ, saving 1 byte if it was stored in 32 bits:

Creating A Thrift Service

Creating a Thrift service first requires creating a Thrift file describing a service, generating the code for the service, and finally writing some supporting code to start the service and client code to call it.

Definition

A Thrift definition file should be familiar to anyone who knows any language with a c syntax. Thrift files use keywords such as struct and int as well as curly braces for containing types and parenthesis for parameter lists. Thrift allows you to transfer both simple and complex types. Let's create a service for managing courses and students.

senum PhoneType {
  "HOME",
  "WORK",
  "MOBILE"
  "OTHER"
}
 
struct Phone {
  1: i32    id,
  2: string number,
  3: PhoneType type
}

Here we have a Phone which contains an id, number and type. The phone number is a simple string, while the type is an enumeration limited to the values "HOME", "WORK", "MOBILE" and "OTHER". Thrift uses the struct keyword to define a simple structure and senum for an enumeration. The Java generated code will create a POJO class for struct as you might expect. Disappointingly senum will not result in an Enum, but rather a simple Java String in the Phone class.

Note the numeric identifiers preceding each element in the struct. These identifiers are used during serialization/deserialization to speed parsing and minimize the size of the metadata. These numeric identifiers are what are passed over the wire rather than the string names of elements.

struct Person {
  1: i32    id,
  2: string firstName,
  3: string lastName,
  4: string email,
  5: list<Phone> phones
}
 
struct Course {
  1: i32    id,
  2: string number,
  3: string name,
  4: Person instructor,
  5: string roomNumber,
  6: list<Person> students
}

Here we have two more structs - Person and Course. Notice that these structs build on one another. The Person contains a list of Phones, while Course contains an instructor (one Person) and students (list of Persons). Thrift supports multiple collection types - list, set and map.

This completes the setup of our types. Let's move on to services.

service CourseService {
  list<string> getCourseInventory(),
  Course getCourse(1:string courseNumber) throws (1: CourseNotFound cnf),
  void addCourse(1:Course course) throws (1: UnacceptableCourse uc)
  void deleteCourse(1:string courseNumber) throws (1: CourseNotFound cnf)
}

Here we have a defined a single service with 4 methods. Note that the arguments to service methods also require ordinals, just like structs. Also note that services can declare thrown exceptions and again each exception requires an ordinal:

exception CourseNotFound {
  1: string message
}
 
exception UnacceptableCourse {
  1: string message
}

Before moving on to code generation, Thrift supports namespaces. For each namespace, you declare the language binding, as well as the namespace. In Java, this defines the package into which the generated code is created. The namespace declaration used in the example is below:

namespace java com.ociweb.jnb.thrift.gen
namespace py com.ociweb.jnb.thrift

This completes our Thrift file.

Code Generation

Thrift supports many languages too varying degrees. The complete list is below. Be careful before assuming that just because your language has some support that it supports all of the features of Thrift. Python for instance, only supports TBinaryProtocol.

• Cocoa
• C++
• C#
• Erlang
• Haskell
• Java
• OCaml
• Perl
• PHP
• Python
• Ruby
• Smalltalk

With this being the Java News Brief, this article will focus on the Java side of Thrift. Python will also be used in order to show how Thrift does indeed support cross-language development. The sample thrift file "course.thrift" used throughout this article requires the following invocations to generate Java and Python code, respectively:

• thrift --gen java course.thrift
• thrift --gen py course.thrift

The Thrift code generator is written in C++. Before running the Thrift code generation tool, you will need to build Thrift from source. If you don't care to bother for this article, simply skip this part and move on. The sample code has all you need to run the Java examples. If you decide to build it, the Thrift wiki has a page explaining this in sufficient detail (see references below).

The thrift code generator produces the following files for java and python, respectively:

|-- gen-java
|   `-- com
|       `-- ociweb
|           `-- jnb
|               `-- thrift
|                   `-- gen
|                       |-- Course.java
|                       |-- CourseNotFoundException.java
|                       |-- CourseService.java
|                       |-- Person.java
|                       |-- Phone.java
|                       `-- UnacceptableCourseException.java
`-- gen-py
    |-- __init__.py
    `-- com
        |-- __init__.py
        `-- ociweb
            |-- __init__.py
            `-- jnb
                |-- __init__.py
                `-- thrift
                    |-- CourseService-remote
                    |-- CourseService.py
                    |-- __init__.py
                    |-- constants.py
                    `-- ttypes.py

Taking a look at the Java results, an individual file was created for each Thrift struct and exception, as you might expect. The senum thrift type did not result in a Java Enum as mentioned above, however. Rather, it resulted in a simple Java String inside the Phone class with a comment in the validate method stating that this is where the values for the type should be verified. Protocol Buffers on the other hand, did produce a Java Enum for the equivalent definition (see source for details). Lastly, a CourseService.java file was generated. This file contains classes to create clients and servers.

The Python results are again what you might expect. All the Thrift struct types as well as the exceptions are in the ttypes.py module, while the client and server code is in the CourseService.py module.

Creating a Java Server

Creating a server in Thrift requires about the same amount of code as the other technologies examined in this article (REST and RMI). Check out the example code and judge for yourself. This example will use the TCompactProtocol with the FramedTransport and a non-blocking server. TFramedTransport is a requirement for using a non-blocking server, since the frames are used to determine when a given request is complete.

final TNonblockingServerSocket socket = new TNonblockingServerSocket(PORT);
final CourseService.Processor processor = new CourseService.Processor(
        new Handler());
final TServer server = new THsHaServer(processor, socket,
        new TFramedTransport.Factory(), new TCompactProtocol.Factory());
 
server.serve();
 
...
 
private static class Handler implements CourseService.Iface {
 
    @Override
    public List<String> getCourseInventory() throws
            TException {
        return db.getCourseList();
    }
 
    @Override
    public Course getCourse(String courseNumber) throws
            CourseNotFoundException, TException {
        final com.ociweb.jnb.thrift.db.Course dbCourse =
            db.getCourse(courseNumber);
        if (dbCourse != null) {
            return ConversionHelper.fromDbCourse(dbCourse);
        }
 
        return null;
    }
 
    @Override
    public void addCourse(Course course) throws
            UnacceptableCourseException, TException {
        com.ociweb.jnb.thrift.db.Course dbCourse =
            ConversionHelper.toDbCourse(course);
        db.addCourse(dbCourse);
    }
 
    @Override
    public void deleteCourse(String courseNumber) throws
            CourseNotFoundException, TException {
        if (db.getCourse(courseNumber) != null) {
            db.deleteCourse(courseNumber);
        }
    }
}

The first few lines before the ellipsis simply setup the server with the protocol, transport and server type we want to use. The handler class is where the implementation of the services is done. There is a fictitious database referenced as "db" that is used for the calls to distill the code to its relevant parts and enable re-use for the comparisons later in this article.

Creating a Java Client

Creating a Java client requires the same basic setup as the server, but does not require implementation of an interface.

//Setup the transport and protocol
final TSocket socket = new TSocket(HOST, PORT);
socket.setTimeout(SOCKET_TIMEOUT);
final TTransport transport = new TFramedTransport(socket);
final TProtocol protocol = new TCompactProtocol(transport);
final CourseService.Client client = new CourseService.Client(protocol);
 
//The transport must be opened before you can begin using
transport.open();
 
//All hooked up, start using the service
List<String> classInv = client.getCourseInventory();
System.out.println("Received " + classInv.size() + " class(es).");
 
client.deleteCourse("WINDOWS_301");
 
classInv = client.getCourseInventory();
System.out.println("Received " + classInv.size() + " class(es).");
 
transport.close();

The first few lines are the corollary to the setup of the server. The next several lines call the getCourseInventory and deleteCourse methods of the service. One thing to note is that while the server is using non-blocking IO, the client is using blocking IO. The equivalent non-blocking client socket was not fully implemented in the release build of Thrift that this example is built on. Each service operation actually calls send_ and recv_ method pairs internally. I tried calling these methods to see if I could get asynchronous behavior but had little luck. There is an "async" modifier that can be added to void return type methods, but the generated code looked no different with or without it. Finally, the generated receive methods don't gracefully return if no response was received when you do call them.

Creating a Python Client

The Python client is effectively the same as the Java client except for syntax.

#Setup the transport and protocol
socket = TSocket.TSocket("localhost", 8000)
socket._timeout = 1000
transport = TTransport.TFramedTransport(socket)
protocol_factory = TBinaryProtocol.TBinaryProtocolFactory()
protocol = protocol_factory.getProtocol(transport)
client = Client(protocol)
 
#The transport must be opened before you can begin using
transport.open()
 
classInv = client.getCourseInventory()
print "Received", len(classInv), "class(es)"
 
client.deleteCourse("WINDOWS_301")
 
classInv = client.getCourseInventory()
print "Received", len(classInv), "class(es)"

Note that the Python code uses the TBinaryProtocol and not TCompactProtocol. The version of Thrift used for this article does not support TCompactProtocol for Python.

Both clients produce the following output against a newly started server:

Received 18 class(es).
Received 17 class(es).

Running Thrift

For more details on running the examples, see the README included with the source. For a quick start, simply run the following from the java directory:

• ant start.thrift.server
• ant run.thrift.client

Comparing Thrift

In order to validate Thrift's value proposition, I decided to compare it with some other service technologies that are also fairly easy to use in practice. Since RESTful webservices seem to be popular of late, I have compared Thrift to REST. Although Protocol Buffers does not include a services infrastructure, it transports objects in a similar fashion to Thrift's TCompactProtocol, thus making it a useful comparison. Lastly, I included RMI, since it uses a binary transport and thus can serve as a "reference implementation" of sorts for Java binary object transport.

For the comparisons, I compared the file sizes and runtime performance of each service technology. For REST, I compared both XML-based and JSON-based REST. For Thrift, I chose the most efficient transport available for Java - TCompactProtocol.

Size Comparison

To compare the sizes, I wrote out essentially the same data for each. Each write includes one Course object with 5 Person objects, and one Phone object. The definitions for each of these types can be seen in the thrift file listed above. To capture the file sizes, I used the following techniques:

The chart and table below show the results. Sizes are in bytes. None of the sizes include TCP/IP overhead.

*Smaller is better.
** Excludes messaging overhead. Includes only transported objects.

Thrift has a clear advantage in the size of its payload particularly compared to RMI and XML-based REST. Protocol Buffers from Google is effectively the same given that the Protocol Buffers number excludes messaging overhead.

Runtime Performance

To compare the runtime performance of Thrift, I created the following scenario:

Test Scenario

• Query the list of Course numbers.
• Fetch the course for each course number.

This scenario is executed 10,000 times. The tests were run on the following systems:

The following table describes each test run:

*Since Protocol Buffers does not include a services infrastructure (unlike Thrift), I wrote my own server for this article.

The chart and table below summarize the results. All times are in seconds.

*Average Time excludes the first run in order to account for server warm-up. Smaller numbers are better.

The tests yielded some interesting observations. In terms of wall time Thrift clearly out-performed REST and RMI. In fact, TCompactProtocol took less than 20% of the time it took REST-XML to transmit the same data. The clear dominance of the binary protocols should not be too surprising, as binary data transmission is well-known to have higher performance than text-based protocols. RMI in fact significantly out-performed JSON-based REST in wall time, despite its significantly larger payload size (61.9% larger).

The CPU percentages yielded some interesting numbers. While the Thrift and Protocol Buffers servers had the highest server CPU percentages, the REST clients had the highest CPU percentages of the clients. For whatever reason Thrift and REST disproportionately place their CPU loads on their clients and servers. Protocol Buffers balanced its load most evenly between client and server, but then again this was a simple quick hand-rolled server that I wrote for this article. While I did not have time to analyze the cause of the CPU load, the Thrift and Protocol Buffers examples needed to do manual conversion of objects between what is transmitted and what is used. The RMI and REST implementations required no such object conversion. This extra bit of work may account for the additional CPU utilization on the Thrift and Protocol Buffers servers.

This test basically just covered throughput of each server. Another useful test would have been too test how each server handled multiple concurrent connections of shorter duration. There was insufficient time to complete a concurrent test for this article, however.

Given the poor performance of REST, there may certainly be higher performing servlet containers than Jetty that could be used as part of this test. Jetty was merely chosen because of its relative ease in implementation and ease in bundling for download of the sample code used in this article. Doing some quick searches, I found one performance comparison that showed Apache Tomcat to be faster than Jetty, and another that showed them at parity. Neither study showed anywhere near a performance difference to make up for the wall time performance of the binary protocols.

All of these technologies are roughly equivalent in the amount of coding complexity required to make them work. This excludes Protocol Buffers of course, as it contains no services infrastructure. It should also be noted that Thrift generates all the code you need for a client or server for each language it supports. Java was the server of choice in this article, but other languages could be used if they are better suited - one of the main reasons Thrift was developed in the first place. That being said, I found many of the implementations incomplete. As mentioned previously, the Python implementation for instance only had the TBinaryProtocol implemented.

Conclusion

Thrift is a powerful library for creating high-performance services that can be called from multiple languages. If your application has a need for multiple languages to communicate where speed is a concern and your clients and servers are co-located, Thrift may well be the choice for you. Thrift might also make a good choice for IPC on a single machine where speed and/or interoperability are a concern.

Thrift was designed for use where clients and servers are co-located, as in a data center. If you consider using Thrift in environments where client and server are not co-located, you should expect to encounter some challenges. In particular, the aforementioned issue with asynchronous calls, as well as lack of security are likely to pose challenges. While the security issue may be solved by a new transport, the issue with asynchronous calls will likely require work in the core areas of Thrift. Plus, since Thrift supports numerous language bindings, you will likely need to make changes for each language you are using.

If the composite service work-around will not work for you, the server-per-service limitation in Thrift may pose a problem in some deployment scenarios. For instance, if the Thrift services are on one side of a firewall and the client's are on the other, some data centers may have a problem with opening up too many ports.

References

• [1] Thrift Home
  http://incubator.apache.org/thrift
• [2] Thrift Wiki
  http://wiki.apache.org/thrift
• [3] Thrift Installation
  http://wiki.apache.org/thrift/ThriftInstallation
• [4] Thrift Installation (Win32)
  http://wiki.apache.org/thrift/ThriftInstallationWin32
• [5] Wikipedia Page on Variable-Length Quantity(VLQ)
  http://en.wikipedia.org/wiki/Variable_length_unsigned_integer
• [6] Protocol Buffers
  http://code.google.com/p/protobuf/
• [7[ RESTful Web Services
  http://java.sun.com/developer/technicalArticles/WebServices/restful/
• [8] Remote Method Invocation Home
  http://java.sun.com/javase/technologies/core/basic/rmi/index.jsp
• [9] Full source for this article
  Jun2009Source.zip