An Introduction to TAO's Asynchronous Message Handling

Middleware News Brief (MNB) features news and technical information about Open Source middleware technologies.

August 22, 2003 - By Rob Martin, Senior Software Engineer

Introduction

CORBA ORBs typically offer application developers a variety of mechanisms to control request processing. The following mechanisms are defined by formal CORBA specifications:

POA threading models, which control concurrent request processing
Real-Time CORBA's priority mechanisms, which provide platform-independent control of native thread priorities
Real-Time CORBA's threadpool mechanisms, which control the allocation of threads to requests at the POA level
Asynchronous Message Invocation (AMI), which allows a client to invoke a synchronous request in an asynchronous manner

Other mechanisms, such as an ORB's threading and concurrency strategies, are defined by ORB implementers.

TAO's Asynchronous Message Handling feature (AMH) offers application developers yet another tool to control request processing in certain situations, as described in this article.

Using AMH, an application can preserve a synchronous request's appearance while processing it in an asynchronous manner. Put simply, AMH allows a servant to delegate the processing of a request and transmission of a reply to a thread other than the one from which the servant was invoked.

Moreover, AMH employs mechanisms that are similar to those used by AMI, so application developers familiar with AMI should have little trouble employing AMH.

AMH is particularly attractive for optimizing middle-tier servers in three-tier architectures, especially when combined with AMI. A common scenario is a middle-tier server that services requests to process large blocks of data. Often, a bulk processing task can be reorganized into a series of smaller processing tasks that may execute concurrently. The results of these subordinate processing tasks are accumulated as they are received, and a reply is formulated and sent to the client as soon as all results are available.

Using AMH, a servant invoked to process a large data block can delegate the task to another thread. That thread decomposes the input data block into several, smaller data blocks and forwards each of the smaller blocks to a separate back-end server using AMI. The resulting requests are processed concurrently by the various back end servers. The results from the back end servers, returned via AMI's callback mechanism, are accumulated as they are received. As soon as the sub-tasks are complete and all results have been accumulated, a reply is formulated and sent to the client using AMH's reply mechanism.

AMH may also be useful in other circumstances to:

De-couple threads allocated to receiving client requests from threads used to process and reply to such requests
Consider a scenario where the overhead required to process a request is substantially greater than the overhead required to receive and dispatch a request. Using AMH, a greater number of threads can be allocated to processing requests than are allocated to the ORB for receiving and dispatching requests.

A CORBA-based service that supports a large client base may employ AMH to prevent excessive consumption of network stack resources by allocating a greater number of threads to the ORB for servicing clients' connections than are allocated to request processing.
Bridge CORBA request/replies to another service using a different middleware
A servant in a middleware bridge delegates request processing to another thread passing the incoming parameters. The delegated thread then formulates another request (in the appropriate format), invokes the target service, and waits for a reply. When the reply is received, the delegated thread uses AMH mechanisms to send a CORBA reply to the client. Meanwhile, the thread originally dispatched to handle the CORBA request is available to perform other ORB-related tasks.
Provide remote access to a single-threaded service
Concurrent CORBA requests can be delegated via a queuing mechanism to a single-threaded active object. The queue of requests is processed serially by the active object's thread, and replies are returned to the clients using AMH's reply mechanism.

The remainder of this article compares synchronous request processing with AMH, explains how IDL-defined interfaces are modified for use via AMH, and presents a sample application.

Additional information regarding AMH or AMI is available from the following sources:

The strategy for and design of AMH is set forth in "Designing an Efficient and Scalable Server-side Asynchrony Model for CORBA" (AMH.pdf) by Darrell Brunsch, Carlos O'Ryan, and Douglas Schmidt.
Additional design details and performance information is contained in Design and Performance of Asynchronous Method Handling for CORBA by Mayur Deshpande, Douglas C. Schmidt, Carlos O'Ryan, and Darrell Brunsch.
AMI is defined by the current CORBA specification (formal/2002-12-06, Chapter 22, Section II). Additional information regarding AMI is also contained in the TAO Developer's Guide, available from OCI.

AMH is included in TAO 1.3a, OCI's commercially supported release, as well as the latest TAO releases from the Distributed Object Computing (DOC) Group.

Additional information regarding Real-Time CORBA's priority and threadpool mechanisms is available in Real-Time CORBA Priority and Threadpool Mechanisms.

Request Processing Aspects of CORBA and TAO contains a discussion of the POA's threading models.

Synchronous Request Processing

Synchronous request processing, also called Synchronous Message Invocation (SMI), is the basic processing model for twoway CORBA invocations. From the client's perspective, a twoway invocation is a blocking call; control does not return to the caller until the invocation has completed.

The following diagram summarizes synchronous request processing, focusing on the most salient points for this discussion of AMH; many details are omitted in the interest of brevity.

Figure 1. Synchronous Message Processing

A client makes a CORBA request, and the proxy forwards the request to the client-side ORB. (The proxy, also called the stub, is not shown in this diagram.)
The client-side ORB sends a request message to the server-side ORB, i.e., the ORB hosting the target CORBA object.
The server-side ORB dispatches the request to a thread, which in turn makes an upcall. The request is dispatched to the appropriate POA, the POA invokes the skeleton, and the skeleton invokes the servant's method. (The thread, POA, and skeleton are not shown in this diagram.)
The servant processes the request and returns.
The return triggers a reply to the client-side ORB.
The server-side ORB sends a reply message.
The client-side ORB receives the reply message and returns the results, if any, to the client.

Once the server-side ORB dispatches the request to a thread, that thread is bound to the request until processing is complete and a reply is sent to the client. Therefore, the thread is effectively unavailable for the request's duration.

In certain scenarios, some ORBs can 'borrow' a thread that is already bound to a request for another task. This can happen with TAO, for example, when a thread is waiting for a reply to an outbound CORBA request made by a servant, and it is dispatched to handle an inbound request. (This is commonly referred to as a nested upcall.) When the ORB borrows a thread that is processing a request, completion of that request is usually postponed until the task for which the ORB borrowed the thread (e.g., handling a nested upcall) is complete.

Asynchronous Message Handling

AMH preserves a synchronous request's semantics while allowing the application to process it asynchronously.

The following diagram depicts a synchronous request processed using AMH. Additional details have been added as needed to explain AMH, but many other details are still left out.

Figure 2. Asynchronous Message Handling

A client issues a CORBA request, which is forwarded to the client-side ORB.
The client-side ORB transmits a request message to the server-side ORB.
The server-side ORB dispatches the request to a thread, which in turn makes an upcall. The request is dispatched to a POA (not shown), and the POA invokes the skeleton.
Prior to invoking the target object, the skeleton creates a response handler object.
- A response handler contains the context information required to send a reply to the client. This object will be invoked when the request is complete to send a reply. Its interface is implied by the target object's interface, which is similar to the way in which an AMI reply handler's interface is implied. Unlike AMI, however, its implementation is also generated by the IDL compiler.
The skeleton invokes the servant, passing a reference to the response handler.
The servant invokes an asynchronous service.
- The asynchronous service can be co-located with the servant, if for example, the servant queues a request for processing by an active object, or it can be remote, invoked via CORBA or perhaps another middleware. The essential requirement is that processing the request be delegated from the current thread to another thread or process. At a minimum, a reference to the response handler must be passed to the other service. Other information, including the inbound request's parameters, may also be passed to the target service.
The CORBA servant has completed its part in processing the request and returns, which ultimately returns the thread to the ORB.
- The thread that was dispatched to process the request is now available for other ORB tasks; meanwhile, the request is processed on a separate thread.
The request is processed by the target service.
Upon completing the request, the target service invokes the response handler to send a reply to the client.
The response handler sends a reply to the client via the server-side ORB.
The client-side ORB receives the reply and returns the results, if any, to the client.

Middle Tier Optimization: AMH with AMI

AMH allows an application to process synchronous requests in an asynchronous manner; AMI allows an application to make synchronous requests in a asynchronous manner. Combined, these features offer a means for optimizing the performance of middle-tier servers by limiting the time period that threads must be dedicated to processing requests from front-end clients and replies from back-end servers. Threads that service client requests are bound to a request only for the time required to forward the request to a back-end server using AMI. Threads dispatched to process back-end replies are bound to a reply only for the time needed to send a reply to the client.

The following diagram depicts a request processed using a combination of AMH and AMI. As before, some details are omitted in the interest of brevity.

Figure 3. Middle-Tier Optimization with AMD and AMI

Figure 3. Middle-tier Optimization with AMH and AMI

The client makes a CORBA request, and the proxy forwards the request to the client-side ORB.
The client-side ORB sends a request message to the server-side ORB.
The server-side ORB dispatches the request to a thread, which makes an upcall. The request is dispatched to the appropriate POA and then to the skeleton.
The skeleton creates a response handler.
The skeleton invokes the servant, passing a reference to the response handler.
The servant creates an instance of an AMI reply handler. The reply handler is invoked by the ORB when a reply is received from the remote server. A reference to the response handler is passed to the reply handler, perhaps upon construction or via a separate initialization step depending upon the design. (This step is not shown here.)
The servant makes a CORBA request to the remote server (the back-end server) using AMI. A reference to the reply handler is passed to the server-side ORB along with the other parameter (only the reply handler is shown here).
The server-side ORB sends a request message to the remote server.
- At this point, control returns to the server-side ORB thus releasing the thread and making it available for other tasks. Consequently, the middle-tier server no longer has any threads bound to the client's request, which will remain the case until a reply is received from the remote server.
The remote server processes the client's request.
The remote server sends a reply to the server-side ORB.
The server-side ORB dispatches the reply handler (details not shown here).
The reply handler invokes the response handler to send a reply to the client.
The response handler invokes the server side ORB to send the reply.
The server-side ORB sends a reply message to the client-side ORB.
- At this point, control returns thus releasing the thread dispatched to process the remote server's reply.
The client-side ORB receives the reply message and returns the results, if any, to the client.

Interfaces and AMH

In addition to preserving the semantics of a synchronous request, AMH also preserves a CORBA object's interface as it is presented to clients. However, the semantics of AMH require a different interface to be realized by the servant, as well as a new interface for the response handler, interfaces which are implied by the original interface.

The IDL compiler, when directed to compile IDL for use with AMH, generates stubs and skeletons from the original IDL, as well as the implied IDL. Moreover, the response handler's implementation is also generated from the implied IDL.

To propagate an exception to the client, an exception holder object and additional methods are also generated from the original interface. For each method appearing in the original interface:

A corresponding method appears in the response handler to obtain an exception holder.
A corresponding method appears in the exception holder to propagate an exception to the client.

When an exception occurs during the processing of a request, the response handler's corresponding method is invoked to obtain an exception holder, and the exception holder's corresponding method is invoked to transmit the exception.

The implied IDL used to produce the skeleton, which declares methods to be realized by the servant, is derived from the original interface as follows:

The string 'AMH_' is prepended to the original interface's name.
- e.g., Interface becomes AMH_Interface
All methods appearing in the original interface also appear in the implied interface.
Method names are preserved, but each method returns void
- e.g., boolean a_method(...) becomes void a_method(...)
Each method is passed a reference to a response handler as its first parameter.
- e.g., short a_method(...) becomes void a_method(in AMH_InterfaceResponseHandler handler, ...)
in mode parameters are unchanged.
- e.g., string a_method(in short arg) becomes void a_method(in AMH_InterfaceResponseHandler handler, in short arg)
inout mode parameters become in mode parameters.
- e.g., long a_method(inout short arg) becomes void a_method(in AMH_InterfaceResponseHandler handler, in short arg)
out parameters are omitted.
- e.g., boolean a_method(out string reply,...) becomes void a_method(in AMH_InterfaceResponseHandler handler,...)

The response handler's interface is derived from the original IDL as follows:

The string 'AMH_' is prepended and the string 'ResponseHandler is appended to the original interface's name.
- e.g., Interface becomes AMH_InterfaceResponseHandler
All methods appearing in the original interface also appear in the response handler's interface.
Method names are preserved, but each method returns void.
- e.g., boolean a_method(...) becomes void a_method(...)
A method's return value becomes an in mode parameter as the first argument to the corresponding response handler method.
- e.g., string a_method(in short arg,...) becomes void a_method(in string return_value,...)
in mode parameters do not appear in the response handler's methods.
- e.g., string a_method(in short arg,...) becomes void a_method(...)
inout mode parameters become in mode parameters.
- e.g., long a_method(inout short arg) becomes void a_method(in short arg)
out mode parameters become in mode parameters,
- e.g., boolean a_method(out string reply) becomes void a_method(in string reply)
For each method in the original interface, a corresponding method appears in the response handler to initialize an exception holder object.
- e.g., Given the method short a_method(...) raises (Exception) appears in the original interface,
The method a_method_except(in AMH_InterfaceExceptionHolder) is implied in the response handler's interface, and the corresponding method is implied in the exception holder.
- e.g., raise_a_method() raises (Exception)

Note that the exception holder object's name is formed by pre-pending the string 'AMH_' and appending the string 'ExceptionHolder' to the original interface's name, and that each of the exception holder's methods' exception specifications have the same signature as the corresponding method's exception specification appearing in the original interface.

Compiling Interfaces for AMH

To compile an interface for use with AMH, pass "-GH" to TAO's IDL compiler. Compile and link applications as usual.

Example Application

The simple prototype application shown in the following diagram models a three tier architecture. A front-end client (Client) invokes a middle-tier service (Middle_Tier_Server), and the middle-tier service delegates each request to a back-end server (Simple_Server).

Figure 4. Example Application

The middle-tier server presents the following interface:

interface MessengerService
{
  boolean send_message( in string user_name,
                        in string subject,
                        inout string message );
};

The back-end server presents a similar interface:

interface Messenger
{
  boolean send_message( in string user_name,
                        in string subject,
                        inout string message );
};

The middle-tier process employs AMH and AMI; however, this has little effect on the client and back-end server applications. Clients make what appear to be oridinary synchronous requests, and back end servers process requests synchronously. Therefore, the remainder of this article will address implementation of the middle tier server.

MessengerService requests are processed by a proxy object, called MessengerProxy_i, which is deployed in the mid-tier server. This object is organized as shown in the following diagram:

Figure 5. Sample Application Service Proxy

Requests from clients are handled via AMH, so the MessengerProxy_i realizes the implied interface AMH_MessengerService.
To invoke the back-end service, MessengerProxy_i keeps a reference to a Messenger CORBA object.
An AMI reply handler, called MessengerReplyHandler_i, which realizes the implied AMI_MessengerHandler interface, is used to handle callbacks that are invoked when the back end messenger returns.
The callback object is created once and re-used for each request, so MessengerProxy_i keeps a reference to the AMH_MessengerHandler CORBA object, as well as the servant.

To send replies to front end clients, MessengerReplyHandler_i needs a reference to the implied AMH response handler, AMH_MessengerServiceResponseHandler.

The definition for MessengerProxy_i follows:

class  MessengerProxy_i : public virtual POA_AMH_MessengerService
{
  public:
    MessengerProxy_i(
      Messenger_ptr messenger,
      MessengerReplyHandler_i *replyHandler,
      AMI_MessengerHandler_ptr replyHandlerObj
    );
 
    virtual ~MessengerProxy_i (void);
 
    virtual void send_message (
      AMH_MessengerServiceResponseHandler_ptr handler,
      const char * user_name,
      const char * subject,
      const char * message
      ) throw(CORBA::SystemException);
 
  private:
    Messenger_var            _messenger;
    MessengerReplyHandler_i *_replyHandler;
    AMI_MessengerHandler_var _replyHandlerObj;
};

Note: According to standard, the string 'POA' is prepended to an interface's name to produce the corresponding skeleton's class name. The implied interface for this service when implemented via AMH is AMH_MessengerService; thus the skeleton's class name is POA_AMH_MessengerService.

When the proxy is created, it is provided with:

A reference to the back-end messenger service
A pointer to the reply handler servant
A CORBA reference to the reply handler

The proxy's implementation of send_message demonstrates how a request is processed, and how these member variables are used:

void MessengerProxy_i::send_message(
  AMH_MessengerServiceResponseHandler_ptr responseHandler,
  const char * user_name,
  const char * subject,
  const char * message
) throw(CORBA::SystemException)
{
 
  _replyHandler->responseHandler(responseHandler);
 
  _messenger->sendc_send_message(
    _replyHandlerObj.in(),
    user_name,
    subject,
    message);
};

First, the reply handler is provided with a reference (via a native pointer) to the response handler created for this invocation; when the reply handler is invoked, it will use this reference to return a reply to the client.

Next the back-end service is invoked asynchronously; accordingly, a reference to the reply handler is passed as the first parameter to the sendc_ method, followed by the parameters defined by the service's interface.

A portion of the reply handler's definition follows:

class MessengerReplyHandler_i : public POA_AMI_MessengerHandler
{
  public:
 
    // some methods omitted
 
    void responseHandler(
      AMH_MessengerServiceResponseHandler_ptr responseHandler
    );
 
    void send_message(CORBA::Boolean ami_return_val,
      const char *message
    );
 
  private:
    AMH_MessengerServiceResponseHandler_var _responseHandler;
};

Note: The implied interface for an AMI reply handler for the Messenger interface is AMI_MessengerHandler; thus the skeleton's class name is POA_AMI_MessengerHandler.

The implementation of responseHandler is:

void MessengerReplyHandler_i::responseHandler(
  AMH_MessengerServiceResponseHandler_ptr responseHandler
)
{
  _responseHandler = 
    AMH_MessengerServiceResponseHandler::_duplicate(responseHandler);
}

Duplicating the response handler reference is crucial; recall that it is passed to the AMH servant as an in mode parameter and so must be duplicated to retain the reference over a context switch.

The method for sending a reply to the client is straight forward:

void MessengerReplyHandler_i::send_message(
  CORBA::Boolean ami_return_val,
  const char *message)
{
  _responseHandler->send_message(ami_return_val, message);
}

It simply forwards its parameters to the corresponding response handler's method, which in turn sends a reply to the client.

The relevant portion of the mid-tier server's main follows (some typical code, such as checks for nil object references omitted):

  // Get a reference to the back-end server
 
  obj = orb->string_to_object( "file://simple_server.ior" );
  Messenger_var messenger = Messenger::_narrow( obj.in() );
 
  // Create the reply handler
  MessengerReplyHandler_i *replyHandler = new MessengerReplyHandler_i;
 
  // Activate reply handler and obtain object reference
  PortableServer::ServantBase_var tmpServer = replyHandler;
 
  PortableServer::ObjectId_var oid =
    poa->activate_object (tmpServer.in ());
 
  obj = poa->id_to_reference (oid.in ());
 
  // narrow reference to appropriate type
  AMI_MessengerHandler_var replyHandlerObj =
    AMI_MessengerHandler::_narrow(obj.in());
 
  // Create the messenger proxy servant,
  MessengerProxy_i *messengerProxy = 
    new MessengerProxy_i(messenger.in(), replyHandler,
      replyHandlerObj.in());
 
  tmpServer = messengerProxy;
 
  oid = poa->activate_object (tmpServer.in ());
 
  CORBA::Object_var server_obj =
    poa->id_to_reference (oid.in ());

First, a reference to the back-end server is obtained, in this case from a file.
Next, the reply handler is created, activated with the root POA, and an object reference obtained.
Then, the messenger proxy is created and initialized with the reference to the back end service, a native pointer to the reply handler servant, and an object reference to the reply handler.
Finally, the proxy is activated with the root POA and an object reference obtained to be exported to clients.

The code for the client and back-end servers is typical of such processes. The client is unaware that its requests are processed asynchronously, so no special setup is required. Similarly, the back end server is unaware that it is invoked asynchronously, and no special setup is required there either.

SUMMARY

This article presented TAO's Asynchronous Message Handling (AMH) feature, another important tool for controlling request processing in CORBA servers. AMH makes it possible to tailor an application to the specific circumstances in which it will be deployed and, when combined with other capabilities such as AMI, can afford significant performance improvements.

Acknowledgments

This article was written by OCI Principal Software Engineer, Rob Martin, and OCI Software Engineer, Stuart Jones.