An Introduction to TAO's Asynchronous Message Handling

An Introduction to TAO's Asynchronous Message Handling

by Rob Martin, Prinicpal Engineer, and Stuart Jones, Software Engineer

August 2003

Introduction

CORBA ORBs typically offer application developers a variety of mechanisms to control request processing. Mechanisms such as POA threading models, which control concurrent request processing, Real-Time CORBA's priority mechanisms, which provides platform-independent control of native thread priorities, Real-Time CORBA's threadpool mechanisms, which controls allocation of threads to requests at the POA level, and Asynchronous Message Invocation (AMI), which allows a client to invoke a synchronous request in an asynchronous manner, are defined by formal CORBA specifications. Other mechanisms, such as an ORB's threading and concurrency strategies are defined by ORB implementers.

TAO's Asynchronous Message Handling feature (AMH) offers application developers yet another tool to control request processing in certain situations as described in this article. Using AMH, an application can preserve a synchronous request's appearance while processing it in an asynchronous manner. Put simply, AMH allows a servant to delegate the processing of a request and transmission of a reply to a thread other than the one from which the servant was invoked. Moreover, AMH employs mechanisms that are similar to those used by AMI, so application developers familiar with AMI should have little trouble employing AMH.

AMH is particularly attractive for optimizing middle-tier servers in three-tier architectures, especially when combined with AMI. A common scenario is a middle-tier server that services requests to process large blocks of data. Often, a bulk processing task can be re-organized into a series of smaller processing tasks that may execute concurrently. The results of these subordinate processing tasks are accumulated as they are received and a reply is formulated and sent to the client as soon as all results are available. Using AMH, a servant invoked to process a large data block can delegate the task to another thread. That thread decomposes the input data block into several, smaller data blocks and forwards each of the smaller blocks to a separate back-end server using AMI. The resulting requests are processed concurrently by the various back end servers. The results from the back end servers, returned via AMI's callback mechanism, are accumulated as they are received. As soon as the sub-tasks are complete and all results have been accumulated, a reply is formulated and sent to the client using AMH's reply mechanism.

AMH may also be useful in other circumstances to:

The remainder of this article compares synchronous request processing with AMH, explains how IDL-defined interfaces are modified for use via AMH, and presents a sample application. Additional information regarding AMH or AMI is available from the following sources:

AMH is included in TAO 1.3a, OCI's commercially supported release, as well as the latest TAO releases from the Distributed Object Computing (DOC) Group.

Additional information regarding Real-Time CORBA's priority and threadpool mechanisms is available in Real-Time CORBA Priority and Threadpool MechanismsRequest Processing Aspects of CORBA and TAO contains a discussion of the POA's threading models.

Synchronous Request Processing

Synchronous request processing, also called Synchronous Message Invocation (SMI), is the basic processing model for twoway CORBA invocations. From the client's perspective, a twoway invocation is a blocking call--control does not return to the caller until the invocation has completed. The following diagram summarizes synchronous request processing focusing on the most salient points for this discussion of AMH; many details are omitted in the interest of brevity.

Synchronous Message Processing

Synchronous Message Processing
 

  1. A client makes a CORBA request and the proxy forwards the request to the client-side ORB. (The proxy, also called the stub, is not shown in this diagram.)
  2. The client-side ORB sends a request message to the server-side ORB, i.e. the ORB hosting the target CORBA object.
  3. The server-side ORB dispatches the request to a thread, which in turn makes an upcall. The request is dispatched to the appropriate POA, the POA invokes the skeleton, and the skeleton invokes the servant's method. (The thread, POA, and skeleton are not shown in this diagram.)
  4. The servant processes the request and returns.
  5. The return triggers a reply to the client-side ORB.
  6. The server-side ORB sends a reply message.
  7. The client-side ORB receives the reply message and returns the results, if any, to the client.

Once the server-side ORB dispatches the request to a thread, that thread is bound to the request until processing is complete and a reply is sent to the client. Therefore, the thread is effectively unavailable for the request's duration. Some ORBs can, in certain scenarios, 'borrow' a thread that is already bound to a request for another task. This can happen with TAO, for example, when a thread is waiting for a reply to an outbound CORBA request made by a servant and it is dispatched to handle an inbound request. (This is commonly referred to as a nested upcall.) When the ORB borrows a thread that is processing a request, completion of that request is usually postponed until the task for which the ORB borrowed the thread, e.g., handling a nested upcall, is complete.

Asynchronous Message Handling

AMH preserves a synchronous request's semantics while allowing the application to process it asynchronously. The following diagram depicts a synchronous request processed using AMH. Additional details have been added as needed to explain AMH, but many other details are still left out.

Asynchronous Message Handling

Asynchronous Message Handling
 

  1. A client issues a CORBA request, which is forwarded to the client-side ORB.
  2. The client-side ORB transmits a request message to the server-side ORB.
  3. The server-side ORB dispatches the request to a thread, which in turn makes an upcall. The request is dispatched to a POA (not shown) and the POA invokes the skeleton.
  4. Prior to invoking the target object, the skeleton creates a response handler object.
    • A response handler contains the context information required to send a reply to the client. This object will be invoked when the request is complete to send a reply. Its interface is implied by the target object's interface, which is similar to the way in which an AMI reply handler's interfaces is implied. Unlike AMI, however, its implementation is also generated by the IDL compiler.
  5. The skeleton invokes the servant passing a reference to the response handler.
  6. The servant invokes an asynchronous service.
    • The asynchronous service can be co-located with the servant, if for example the servant queues a request for processing by an active object, or it can be remote, invoked via CORBA or perhaps another middleware. The essential requirement is that processing the request be delegated from the current thread to another thread or process. At a minimum, a reference to the response handler must be passed to the other service. Other information, including the inbound request's parameters may also be passed to the target service.
  7. The CORBA servant has completed its part in processing the request and returns, which ultimately returns the thread to the ORB.
    • The thread that was dispatched to process the request is now available for other ORB tasks; meanwhile, the request is processed on a separate thread.
  8. The request is processed by the target service.
  9. Upon completing the request, the target service invokes the response handler to send a reply to the client.
  10. The response handler sends a reply to the client via the server-side ORB.
  11. The client-side ORB receives the reply and returns the results, if any, to the client.

Middle Tier Optimization: AMH with AMI

AMH allows an application to process synchronous requests in an asynchronous manner; AMI allows an application to make synchronous requests in a asynchronous manner. Combined, these features offer a means for optimizing the performance of middle-tier servers by limiting the time period that threads must be dedicated to processing requests from the front-end clients and replies from the back-end servers. Threads that service client requests are bound to a request only for the time required to forward the request to a back-end server using AMI. Threads dispatched to process back-end replies are bound to a reply only for the time needed to send a reply to the client.

The following diagram depicts a request processed using a combination of AMH and AMI. As before, some details are omitted in the interest of brevity.

Middle-Tier Optimization with AMD and AMI

Middle-tier Optimization with AMH and AMI
 

  1. The client makes a CORBA request and the proxy forwards the request to the client-side ORB.
  2. The client-side ORB sends a request message to the server-side ORB.
  3. The server-side ORB dispatches the request to a thread, which makes an upcall. The request is dispatched to the appropriate POA, and then to the skeleton.
  4. The skeleton creates a response handler.
  5. The skeleton invokes the servant passing a reference to the response handler.
  6. The servant creates an instance of an AMI reply handler. The reply handler is invoked by the ORB when a reply is received from the remote server. A reference to the response handler is passed to the reply handler, perhaps upon construction or via a separate initialization step depending upon the design. (This step is not shown here.)
  7. The servant makes a CORBA request to the remote server (the back-end server) using AMI. A reference to the reply handler is passed to the server-side ORB along with the other parameter (only the reply handler is shown here).
  8. The server-side ORB sends a request message to the remote server.
    • At this point, control returns to the server-side ORB thus releasing the thread and making it available for other tasks. Consequently, the middle-tier server no longer has any threads bound to the client's request, which will remain the case until a reply is received from the remote server.
  9. The remote server processes the client's request.
  10. The remote server sends a reply to the server-side ORB.
  11. The server-side ORB dispatches the reply handler (details not shown here).
  12. The reply handler invokes the response handler to send a reply to the client.
  13. The response handler invokes the server side ORB to send the reply.
  14. The server-side ORB sends a reply message to the client-side ORB.
    • At this point, control returns thus releasing the thread dispatched to process the remote server's reply.
  15. The client-side ORB receives the reply message and returns the results, if any, to the client.

Interfaces and AMH

In addition to preserving the semantics of a synchronous request, AMH also preserves a CORBA object's interface as it is presented to clients. However, the semantics of AMH require a different interface to be realized by the servant as well as a new interface for the response handler, interfaces which are implied by the original interface. The IDL compiler, when directed to compile IDL for use with AMH, generates stubs and skeletons from the original IDL as well as the implied IDL. Moreover, the response handler's implementation is also generated from the implied IDL.

To propagate an exception to the client, an exception holder object and additional methods are also generated from the original interface. For each method appearing in the original interface:

When an exception occurs during the processing of a request, the response handler's corresponding method is invoked to obtain an exception holder, and the exception holder's corresponding method is invoked to transmit the exception.

The implied IDL used to produce the skeleton, which declares methods to be realized by the servant, is derived from the original interface as follows:

The response handler's interface is derived from the original IDL as follows:

Note that the exception holder object's name is formed by pre-pending the string 'AMH_' and appending the string 'ExceptionHolder' to the original interface's name, and that each of the exception holder's methods' exception specifications have the same signature as the corresponding method's exception specification appearing in the original interface.

Compiling Interfaces for AMH

To compile an interface for use with AMH, pass "-GH" to TAO's IDL compiler. Compile and link applications as usual.

Example Application

The simple prototype application shown in the following diagram models a three tier architecture. A front-end client (Client) invokes a middle-tier service (Middle_Tier_Server), and the middle-tier service delegates each request to a back-end server (Simple_Server).

Example Application

Example Application

 

The middle-tier server presents the following interface:

interface MessengerService
{
  boolean send_message( in string user_name,
                        in string subject,
                        inout string message );
};


The back-end server presents a similar interface:

interface Messenger
{
  boolean send_message( in string user_name,
                        in string subject,
                        inout string message );
};

The middle-tier process employs AMH and AMI; however, this has little effect on the client and back-end server applications. Clients make what appear to be oridinary synchronous requests, and back end servers process requests synchronously. Therefore, the remainder of this article will address implementation of the middle tier server.

MessengerService requests are processed by a proxy object, called MessengerProxy_i, which is deployed in the mid-tier server. This object is organized as shown in the following diagram:

Sample Application Service Proxy

Sample Application Service Proxy
 

Requests from clients are handled via AMH, so the MessengerProxy_i realizes the implied interfaceAMH_MessengerService. To invoke the back-end service, MessengerProxy_i keeps a reference to a MessengerCORBA object. An AMI reply handler, called MessengerReplyHandler_i, which realizes the implied AMI_MessengerHandler interface, is used to handle callbacks that are invoked when the back end messenger returns. The callback object is created once and re-used for each request, so MessengerProxy_i keeps a reference to the AMH_MessengerHandler CORBA object as well as the servant. To send replies to front end clients, MessengerReplyHandler_i needs a reference to the implied AMH response handler,AMH_MessengerServiceResponseHandler.

The definition for MessengerProxy_i follows:

  1. class MessengerProxy_i : public virtual POA_AMH_MessengerService
  2. {
  3. public:
  4. MessengerProxy_i(
  5. Messenger_ptr messenger,
  6. MessengerReplyHandler_i *replyHandler,
  7. AMI_MessengerHandler_ptr replyHandlerObj
  8. );
  9.  
  10. virtual ~MessengerProxy_i (void);
  11.  
  12. virtual void send_message (
  13. AMH_MessengerServiceResponseHandler_ptr handler,
  14. const char * user_name,
  15. const char * subject,
  16. const char * message
  17. ) throw(CORBA::SystemException);
  18.  
  19. private:
  20. Messenger_var _messenger;
  21. MessengerReplyHandler_i *_replyHandler;
  22. AMI_MessengerHandler_var _replyHandlerObj;
  23. };

Note: according to standard, the string 'POA' is prepended to an interface's name to produce the corresponding skeleton's class name. The implied interface for this service when implemented via AMH is AMH_MessengerService; thus the skeleton's class name is POA_AMH_MessengerService.

When the proxy is created, it is provided with a reference to the back-end messenger service, a pointer to the reply handler servant, and a CORBA reference to the reply handler as well. The proxy's implementation of send_message demonstrates how a request is processed, and how these member variables are used:

  1. void MessengerProxy_i::send_message(
  2. AMH_MessengerServiceResponseHandler_ptr responseHandler,
  3. const char * user_name,
  4. const char * subject,
  5. const char * message
  6. ) throw(CORBA::SystemException)
  7. {
  8.  
  9. _replyHandler->responseHandler(responseHandler);
  10.  
  11. _messenger->sendc_send_message(
  12. _replyHandlerObj.in(),
  13. user_name,
  14. subject,
  15. message);
  16. };

First, the reply handler is provided with a reference (via a native pointer) to the response handler created for this invocation; when the reply handler is invoked, it will use this reference to return a reply to the client. Next the back-end service is invoked asynchronously; accordingly, a reference to the reply handler is passed as the first parameter to thesendc_ method followed by the parameters defined by the service's interface.

A portion of the reply handler's definition follows:

  1. class MessengerReplyHandler_i : public POA_AMI_MessengerHandler
  2. {
  3. public:
  4.  
  5. // some methods omitted
  6.  
  7. void responseHandler(
  8. AMH_MessengerServiceResponseHandler_ptr responseHandler
  9. );
  10.  
  11. void send_message(CORBA::Boolean ami_return_val,
  12. const char *message
  13. );
  14.  
  15. private:
  16. AMH_MessengerServiceResponseHandler_var _responseHandler;
  17. };

Note: the implied interface for an AMI reply handler for the Messenger interface is AMI_MessengerHandler; thus the skeleton's class name is POA_AMI_MessengerHandler.

The implementation of responseHandler is:

  1. void MessengerReplyHandler_i::responseHandler(
  2. AMH_MessengerServiceResponseHandler_ptr responseHandler
  3. )
  4. {
  5. _responseHandler =
  6. AMH_MessengerServiceResponseHandler::_duplicate(responseHandler);
  7. }

Duplicating the response handler reference is crucial; recall that it is passed to the AMH servant as an in mode parameter and so must be duplicated to retain the reference over a context switch.

The method for sending a reply to the client is straight forward:

  1. void MessengerReplyHandler_i::send_message(
  2. CORBA::Boolean ami_return_val,
  3. const char *message)
  4. {
  5. _responseHandler->send_message(ami_return_val, message);
  6. }

It simply forwards its parameters to the corresponding response handler's method, which in turn sends a reply to the client.

The relevant portion of the mid-tier server's main follows (some typical code, such as checks for nil object references omitted):

  1. // Get a reference to the back-end server
  2.  
  3. obj = orb->string_to_object( "file://simple_server.ior" );
  4. Messenger_var messenger = Messenger::_narrow( obj.in() );
  5.  
  6. // Create the reply handler
  7. MessengerReplyHandler_i *replyHandler = new MessengerReplyHandler_i;
  8.  
  9. // Activate reply handler and obtain object reference
  10. PortableServer::ServantBase_var tmpServer = replyHandler;
  11.  
  12. PortableServer::ObjectId_var oid =
  13. poa->activate_object (tmpServer.in ());
  14.  
  15. obj = poa->id_to_reference (oid.in ());
  16.  
  17. // narrow reference to appropriate type
  18. AMI_MessengerHandler_var replyHandlerObj =
  19. AMI_MessengerHandler::_narrow(obj.in());
  20.  
  21. // Create the messenger proxy servant,
  22. MessengerProxy_i *messengerProxy =
  23. new MessengerProxy_i(messenger.in(), replyHandler,
  24. replyHandlerObj.in());
  25.  
  26. tmpServer = messengerProxy;
  27.  
  28. oid = poa->activate_object (tmpServer.in ());
  29.  
  30. CORBA::Object_var server_obj =
  31. poa->id_to_reference (oid.in ());

First, a reference to the back-end server is obtained, in this case from a file. Next, the reply handler is created, activated with the root POA, and an object reference obtained. Then, the messenger proxy is created and initialized with the reference to the back end service, a native pointer to the reply handler servant, and an object reference to the reply handler. Finally, the proxy is activated with the root POA and an object reference obtained to be exported to clients.

The code for the client and back-end servers typical of such processes. The client is unaware that its requests are processed asynchronously, so no special setup is required. Similarly, the back end server is unaware that it is invoked asynchronously, and no special setup is required there either.

Summary

This article presented TAO's Asynchronous Message Handling (AMH) feature, another important tool for controlling request processing in CORBA servers. AMH makes it possible to tailor an application to the specific circumstances in which it will be deployed and, when combined with other capabilities such as AMI, can afford significant performance improvements.

The Middleware News Brief is a periodic newsletter. The purpose and intent of this publication is to advance and inform on features, news, and technical information about Open Source, middleware technologies (e.g., TAO, OpenDDS, JacORB), including case studies and other technical content.

© Copyright Object Computing, Inc. 1993, 2016. All rights reserved

Subscribe

secret