Request-Processing Aspects of CORBA and TAO

Middleware News Brief (MNB) features news and technical information about Open Source middleware technologies.

February 20, 2003 - By Rob Martin, Senior Software Engineer

Introduction

CORBA ORBs are used in software systems that span a variety of problem domains, including finance and banking, telecommunications network provisioning and maintenance, media collection and dissemination, aerospace, and defense [http://www.corba.org/success.htm]. Consequently, CORBA ORBs [formal/02-12-06] are subject to wide-ranging performance requirements used in sequential as well as concurrent processing scenarios and deployed on many different platforms. To be successful across such a broad range of applications, an ORB must provide some means of tailoring its behavior to a software application's specific requirements. Controlling how CORBA requests are processed is one way in which an ORB can be aligned to meet an application's needs.

Processing a CORBA request can be generalized as a two-step process:

The ORB dispatches the request to a POA.
To dispatch a request to a POA, an ORB selects a thread to process the request, identifies the target POA, and invokes the POA to process the request.
The POA dispatches the request to an application object.
To dispatch a request to an application object, a POA applies a threading model to enforce constraints on concurrent processing and a request processing policy to identify and invoke the target object.

The CORBA specification defines:

The POA's threading models
The policy used to specify a POA's applicable threading model
The ORB's thread-related interfaces

The specification does not, however, define the strategies an ORB may use when assigning requests to threads or the mechanisms for choosing a strategy or combination of strategies that fulfill a specific application's requirements.

All in all, the specification provides only a narrow treatment of these issues, leaving much to the ORB implementer's discretion.

Modern, multi-threaded ORBs typically offer a variety of strategies for mapping requests to threads. Thus, much is also left to the application developer's discretion. Choosing appropriate strategies to control request processing behavior depends upon an application's characteristics:

Is the application characterized by infrequent, long-running requests? by frequent, short-running requests? by a mixture of requests with varied run times?
Will the application support a large client base? a small client base?
Are the application's clients associated with a server for short periods of time, perhaps only a single request? for a long period of time spanning many requests?
To what extent will concurrent requests contend for shared resources?

Other factors may also influence or constrain request processing strategies:

Will the application software depend upon software libraries or legacy code that is not multi-thread safe?
Are there constraints in the processing environment such that certain portions of code must be executed on the main thread?

Careful examination of an application's request processing characteristics and related factors is an important step when considering potential request dispatching strategies.

This article addresses request-processing aspects of CORBA and TAO [The ACE ORB (TAO) is a standard-compliant CORBA ORB developed and maintained by the Distributed Object Computing Group (DOC Group) and commercially supported by OCI.], specifically an ORB's thread-related operations and the POA's threading models as set forth by the CORBA specification.

TAO's implementation is largely based upon software patterns that have emerged from distributed system and distributed real-time embedded (DRE) system research in recent years. The Related Topics section contains references to material that presents several software patterns of particular interest to readers wanting to understand or examine TAO's implementation. Some readers, especially those involved in DRE system development, may find the Real-Time CORBA Specification's (formal/02-08-02) threading and concurrent processing capabilities useful. (TAO includes a RT CORBA implementation, but that is beyond the scope of these articles.)

Thread-Related ORB Operations

The ORB's thread-related operations are used to:

Provide one or more threads to an ORB
Shut down an ORB
Destroy an ORB

The relevant portions of the ORB's interface definition are:

module CORBA
{
  // Portions of the CORBA module omitted.
	
  interface ORB
  {
    // Portions of the ORB interface omitted.
		
    boolean work_pending();
    void perform_work();
    void run();
    void shutdown( in boolean wait_for_completion );
    void destroy();
  }
}

ORB::run() provides a thread to the ORB.
- In appropriate circumstances, this operation can be called from multiple threads to provide more than one thread to an ORB.
- This is a blocking operation from the application's perspective. Each thread that calls ORB::run() becomes dedicated to the ORB.
- ORB::run() does not return until the ORB has shut down.
ORB::perform_work() provides a thread to an ORB for a single unit of work.
- This is a non-blocking operation from the application's perspective; the application can perform tasks un-related to the ORB between calls to ORB::perform_work().
- The definition for a unit of work is left to ORB implementers; often it is a single invocation.
- ORB::perform_work() is best used in conjunction with ORB::work_pending() to implement a polling loop that interleaves ORB processing with other activities, such as servicing another external interface.
ORB::work_pending() indicates whether or not an ORB has an immediate need for a thread to perform ORB-related tasks.
- The CORBA specification states that ORB::work_pending() and ORB::perform_work() are intended for use only with the main thread. In practice, it isn't clear the extent to which this restriction is implemented.
ORB::shutdown() instructs an ORB to halt processing.
- This operation is typically invoked just prior to the ORB's destruction.
- If wait_for_completion is TRUE, this operation blocks until the ORB has concluded request processing and other object adapter related activities and all object adapters have been destroyed.
- If wait_for_completion is FALSE, ORB::shutdown() may return before the ORB completes its shutdown process.
- Some implementations may require a call to ORB::run() or ORB::perform_work() after a call to ORB::shutdown() with wait_for_completion == FALSE so the ORB can finish shutting down.
- An ORB continues its normal processing tasks while it is shutting down so there may be a significant delay, after ORB::shutdown() is called, before the ORB actually stops processing. For this reason, some implementations may allow an application to specify a time limit for the shutdown operation. If the ORB has not shut down gracefully within the allotted time, the ORB may then conduct a forced shutdown.
ORB::destroy() destroys an ORB and releases its resources so they can be reclaimed by the application.
- This operation will initiate the shutdown process when called on an ORB that has not been shut down.
- If ORB::destroy()initiates the shutdown process, it blocks until the ORB has shut down, as if ORB::shutdown() had been called with wait_for_completion == TRUE.

Exceptions Raised by Thread-Related Operations

The following exceptions may be raised by thread-related operations or by circumstances arising from the use of thread-related operations:

ORB::work_pending() or ORB::perform_work called on an ORB after it has been shut down raises BAD_INV_ORDER.
ORB::shutdown() with wait_for_completion == TRUE called from a thread processing a CORBA request raises BAD_INV_ORDER with an OMG minor code of 3 and a completion status of COMPLETED_NO.
Any operation other than duplicate(), release(), or is_nil() invoked on an ORB after it has shut down, or invoked on an object reference obtained from an ORB that has since shut down raises BAD_INV_ORDER with an OMG minor code of 4.
- Scenarios may arise in which operations other than those cited here may be called while an ORB is shutting down. Application developers should be prepared to handle this exception when such scenarios may occur because the time required to shut down an ORB is not predictable. (See the next section's discussion.)
Any operation invoked on an ORB after its destruction raises OBJECT_NOT_EXIST.
ORB::destroy() called from a thread processing an invocation raises BAD_INV_ORDER with an OMG minor code of 3.

Guidelines For Application Developers

Server and hybrid applications (hybrid applications act as a CORBA client as well as a CORBA server) must make one or more threads available to the ORB via ORB::run() or ORB::perform_work().

A thread from which ORB::run() is called becomes dedicated to the ORB. A thread that calls ORB::perform_work() can be used to perform other tasks in addition to ORB-related tasks.

Pure client applications do not need to use the thread-related operations. (Applications that behave largely as clients but employ a callback object or Asynchronous Message Invocation [AMI] to receive a server's replies are considered hybrid applications.)

Some common scenarios are:

A single-threaded server that has all of its tasks initiated via CORBA invocations typically calls ORB::run() from the main thread after the application is initialized.
A multi-threaded server that has all of its tasks initiated via CORBA invocations typically calls ORB::run() from each thread.
A single-threaded server that has processing tasks un-related to CORBA invocations implements a polling loop, using ORB::work_pending() and ORB::perform_work(), to interleave CORBA invocations with other processing tasks.
A typical polling loop is similar to the following:

// Code fragment
for(;;)
{
  if(orb->work_pending())
  {
    orb->perform_work();
  }
  // do other tasks
}

In most cases, ORB::perform_work() should be called only when ORB::work_pending() returns TRUE to prevent the application from blocking in the absence of CORBA invocations. However, some ORB implementations may allow time-bounded calls to ORB::perform_work() and thus remove the need to call ORB::work_pending().

A multi-threaded server that has tasks un-related to CORBA invocations has two options:

Dedicate each thread to ORB-related tasks with ORB::run() or to other tasks using the appropriate mechanism.
Dedicate a group of threads to the ORB with ORB::run() and use a polling loop to interleave ORB-related and other tasks on another thread or group of threads.

Whether or not to use multiple threads for tasks unrelated to the ORB depends upon the nature of those tasks and the applicable concurrency constraints. When a polling loop is used in multi-threaded applications, it is often executed on the main thread after other threads that are dedicated to the ORB have been spawned.

The run time of ORB::perform_work() is not deterministic. Application developers should pay careful attention to an ORB's implementation and their application's design when interleaving CORBA invocations with other processing tasks via ORB::perform_work().

Numerous factors influence the run time of ORB::perform_work(), including the following:

Long-running CORBA requests
Outbound CORBA requests that are issued during the processing of an inbound CORBA request
The ORB's definition of a unit of work

Some implementations extend ORB::perform_work() to permit specification of a maximum run time, but typically this is not sufficient to place an absolute limit on ORB::perform_work()'s run time. For example, when a synchronous outbound request is issued during the processing of an inbound request, the run time for the inbound request becomes a function of the outbound request's run time, which is beyond the scope of the local ORB.

Terminating a CORBA-based application gracefully is often a complicated matter because the ORB continues normal processing while it shuts down. An application dedicated to processing requests can be signaled to shut down via a CORBA request. An application that performs tasks un-related to CORBA requests can be signaled to shut down via a CORBA request as well as any other interfaces that are serviced outside the scope of CORBA requests.

If signaled to shut down via a CORBA interface, ORB::shutdown() can be invoked with wait_for_completion == FALSE from the thread that processes the request; invoking ORB::shutdown() with wait_for_completion == TRUE would raise an exception in this situation.

If signaled to shut down outside the scope of a CORBA request, the responding thread can invoke ORB::shutdown() with wait_for_completion == TRUE or FALSE; wait_for_completion == TRUE in this case will block the calling thread until the ORB has shut down.

Once ORB::shutdown() has been called, all calls to ORB::run() will return once the ORB has shut down. However, the time required to shut down an ORB depends upon the volume of client activity when shutdown is requested. In extreme cases, a complete cessation of client activity may be necessary to allow the ORB to shut down.

Here are some options for shutting down a CORBA-based application:

The application can be killed with the appropriate platform-specific mechanism. This is the simplest solution but it is not graceful and may have undesirable side-effects.
ORB::shutdown() can be called with wait_for_completion == FALSE from a thread processing a CORBA request that was invoked via some maintenance or administrative interface. All calls to ORB::run() will return once the ORB has shut down and thus release the threads dedicated to the ORB. Some mechanism should be used to prevent the main thread from exiting before the other threads exit.
The application may continue to run for a relatively long time. If this is unacceptable, another mechanism should be used to forcibly terminate the application.
The application can block the main thread on a condition variable or sempahore until the application is signaled to shut down. (In this case, the main thread is not used for ORB-related tasks.) Once un-blocked, the main thread calls ORB::shutdown() to terminate ORB-related processing. If ORB::shutdown() is called with wait_for_completion == FALSE, some other mechanism should be used to prevent the main thread from exiting before all other threads have exited.

This last option provides some additional flexibility. After the main thread initiates shutdown, it might then block until all other threads have exited or

a specified time interval has elapsed. If the other threads exit before the time interval elapses, the main thread conducts any other cleanup activities and then exits. If the time period elapses, the main thread performs some, perhaps not all, of the usual cleanup activity and then exits causing a forced termination.

Request Processing

A CORBA request is processed in two stages:

The ORB assigns the request to a thread and dispatches it to a POA.
The POA then dispatches the request to an application object.

Strategies employed by a multi-threaded ORB as it receives and dispatches requests determine the extent to which requests may be processed concurrently. These strategies are usually described as a ratio between threads and requests as follows:

thread-per-request. A separate thread is spawned to process each request.
thread-per-connection. A separate thread is spawned to process all requests received via a distinct network connection.
thread-per-client. A separate thread is spawned to process all requests received from a distinct client.
thread-pool. Requests are distributed among a group of threads.

Each strategy has its strengths and weaknesses, which motivates understanding an application's request-processing characteristics to choose the most appropriate strategy or combination of strategies. A complete discussion of request dispatching strategies is beyond the scope of this article.

The remainder of this article is devoted to the POA's threading models. (For additional information regarding threading and concurrency strategies, see the Related Topics section at the end of this article.)

POA Threading Models

The POA threading models establish concurrency constraints that are imposed during request processing in a multi-threaded environment. These constraints further qualify the extent to which CORBA requests may be processed concurrently.

A POA's threading model is determined by its ThreadPolicy value. There are three standard ThreadPolicy values:

ORB_CTRL_MODEL. Requests are dispatched to application objects from the thread with which the ORB invoked the POA. Concurrent servant upcalls may occur if the ORB's effective strategies allow requests to be dispatched concurrently.
SINGLE_THREAD_MODEL. The POA dispatches requests to application objects sequentially. Concurrent servant upcalls shall not occur at the POA level.
MAIN_THREAD_MODEL. Requests for all main-threaded POAs are dispatched sequentially. Concurrent servant upcalls shall not occur within the scope of all main-threaded POAs.

ORB Controlled Model

An ORB-controlled POA places no constraints on concurrent requests; the POA effectively abdicates responsibility for threading and concurrent request processing to the ORB.

Figure 1. ORB-Controlled Threading Model

Concurrent servant upcalls may occur at the POA level in multi-threaded environments if the ORB dispatches requests concurrently.

Application objects activated by an ORB-controlled POA in a multi-threaded environment should be multi-thread safe.

Single-Thread Model

A single-threaded POA constrains request processing such that concurrent upcalls shall not occur at the POA level, even in multi-threaded environments.

Figure 2. Single-Thread Threading Model

A multi-threaded ORB may dispatch concurrent requests to a single-threaded POA, but the subsequent servant upcalls will occur sequentially; thus an application object activated by a single-threaded POA will not be subject to concurrent requests. However, an application object activated by multiple single-threaded POAs may be subject to concurrent requests in a multi-threaded environment.

Figure 3. Servant-Activated by Multiple Single-Threaded POAs

Activating an application object with multiple POAs is not recommended.

Main Thread Model

Requests are dispatched to all main-threaded POAs sequentially. This effectively serializes requests dispatched by main-threaded POAs at the ORB level.

Figure 4. Main-Thread Model

An application object activated by a main-threaded POA will not be subject to concurrent requests. The main-threaded model is also applicable in processing environments where some code must be executed on the main thread. In such environments, the main-thread model insures that servant upcalls are processed on that thread. However, the application must make the main thread available to the ORB via ORB::run() or ORB::perform_work().

CONTROLLING A POA'S THREADING MODEL

A POA's threading model is determined by its ThreadPolicy. The default ThreadPolicy value is ORB_CTRL_MODEL. A POA's ThreadPolicy, like all other POA policies, can be assigned only when the POA is created.

The relevant portions of the PortableServer module and the POA's interface are:

module PortableServer {
 
  // Portions of the PortableServer module omitted.
 
  const CORBA::PolicyType THREAD_POLICY_ID = 16;
 
  enum ThreadPolicyValue {
    ORB_CTRL_MODEL,
    SINGLE_THREAD_MODEL,
    MAIN_THREAD_MODEL
  };
 
  local interface ThreadPolicy : CORBA::Policy {
    readonly attribute ThreadPolicyValue value;
  };
 
  local interface POA {
 
    // Portions of interface POA omitted.
 
    POA create_POA(
      in string adapter_name;
      in POAManager aPOAManager,
      in CORBA::PolicyList policies
      ) raises (AdapterAlreadyExists, InvalidPolicy);
 
    ThreadPolicy create_thread_policy(
      in ThreadPolicyValue value);
 
  };
};

The following code fragment demonstrates how to create a single-threaded POA:

  // Portions of int main(int, char **argv) omitted.
 
  // Init the ORB.
  CORBA::ORB_var orb = CORBA::ORB_init (argc, argv);
 
  // Get the Root POA.
  CORBA::Object_var obj = orb->resolve_initial_references ("RootPOA");
  PortableServer::POA_var poa = PortableServer::POA::_narrow (obj.in());
 
  // Create and populate a policy list.
  CORBA::PolicyList policies(1);
  policies[0] = poa->create_thread_policy(PortableServer::SINGLE_THREAD_MODEL);
 
  // Use the Root POA's POAManager.
  PortableServer::POAManager_var mgr = poa->the_POAManager ();
 
  // Create a child POA.
  // Threading model is single-threaded.
  // All other policies assume the default value.
  PortableServer::POA_var st_poa = poa->create_POA("ST POA", mgr.in(), policies);
 
  // Release memory allocated to the policy list.
  policies[0]->destroy();

GUIDELINES FOR APPLICATION DEVELOPERS

The ORB-controlled threading model (ORB_CTRL_MODEL) allows application developers to make the most of an ORB's concurrent processing capabilities and performance optimization strategies. An application using this threading model can also take advantage of fine-grained contention management techniques to resolve application objects' contention for shared resources and services. However, this model obligates application developers to assume concurrent requests will occur, and identify and resolve points of contention that may arise during concurrent CORBA requests.

The following scenarios might motivate use of the ORB-controlled threading model for a CORBA-based application that is subject to concurrent requests:

The application's mission is to provide access to and resolve contention for a shared resource or service, i.e. the application is effectively a contention manager so use of the ORB-controlled threading model does not add additional complexity.
The application serializes and dispatches events, perhaps received from disparate sources, to discrete event processors (application objects) so multiple events can be processed concurrently. As with the previous scenario, applications such as this are obligated to resolve contention.
The application is a CORBA interface to an existing system that permits concurrent invocations.
Acceptable performance can only be achieved with concurrent processing and fine-grained contention management.
The application is stateless, i.e., the context carried with each CORBA request is sufficient to process the request.

The ORB-controlled model might also be appropriate when the ORB offers a serialized request dispatching strategy that out-performs single-threaded or main-threaded POAs.

The single-threaded model (SINGLE_THREAD_MODEL) can simplify an application's design and implementation by guaranteeing that an application object will not be subject to concurrent invocations from a distinct POA. Applied within its constraints, this model shifts some responsibility for contention management away from the application. However, this mechanism can only resolve contention for distinct servants arising from concurrent requests; it can not resolve contention for services and resources shared by application objects. Resolving contention for commonly-used services and resources remains the application's responsibility.

The single-threaded model is appropriate when:

The likelihood of concurrent requests is acceptably low.
The increased latency caused by the serialization of requests is acceptable.
The potential performance improvement resulting from concurrent request processing does not justify the added complexity associated with fine-grained contention management.
An application depends upon software libraries or legacy code that effectively prohibits concurrent processing.
An application employs an ORB that does not provide a suitable request serialization capability.

The main-thread model (MAIN_THREAD_MODEL) is applicable in two circumstances:

An application's environment mandates that certain portions of code execute only on the main thread. However, the application is obligated to make the main thread available to the ORB by calling ORB::run() orORB::perform_work() from the main thread.
An application requires protection from concurrent requests more rigorous than that provided by the single-thread model.

This model imposes the most stringent constraints upon an application and likely reduces a multi-threaded ORB's efficiency. Application developers may want to compare performance of this threading model with that of a single-threaded ORB before committing to this model.

SUMMARY

This article examined aspects of CORBA request processing related to threading and concurrent processing as set forth by the CORBA specification. The material presented here discusses the ORB's thread-related operations and their use, as well as the definition and use of the POA's threading models. Guidelines are offered to help developers consider an application's requirements and choose strategies that will fulfill those requirements.