Scaling, Distributing, and Tuning CORBA Applications

     Previous  Next    Open TOC in new window  Open Index in new window  View as PDF - New Window  Get Adobe Reader - New Window
Content starts here

Scaling CORBA Server Applications

This topic includes the following sections:

Using the Production sample application as an example, this topic demonstrates scaling an CORBA C++ application to increase its processing capability. Before you begin, be sure to read:

 


About Scaling the Production Sample Application

The Production sample application provides the same end-user functionality as the Wrapper sample application. The Production sample application demonstrates how to use features of the Oracle Tuxedo software to scale an existing Oracle Tuxedo application.

This section includes the following topics:

Design Goals

The primary design goal of the Production sample application is to significantly increase the number of client applications it can accommodate by:

How the Application Has Been Scaled

To accommodate these design goals, the Production sample application has been scaled by:

Note: To make the Production sample application easy to use, this application is configured on the Oracle Tuxedo software kit to run on one machine, using one database. The examples shown in this chapter, however, show running this application on two machines using two databases.
Note: The Production sample application is designed so that it can be configured to run on several machines and to use multiple databases. Changing the configuration to multiple machines and databases involves modifying the UBBCONFIG file and partitioning the databases, which is described in Scaling the Application Further.

The sections that follow describe how the Production sample application uses replicated server processes and server groups, object state management, and factory-based routing to meet its scalability goals.

 


Changing the OMG IDL

The only OMG IDL changes for the Production sample application are limited to the find_registrar() and find_teller() operations on, respectively, the RegistrarFactory and TellerFactory objects. These two operations need to be modified to require, respectively, a student ID and account number, which are needed to implement factory-based routing. See Scaling with Factory-based Routing to read about how the Production sample application implements and uses factory-based routing.

 


Using a Stateless Object Model

This section describes how object state management is used with the Registrar and Teller objects in the Production sample applications to increase the application’s scalability. For an introduction to object state management, see Using Object State Management.

To increase scalability, the Registrar and Teller objects are configured in the Production server application with the method activation policy. The method activation policy assigned to these two objects results in the following behavior changes:

With the Basic through the Wrapper sample applications, the Registrar object was process-bound (process activation policy). All client requests on the Registrar object invariably went to the same object instance in the memory of the server machine. The Basic sample application design may be adequate for a small-scale deployment. However, as client application demands increase, client requests on the Registrar object eventually become queued, and response time drops.

However, when the Registrar and Teller objects are stateless (method activation policy), and the server processes that manage these objects are replicated, the Registrar and Teller objects can process multiple client requests in parallel. The only constraint on the number of simultaneous client requests that these objects can handle is the number of server processes that are available that can instantiate the Registrar and Teller objects. These stateless objects, thereby, make for more efficient use of machine resources and reduced client response time.

Most importantly, so that Oracle Tuxedo CORBA can instantiate copies of the Registrar and Teller objects in each of the replicated server processes, each copy of these objects must be unique. To make each instance of these objects unique, the factories for those objects must assign unique object IDs to them.

For the Oracle Tuxedo application to instantiate copies of the Registrar and Teller objects in each of the replicated server application processes, each copy of the Registrar and Teller objects have an unique object ID (OID). The factories that create these objects are responsible for assigning them unique OIDs. For information about generating unique object IDs, see Creating CORBA Server Applications . For more information about other design considerations, see Additional Design Considerations.

 


Scaling by Replicating Server Processes and Server Groups

This topic includes the following sections:

This topic describes how the Production sample application was scaled by replicating server processes and server groups. For an introduction to this topic, see Replicating Server Processes and Server Groups.

Replicating Server Processes in the Production Application

This section describes how the Production sample application replicates server applications. For an introduction to this feature, see Replicating Server Processes.

Figure 2-1 shows the replicated ORA_GRP and APP_GRP groups running on a single machine.

When a request arrives for either of these groups, the Oracle Tuxedo domain has several server processes available that can process the request, and the Oracle Tuxedo domain can choose the server process that is the least busy.

In Figure 2-1, note the following points:

Replicating Server Groups in the Production Application

This section describes how the Production sample application replicates server groups. For an introduction to this feature, see Replicating Server Groups.

Figure 2-2 shows the Production sample application groups replicated on another machine, as specified in the application’s UBBCONFIG file, as ORA_GRP2 and APP_GRP2.

Figure 2-2 Replicating Server Groups Across Machines

Replicating Server Groups Across Machines

In Figure 2-2, the only difference between the content of the groups on Production Machines 1 and 2 is the database:

Note: The course information table in both databases is identical.

Note that the student information in a given database may be completely unrelated to the account information in the same database.

For more information about how the Production sample application uses factory-based routing to distribute the application’s processing load across multiple machines, see Scaling with Factory-based Routing.

Configuring Replicated Server Processes and Groups in the Production Application

Listing 2-1 shows excerpts from the GROUPS and SERVERS sections of the UBBCONFIG file for the Production sample application.

Listing 2-1 GROUPS and SERVERS Sections in a UBBCONFIG File
*GROUPS
    APP_GRP1
        LMID = SITE1
        GRPNO = 2
        TMSNAME = TMS
    APP_GRP2
        LMID = SITE1
        GRPNO = 3
        TMSNAME = TMS
    ORA_GRP1
        LMID = SITE1
        GRPNO = 4
        OPENINFO = "ORACLE_XA:Oracle_XA+Acc=P/scott/..."
        CLOSEINFO = ""
        TMSNAME = "TMS_ORA"
    ORA_GRP2
        LMID = SITE1
        GRPNO = 5
        OPENINFO = "ORACLE_XA:Oracle_XA+Acc=P/scott/..."
        CLOSEINFO = ""
        TMSNAME = "TMS_ORA"
                
*SERVERS
    # By default, activate 2 instances of each server
    # and allow the administrator to activate up to 5
    # instances of each server
    DEFAULT:
        MIN = 2
        MAX = 5
    tellp_server
        SRVGRP = ORA_GRP1
        SRVID = 10
        RESTART = N
    tellp_server
        SRVGRP = ORA_GRP2
        SRVID = 10
        RESTART = N
    billp_server
        SRVGRP = APP_GRP1
        SRVID = 10
        RESTART = N
    billp_server
        SRVGRP = APP_GRP2
        SRVID = 10
        RESTART = N
    univp_server
        SRVGRP = ORA_GRP1
        SRVID = 20
        RESTART = N
    univp_server
        SRVGRP = ORA_GRP2
        SRVID = 20
        RESTART = N

 


Scaling with Factory-based Routing

This topic includes the following sections:

This topic describes how the Production sample application was scaled using factory-based routing. For an introduction to factory-based routing, see Using Factory-based Routing (CORBA Servers Only).

About Factory-based Routing in the Production Application

This section describes how the Production sample application uses a factory-based routing. For an introduction to this feature, see Using Factory-based Routing (CORBA Servers Only).

You can use factory-based routing to expand the load-balancing and scalability features of Oracle Tuxedo CORBA. In the Production sample application, you can use factory-based routing to send requests to register one subset of students to one machine, and requests for another subset of students to another machine. As you increase your application’s processing capability, you can easily modify the factory-based routing in your application to add more machines.

The primary design consideration regarding implementing factory-based routing in the Production sample application is in choosing the value on which routing is based. The Production sample application uses factory-based routing in the following ways:

Configuring Factory-based Routing in the UBBCONFIG File

The University Production sample application demonstrates how to implement factory-based routing. The INTERFACES, ROUTING, and GROUPS sections from the ubb_b.nt configuration file show how you can implement factory-based routing in an Oracle Tuxedo CORBA application. You can find the ubb_p.nt or ubb_p.mk UBBCONFIG files for this sample in the directory where the Oracle Tuxedo software is installed (see the \samples\corba\university\production subdirectory).

The UBBCONFIG file must specify the following data in the INTERFACES and ROUTING sections, as well as how groups and machines are identified.

  1. The INTERFACES section lists the names of the interfaces for which you want to enable factory-based routing. For each interface, this section specifies the kinds of criteria on which the interface routes. This section specifies the routing criteria via an identifier, FACTORYROUTING, as shown in Listing 2-2.
  2. Listing 2-2 INTERFACES Section of a UBBCONFIG File
    INTERFACES
        "IDL:beasys.com/UniversityP/Registrar:1.0"
    FACTORYROUTING = STU_ID
        "IDL:beasys.com/BillingP/Teller:1.0"
            FACTORYROUTING = ACT_NUM

    Listing 2-2 shows the fully qualified interface names for the two interfaces in the Production sample in which factory-based routing is used. The FACTORYROUTING identifier specifies the names of the routing values, which are STU_ID and ACT_NUM, respectively.

  3. The ROUTING section specifies the parameters in Table 2-1 for each routing value.
  4. Table 2-1 Parameters Specified in the ROUTING Section 
    Parameter
    Description
    TYPE
    Specifies the type of routing. In the Production sample, the type of routing is factory-based routing. Therefore, this parameter is defined as FACTORY.
    FIELD
    Specifies the variable name that the factory inserts in the routing value. In the Production sample, the field parameters are student_id and account_number, respectively.
    FIELDTYPE
    Specifies the data type of the routing value. In the Production sample, the field types for student_id and account_number are long.
    RANGES
    Specifies the values that are routed to each group.

    Listing 2-3 shows the ROUTING section of the UBBCONFIG file used in the Production sample application.

    Listing 2-3 ROUTING Section of the UBBCONFIG File
    ROUTING
        STU_ID
            FIELD = "student_id"
            TYPE = FACTORY
            FIELDTYPE = LONG
            RANGES = "100001-100005:ORA_GRP1,100006-100010:ORA_GRP2"
        ACT_NUM
            FIELD = "account_number"
            TYPE = FACTORY
            FIELDTYPE = LONG
            RANGES = "200010-200014:APP_GRP1,200015-200019:APP_GRP2"

    Listing 2-3 shows that Registrar object references for students with IDs in one range are routed to one server group, and Registrar object references for students with IDs in another range are routed to another group. Likewise, Teller object references for accounts in one range are routed to one server group, and Teller object references for accounts in another range are routed to another group.

  5. The groups specified by the RANGES identifier in the ROUTING section of the UBBCONFIG file need to be identified and configured. For example, the Production sample specifies four groups: APP_GRP1, APP_GRP2, ORA_GRP1, and ORA_GRP2. These groups need to be configured, and the machines on which they run need to be identified.
  6. Listing 2-4 shows the GROUPS section of the Production sample UBBCONFIG file, in which the ORA_GRP1 and ORA_GRP2 groups are configured. Notice how the names in the GROUPS section match the group names specified in the RANGES parameter in the ROUTING section. This is critical for factory-based routing to work correctly. Furthermore, any change in the way groups are configured in an application must be reflected in the ROUTING section. (Note that the Production sample packaged with the Oracle Tuxedo software is configured to run entirely on one machine. However, you can easily configure this application to run on multiple machines.)

    Listing 2-4 GROUPS Section of a UBBCONFIG File
    *GROUPS
        APP_GRP1
            LMID = SITE1
            GRPNO = 2
            TMSNAME = TMS
        APP_GRP2
            LMID = SITE1
            GRPNO = 3
            TMSNAME = TMS
        ORA_GRP1
            LMID = SITE1
            GRPNO = 4
            OPENINFO = "ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SesTm=100+LogDir=.+MaxCur=5"
            CLOSEINFO = ""
            TMSNAME = "TMS_ORA"
        ORA_GRP2
            LMID = SITE1
            GRPNO = 5
    OPENINFO = "ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SesTm=100+LogDir=.+MaxCur=5"
            CLOSEINFO = ""
            TMSNAME = "TMS_ORA"

Implementing Factory-based Routing in a Factory

Factories implement factory-based routing in the way the invocation to the TP::create_object_reference() operation is implemented. This operation has the C++ binding in Listing 2-5.

Listing 2-5 C++ Binding for create_object_reference
CORBA::Object_ptr  TP::create_object_reference (
                       const char* interfaceName,
                       const PortableServer::oid &stroid,
                        CORBA::NVlist_ptr criteria);

The third parameter to this operation, criteria, specifies a list of named values to be used for factory-based routing. To implement factory-based routing in a factory, you need to build the NVlist. The use of factory-based routing is optional and is dependent on this argument. Instead of using factory-based routing, you can pass a value of 0 (zero) for this argument.

As stated previously, the RegistrarFactory object in the Production sample application specifies the value STU_ID. This value must exactly match the following information in the UBBCONFIG file:

The RegistrarFactory object inserts the student ID into the NVlist using the code shown in Listing 2-6.

Listing 2-6 NVlist in the RegistrarFactory Object
// put the student id (which is the routing criteria)
// into a CORBA NVList:
CORBA::NVList_var v_criteria;
TP::orb()->create_list(1, v_criteria.out());
CORBA::Any any;
any <<= (CORBA::Long)student;
v_criteria->add_value("student_id", any, 0);

The RegistrarFactory object has the invocation to the TP::create_object_reference() operation, shown in Listing 2-7, passing the NVlist created in Listing 2-6.

Listing 2-7 Invoking create_object_reference in the RegistrarFactory Object
// create the registrar object reference using
// the routing criteria :
CORBA::Object_var v_reg_oref =
    TP::create_object_reference(
        UniversityP::_tc_Registrar->id(),
        object_id,
        v_criteria.in()
    );

The Production sample application also uses factory-based routing in the TellerFactory object to determine the group in which Teller objects should be instantiated based on an account number.

What Happens at Run Time

When you implement factory-based routing in a factory, Oracle Tuxedo CORBA generates an object reference. The following example shows how the client application gets an object reference to a Registrar object when factory-based routing is implemented.

  1. The client application invokes the RegistrarFactory object, requesting a reference to a Registrar object. The request includes a student ID.
  2. The RegistrarFactory inserts the student ID into an NVlist, which is used as the routing criteria.
  3. The RegistrarFactory invokes the TP::create_object_reference() operation, passing the Registrar interface name, a unique OID, and the NVlist.
  4. Oracle Tuxedo CORBA compares the contents of the routing tables with the value in the NVlist to determine a group ID.
  5. Oracle Tuxedo CORBA inserts information about the group into the object reference.

When the client application subsequently invokes an object using the object reference, Oracle Tuxedo CORBA routes the request to the group specified in the object reference.

Note: If you use the process-entity design pattern, you should use caution in how you implement factory-based routing. The object can service only those entities that are contained in the group’s database.

 


Additional Design Considerations

This topic includes the following sections:

About the Additional Design Considerations

When designing the Registrar and Teller objects, you should ensure that:

These objects must have unique object IDs (OIDs) and must be method-bound (that is, they must have the method activation policy assigned to them).

Instantiating the Registrar and Teller Objects

In the University server applications that are less sophisticated than the Production sample application, the run-time behavior of the Registrar and Teller objects was simpler:

However, because the University and Billing server processes are now replicated, Oracle Tuxedo CORBA must be able to differentiate among multiple instances of the Registrar and Teller objects. For example, if there are two University server processes running in a group, Oracle Tuxedo CORBA must have a means to distinguish between the Registrar object running in the first University server process and the Registrar object running in the second University server process. To distinguish multiple instances of these objects, each object instance must be unique.

To make each Registrar and Teller object unique, the factories for those objects must change the way in which they make object references to them. For example, when the RegistrarFactory object in the Basic sample application created an object reference to the Registrar object, the TP::create_object_reference() operation specified an OID that consisted only of the string registrar. However, in the Production sample application, the same TP::create_object_reference() operation uses a generated unique OID instead.

As a result of giving each Registrar and Teller object a unique OID, multiple instances of these objects may be running simultaneously in the Oracle Tuxedo domain. This characteristic is typical of the stateless object model, and is an example of how the Oracle Tuxedo domain can be highly scalable while it offers high performance.

Finally, because unique Registrar and Teller objects need to be brought into memory for each client request on them, it is critical that these objects be deactivated when the invocations on them are completed so that any object state associated with them does not remain idle in memory. The Production server application addresses this issue by assigning the method activation policy to these two objects in the Implementation Configuration File (ICF).

Ensuring That Student Registration Occurs in the Correct Server Group

The primary scalability advantage of using replicated server groups is being able to distribute processing across multiple machines. However, if your application interacts with a database, which is the case with the University sample applications, it is critical that you consider the impact of these multiple server groups on the database interactions.

In many cases, you may have one database associated with each machine in your deployment. If your server application is distributed across multiple machines, you must consider how you set up your databases.

The Production sample application, as described in this chapter, uses two databases. However, this application can easily be configured to accommodate more. The system administrator can decide on how many databases to use.

In the Production sample application, the student and account information is partitioned across the two databases, but course information is identical. Having identical course information in both databases is not a problem because the course information is read-only for the purposes of course registration. However, the student and account information is read-write. If multiple databases were also to contain identical data for students and accounts (that is, the database is not partitioned), the application would need to deal with the overhead of synchronizing the updates to student and account information across all the databases each time any student or account information were to change.

The Production sample application uses factory-based routing to send one set of requests to one machine, and another set to the other machine. How factory-based routing is implemented in the RegistrarFactory object depends on the way in which references to Registrar objects are created.

For example, when the client application sends a request to the RegistrarFactory object to get an object reference to a Registrar object, the client application includes a student ID in that request. The client application must use the object reference that the RegistrarFactory object returns to make all subsequent invocations on a Registrar object on a particular student’s behalf, because the object reference returned by the factory is group-specific. Therefore, for example, when the client application subsequently invokes the get_student_details() operation on the Registrar object, the client application can be assured that the Registrar object is active in the server group associated with the database containing data for that student.

To show how this works, consider the following execution scenario, which is implemented in the Production sample application:

  1. The client application invokes the find_registrar() operation on the RegistrarFactory object. Included in this invocation is the student ID 1000003.
  2. Oracle Tuxedo CORBA routes the client request to any RegistrarFactory object.
  3. The RegistrarFactory object uses the student ID to create an object reference to a Registrar object in ORA_GRP1, based on the routing information in the UBBCONFIG file, and returns that object reference to the client application.
  4. The client application invokes the register_for_courses() operation on the Registrar object.
  5. Oracle Tuxedo CORBA receives the client request and routes it to the server group specified in the object reference. In this case, the client request goes to the University server process in ORA_GRP1, which is on Production Machine 1.
  6. The University server process instantiates a Registrar object and sends the client invocation to it.

The RegistrarFactory object from the preceding scenario returns to the client application a unique reference to a Registrar object that can be instantiated only in ORA_GRP1, which runs on Production Machine 1 and has a database containing student data for students with IDs in the range 100001 to 100005. Therefore, when the client application sends subsequent requests to this Registrar object on behalf of a given student, the Registrar object interacts with the correct database.

Ensuring That the Teller Object Is Instantiated in the Correct Server Group

When the Registrar object needs a Teller object, the Registrar object invokes the TellerFactory object, using the TellerFactory object reference cached in the University Server object.

However, because factory-based routing is used in the TellerFactory object, the Registrar object passes the student’s account number when the Registrar object requests a reference to a Teller object. This way, the TellerFactory object creates a reference to a Teller object in the group that has the correct database.

Note: For the Production sample application to work properly, it is essential that the system administrator configures the server groups and the databases properly. In particular, the system administrator must make sure that a match exists between the routing criteria specified in the routing tables and the databases to which requests using those criteria are routed. Using the Production sample as an example, the database in a given group must contain the correct student and account information for the requests that are routed to that group.

 


Scaling the Application Further

In the future, the system administrator of the Production sample application may want to add capacity to the Oracle Tuxedo domain. For example, the University may eventually experience a large increase in the student population, or the Production application may be scaled up to accommodate the course registration process for an entire state university system, encompassing several campuses. This can be done without modifying or rebuilding the application.

The system administrator can continually add capacity by:

Note: If you add capacity to an existing Oracle Tuxedo CORBA application that uses a database, you must also consider the impact on how the database is set up, particularly when you are using factory-based routing. For example, if the Production sample application is distributed across six machines, the database on each machine must be set up appropriately and in accordance with the routing tables in the UBBCONFIG file.

  Back to Top       Previous  Next