Scaling, Distributing, and Tuning CORBA Applications
This topic includes the following sections:
Using the Production sample application as an example, this topic demonstrates scaling an CORBA C++ application to increase its processing capability. Before you begin, be sure to read:
The Production sample application provides the same end-user functionality as the Wrapper sample application. The Production sample application demonstrates how to use features of the BEA Tuxedo software to scale an existing BEA Tuxedo application.
This section includes the following topics:
The primary design goal of the Production sample application is to significantly increase the number of client applications it can accommodate by:
To accommodate these design goals, the Production sample application has been scaled by:
ORA_GRP
and APP_GRP
server groups defined in the UBBCONFIG
file).ORA_GRP
and APP_GRP
server groups on an additional server machine, Production Machine 2, and also partitioning the database.This makes these objects available on a per-client application (and not per-process) basis, thereby accommodating a parallel processing capability.
Note: To make the Production sample application easy to use, this application is configured on the BEA Tuxedo software kit to run on one machine, using one database. The examples shown in this chapter, however, show running this application on two machines using two databases.
The Production sample application is designed so that it can be configured to run on several machines and to use multiple databases. Changing the configuration to multiple machines and databases involves modifying the UBBCONFIG
file and partitioning the databases, which is described in Scaling the Application Further.
The sections that follow describe how the Production sample application uses replicated server processes and server groups, object state management, and factory-based routing to meet its scalability goals.
The only OMG IDL changes for the Production sample application are limited to the find_registrar()
and find_teller()
operations on, respectively, the RegistrarFactory
and TellerFactory
objects. These two operations need to be modified to require, respectively, a student ID and account number, which are needed to implement factory-based routing. See Scaling with Factory-based Routing to read about how the Production sample application implements and uses factory-based routing.
This section describes how object state management is used with the Registrar
and Teller
objects in the Production sample applications to increase the application's scalability. For an introduction to object state management, see Using Object State Management.
To increase scalability, the Registrar
and Teller
objects are configured in the Production server application with the method
activation policy. The method
activation policy assigned to these two objects results in the following behavior changes:
With the Basic through the Wrapper sample applications, the Registrar
object was process-bound (process
activation policy). All client requests on the Registrar
object invariably went to the same object instance in the memory of the server machine. The Basic sample application design may be adequate for a small-scale deployment. However, as client application demands increase, client requests on the Registrar
object eventually become queued, and response time drops.
However, when the Registrar
and Teller
objects are stateless (method
activation policy), and the server processes that manage these objects are replicated, the Registrar
and Teller
objects can process multiple client requests in parallel. The only constraint on the number of simultaneous client requests that these objects can handle is the number of server processes that are available that can instantiate the Registrar
and Teller
objects. These stateless objects, thereby, make for more efficient use of machine resources and reduced client response time.
Most importantly, so that BEA Tuxedo CORBA can instantiate copies of the Registrar
and Teller
objects in each of the replicated server processes, each copy of these objects must be unique. To make each instance of these objects unique, the factories for those objects must assign unique object IDs to them.
For the BEA Tuxedo application to instantiate copies of the Registrar
and Teller
objects in each of the replicated server application processes, each copy of the Registrar
and Teller
objects have an unique object ID (OID). The factories that create these objects are responsible for assigning them unique OIDs. For information about generating unique object IDs, see Creating CORBA Server Applications. For more information about other design considerations, see Additional Design Considerations.
This topic includes the following sections:
This topic describes how the Production sample application was scaled by replicating server processes and server groups. For an introduction to this topic, see Replicating Server Processes and Server Groups.
This section describes how the Production sample application replicates server applications. For an introduction to this feature, see Replicating Server Processes.
Figure 2-1 shows the replicated ORA_GRP
and APP_GRP
groups running on a single machine.
ORA_GRP
group.APP_GRP
group.Figure 2-1 Replicated Server Groups in the Production Sample
When a request arrives for either of these groups, the BEA Tuxedo domain has several server processes available that can process the request, and the BEA Tuxedo domain can choose the server process that is the least busy.
In Figure 2-1, note the following points:
RegistrarFactory
, Registrar
, TellerFactory
, or Teller
objects within a given server process.CourseSynopsisEnumerator
objects in any University server process.This section describes how the Production sample application replicates server groups. For an introduction to this feature, see Replicating Server Groups.
Figure 2-2 shows the Production sample application groups replicated on another machine, as specified in the application's UBBCONFIG
file, as ORA_GRP2
and APP_GRP2
.
Figure 2-2 Replicating Server Groups Across Machines
In Figure 2-2, the only difference between the content of the groups on Production Machines 1 and 2 is the database:
100001
and 100005
.100006
and 100010
.Note: The course information table in both databases is identical.
Note that the student information in a given database may be completely unrelated to the account information in the same database.
For more information about how the Production sample application uses factory-based routing to distribute the application's processing load across multiple machines, see Scaling with Factory-based Routing.
Listing 2-1 shows excerpts from the GROUPS
and SERVERS
sections of the UBBCONFIG
file for the Production sample application.
Listing 2-1 GROUPS and SERVERS Sections in a UBBCONFIG File
*GROUPS
APP_GRP1
LMID = SITE1
GRPNO = 2
TMSNAME = TMS
APP_GRP2
LMID = SITE1
GRPNO = 3
TMSNAME = TMS
ORA_GRP1
LMID = SITE1
GRPNO = 4
OPENINFO = "ORACLE_XA:Oracle_XA+Acc=P/scott/..."
CLOSEINFO = ""
TMSNAME = "TMS_ORA"
ORA_GRP2
LMID = SITE1
GRPNO = 5
OPENINFO = "ORACLE_XA:Oracle_XA+Acc=P/scott/..."
CLOSEINFO = ""
TMSNAME = "TMS_ORA"
*SERVERS
# By default, activate 2 instances of each server
# and allow the administrator to activate up to 5
# instances of each server
DEFAULT:
MIN = 2
MAX = 5
tellp_server
SRVGRP = ORA_GRP1
SRVID = 10
RESTART = N
tellp_server
SRVGRP = ORA_GRP2
SRVID = 10
RESTART = N
billp_server
SRVGRP = APP_GRP1
SRVID = 10
RESTART = N
billp_server
SRVGRP = APP_GRP2
SRVID = 10
RESTART = N
univp_server
SRVGRP = ORA_GRP1
SRVID = 20
RESTART = N
univp_server
SRVGRP = ORA_GRP2
SRVID = 20
RESTART = N
This topic includes the following sections:
This topic describes how the Production sample application was scaled using factory-based routing. For an introduction to factory-based routing, see Using Factory-based Routing (CORBA Servers Only).
This section describes how the Production sample application uses a factory-based routing. For an introduction to this feature, see Using Factory-based Routing (CORBA Servers Only).
You can use factory-based routing to expand the load-balancing and scalability features of BEA Tuxedo CORBA. In the Production sample application, you can use factory-based routing to send requests to register one subset of students to one machine, and requests for another subset of students to another machine. As you increase your application's processing capability, you can easily modify the factory-based routing in your application to add more machines.
The primary design consideration regarding implementing factory-based routing in the Production sample application is in choosing the value on which routing is based. The Production sample application uses factory-based routing in the following ways:
Registrar
object are routed based on the student ID. Requests from student ID 100001
to 100005
go to Production Machine 1. Requests from student ID 100006
to 100010
go to Production Machine 2.Registrar
object to the Teller
object are routed based on account number. Billing requests for account 200010
to 200014
go to Production Machine 1. Billing requests for account 200015
to 200019
go to Production Machine 2.The University Production sample application demonstrates how to implement factory-based routing. The INTERFACES, ROUTING
, and GROUPS
sections from the ubb_b.nt
configuration file show how you can implement factory-based routing in a BEA Tuxedo CORBA application. You can find the ubb_p.nt
or ubb_p.mk
UBBCONFIG
files for this sample in the directory where the BEA Tuxedo software is installed (see the \samples\corba\university\production
subdirectory).
The UBBCONFIG
file must specify the following data in the INTERFACES
and ROUTING
sections, as well as how groups and machines are identified.
INTERFACES
section lists the names of the interfaces for which you want to enable factory-based routing. For each interface, this section specifies the kinds of criteria on which the interface routes. This section specifies the routing criteria via an identifier, FACTORYROUTING
, as shown in Listing 2-2.Listing 2-2 INTERFACES Section of a UBBCONFIG File
INTERFACES
"IDL:beasys.com/UniversityP/Registrar:1.0"
FACTORYROUTING = STU_ID
"IDL:beasys.com/BillingP/Teller:1.0"
FACTORYROUTING = ACT_NUM
Listing 2-2 shows the fully qualified interface names for the two interfaces in the Production sample in which factory-based routing is used. The FACTORYROUTING
identifier specifies the names of the routing values, which are STU_ID
and ACT_NUM
, respectively.
ROUTING
section specifies the parameters in Table 2-1 for each routing value.Listing 2-3 shows the ROUTING
section of the UBBCONFIG
file used in the Production sample application.
Listing 2-3 ROUTING Section of the UBBCONFIG File
ROUTING
STU_ID
FIELD = "student_id"
TYPE = FACTORY
FIELDTYPE = LONG
RANGES = "100001-100005:ORA_GRP1,100006-100010:ORA_GRP2"
ACT_NUM
FIELD = "account_number"
TYPE = FACTORY
FIELDTYPE = LONG
RANGES = "200010-200014:APP_GRP1,200015-200019:APP_GRP2"
Listing 2-3 shows that Registrar
object references for students with IDs in one range are routed to one server group, and Registrar
object references for students with IDs in another range are routed to another group. Likewise, Teller
object references for accounts in one range are routed to one server group, and Teller
object references for accounts in another range are routed to another group.
RANGES
identifier in the ROUTING
section of the UBBCONFIG
file need to be identified and configured. For example, the Production sample specifies four groups: APP_GRP1
, APP_GRP2
, ORA_GRP1
, and ORA_GRP2
. These groups need to be configured, and the machines on which they run need to be identified.Listing 2-4 shows the GROUPS
section of the Production sample UBBCONFIG
file, in which the ORA_GRP1
and ORA_GRP2
groups are configured. Notice how the names in the GROUPS
section match the group names specified in the RANGES
parameter in the ROUTING
section. This is critical for factory-based routing to work correctly. Furthermore, any change in the way groups are configured in an application must be reflected in the ROUTING
section. (Note that the Production sample packaged with the BEA Tuxedo software is configured to run entirely on one machine. However, you can easily configure this application to run on multiple machines.)
Listing 2-4 GROUPS Section of a UBBCONFIG File
*GROUPS
APP_GRP1
LMID = SITE1
GRPNO = 2
TMSNAME = TMS
APP_GRP2
LMID = SITE1
GRPNO = 3
TMSNAME = TMS
ORA_GRP1
LMID = SITE1
GRPNO = 4
OPENINFO = "ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SesTm=100+LogDir=.+MaxCur=5"
CLOSEINFO = ""
TMSNAME = "TMS_ORA"
ORA_GRP2
LMID = SITE1
GRPNO = 5
OPENINFO = "ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SesTm=100+LogDir=.+MaxCur=5"
CLOSEINFO = ""
TMSNAME = "TMS_ORA"
Factories implement factory-based routing in the way the invocation to the TP::create_object_reference()
operation is implemented. This operation has the C++ binding in Listing 2-5.
Listing 2-5 C++ Binding for create_object_reference
CORBA::Object_ptr TP::create_object_reference (
const char* interfaceName,
const PortableServer::oid &stroid,
CORBA::NVlist_ptr criteria);
The third parameter to this operation, criteria
, specifies a list of named values to be used for factory-based routing. To implement factory-based routing in a factory, you need to build the NVlist
. The use of factory-based routing is optional and is dependent on this argument. Instead of using factory-based routing, you can pass a value of 0
(zero) for this argument.
As stated previously, the RegistrarFactory
object in the Production sample application specifies the value STU_ID
. This value must exactly match the following information in the UBBCONFIG
file:
FACTORYROUTING
identifier in the INTERFACES
section.ROUTING
section.The RegistrarFactory
object inserts the student ID into the NVlist
using the code shown in Listing 2-6.
Listing 2-6 NVlist in the RegistrarFactory Object
// put the student id (which is the routing criteria)
// into a CORBA NVList:
CORBA::NVList_var v_criteria;
TP::orb()->create_list(1, v_criteria.out());
CORBA::Any any;
any <<= (CORBA::Long)student;
v_criteria->add_value("student_id", any, 0);
The RegistrarFactory
object has the invocation to the TP::create_object_reference()
operation, shown in Listing 2-7, passing the NVlist
created in Listing 2-6.
Listing 2-7 Invoking create_object_reference in the RegistrarFactory Object
// create the registrar object reference using
// the routing criteria :
CORBA::Object_var v_reg_oref =
TP::create_object_reference(
UniversityP::_tc_Registrar->id(),
object_id,
v_criteria.in()
);
The Production sample application also uses factory-based routing in the TellerFactory
object to determine the group in which Teller
objects should be instantiated based on an account number.
When you implement factory-based routing in a factory, BEA Tuxedo CORBA generates an object reference. The following example shows how the client application gets an object reference to a Registrar
object when factory-based routing is implemented.
RegistrarFactory
object, requesting a reference to a Registrar
object. The request includes a student ID.RegistrarFactory
invokes the TP::create_object_reference()
operation, passing the Registrar
interface name, a unique OID, and the NVlist
.When the client application subsequently invokes an object using the object reference, BEA Tuxedo CORBA routes the request to the group specified in the object reference.
Note: If you use the process-entity design pattern, you should use caution in how you implement factory-based routing. The object can service only those entities that are contained in the group's database.
This topic includes the following sections:
When designing the Registrar
and Teller
objects, you should ensure that:
Registrar
and Teller
objects work properly for the Production deployment environment; namely, across multiple replicated server processes and multiple groups. Given that the University and Billing server processes are replicated, the design must consider how these two objects should be instantiated.These objects must have unique object IDs (OIDs) and must be method-bound (that is, they must have the method
activation policy assigned to them).
In the University server applications that are less sophisticated than the Production sample application, the run-time behavior of the Registrar
and Teller
objects was simpler:
However, because the University and Billing server processes are now replicated, BEA Tuxedo CORBA must be able to differentiate among multiple instances of the Registrar
and Teller
objects. For example, if there are two University server processes running in a group, BEA Tuxedo CORBA must have a means to distinguish between the Registrar
object running in the first University server process and the Registrar
object running in the second University server process. To distinguish multiple instances of these objects, each object instance must be unique.
To make each Registrar
and Teller
object unique, the factories for those objects must change the way in which they make object references to them. For example, when the RegistrarFactory
object in the Basic sample application created an object reference to the Registrar
object, the TP::create_object_reference()
operation specified an OID that consisted only of the string registrar
. However, in the Production sample application, the same TP::create_object_reference()
operation uses a generated unique OID instead.
As a result of giving each Registrar
and Teller
object a unique OID, multiple instances of these objects may be running simultaneously in the BEA Tuxedo domain. This characteristic is typical of the stateless object model, and is an example of how the BEA Tuxedo domain can be highly scalable while it offers high performance.
Finally, because unique Registrar
and Teller
objects need to be brought into memory for each client request on them, it is critical that these objects be deactivated when the invocations on them are completed so that any object state associated with them does not remain idle in memory. The Production server application addresses this issue by assigning the method
activation policy to these two objects in the Implementation Configuration File (ICF).
The primary scalability advantage of using replicated server groups is being able to distribute processing across multiple machines. However, if your application interacts with a database, which is the case with the University sample applications, it is critical that you consider the impact of these multiple server groups on the database interactions.
In many cases, you may have one database associated with each machine in your deployment. If your server application is distributed across multiple machines, you must consider how you set up your databases.
The Production sample application, as described in this chapter, uses two databases. However, this application can easily be configured to accommodate more. The system administrator can decide on how many databases to use.
In the Production sample application, the student and account information is partitioned across the two databases, but course information is identical. Having identical course information in both databases is not a problem because the course information is read-only for the purposes of course registration. However, the student and account information is read-write. If multiple databases were also to contain identical data for students and accounts (that is, the database is not partitioned), the application would need to deal with the overhead of synchronizing the updates to student and account information across all the databases each time any student or account information were to change.
The Production sample application uses factory-based routing to send one set of requests to one machine, and another set to the other machine. How factory-based routing is implemented in the RegistrarFactory
object depends on the way in which references to Registrar
objects are created.
For example, when the client application sends a request to the RegistrarFactory
object to get an object reference to a Registrar
object, the client application includes a student ID in that request. The client application must use the object reference that the RegistrarFactory
object returns to make all subsequent invocations on a Registrar
object on a particular student's behalf, because the object reference returned by the factory is group-specific. Therefore, for example, when the client application subsequently invokes the get_student_details()
operation on the Registrar
object, the client application can be assured that the Registrar
object is active in the server group associated with the database containing data for that student.
To show how this works, consider the following execution scenario, which is implemented in the Production sample application:
find_registrar()
operation on the RegistrarFactory
object. Included in this invocation is the student ID 1000003
.RegistrarFactory
object uses the student ID to create an object reference to a Registrar
object in ORA_GRP1
, based on the routing information in the UBBCONFIG
file, and returns that object reference to the client application.The RegistrarFactory
object from the preceding scenario returns to the client application a unique reference to a Registrar
object that can be instantiated only in ORA_GRP1
, which runs on Production Machine 1 and has a database containing student data for students with IDs in the range 100001
to 100005
. Therefore, when the client application sends subsequent requests to this Registrar
object on behalf of a given student, the Registrar
object interacts with the correct database.
When the Registrar
object needs a Teller
object, the Registrar
object invokes the TellerFactory
object, using the TellerFactory
object reference cached in the University Server object.
However, because factory-based routing is used in the TellerFactory
object, the Registrar
object passes the student's account number when the Registrar
object requests a reference to a Teller
object. This way, the TellerFactory
object creates a reference to a Teller
object in the group that has the correct database.
Note: For the Production sample application to work properly, it is essential that the system administrator configures the server groups and the databases properly. In particular, the system administrator must make sure that a match exists between the routing criteria specified in the routing tables and the databases to which requests using those criteria are routed. Using the Production sample as an example, the database in a given group must contain the correct student and account information for the requests that are routed to that group.
In the future, the system administrator of the Production sample application may want to add capacity to the BEA Tuxedo domain. For example, the University may eventually experience a large increase in the student population, or the Production application may be scaled up to accommodate the course registration process for an entire state university system, encompassing several campuses. This can be done without modifying or rebuilding the application.
The system administrator can continually add capacity by:
The system administrator must modify the UBBCONFIG
file to specify the additional server groups, the server processes that run in those groups, and the machines on which the server groups run.
For example, instead of routing to the two existing groups in the Production sample application, the system administrator can modify the routing rules in the UBBCONFIG
file to partition the application further among additional server groups added to the BEA Tuxedo domain. Any modification to the routing tables must match the information for the configured server groups and machines in the UBBCONFIG
file.
Note: If you add capacity to an existing BEA Tuxedo CORBA application that uses a database, you must also consider the impact on how the database is set up, particularly when you are using factory-based routing. For example, if the Production sample application is distributed across six machines, the database on each machine must be set up appropriately and in accordance with the routing tables in the UBBCONFIG
file.