Running /Domain

What This Chapter is About

This chapter covers:

  1. /Domain run-time administrative commands and servers
  2. Transaction Management

Run-time Administration

/Domain integrates with an existing System/T application by adding entries for domain gateways groups and gateway servers in the TUXCONFIG file. The existing tmconfig(1) and tmadmin(1) commands can be used to add this new configuration dynamically to a running System/T application. The tmadmin command can also be used to list the information contained in the Bulletin Board for /Domain gateway groups and their gateways.

Therefore, in addition to tmadmin, a new administrative command, dmadmin(1) is provided. This command provides administrators with the run-time administration capabilities to maintain the /Domain configuration and active gateway groups. Figure 1 shows the relationship between administrative commands and servers.

Fig. 1: /Domain run-time administration

Figure 1 shows the components of the /Domain administrative subsystem: the dmadmin(1) command, the /Domain administrative server, DMADM(5), the gateway group administrative server, GWADM(5), and the /Domain configuration file, BDMCONFIG. The dmadmin command can be used by the administrator to update the BDMCONFIG file while the BEA TUXEDO application is running. The command acts as a front-end process that translates administrative commands to service requests to the DMADMIN service, which is a generic administrative service advertised by the DMADM server. The DMADMIN service invokes the validation, retrieval, or update functions provided within the DMADM server to maintain the BDMCONFIG file. The DMADM server also provides a registration service to gateway groups. This registration service is requested by GWADM servers as part of their initialization procedure. The registration service downloads the configuration information required by the requesting gateway group. The DMADM server maintains the list of registered gateway groups, and propagates to these groups any changes made to the configuration.

Figure 1 also shows some of the functionality provided by the GWADM server. This server registers with the DMADM server to obtain the configuration information used by the corresponding gateway group. The GWADM accepts queries from dmadmin to obtain run-time statistics or to change the run-time options of the corresponding gateway group. Periodically, the GWADM server sends an ``I-am-alive'' message to the DMADM server. If no reply is received from the DMADM server, the GWADM server registers again. This mechanism, makes sure the GWADM server always has the latest copy of the /Domain configuration for its group.

Another run-time feature is the capability to specify gateway parameters when a gateway group is booted. This capability requires the use of the CLOPT parameter when the GWADM server is defined in the *SERVERS section of the TUXCONFIG file.

The main capabilities of the the run-time administration feature are as follows:

The capabilities listed above are described in more detail later in this chapter.

Migration Restriction

The migration of a domain gateway group to another machine is possible. However, instantiations can only be migrated across machines of the same type.

Security Limitation

For security reasons, the domain administrative command, dmadmin, cannot be run from a workstation when /Domain is used with an older release of BEA TUXEDO. Older releases cannot distinguish application administrators from ordinary users accessing the system from a workstation.

dmadmin(1)

The dmadmin command is used by application administrators for the interactive administration of the information stored in the BDMCONFIG file and the different gateway groups running within a particular BEA TUXEDO application. dmadmin is used to obtain statistics or other information gathered by gateway groups, to change the gateway group parameters, and to add (or update) new information to the BDMCONFIG file.

dmadmin(1) is designed for extensibility. In future releases, new commands will be integrated with only minor changes to the software.

dmadmin is implemented as a front-end to the DMADM and GWADM servers. The communication between the two servers is done via FML typed buffers.

A copy of the dmadmin(1) manual page is included in the BEA TUXEDO Reference Manual: Section 1

Run-time Deletions

Run-time deletions to the BDMCONFIG file can be performed only when the changes do not involve an active gateway group.

DMADM(5)

The domain administrative server, DMADM(5) is a BEA TUXEDO-supplied server that provides run-time administration of the BDMCONFIG file. Its main function is to maintain the BDMCONFIG file and to support a list of registered gateway groups. The DMADM propagates run-time configuration changes to the registered gateway groups (see Figure 1).

The DMADM server advertises two services:

  1. DMADMIN, that is used by dmadmin and GWADM servers
  2. a service whose name is derived from the server identifier, DMADM_<svrid>. DMADM_<svrid> is used by registered GWADM servers for specific administrative functions (for example, to refresh the gateway group configuration information or to signal that a GWADM is still registered).

A copy of the DMADM(5) manual page is included in the BEA TUXEDO Reference Manual: Section 5

The DMADM server must be defined in the *SERVERS section of the TUXCONFIG file as a server running within a group (for example, DMADMGRP). There should be only one instance of the DMADM server in this group and it must be defined with no reply queue, that is, REPLYQ=N.

The administrator may define only one copy of the BDMCONFIG file per BEA TUXEDO application (that is, domain).

GWADM(5)

The gateway administrative server, GWADM(5) is a BEA TUXEDO-supplied server that provides administrative functions for a /Domain gateway group.

The main functions of the GWADM server are:

The GWADM server advertises two services;

  1. a generic service, GWADMIN,
  2. a service name based on the local domain name (the value of the LDOM keyword in the BDMCONFIG).

The dmadmin command uses the services to retrieve information from all active gateway groups or from a specific gateway group.

The GWADM server must be defined in the *SERVERS section of the TUXCONFIG file. It should not be part of the MSSQ used by the gateways associated with the group and it must not have a reply queue, that is, REPLYQ=N must be specified. It must be the first server booted within the gateway group; that is, it must either have a SEQUENCE number or it must be defined ahead of the gateway servers.

The GWADM server requires the existence of a DMADM server. Specifically, a DMADM server must be booted before that GWADM is booted.

A copy of the GWADM(5) manual page is included in the BEA TUXEDO Reference Manual: Section 5

The GWADM server must create the shared memory required by the gateway group to populate the configuration tables with information received from the DMADM server. The GWADM server uses IPC_PRIVATE with shmget and stores the ipckey returned in the shmid field of its registry entry in the Bulletin Board. Gateways can learn the ipckey by retrieving the GWADM registry entry and checking the shmid field.

dmgwopts(5)

/Domain gateways are special System/T servers defined in the TUXCONFIG file. Run-time options can be assigned to gateways as described in the dwgopts(5) manual page. A copy of the dwgopts(5) manual page is included in the BEA TUXEDO Reference Manual: Section 5

/Domain gateways should not be specified as members of a MSSQ set. They should not have a reply queue, REPLYQ=N; they may be restartable.

Transaction Management

Application programmers can request the execution of remote services within a transaction. Also, users at remote domains can request the execution of local services within a transaction. Therefore, /Domain coordinates the mapping of remote transactions to local transactions, and the sane termination (commitment or rollback) of these transactions.

BEA TUXEDO architecture uses a separate process (the TMS) to coordinate the commitment and the recovery of transaction branches accessing a particular group. In /Domain, however, this architecture would require extra messages from the gateway to the TMS server to process the commitment for incoming transactions. To simplify the /Domain architecture and to reduce the number of messages, the TMS code has been integrated with the gateway code.

Hence, /Domain gateways can process the transaction protocol used by System/T. System/T transaction protocol requires that the gateway group advertise the TMS service. This advertisement is done when the first gateway is booted. Once the TMS service is advertised, any transaction control messages directed to the gateway group are put on the gateways' MSSQ.

/Domain gateway groups should be defined in the TUXCONFIG file without the parameters TMSNAME, TMSCOUNT, OPENINFO and CLOSEINFO. These four parameters apply to groups that use an XA-compliant resource manager; /Domain gateways do not use the XA interface.

The commitment protocol across domains is strictly hierarchical. It is not possible to flatten the transaction tree because the structure of the transaction tree is not fully known at every domain, that is, a superior only knows its immediate subordinate domains. Flattening the tree would also require the root domain to be fully connected to all domains participating in the transaction.

The following sections describe the transaction management capabilities that are provided by domain gateways. These capabilities are:

TMS Functionality

In BEA TUXEDO, the TMS is a special server that is implicitly associated with server groups that use X/Open XA-compliant resource managers. The TMS server releases application servers from the delays associated with the distributed two-phase commitment protocol. TMSs coordinate the commitment of a transaction via special service requests to the TMS service, which is offered by all TMS servers.

In /Domain, gateways are not associated with an XA-compliant resource manager. The Transaction Processing Working Group (TPWG) of X/Open has proposed an advanced XA interface. System/T elected not to use it because the interface did not match the highly asynchronous and non-blocking model required by the gateway. While /Domain gateways do not use a separate TMS server, they do offer the TMS capability, which allows gateways to coordinate the two-phase commitment of transactions executed across domains. This is accomplished as follows:

  1. /Domain gateways advertise the TMS service and perform all operations associated with that service. Messages sent to this service are placed on the MSSQ used by the gateways and the gateways are able to process transaction management functions for transactions associated with the group.
  2. A gateway can act as a subordinate of transactions coordinated by another group within the domain. In this case, the gateway is a superior of the transaction branches executed in other remote domains (see Figure 3).

    Fig. 2: The gateway as subordinate/coordinator

  3. When acting as a subordinate of a transaction coordinated by a remote domain, the gateway also acts as the coordinator for all groups in the local domain accessed by the transaction (see Figure 3).
  4. Gateways manage transaction commitment for a particular client or for a server that uses the forwarding service with the AUTOTRAN capability. When this combination is used, the last server (the /Domain gateway) in the forward chain issues the commit and becomes the coordinator of the transaction; as a matter of fact, /Domain gateways always act as the last server in a forward chain (see Figure 4).

    Fig. 3: The Gateway managing commitment for a client

  5. Gateways automatically start/terminate transactions for remote services specified with the AUTOTRAN capability. This capability is required when the application administrator wants to enforce reliable network communication with remote services. Application administrators specify this capability by setting the AUTOTRAN, parameter to Y in the corresponding remote service definition (see the *DM_REMOTE_SERVICES section of the dmconfig(5) manual page in the BEA TUXEDO Reference Manual: Section 5
  6. Gateways map the BEA TUXEDO transaction protocol to the networking transaction protocol used for the interoperation with remote domains. It goes without saying, this mapping is instantiation-dependent.

READONLY Optimization

The READONLY optimization in the two-phase commitment protocol is supported only in the /TDOMAIN instantiation.

GTRID Mapping

A GTRID is a Global Transaction IDentifier. GTRID mapping defines how a transaction tree that crosses domain boundaries is constructed.

In System/T, a transaction tree is a two-level tree where the root is the group coordinating a global transaction and the nodes are other groups involved in the transaction. Each group performs its part of the global transaction independently from the parts done by other groups. Each group, therefore, implicitly defines a transaction branch. System/T, through Transaction Manager Servers, TMSs, coordinates the completion of the global transaction, making sure each branch completes.

Tightly-coupled or Loosely-coupled

In the X/Open DTP Model, a Transaction Manager, TM, can construct transaction trees by defining either tightly-coupled or loosely-coupled relationships with a Resource Manager, RM, by the way it interprets the transaction identifiers, XIDs used by the XA interface.

A "tightly-coupled relationship" is one where the same transaction identifier, XID, is used by all processes participating in the same global transaction and accessing the same RM. This relationship maximizes data sharing between processes; XA-compliant RMs expect to share locks for resources used by processes having the same XID. System/T achieves the tightly-coupled relationship via the group concept; that is, work done by a group on behalf of a given global transaction belongs to the same transaction branch; all the processes are given the same XID.

In a "loosely-coupled relationship," the TM generates a transaction branch for each part of the work in support of the global transaction. The RM handles each transaction branch separately; there is no sharing of data or of locks between the transaction branches. Deadlocks between transaction branches can occur. A deadlock results in the rollback of the global transaction. In System/T, when different groups participate in the same global transaction each group defines a transaction branch; this results in a loosely-coupled relationship.

Global Transactions Across Domains

The first difference between a global transaction in a single BEA TUXEDO application and global transactions across domains is that in the /Domain framework the transaction tree cannot be flattened to a two-level tree. There are two reasons for this:

  1. the transaction may involve more domains than can be known from the root domain (where the transaction is controlled), so the structure of the transaction tree cannot be fully known.
  2. the root domain would have to be directly connected to all domains in the transaction.

This means that the commitment protocol across domains must be hierarchical. Even a loop-back service request defines a new branch in the transaction tree.

Note: A loop-back request is one that goes to another domain and then comes back to be processed in the original domain. For example, domain A requests a service of domain B. The service in domain B requests another service in domain A, The transaction tree has two branches at the network level: a branch b1 from A to B and a branch b2 from B to A. Domain A cannot commit the work done on branch b2 before receiving commit instructions from B.

The structure of the transaction tree for global transactions across domains also depends on the distributed transaction processing protocol used by the particular /Domain instantiation. For example, in the OSI-TP protocol each dialogue (the OSI-TP word for a service request) is associated with a different transaction branch. In System/T, the OSI-TP instantiation uses a dialogue for each service request, so each service request is mapped to a separate transaction branch. The XAP-TP interface hides this mapping and provides a mechanism by which an entire OSI-TP sub-tree can be referenced by a user-defined identifier (In the System/T implementation, this identifier is the GTRID). The GTRID is used to instruct XAP-TP how a transaction tree must be constructed, that is which dialogues must be included within a given OSI-TP transaction. Therefore, from the System/T perspective a whole OSI-TP subtree can be managed as single transaction branch.

This property, however, only applies to outgoing service requests (that is, from the root domain to subordinate domains). It cannot be applied to incoming service requests. The OSI-TP instantiation consequently implements a loosely-coupled relationship; each incoming service request is mapped to a new System/T global transaction.

The /TDOMAIN instantiation tries to optimize GTRID mapping by implementing a tightly-coupled relationship. In /TDOMAIN service requests issued on behalf of the same global transaction are mapped to the same network transaction branch. Therefore, incoming service requests can be mapped to a single System/T transaction. However, the hierarchical structure of inter-domain communication and the inter-domain transaction tree must still be maintained.

The optimization that /TDOMAIN introduces applies only to a single domain. When two or more domains are involved in the transaction, the network transaction tree contains at least one branch per domain interaction. Hence, across domains, the network transaction tree remains loosely-coupled. There will be as many branches as there are domains involved in the transaction (even if all branches access the same resource manager instance).

/Domain gateway groups implement a loosely-coupled relationship because they generate different transaction branches for inter-domain transactions.

Figure 5 shows the service request graph for a client that generates three service requests, one local (r0) and two remote (r2 and r3). Request r0 goes to a local service (Svc0), which generates another remote service request (r1). Request r1 goes to remote service Rsvc1, which issues a loop-back service request r4 to local service Svc4. Svc0 and Svc4 execute in different groups (G0 and G4). The /Domain gateway executes within another group (GW), and the remote services Rscv1, Rsvc2, and Rsvc3 execute in another domain (domain B).

Fig. 4: Service request graph

For the service request graph shown above in Figure 5, Figure 6 shows the transaction tree for the OSI-TP instantiation and Figure 7 shows the transaction tree produced by the /TDOMAIN instantiation. The figures assume that both domains (A and B) are BEA TUXEDO applications. The OSITP instantiation is loosely-coupled because of the OSI-TP protocol. The transaction tree for this instantiation shows group G0 in Domain A coordinating the global transaction started by the client. Group G0 coordinates group GW. Requests r1, r2, and r4 are mapped each to an OSI-TP dialogue and therefore to an OSI-TP transaction branch. However, OSI-TP uses the XAP-TP feature that allows an entire OSI-TP transaction to be referred by a unique identifier (T1) and uses this identifier for requests r1, r2, and r3. It is up to XAP-TP to generate OSI-TP transaction identifiers and to construct the corresponding OSI-TP transaction tree. The generic part of /Domain only needs to map service requests r1, r2, and r3 to the T1 identifier. In domain B, OSI-TP uses the rule that new transaction branches must be mapped a new BEA TUXEDO transaction. Therefore, OSI-TP transaction branches r1, r2, and r3 get mapped to three different BEA TUXEDO transactions (the corresponding mapping is represented by identifiers T2, T3, and T4). The graph shows the gateway group GW in Domain B coordinating three BEA TUXEDO transactions on group G1. Finally, there is the loop-back service request r4 that generates another branch in the transaction tree. OSI-TP maps this request to identifier T2, but XAP-TP generates a new branch in its transaction tree (r4: B to A'). This is a new transaction branch on Domain A, and therefore, the gateway generates a new mapping T5 to a new BEA TUXEDO transaction. Hence, the transaction graph shows that gateway group GW on Domain A coordinates group G4.

Notice that the hierarchical nature of the OSI-TP protocol is fully enforced by these mappings. However, because these mappings introduce a loosely-coupled relationship, the probability for intra-transaction deadlock is increased (e.g., there are three BEA TUXEDO transactions accessing the RM represented by group G1).

Fig. 5: Transaction tree, /OSITP

The /TDOMAIN instantiation provides a tightly-coupled integration that solves this deadlock problem by minimizing the number of transaction branches required in the interoperation between two domains. The corresponding transaction tree is shown in Figure 7. Notice that the gateway still must perform mappings between a BEA TUXEDO transaction and a network transaction, and that the hierarchical nature of the communication between domains must be strictly enforced. The figure shows that requests r1, r2, and r3 are mapped to a single /TDOMAIN transaction branch. Therefore, on domain B only one BEA TUXEDO transaction needs to be generated; T2 represents this mapping and the graph shows gateway group GW on domain B coordinating group G1. Request r4 is mapped to identifier T2 on Domain B, but /TDOMAIN will generate a new branch in its transaction tree (r4: B to A'). This is a new transaction branch on Domain A, and therefore, the gateway generates a new mapping T3 to a new BEA TUXEDO transaction. The graph shows that gateway group GW on domain A also coordinates group G4. Hence, the hierarchical nature of inter-domain communication is fully enforced with this mapping: group G4 cannot commit before group G1.

Fig. 6: Transaction tree, /TDOMAIN

Summary of Transaction Management

The following statements can be made to sum up /Domain transaction management:

  1. Gateways generate mappings from a BEA TUXEDO transaction to a network transaction. A new mapping generated per BEA TUXEDO transaction or per incoming network transaction branch.
  2. Each instantiation handles its own representation of the network transaction tree. Instantiations observe the hierarchical nature of the inter-domain communication.

Logging

Logging is used to keep track of the progress of the two-phase commit protocol. The information stored in the log is used to make sure the transaction is completed in the event of a network failure or machine crash.

To ensure completion of transactions across domains, /Domain gateways log the mapping between local and remote identifiers. Along with this information, the /Domain transaction management facility records the decisions during the different phases of the commitment protocol, and the information about the different remote domains involved in the transaction. In the OSI-TP case, the XAP-TP interface logs the information required for the recovery of the OSI-TP protocol machine. The information is referred to as a blob, (for binary large object); it is kept in the same log record as the commit information to make recovery easier.

/Domain log records have a different structure from the log records stored in System/T's TLOG. The TLOG records are fixed size and are stored in a single page. /Domain log records are variable size; more than one page may be required to store the record. The /Domain logging mechanism, DMTLOG, has the ability to store variable size log records.

For the case where a TMS is the superior of a /Domain gateway group, the System/T TLOG is still required to coordinate the commitment

How Logging Works

The logging functionality is provided by the GWADM administrative server. Each /Domain gateway group has its own log. The specification of the DMTLOG is included in the /Domain configuration file, DMCONFIG, for each local domain by means of the required parameter DMTLOGDEV=string and the two optional parameters, DMTLOGNAME=identifier and DMTLOGSIZE=numeric in the *DM_LOCAL_DOMAINS section. (See the dmconfig(5) manual page in the BEA TUXEDO Reference Manual: Section 5 The administrator can optionally create a DMTLOG with the run-time administration facility (see the dmadmin(1) manual page in the BEA TUXEDO Reference Manual: Section 1

If the DMTLOG has not been created when a /Domain gateway group is booted, the distinguished gateway automatically creates the log. The information for this automatic creation is extracted from the BDMCONFIG file.

Until the logging device is specified in the BDMCONFIG file the /Domain gateway group cannot process requests in transaction mode and the gateway group cannot offer the TMS service.

To coordinate the commit protocol, /Domain gateways require the following two log records:

When the transaction has been committed at all nodes, these log records of the transaction are removed.

In the OSI-TP protocol, two types of heuristic records are logged:

Heuristic log records persist until they are explicitly removed by the administrator. This persistence is required to provide the right information during recovery after a crash, and to provide diagnostic information for administrators.

The administrator uses the forgettran command of dmadmin(1) to remove heuristic records when they are no longer needed.

Migration

Migration is possible only between machines of the same type.

Recovery

When a /Domain gateway group is booted, the distinguished gateway performs an automatic warm-start of the DMTLOG. The warm-start includes scanning the log to see if any transactions were not completed. If uncompleted transactions are found, action is taken to move them to completion.

In OSI-TP any blobs stored in the DMTLOG with a transaction record are passed to the network access module. which uses the blobs to reconstruct its internal state and to recover any failed connection.

In the case of heuristic decisions, if the /Domain gateway group is a subordinate of a local TMS and the heuristic decision has been indicated, the TMS generates a TMS_STATUS message to learn the final decision.