Configuring Oracle Clusterware Management With the cluster.oracle.ini File
The information in the cluster.oracle.ini
file is used to create
Oracle Clusterware resources that manage TimesTen databases, TimesTen processes, user
applications, and virtual IP addresses. Create an initialization file called
cluster.oracle.ini
as a text file.
Note:
See TimesTen Configuration Attributes for Oracle Clusterware for details on all of the attributes that can be used in the cluster.oracle.ini
file.
The ttCWAdmin -create
command reads this file for configuration
information, so the location of the text file must be reachable and readable by
ttCWAdmin
. The ttCWAdmin
utility is used to
administer TimesTen active standby pairs in a cluster that is managed by Oracle
Clusterware.
It is recommended that you place this file in the TimesTen daemon home directory on the host for the active database. However, you can place this file in any directory or shared drive on the same host as where you run the ttCWAdmin -create
command.
The default location for this file is in the timesten_home
/conf
directory. If you place this file in another location, identify the path of the location with the -ttclusterini
option.
The entry name in the cluster.oracle.ini
file must be the same as
an existing system DSN in the sys.odbc.ini
file. For example,
[basicDSN]
is the entry name in the
cluster.oracle.ini
file described in Configuring Basic Availability. [basicDSN]
must also be the
DataStore
and Data Source Name
data store
attributes in the sys.odbc.ini
files on each host. For example, the
sys.odbc.ini
file for the basicDSN
DSN on
host1
might be:
[basicDSN] DataStore=/path1/basicDSN LogDir=/path1/log DatabaseCharacterSet=AL32UTF8 ConnectionCharacterSet=AL32UTF8
The sys.odbc.ini
file for basicDSN
on host2
can have a different path, but all other attributes should be the same:
[basicDSN] DataStore=/path2/basicDSN LogDir=/path2/log DatabaseCharacterSet=AL32UTF8 ConnectionCharacterSet=AL32UTF8
The following sections demonstrate sample configurations of the cluster.oracle.ini
file:
Configuring Basic Availability
This example shows an active standby pair with no subscribers.
The host for the active database is the first MasterHost
defined (host1
) and the standby database is the second
MasterHost
in the list (host2
). Each host in the
list is delimited by commas. You can include spaces for readability, if desired.
[basicDSN] MasterHosts=host1,host2
The following is an example of a cluster.oracle.ini
file for an
active standby pair with one subscriber on host3
:
[basicSubscriberDSN] MasterHosts=host1,host2 SubscriberHosts=host3
Configuring Advanced Availability
Advanced availability involves configuring spare master or subscriber hosts that are idle until needed to replace master or subscriber hosts (used in the active standby pair replication scheme) that either shut down unexpectedly or experience an unrecoverable error.
As mentioned in Configuring Basic Availability, the MasterHosts
attribute in the cluster.oracle.ini
file configures the hosts that are
used as the master nodes. For an active standby pair replication scheme, you only need
two master hosts (one to become the active and one to become the standby). In the event
of a failure, the host that did not fail becomes the active (if not already the active)
and the failed host is recovered and becomes the standby. However, if the failed host
cannot be recovered and if you specified more than two hosts as master hosts in the
cluster.oracle.ini
file, then the next master host in the list can
be instantiated to take the place of an unrecoverable master host.
For example, the following shows a configuration of several master hosts. The first two master hosts (host1
and host2
) become the active and the standby; the latter two master hosts (host3
and host4
) can be used to take the place of either host1
or host2
if either encounter an unrecoverable failure.
MasterHosts=host1,host2,host3,host4
When you configure more than two multiple hosts, you should also configure two virtual IP (VIP) addresses used only by Oracle Clusterware resources that manage TimesTen resources. With these VIP addresses, TimesTen internal processes (those that manage replication) are isolated from any master host changes that may occur because of an unrecoverable host error.
Note:
As described in Create the Oracle Clusterware Resources to Manage Virtual IP Addresses, the Oracle Clusterware resource that manage these
VIP addresses (used in advanced availability) are created with the ttCWAdmin
-createVIPs
command.
These VIP addresses must be different from any other VIP addresses defined for Oracle Clusterware use or any VIP addresses that are to be used by user applications. Furthermore, if an application does use these VIP addresses, then the application may encounter errors when a master host fails (either recoverable or unrecoverable). These VIP addresses cannot be used by a user application as a method for client failover or as a method to isolate themselves if an active database and standby database switch.
Specify two VIP addresses in the MasterVIP
parameter, one for each master host in the active standby pair replication scheme. The VIP addresses specified for the TimesTen cluster must be different from any VIP addresses already defined and used by Oracle Clusterware. In particular, the VIP addresses that are created during the Oracle Clusterware install cannot be used with TimesTen.
MasterVIP=192.168.1.1, 192.168.1.2
The following parameters are also associated with advanced availability in the cluster.oracle.ini
file:
-
SubscriberHosts
, similar toMasterHosts
lists the host names that can contain subscriber databases. -
SubscriberVIP
, similar toMasterVIP
, provides VIP addresses that can be used by TimesTen internally to manage a subscriber node. -
VIPInterface
is the name of the public network adaptor. -
VIPNetMask
defines the netmask of the virtual IP addresses.
In the following example, the hosts for the active database and the standby database are host1
and host2
. The hosts available for instantiation in case of an unrecoverable error are host3
and host4
. There are no subscriber nodes. VIPInterface
is the name of the public network adaptor. VIPNetMask
defines the netmask of the virtual IP addresses.
[advancedDSN] MasterHosts=host1,host2,host3,host4 MasterVIP=192.168.1.1, 192.168.1.2 VIPInterface=eth0 VIPNetMask=255.255.255.0
The following example configures a single subscriber on host4
. There is one extra host defined in SubscriberHosts
that can be used for failover of the master databases and one extra node that can be used for failover of the subscriber database. MasterVIP
and SubscriberVIP
specify the virtual IP addresses defined for the master and subscriber hosts.
[advancedSubscriberDSN] MasterHosts=host1,host2,host3 SubscriberHosts=host4,host5 MasterVIP=192.168.1.1, 192.168.1.2 SubscriberVIP=192.168.1.3 VIPInterface=eth0 VIPNetMask=255.255.255.0
Ensure that the extra master nodes:
-
Have TimesTen installed
-
Have the direct mode application installed if this is part of the configuration. See Implementing Application Failover.
Including Cache Groups in the Active Standby Pair
If the active standby pair replicates one or more AWT or read-only cache groups, set the CacheConnect
attribute to y
.
This example sets the CacheConnect
attribute to y
. The example specifies an active standby pair with one subscriber in an advanced availability configuration. The active standby pair replicates one or more cache groups.
[advancedCacheDSN] MasterHosts=host1,host2,host3 SubscriberHosts=host4, host5 MasterVIP=192.168.1.1, 192.168.1.2 SubscriberVIP=192.168.1.3 VIPInterface=eth0 VIPNetMask=255.255.255.0 CacheConnect=y
Implementing Application Failover
TimesTen integration with Oracle Clusterware can facilitate the failover of a TimesTen application that is linked to any of the databases in the active standby pair.
TimesTen can manage both direct and client/server mode applications that are on the same host as Oracle Clusterware and TimesTen.
The required attributes in the cluster.oracle.ini
file for failing over a TimesTen application are as follows:
-
AppName
- Name of the application to be managed by Oracle Clusterware -
AppStartCmd
- Command line for starting the application -
AppStopCmd
- Command line for stopping the application -
AppCheckCmd
- Command line for running an application that checks the status of the application specified byAppName
-
AppType
- Determines the database to which the application is linked. The possible values areActive
,Standby
,DualMaster
,Subscriber (all)
andSubscriber
[
index
]
.
There are also several optional attributes that you can configure, such as AppFailureThreshold
, DatabaseFailoverDelay
, and AppScriptTimeout
. Table A-3 lists and describes all optional attributes and their default values.
The TimesTen application monitor process uses the user-supplied script or program specified by AppCheckCmd
to monitor the application. The script that checks the status of the application must be written to return 0
for success and a nonzero number for failure. When Oracle Clusterware detects a nonzero value, it takes action to recover the failed application.
This example shows advanced availability configured for an active standby pair with
no subscribers. The reader
application is an application that queries
the data in the standby database. AppStartCmd
,
AppStopCmd
and AppCheckCmd
can include arguments
such as start
, stop
and check
commands.
Note:
Do not use quotes in the values for AppStartCmd
, AppStopCmd
and AppCheckCmd
.
[appDSN] MasterHosts=host1,host2,host3,host4 MasterVIP=192.168.1.1, 192.168.1.2 VIPInterface=eth0 VIPNetMask=255.255.255.0 AppName=reader AppType=Standby AppStartCmd=/mycluster/reader/app_start.sh start AppStopCmd=/mycluster/reader/app_stop.sh stop AppCheckCmd=/mycluster/reader/app_check.sh check
You can configure failover for more than one application. Use AppName
to name the application and provide values for AppType
, AppStartCmd
, AppStopCmd
and AppCheckCmd
immediately following the AppName
attribute. You can include blank lines for readability. For example:
[app2DSN] MasterHosts=host1,host2,host3,host4 MasterVIP=192.168.1.1, 192.168.1.2 VIPInterface=eth0 VIPNetMask=255.255.255.0 AppName=reader AppType=Standby AppStartCmd=/mycluster/reader/app_start.sh AppStopCmd=/mycluster/reader/app_stop.sh AppCheckCmd=/mycluster/reader/app_check.sh AppName=update AppType=Active AppStartCmd=/mycluster/update/app2_start.sh AppStopCmd=/mycluster/update/app2_stop.sh AppCheckCmd=/mycluster/update/app2_check.sh
If you set AppType
to DualMaster
, the application starts on both the active and the standby hosts. The failure of the application on the active host causes the active database and all other applications on the host to fail over to the standby host. You can configure the failure interval, the number of restart attempts, and the uptime threshold by setting the AppFailureInterval
, AppRestartAttempts
and AppUptimeThreshold
attributes. These attributes have default values. For example:
[appDualDSN] MasterHosts=host1,host2,host3,host4 MasterVIP=192.168.1.1, 192.168.1.2 VIPInterface=eth0 VIPNetMask=255.255.255.0 AppName=update AppType=DualMaster AppStartCmd=/mycluster/update/app2_start.sh AppStopCmd=/mycluster/update/app2_stop.sh AppCheckCmd=/mycluster/update/app2_check.sh AppRestartAttempts=5 AppUptimeThreshold=300 AppFailureInterval=30
Configuring for Recovery When Both Master Nodes Permanently Fail
If both master nodes fail and then come back up, Oracle Clusterware can automatically recover the master databases.
Automatic recovery of a temporary dual failure requires the following:
-
RETURN TWOSAFE
is not specified for the active standby pair. -
AutoRecover
is set toy
. -
RepBackupDir
specifies a directory on shared storage. -
RepBackupPeriod
is set to a value greater than0
.
If both master nodes fail permanently, Oracle Clusterware can automatically recover the master databases to two new nodes if the following is true:
-
Advanced availability is configured (virtual IP addresses and at least four hosts).
-
The active standby pair does not replicate cache groups.
-
RETURN TWOSAFE
is not specified. -
AutoRecover
is set toy
. -
RepBackupDir
specifies a directory on shared storage. -
RepBackupPeriod
must be set to a value greater than0
.
TimesTen first performs a full backup of the active database and then performs incremental backups. You can specify the optional attribute RepFullBackupCycle
to manage when TimesTen performs subsequent full backup. By default, TimesTen performs a full backup after every five incremental backups.
If RepBackupDir
and RepBackupPeriod
are configured
for backups, TimesTen performs backups for any master database that becomes active. It
does not delete backups that were performed for a database that used to be the active
and has become the standby unless the database becomes the active again. Ensure that the
shared storage has enough space for two complete database backups. The ttCWAdmin
-restore
command automatically chooses the correct backup files.
Incremental backups increase the amount of log records in the transaction log files. Ensure that the values of RepBackupPeriod
and RepFullBackupCycle
are small enough to prevent a large amount of log records in the transaction log file.
This example shows attribute settings for automatic recovery.
[autorecoveryDSN]
MasterHosts=host1,host2,host3,host4
MasterVIP=192.168.1.1, 192.168.1.2
VIPInterface=eth0
VIPNetMask=255.255.255.0
AutoRecover=y
RepBackupDir=/shared_drive/dsbackup
RepBackupPeriod=3600
If you have cache groups in the active standby pair or prefer to recover manually
from failure of both master hosts, ensure that AutoRecover
is set to
n
(the default). Manual recovery requires the following:
-
RepBackupDir
specifies a directory on shared storage -
RepBackupPeriod
must be set to a value greater than0
This example shows attribute settings for manual recovery. The default value for AutoRecover
is n
, so it is not included in the file.
[manrecoveryDSN]
MasterHosts=host1,host2,host3
MasterVIP=192.168.1.1, 192.168.1.2
VIPInterface=eth0
VIPNetMask=255.255.255.0
RepBackupDir=/shared_drive/dsbackup
RepBackupPeriod=3600
Using the RepDDL Attribute
The RepDDL
attribute represents the SQL statement that creates the active standby pair.
The RepDDL
attribute is optional. You can use it to exclude tables, cache groups and sequences from the active standby pair.
If you include RepDDL
in the cluster.oracle.ini
file, do not specify ReturnServiceAttribute
, MasterStoreAttribute
or SubscriberStoreAttribute
in the cluster.oracle.ini
file. Include those replication settings in the RepDDL
attribute.
When you specify a value for RepDDL
, use the <DSN>
macro for the database file name prefix. Use the <MASTERHOST[1]>
and <MASTERHOST[2]>
macros to specify the master host names. TimesTen substitutes the correct values from the MasterHosts
or MasterVIP
attributes, depending on whether your configuration uses virtual IP addresses. Similarly, use the <SUBSCRIBERHOST[
n
]>
macro to specify subscriber host names, where n
is a number from 1 to the total number of SubscriberHosts
attribute values or 1 to the total number of SubscriberVIP
attribute values if virtual IP addresses are used.
Use the RepDDL
attribute to exclude tables, cache groups, and
sequences from the active standby pair:
[excludeDSN] MasterHosts=host1,host2,host3,host4 SubscriberHosts=host5,host6 MasterVIP=192.168.1.1, 192.168.1.2 SubscriberVIP=192.168.1.3 VIPInterface=eth0 VIPNetMask=255.255.255.0 RepDDL=CREATE ACTIVE STANDBY PAIR \ <DSN> ON <MASTERHOST[1]>, <DSN> ON <MASTERHOST[2]> SUBSCRIBER <DSN> ON <SUBSCRIBERHOST[1]>\ EXCLUDE TABLE pat.salaries, \ EXCLUDE CACHE GROUP terry.salupdate, \ EXCLUDE SEQUENCE ttuser.empcount
The replication agent transmitter obtains route information as follows, in order of priority:
-
From the
ROUTE
clause in theRepDDL
setting, if aROUTE
clause is specified. Do not specify aROUTE
clause if you are configuring advanced availability. -
From Oracle Clusterware, which provides the private host names and public host names of the local and remote hosts as well as the remote daemon port number. The private host name is preferred over the public host name. If the replication agent transmitter cannot connect to the IPC socket, it attempts to connect to the remote daemon using information that Oracle Clusterware maintains about the replication scheme.
-
From the active and standby hosts. If they fail, then the replication agent chooses the connection method based on host name.
This is an example of specifying the ROUTE
clause in RepDDL
:
[routeDSN] MasterHosts=host1,host2,host3,host4 RepDDL=CREATE ACTIVE STANDBY PAIR \ <DSN> ON <MASTERHOST[1]>, <DSN> ON <MASTERHOST[2]>\ ROUTE MASTER <DSN> ON <MASTERHOST[1]> SUBSCRIBER <DSN> ON <MASTERHOST[2]>\ MASTERIP "192.168.1.2" PRIORITY 1\ SUBSCRIBERIP "192.168.1.3" PRIORITY 1\ MASTERIP "10.0.0.1" PRIORITY 2\ SUBSCRIBERIP "10.0.0.2" PRIORITY 2\ MASTERIP "140.87.11.203" PRIORITY 3\ SUBSCRIBERIP "140.87.11.204" PRIORITY 3\ ROUTE MASTER <DSN> ON <MASTERHOST[2]> SUBSCRIBER <DSN> ON <MASTERHOST[1]>\ MASTERIP "192.168.1.3" PRIORITY 1\ SUBSCRIBERIP "192.168.1.2" PRIORITY 1\ MASTERIP "10.0.0.2" PRIORITY 2\ SUBSCRIBERIP "10.0.0.1" PRIORITY 2\ MASTERIP "140.87.11.204" PRIORITY 3\ SUBSCRIBERIP "140.87.11.203" PRIORITY 3\