Oracle RAC and Disaster Recovery
Oracle Real Application Clusters (RAC) can provide database availability within an NMS site. Oracle Data Guard provides replication and database role transitions between sites. A RAC database has multiple database instances but represents one database in the Data Guard configuration. Configure each NMS site against the appropriate database, rather than creating an NMS site for each RAC instance.
RAC does not make the NMS Services tier active on multiple nodes at once. Keep one active NMS Services instance for each operational NMS instance. Alternate application nodes can run warm services as described in
Warm Services.
Preparing RAC for NMS Disaster Recovery
1. Have the database administrator configure RAC and Data Guard for the supported Oracle Database release before configuring NMS recovery. Configure the broker's shared files and server parameter file for RAC, start the broker on the participating databases, and verify replication between the primary and physical standby databases. Use the Oracle Database documentation for the installed release for the database installation and broker configuration commands.
2. Define application database services and connection identifiers separately from the broker connection identifiers. For NMS application connections, use SCAN addresses and SERVICE_NAME rather than pinning the connection to one RAC instance SID. Configure service placement and availability for the intended database roles. The write service must be available when that database is primary. If warm services will run against a physical standby, provide a connection that can query the standby while it is open for read access.
3. Provide a DGMGRL connect identifier for each site's database. It must work from the NMS Services hosts that will execute recovery and remain usable for its intended database roles. Configure the broker's DGConnectIdentifier to reach the RAC instances, including connect-time failover, according to the broker requirements. Keep this distinct from the role-managed application service. Database restart connectivity is a separate broker requirement: validate the applicable Clusterware or static-listener setup rather than assuming an application service or a SCAN connection replaces it.
4. Configure the NMS administrative environments, site-specific
CES_PARAMETERS, passwordless SSH, Oracle Wallet, and WebLogic startup configuration described in
Prerequisites.
RDBMS_HOST and
RDBMS_ADMIN identify the NMS schema connections; they are not the database name used in the broker configuration.
5. Run nms-ha-config init in the initial administrative environment. Supply the DGMGRL connect identifier, credentials for a user with SYSDG privilege, the database selected from the broker configuration, and the NMS Services hostname. Associate each additional NMS site with its broker database name and DGMGRL connect identifier. Use the database name reported by DGMGRL show configuration, not an individual RAC instance name or SID. The NMS Services hostname is the application host used to stop and synchronize NMS Services, not the RAC SCAN hostname.
6. Copy $NMS_HOME/etc/nms_sites.cfg to each administrative environment at each site. In each environment, populate the local Oracle Wallet entries by running:
nms-ha-config update --wallet $NMS_SITE_NAME
When configuration already exists, use nms-ha-config update to change a site's database name, NMS Services hostname, or DGMGRL connect identifier instead of repeating init. For example:
nms-ha-config update --dbname <broker_database_name> --connect <dgmgrl_connect_identifier> --wallet <site_name>
7. Configure WebLogic application data sources for the RAC application service. The
nms-wls-config JDBC configuration provides
oracle_host,
oracle_port, and
oracle_service_name; its
db_url option can specify a complete JDBC descriptor. If using GridLink, also configure
grid_link and the appropriate Oracle Notification Service connection. Continue to configure
oracle_host when
db_url is supplied, because
nms-wls-config uses it to build the ONS node list. Configure NMS Monitor data sources separately so that each can identify and query its own site's database, including the standby database; see
Create WebLogic Cluster below.
Validation Before Production Use
From each NMS administrative environment, verify that the configured DGMGRL connect identifiers and wallet credentials can access all configured databases and that broker validation reports the intended standby ready for the requested operation. Confirm that all NMS application connections and site-specific monitoring connections reach the intended database and schema.
Validate RAC instance recovery separately from NMS site recovery. In a non-production environment matching the production topology, test loss or relocation of a RAC instance and verify recovery of the NMS database connections without changing the NMS site unnecessarily. Then test a planned site switchover and switchback using nms-ha-switchover. Confirm database roles, current NMS site, file synchronization, service state, WebLogic connectivity, and integrations. Test the nms-ha-failover procedure separately under the project's recovery plan. Recovery time and connection behavior must be measured for the deployed topology.