Skip Headers
Oracle® Enterprise Manager Cloud Control Administrator's Guide
12c Release 3 (12.1.0.3)

E24473-27
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

D Standby OMSs using Standby WebLogic Domain

The following standby OMS implementation is used for Enterprise Manager Release 12.1.0.2 and earlier installations. For Enterprise Manager Release 12.1.0.3 and later, see Chapter 30, "Enterprise Manager Disaster Recovery."

IMPORTANT:

Standby OMSs using Standby WebLogic Domain are still supported but have been deprecated and may be desupported in a future release (see My Oracle Support Note 1563541.1 for details).

D.1 Configuring Standby Management Services on a Standby Site

Consider the following before installing the standby Management Services.

Oracle recommends that this activity be done during a lean period or during a planned maintenance window. When new Oracle Management Service instances are installed on the standby site, they are initially configured to connect to the Management Repository database on the primary site. Some workload will be taken up by the new Management Service. This could result in temporary loss in performance if the standby site Management Services are located far away from the primary site Management Repository database. However there would be no data loss and the performance would recover once the standby Management Services are shutdown post configuration.

Prerequisites

  • The primary site must be configured as per Cloud Control MAA guidelines described in previous sections. This includes Management Services fronted by an SLB and all Management Agents configured to upload to Management Services by the SLB.

  • The standby site must be similar to the primary site in terms of hardware and network resources to ensure there is no loss of performance when failover happens.

  • Configure storage used by the software library to be replicated at the primary and standby site. In the event of a site outage, the contents of this storage must be made available on the standby site using hardware vendor disk level replication technologies.

  • The shared storage used for the software library must be made available on the standby site using the same paths as the primary site.

  • For complete redundancy in a disaster recovery environment, a second load balancer must be installed at the standby site. The secondary SLB must be configured in the same fashion as the primary. Some SLB vendors (such as F5 Networks) offer additional services that can be used to pass control of the Virtual IP presented by the SLB on the primary site to the SLB on the standby site in the event of a site outage. This can be used to facilitate automatic switching of Management Agent traffic from the primary site to the standby site.

D.1.1 Installing the First Standby Management Service

Install the first standby Management Service using the following steps:

  1. Copy the emkey to the Management Repository by running the following command on the first Management Service on the primary site:

    emctl config emkey -copy_to_repos

  2. Export the configuration from the first Management Service on the primary site using:

    emctl exportconfig oms -dir <location for the export file>

    After the configuration is exported, do not make any configuration changes to the primary site till the standby management service is configured.

  3. Install a Management Agent on the standby host if one does not already exist.

  4. Perform a software-only install of the Enterprise Manager software using a modified version of the ”Add Management Service” Deployment Procedure.

    1. From the Enterprise menu, select Provisioning and Patching, then select Procedure Library.

    2. Select Add Management Service procedure and click Create Like.

    3. Go to the Procedure Steps tab and select and disable the steps - ”Configure Management Service”, ”Targets Discovery” and ”Post Configuration Tasks”.

    4. Save the modified deployment procedure and use it to install the Enterprise Manager software on the standby OMS host.

    5. After the Deployment Procedure completes, delete the file emInstanceMapping.properties from <OMS Oracle Home>/sysman/config on the standby OMS host.

  5. Configure the Management Service by running omsca in standby mode. Choose a different domain name for the standby. For example, if the primary WebLogic domain is GCDomain, choose GCDomainStby.

    omsca standby –EM_DOMAIN_NAME GCDomainStby –NM_USER nodemanager -AS_USERNAME weblogic –nostart

    When prompted for the Administration Server host and EM Instance host, enter the standby OMS hostname (or accept the default).

    When prompted for the passwords, provide the same passwords as the primary site.

    When prompted for Management Repository details, provide the primary database details.

  6. Configure the required plug-ins by running the following command:

    pluginca -action deploy -isFirstOMS true -plugins <plugin-list> -oracleHome <oms oracle home> -middlewareHome <wls middleware home>

    where plugin-list is the list of plug-ins returned by the SQL query

    SELECT epv.plugin_id, epv.version FROM em_plugin_version epv, em_current_deployed_plugin ecp WHERE epv.plugin_type NOT IN ( 'BUILT_IN_TARGET_TYPE' , 'INSTALL_HOME') AND ecp.dest_type='2' AND epv.plugin_version_id = ecp.plugin_version_id;

    and is a comma separate list in the following format: <plugin-id>=<plugin-version>,<plugin-id>=<plugin-version>,…

    Example: "oracle.sysman.empa=12.1.0.1.0,oracle.sysman.mos=12.1.0.1.0,oracle.sysman.emas=12.1.0.1.0,oracle.sysman.emfa=12.1.0.1.0,oracle.sysman.db=12.1.0.1.0,oracle.sysman.emct=12.1.0.1.0,oracle.sysman.vt=12.1.0.1.0,oracle.sysman.ssa=12.1.0.1.0"

  7. Copy over the configuration exported from the primary Management Service in step 2 above to the standby Management Service host. Import the exported configuration on the standby Management Service using:

    emctl importconfig oms -file <full path of the export file>

    Note this command emits a warning about a failed export and prompts for confirmation to proceed. The warning can be ignored by entering "y" to proceed.

    Note this command will start the Management Service.

  8. Stop the Management Service but leave the Administration Server running using:

    emctl stop oms

  9. Add the standby Weblogic Domain and associated targets:

    The standby Weblogic Domain and associated targets can be added using the Guided Discovery process.

    From the Setup menu, select Add Target , then select Add Targets Manually. Select Oracle Fusion Middleware from the Target Types menu. Use the secure port (typically 7101) and, under Advanced, set the JMX Protocol to t3s.

    Note that the WebLogic targets, except the Administration Server, will be shown as down as the standby OMS is down at this stage.

  10. If you have Single Sign On configured on the primary site, follow the same steps to configure SSO on the standby OMS.

  11. If you have Real User Experience Insight, AD4J Manager or BI Publisher configured on the primary site, follow the same steps to configure them on the standby OMS.

D.1.2 Installing the First Standby Management Service (Oracle Installer, Software Only Mode)

  1. Copy the emkey to the Management Repository by running the following command on the first OMS on the primary site:

    emctl config emkey -copy_to_repos
    
  2. Export the configuration from the first OMS on the primary site.

    emctl exportconfig oms -dir <location for the export file>
    

    After the configuration is exported, do not make any configuration changes to the primary site until the standby OMS is configured.

  3. On the remote host, perform a software-only installation of the standby OMS as described in the Oracle® Enterprise Manager Cloud Control Advanced Installation and Configuration Guide chapter "Installing Enterprise Manager Software Now and Configuring Later" (Installing in Graphical Mode). Ensure that you install the software binaries in the same middleware location as that of the primary OMS.

    Note:

    Make sure you select same set of plug-ins as the primary OMS.

    If you have installed extra plug-ins which are not part of the installation, then complete the following steps.

    1. Connect to repository and execute the following query to obtain the list of installed plug-ins:

      SELECT epv.plugin_id "plugin id", epv.version "version", epv.rev_version "revision", epv.su_entity_id "update id"
      FROM em_plugin_version epv, em_current_deployed_plugin ecp
      WHERE epv.plugin_type NOT IN ('BUILT_IN_TARGET_TYPE', 'INSTALL_HOME')
      AND ecp.dest_type='2'
      AND epv.plugin_version_id = ecp.plugin_version_id;
      

      Write down the returned list of plug-ins installed.

    2. Extract the plug-in archives from the primary site.

      emcli export_update -id=<update id> -host=<standby OMS host> -dir=<directory to export archives> <host credential options>
      

      where <update id> is the value returned by the four column of query above.

      Note that this command generates a file with a .zip extension. Rename the file from <file_name>.zip to <file_name>.opar .

      Repeat above steps for all plugins. In standby OMS host, place all .opar files in one directory <plugin dir >.

    3. Invoke runInstaller and choose software-only install. This will install only the middleware home and OMS Oracle homes.

    4. To install the required plug-ins, run the plug-in install script.

      <OMS_HOME>/sysman/install/PluginInstall.sh -pluginLocation <plugin dir>
      

      where <plugin dir> is the directory where you placed extra plug-ins in step two above. Running this command displays a screen that will let you select the plug-ins to install. Verify that all the plug-ins listed in step 1 appear and that you select them. If there is a mismatch, OMS configuration may fail at a later step.

  4. On the software-only installation, apply all the patches you applied on the primary OMS so that both the primary and standby OMSs are identical and are in sync.

  5. Configure the OMS by running omsca in standby mode. Choose a different domain name for the standby. For example, if the primary WebLogic domain is GCDomain, choose GCDomainStby.

    omsca standby -EM_DOMAIN_NAME GCDomainStby -NM_USER nodemanager -AS_USERNAME weblogic -nostart
    

    When prompted for the Administration Server host and Enterprise Manager Instance host, enter the standby OMS hostname (or accept the default).

    When prompted for the passwords, provide the same passwords as the primary site.

    When prompted for Management Repository details, provide the primary database details.

  6. Configure the required plug-ins by running the following command:

    pluginca -action deploy -isFirstOMS true -plugins <plug-in list> -oracleHome <oms oracle home> -middlewareHome <wls middleware home>
    

    where plug-in list is the list of plug-ins returned by the SQL query.

    SELECT epv.plugin_id, epv.version FROM em_plugin_version epv, em_current_deployed_plugin ecp WHERE epv.plugin_type NOT IN ( 'BUILT_IN_TARGET_TYPE' , 'INSTALL_HOME') AND ecp.dest_type='2' AND epv.plugin_version_id = ecp.plugin_version_id;
    

    The plug-in list must be a comma separate list in the following format:

    <plugin-id>=<plugin-version>,<plugin-id>=<plugin-version>,…

    Example:

    "oracle.sysman.empa=12.1.0.1.0,oracle.sysman.mos=12.1.0.1.0,oracle.sysman.emas=12.1.0.1.0,oracle.sysman.emfa=12.1.0.1.0,oracle.sysman.db=12.1.0.1.0,oracle.sysman.emct=12.1.0.1.0,oracle.sysman.vt=12.1.0.1.0,oracle.sysman.ssa=12.1.0.1.0"
    
  7. Copy over the configuration exported from the primary OMS in step 2 above to the standby OMS host. Import the exported configuration on the standby OMS:

    emctl importconfig oms -file <full path of the export file>
    

    Note that this command generates a warning about a failed export and prompts for confirmation in order to proceed. The warning can be ignored by entering "y" to proceed.

    This command will automatically start the OMS.

  8. Stop the Management Service but leave the Administration Server running using:

    emctl stop oms
    
  9. Configure the Management Agent on the second OMS host by running the following command from the OMS home:

    $<AGENT_HOME>/sysman/install/agentDeploy.sh AGENT_BASE_DIR=<middleware_home>/agent OMS_HOST=<oms_host_name> EM_UPLOAD_PORT=<oms_port> AGENT_REGISTRATION_PASSWORD=<password> -configOnly
    

    Note: If you have an SLB configured, then directly specify the host and port of the primary load balancer.If no SLB is configured, then use the secure upload port of primary OMS.

    Example:

    ./agentDeploy.sh AGENT_BASE_DIR=$MIDDLEWARE_BASE/agent OMS_HOST=prim_oms_hhost.domain.com EM_UPLOAD_PORT=4900 AGENT_REGISTRATION_PASSWORD=password -configOnly
    
  10. Deploy the required plug-ins on the Management Agent.

    For information about deploying plug-ins, refer to the section Deploying and Updating Plug-ins in the Chapter 13, "Updating Cloud Control".

  11. Add the standby WebLogic Domain and associated targets:

    The standby WebLogic Domain and associated targets can be added using the Guided Discovery process.

    From the Setup menu, select Add Target , then select Add Targets Manually. Select Oracle Fusion Middleware from the Target Types menu. Use the secure port (typically 7101) and, under Advanced, set the JMX Protocol to t3s.Note that the WebLogic targets, except the Administration Server, will be shown as "down" since the standby OMS is down at this stage.

  12. If you have Single Sign-On configured on the primary site, follow the same steps to configure SSO on the standby OMS.

  13. If you have Real User Experience Insight, AD4J Manager or BI Publisher configured on the primary site, follow the same steps to configure them on the standby OMS.

D.1.3 Installing Additional Standby Management Services

It is recommended that your standby site be similar in configuration as your primary site. This means configuring multiple OMS on your standby site, similar to your primary site. Install additional standby Management Services as per the procedure listed below under ”Additional Standby Management Services”.

  1. Start the standby Administration Server by running the following command on the first standby Management Service:

    emctl start oms –admin_only

  2. Export the configuration from the first Management Service on the primary site using:

    emctl exportconfig oms -dir <location for the export file>

    After the configuration is exported, do not make any configuration changes to the primary site still the standby management service is configured.

  3. Install a Management Agent on the standby host.

  4. Perform a software-only install of the Enterprise Manager software using a modified version of ”Add Management Service” Deployment Procedure.

    From the Enterprise menu, select Provisioning and Patching, then select Procedure Library.

    Select Add Management Service procedure and then click Create Like.

    Go to the Procedure Steps tab and select and disable the steps - ”Configure Management Service”, ”Targets Discovery” and ”Post Configuration Tasks”.

    Save the modified deployment procedure and use it to install the Enterprise Manager software on the standby OMS host.

    After the Deployment Procedure completes, delete the file emInstanceMapping.properties from <OMS Oracle Home>/sysman/config on the standby OMS host.

  5. Configure the Management Service by running omsca.

    omsca add –nostart

    When prompted for Management Repository details, provide the primary database details.When prompted for Administration Server details, provide the standby administration server details.

  6. Configure the required plug-ins by running the following command:

    pluginca -action deploy –isFirstOMS false -plugins <plugin-list> -oracleHome <oms oracle home> -middlewareHome <wls middleware home>

    where plugin-list is the list of plug-ins returned by the SQL query above and is a comma separate list in the following format: <plugin-id>=<plugin-version>,<plugin-id>=<plugin-version>,…

    Example "oracle.sysman.empa=12.1.0.1.0,oracle.sysman.mos=12.1.0.1.0,oracle.sysman.emas=12.1.0.1.0,oracle.sysman.emfa=12.1.0.1.0,oracle.sysman.db=12.1.0.1.0,oracle.sysman.emct=12.1.0.1.0,oracle.sysman.vt=12.1.0.1.0,oracle.sysman.ssa=12.1.0.1.0"

  7. Copy over the configuration exported from the primary Management Service in step 1 above to the standby Management Service host. Import the exported configuration on the standby Management Service using:

    emctl importconfig oms -file <full path of the export file>

    Note this command emits a warning about a failed export and prompts for confirmation to proceed. The warning can be ignored by entering "y" to proceed.

    Note this command will start the Management Service.

  8. Stop the Management Service using:

    emctl stop oms

  9. Refresh the standby domain target from the console. This will present a guided workflow to discover and add the new managed server and associated targets.

  10. If you have Single Sign On configured on the primary site, follow the same steps to configure SSO on the standby OMS.

  11. If you have Real User Experience Insight, AD4J Manager or BI Publisher configured on the primary site, follow the same steps to configure them on the standby OMS.

D.1.4 Installing Additional Standby Management Services (Oracle Installer, Software Only Mode)

  1. Start the standby Administration Server by running the following command on the first standby Management Service:

    emctl start oms –admin_only

  2. Export the configuration from the first Management Service on the primary site using:

    emctl exportconfig oms -dir <location for the export file>

    After the configuration is exported, do not make any configuration changes to the primary site still the standby management service is configured.

  3. On the remote host, perform a software-only installation of the standby OMS as described in the Oracle® Enterprise Manager Cloud Control Advanced Installation and Configuration Guide chapter "Installing Enterprise Manager Software Now and Configuring Later" (Installing in Graphical Mode). Ensure that you install the software binaries in the same middleware location as that of the primary OMS.

    Note:

    Make sure you select same set of plug-ins as the primary OMS.

    If you have installed extra plug-ins which are not part of the installation, then complete the following steps.

    1. Connect to repository and execute the following query to obtain the list of installed plug-ins:

      SELECT epv.plugin_id "plugin id", epv.version "version", epv.rev_version "revision", epv.su_entity_id "update id"
      FROM em_plugin_version epv, em_current_deployed_plugin ecp
      WHERE epv.plugin_type NOT IN ('BUILT_IN_TARGET_TYPE', 'INSTALL_HOME')
      AND ecp.dest_type='2'
      AND epv.plugin_version_id = ecp.plugin_version_id;
      

      Write down the returned list of plug-ins installed.

    2. Extract the plug-in archives from the primary site.

      emcli export_update -id=<update id> -host=<standby OMS host> -dir=<directory to export archives> <host credential options>
      

      where <update id> is the value returned by the four column of query above.

      Note that this command generates a file with a .zip extension. Rename the file from <file_name>.zip to <file_name>.opar .

      Repeat above steps for all plugins. In standby OMS host, place all .opar files in one directory <plugin dir >.

    3. Invoke runInstaller and choose software-only install. This will install only the middleware home and OMS Oracle homes.

    4. To install the required plug-ins, run the plug-in install script.

      <OMS_HOME>/sysman/install/PluginInstall.sh -pluginLocation <plugin dir>
      

      where <plugin dir> is the directory where you placed extra plug-ins in step two above. Running this command displays a screen that will let you select the plug-ins to install. Verify that all the plug-ins listed in step 1 appear and that you select them. If there is a mismatch, OMS configuration may fail at a later step.

  4. On the software-only installation, apply all the patches you applied on the primary OMS so that both the primary and standby OMSs are identical and are in sync.

  5. Configure the Management Service by running omsca.

    omsca add –nostart

    When prompted for Management Repository details, provide the primary database details. When prompted for Administration Server details, provide the standby Administration Server details.To keep the ports consistent with your primary site, provide all the ports on the command line itself. For example:

    omsca add -MSPORT 7202 -MS_HTTPS_PORT 7301 -EM_NODEMGR_PORT 7403 -EM_UPLOAD_PORT 4889 -EM_UPLOAD_HTTPS_PORT 4900 -EM_CONSOLE_PORT 7788 -EM_CONSOLE_HTTPS_PORT 7799 -nostart
    
  6. Configure the required plug-ins by running the following command:

    pluginca -action deploy –isFirstOMS false -plugins <plugin-list> -oracleHome <oms oracle home> -middlewareHome <wls middleware home>

    where the plugin-list is the list of plug-ins returned by the SQL query above and is a comma separated list in the following format: <plugin-id>=<plugin-version>,<plugin-id>=<plugin-version>,…

    Example "oracle.sysman.empa=12.1.0.1.0,oracle.sysman.mos=12.1.0.1.0,oracle.sysman.emas=12.1.0.1.0,oracle.sysman.emfa=12.1.0.1.0,oracle.sysman.db=12.1.0.1.0,oracle.sysman.emct=12.1.0.1.0,oracle.sysman.vt=12.1.0.1.0,oracle.sysman.ssa=12.1.0.1.0"

  7. Copy over the configuration exported from the primary Management Service in step 1 above to the standby Management Service host. Import the exported configuration on the standby Management Service using:

    emctl importconfig oms -file <full path of the export file>

    Note this command emits a warning about a failed export and prompts for confirmation to proceed. The warning can be ignored by entering "y" to proceed.

    Note this command will automatically start the OMS.

  8. Stop the OMS using:

    emctl stop oms

  9. Configure the Management Agent on the second OMS host by running the following command from the OMS home:

    $<AGENT_HOME>/sysman/install/agentDeploy.sh AGENT_BASE_DIR=<middleware_home>/agent OMS_HOST=<second_oms_host_name>EM_UPLOAD_PORT=<second_oms_port> AGENT_REGISTRATION_PASSWORD=<password> -configOnly
    

    Note: If you have an SLB configured, then directly specify the host and port of the primary load balancer.If no SLB is configured, then use secure upload port of primary OMS.

  10. Deploy the required plug-ins on the Management Agent.

    For information about deploying plug-ins, refer to the section Deploying and Updating Plug-ins in the Chapter 13, "Updating Cloud Control".

  11. Refresh the standby domain target from the console. This will present a guided workflow to discover and add the new managed server and associated targets.

  12. If you have Single Sign On configured on the primary site, follow the same steps to configure SSO on the standby OMS.

  13. If you have Real User Experience Insight, AD4J Manager or BI Publisher configured on the primary site, follow the same steps to configure them on the standby OMS.

D.1.5 Validating Your Installation and Complete the Setup

Update the standby SLB configuration by adding the standby Management Service(s) to the different pools on the SLB. Setup monitors for the new Management Service.

D.1.5.1 Keeping the Standby Site in Sync

After the initial setup of the standby site, the standby site has to be kept in sync with the changes done on primary site. Transactions on the primary Management Repository get propagated to the Standby Management Repository automatically through Data Guard but the OMS side changes have to be redone manually on the standby site. The following sections describe this procedure for typical activities.

Applying patches

When patches are applied on the primary site Management Services, they have to be applied on the standby site Management Services too. Note that patches typically update the Oracle Homes (via the Opatch apply command) and optionally might require scripts to be run against the Management Repository. On the standby site, it is sufficient to update the Oracle Homes (via the opatch apply command) and skip the running of scripts on the Management Repository because database changes are automatically propagated to the standby site using Data Guard.

Managing Plug-ins

When new plug-ins are deployed on the primary site or existing plug-ins upgraded or un-deployed on the primary site, the following procedures needs to be run on the standby site too to keep the Standby Management Services in sync. Note if the Standby Management Services are not kept in sync, they would fail to start when a switchover or failover operation is attempted.

The procedure below assumes that the standby site was setup as per the documented process and the standby management services are currently down and point to the primary repository. The plug-in(s) deployment on the primary site has been completed successfully.

Deploying a New Plug-in or Upgrading a Plug-in on Standby Site

  1. Extract the plug-in archives from the primary site

    Go to the Self Update Home, click on Plug-ins, select the required plug-in and select export from the Action table menu. Note the EM CLI command from the popup that gets displayed.

    emcli export_update -id=<update id> -deep -host=<standby OMS host> -dir=<directory to export archives> <host credential options>

    Note that an additional option ”-deep” is required. This command would create 4 files on the destination directory specified. The filename <version>_OMS_<platform>_<revision>.zip is the one to be used in next step.

  2. Start the Standby Administration Server, if it is down.

    emctl start oms -admin_only

  3. Run prerequisite checks for the plug-in and apply the required patch reported by after running the following command.

    ./emctl plugin deploy -action prereqcheck -isFirstOMS false -archives <exported_plugin_archive>
    

    where <exported_plugin_archive> is the zip file that was exported in Step 1: <version>_OMS_<platform>_<revision>.zip

  4. Configure the plug-in on the first standby OMS Oracle Home.

    ./emctl plugin deploy -action deploy -isFirstOMS false -archives <exported_plugin_archive>
    

    where exported_plugin_archive represents the zip file <version>_OMS_<platform>_<revision>.zip

  5. Repeat steps 3 and 4 for each standby additional OMS/

    ./emctl plugin deploy -action prereqcheck -isFirstOMS true -archives <exported_plugin_archive>
    
    ./emctl plugin deploy -action deploy -isFirstOMS true -archives <exported_plugin_archive>
    

    This completes the plug-in deployment on the standby site.

Sync up Sysman Credentials

When administrators modify sysman credentials on the primary site, the following procedure must be run on the standby site in order to keep the standby management service in sync with respect to sysman credentials.

If the sysman credentials for the primary and standby sites are not kept in sync, they will fail to start when switchover or failover opertations are attempted. In addition, the administratator will not be able to log into the standby OMS once the switchover operation is complete.

The procedure below assumes that the standby site was set up according to the documented process and the standby management services are currently down and point to the primary repository. The plug-in(s) deployment on the primary site has been completed successfully.

Perform following steps on standby management service

  1. Start the standby Administration Server, if it is down

    emctl start oms –admin_only

  2. Change password for the sysman user. You must also provide the SYS password.

    emctl config oms -change_repos_pwd -use_sys_pwd -sys_pwd <sys_pwd>

  3. Stop the standby Administration Server,

    emctl stop oms –all