17 Recovering Your Environment

This chapter describes recommended recovery strategies and procedures for recovering Oracle Fusion Middleware from different types of failures and outages, such as media failures or loss of host.

This chapter includes the following sections:

17.1 Overview of Recovering Your Environment

This section provides an overview of recovery strategies for outages that involve actual data loss or corruption, host failure, or media failure where the host or disk cannot be restarted and they are permanently lost. This type of failure requires some type of data restoration before the Oracle Fusion Middleware environment can be restarted and continue with normal processing.

Note:

The procedures in this chapter assume that no administrative changes were made since the last backup. If administrative changes were made since the last backup, they must be reapplied after recovery is complete.

When you restore the files, use your preferred tool to extract the compressed files, as described in Section 15.4.

Ensure that the tool you are using preserves the permissions and timestamps of the files.

Rename existing files and directories before you begin restoring the files from backup so that you do not unintentionally override necessary files.

When you recover your environment, it is important to recover the entities in the correct order:

  1. The database, if it needs to be recovered. See Section 17.2.9 and Section 17.3.7.

  2. The Oracle Home, if it needs to be recovered. See Section 17.2.1.

  3. The entire domain, if it needs to be recovered. See Section 17.2.2 and Section 17.3.1 for recovering a WebLogic Server managed domain. See Section 17.2.3 for recovering a standalone domain.

  4. The Administration Server, if you do not need to recover the domain. See Section 17.2.4 and Section 17.3.3,

  5. The Managed Servers, if they are not in the Administration Server domain directory and they need to be recovered. See Section 17.2.5 and Section 17.3.4.

    Java components are recovered when you recover the Managed Server. System components are recovered when you recover the domain.

    However, when you have a loss of host, you may need to take additional steps to recover components. See Section 17.3.5

  6. Applications, if they need to be recovered. See Section 17.2.8.

17.2 Recovering After Data Loss, Corruption, Media Failure, or Application Malfunction

This section describes recovery strategies for outages that involve actual data loss or corruption, or media failure where the disk cannot be restored. It also describes recovery strategies for applications that are no longer functioning properly. This type of failure requires some type of data restoration before the Oracle Fusion Middleware environment can be restarted and continue with normal processing. It contains the following topics:

Note:

You can only restore an entity to the same path as the original entity. That path can be on the same host or a different host.

17.2.1 Recovering the Oracle Home

You can recover the Oracle home that was corrupted or from which files were deleted.

To recover the Oracle home:

  1. Stop all relevant processes. That is, stop all processes that are related to the domain, such as the Administration Server, Node Manager, and Managed Servers. For example, to stop the Administration Server on Linux:

    DOMAIN_HOME/bin/stopWebLogic.sh username password [admin_url]
    
  2. Change to the directory that you want to be the parent directory of the Oracle home directory. Use the same directory structure as in the original environment.

  3. Recover the Oracle home directory from backup. For example:

    (UNIX) tar -xf oracle_home_backup_042013.tar
    (Windows) jar xf oracle_home_backup_042013.jar
    
  4. Start all relevant processes. That is, start all processes that run in the Oracle home, such as the Administration Server and Managed Servers. For example, start the Administration Server:

    DOMAIN_HOME/bin/startWebLogic.sh 
    

17.2.2 Recovering an Oracle WebLogic Server Domain

You can recover an Oracle WebLogic Server domain that was corrupted or deleted from the file system.

Caution:

Performing a domain-level recovery can impact other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

To recover an Oracle WebLogic Server domain that was corrupted or deleted from the file system:

  1. Stop all relevant processes. That is, stop all processes that are related to the domain, such as the Administration Server, Managed Servers, and any system components. For example, stop the Administration Server:

    DOMAIN_HOME/bin/stopWebLogic.sh username password [admin_url]
    

    For information on stopping system components such as Oracle HTTP Server, see Section 4.3.2.

  2. Change to the directory that you want to be the parent directory of the domain home directory. Use the same directory structure as in the original environment.

  3. Recover the domain directory from backup:

    (UNIX) tar -xf domain_backup_042013.tar 
    (Windows) jar xf domain_backup_042013.jar 
    
  4. Start all relevant processes. That is, start all processes that are related to the domain, such as the Administration Server, Managed Servers, and any system components. For example, start the Administration Server:

    DOMAIN_HOME/bin/startWebLogic.sh 
    

    For information on starting system components such as Oracle HTTP Server, see Section 4.3.2.

  5. If you cannot start the Administration Server, recover it, as described in Section 17.2.4.

  6. If you cannot start a Managed Server, recover it, as described in Section 17.2.5.

17.2.3 Recovering a Standalone Domain

You can recover a standalone domain that contains system components such as Oracle HTTP Server, that was corrupted or deleted from the file system.

To recover a standalone domain:

  1. Stop Node Manager as described in Section 4.2.2.

  2. Recover the domain home:

    (UNIX) tar xf domain_backup_05_21_2013.tar
    (Windows) jar xf domain_backup_05_21_2013.jar
    
  3. Start Node Manager:

    (UNIX) DOMAIN_HOME/bin/startNodeManager.sh
    (Windows) DOMAIN_HOME\bin\startNodeManager.cmd
    
  4. Start any system components, such as Oracle HTTP Server, that are in the domain:

    (UNIX) Domain_Home/bin/startComponent.sh ohs1
    (Windows) Domain_Home\bin\startComponent.cmd ohs1
    

17.2.4 Recovering the Administration Server Configuration

If the Administration Server configuration has been lost because of file deletion or file system corruption, the Administration Server console continues to function if it was already started when the problem occurred. To prevent the Administration Server from prompting for a user name and password, see Section 4.2.4.

Caution:

Performing a domain-level recovery can impact other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

To recover the Administration Server configuration:

  1. Stop all processes, including the Administration Server, Managed Servers, and Node Manager, if they are started. For example, to stop the Administration Server:

    DOMAIN_HOME/bin/stopWebLogic.sh username password [admin_url]
    
  2. Recover the Administration Server configuration by recovering the domain home backup to a temporary location. Then, restore the config directory to the following location:

    DOMAIN_HOME/config
    
  3. Start the Administration Server. For example:

    DOMAIN_HOME/bin/startWebLogic.sh 
    
  4. Verify that the Administration Server starts properly and is accessible.

On the next configuration change, the configuration from the Administration Server is pushed to the Managed Servers. On each Managed Server restart, the configuration is retrieved from the Administration Server.

17.2.5 Recovering a Managed Server

You can recover a Managed Server's files, including its configuration files if they are deleted or corrupted.

The following topics describe how to recover a Managed Server's files:

17.2.5.1 Recovering a Managed Server When It Cannot Be Started

In this scenario, the Managed Server is not on the same host as the Administration Server, and it does not operate properly or cannot be started because the configuration has been deleted or corrupted or the configuration was mistakenly changed and you cannot ascertain what was changed.

To recover a Managed Server when it cannot be started:

  1. If the Administration Server is not reachable, recover the Administration Server, as described in Section 17.2.4.

  2. If the Managed Server fails to start, recover the Oracle home from the backup, if required. For example:

    tar -xf oracle_home_backup_042013.tar 
    
  3. Stop Node Manager as described in Section 4.2.2.

  4. If the file system is lost, take the following steps:

    1. Create a domain template jar file for the Administration Server, using the pack utility on the Administration Server host. For example:

      pack.sh -domain=/scratch/oracle/config/domains/domain_name
         -template=/scratch/temp.jar -template_name=test_install 
         -template_author=myname -log=/scratch/logs/my.log -managed=true
      

      Specifying the -managed=true option packs up only the Managed Servers. If you want to pack the entire domain, omit this option.

    2. Unpack the domain template jar file, using the unpack utility on the Managed Server host. In the following example, temp.jar is the archive created by the pack command:

      unpack.sh -template=/scratch/temp.jar
         -domain=/scratch/oracle/config/domains/domain_name
         -log=/scratch/logs/new.log -log_priority=info
      

      Note:

      • The following directory must exist. If it does not the unpack command fails.

        ORACLE_HOME/config/domains/
        
      • The unpack command provides an -overwrite_domain option, which allows unpacking a Managed Server template into an existing domain and existing applications directories. For any file that is overwritten, a backup copy of the original is created. Use the -overwrite_domain option, if required for your deployment.

      • By default, applications are stored in the following directory unless you pass another location using the -app_dir argument:

        ORACLE_HOME/user_projects/applications/Domain_Name
        
    3. Ensure that the application artifacts are accessible from the Managed Server host. That is, if the application artifacts are not on the same server as the Managed Server, they must be in a location accessible by the Managed Server.

      Note:

      • For stage mode applications, the Administration server copies the application bits to the staged directories on the Managed Server hosts.

      • For nostage and external_stage mode applications, ensure that application files are available in the stage directories of the Managed Server.

      See Deploying Applications to Oracle WebLogic Server for information about stage, nostage and external_stage modes.

  5. Start the Managed Server. For example:

    DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url 
    

    The Managed Server connects to the Administration Server and updates its configuration changes.

17.2.5.2 Recovering a Managed Server When It Does Not Function Correctly

In this scenario, the Managed Server is running, but the file system for the Managed Server has been lost or corrupted.

To recover the Managed Server:

  1. Stop the Managed Server. For example:

    DOMAIN_HOME/bin/stopManagedWeblogic.sh managed_server_name admin_url
            username password
    
  2. Recover the Oracle home from the backup, if required:

    tar -xf oracle_home_backup_042013.tar 
    
  3. Stop Node Manager as described in Section 4.2.2.

  4. Create a domain template jar file for the Administration Server, using the pack utility on the Administration Server host. For example:

    pack.sh -domain=/scratch/oracle/config/domains/WLS_domain
      -template=/scratch/temp.jar -template_name=test_install 
      -template_author=myname -log=/scratch/logs/my.log -managed=true
    

    Specifying the -managed=true option packs up only the Managed Servers. If you want to pack the entire domain, omit this option.

  5. Unpack the domain template jar file, using the unpack utility on the Managed Server host. In the following example, temp.jar is the archive created by the pack command:

    unpack.sh -template=/scratch/temp.jar
       -domain=/scratch/oracle/config/domains/WLS_domain
       -log=/scratch/logs/new.log -log_priority=info
    

    Note:

    • The following directory must exist. If it does not the unpack command fails.

      ORACLE_HOME/config/domains/
      
    • The unpack command provides an -overwrite_domain option, which allows unpacking a Managed Server template into an existing domain and existing applications directories. For any file that is overwritten, a backup copy of the original is created. Use the -overwrite_domain option, if required for your deployment.

    • By default, applications are stored in the following directory unless you pass another location using the -app_dir argument:

      ORACLE_HOME/user_projects/applications/Domain_Name
      
  6. Ensure that the application artifacts are accessible from the Managed Server host. That is, if the application artifacts are not on the same server as the Managed Server, they must be in a location accessible by the Managed Server.

    Note:

    • For stage mode applications, the Administration server copies the application bits to the staged directories on the Managed Server hosts.

    • For nostage and external_stage mode applications, ensure that application files are available in the stage directories of the Managed Server.

    See Deploying Applications to Oracle WebLogic Server for information about deploying applications.

  7. Restart the Managed Server. For example:

    DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url
    

17.2.6 Recovering Components

For components such as Oracle WebLogic Server and Oracle HTTP Server, you recover the components using the procedures in the following topics:

17.2.6.1 Recovering a Component That Is Not Functioning Properly

You can recover a component if the component's files have been deleted or corrupted or if the component cannot be started or is not functioning properly because the component's configuration was changed and committed. You may not be able to ascertain what change is causing the problem and you want to revert to an earlier version.

  • For Java components, you recover the Managed Server, as described in Section 17.2.5.

  • For system components, such as Oracle HTTP Server, in a standalone domain, you recover the domain, as described in Section 17.2.3.

  • For system components, such as Oracle HTTP Server, in an Oracle WebLogic Server domain, you recover the domain as described in Section 17.2.2.

17.2.6.2 Recovering a Component After Cluster Configuration Change

You can recover components in a cluster when the components cannot be started or are not functioning properly because the configuration was changed and committed at the cluster level. You may not be able to ascertain what change is causing the problem and you want to revert to an earlier version.

Caution:

Performing a domain-level recovery can impact other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

To recover the components:

  1. Stop the cluster using the WLST stop command:

    shutdown('cluster_name','Cluster')
    
  2. Stop all processes, such as the Managed Servers and the Administration Server. For example, to stop the Administration Server on Linux:

    DOMAIN_HOME/bin/stopWebLogic.sh username password [admin_url]
    
  3. Recover the Administration Server configuration by recovering the domain home backup to a temporary location. Then, restore the config directory to the following location:

    DOMAIN_HOME/config
    
  4. Start the Administration Server. For example:

    DOMAIN_HOME/bin/startWebLogic.sh 
    
  5. Start the cluster. You can use the Oracle WebLogic Server Administration Console or WLST. For example, to use the WLST start command:

    start('cluster_name', 'Cluster')
    

The latest configuration is retrieved from the Administration Server to every member of the cluster.

17.2.7 Recovering a Cluster

The following topics describe how to recover a cluster:

17.2.7.1 Recovering a Cluster After Deletion or Cluster-Level Configuration Changes

In this scenario, the cluster has been erroneously deleted or the cluster-level configuration, such as the JMS configuration or container-level data sources, was mistakenly changed and committed. The server cannot be started or does not operate properly or the services running inside the server are not starting. You cannot ascertain what was changed.

Caution:

Performing a domain-level recovery can impact other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

If the configuration changes are few, then the easiest way is to redo the configuration changes. If that is not feasible, use the following procedure to recover the configuration:

  1. Stop the cluster. You can use the Oracle WebLogic Server Administration Console or WLST. For example, to use WLST:

    shutdown('cluster_name', 'Cluster')
    
  2. Stop the Administration Server. For example:

    DOMAIN_HOME/bin/stopWebLogic.sh username password [admin_url]
    
  3. Recover the Administration Server configuration by recovering the domain home backup to a temporary location. Then, restore the config directory to the following location:

    DOMAIN_HOME/config
    
  4. Start the Administration Server. For example:

    DOMAIN_HOME/bin/startWebLogic.sh 
    
  5. Start the cluster. You can use the Oracle WebLogic Server Administration Console or WLST. For example, to use WLST:

    start('cluster_name', 'Cluster')
    

17.2.7.2 Recovering a Cluster After Membership Is Mistakenly Modified

You can recover a cluster when the cluster's membership has been mistakenly modified. For example, if you inadvertently delete a member from the cluster, you can restore the member to the cluster.

Caution:

Performing a domain-level recovery can impact other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

To recover the cluster membership:

  1. Stop all processes, such as the Managed Servers and the Administration Server. For example, to stop the Administration Server on Linux:

    DOMAIN_HOME/bin/stopWebLogic.sh username password [admin_url]
    
  2. Recover the Administration Server configuration by recovering the domain home backup to a temporary location. Then, restore the config directory to the following location:

    DOMAIN_HOME/config
    
  3. Start the Administration Server. For example:

    DOMAIN_HOME/bin/startWebLogic.sh 
    

    The deleted member is now back in the cluster.

  4. Start all processes, such as the Managed Servers. For example, to start the Managed Server on Linux, use the following script:

    DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url 
    
  5. Start the cluster. You can use the Oracle WebLogic Server Administration Console or WLST. For example, to use WLST:

    start('cluster_name', 'Cluster')
    

    The deleted member is now part of the cluster.

  6. Start all cluster members if they are not started:

    DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url 
    

17.2.8 Recovering Applications

The following topics describe how to recover an application:

Note the following about recovering applications:

  • If the application is staged, the Administration server copies the application bits to the staged directories on the Managed Server hosts.

  • If the deployment mode is nostage or external_stage, ensure that additional application artifacts are available. For example, applications may reside in directories outside of the domain directory. Make your application files available to the new Administration Server by copying them from backups or by using a shared disk. Your application files should be available in the same relative location on the new file system as on the file system of the original Administration Server.

See Also:

Deploying Applications to Oracle WebLogic Server for information about deploying applications

17.2.8.1 Recovering Application Artifacts

If an application's artifacts, such as the .ear file, have been lost or corrupted, you can recover the application.

To recover the application:

  1. Start the Managed Server to which the application was deployed. For example:

    DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url 
    

    This synchronizes the configuration with the Administration Server.

    On each Managed Server restart, the configuration and application artifacts are retrieved from the Administration Server.

17.2.8.2 Recovering a Redeployed Application That Is No Longer Functional

If a Java EE application was redeployed to a Managed Server (whether or not the Managed Server is part of a cluster) and the application is no longer functional, you can recover it.

To recover the application:

  1. Recover the application files from backup, if needed.

  2. Redeploy the old version of the application from the backup.

    You cannot just copy the original ear file. Even if the original ear file (from the backup) is copied back to the Managed Server stage directory and you restart the Managed Server, the application is still not recovered. You must redeploy the original version.

17.2.8.3 Recovering an Undeployed Application

If a deployed application was undeployed from Oracle WebLogic Server, you can recover it.

To recover the application:

  1. Recover the application files from backup, if needed.

  2. Redeploy the old version of the application from the backup. If the application was deployed to a cluster, redeploy the application to the same cluster.

    You cannot just copy the original ear file. Even if the original ear file (from the backup) is copied back to the Managed Server stage directory and you restart the Managed Server, the application is still not recovered. You must redeploy the original version.

17.2.8.4 Recovering a Composite Application

A new version of a composite application was redeployed to a Managed Server or cluster. The application is no longer functional.

To recover the application:

  1. Recover the application files from backup, if needed.

  2. Redeploy the old version of the application. If the application was deployed to a cluster, redeploy the application to the same cluster.

17.2.9 Recovering a Database

If your database that contains your metadata repository, including the MDS Repository, is corrupted, you can recover it using RMAN. You can recover the database at the desired granularity, either a full recovery or a tablespace recovery.

For best results, recover the database to the most current state, using point-in-time recovery (if the database is configured in Archive Log Mode.) This ensures that the latest data is recovered. For example:

rman> restore database;
rman> recover database;

See Creating Schemas with the Repository Creation Utility for the schemas used by each component.

For detailed steps, see the Oracle Database Backup and Recovery User's Guide.

17.3 Recovering After Loss of Host

This section describes how to recover your Oracle Fusion Middleware environment after losing the original operating environment. For example, you could have a serious system malfunction or loss of media. The sections includes the following topics:

Note:

When you are recovering in the case of loss of host, you must restore the files using the same path as on the original host.

17.3.1 Recovering After Loss of Oracle WebLogic Server Domain Host

To recover an Oracle WebLogic Server domain:

  1. Change to the directory that you want to be the parent directory of the Domain home directory. Use the same directory structure as in the original environment.

  2. Recover the domain directory from backup:

    (UNIX) tar -xf domain_backup_042013.tar 
    (Windows) jar xf domain_backup_042013.jar 
    
  3. Start all relevant processes. That is, start all processes that are related to the domain. For example, start the Administration Server:

    DOMAIN_HOME/bin/startWebLogic.sh 
    
  4. If you cannot start the Administration Server, recover it, as described in Section 17.3.3.

  5. If you cannot start a Managed Server, recover it, as described in Section 17.3.4.

17.3.2 Recovering After Loss of Standalone Domain Host

If you lose a host that contains a standalone domain, you can recover it to the same host or a different host, as described in the following topics:

17.3.2.1 Recovering a Standalone Domain to the Same Host

In this scenario, you recover the standalone domain to the same host after the operating system has been reinstalled.

To recover the standalone domain to the same host:

  1. If Node Manager or a system component, such as Oracle HTTP Server are running, stop them.

  2. If it is corrupted, recover the Oracle home:

    (UNIX) tar xf oracle_home_backup_05_21_2013.tar
    (Windows) jar xf oracle_home_backup_05_21_2013.jar
    
  3. Recover the domain home:

    (UNIX) tar xf domain_backup_05_21_2013.tar
    (Windows) jar xf domain_backup_05_21_2013.jar
    
  4. Start Node Manager:

    (UNIX) DOMAIN_HOME/bin/startNodeManager.sh
    (Windows) DOMAIN_HOME\bin\startNodeManager.cmd
    
  5. Start any system components, such as Oracle HTTP Server, that are in the domain:

    (UNIX) Domain_Home/bin/startComponent.sh ohs1
    (Windows) Domain_Home\bin\startComponent.cmd ohs1
    

17.3.2.2 Recovering a Standalone Domain to a Different Host

In this scenario, you recover the standalone domain to a different host.

To recover the standalone domain to a different host:

  1. Recover the Oracle home:

    (UNIX) tar xf oracle_home_backup_05_21_2013.tar
    (Windows) jar xf oracle_home_backup_05_21_2013.jar
    
  2. Recover the domain home:

    (UNIX) tar xf domain_backup_05_21_2013.tar
    (Windows) jar xf domain_backup_05_21_2013.jar
    
  3. In a standalone domain, by default, Node Manager is listening on localhost. However, if it is not, you can update the ListenAddress by using the following WLST commands:

    readDomain('Domain_Home')
    cd('/')
    cd('NMProperties')
    set('ListenAddress','localhost')
    set('ListenPort',9001)
    
  4. Start Node Manager:

    (UNIX) DOMAIN_HOME/bin/startNodeManager.sh
    (Windows) DOMAIN_HOME\bin\startNodeManager.cmd
    
  5. Update any system component configuration files manually.

    See Section 17.3.5 for details for specific components.

  6. Start any system components, such as Oracle HTTP Server, that are in the domain. For example:

    (UNIX) Domain_Home/bin/startComponent.sh ohs1
    (Windows) Domain_Home\bin\startComponent.cmd ohs1
    
  7. Update the Oracle Inventory, as described in Section 17.3.6.4.

  8. For Windows, update the Windows Registry, as described in Section 17.3.6.5.

17.3.3 Recovering After Loss of Administration Server Host

If you lose a host that contains the Administration Server, you can recover it to the same host or a different host, as described in the following topics:

17.3.3.1 Recovering the Administration Server to the Same Host

In this scenario, you recover the Administration Server either to the same host after the operating system has been reinstalled or to a new host that has the same host name. For example, the Administration Server is running on Host A and the Managed Server is running on Host B. Host A has failed for some reason and the Administration Server must be recovered.

To recover the Administration Server to the same host:

  1. Recover the file system. For example, recover the domain containing the Administration Server, as described in Section 17.3.1.

  2. Attempt to start the Administration Server. For example:

    DOMAIN_HOME/bin/startWebLogic.sh 
    

    If the Administration Server starts, you do not need to take any further steps.

  3. If the Administration Server fails to start, take the following steps on Host A:

    1. Stop all relevant processes. That is, stop all processes that are related to the domain, such as the Administration Server and Managed Servers. For example, to stop the Administration Server on Linux:

      DOMAIN_HOME/bin/stopWebLogic.sh username password [admin_url]
      
    2. Recover the Oracle home, if needed:

      tar -xf oracle_home_backup_042013.tar 
      
    3. If the domain directory does not reside in the Oracle home, recover the domain directory from backup. First change to the directory that you want to be the parent of the Domain home, then:

      tar -xf domain_backup_042013.tar 
      
    4. Start the Administration Server. For example:

      DOMAIN_HOME/bin/startWebLogic.sh 
      
    5. Start the Managed Servers, specifying the Administration URL for the host:

      DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url 
      
    6. Start Node Manager:

      cd DOMAIN_HOME/bin
      ./startNodeManager.sh
      

17.3.3.2 Recovering the Administration Server to a Different Host

In this scenario, the Administration Server is running on Host A and the Managed Server is running on Host B. Host A has failed for some reason and the Administration Server must be moved to Host C.

To recover the Administration Server to a different host:

  1. Recover the Oracle home to Host C (the new Host):

    tar -xf oracle_home_backup_042013.tar 
    
  2. If the domain directory does not reside in the Oracle home, recover the domain directory from backup. First change to the directory that you want to be the parent of the Domain home, then:

    tar -xf domain_backup_042013.tar 
    
  3. If the Administration Server has a Listen address, create a new machine with the new host name, as described in Section 17.3.6.3.

  4. Start the Administration Server. For example:

    DOMAIN_HOME/bin/startWebLogic.sh 
    
  5. Using WLST, connect to the Administration Server and then enroll Node Manager running in the new host with the Administration Server:

    connect('username','password','t3://host:port')
    nmEnroll('/scratch/oracle/config/domains/domain_name',
      'DOMAIN_HOME/nodemanager')
    
  6. Edit the Node Manager properties file, changing the Listen Address property. For a domain-based Node Manager, the file is located at:

    DOMAIN_HOME/nodemanager/nodemanager.properties
    

    Alternatively, you can use the following WLST commands to change the property:

    readDomain('Domain_Home')
    cd('/')
    cd('NMProperties')
    set('ListenAddress','localhost')
    set('ListenPort',port_num)
    
  7. Start Node Manager on Host C if it was configured on the original host:

    cd DOMAIN_HOME/bin
    ./startNodeManager.sh
    
  8. Start the Managed Servers. The section "Restarting a Failed Administration Server" in the Administering Server Startup and Shutdown for Oracle WebLogic Server describes different ways to restart them, depending on how they were configured.

    One option is to use the following script, specifying the Administration URL of the new host:

    DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url 
    
  9. Ensure that additional application artifacts are available. For example, if the deployment mode is nostage or external_stage, applications may reside in directories outside of the domain directory. Make your application files available to the new Administration Server by copying them from backups or by using a shared disk. Your application files should be available in the same relative location on the new file system as on the file system of the original Administration Server.

    If the application is staged, the Administration Server copies the application bits to the staged directories on the Managed Server hosts.

  10. Update Oracle Inventory, as described in Section 17.3.6.4.

  11. On Windows, recover the Windows Registry, as described in Section 17.3.6.5

  12. If your environment contains Oracle HTTP Server, modify the mod_wl_ohs.conf file, as described in Section 17.3.6.2.

Now you can start and stop the Managed Server on Host B using the Administration Console running on Host C.

If you are recovering the Administration Server for a Web Tier installation, see Section 17.3.6 for information about additional actions you must take.

17.3.4 Recovering After Loss of Managed Server Host

If you lose a host that contains a Managed Server, you can recover it to the same host or a different host, as described in the following topics:

17.3.4.1 Recovering a Managed Server to the Same Host

In this scenario, you recover a Managed Server to the same host after the operating system has been reinstalled or to a new host that has the same host name. The Administration Server is running on Host A and the Managed Server is running on Host B. Host B failed for some reason and the Managed Server must be recovered to Host B.

To recover a Managed Server to the same host:

  1. Start Node Manager on Host B:

    cd DOMAIN_HOME/bin
    ./startNodeManager.sh
    
  2. Start the Managed Server. For example:

    DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url 
    

    If the Managed Server starts, it connects to the Administration Server and updates its configuration changes. You do not need to take any further steps.

  3. If the Managed Server fails to start or if the file system is lost, take the following steps:

    1. Recover the Oracle home to Host B from the backup, if required:

      tar -xf oracle_home_backup_042013.tar 
      
    2. Stop Node Manager as described in Section 4.2.2.

    3. Create a domain template jar file for the Administration Server running in Host A, using the pack utility. For example:

      pack.sh -domain=/scratch/oracle/config/domains/domain_name
        -template=/scratch/temp.jar -template_name=test_install 
        -template_author=myname -log=/scratch/logs/my.log -managed=true
      

      Specifying the -managed=true option packs up only the Managed Servers. If you want to pack the entire domain, omit this option.

    4. Unpack the domain template jar file in Host B, using the unpack utility:

      unpack.sh -template=/scratch/temp.jar
        -domain=/scratch/oracle/config/domains/domain_name 
        -log=/scratch/logs/new.log -log_priority=info
      
    5. Ensure that the application artifacts are accessible from the Managed Server host. That is, if the application artifacts are not on the same server as the Managed Server, they must be in a location accessible by the Managed Server.

      Note:

      • For applications that are deployed in nostage and external_stage mode, copy the application artifacts from the Administration Server host directory.

      • For applications that are deployed in stage mode, the Administration server copies the application bits to the staged directories on the Managed Server hosts.

      See Deploying Applications to Oracle WebLogic Server for information about deploying applications.

    6. Update the Node Manager property ListenAddress by using the following WLST commands:

      readDomain('Domain_Home')
      cd('/')
      cd('NMProperties')
      set('ListenAddress','localhost')
      set('ListenPort',9001)
      
    7. If Node Manager is not started, start it:

      cd DOMAIN_HOME/bin
      ./startNodeManager.sh
      
    8. Start the Managed Server. For example:

      DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url 
      

      The Managed Server connects to the Administration Server and updates its configuration changes.

17.3.4.2 Recovering a Managed Server to a Different Host

In this scenario, the Administration Server is running on Host A and the Managed Server is running on Host B. Host B failed for some reason and the Managed Server must be recovered to Host C. There are two machines, which are logical representations of the computer that hosts one or more WebLogic Servers, machine_1 on Host A and machine_2 on Host B.

Important:

Recover the Oracle home to the same location as the original.

To recover a Managed Server to a different host:

  1. Recover the Oracle home for the Managed Server to Host C.

    tar -xf oracle_home_backup_042013.tar 
    
  2. Reconfigure the topology to point to the new host:

    1. To avoid an inconsistent backup, do not make any configuration changes until the backup is completed. To ensure that no changes are made in the WebLogic Server domain, lock the WebLogic Server configuration, as described in Section 2.3.2.

    2. In the WebLogic Server Administration Console, change the configuration of machine_2, to point it to the new host:

      From the left pane of the Console, expand Environment and then select Machines. Select machine_2 and select the Configuration tab. Then select the Node Manager tab. Change the Listen Address to the address for Host C. Click Save.

      If you identify the Listen Address by IP address, you must disable Host Name Verification on the Administration Servers that access Node Manager. For more information and instructions, see "Using Hostname Verification" in Administering Security for Oracle WebLogic Server.

    3. Change the Managed Server configuration to point to the new host:

      From the left pane of the Console, expand Environment and then Servers. Then, select the name of the server. Select the Configuration tab, then the General tab.

      Change the Machine to machine_2.

      Change Listen Address to the new host. (If the listening address was set to blank, you do not need to change it.)

    4. Unlock the Oracle WebLogic Server configuration by clicking Release Configuration on the WebLogic Server Administration Console,

  3. Take any additional steps needed for components as described in Table 17-1.

  4. Stop Node Manager as described in Section 4.2.2.

  5. Create a domain template jar file from the Administration Server running in Host A, using the pack utility. For example:

    pack.sh -domain=/scratch/oracle/config/domains/domain_name
      -template=/scratch/temp.jar -template_name=test_install 
      -template_author=myname -log=/scratch/logs/my.log -managed=true
    

    Specifying the -managed=true option packs up only the Managed Servers. If you want to pack the entire domain, omit this option.

  6. Unpack the domain template jar file on Host C, using the unpack utility:

    unpack.sh -template=/scratch/temp.jar
       -domain=/scratch/oracle/config/domains/domain_name
       -log=/scratch/logs/new.log -log_priority=info
    
  7. Ensure that the application artifacts are accessible from the Managed Server host. That is, if the application artifacts are not on the same server as the Managed Server, they must be in a location accessible by the Managed Server.

    Note:

    • For applications that are deployed in nostage and external_stage mode, copy the application artifacts from the Administration Server host directory.

    • For applications that are deployed in stage mode, the Administration server copies the application bits to the staged directories on the Managed Server hosts.

    See Deploying Applications to Oracle WebLogic Server for information about deploying applications.

  8. Update the ListenAddress by using the following WLST commands:

    readDomain('Domain_Home')
    cd('/')
    cd('NMProperties')
    set('ListenAddress','localhost')
    set('ListenPort',9001)
    
  9. Start Node Manager on Host C, if it is not started:

    cd DOMAIN_HOME/bin
    ./startNodeManager.sh
    
  10. Start the Managed Server. For example:

    DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name admin_url 
    

    The Managed Server connects to the Administration Server and updates its configuration changes.

  11. Update Oracle Inventory, as described in Section 17.3.6.4.

  12. On Windows, recover the Windows Registry, as described in Section 17.3.6.5

  13. If your environment contains Oracle HTTP Server, modify the mod_wl_ohs.conf file, as described in Section 17.3.6.2.

Now you can start and stop the Managed Server on Host C using the Administration Server running on Host A.

17.3.5 Recovering After Loss of Component Host

If you lose a host that contains a component (and its Managed Server, if applicable), you can recover most components to the same host or a different host using the procedures described in the following topics:

Some components require additional actions, which are described in the sections listed in Table 17-1.

Table 17-1 Recovery Procedures for Loss of Host for Particular Components

Component Procedure
 
 
 
 
 

Oracle Data Integrator

Section 17.3.5.6

 
 
 
 
 
 
 

Oracle HTTP Server

Section 17.3.5.5.2

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

17.3.5.1 Recovering a Java Component to the Same Host

To recover a Java component to the same host:

  1. Recover the Managed Server, as described in Section 17.3.4.1.

  2. If the component requires additional steps, as noted in Table 17-1, take those steps.

17.3.5.2 Recovering a Java Component to a Different Host

To recover a Java component to a different host, recover the Managed Server, as described in Section 17.3.4.2.

However, note that some components require additional steps, as noted in Table 17-1.

17.3.5.3 Recovering a System Component to the Same Host

To recover a system component, such as Oracle HTTP Server, to the same host:

  • For system components, such as Oracle HTTP Server, in a standalone domain, you recover the domain, as described in Section 17.3.2.

  • For system components, such as Oracle HTTP Server in an Oracle WebLogic Server domain, you recover the domain, as described in Section 17.3.1.

However, note that some components require additional steps, as noted in Table 17-1.

17.3.5.4 Recovering a System Component to a Different Host

To recover a system component, such as Oracle HTTP Server, to a different host:,

  • For system components in a standalone domain, you recover the domain, as described in Section 17.3.2.

  • For system components in an Oracle WebLogic Server domain, you recover the domain, as described in Section 17.3.1.

However, note that most components require additional steps, as noted in Table 17-1.

17.3.5.5 Recovering Web Tier Components to a Different Host

The Web tier consists of Oracle HTTP Server. The following topics describe how to recover these components to a different host:

17.3.5.5.1 Recovering Oracle HTTP Server in a Standalone Domain to a Different Host

To recover Oracle HTTP Server in a standalone domain:

  1. Follow steps 1 through 4 in Section 17.3.2.

  2. Update the configuration files in the following directory:

    (UNIX) DOMAIN_HOME/config/fmwconfig/components/OHS/instance_name
    (Windows) DOMAIN_HOME\config\fmwconfig\components\OHS\instance_name
    

    For example, update the IP Address and host name in httpd.conf, admin.conf, and mod_wl_ohs.conf (if required).

  3. Follow steps 6 through 8 in Section 17.3.2.

17.3.5.5.2 Recovering Oracle HTTP Server in a WebLogic Server Domain to a Different Host

To recover Oracle HTTP Server in an Oracle WebLogic Server domain to a different host:

  1. Recover the domain, as described in Section 17.3.1.

  2. Change the configuration of the Oracle HTTP Server instance that was on Host B:

    1. In Fusion Middleware Control,from the navigation pane, expand the domain, then Web Tier.

    2. Select the Oracle HTTP Server instance, such as ohs1.

    3. From the Oracle HTTP Server menu, select Administration, then Ports Configuration.

    4. For each port in the table, select the port, then click Edit. Change the IP Address.

      Note that if ANY is selected, you do not need to make any changes.

    5. Click OK.

  3. Update the mod_wl_ohs wiring for each Oracle HTTP Server instance:

    1. In Fusion Middleware Control,from the navigation pane, expand the domain, then Web Tier.

    2. Select the Oracle HTTP Server instance, such as ohs1.

    3. From the Oracle HTTP Server menu, select Administration, then mod_wl_ohs Configuration.

    4. In the Locations section, click AutoFill.

      All valid WebLogic Server endpoint locations are displayed.

    5. Click Apply.

  4. Restart any Oracle HTTP Server instances that are not on the failed machine by navigating to that instance and clicking Start Up.

  5. Start the Oracle HTTP Server instances on Host C by navigating to that instance and clicking Start Up.

17.3.5.6 Recovering Oracle Data Integrator to a Different Host

To recover Oracle Data Integrator, following the procedures in one or both of these topics, depending on the failure:

17.3.5.6.1 Recovering Oracle Data Integrator Repository

If the Oracle Data Integrator Repository must be restored to a different host:

  1. Restore the database, as described in Section 17.3.7.

  2. Connect to the restored Oracle Data Integrator repository using ODI Studio. Create a new connection for the master repository to the new database host, as described in "Connecting to the Master Repository" in the Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator.

  3. Edit each of the Work Repositories. Click Connection and edit the connection information so that the JDBC URL points to the new database host containing the work repository.

  4. For the Oracle Data Integrator JEE Agent repository configuration, in the Oracle WebLogic Server configuration, edit the data sources to match the new database host location.

  5. Update the Oracle Data Integrator Standalone Agent Repository configuration using the following WLST offline commands:

    cd ORACLE_HOME/odi/common/bin
    ./wlst.sh
    readDomain('DOMAIN_DIRECTORY')
    cd('/JdbcSystemResource/OdiMasterRepository/JdbcResource/OdiMasterRepository/JDBCDriverParams/NO_NAME_0');
    set('URL','NEW_JDBC_URL_TO_RECOVERED_DB');
    updateDomain();
    exit();
    
17.3.5.6.2 Recovering Oracle Data Integrator Agents to a Different Host

To recover Oracle Data Integrator agents to a different host:

  1. Restore Oracle Data Integrator JEE Agent by restoring the Managed Server, as described in Section 17.3.4.

  2. Restore the Oracle Data Integrator Standalone system component, as described in Section 17.3.5

  3. Use ODI Studio to edit each physical agent's configuration and provide the updated Host Name value and, if changed, the Port value.

  4. Update Oracle Data Integrator Standalone Agent's host and port configuration using the following commands:

    cd ORACLE_HOME/odi/common/bin
    ./wlst.sh
    readDomain('DOMAIN_HOME')
    cd('ODI/ODI_STANDALONE_AGENT_NAME')
    set("ServerListenAddress",'UPDATED_HOST_NAME');
    set("ServerListenPort",'UPDATED_PORT');
    updateDomain();
    exit();
    
  5. Restart the standalone agents and the Oracle Data Integrator applications deployed in Oracle WebLogic Server.

17.3.6 Additional Actions for Recovering Entities After Loss of Host

Depending on the entity that you are recovering, you may need to take additional actions after loss of host. The sections about each entity may require you to follow one or more of the following procedures. If so, that is noted in the section describing how to recover the entity.

The following topics describe the actions you may need to take:

17.3.6.1 Recovering Fusion Middleware Control to a Different Host

To recover Fusion Middleware Control to a different host, update properties using the System MBean Browser:

  1. In Fusion Middleware Control, from the WebLogic Domain menu, select System MBean Browser.

  2. In the System MBean Browser pane, expand Application-Defined MBeans, then emoms.props, then Server: AdminServer, then Application: em, and then Properties.

  3. Click emoms.properties.

  4. In the Attributes pane, select the Operations tab and click setProperty.

  5. Change the value of the following properties to the new host name:

    • oracle.sysman.emSDK.svlt.ConsoleServerHost

    • oracle.sysman.emSDK.svlt.ConsoleServerName

    For example, for Key, enter oracle.sysman.emSDK.svlt.ConsoleServerHost. Then, for value, enter host.example.com:7001_Management_Service.

  6. Click Invoke.

17.3.6.2 Modifying the mod_wl_ohs.conf File

When you recover an Administration Server or a Managed Server to a different host and your environment includes Oracle HTTP Server, you must modify the following file on the new host:

(UNIX) DOMAIN_HOME/config/fmwconfig/components/OHS/ohs_name/mod_wl_ohs.conf
(Windows) DOMAIN_HOME\config\fmwconfig\components\OHS\ohs_name\mod_wl_ohs.conf

Note that with Oracle HTTP Server in a WebLogic Server domain, this directory is in the Domain home of the Administration Server. With Oracle HTTP Server in a standalone domain, this directory is the Domain home of Oracle HTTP Server.

Modify all of the instances of the host name, port, and clusters (elements such as WebLogicHost, WebLogicPort, and WebLogicCluster) entries in that file. For example:

<Location /console>
    SetHandler weblogic-handler
    WebLogicHost Admin_Host
    WeblogicPort Admin_Port
    WLProxySSL ON
    WLProxySSLPassThrough ON
</Location>

17.3.6.3 Creating a New Machine for Certain Components

If the Administration Server has a Listen address, you must create a new machine with the new host name before you start the Administration Server:

Take the following steps:

  1. Create a new machine with the new host name. Use the following WLST commands, in offline mode:

    wls:/offline> readDomain('DOMAIN_HOME')
    wls:/offline/sampledomain> machine = create('newhostname', 'Machine')
    wls:/offline/sampledomain> cd('/Machine/newhostname')
    wls:/offline/sampledomain> nm = create('newhostname', 'NodeManager')
    wls:/offline/sampledomain> cd('/Machine/newhostname/NodeManager/newhostname')
    wls:/offline/sampledomain> set('ListenAddress', 'newhostname')
    wls:/offline/sampledomain> updateDomain()
    wls:/offline/sampledomain> exit()
    
  2. For the Administration Server, set the machine with the new host name, using the following WLST command, in offline mode:

    wls:/offline> readDomain('DOMAIN_HOME')
    wls:/offline/sampledomain> cd ('/Machine/newhostname')
    wls:/offline/sampledomain> machine = cmo
    wls:/offline/sampledomain> cd ('/Server/AdminServer')
    wls:/offline/sampledomain> set('Machine', machine)
    wls:/offline/sampledomain> updateDomain()
    wls:/offline/sampledomain> exit()
    
  3. Set the listen port for the Administration Server:

    wls:/offline/sampledomain> readDomain('DOMAIN_HOME')
    wls:/offline/sampledomain> cd('/Server/AdminServer')
    wls:/offline/sampledomain> cmo.setListenPort(8001)
    wls:/offline/sampledomain> updateDomain()
    wls:/offline/sampledomain> exit()
    
  4. If required, update the Administration Server listen address, using WLST:

    readDomain('DOMAIN_HOME')
    cd('/Server/AdminServer')
    cmo.getListenAddress()
    cmo.setListenAddress('newhostname')
    updateDomain()
    exit()
    

17.3.6.4 Updating Oracle Inventory

For many components, when you recover to a different host, as in the case of loss of host, you must update the Oracle inventory. To do so, execute the following script:

(UNIX) ORACLE_HOME/oui/bin/attachHome.sh
(Windows) ORACLE_HOME\oui\bin\attachHome.cmd

17.3.6.5 Recovering the Windows Registry

When you recover any component to a different host on Windows, as in the case of loss of host, you must import any Windows Registry keys related to Oracle Fusion Middleware to the new host. (You exported the Registry keys in Section 16.3.3.)

Recover the following Registry key.

HKEY_LOCAL_MACHINE\Software\Oracle

In addition, recover each node that begins with Oracle within the following registry keys:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\Services

To import a key that you have previously exported, use the following command:

regedit  /I  FileName

For example:

regedit /I C:\oracleregistry.reg 

You can also use the Registry Editor to import the key. See the Registry Editor Help for more information.

17.3.7 Recovering After Loss of Database Host

If the host that contained your database is lost, you can recover the database using RMAN.

For example:

rman> restore database;
rman> recover database;

For best results, recover the database to the most current state, using point-in-time recovery (if the database is configured in Archive Log Mode.) This ensures that the latest data is recovered. Also, use the same name for the database. Note the following:

See Creating Schemas with the Repository Creation Utility for the schemas used by each component.

For detailed steps about recovering a database and using RMAN, see the Oracle Database Backup and Recovery User's Guide.