10 Back Up and Recover Oracle Fusion Applications

This section describes recommended back up and recovery strategies and procedures for recovering the Oracle Fusion Applications environment from different types of failures and outages, such as data loss or corruption, host failure, or media failure.

The following topics are discussed:

10.1 Introduction to Backup and Recovery

An Oracle Fusion Applications environment can consist of different Oracle Fusion Applications product families. It is built on Oracle Fusion Middleware, which contains Oracle WebLogic Server domains with Java components, such as Oracle SOA Suite, and system components, such as Oracle HTTP Server. It also contains a separate Oracle WebLogic Server domain with Identity Management components, such as Oracle Internet Directory and Oracle Virtual Directory.

Oracle Fusion Applications and some Oracle Fusion Middleware components use Oracle Database instances to store data and metadata.

Oracle Fusion Applications uses at least three databases:

  • A database that holds the Oracle Fusion Applications data.

  • A database that holds the Oracle Identity repository.

The installations of an Oracle Fusion Applications environment are interdependent in that they contain configuration information, applications, and data that are kept in synchronization.

It is, therefore, important to consider the entire Oracle Fusion Applications environment when performing backup and recovery. The entire Oracle Fusion Applications environment should be backed up as soon as it has been installed and configured, then periodically. If a loss occurs, the environment can be restored to a consistent state.

10.2 Back Up and Recovery Tools

To back up and recover the Oracle Fusion Applications environment, the following options can be used:

  • Oracle Enterprise Manager Cloud Control

    With Cloud Control, it is possible to back up and restore the directories and databases that comprise an Oracle Fusion Applications installation. Because Cloud Control uses Oracle Secure Backup to perform Oracle Fusion Applications backups, all backups are done to tape, which is the media supported by Oracle Secure Backup.

  • Any operating system or third-party tool which provides a consistent image back up of the file system, which can be later restored. If the storage has features such as storage snapshot, this is recommended. If the storage system does not provide such features, file copy utilities can be used such as copy, xcopy, tar, or jar. However, these utilities may result in an inconsistent backup unless the system is in a quiesced state.

    In addition, make sure that the utilities:

    For example, for UNIX, tar can be used.

    • Preserve symbolic links

    • Support long file names

    • Preserve the permissions and ownership of the files

    See Modes of Backup for descriptions of online and offline backups.

  • Oracle Recovery Manager (RMAN) to back up database-based metadata repositories and any databases used by Oracle Fusion Applications. With RMAN, full backups or incremental backups can be performed. For information about backing up databases, see the Oracle Database Backup and Recovery User's Guide. For information about recovering databases, see Recover the Databases and Recover the Databases After Loss of Host.

If the backups need to be retained for a longer duration, consider backing up to tape, for example using Oracle Secure Backup.

For optimized backup time and fast restore times for the file system, a storage snapshot feature provided by a storage vendor can be used. Snapshots are point-in-time, read-only copies of the file system. Snapshots normally use copy-on-write mechanisms and hence do not occupy any space at the beginning. When new data is written, the old copy is written to the snapshot. Most of the storage vendors support an unlimited number of snapshots and allow creating snapshots manually or automatically without any user interventions. Snapshots can be taken quickly, thus reducing the time taken when performing a backup. Taking regular snapshots of the Oracle Fusion Applications configuration helps to restore the Oracle Fusion Applications environment quickly in the event of configuration corruption or data loss. Snapshots provide rollback capabilities. When a rollback occurs, any newer snapshots (and clones of newer snapshots) are destroyed, and the active data reverts to the state when the snapshot was taken.

10.3 Back Up the Environment Overview

The environment should be backed up when Oracle Fusion Applications is installed and configured and on a regular basis. It is possible to back up the full environment or only parts of it. The backups can be performed in online or offline mode.

This section includes the following topics:

10.3.1 Modes of Backup

It is possible to back up the Oracle Fusion Applications environment offline or online:

  • With an offline backup, the environment must be shut down before performing the backup. When an offline backup is performed, the Administration Server, all Managed Servers in the domain, and all system components in the Oracle instances should be shut down.

    Back up the environment offline immediately after installation and after applying upgrades.

  • With an online backup, do not shut down the environment before backing up the files. To avoid an inconsistent backup, do not make any configuration changes until the backup is completed. To ensure that no changes are made in the WebLogic Server domain, lock the WebLogic Server configuration, as described in Lock the WebLogic Server Configuration.

    During an online backup, applications can continue to run during the backup, so the business is not affected.

10.3.2 Types of Backups

The Oracle Fusion Applications file system and the Oracle Fusion Applications databases should be backed up.

For the Oracle Fusion Applications file system, a full backup or a partial backup can be performed.

To perform a full backup of the file system, back up the binary files as well as the configuration files.

Binary files are static files and directories that do not change frequently. These include:

  • The Applications base directory, which is the top-level directory containing Oracle Fusion Applications, Middleware homes, and Oracle homes.

  • The Middleware home (MW_HOME). A Middleware home consists of a WebLogic Server home, an Oracle Common home, and optionally one or more Oracle homes and one or more Oracle instances.

  • OraInventory

  • On UNIX, the oraInst.loc file, which is located, by default, in the following directory:

    (Linux) /etc
    (Other UNIX systems) /var/opt/oracle
    
  • On UNIX, the oratab file, which is located in the following directory:

    /etc
    
  • The beahomelist file, which is located at:

    user_home/bea/beahomelist
    

Configuration files are those files that change frequently. Configuration files include:

  • The Applications configuration directory, which is the top-level directory containing domains and Oracle instances.

  • Domain directories of the Administration Server and the Managed Servers. The Oracle Fusion Applications environment can consist of multiple domains (for example, CRMDomain and FinancialDomain). Each domain consists of an Administration Server and one or more Managed Servers.

    Unless stated in the backup recommendations for a particular type of Oracle Fusion application, it is not needed to back up Managed Server directories separately because the Administration Server contains information about all of the Managed Servers in its domain.

  • All Oracle instance homes, which reside by default in the MW_HOME, but can be configured to be in a different location.

For the Oracle Fusion Applications database and related databases, full or incremental backups can be performed. Use Oracle Recovery Manager to back up an Oracle Database instance.

The databases must be synchronized when restoring them.

10.3.3 Recommended Backup Strategy

This section outlines the recommended strategy for performing backups. Using this strategy ensures that the recovery procedures in this book can be performed.

It is very important to store the backups in a secure location, that is, not on the same hardware that contains the Oracle Fusion Applications environment.

  • Perform a full offline backup: Back up the binary files and directories and the configuration files described in Types of Backups. If the Applications base directory is shared, it only needs to be backed up once. If the Applications base directory is not shared, back it up on each host in the Oracle Fusion Applications environment. Perform a full offline backup at the following times:

    • Immediately after installing Oracle Fusion Applications

    • Immediately after an operating system software upgrade

    • Immediately before upgrading the Oracle Fusion Applications environment

    • Immediately after upgrading the Oracle Fusion Applications environment

    • Immediately before patching the Oracle Fusion Applications environment

  • Perform an online backup of configuration files: Back up the configuration files described in Types of Backups. Backing up the configuration files makes possible to restore the environment to a consistent state as of the time of the most recent configuration and metadata backup. The configuration files can be backed up at the following granularity:

    • The Applications configuration directory.

    • The domain.

    • The instance. The files should be backed up in the following directories:

      For the Fusion Applications components domains: instance/domains/host_name/<Product>Domain
      For the Webtier home: instance/CommonDomain_webtier
      For the BI Instance: instance/BIInstance
      

    To avoid an inconsistent backup, do not make any configuration changes until the backup completes.

    Perform an online backup of configuration files at the following times:

    • On a regular basis. Oracle recommends to back up configuration files nightly.

    • Before making configuration changes to an Administration Server, a Managed Server, or application.

    • Immediately after patching the Oracle Fusion Applications environment.

    • After making configuration changes to an Administration Server, a Managed Server, Oracle instance, or application.

    • After a major change to the deployment architecture, such as creating servers or clusters.

  • Perform a full or incremental backup of the databases: Use RMAN to backup the databases.

Consider the following recommendations:

  • To ensure that no changes are made in the WebLogic Server domains, lock the WebLogic Server configuration for all the domains in Oracle Fusion Applications environment, as described in the Lock the WebLogic Server Configuration.

  • When the backup is created, name the archive file with a unique name. Consider appending the date and time to the name. For example, if a backup of the Applications base directory is created on March 2 2016, name the backup:

    ApplBase_backup_030216.tar
    

10.4 Recover the Environment Overview

If the environment suffers from critical failures that involve actual data corruption, data loss, or loss of host, all or part of the environment must be recovered.

This section includes the following topics:

10.4.1 Types of Recovery

Recovery strategies make possible to recover from critical failures that involve actual data corruption, data loss, or loss of host. Depending on the type of loss, the Oracle Fusion Applications environment can be recovered in part or in full. The following can be recovered:

  • The Applications base directory

  • The Oracle WebLogic Server domains containing Oracle Fusion Applications

  • The Administration Server for the Oracle Fusion Applications domains

  • Managed Servers

  • The Middleware home

  • An Oracle home

  • An Oracle instance home

  • Any database used by Oracle Fusion Applications. Use Oracle Recovery Manager (RMAN) to recover an Oracle Database instance. See Recover the Databases and Recover the Databases After Loss of Host.

With Oracle Fusion Applications, all or some of the files can be installed on shared storage. Some of the procedures will differ depending on whether all files are on shared storage. The options are:

  • All files are on a shared file system.

  • All binary files are on a shared file system as well as all Administration Servers. Managed Servers are on local file systems.

10.4.2 Recommended Recovery Strategies

The following are key points about recovery:

  • All or part of theOracle Fusion Applications environment must be offline while performing recovery.

    Stop the relevant processes. The stopped processes depend on the granularity of the recovery. For example, if only one domain is being recovered, shut down the corresponding Administration Server and Managed Servers.

  • Rename existing files and directories before begin restoring the files from backup so that necessary files are not unintentionally overridden.

  • Although, in some cases, it may appear that only one or two files are lost or corrupted, the directory structure should be restored for the entire element, such as an Oracle instance home or a component, rather than just restoring one or two files. In this way, it is more likely to guarantee a successful recovery.

  • If a database needs to be recovered, perform a complete recovery to recover the database to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database as close to the current time as possible. (It is possible to use point-in-time recovery if the database is configured in Archive Log Mode. This is typically a time right before the database failure occurred.) Oracle recommends using Archive Log Mode for production databases.

    The databases must be kept synchronized.

10.5 Prerequisites for Using Cloud Control to Back Up or Restore the Environment

Before backing up or restoring an Oracle Fusion Applications environment using Oracle Enterprise Manager Cloud Control, ensure the following prerequisites are met:

  • Cloud Control 13c Agents must be installed on all hosts in the Oracle Fusion Applications environment where components (such as Oracle WebLogic Server Administration and Managed Servers) are running, and the hosts must be discovered in Cloud Control. (Although it is possible to monitor an Oracle Fusion Applications environment without having Agents on every host, backup and restore requires a local Agent on each host).

  • An Oracle Secure Backup domain target must be discovered in Cloud Control. This requires an Oracle Secure Backup server installation, and all hosts that are part of the Oracle Fusion Applications environment must be configured as Oracle Secure Backup client hosts.

  • A Fusion Instance target representing the Oracle Fusion Applications environment must be discovered in Cloud Control. The Fusion Instance target must be discovered through a local Agent that is running on one of the hosts in the Oracle Fusion Applications environment.

  • A backup configuration must be created using Cloud Control. A backup configuration contains the settings that will be used for the database and file backups, and is required to perform an Oracle Fusion Applications backup. See Configure Cloud Control Backups for more details on creating a backup configuration.

  • Before performing a backup or restoring through Cloud Control, all Oracle WebLogic Server instances and other components must be properly shut down in the Oracle Fusion Applications environment. Refer to Recover After Data Loss, Corruption, or Media Failure and Recover After Loss of Host for details on what components must be shut down and how to shut them down. Optionally, scripts that perform the required shutdown of components on each host can be invoked as user-specified pre-backup or pre-restore scripts within the Cloud Control backup procedure. For more details on user-specified scripts, see Back Up Oracle Fusion Applications Using Cloud Control.

  • If the Oracle Fusion Applications repository or other databases are included in an Oracle Fusion Applications restore, the databases must be started in mounted mode before submitting the restore procedure. If a complete restore of the databases is required, it may be necessary to first perform a SPFILE or controlfile restore or both for the individual databases through the database Perform Recovery wizard, which can be accessed from the Availability menu on the Cloud Control database home page. (Go to the home page of the database that must be restored, select Availability, then select Backup and Recovery, then select Perform Recovery.)

For information about using Cloud Control to back up and recover the Oracle Fusion Applications environment, see Perform a Backup Using Cloud Control and Recover Using Cloud Control.

For more information on installing and configuring Cloud Control, refer to the Oracle Enterprise Manager Cloud Control Basic Installation Guide in the Oracle Enterprise Manager documentation library.

For more information on installing Oracle Secure Backup, refer to the Oracle Secure Backup Installation and Configuration Guide in the Oracle Database Documentation library.

10.6 Backup and Recovery Recommendations for Oracle Fusion Applications

The topics in this section describe backup and recovery recommendations for specific Oracle Fusion Applications components and for components that are related to Oracle Fusion Applications.

Table 10-1 describes what must be backed up and recovered for each Oracle Fusion Applications component.

Depending upon the extent of failure, recovery should be performed at the desired granularity. Note the following:

Table 10-1 describes the backup and recovery recommendations for Oracle Fusion Applications and related components.

Table 10-1 Backup and Recovery Recommendations for Oracle Fusion Applications Components

Component Database Dependencies Backup Recommendation Recovery Recommendations Additional Information

Oracle E-Mail and Web Marketing

None

The Managed Servers to which Email Sending Domain (ESD), Click-Through Daemon (CTD) are deployed

The directory for Bounce-Handling Daemon (BHD)

The Managed Servers to which Email Sending Domain (ESD), Click-Through Daemon (CTD) are deployed

The installation directory for Bounce-Handling Daemon (BHD)

Recover Oracle E-Mail and Web Marketing.

Oracle Enterprise Crawl and Search Framework

The database containing Oracle Fusion Applications schemas

The domain

The database

The domain

The database, if necessary

Recover Oracle Enterprise Crawl and Search Framework.

Oracle Enterprise Scheduler

The databases containing Oracle Fusion Applications, Oracle Enterprise Scheduler, and Oracle Fusion Middleware schemas

The domain and the Oracle home

The databases

The domain home and the Oracle home, as needed

The databases, if necessary.

Recover Oracle Enterprise Scheduler.

Oracle Fusion Applications Technology

The databases containing Oracle Fusion Applications, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

The Applications base directory and the Administration Server domain in which Oracle Fusion Applications Technology is deployed.

Oracle WebCenter Content

The databases.

The Managed Server.

Oracle WebCenter Content.

The databases, if necessary

N/A

Oracle Fusion Customer Relationship Management

The databases containing Oracle Fusion Applications, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

The Applications base directory and the Administration Server domain in which Oracle Fusion CRM is deployed.

Oracle WebCenter Content and the standalone Java servers used for Oracle E-Mail and Web Marketing,.

The databases.

The Managed Server

If needed, the standalone Java servers used for Oracle E-Mail and Web Marketing

Oracle WebCenter Content

The databases, if necessary

Recover Oracle Fusion Customer Relationship Management.

Oracle Fusion Financials

The databases containing the Oracle Fusion Applications, Oracle Essbase, and Oracle Fusion Middleware schemas

The Applications base directory and the Administration Server domain in which Oracle Fusion Financials is deployed.

Oracle WebCenter Content

The databases.

The Managed Server

Oracle WebCenter Content

The databases, if necessary

N/A

Oracle Fusion Human Capital Management

The databases containing Oracle Fusion Applications, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

The Applications base directory and the Administration Server domain in which Oracle Fusion Human Capital Management is deployed.

Oracle WebCenter Content

The databases.

The Managed Server

Oracle WebCenter Content

The databases, if necessary

N/A

Oracle Fusion Incentive Compensation

The databases containing Oracle Fusion Applications, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

The Applications base directory and the Administration Server domain in which Oracle Fusion Incentive Compensation is deployed.

Oracle WebCenter Content

The databases.

The Managed Server

Oracle WebCenter Content

The databases, if necessary

N/A

Oracle Fusion Procurement

The databases containing Oracle Fusion Applications, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

The Applications base directory and the Administration Server domain in which Oracle Fusion Procurement is deployed.

Oracle WebCenter Content

The databases.

The Managed Server

The databases, if necessary

Recover Oracle Fusion Procurement.

For loss of host, see Recover Oracle Fusion Procurement After Loss of Host.

Oracle Fusion Project

The databases containing Oracle Fusion Applications, Oracle Essbase, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

The Applications base directory and the Administration Server domain in which Oracle Fusion Project is deployed.

Oracle WebCenter Content

The databases.

The Managed Server

Oracle WebCenter Content

The databases, if necessary

N/A

Oracle Fusion Setup

The databases containing Oracle Fusion Applications, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

The Applications base directory and the Administration Server domain in which Oracle Fusion Setup is deployed.

Oracle WebCenter Content

The databases.

The Managed Server

Oracle WebCenter Content

The databases, if necessary

N/A

Oracle Fusion Supply Chain Management

The databases containing Oracle Fusion Applications, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

The Applications base directory and the Administration Server domain in which Oracle Fusion Supply Chain Management is deployed.

Oracle WebCenter Content

The databases.

The Managed Server

Oracle WebCenter Content

The databases, if necessary

For loss of host, see Recover Oracle Fusion Supply Chain Management After Loss of Host.

Oracle WebCenter Content

The OCS schema

The domain, the Oracle home, and database containing the OCS schema. If the Vault and WebLayout directories are not located in the domain directory, back up their directories, specified in:

DOMAIN_HOME/ucm/CONTEXT-ROOT/config/config.cfg

The following directory, located in a shared file system:

DOMAIN_HOME/ucm/CONTEXT-ROOT/config

The domain and the shared file system containing the Vault and WebLayout directories, depending on the severity of the failure.

The database, if necessary.

Recover Oracle WebCenter Content.

For loss of host, see Recover Oracle WebCenter Content to a Different Host.

This section includes the following topics:

10.6.1 Backup and Recovery Recommendations for Oracle Fusion Customer Relationship Management

This section describes the Oracle Fusion Customer Relationship Management data that must be backed up and restored.

Configuration Files

Most configuration data is stored in the Oracle WebLogic Server domain.

In addition, Oracle Fusion Customer Relationship Management uses configuration data for Java servers for Oracle E-Mail and Web Marketing. See Backup and Recovery Recommendations for Java Servers for Oracle E-Mail and Web Marketing for information about backing up and recovering Oracle E-Mail and Web Marketing.

Dependencies on Oracle Fusion Middleware Components

Oracle Metadata Services, Oracle Business Intelligence metadata, Oracle Real-Time Decisions, an LDAP provider such as Oracle Internet Directory, Oracle WebCenter Portal (including tagging, group spaces, and the Oracle WebCenter Portal schemas), portlet metadata, Oracle SOA Suite, Oracle ADF user interface, , Oracle WebCenter Content, Oracle Enterprise Crawl and Search Framework metadata and data, including Oracle Secure Enterprise Search, Oracle Product Data Quality

Database Repository Dependencies

The databases containing Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

Backup Recommendations

Back up the Applications base directory and the Administration Server domain in which Oracle Fusion Customer Relationship Management is deployed. Back up the standalone Java servers used for Oracle E-Mail and Web Marketing, as described in Backup and Recovery Recommendations for Java Servers for Oracle E-Mail and Web Marketing.

Back up the databases containing the Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content.

Back up Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recovery Recommendations

Recover the Managed Server to which the application is deployed. If needed, recover the standalone Java servers used for Oracle E-Mail and Web Marketing, as described in Backup and Recovery Recommendations for Java Servers for Oracle E-Mail and Web Marketing.

Depending upon the extent of failure, recovery should be performed at the desired granularity. For the steps to recover Oracle Fusion Customer Relationship Management, including for loss of host, see Recover Oracle Fusion Customer Relationship Management.

Recover Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recover the databases containing the Oracle Fusion Applications, Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content.

If needed, perform a complete recovery to recover the databases to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible. Then reconcile the databases.

10.6.2 Backup and Recovery Recommendations for Oracle Fusion Financials

This section describes the Oracle Fusion Financials data that must be backed up and restored.

Configuration Files

Configuration data is stored in the Oracle WebLogic Server domain.

Dependencies on Oracle Fusion Middleware Components

Oracle Application Development Framework, Oracle BPEL Process Manager, Oracle WebCenter Portal (portlets), Oracle Metadata Services, Oracle WebCenter Content: Imaging, Oracle Enterprise Scheduler, Oracle BI Enterprise Edition, Oracle Essbase, and Applications Technology Group

Dependencies on Third-Party Products

None.

Database Repository Dependencies

The databases containing the Oracle Fusion Applications, the LDAP store, Oracle Essbase, and Oracle Fusion Middleware schemas

Backup Recommendations

Back up the Applications base directory and the Administration Server domain in which Oracle Fusion Financials is deployed.

Back up Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content

Back up the databases containing the Oracle Fusion Applications, Oracle Essbase, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content.

Recovery Recommendations

Recover the Managed Server to which the application is deployed. For information about recovering Oracle Essbase, see Recover Essbase In Clustered Environment After Loss of Host.

Depending upon the extent of failure, recovery should be performed at the desired granularity. See Recover After Data Loss, Corruption, or Media Failure.

Recover Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recover the databases to the most recent point in time, if needed.

If needed, perform a complete recovery to recover the databases to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible. Then, reconcile the databases.

10.6.3 Backup and Recovery Recommendations for Oracle Fusion Human Capital Management

This section describes the Oracle Fusion Human Capital Management data that must be backed up and restored.

Configuration Files

Most configuration data is stored in the Oracle WebLogic Server domain.

Dependencies on Oracle Fusion Middleware Components

Oracle Application Development Framework, Oracle BPEL Process Manager, Oracle WebCenter Portal (portlets), Oracle Metadata Services, Oracle Enterprise Scheduler, Oracle Business Intelligence Publisher, an LDAP provider such as Oracle Internet Directory, Oracle Fusion Middleware Extensions for Applications, Oracle WebCenter Content.

Dependencies on Third-Party Products

None.

Database Repository Dependencies

The databases containing Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content.

Backup Recommendations

Back up the Applications base directory and the Administration Server domain in which Oracle Fusion Human Capital Management is deployed.

Back up Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Back up the databases containing the LDAP store, Oracle Fusion Applications, Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content.

Recovery Recommendations

Recover the Managed Server to which the application is deployed.

Depending upon the extent of failure, recovery should be performed at the desired granularity. See Recover After Data Loss, Corruption, or Media Failure.

Recover Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recover the databases to the most recent point in time, if needed. Note that the databases, including the LDAP store, the database containing Oracle Fusion Human Capital Management schemas and the Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content must be kept synchronized. If one is restored, restore the others to the same point in time.

There may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible. Then, reconcile the databases.

10.6.4 Backup and Recovery Recommendations for Oracle Fusion Supply Chain Management

This section describes the Oracle Fusion Supply Chain Management data that must be backed up and restored.

Configuration Files

Configuration data is stored in the Oracle WebLogic Server domain.

Dependencies on Oracle Fusion Middleware Components

Oracle Application Development Framework, Oracle SOA Suite, Oracle Enterprise Scheduler.

Dependencies on Third-Party Products

None.

Database Repository Dependencies

The databases containing Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content.

Backup Recommendations

Back up the Applications base directory and the Administration Server domain in which Oracle Fusion Supply Chain Management is deployed.

Back up Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Back up the databases containing the Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content.

Recovery Recommendations

Recover the Managed Server to which the application is deployed.

Recover Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recover the databases to the most recent point in time, if needed.

Depending upon the extent of failure, recovery should be performed at the desired granularity. See Recover After Data Loss, Corruption, or Media Failure. For the steps specific to recovering from loss of host, see Recover Oracle Fusion Supply Chain Management After Loss of Host.

10.6.5 Backup and Recovery Recommendations for Oracle Fusion Project

This section describes the Oracle Fusion Project data that must be backed up and restored.

Configuration Files

Most configuration data is stored in the Oracle WebLogic Server domain. In addition, configuration data is stored in the Oracle Essbase database.

Dependencies on Oracle Fusion Middleware Components

Oracle Metadata Services, an LDAP provider such as Oracle Internet Directory, Oracle WebCenter Portal (including tagging, group spaces, and the Oracle WebCenter Portal schemas), Oracle SOA Suite, Oracle Application Development Framework user interface, Oracle Enterprise Crawl and Search Framework metadata and data, Oracle Enterprise Scheduler, Oracle Essbase.

Dependencies on Third-Party Products

None.

Database Repository Dependencies

The databases containing the Oracle Fusion Applications, Oracle Essbase, the LDAP store, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content

Backup Recommendations

Back up the Applications base directory and the Administration Server domain in which Oracle Fusion Project is deployed.

Back up Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Back up the Oracle Essbase database used by Oracle Fusion Project. Back up the databases containing the Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas.

Recovery Recommendations

Recover the Managed Server to which the application is deployed.

Recover Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recover the databases to the most recent point in time, if needed.

Depending upon the extent of failure, recovery should be performed at the desired granularity. See Recover After Data Loss, Corruption, or Media Failure.

10.6.6 Backup and Recovery Recommendations for Oracle Fusion Procurement

This section describes the Oracle Fusion Procurement data that must be backed up and restored.

Configuration Files

Most configuration data is stored in the Oracle WebLogic Server domain. In addition, the certificate file, which is determined by the Profile Option, contains a certificate for SSL connections made to supplier web sites to let requisitioners search or browse catalogs.

Dependencies on Oracle Fusion Middleware Components

Oracle Application Development Framework, Oracle BPEL Process Manager, Oracle B2B, Oracle Fusion Middleware Extensions for Applications, Oracle Enterprise Scheduler, Oracle Business Intelligence Publisher, Oracle Streams, Advanced Queuing (AQ), Java Message Service (JMS), JMS queues with Oracle B2B.

Dependencies on Third-Party Products

CUPS software, a portable printing layer for UNIX systems.

Database Repository Dependencies

The databases containing Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content.

Backup Recommendations

Back up the Applications base directory and the Administration Server domain in which Oracle Fusion Procurement is deployed. Back up the certificate file.

Back up Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Back up the databases containing the Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content.

Recovery Recommendations

Recover the Managed Server to which the application is deployed.

Depending upon the extent of failure, recovery should be performed at the desired granularity. For the steps to recover Oracle Fusion Procurement, see Recover Oracle Fusion Procurement. For the steps specific to recovering from loss of host, see Recover Oracle Fusion Procurement After Loss of Host.

For information about recovering JMS, see Backup and Recovery Recommendations for Oracle WebLogic Server JMS in the Administering Oracle Fusion Middleware.

Recover Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recover the databases to the most recent point in time, if needed.

If needed, perform a complete recovery to recover the databases to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible. Then, reconcile the databases.

10.6.7 Backup and Recovery Recommendations for Oracle Fusion Incentive Compensation

This section describes the Oracle Fusion Incentive Compensation data that must be backed up and restored.

Configuration Files

Configuration data is stored in the Oracle WebLogic Server domain.

Dependencies on Oracle Fusion Middleware Components

Oracle Application Development Framework, Oracle SOA Suite, Oracle Data Integrator, Oracle BI Enterprise Edition, Oracle Enterprise Scheduler, Oracle Enterprise Crawl and Search Framework, Oracle Secure Enterprise Search.

Dependencies on Third-Party Products

None.

Database Repository Dependencies

The databases containing Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content.

Backup Recommendations

Back up the Applications base directory and the Administration Server domain in which Oracle Fusion Incentive Compensation is deployed.

Back up Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Back up the databases containing the Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content.

Recovery Recommendations

Recover the Managed Server to which the application is deployed.

Depending upon the extent of failure, recovery should be performed at the desired granularity. See Recover After Data Loss, Corruption, or Media Failure.

Recover Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recover the databases to the most recent point in time, if needed.

If needed, perform a complete recovery to recover the databases to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible. Then, reconcile the databases.

10.6.8 Backup and Recovery Recommendations for Oracle Fusion Applications Technology

This section describes the Oracle Fusion Applications Technology data that must be backed up and restored.

Configuration Files

Configuration data is stored in the Oracle WebLogic Server domain.

Dependencies on Oracle Fusion Middleware Components

Oracle Application Development Framework, Oracle SOA Suite, Federated Worklist, Oracle WebCenter Portal (including portlets, tagging, group spaces, and forums), Oracle WebCenter Portal, MDS, portlet metadata, Oracle Enterprise Crawl and Search Framework metadata and data, and Oracle Secure Enterprise Search, Oracle BI Enterprise Edition, Oracle Fusion Topology Manager, Applications Technology Group.

Dependencies on Third-Party Products

None.

Database Repository Dependencies

The databases containing Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content.

Backup Recommendations

Back up the Applications base directory and the Administration Server domain in which Oracle Fusion Applications Technology is deployed.

Back up Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Back up the databases containing the Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content.

Recovery Recommendations

Recover the Managed Server to which the application is deployed.

Depending upon the extent of failure, recovery should be performed at the desired granularity. See Recover After Data Loss, Corruption, or Media Failure.

Recover Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recover the databases to the most recent point in time, if needed.

If needed, perform a complete recovery to recover the databases to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible. Then, reconcile the databases.

10.6.9 Backup and Recovery Recommendations for Oracle Fusion Setup

This section describes the Oracle Fusion Setup data that must be backed up and restored.

Configuration Files

Configuration data is stored in the Oracle WebLogic Server domain.

Dependencies on Oracle Fusion Middleware Components

Oracle Application Development Framework, Oracle BPEL Process Manager, Oracle WebCenter Content, Oracle WebCenter Portal (portlets), Oracle Enterprise Scheduler, Identity Governance Framework.

Dependencies on Third-Party Products

None.

Database Repository Dependencies

The databases containing Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content.

Backup Recommendations

Back up the Applications base directory and the Administration Server domain in which Oracle Fusion Setup is deployed.

Back up Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Back up the databases containing the Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content.

Recovery Recommendations

Recover the Managed Server to which the application is deployed.

Depending upon the extent of failure, recovery should be performed at the desired granularity. See Recover After Data Loss, Corruption, or Media Failure.

Recover Oracle WebCenter Content, as described in Backup and Recovery Recommendations for Oracle WebCenter Content.

Recover the databases containing the Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas, including those for Oracle WebCenter Content.

If needed, perform a complete recovery to recover the databases to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible. Then, reconcile the databases.

10.6.10 Backup and Recovery Recommendations for Oracle WebCenter Content

This section describes the Oracle WebCenter Content data that must be backed up and restored.

Configuration Files

DOMAIN_HOME/ucm/CONTEXT-ROOT/bin/intradoc.cfg
DOMAIN_HOME/ucm/CONTEXT-ROOT/config/config.cfg

Database Repository Dependencies

OCS schema.

Backup Recommendations

Back up the domain, the Oracle home, and database containing the OCS schema. If the Vault and WebLayout directories are not located in the domain directory, back up their directories, which are specified in:

DOMAIN_HOME/ucm/CONTEXT-ROOT/config/config.cfg

Also, back up the following directory, which is located in a shared file system:

DOMAIN_HOME/ucm/CONTEXT-ROOT/config

Recovery Recommendations

Restore the domain and the shared file system containing the Vault and WebLayout directories, depending on the severity of the failure.

Recover the database containing the OCS schema to the most recent point in time, if needed.

For the steps to recover Oracle WebCenter Content, see Recover Oracle WebCenter Content. For the steps specific to recovering from loss of host, see Recover Oracle WebCenter Content to a Different Host.

10.6.11 Backup and Recovery Recommendations for Oracle Enterprise Scheduler

This section describes the Oracle Enterprise Scheduler data that must be backed up and restored.

Configuration Files

Configuration data is stored in the Oracle WebLogic Server domain.

Dependencies on Oracle Fusion Middleware Components

None.

Dependencies on Third-Party Products

None.

Database Repository Dependencies

The databases containing Oracle Fusion Applications, Oracle Enterprise Scheduler, LDAP store, and Oracle Fusion Middleware schemas, including Oracle WebCenter Content.

Backup Recommendations

Back up the Oracle home and the domain home.

Back up the databases containing the Oracle Fusion Applications, the LDAP store, and Oracle Fusion Middleware schemas.

Recovery Recommendations

Recover the domain home and the Oracle home, as needed.

Depending upon the extent of failure, recovery should be performed at the desired granularity. For the steps to recover Oracle Enterprise Scheduler, including for loss of host, see Recover Oracle Enterprise Scheduler.

Recover the databases to the most recent point in time, if needed.

If needed, perform a complete recovery to recover the databases to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible. Then, reconcile the databases.

10.6.12 Backup and Recovery Recommendations for Oracle Enterprise Crawl and Search Framework

This section describes the Oracle Enterprise Crawl and Search Framework data that must be backed up and restored.

Configuration Files

Configuration data is stored in the Oracle WebLogic Server domain.

Dependencies on Oracle Fusion Middleware Components

Oracle Application Development Framework.

Dependencies on Third-Party Products

None.

Database Repository Dependencies

The database containing Oracle Fusion Applications schemas.

Backup Recommendations

Back up the domain in which Oracle Enterprise Crawl and Search Framework is deployed.

Back up the database containing the Oracle Fusion Applications schemas.

Recovery Recommendations

Recover the domain in which Oracle Enterprise Crawl and Search Framework is deployed.

Depending upon the extent of failure, recovery should be performed at the desired granularity. For the steps to recover Oracle Enterprise Crawl and Search Framework, including for loss of host, see Recover Oracle Enterprise Crawl and Search Framework.

Recover the database containing the Oracle Fusion Applications schemas.

If needed, perform a complete recovery to recover the databases to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible. Then, reconcile the databases.

10.6.13 Backup and Recovery Recommendations for Java Servers for Oracle E-Mail and Web Marketing

Oracle E-Mail and Web Marketing is provided with Oracle Fusion Customer Relationship Management. Oracle E-Mail and Web Marketing provides three components: Email Sending Domain (ESD), Click-Through Daemon (CTD), and Bounce-Handling Daemon (BHD).

This section describes the Oracle E-Mail and Web Marketing data that must be backed up and restored.

Configuration Files

Configuration data is stored in the Oracle WebLogic Server domain.

Dependencies on Oracle Fusion Middleware Components

None.

Dependencies on Third-Party Products

None.

Database Repository Dependencies

None.

Backup Recommendations

Back up the Managed Servers to which ESD and CTD are deployed. Back up the directory for BHD.

Recovery Recommendations

Restore the Managed Servers to which ESD and CTD are deployed. Restore the directory for BHD.

Depending upon the extent of failure, recovery should be performed at the desired granularity. For the steps to recover Oracle E-Mail and Web Marketing, including for loss of host, see Recover Oracle E-Mail and Web Marketing.

10.7 Perform a Backup

A full offline backup or an online or offline backup of configuration files can be performed.

This section includes the following topics:

10.7.1 Perform a Backup Using Cloud Control

Cloud Control can be used to back up Oracle Fusion Applications, as described in the following topics:

10.7.1.1 Configure Cloud Control Backups

A backup configuration must be created before performing a backup or restoring using Cloud Control. A backup configuration contains settings for database and file backups. Create the configuration using the Backup Configurations page. Multiple backup configurations can be created, using different settings. Then, the configurations can be used in subsequent backup and restore operations.

To create a new backup configuration using Cloud Control, perform the following steps:

  1. From the Targets menu, select Fusion Applications.

    The Fusion Applications target home page is displayed.

  2. Click the Fusion Instance that needs to be backed up.

    The Fusion Instance home page is displayed.

  3. From the Fusion Instance menu, choose Control, then Backup Configurations.

    The Backup Configurations page is displayed, as shown in the following figure:

    Figure 10-1 Backup Configurations Page



  4. Click Create.

    The Create Backup Configurations page is displayed. It contains three tabs: Storage, Policy, and Recovery Catalog. The default settings or customize individual settings can be used.

  5. On the Storage tab, specify settings related to disk and tape backup. Because Cloud Control always backs up Oracle Fusion Applications to tape, the disk settings are not applicable.

    To specify tape settings, first select an Oracle Secure Backup Domain, after which separate tape settings for database and files are shown. Database settings include datafile and archived log copies, backup type, and media management parameters. File settings include media family and devices. Specify Oracle Secure Backup domain and host credentials.

    The following figure shows the Storage tab:

    Figure 10-2 Create Backup Configuration Page



  6. On the Policy tab, specify policy settings for database and tape backups. Database settings include backup retention, compression, and encryption. Separate database encryption settings are specified for disk and tape backups. File settings include Oracle Secure Backup job and encryption settings.

    The following figure shows the Policy tab:

  7. If a recovery catalog is being used for database backups, specify the catalog on the Recovery Catalog tab, which is shown in the following figure:

    Figure 10-4 Recovery Catalog Tab



  8. Click Save.

For more information on the database backup settings in these pages, refer to the Oracle Cloud Oracle Database Backup and Recovery Reference Guide in the Oracle Database documentation library.

For information on the Oracle Secure Backup settings in these pages, refer to the obtool backup command options in the Oracle Secure Backup Reference Guide in the Oracle Database documentation library.

10.7.1.2 Back Up Oracle Fusion Applications Using Cloud Control

By default, Cloud Control backs up the core directories and databases that comprise an Oracle Fusion Applications installation. It can include additional files and databases in the backup. It is also possible to execute user-defined pre-backup and post-backup scripts.

Because Cloud Control uses Oracle Secure Backup to perform the backups, all backups are done to tape, which is the media supported by Oracle Secure Backup.

To back up an Oracle Fusion Applications environment using Cloud Control:

  1. Ensure that the prerequisites have been met, as described in Prerequisites for Using Cloud Control to Back Up or Restore the Environment.

  2. From the Targets menu, choose Fusion Applications.

    The Fusion Applications target home page is displayed.

  3. Click the Fusion Instance that needs to be backed up.

    The Fusion Instance home page is displayed.

  4. From the Fusion Instance menu, choose Control, then Schedule Backup.

    The Schedule Backup wizard is displayed, with the first step, Backup Scope, as shown in the following figure:

    Figure 10-5 Schedule Backup: Backup Scope Page



  5. In the Backup Scope step:

    1. Select the Oracle Secure Backup domain to use for the backup.

    2. Select the scope of the backup among the following list:

      • Software and Configuration Directories:

        Backs up all directories in the Oracle Fusion Applications installation, including the Applications Home directory (the root of the Oracle Fusion Applications installation) and any other directories (for example, Domain homes) that may be installed in a different location. Cloud Control will determine the distinct file systems that comprise the installation across all hosts in the environment, and will include the top-level directories needed to constitute a complete backup.

      • Configuration Directories:

        Backs up only the configuration-related directories, including the Domain home of the Administrative Server for each Oracle WebLogic Server domain. The following options can be selected:

        • All: Back up all configuration directories.

        • Custom: Narrow the scope of the backup by selecting specific Oracle Fusion Applications components. (On the next step, it is possible to select which of their associated directories to include in the backup.)

    Click Next.

  6. In the Directories step, specify the Oracle Fusion Applications directories to include in the backup. By default, the directories that correspond to the backup scope choice (made on the Backup Scope page) are included. If the Custom option is chosen, the directories from the list can be removed. (The list of directories that correspond to the backup scope are automatically determined by Cloud Control. It is not possible to individually select or remove subdirectories from the backup.) Click Add to specify additional files or directories to include in the backup from outside the core Oracle Fusion Applications installation. Any directories or files that reside on a discovered Oracle Secure Backup client host can be included.

    Click Next.

  7. In the Databases step, click Add to specify additional databases to add to the backup. By default, the Oracle Fusion Applications repository database is included in the backup.

    Click Next.

  8. In the Configuration step, select the backup configuration to use for the backup. (See the Configure Cloud Control Backups for details on creating a backup configuration.)

    Click Next.

  9. In the Options step:

    1. Select the backup type:

      • Full: A full backup of the databases and files will be performed. If the database portion of the backup is to be used as the base of an incremental backup strategy, select the Use as the base of an incremental backup strategy option.

      • Incremental: An incremental backup of the databases and files will be performed.

    2. Select the backup mode:

      • Online: The databases will remain running during the backup. Manually quiesce or shut down the Oracle Fusion Applications processes before submitting the backup, as this is not done automatically by Cloud Control.

      • Offline: The databases will be shutdown automatically before the backup. Manually shut down the Oracle Fusion Applications processes before submitting the backup, as this is not done automatically by Cloud Control.

      See Start and Stop an Oracle Fusion Applications Environment for information about stopping Oracle Fusion Applications processes. Alternatively, scripts can be specified to shutdown the processes in Step 10.

    Click Next.

  10. In the Scripts step, specify the host name and path of scripts to be run before and after the backup. Post-backup scripts can be specified to run upon success or failure of the backup. The scripts can be run on any discovered host. The scripts must be executable files that exist on the specified hosts. Any type of executable file can be specified.

    Click Next.

  11. In the Credentials step, if one or more databases is included in the backup, specify the database and host credentials that will be used to back up the databases.

    Click Next.

  12. In the Schedule step, specify the procedure name, description, and scheduling information.

    Click Next.

  13. In the Review step, verify all the input specified in the wizard. Make changes as needed and click Submit. A Cloud Control deployment procedure is submitted to perform the backup, and a confirmation page appears with a link to the deployment procedure status page.

For a description of the backup types, see the Oracle Database Backup and Recovery User's Guide in the Oracle database documentation library.

10.7.2 Perform a Full Offline Backup Using the Command Line

To perform a full offline backup, copy the file system artifacts and database repositories corresponding to Oracle Fusion Applications. Use the preferred tool for archiving and compressing, as described in Back Up the Environment Overview. Ensure that the tool being used preserves the permissions of the files.

To perform a full offline backup:

  1. Stop all processes. See Start and Stop an Oracle Fusion Applications Environment.
  2. Back up the Applications base directory on all hosts. For example:
    (UNIX) tar -cpf ApplBase_backup_030214.tar APPLICATIONS_BASE/*
    
  3. Back up the Applications configuration directory on all hosts. For example:
    (UNIX) tar -cpf ApplConfig_backup_030214.tar APPLICATIONS_CONFIG/*
    
  4. If a domain is not located within the Applications configuration home, back up the domains separately. This backs up the Managed Servers that are running Java components such as Oracle SOA Suite and Oracle WebCenter Portal.

    For example:

    (UNIX) tar -cpf domain_home_backup_030214.tar APPLICATIONS_CONFIG/instance/domains/domain_name/*
    

    In most cases, it is not needed to back up the Managed Server directories separately, because the Administration Server domain contains information about the Managed Servers in its domain. See Recommended Backup Strategy for information about what is needed to back up.

  5. If the Oracle instance home is not located within the Applications base home, back up the Oracle instance home. The Oracle instance home contains configuration information about system components, such as the Global Order Promising component of Oracle Fusion Supply Chain Management.

    For example:

    (UNIX) tar -cpf instance_home_backup_030214.tar ORACLE_INSTANCE/*
    
  6. If a Managed Server is not located within the domain, back up the Managed Server directory. For example:
    (UNIX) tar -cpf man_server1_backup_030214.tar APPLICATIONS_CONFIG/instance/domains/domain_name/servers/server_name/*
    
  7. Back up the OraInventory directory. For example:
    (UNIX) tar -cpf Inven_home_backup_030214 /scratch/oracle/OraInventory
    
  8. On UNIX, back up the OraInst.loc file, which is located in the following directory:
    (Linux) /etc
    (Other UNIX systems) /var/opt/oracle
    
  9. On UNIX, back up the oratab file, which is located in the following directory:
    /etc
    

    The oratab file is located on the database host.

  10. Back up the databases using the Oracle Recovery Manager (RMAN). For detailed steps, see Recover the Databases and Recover the Databases After Loss of Host.
  11. Create a record of the Oracle Fusion Applications environment. See Create a Record of the Oracle Fusion Applications Configuration.

10.7.3 Perform an Online Backup of Configuration Files Using the Command Line

A backup of configuration files should be performed on a regular basis and at the times described in Recommended Backup Strategy.

Follow the next steps to back up configuration files:

  1. To avoid an inconsistent backup, do not make any configuration changes until the backup is completed. To ensure that no changes are made in the WebLogic Server domain, lock the WebLogic Server configuration, as described in Lock the WebLogic Server Configuration
  2. Back up the domain directories. This backs up the Managed Servers that are running Java components such as Oracle SOA Suite and Oracle WebCenter Portal. For example:
    (UNIX) tar -cpf domain_home_backup_030214.tar APPLICATIONS_CONFIG/instance/domains/domain_name/*
    
  3. Back up the Oracle instance home. This backs up the system components, such as Oracle HTTP Server. For example:
    (UNIX) tar -cpf instance_home_backup_030214.tar ORACLE_INSTANCE/*
    
  4. Back up the databases using the Oracle Recovery Manager (RMAN). For detailed steps, see Recover the Databases and Recover the Databases After Loss of Host.

10.8 Create a Record of the Oracle Fusion Applications Configuration

If the Oracle Fusion Applications environment needs to be restored and recovered, it is important to have all the necessary information available. This is especially true in the event of a hardware loss that requires to reconstruct all or part of the Oracle Fusion Applications environment on a new disk or host.

An up-to-date record of the Oracle Fusion Applications environment that includes the information listed in this section should be maintained. This information should be kept both in hardcopy and electronic form. The electronic form should be stored on a host or email system that is completely separate from the Oracle Fusion Applications environment.

The Oracle Fusion Applications hardware and software configuration record should include:

  • The following information for each host in the environment:

    • Host name

    • Virtual host name (if any)

    • Domain name

    • IP address

    • Hardware platform

    • Operating system release level and patch information

  • The following information for each Oracle Fusion Applications installation in the environment:

    • Installation type (for example, Oracle SOA Suite, Oracle Fusion Supply Chain Management)

    • Host on which the installation resides

    • User name, user ID number, group name, group ID number, environment profile, and type of shell for the operating system user that owns the Oracle home (/etc/passwd and /etc/group entries)

    • Directory structure, mount points, and full path for the Applications base directory, Applications configuration directory, Middleware home, Oracle Common home, Oracle homes, Oracle WebLogic Server domain homes, and the Oracle instance homes

    • Amount of disk space used by the installation

    • Port numbers used by the installation

  • The following information for the databases containing the metadata for components and any other databases used by Oracle Fusion Applications:

    • Host name

    • Database version and patch level

    • Base language

    • Character set

    • Global database name

    • SID

  • The following information about backups:

    • The time of the backup

    • The contents of the backup. For example, a full backup, a backup of a domain

    • The tool used to create the backup

    • Where the backup is stored

10.9 Recover Using Cloud Control

An Oracle Fusion Applications environment can be recovered with Cloud Control. Cloud Control restores the directories and databases of a Oracle Fusion Applications backup. All directories and databases of the backup are restored by default. Optionally, individual directories and databases can be selected to be restored.

The restore capability provided by Cloud Control is one part of a larger process that must be performed to recover an Oracle Fusion Applications environment or individual Oracle Fusion Applications components. After restoring Oracle Fusion Applications directories and databases using Cloud Control, additional manual steps must be performed depending on the component that is being restored. Those steps are described in Recover After Data Loss, Corruption, or Media Failure and Recover After Loss of Host.

To recover Oracle Fusion Applications with Cloud Control:

  1. Ensure the prerequisites have been met, as described in Prerequisites for Using Cloud Control to Back Up or Restore the Environment.
  2. From the Targets menu, choose Fusion Applications.

    The Fusion Applications target home page is displayed.

  3. Click the Fusion Instance that to be recovered.

    The Fusion Instance home page is displayed.

  4. From the Fusion Instance menu, choose Control, then Perform Restore.

    The Perform Restore wizard is displayed, as shown in the following figure:

    Figure 10-6 Perform Restore: Select Backup Page



  5. In the Select Backup step, select the backup to restore from the list of available backups for the Fusion Instance target.

    Click Next.

  6. In the Components step, the Oracle Fusion Applications components and databases included in the selected backup are listed. Select one or more components to include in the restore. The directories that correspond to the selected components will be shown on the next page. If multiple components are selected that have common directories, those common directories will be included along with any directories that are specific to individual components.

    Click Next.

  7. In the Directories step, the directories that correspond to the selected Oracle Fusion Applications components are listed in the first table. By default, they will all be restored to their original locations. Directories that should not be included in the restore from the table can be removed. The destination host name and path where the directories will be restored can be changed. (Note that Oracle Fusion Applications directories must be restored to their original paths to successfully recover the environment. However, restoring to different paths is supported to provide the option of restoring to a temporary staging location in preparation for eventual movement to the final location.)

    While complete directories can be removed from the list, individual subdirectories or files under the top-level directories shown in the list cannot be selected or removed. If a more fine-grained restore granularity is desired, individual files and subdirectories can be restored by using the Perform Restore wizard available from the Oracle Secure Backup Domain home page. (From that home page, select Manage, then File System Backup/Restore, and then Perform Restore.)

    Any additional files or directories included in the backup are listed in a separate table, and will be restored to their original locations by default. Directories and files can be removed from the table and the destination host name and path can be changed.

    Click Next.

  8. In the Configuration step, the original backup configuration used to create the backup is displayed. If the backup configuration is still available it will be used by default and no user action is required on this page. If the configuration is not available, select an alternate backup configuration that has settings similar to backup configuration used to create the backup.

    Click Next.

  9. In the Scripts step, specify pre-restore and post-restore scripts that will be executed as part of the restore operation. Scripts can be run on any discovered host. The scripts must be executable files that exist on the specified hosts.

    The Oracle Fusion Applications components being restored must be shut down before being restored. Because this is not done automatically by Cloud Control, the appropriate shutdown scripts can be executed as part of the restore procedure by specifying them as pre-restore scripts.

    Click Next.

  10. In the Credentials step, if one or more databases is included in the restore operation, specify both database and host credentials that will be used to restore the databases.

    Click Next.

  11. In the Schedule step, specify the procedure name, description, and scheduling information.

    Click Next.

  12. In the Review step, verify all the input specified in the wizard. Make changes as needed and click Submit.

10.10 Recover After Data Loss, Corruption, or Media Failure

This section describes recovery strategies for outages that involve actual data loss or corruption, or media failure where the disk cannot be restored. It also describes recovery strategies for applications that are no longer functioning properly. This type of failure requires some type of data restoration before the Oracle Fusion Applications environment can be restarted and continue with normal processing.

Depending on the extent of the failure, the Applications base directory, the Middleware homes, the Administration Server, a Managed Server, or the database can be recovered. Some Oracle Fusion Applications components require additional steps, which are described in subsequent sections.

This section includes the following topics:

This section also includes the following topics which describe additional considerations that may need to be taken for particular components:

For information about recovering Oracle Fusion Middleware components, such as Oracle SOA Suite, see the Recovering Your Environment section in the Administering Oracle Fusion Middleware in the Oracle Fusion Middleware Documentation library.

10.10.1 Recover the Applications Base Directory

To recover an Applications base directory that was corrupted or from which files were deleted:

  1. Stop all relevant processes that are related to Oracle Fusion Applications, such as the Administration Server, Node Manager, Managed Servers, and Oracle instances, as described in Start and Stop an Oracle Fusion Applications Environment.
  2. Recover the Applications base directory from the backup file. For example:
    (UNIX) tar -xf ApplBase_backup_030214.tar
    
  3. Start all relevant processes that run in the Applications base home, as described in Start and Stop an Oracle Fusion Applications Environment.

10.10.2 Recover a Middleware Home

To recover a Middleware home that was corrupted or from which files were deleted:

  1. Stop all relevant processes that run in the Middleware home, such as the Administration Server, Node Manager, Managed Servers, and Oracle instances, as described in Start and Stop an Oracle Fusion Applications Environment.
  2. Recover the Middleware home directory from the backup file. For example:
    cd MW_HOME
    (UNIX) tar -xf mw_home_backup_030214.tar
    
  3. Start all relevant processes that run in the Middleware home, as described in Start and Stop an Oracle Fusion Applications Environment.

10.10.3 Recover an Oracle WebLogic Server Domain

To recover an Oracle WebLogic Server domain that was corrupted or deleted from the file system:

  1. Stop all relevant processes that are related to the domain, such as the Administration Server and Managed Servers, as described in Start and Stop an Oracle Fusion Applications Environment.
  2. Recover the domain directory from the backup file:
    cd DOMAIN_HOME
    (UNIX) tar -xf domain_backup_030214.tar 
    
  3. Start all relevant processes that are related to the domain, as described in Start and Stop an Oracle Fusion Applications Environment.
  4. If the Administration Server cannot start , recover it, as described in Recover the Administration Server Configuration.
  5. If a Managed Server cannot start, recover it, as described in Recover a Managed Server.

10.10.4 Recover an Oracle Home

To recover an Oracle home from the backup file:

  1. Recover the Oracle home to the original directory from a backup file. For example:
    cd ORACLE_HOME
    (UNIX) tar -xf Oracle_home_backup_030214.tar 
    
  2. Restart the Managed Server to which applications are deployed, using the WLST start command. For example:
    wls:/mydomain/serverConfig> start('myserver','Server')
    

10.10.5 Recover an Oracle Instance Home

An Oracle instance home contains configuration information for system components, such as Oracle HTTP Server or Oracle Internet Directory.

This section includes the following topics:

10.10.5.1 Recover After Oracle Instance Home Deleted from File System

To recover an Oracle instance home that was corrupted or deleted from the file system:

  1. Stop all relevant processes by killing all processes related to that Oracle instance.
  2. Recover the Oracle instance home directory from a backup file. For example:
    cd ORACLE_INSTANCE
    (UNIX) tar -xf instance_home_backup_030214.tar 
    
  3. Start all relevant processes related to that Oracle instance:
    opmnctl startall
    

10.10.5.2 Recover After Oracle Instance Home Deregistered

An Oracle instance must be registered with the domain. To recover an Oracle instance home that was deregistered from the domain:

  1. Recover the Oracle instance home directory from a backup file. For example:
    cd ORACLE_INSTANCE
    (UNIX) tar -xf instance_home_backup_030214.tar 
    
  2. Register the Oracle instance, along with all of its components, with the Administration Server, using the opmnctl registerinstance command. For example:
    opmnctl registerinstance -adminHost admin_server_host 
         -adminPort admin_server_port -adminUsername username 
         -adminPassword password
         -oracleInstance ORACLE_INSTANCE_dir -oracleHome ORACLE_HOME_dir
         -instanceName Instance_name -wlserverHome Middleware_Home
    

10.10.6 Recover the Administration Server Configuration

If the Administration Server configuration has been lost because of file deletion or file system corruption, the Administration Server console continues to function if it was already started when the problem occurred. The Administration Server directory is regenerated automatically, except for security information. Consequently, whenever the Administration Server is started , it prompts for a user name and password. To prevent this, the configuration can be recovered.

Caution:

Performing a domain-level recovery can affect other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

To recover the Administration Server configuration:

  1. Stop all processes, including the Administration Server, Managed Servers, and Node Manager, if they are started, as described in Start and Stop an Oracle Fusion Applications Environment.
  2. Recover the Administration Server configuration by recovering the domain home backup to a temporary location. Then, restore the config directory to the following location:
    (UNIX) DOMAIN_HOME/config
    
  3. Start the Administration Server as described in Start and Stop an Oracle Fusion Applications Environment.
  4. Verify that the Administration Server starts properly and is accessible.
  5. Start other processes, as described in Start and Stop an Oracle Fusion Applications Environment.

On the next configuration change, the configuration from the Administration Server is pushed to the Managed Servers. On each Managed Server restart, the configuration is retrieved from the Administration Server.

10.10.7 Recover a Managed Server

For many Oracle Fusion Applications, recover the Managed Server in which the application is deployed. In addition, for some components, additional steps may need to be taken , which are described in Additional Steps for Recovering Particular Oracle Fusion Applications Components and Recover Components Related to Oracle Fusion Applications.

In this scenario, the Managed Server does not operate properly or cannot be started because the configuration has been deleted or corrupted or the configuration was mistakenly changed and cannot ascertain what was changed.

To recover a Managed Server:

  1. If the Administration Server is not reachable, recover the Administration Server, as described in Recover the Administration Server Configuration.

  2. If the Managed Server fails to start or if the file system is lost:

    1. Recover the Middleware home from the backup file, if required. For example:

      (UNIX) tar -xf mw_home_backup_030214.tar 
      
    2. Create a domain template jar file for the Administration Server, using the pack utility.

      For example, on UNIX:

      pack.sh -domain=APPLICATIONS_CONFIG/instance/domains/domain_name
         -template=/scratch/temp.jar -template_name=domain1 
         -template_author=myname -log=/scratch/logs/my.log -managed=true
      

      Specifying the -managed=true option packs up only the Managed Servers. If the entire domain will be packed, omit this option.

    3. Unpack the domain template JAR file, using the unpack utility.

      For example, on UNIX:

      unpack.sh -template=/scratch/temp.jar
         -domain=APPLICATIONS_CONFIG/instance/domains/domain_name
         -log=/scratch/logs/new.log -log_priority=info
      
    4. Ensure that the application artifacts are accessible from the Managed Server host. That is, if the application artifacts are not on the same server as the Managed Server, they must be in a location accessible by the Managed Server.

      Consider the following:

      • For stage mode applications, the Administration Server copies the application bits to the staged directories on the Managed Server hosts.

      • For nostage and external-stage mode applications, ensure that application files are available in the stage directories of the Managed Server.

    5. Start the Managed Server, as described in Start and Stop an Oracle Fusion Applications Environment.

      The Managed Server connects to the Administration Server and updates its configuration changes.

10.10.8 Recover the Databases

The databases used by Oracle Fusion Applications can be recovered using RMAN. The databases at whatever level is appropriate can be recovered by performing a restore and recover of the full database, a tablespace, or a data file.

Oracle Fusion Customer Relationship Management uses the following databases: Oracle Fusion Applications database, Oracle WebCenter Content, and Oracle Identity database. Consistency among the databases must be maintained. If any of these databases are recovered to a different point in time, reconciling the databases may be needed.

If needed, perform a complete recovery to recover the databases to the most current state. However, there may be some situations where there are not all the required logs to accomplish complete recovery. In that situation, perform an incomplete recovery (point-in-time recovery) to recover the database to as close to the current time as possible.

For information about recovering databases, see the Oracle Database Backup and Recovery User's Guide.

10.10.9 Additional Steps for Recovering Particular Oracle Fusion Applications Components

In most cases, to recover Oracle Fusion Applications, a Middleware home, a domain, a server, an Oracle home, or an Oracle instance are recovered, depending on the extent of the failure, as stated in Recover After Data Loss, Corruption, or Media Failure. However, additional steps may need to be taken for particular components.

This section includes the following topics:

10.10.9.1 Recover Oracle Fusion Customer Relationship Management

To recover Oracle Fusion Customer Relationship Management:

  1. Recover the Managed Server to which Oracle Fusion Customer Relationship Management is deployed, as described in Recover a Managed Server.
  2. Recover the Java servers for Oracle E-Mail and Web Marketing, as described in Recover Oracle E-Mail and Web Marketing.
  3. The Oracle Product Data Quality repository (the FUSION_DQ schema) and the customer source must be kept synchronized. If one is restored, the other must be restored to the same point in time. In addition, the Data Quality engine server artifacts in the file system, the Data Quality Admin Configuration data in the customer master (fusion), and the deployment topology must be kept synchronized.

For a complete case study for recovering Oracle Fusion Customer Relationship Management in different recovery scenarios, see A Case Study: Recover Oracle Fusion Customer Relationship Management.

10.10.9.2 Recover Oracle Fusion Procurement

To recover Oracle Fusion Procurement:

  1. Recover the Managed Server to which the application is deployed, as described in Recover a Managed Server.
  2. If a certificate file is used to make SSL connections to supplier's web sites, make sure that the certificate file exists in the location that was specified. If it does not, recover it from the backup file.
  3. Oracle Fusion Procurement is dependent on JMS. For information about recovering JMS, see section Backup and Recovery Recommendations for Oracle WebLogic Server JMS in the Administering Oracle Fusion Middleware.

The Oracle Fusion Procurement is dependent on CUPS software, a portable printing layer for UNIX systems.

10.10.10 Recover Components Related to Oracle Fusion Applications

Components related to Oracle Fusion Applications.

This section includes the following topics:

10.10.10.1 Recover Oracle HTTP Server

To recover Oracle HTTP Server, the Oracle instance that contains Oracle HTTP Server must be recovered, as described in Recover an Oracle Instance Home.

10.10.10.2 Recover Oracle WebCenter Content

To recover Oracle WebCenter Content:

  1. If necessary, restore the database, as described in Recover the Databases.
  2. Restore the domain, as described in Recover an Oracle WebLogic Server Domain.
  3. If the Vault, WebLayout, or Search directories are not located in the domain directory, restore those directories, if necessary. For example, if the Vault directory is located on a shared drive in /net/home/vault, restore it from backup:
    cd /net/home/vault
    tar -xf vault_backup_042014.tar 
    

The database and the shared file system should be restored at the same time. If this cannot be done, the IDCAnalyse utility can be used to determine if there are any inconsistencies between the database and the shared file system. If there are, a manual recovery can be performed using IDCAnalyse.

10.10.10.3 Recover Oracle Enterprise Scheduler

To recover Oracle Enterprise Scheduler:

  1. Recover the domain directory from the backup file, as described in Recover an Oracle WebLogic Server Domain.
  2. Recover the Oracle home to the original directory from the backup file, as described in Recover an Oracle Home.
  3. If necessary, recover the database containing the Oracle Fusion applications and MDS schemas to the most recent point in time, if needed, as described in Recover the Databases.

10.10.10.4 Recover Oracle Enterprise Crawl and Search Framework

To recover Oracle Enterprise Crawl and Search Framework:

  1. Recover the domain directory from the backup file, as described in Recover an Oracle WebLogic Server Domain.
  2. If necessary, recover the database containing schemas related to Oracle Enterprise Crawl and Search Framework, as described in Recover the Databases.
  3. Reconcile the databases.

10.10.10.5 Recover Oracle E-Mail and Web Marketing

To recover Oracle E-Mail and Web Marketing:

  1. Recover the Managed Servers to which the Email Sending Daemon and Click Thru Daemon are deployed, as described in Recover a Managed Server.
  2. Recover the installation directory for the Bounce Handling Daemon.

10.10.10.6 Recover Oracle BI Enterprise Edition

The following topics describe how to recover Oracle BI EE:

When recovering Oracle BI EE, the Oracle BI Presentation Catalog and the Oracle BI EE repository (RPD) must be restored to the same point in time, by using the same backup set.

10.10.10.6.1 Recover Oracle BI Enterprise Edition in a Non-Clustered Environment

This scenario assumes that Oracle BI Enterprise Edition is running in a non-clustered environment.

  1. Restore the following, depending on the extent of the failure:
  2. Recover the Administration Server, as described in Recover the Administration Server Configuration.
  3. Recover the Managed Server, as described in Recover a Managed Server.
  4. If necessary, restore the database containing the Oracle BI EE schemas. See Recover the Databases.
  5. Reconcile the LDAP Database with the Oracle BI EE repository (RPD), as described in Reconcile the LDAP Database with the Oracle BI EE Repository (RPD), and with the Oracle BI Presentation Catalog, as described in Reconcile the LDAP Database with Oracle BI Presentation Catalog.
10.10.10.6.2 Recover Oracle BI Enterprise Edition in a Clustered Environment

This scenario assumes that Oracle BI Enterprise Edition is running in a clustered environment.

To recover Oracle BI EE in a clustered environment:

  1. Restore the following, depending on the extent of the failure:
  2. Recover the Administration Server, as described in Recover the Administration Server Configuration.
  3. Recover all Managed Servers, as described in Recover a Managed Server.
  4. If necessary, restore the database containing the Oracle BI EE schemas, as described in Recover the Databases.
  5. Reconcile the LDAP Database with the Oracle BI EE repository (RPD), as described in Reconcile the LDAP Database with the Oracle BI EE Repository (RPD), and with the Oracle BI Presentation Catalog, as described in Reconcile the LDAP Database with Oracle BI Presentation Catalog.
10.10.10.6.3 Reconcile the LDAP Database with the Oracle BI EE Repository (RPD)

The LDAP database must be reconciled with the Oracle BI EE repository (RPD).

Oracle BI Enterprise Edition provides a method to perform synchronization. Automatic synchronization can be enabled, at all times, or temporarily to perform the synchronization. (See section NQSConfig.INI File Configuration Settings in the System Administrator's Guide for Oracle Business Intelligence Enterprise Edition for information about editing the NQSConfig.ini file.)

  • To enable synchronization:

    1. Edit the following file:

      INSTANCE_HOME/config/OracleBIServerComponent/coreapplication_obis1/NQSConfig.INI 
      

      Set the flag FMW_UPDATE_ROLE_AND_USER_REF_GUIDS to yes.

    2. Restart the servers. The information in the LDAP database and RPD is synchronized.

  • To disable synchronization:

    1. To disable synchronization, edit the following file:

      INSTANCE_HOME/config/OracleBIServerComponent/coreapplication_obis1/NQSConfig.INI 
      

      Set the flag FMW_UPDATE_ROLE_AND_USER_REF_GUIDS to no.

    2. Restart the servers.

10.10.10.6.4 Reconcile the LDAP Database with Oracle BI Presentation Catalog

If the LDAP database is restored to a previous point in time resulting in the LDAP database being behind in time to the Oracle BI Presentation Catalog, use the following command to reconcile the LDAP database with the Oracle BI Presentation Catalog:

runcat.cmd -cmd forgetAccounts

For information about the runcat command, see the help:

./runcat.sh -cmd maintenanceMode -help

10.10.10.7 Recover Oracle Business Intelligence Publisher

To recover Oracle Business Intelligence Publisher:

  1. Recover the Managed Server containing the Oracle Business Intelligence Publisher component, as described in Recover a Managed Server.
  2. If necessary, restore the database containing the Oracle Business Intelligence Publisher schemas, as described in Recover the Databases.

If backup artifacts are restored from different times, then user accounts, user reports, and user permissions revert to the restored version. Restore all artifacts from the same point in time.

10.10.10.8 Recover Oracle Real-Time Decisions

To recover Oracle Real-Time Decisions:

  1. Recover the Managed Server containing the Oracle Real-Time Decisions component, as described in Recover a Managed Server.

If backup artifacts are restored from a different time, the analytic models miss a period of learning, but their intelligence is unaffected.

10.10.10.9 Recover Oracle Essbase

To recover Oracle Essbase:

  1. Recover the Middleware home, as described in Recover a Middleware Home
  2. If the Oracle instance does not reside in the Middleware home, recover it, as described in Recover an Oracle Instance Home

10.10.10.10 Recover Oracle Hyperion Calculation Manager

To recover Oracle Hyperion Calculation Manager:

  1. Recover the Managed Server to which Oracle Hyperion Calculation Manager is deployed, as described in Recover a Managed Server.
  2. If necessary, recover the database to the most recent point in time, or import the artifacts using the Oracle Hyperion Calculation Manager import tool.

10.10.10.11 Recover Oracle Hyperion Financial Reporting

To recover Oracle Hyperion Financial Reporting:

  1. Recover the Oracle Hyperion Financial Reporting Oracle home, as described in Recover a Managed Server.
  2. Recover the Oracle instance, as described in Recover an Oracle Instance Home.
  3. If necessary, recover the database to the most recent point in time.

10.10.10.12 Recover Oracle Hyperion Smart View

To recover Oracle Hyperion Smart View:

  1. Restore the Oracle Hyperion Smart View data files:
    • Microsoft Excel files (XLS and XLSX)

    • Microsoft Word files (DOC and DOCX)

    • Microsoft Powerpoint files (PPT and PPTX)

10.11 Recover After Loss of Host

This section describes recovery strategies after losing the original operating environment. For example, a serious system malfunction or loss of media could happen.

Depending on the extent of the failure, the Applications base directory, the Administration Server, a Managed Server, or the database can be recovered. Some Oracle Fusion Applications components require additional steps, which are described in subsequent sections.

This section includes the following topics:

This section also includes the following topics, which describe additional considerations that may need to be taken for Oracle Fusion Applications components:

10.11.1 Recover the Applications Base Directory After Loss of Host

The Applications base directory can be recovered if the host that contains the directory is lost.

To recover the Applications base directory:

  1. Recover the Applications base directory from the backup file. For example:
    cd APPLICATIONS_BASE
    (UNIX) tar -xf ApplBase_backup_030214.tar
    
  2. Start all relevant processes. That is, start all processes that run in the Applications base home, as described in Start and Stop an Oracle Fusion Applications Environment.

10.11.2 Recover After Loss of Administration Server Host

If a host that contains the Administration Server is lost, it can be recovered to the same host or a different host.

This section includes the following topics:

10.11.2.1 Recover the Administration Server to the Same Host

In this scenario, recover the Administration Server either to the same host after the operating system has been reinstalled or to a new host that has the same host name. For example, the Administration Server is running on Host A and the Managed Server is running on Host B. Host A has failed for some reason and the Administration Server must be recovered.

To recover the Administration Server:

  1. Recover the file system. For example, recover the domain containing the Administration Server, as described in Recover an Oracle WebLogic Server Domain.

  2. Attempt to start the Administration Server, as described in Start and Stop an Oracle Fusion Applications Environment.

    If the Administration Server starts, any further steps need to be taken.

  3. If the Administration Server fails to start, take the following steps on Host A:

    1. Stop all relevant processes. That is, stop all processes that are related to the domain, such as the Administration Server and Managed Servers, as described in Start and Stop an Oracle Fusion Applications Environment.

    2. Recover the domain directory from the backup file:

      cd DOMAIN_HOME
      (UNIX) tar -xf domain_backup_030214.tar 
      

      This restores the Administration Server as well as the Managed Servers in the domain.

    3. Start the Administration Server and Managed Servers, as described in Start and Stop an Oracle Fusion Applications Environment.

    4. Start the Node Manager:

      ORACLE_COMMON_HOME/common/bin/wlst.sh
      wls:/offline> startNodeManager()
      

10.11.2.2 Recover the Administration Server to a Different Host

In this scenario, the Administration Server is running on Host A and the Managed Server is running on Host B. Host A has failed for some reason and the Administration Server must be moved to Host C.

This scenario assumes that the shared location containing the binary files and the Administration Server configuration is intact. If it is not, follow the steps in Recover a Middleware Home through Recover an Oracle Instance Home.

To recover the Administration Server to a different host:

  1. Because a new host has been included, mount the file system to the new host, Host C.

  2. If the Administration Server has a listen address, create a new machine with the new host name, as described in Create a New Machine for the New Host Name.

  3. When moving.jks files to another host, warnings may be received from the host name verification. Those SSL certificates can be regenerated in the new host to set up SSL and configure a custom host name verifier.

    The .jks files are located in the following directory:

    (UNIX) APPLICATIONS_BASE/fusionapps/wlserver_10.3/server/lib 
    
  4. Start the Node Manager on Host C if it was configured on the original host:

    ORACLE_COMMON_HOME/common/bin/wlst.sh
    wls:/offline> startNodeManager()
    
  5. Start the Administration Server, as described in Start and Stop an Oracle Fusion Applications Environment.

  6. If the Managed Servers are on the same failed host as the Administration Server, restore the Managed Servers, as described in Recover a Managed Server to a Different Host.

  7. Start the Managed Servers, as described in Start and Stop an Oracle Fusion Applications Environment.

  8. Ensure that additional application artifacts are available. For example, if the deployment mode is nostage or external-stage, applications may reside in directories outside of the domain directory. Make the application files available to the new Administration Server by copying them from backups or by using a shared disk. The application files should be available in the same relative location on the new file system as on the file system of the original Administration Server.

    If the application is staged, the Administration Server copies the application bits to the staged directories on the Managed Server hosts.

  9. If the environment contains Oracle HTTP Server, modify the FusionVirtualHost_x.conf file, as described in Modify the FusionVirtualHost_x.conf File.

  10. Update Oracle Inventory, as described in Update Oracle Inventory.

  11. Edit the targets.xml file for Fusion Middleware Control, as described in Change the Host Name in the targets.xml File for Fusion Middleware Control.

  12. Oracle Management Service, which is part of Fusion Middleware Control, is on the original host and is recovered to the new host when the Administration Server is restored. Oracle Management Agent connects to Oracle Management Service to monitor certain components. If the environment contains components, such as Oracle Internet Directory and Oracle Virtual Directory, that use Oracle Management Agent, but they are located on a different host, the following steps must be taken on each host containing the components. For example, the Administration Server was on Host A, but is restored, along with Oracle Management Service, to Host B. Oracle Internet Directory is on Host C and Oracle Virtual Directory is on Host D. These steps must be taken on both Host C and Host D.

    1. Edit the following file:

      (UNIX) ORACLE_INSTANCE/EMAGENT/emagent_name/sysman/config/emd.properties
      (Windows)ORACLE_INSTANCE\EMAGENT\emagent_name\sysman\config\emd.properties
      

      Update the following entries, replacing the host name with the new host for the Administration Server:

      emdWalletSrcUrl=http://newhost.domain.com:port/em/wallets/emd 
      REPOSITORY_URL=http://newhost.domain.com:port/em/upload/
      
    2. Shut down and restart the EM Agent process.

      For example, on UNIX:

      cd ORACLE_INSTANCE/EMAGENT/emagent_dir
      ./emctl stop agent
      ./emctl start agent
      ./emctl status agent
      

      For example, on Windows:

      cd ORACLE_INSTANCE\EMAGENT\emagent_dir
      emctl stop agent
      emctl start agent
      emctl status agent
      

      The status command shows the REPOSITORY_URL pointing to the new host.

For information about configuring a custom host name verifier, see section Configure a Custom Host Name Verifier in the Oracle WebLogic Server Administration Console Help.

For information about regenerating SSL certificates, see section Configuring SSL in Administering Security for Oracle WebLogic Server 12c (12.2.1) in the Oracle Fusion Middleware Documentation library.

For descriptions of different ways to restart Managed Servers, depending on how they were configured, see section Restarting a Failed Administration Server in the Administering Server Startup and Shutdown for Oracle WebLogic Server n the Oracle Fusion Middleware d=Documentation library.

10.11.3 Recover After Loss of Managed Server Host

If a host that contains a Managed Server is lost, it can be recovered to the same host or a different host.

This section includes the following topics:

10.11.3.1 Recover a Managed Server to the Same Host

In this scenario, recover a Managed Server to the same host after the operating system has been reinstalled or to a new host that has the same host name. The Administration Server is running on Host A and the Managed Server is running on Host B. Host B failed for some reason and the Managed Server must be recovered to Host B.

  1. Start the Node Manager on Host B:

    ORACLE_COMMON_HOME/common/bin/wlst.sh
    wls:/offline> startNodeManager()
    
  2. Start the Managed Server, as described in Start and Stop an Oracle Fusion Applications Environment.

    If the Managed Server starts, it connects to the Administration Server and updates its configuration changes. Any further steps need to be taken.

  3. If the Managed Server fails to start or if the file system is lost:

    1. Stop the Node Manager:

      ORACLE_COMMON_HOME/common/bin/wlst.sh
      wls:/offline> stopNodeManager()
      
    2. Recover the Middleware home to Host B from the backup file, if required:

      cd MW_HOME
      (UNIX) tar -xf mw_home_backup_030214.tar 
      
    3. Create a domain template jar file for the Administration Server running in Host A, using the pack utility.

      For example, on UNIX:

      pack.sh -domain=APPLICATIONS_CONFIG/instance/domains/domain_name
         -template=/scratch/temp.jar -template_name=domain1
         -template_author=myname -log=/scratch/logs/my.log -managed=true
      

      Specifying the -managed=true option packs up only the Managed Servers. If the entire domain is packed, omit this option.

    4. Unpack the domain template jar file in Host B, using the unpack utility.

      For example, on UNIX:

      unpack.sh -template=/scratch/temp.jar
         -domain=APPLICATIONS_CONFIG/instance/domains/domain_name
         -log=/scratch/logs/new.log -log_priority=info
      
    5. Ensure that the application artifacts are accessible from the Managed Server host. That is, if the application artifacts are not on the same server as the Managed Server, they must be in a location accessible by the Managed Server.

      • For applications that are deployed in nostage or external-stage mode, copy the application artifacts from the Administration Server host directory.

      • For applications that are deployed in stage mode, the Administration Server copies the application bits to the staged directories on the Managed Server hosts.

      If the Node Manager is not started, start it:

      ORACLE_COMMON_HOME/common/bin/wlst.sh
      wls:/offline> startNodeManager()
      
    6. Start the Managed Server, as described in Start and Stop an Oracle Fusion Applications Environment.

      The Managed Server connects to the Administration Server and updates its configuration changes.

10.11.3.2 Recover a Managed Server to a Different Host

In this scenario, the Administration Server is running on Host A and the Managed Server is running on Host B. Host B failed for some reason and the Managed Server must be recovered to Host C.

Recover the Middleware home to the same location as the original.

To recover a Managed Server to a different host:

  1. If the Managed Server configuration files are local to the host (that is, they are not on a shared file system):

    1. Recover the Middleware home for the Managed Server to Host C.

      cd MW_HOME
      (UNIX) tar -xf mw_home_backup_030214.tar 
      

      Note that when the Middleware home is restored, all of the domains are being restored because they are on a shared file system.

    2. Create a domain template jar file from the Administration Server running in Host A, using the pack utility.

      For example, on UNIX:

      pack.sh -domain=APPLICATIONS_CONFIG/instance/domains/domain_name
         -template=/scratch/temp.jar -template_name=domain1
         -template_author=myname -log=/scratch/logs/my.log -managed=true
      

      Specifying the -managed=true option packs up only the Managed Servers. If the entire domain is packed, omit this option.

    3. Unpack the domain template jar file on Host C, using the unpack utility.

      For example, on UNIX:

      unpack.sh -template=/scratch/temp.jar
         -domain=APPLICATIONS_CONFIG/instance/domains/domain_name
         -log=/scratch/logs/new.log -log_priority=info
      

      When recovering to a different domain home, use the -app_dir switch in the unpack command.

  2. Ensure that the application artifacts are accessible from the Managed Server host. That is, if the application artifacts are not on the same server as the Managed Server, they must be in a location accessible by the Managed Server.

    • For applications that are deployed in nostage or external-stage mode, copy the application artifacts from the Administration Server host directory.

    • For applications that are deployed in stage mode, the Administration Server copies the application bits to the staged directories on the Managed Server hosts.

  3. If the Managed Server is not co-located with the Administration Server, take the following steps. (These steps are not needed if the Managed Server is co-located with the Administration Server.)

    1. Edit the nodemanager.domains file, specifying the domain names and domain directories. Use the following format:

      domain_name=domain_directory
      
    2. Start the Node Manager on Host C, if it is not started:

      ORACLE_COMMON_HOME/common/bin/wlst.sh
      wls:/offline> startNodeManager()
      
    3. Using WLST, connect to the Administration Server and then enroll the Node Manager running in the new host with the Administration Server:

      connect('username','password','host:port')
      nmEnroll('APPLICATIONS_CONFIG/instance/domains/domain_name',
        'MW_HOME/wlserver_n/common/nodemanager/instance_name')
      
    4. Change the Managed Server configuration to point to the new host:

      • In the WebLogic Server Administration Console, create a machine, which is a logical representation of the computer that hosts one or more WebLogic Servers, and point it to the new host. (From the Home page, select Machines. Then, click New.) Follow the directions in the Administration Console help.

        If the listen address is identified by using an IP address, the host name verification must be disabled on the Administration Servers that access Node Manager. For more information, see section Using Host Name Verification in Administering Security for Oracle WebLogic Server 12c (12.2.1).

      • Change the Managed Server configuration to point to the new machine. (From the left pane of the Console, expand Environment and then Servers. Then, select the name of the server. Select the Configuration tab, then the General tab. In the Machine field, select the machine to which the server is assigned.)

        Change Listen Address to the new host. (If the listen address was set to blank, it does not need to be changed.)

      These steps only need to be taken once for all Managed Servers on the same host.

  4. When moving .jks files to another host, warnings may be received from the host name verification. Those SSL certificates can be regenerated in the new host to set up SSL and configure a custom host name verifier. See section Configure a Custom Host Name Verifier in the Oracle WebLogic Server Administration Console Help. Also, see section Configuring SSL in Administering Security for Oracle WebLogic Server 12c (12.2.1).

    The .jks files are located in the following directory:

    (UNIX) APPLICATIONS_BASE/fusionapps/wlserver_10.3/server/lib 
    
  5. Start the Managed Server, as described in Start and Stop an Oracle Fusion Applications Environment.

    The Managed Server connects to the Administration Server and updates its configuration changes.

  6. If the environment contains Oracle HTTP Server, modify the FusionVirtualHost_x.conf file, as described in Modify the FusionVirtualHost_x.conf File.

  7. Edit the targets.xml file for Fusion Middleware Control, as described in Change the Host Name in the targets.xml File for Fusion Middleware Control.

Now the Managed Server can be started and stopped on Host C using the Administration Server running on Host A.

10.11.4 Recover the Databases After Loss of Host

If the physical host where the database resides is lost, the database can be recovered using RMAN.

See section Restoring a Database on a New Host in the Oracle Database Backup and Recovery User's Guide to learn how to use RMAN to recover the database in the event of a complete failure on the primary database host computer.

Oracle Fusion Customer Relationship Management uses the following databases: Oracle Fusion Applications database, Oracle WebCenter Content, and Oracle Identity database. Consistency must be maintained among the databases. If any of these databases are recovered to a different point in time, the databases may need to be reconciled.

10.11.5 Additional Actions for Recovering Entities After Loss of Host

Depending on the entity that is being recovered, additional actions may need to be taken after loss of host. The sections about each entity may require one or more of the following procedures to be followed. If so, that is noted in the section describing how to recover the entity.

This section includes the following topics:

10.11.5.1 Change the Host Name in the targets.xml File for Fusion Middleware Control

When recovering a component to a different host, the targets.xml file for Fusion Middleware Control must be updated. The file is located at:

(UNIX) APPLICATIONS_CONFIG/instance/domains/hostname/domain_name/sysman/state/targets.xml

In the file, change the host name to the new host name for components that are recovered to a different host. The following shows an example of the targets.xml file:

<Targets>
<Target TYPE="oracle_ias_farm" NAME="Farm_CRMDomain" DISPLAY_NAME="Farm_CRMDomain">
     <Property NAME="MachineName" VALUE="example.domain.com"/>
     <Property NAME="Port" VALUE="13001"/>
     <Property NAME="isLocal" VALUE="true"/>
     <Property NAME="Protocol" VALUE="t3"/>
     <Property NAME="serviceURL" VALUE="service:jmx:t3://example.domain.com:13001/jndi/weblogic.management.mbeanservers.domainruntime"/>
     <Property NAME="WebLogicHome" VALUE="/scratch//appsbase/fusionapps/wlserver_10.3"/>
      <Property NAME="DomainHome" VALUE="/scratch//appsbase/instance/domains/example.domain.com/CRMDomain"/>
</Target>

10.11.5.2 Modify the FusionVirtualHost_x.conf File

When an Administration Server or a Managed Server is recovered to a different host and the environment includes Oracle HTTP Server, the FusionVirtualHost_x.conf file must be modified on the new host. There is a separate file for each domain, for example FusionVirtualHost_fin.conf. The files are located in:

(UNIX) APPLICATIONS_CONFIG/instance/CommonDomain_webtier/config/OHS/ohs_name/moduleconf/FusionVirtualHost_x.conf

Modify all of the instances of the host name and clusters (elements such as WebLogicHost and WebLogicCluster) entries in that file. For example:

<Location /console>
    SetHandler weblogic-handler
    WebLogicHost Admin_Host
    WeblogicPort Admin_Port
    WLProxySSL ON
    WLProxySSLPassThrough ON
</Location>
 .
 .
 .
<Location /soa-infra>
   SetHandler weblogic-handler
   WebLogicCluster SOAHOST1:8001,*SOAHOST2*:*8001*
   WLProxySSL ON
   WLProxySSLPassThrough ON
</Location> 

10.11.5.3 Create a New Machine for the New Host Name

If the Administration Server has a listen address, a new machine must be created with the new host name and set the listen address, before starting the Administration Server. A machine is a logical representation of the computer that hosts one or more WebLogic Servers.

To create a new machine:

  1. Create a new machine with the new host name. Use the following WLST commands, in offline mode:
    readDomain('Domain_Home')
    machine = create('newhostname', 'Machine')
    cd('/Machine/newhostname')
    nm = create('newhostname', 'NodeManager')
    
    cd('/Machine/newhostname/NodeManager/newhostname')
    set('ListenAddress', 'newhostname')
    updateDomain()
    
  2. For the Administration Server, set the machine with the new host name, using the following WLST command, in offline mode:
    readDomain('DomainHome')
    cd ('/Machine/newhostname')
    machine = cmo
    cd ('/Server/AdminServer')
    set('Machine', machine)
    updateDomain()
    
  3. Set the listen address for the Administration Server:
    readDomain("DomainHome")
    cd("servers/AdminServer")
    cmo.setListenPort(8001)
    updateDomain()
    exit()
    

10.11.5.4 Update Oracle Inventory

For many components, when recovering to a different host, as in the case of loss of host, the Oracle Inventory must be updated on UNIX. To do so, execute the following script:

ORACLE_COMMON_HOME/oui/bin/attachHome.sh 

In addition, beahomelist must be updated to edit the location of a Middleware home. Edit the following file to update the Middleware home information:

(UNIX) user_home/bea/beahomelist

10.11.6 Recover Oracle Fusion Applications After Loss of Host

In most cases, to recover Oracle Fusion Applications, the entire Middleware home, a domain, a server, an Oracle home, or an Oracle instance, must be recovered depending on the extent of the failure. However, additional steps may need to be taken for particular components.

This section includes the following topics:

10.11.6.1 Recover Oracle Fusion Supply Chain Management After Loss of Host

If a host that contains Oracle Fusion Supply Chain Management is lost, it can be recovered to the same host or a different host.

To recover Oracle Fusion Supply Chain Management to the same host, recover the Managed Server to which the application is deployed, as described in Recover After Loss of Managed Server Host.

To recover Oracle Fusion Supply Chain Management to a different host:

  1. Recover the Managed Server to which the application is deployed, as described in Recover After Loss of Managed Server Host.
  2. Recover the Oracle instance for Global Order Promising, as described in Recover an Oracle Instance Home.

10.11.6.2 Recover Oracle Fusion Procurement After Loss of Host

If a host that contains Oracle Fusion Procurement is lost, it can be recovered to the same host or a different host.

To recover Oracle Fusion Procurement to the same host, recover the Managed Server to which the application is deployed, as described in Recover After Loss of Managed Server Host.

To recover Oracle Fusion Procurement to a different host:

  1. Recover the Managed Server to which the application is deployed, as described in Recover After Loss of Managed Server Host.
  2. Ensure that Oracle Business Intelligence Publisher and CUPS IP address and port number reflects the different host.
  3. Oracle Fusion Procurement is dependent on JMS. For information about recovering JMS, see section Backup and Recovery Recommendations for Oracle WebLogic Server JMS in the Administering Oracle Fusion Middleware.

10.11.7 Recover Components Related to Oracle Fusion Applications

In most cases, to recover components related to Oracle Fusion Applications, the entire Middleware home, a domain, a server, an Oracle home, or an Oracle instance, must be recovered depending on the extent of the failure. Additional steps may need to be taken for particular components related to Oracle Fusion Applications.

Table 10-2 provides links to the procedures for recovering after loss of host.

10.11.7.1 Recover Oracle HTTP Server After Loss of Host

To recover Oracle HTTP Server to the same host, recover the Oracle instance, as described in Recover an Oracle Instance Home.

To recover Oracle HTTP Server to a different host:

  1. Recover the Middleware home, as described in Recover a Middleware Home.
  2. Start all relevant processes, as described in Start and Stop an Oracle Fusion Applications Environment.
  3. Update the registration of the Oracle instance with the Administration Server, using the opmnctl updateinstanceregistration command on the new host. For example:
    opmnctl updateinstanceregistration -adminHost admin_server_host 
    

    This command updates the OPMN instance.properties file.

  4. Update the registration of the component with the Administration Server, using the opmnctl updatecomponentregistration command on the new host. For example, to update the registration for Oracle HTTP Server, use the following command:
    opmnctl updatecomponentregistration -Host new_host -Port nonSSLPort
        -componentName ohs1 -componentType OHS
    
  5. Edit the targets.xml file for Fusion Middleware Control, as described in Change the Host Name in the targets.xml File for Fusion Middleware Control.
  6. Modify the ServerName entry in the following file to have the new host name:
    (UNIX) ORACLE_INSTANCE/config/OHS/ohs_name/httpd.conf
    
  7. Modify the FusionVirtualHost_x.conf file, as described in Modify the FusionVirtualHost_x.conf File.
  8. If the Managed Server is in a cluster, modify the following files (there is more than one file, with the name ear_name.ear):
    (UNIX) APPLICATIONS_CONFIG/fusionapps/applications/domain_name/deploy/ear_name.ear/APP-INF/classes/wf_client_config.xml
    

    Change the rootEndPointURL element so that it points to the Oracle HTTP Server or Load Balancer for the cluster on the new environment.

10.11.7.2 Recover Oracle WebCenter Content to a Different Host

To recover Oracle WebCenter Content to a different host:

  1. If the database must be restored to a different host, restore it, as described in Recover Databases for Oracle Fusion Customer Relationship Management.
  2. Restore the domain, as described in Recover an Oracle WebLogic Server Domain.
  3. If the Vault, WebLayout, or Search directories are not located in the domain directory, restore those directories, if necessary. For example, if the Vault directory is located on a shared drive in /net/home/vault, restore it from backup:
    cd /net/home/vault
    tar -xf vault_backup_042014.tar 
    
  4. Edit the following file:
    DOMAIN_HOME/ucm_domain/ucm/cs/config/config.cfg
    

    In the file, change the HttpServerAddress setting to specify the new host. For example:

    HttpServerAddress=hostname:port_number
    

The database and the shared file system should be restored at the same time. If this cannot be done, the IDCAnalyse utility can be used to determine if there are any inconsistencies between the database and the shared file system. If there are, a manual recovery can be performed using IDCAnalyse.

10.11.7.3 Recover Oracle Enterprise Crawl and Search Framework After Loss of Host

To recover Oracle Enterprise Crawl and Search Framework to a different host, follow the procedure in Recover Oracle Enterprise Crawl and Search Framework.

10.11.7.4 Recover Oracle E-Mail and Web Marketing After Loss of Host

To recover Oracle E-Mail and Web Marketing, follow the procedure in Recover Oracle E-Mail and Web Marketing.

10.11.7.5 Recover Essbase In Clustered Environment After Loss of Host

To recover after loss of host, the entire Middleware home, a domain, a server, an Oracle home, or an Oracle instance, must be recovered depending on the extent of the failure.

However, if the failed node contained Oracle Essbase and was clustered, secondary instances of Oracle Essbase Agent must be configured so that they are distributed for high availability. For more information, see Configuring and Validating Oracle Essbase Clustering in the Oracle Fusion Applications Installation Guide.

10.11.7.6 Recover Oracle BI Enterprise Edition to a Different Host

10.11.7.6.1 Recover Oracle BI EE to a Different Host in a Non-Clustered Environment

The steps taken to recover Oracle BI EE to a different host depend on the operating system. The host must have the same name as the original host.

10.11.7.6.2 Recover Oracle BI EE to a Different Host in a Clustered Environment

In this scenario, the Oracle BI EE cluster is on two hosts, Host A and Host B. Host A contains instance1 and Host B contains instance2. Host A must be replaced for some reason, such as a host crash, and it must be recovered to Host C and the system must be scaled out so that Host C contains instance3.

Take the following steps:

  1. Restore the Middleware home from backup, as described in Recover a Middleware Home, overwriting the Middleware home created with the new installation.

  2. Restore the database containing the Oracle BI EE schemas, if necessary. See Recover the Databases After Loss of Host.

  3. If the failed node contained the Administration Server, recover it, as described in steps 1 through 5 in Recover the Administration Server to a Different Host.

  4. Scale out the Oracle BI EE system, as described in section Scaling Out the BI System on APPHOST2 in the Enterprise Deployment Guide for Oracle Business Intelligence.

    Consider the following:

    • When entering the directory specifications for the Domain Home and Applications Home, enter specifications for directories that do not yet exist or that are empty.

    • If the Domain Home field is empty, update the following file with the domain directory:

      MW_HOME/wlserver_10.3/common/nodemanager/nodemanager.domains
       
      

      Before starting Node Manager, take the following steps:

      1. Stop Node Manager, if it is running.

      2. Run the setNMProps.sh script, which is located in the ORACLE_COMMON_HOME/common/bin directory, to set the StartScriptEnabled property to true before starting Node Manager:

        cd ORACLE_COMMON_HOME/common/bin
        ./setNMProps.sh
        
      3. Restart Node Manager and enable dynamic registration using the following commands:

        cd WL_HOME/server/bin 
        export JAVA_OPTIONS=-DDomainRegistrationEnabled=true
        ./startNodeManager.sh
        

        It is important to set -DDomainRegistrationEnabled=true whenever a Node Manager which must manage the Administration Server is started. If there is no Administration Server on this computer, and if this computer is not an Administration Server failover node, then Node Manager can be started as follows:

        ./startNodeManager.sh
        
  5. Scale out the system components, as described in section Scaling Out the System Components in the Enterprise Deployment Guide for Oracle Business Intelligence. Fusion Middleware Control prompts to restart the instances after changing their configuration. Restart the instances.

    Because instance1 on Host A is no longer available, its count of BI Servers, Presentation Services, and JavaHosts must be modified to be 0. Fusion Middleware Control prompts to restart the instances after changing their configuration. Restart the instances.

  6. Make instance2 the primary instance and instance3 the secondary instance using Fusion Middleware Control:

    1. Make instance 2 the primary instance and specify the secondary instance as none. Activate and restart the instance as prompted by Fusion Middleware Control.

    2. Make instance3 the secondary instance. Activate and restart the instance as prompted by Fusion Middleware Control.

    See section Configuring Secondary Instances of Singleton System Components in the Enterprise Deployment Guide for Oracle Business Intelligence for more information.

  7. Set the listen address of the bi_servern Managed Server, as described in section Setting the Listen Address for the bi_server2 Managed Server in the Enterprise Deployment Guide for Oracle Business Intelligence.

  8. Disable host name verification for the bi_servern Managed Server, as described in section Disabling Host Name Verification for the bi_server2 Managed Server in the Enterprise Deployment Guide for Oracle Business Intelligence.

  9. Depending on the configuration, perform additional configuration, as described in section Performing Additional Configuration for Oracle Business Intelligence Availability in the Enterprise Deployment Guide for Oracle Business Intelligence.

  10. If Oracle HTTP Server is installed, set the front-end HTTP host and port for the Oracle WebLogic Server cluster to ensure that Oracle BI Search URLs are set correctly, as described in section Setting the Frontend URL for the Administration Console in the Enterprise Deployment Guide for Oracle Business Intelligence.

  11. Configure Node Manager for the Managed Servers as described in section Configuring Node Manager for the Managed Servers in the Enterprise Deployment Guide for Oracle Business Intelligence.

  12. Start the Oracle BI EE Managed Server and all of the OPMN-managed components.

10.11.7.6.3 Additional Steps for Recovering Oracle BI EE

Depending on the environment, additional steps may need to be taken after performing the steps in Recover Oracle BI EE to a Different Host in a Clustered Environment:

  • If the failed host contained the master BI Server, primary cluster controller, and primary Oracle BI Scheduler and the new instance is the master BI Server, take the following steps as appropriate. If instance2 is left as the master BI server, no additional steps need to be taken.

    • If the master BI Server is lost:

      1. Stop Oracle WebLogic Server and OPMN processes on all nodes.

      2. Update the following configuration file to designate a new master BI Server:

        DOAMIN_HOME/config/fmwconfig/biee-domain.xml
        

        In the section <AvailabilityOptions>, edit the following:

        masterBIServerOracleInstanceId="instance_name"
        masterBIServerComponentId="component_id"
        
        

        Also update the following settings:

        <OracleInstance id="instance1" host="hostname"
        instanceHome="path_to_instance_home" opmnLocalPort="9500" opmnRemotePort="number"> 
        <SchedulerOptions dataSource="(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname(PORT=number)))
        
      3. Copy the file to the other host.

      4. Restart the Administration Server and the Managed Servers.

    • If the primary cluster controller or scheduler is lost, it fails over to the standby cluster controller or scheduler. It must be determined whether to reconfigure it to be the primary cluster controller or scheduler or leave it as secondary that has been activated because the primary components have failed. For more information, see section Configuring Secondary Instances of Singleton System Components in the Enterprise Deployment Guide for Oracle Business Intelligence.

  • If the failed host contained the BI Server, the secondary cluster controller, and the secondary Oracle BI Scheduler, designate the new host as the secondary cluster controller or scheduler.

  • If the failed host contained the BI Server and system components such as BI Presentation Services and BI Java hosts, no additional steps are needed.

  • If the failed host contained the following related components, recover them:

10.11.7.7 Recover Oracle Business Intelligence Publisher to a Different Host

To recover Oracle Business Intelligence Publisher to a different host:

  1. Recover the Managed Server containing the Oracle Business Intelligence Publisher component, as described in Recover After Loss of Managed Server Host.

  2. Restore the database containing the Oracle Business Intelligence Publisher schemas, if necessary. See Recover the Databases After Loss of Host.

  3. Modify the server value for Oracle BI Presentation Services:

    1. Open the BI Publisher application at http://hostname:port/xmlpserver and log in.

    2. Click Administration, then Integration, then Oracle BI Presentation Services.

    3. Change Server to the new host name.

    4. Click Apply.

  4. To transform Oracle BI Publisher to work in a Cold Failover Cluster environment, the BI Scheduler 's JMS configuration must be changed:

    1. In the BI Publisher application, click Administration.

    2. In the System Administration section, click Scheduler Configuration.

    3. Change Weblogic JNDI URL to the new host URL. For example, t3://hostname:port.

    4. Click Apply.

  5. If a Cold Failover Cluster is being used, configure the Managed Server to listen on the virtual IP address. See section Transforming Oracle WebLogic Managed Servers in the High Availability Guide. Then restart the Managed Server using the Administration Console or the WLST command line.

  6. In BI Publisher, data sources that refer to this BI Enterprise Edition instance should change or be created (if new using the new virtual host). To change the data sources:

    1. In the BI Publisher application:

    2. Click JDBC Connection under Data Sources.

    3. Edit any data source for BI Enterprise Edition for this instance to reflect the values for the new host.

If backup artifacts are restored from different time, then user accounts, user reports, and user permissions revert to the restored version. Restore all artifacts from the same point in time.

10.11.7.8 Recover Oracle Real-Time Decisions to a Different Host

To recover Oracle Real-Time Decisions to a different host:

  1. Recover the Managed Server containing the Oracle Real-Time Decisions component, as described in Recover After Loss of Managed Server Host.

Note that if backup artifacts are restored from different time, the analytic models miss a period of learning, but their intelligence is unaffected.

10.11.7.9 Recover Oracle Hyperion Calculation Manager After Loss of Host

To recover Oracle Hyperion Calculation Manager after loss of host, follow the procedure in Recover Oracle Hyperion Calculation Manager.

If the database must be restored, restore the database and import Calculation Manager rules and rule sets from the exported file.

In Calculation Manager, select File, and then Import.

10.11.7.10 Recover Oracle Hyperion Financial Reporting After Loss of Host

If a host that contains Oracle Hyperion Financial Reporting is lost, it can be recovered to the same host or a different host. To recover Oracle Hyperion Financial Reporting, recover the Oracle home, as described in Recover an Oracle Home and the Oracle instance, as described in Recover an Oracle Instance Home.

If the database host has changed, update the host name using the following commands:

Epmsys_registry updateproperty financial_reporting_product/@host new_host
Epmsys_registry updateproperty financial_reporting_product/logical_web_app/@host new_host
Epmsys_registry updateproperty financial_reporting_product/logical_web_app/financial_reporting_web_app@host new_host

10.12 A Case Study: Recover Oracle Fusion Customer Relationship Management

This section provides a case study of recovering Oracle Fusion Customer Relationship Management and all of the other Oracle Fusion Applications offerings and Oracle Fusion Middleware components that are installed when Oracle Fusion Customer Relationship Management is installed.

All or part of theOracle Fusion Applications environment can be recovered, as described in Recover the Environment Overview.

This section includes the following topics:

10.12.1 The Recovery Case Study Scenario

In this case study, the following domains were created when Oracle Fusion Customer Relationship Management was installed and provisioned. The Oracle Fusion application files are installed in shared storage, which is mounted on all hosts that run the applications. A single copy of binary files is shared across multiple hosts. The Administration Server configuration is also shared. However, the Managed Server configurations are not shared.

Oracle Fusion Applications Component Domain Name

Oracle Fusion Customer Relationship Management

CRMDomain

Oracle Fusion Financials

FinancialDomain

Oracle Fusion Human Capital Management

HCMDomain

Oracle Fusion Supply Chain Management

SCMDomain

Oracle Fusion Setup

CommonDomain

In addition, Oracle Fusion Customer Relationship Management uses the following Oracle Fusion Middleware components:

  • Oracle WebCenter Portal

  • Oracle Business Intelligence

  • Oracle Essbase

  • Oracle Enterprise Scheduler

  • Oracle SOA Suite

  • Oracle WebCenter Content

  • Oracle Application Development Framework

Oracle Fusion Customer Relationship Management is installed in the following directory:

/scratch/oracle/APPLICATIONS_BASE

The APPLICATIONS_BASE directory contains:

  • The Oracle Fusion Applications Middleware home directory, which is located at:

    /scratch/oracle/APPLICATIONS_BASE/fusionapps
    
  • One Middleware home that hosts the Java EE components, including Oracle SOA Suite and Oracle WebCenter Portal, and is located at:

    /scratch/oracle/APPLICATIONS_BASE/fusionapps
    
  • A second Middleware home that hosts system components such as Oracle HTTP Server and is located at:

    /scratch/oracle/APPLICATIONS_BASE/webtier_mw_home
    

In this case study,

10.12.2 Recover the Middleware Home Containing Java EE Components

In this scenario, the Oracle Fusion Applications Middleware home, which contains Oracle Fusion Applications instances, and the Java EE components, Oracle SOA Suite and Oracle WebCenter Portal is recovered, after data loss or corruption.

To recover the Middleware home:

  1. Stop all relevant processes using Fusion Applications Control. That is, stop all Managed Servers that are related to Oracle SOA Suite and Oracle WebCenter Portal. For example, stop the FINSOAServer_1 (in the FinancialDomain), CRMSOAServer_1 (in the CRMDomain), SCMSOAServer_1 (in the SCMDomain), and FSSOAServer (in the HCMDomain). Stop the Administration Server, as described in Start and Stop an Oracle Fusion Applications Environment.
  2. From the original Middleware home directory, move the files to a backup directory:
    mv /scratch/oracle/APPLICATIONS_BASE/fusionapps
        /scratch/oracle/APPLICATIONS_BASE/fusionapps_backup
    

    This ensures that any needed files are not overwritten.

  3. Create a staging directory and restore the Applications base directory to that directory:
    cd /scratch/stage/
    tar -xvf crm_ApplBase_backup_030214.tar
    
  4. From that stage directory, copy the Middleware home to the original location:
    cp /scratch/stage/APPLICATIONS_BASE/fusionapps
       /scratch/oracle/APPLICATIONS_BASE/fusionapps
    
  5. Start all relevant processes, as described in Start and Stop an Oracle Fusion Applications Environment.

10.12.3 Recover the Web Tier Middleware Home

In this scenario, the Web tier Middleware home that contains system components, such as Oracle HTTP Server is recovered, after data loss or corruption.

To recover the Middleware home:

  1. Stop all relevant processes, such as all processes running in the Oracle instance, using the following command:
    opmnctl stopall
    
  2. From the original Middleware home directory, move the files to a backup directory:
    mv /scratch/oracle/APPLICATIONS_BASE/webtier_mw_home
      /scratch/oracle/APPLICATIONS_BASE/webtier_mw_home_backup
    

    This ensures that any needed files are not overwritten.

  3. Create a staging directory and restore the Applications base directory to that directory:
    cd /scratch/stage
    tar -xvf crm_ApplBase_backup_030214.tar
    
  4. From that stage directory, copy the Middleware home to the original location:
    cp /scratch/stage/APPLICATIONS_BASE/webtier_mwhome
      /scratch/oracle/APPLICATIONS_BASE/webtier_mwhome
    
  5. Start all relevant processes, using the following command:
    opmnctl startall
    

10.12.4 Recover the Oracle Fusion Customer Relationship Management Domain

In this scenario, the domain that contains Oracle Fusion Customer Relationship Management is recovered after data loss or corruption. This procedure shows recovering the domain CRMDomain, but the same procedure can be used for other Oracle Fusion Applications domains.

To recover the domain:

  1. Stop all Managed Servers in the CRMDomain, as described in Start and Stop an Oracle Fusion Applications Environment.
  2. Stop the Administration Server for the CRMDomain, as described in Start and Stop an Oracle Fusion Applications Environment.
  3. From the original domain directory, move the files to a backup directory:
    mv /scratch/oracle/APPLICATIONS_CONFIG/instance/domains/CRMDomain
      /scratch/oracle/APPLICATIONS_CONFIG/instance/domains/CRMDomain_backup
    

    This ensures that any needed files are not overwritten.

  4. Create a staging directory and restore the Applications base directory to that directory:
    cd /scratch/stage
    tar -xvf crm_ApplBase_backup_030214.tar
    
  5. From that stage directory, copy the domain to the original location:
    cp /scratch/stage/APPLICATIONS_CONFIG/instance/domains/CRMDomain
      /scratch/oracle/APPLICATIONS_CONFIG/instance/domains/CRMDomain
    
  6. Start the Administration Server, as described in Start and Stop an Oracle Fusion Applications Environment.
  7. Start all Managed Servers, as described in Start and Stop an Oracle Fusion Applications Environment.

10.12.5 Recover Servers When the Installation Directory Is Shared Between Hosts

When the applications directory is shared between hosts, and an Administration Server or a Managed Server need to be recovered, the entire domain must be recovered.

The following table shows a sample domain mapping and provides guidelines for recovering an Oracle WebLogic Server Administration Server or any Managed Server running in the domain. The list of servers in the table is not complete. Refer to the particular Oracle Fusion application offering for the complete list of servers.

Domain Name Servers Running in Domain What to Recover

CRMDomain

AdminServer

CRMCommonServer_1

CRMPerformanceServer_1

CRMAnalyticsServer_1

CRMSearchServer_1

CRMODIServer_1

CRMSOAServer_1

CustomerServer_1

SalesServer_1

MarketingServer_1

OrderCaptureServer_1

The entire CRMDomain. For example:

APPLICATIONS_CONFIG/instance/domains/CRMDomain

FinancialDomain

AdminServer

FinCommonServer_1

FinAnalyticsServer_1

FinSOAServer_1

LedgerServer_1

ESSServer_1

The entire FinancialDomain. For example:

APPLICATIONS_CONFIG/instance/domains/FinancialDomain

SCMDomain

AdminServer

ProductManagementServer_1

SCMSOAServer_1

The entire SCMDomain. For example:

APPLICATIONS_CONFIG/instance/domains/SCMDomain

HCMDomain

AdminServer

CoreSetupServer_1

FINSOAServer_1

TalentServer_1

ESSServer_1

HCMAnalyticsServer_1

The entire HCMDomain. For example:

APPLICATIONS_CONFIG/instance/domains/HCMDomain

SetupDomain

AdminServer

HomePageServer_1

HelpPortalServer_1

FunctionalSetupServer_1

WCServices_1

SpacesServer_1

The entire SetupDomain. For example:

APPLICATIONS_CONFIG/instance/domains/SEtupDomain

10.12.6 Recover Servers When the Managed Server Configuration Is Local

If domain directory of the Managed Server is different than the Administration Server domain directory, and a Managed Server needs to be recovered, use pack and unpack to recover the Managed Server.

For example, to pack and unpack the Managed Server FinCommonServer_1:

  1. If the Administration Server is not reachable, recover the Administration Server, as described in Recover the Administration Server Configuration.

  2. If the Managed Server fails to start or if the file system is lost:

    1. Recover the Middleware home from the backup file, if required. For example:

      cd MW_HOME
      tar -xf mw_home_backup_030214.tar 
      
    2. Create a domain template jar file for the FinancialDomain Administration Server, using the pack utility. For example:

      pack.sh -domain=APPLICATIONS_CONFIG/instance/domains/FinancialDomain
         -template=/scratch/temp.jar -template_name=domain1 
         -template_author=myname -log=/scratch/logs/my.log -managed=true
      

      Specifying the -managed=true option packs up only the Managed Servers. If the entire domain is packed, omit this option.

    3. Unpack the domain template JAR file, using the unpack utility. For example:

      unpack.sh -template=/scratch/temp.jar
         -domain=APPLICATIONS_CONFIG/instance/domains/FinancialDomain
         -log=/scratch/logs/my.lognew.log -log_priority=info
      
    4. Ensure that the application artifacts are accessible from the Managed Server host. That is, if the application artifacts are not on the same server as the Managed Server, they must be in a location accessible by the Managed Server.

      • For stage mode applications, the Administration Server copies the application bits to the staged directories on the Managed Server hosts.

      • For nostage and external-stage mode applications, ensure that application files are available in the stage directories of the Managed Server.

    5. Start the Managed Server, as described in Start and Stop an Oracle Fusion Applications Environment.

      The Managed Server connects to the Administration Server and updates its configuration changes.

10.12.7 Recover an Oracle Instance in an Oracle Fusion Customer Relationship Management Installation

To recover the Oracle instance ohs in an Oracle Fusion Customer Relationship Management installation:

  1. Stop all relevant processes using opmnctl or kill all processes that are related to that Oracle instance. For example:
    opmnctl stopall
    
  2. Copy the original Oracle instance to a backup location in case some configuration data needs to be retrieved:
    cp /scratch/CRM/APPLICATIONS_CONFIG/webtier_mw_home/webtier/ohs 
       /scratch/CRM/APPLICATIONS_CONFIG/webtier_mw_home/webtier/ohs_backup
    
  3. Restore the Oracle instance home from the backup file to a stage directory by restoring the Applications configuration directory:
    mkdir stage
    cd stage
    tar -xvf crm_ApplConfig_backup_030214.tar
    
  4. Copy the restored Oracle instance from step 3 to the original location:
     cp /scratch/stage/APPLICATIONS_CONFIG/webtier_mw_home/webtier/ohs/  
        /scratch/CRM/APPLICATIONS_CONFIG/webtier_mw_home/webtier/ohs
    
  5. Start all relevant processes. That is, start all processes that are related to that Oracle instance:
    opmnctl startall
    

If the Oracle instance containing Oracle HTTP Server is being recovered to a new host after loss of host, see Recover Oracle HTTP Server After Loss of Host.

10.12.8 Recover an Oracle Fusion Customer Relationship Management Cluster

In this scenario, the cluster CRMPerformanceCluster has been erroneously deleted or the cluster-level configuration, such as the JMS configuration or container-level data sources, was mistakenly changed and committed. The server CRMPerformanceServer_1 cannot be started or does not operate properly or the services running inside the server are not starting. What was changed cannot be ascertained.

This procedure can be used to recover a cluster if the membership is mistakenly deleted

Caution:

Performing a domain-level recovery can affect other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

If the configuration changes are few, then the easiest way is to redo the configuration changes. If that is not feasible, restore the domain to which the cluster belongs. (See the table in Recover Servers When the Installation Directory Is Shared Between Hosts for Managed Servers to domain mappings.) Take the following steps:

  1. Stop the cluster:
    stop('CRMPerformanceCluster', 'Cluster')
    
  2. Stop the Administration Server, as described in Start and Stop an Oracle Fusion Applications Environment.
  3. Copy the original CRMDomain to a backup location in case some configuration data needs to be retrieved:
    cp /scratch/CRM/APPLICATIONS_CONFIG/instance/domains/CRMDomain 
       /scratch/CRM/APPLICATIONS_CONFIG/instance/domains/CRMDomain_backup
    
  4. Restore the CRMDomain home from the backup file to a stage directory by restoring the Applications configuration directory:
    mkdir stage
    cd stage
    tar -xvf crm_ApplConfig_backup_030214.tar
    
  5. Copy the restored CRMDomain from step 4 to the original location:
    cp /scratch/stage/APPLICATIONS_CONFIG/instance/domains/CRMDomain
       /scratch/CRM/APPLICATIONS_CONFIG/instance/domains/CRMDomain
    
  6. Start the Administration Server, as described in Start and Stop an Oracle Fusion Applications Environment.
  7. Start the cluster. The Oracle WebLogic Server Administration Console or WLST can be used. For example, to use the WLST start command:
    start('CRMPerformanceCluster', 'Cluster')
    

10.12.9 Recover Databases for Oracle Fusion Customer Relationship Management

The databases can be recovered using RMAN at whatever level is appropriate by performing a restore and recovery of either the full databases, a tablespace, or a data file.

Oracle Fusion Customer Relationship Management uses the following databases: Oracle Fusion Applications database, Oracle WebCenter Content, and Oracle Identity database. Consistency must be maintained among the databases. If any of these databases are recovered to a different point in time, the databases may need to be reconciled.

For more information, see the following topics: