Skip Headers
Oracle® Fusion Middleware 2 Day Administration Guide
11g Release 1 (11.1.1)

Part Number E10064-04
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Master Index
Master Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

8 Backing Up and Recovering Oracle Fusion Middleware

Backup and recovery refers to the strategies and procedures involved in guarding against hardware failures and data loss, and reconstructing data should a loss occur. This chapter describes the basic backup and recovery strategies and procedures.

This chapter contains the following topics:

8.1 Overview of Backup and Recovery

An Oracle Fusion Middleware environment can consist of different components and configurations. A typical Oracle Fusion Middleware environment contains an Oracle WebLogic Server domain with Java components, such as Oracle SOA Suite, and a WebLogic Server domain with Identity Management components. It can also include Oracle instances containing system components such as Oracle HTTP Server, Oracle Web Cache, Oracle Internet Directory, and Oracle Virtual Directory.

The installations of an Oracle Fusion Middleware environment are interdependent in that they contain configuration information, applications, and data that are kept in synchronization. For example, when you perform a configuration change, information in configuration files is updated. When you deploy an application, you might deploy it to all Managed Servers in a domain or cluster.

It is, therefore, important to consider your entire Oracle Fusion Middleware environment when performing backup and recovery. You should back up your entire Oracle Fusion Middleware environment at once, then periodically. If a loss occurs, you can restore your environment to a consistent state.

8.1.1 Understanding Backup Operations

To back up your Oracle Fusion Middleware environment, you can use:

  • File copy utilities such as copy, xcopy, tar, or jar. Make sure that the utilities:

    • Preserve symbolic links

    • Support long file names

    • Preserve the permissions and ownership of the files

    For example:

    • On Windows, for online backups, use copy; for offline backups, use copy, xcopy, or jar. Do not use Winzip because it does not work with long filenames or extensions.

      Note that for some versions of Windows, any file name with more than 256 characters fails. You can use the xcopy command with the following switches to work around this issue:

      xcopy /s/e  "C:\Temp\*.*"  "C:\copy"
      

      See the xcopy help for more information about syntax and restrictions.

    • On Linux and UNIX, for online and offline backups, use tar.

  • Oracle Recovery Manager (RMAN) to back up database-based metadata repositories and any databases used by Oracle Fusion Middleware. With RMAN, you can perform full backups or incremental backups. See Oracle Database Backup and Recovery User's Guide for information about using RMAN to back up a database.

If you want to retain your backups for a longer duration, you may want to back up to tape, for example using Oracle Secure Backup.

You can also configure Oracle WebLogic Server to make backup copies of the configuration files. This facilitates recovery in cases where configuration changes need to be reversed or in the unlikely case that configuration files become corrupted. When the Administration Server starts, it saves a .jar file named config-booted.jar that contains the configuration files. When you make changes to the configuration files, the old files are saved in the configArchive directory under the domain directory, in a .jar file with a sequentially numbered name such as config-1.jar. However, the configuration archive is always local to the Administration Server host. It is a best practice to back up the archives to an external location.

8.1.1.1 Types of Backups

  • An offline backup means that you must shut down the environment before backing up the files. When you perform an offline backup, the Administration Server, all Managed Servers in the domain, and all system components in the Oracle instances should be shut down.

    Back up the environment offline immediately after installation and after applying any patches or upgrades.

  • An online backup means that you do not shut down the environment before backing up the files. To avoid an inconsistent backup, do not make any configuration changes until the backup is completed. To ensure that no changes are made in the Oracle WebLogic Server domain, lock the WebLogic Server configuration, as described in Section 2.3.2.

You can perform backups on your full Oracle Fusion Middleware environment, or on the run-time artifacts, those files that change frequently.

To perform a full backup, you should back up the static files and directories, as well as run-time artifacts.

Static files and directories are those that do not change frequently. These include:

  • The Middleware home (MW_HOME). A Middleware home consists of a WebLogic Server home (the Oracle WebLogic Server product directories), an Oracle Common home, and optionally an Oracle home. It can also contain the user_projects directories, which contains Oracle WebLogic Server domains, and Oracle instance homes, which are not static files.

  • OraInventory

  • On Linux and UNIX, the OraInst.loc file, which is located in the following directory:

    (Linux and IBM AIX) /etc
    (Other UNIX systems) /var/opt/oracle
    
  • On Linux and UNIX, the oratab file, which is located in the following directory:

    /etc
    
  • The beahomelist file, which is located at:

    (UNIX) user_home/bea/beahomelist
    (Windows) C:\bea\beahomelist
    
  • On Windows, the following registry key:

    HKEY_LOCAL_MACHINE\Software\Oracle
    

    In addition, for system components, such as Oracle Web Cache, you must back up the following Windows Registry key:

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services
    

Run-time artifacts are those files that change frequently. Back up these files when you perform a full backup and on a regular basis. Run-time artifacts include:

  • Domain directories of the Administration Server and the Managed Servers (by default, a domain directory resides in MW_HOME, but it can be configured by the user to point to a different location)

    In most cases, you do not need to back up Managed Server directories separately because the Administration Server contains information about all of the Managed Servers in its domain.

  • All Oracle instance homes, which reside, by default, in the MW_HOME but can be configured to be in a different location

  • Application artifacts, such as .ear or .war files that reside outside of the domain

    You do not need to backup application artifacts in a Managed Server domain because they can be retrieved from the Administration Server during Managed Server startup.

  • Database artifacts such as the MDS repository

  • Any database-based metadata repositories used by Oracle Fusion Middleware. You use Oracle Recovery Manager (RMAN) to back up an Oracle database

  • Persistent stores, such as JMS Providers and transaction logs, which reside, by default in the user_projects directory, but can be configured in a different location

8.1.1.2 Recommended Backup Strategy

This section outlines the recommended strategy for performing backups. Using this strategy ensures that you will be able to perform the recovery procedures in this book.

  • Perform a full offline backup. This involves backing up the entities described in Section 8.1.1.1. Perform an offline full backup at the following times:

    • Immediately after you install Oracle Fusion Middleware

    • Immediately before upgrading your Oracle Fusion Middleware environment

    • Immediately after an operating system software upgrade

    • Immediately after upgrading or patching Oracle Fusion Middleware

  • Perform an online backup of run-time artifacts. This involves backing up the run-time artifacts described in Section 8.1.1.1. Backing up the run-time artifacts enables you to restore your environment to a consistent state as of the time of your most recent configuration and metadata backup. To avoid an inconsistent backup, do not make any configuration changes until backup completes. Perform an online backup of run-time artifacts at the following times:

    • On a regular basis. Oracle recommends that you back up run-time artifacts nightly.

    • Prior to making configuration changes to a component.

    • After making configuration changes to a component.

    • Prior to deploying a custom Java EE application to a Managed Server or cluster.

    • After a major change to the deployment architecture, such as creating servers or clusters.

  • Perform a full or incremental backup of your databases: Use RMAN to backup your databases. See the Oracle Database Backup and Recovery User's Guide for information about using RMAN and for suggested methods of backing up the databases.

8.1.2 Understanding Recovery Operations

Recovery strategies enable you to recover from critical failures that involve actual data loss. Depending on the type of loss, they can involve recovering any combination of the following types of files:

  • Oracle software files

  • Configuration files

  • Oracle system files

  • Windows Registry key

  • Application artifacts

You can recover your Oracle Fusion Middleware environment while Oracle Fusion Middleware is offline.

To recover your Oracle Fusion Middleware environment, you can use:

  • File copy utilities such as copy, xcopy, or tar

    When you restore the files, use your preferred tool to extract the compressed files:

    • On Windows, for online recovery, use copy; for offline recovery, use copy, xcopy, or jar.

      Note that for some versions of Windows, any file name with more than 256 characters will fail. You can use the xcopy command with the following switches to work around this issue:

      xcopy /s/e  "C:\Temp\*.*"  "C:\copy"
      

      See the xcopy help for more information about syntax and restrictions.

      Do not use Winzip because it does not work with long filenames or extensions.

    • On Linux and UNIX, use tar.

    Ensure that the tool you are using preserves the permissions and timestamps of the files.

  • Oracle Recovery Manager (RMAN) to recover database-based metadata repositories

8.1.2.1 Types of Recovery

You can recover your Oracle Fusion Middleware environment in part or in full. You can recover the following:

  • The Middleware home

  • An Oracle WebLogic Server domain

  • The Oracle WebLogic Server Administration Server

  • An Oracle WebLogic Server Managed Server

  • An Oracle home

  • An Oracle instance

  • A component, such as Oracle SOA Suite or Oracle HTTP Server

  • An Oracle WebLogic Server cluster

  • Deployed applications

8.1.2.2 Recommended Recovery Strategies

Note the following key points about recovery:

  • Your Oracle Fusion Middleware environment must be offline while you are performing recovery.

  • Rename important existing files and directories before you begin restoring the files from backup so that you do not unintentionally override necessary files.

  • Although, in some cases, it may appear that only one or two files are lost or corrupted, you should restore the directory structure for the entire element, such as an Oracle instance or a component, rather than just restoring one or two files. In this way, you are more likely to guarantee a successful recovery.

  • Recover the database to the most current state, using point-in-time recovery (if the database is configured in Archive Log Mode). This is typically a time right before the database failure occurred.

8.2 Backing Up Your Environment

The following topics describe how to perform different types of backups:

8.2.1 Performing a Full Offline Backup

To perform a full offline backup, you copy the directories that contain Oracle Fusion Middleware files.

Archive and compress the source Middleware home, using your preferred tool for archiving. Ensure that the tool you are using preserves the permissions of the files, as described in Section 8.1.1.

The following example shows how to archive and compress the Middleware home on Linux:

cd Middleware_Home 
tar cf - * | gzip > Middleware_Home.tar.gz

The tar utility may issue warnings if the sticky bit is set on some files. You can safely ignore these warnings.

To perform a full offline backup:

  1. Shut down all processes in the Middleware home. For example, shut down the Managed Servers, the Administration Server, and any Oracle instances running in the Middleware home.

  2. Back up the Middleware home (MW_HOME) on all hosts. For example:

    tar -cf mw_home_backup_061112.tar MW_HOME/*
    
  3. If the domain is not located within the Middleware home, back up the Administration Server domain separately. This backs up Java components such as Oracle SOA Suite and Oracle WebCenter Portal.

    For example:

    tar -cf domain_home_backup_061112.tar MW_HOME/user_projects/domains/domain_name/*
    

    In most cases, you do not need to backup the Managed Server directories separately, because the Administration Server domain contains information about the Managed Servers in its domain.

  4. If the Oracle instance home is not located within the Middleware home, back up the Oracle instance home. The Oracle instance home contains configuration information about system components, such as Oracle HTTP Server or Oracle Internet Directory. (See Section 2.4.2 for a list of system components.)

    For example:

    tar -cf instance_home_backup_061112.tar ORACLE_INSTANCE/*
    
  5. If a Managed Server is not located within the domain directory, back up the Managed Server directory. For example:

    tar -cf mg1_home_backup_061112.tar MW_HOME/user_projects/domains/domain_name/servers/server_name/*
    
  6. Back up the OraInventory directory. For example:

    tar -cf Inven_home_backup_061112.tar /scratch/oracle/OraInventory
    
  7. On Linux and UNIX, back up the OraInst.loc file, which is located in the following directory:

    (Linux and IBM AIX) /etc
    (Other UNIX systems) /var/opt/oracle
    
  8. On Linux and UNIX, backup the oratab file, which is located in the following directory:

    /etc
    
  9. Backup the database repositories using the Oracle Recovery Manager (RMAN). For detailed steps, see the Oracle Database Backup and Recovery User's Guide, which is available at:

    http://www.oracle.com/technology/documentation/database.html
    
  10. On Windows, export the following registry key:

    HKEY_LOCAL_MACHINE\Software\Oracle
    

    In addition, for system components, such as Oracle Web Cache, export the following Windows Registry key:

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services
    

    To export a key, use the following command:

    regedit /E  FileName Key
    

    For example:

    regedit /E C:\oracleregistry.reg HKEY_LOCAL_MACHINE/oracle 
    

    You can also use the Registry Editor to export the key. See the Registry Editor Help for more information.

  11. Create a record of your Oracle Fusion Middleware environment. See Section 8.4.

8.2.2 Performing an Online Backup of Run-Time Artifacts

You should perform a backup of run-time artifacts on a regular basis and at the times described in Section 8.1.1.2.

To back up run-time artifacts:

  1. To avoid an inconsistent backup, do not make any configuration changes until the backup is completed. To ensure that no changes are made in the Oracle WebLogic Server domain, lock the WebLogic Server configuration, as described in Section 2.3.2.

  2. Back up the Administration Server domain directories. This backs up Java components such as Oracle SOA Suite and Oracle WebCenter Portal. For example:

    tar -cf domain_home_backup_061112.tar MW_HOME/user_projects/domains/domain_name/*
    

    For Oracle Portal, Oracle Reports, Oracle Forms Services, and Oracle Business Intelligence Discoverer, you must back up the Managed Server domain directories, in addition to the Administration Server domain.

  3. Back up the Oracle instance home. This backs up the system components, such as Oracle HTTP Server. For example:

    tar -cf instance_home_backup_061112.tar ORACLE_INSTANCE/*
    
  4. Backup the database repositories using the Oracle Recovery Manager (RMAN). For detailed steps, see the Oracle Database Backup and Recovery User's Guide, which is available at:

    http://www.oracle.com/technology/documentation/database.html
    
  5. Create a record of your Oracle Fusion Middleware environment. See Section 8.4.

8.3 Recovering After Data Loss, Corruption, or Media Failure

This section describes recovery strategies for outages that involve actual data loss or corruption, or media failure where the disk cannot be restored. This type of failure requires some type of data restoration before the Oracle Fusion Middleware environment can be restarted and continue with normal processing. It contains the following topics:

For information about recovering after the loss of a host, see "Recovery After Loss of Host" in the Oracle Fusion Middleware Administrator's Guide.

8.3.1 Recovering a Middleware Home

You can recover a Middleware home that was corrupted or from which files were deleted.

To recover a Middleware home:

  1. Stop all relevant processes. That is, stop all processes that are related to the domain, such as the Administration Server, Node Manager, and Managed Servers. For example, to stop the Administration Server on Linux:

    DOMAIN_HOME/bin/stopWeblogic.sh username password [admin_url]
    
  2. Recover the Middleware home directory from backup. For example:

    cd MW_HOME
    (UNIX) tar -xf mw_home_backup_061112.tar
    (Windows) jar xtf mw_home_backup_061112.jar
    
  3. Start all relevant processes. That is, start all processes that run in that Middleware home. For example, to start the Administration Server:

    DOMAIN_HOME/bin/startWebLogic.sh -Dweblogic.management.username=username
     -Dweblogic.management.password=password
     -Dweblogic.system.StoreBootIdentity=true
    

8.3.2 Recovering an Oracle WebLogic Server Domain

You can recover an Oracle WebLogic Server domain that was corrupted or deleted from the file system.

Caution:

Performing a domain-level recovery can impact other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

To recover an Oracle WebLogic Server domain:

  1. Stop all relevant processes. That is, stop all processes that are related to the domain, such as the Administration Server and Managed Servers. For example, stop the Administration Server:

    DOMAIN_HOME/bin/stopWeblogic.sh username password [admin_url]
    
  2. Recover the domain directory from backup. For example:

    cd MW_HOME/user_projects/domains/domain_name
    (UNIX) tar -xf domain_backup_061112.tar 
    (Windows) jar xtf domain_backup_061112.jar 
    
  3. Start all relevant processes. That is, start all processes that are related to the domain. For example, start the Administration Server:

    DOMAIN_HOME/bin/startWebLogic.sh -Dweblogic.management.username=username
     -Dweblogic.management.password=password
     -Dweblogic.system.StoreBootIdentity=true
    
  4. If you cannot start the Administration Server, recover it, as described in Section 8.3.3.

  5. If you cannot start a Managed Server, recover it, as described in Section 8.3.4.

8.3.3 Recovering the Administration Server Configuration

If the Administration Server configuration has been lost because of file deletion or file system corruption, the Administration Server console continues to function if it was already started when the problem occurred. The Administration Server directory is regenerated automatically, except for security information. As a result, whenever you start the Administration Server, it prompts for a user name and password. To prevent this, you can recover the configuration.

Caution:

Performing a domain-level recovery can impact other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

To recover the Administration Server configuration:

  1. Stop all processes, including the Administration Server, Managed Servers, and Node Manager if they are started. For example, to stop the Administration Server on Linux, use the following script:

    DOMAIN_HOME/bin/stopWeblogic.sh username password admin_url
    
  2. Recover the Administration Server configuration by recovering the domain home backup to a temporary location. Then, restore the config directory to the following location:

    DOMAIN_HOME/config
    
  3. Start the Administration Server. You can use WLST or the following script:

    DOMAIN_HOME/bin/startWebLogic.sh -Dweblogic.management.username=username
       -Dweblogic.management.password=password
       -Dweblogic.system.StoreBootIdentity=true
    
  4. Verify that the Administration Server starts properly and is accessible.

On the next configuration change, the configuration from the Administration Server is pushed to the Managed Servers. On each Managed Server restart, the configuration is retrieved from the Administration Server.

8.3.4 Recovering a Managed Server

In this scenario, the Managed Server does not operate properly or cannot be started because the configuration has been deleted or corrupted or the configuration was mistakenly changed and you cannot ascertain what was changed.

To recover a Managed Server when it cannot be started:

  1. If the Administration Server is not reachable, recover the Administration Server, as described in Section 8.3.3.

  2. If the Managed Server fails to start or if the file system is lost, take the following steps:

    1. Recover the Middleware home from the backup, if required. For example:

      tar -xf mw_home_backup_061112.tar 
      
    2. Create a domain template jar file for the Administration Server, using the pack utility. For example:

      pack.sh -domain=MW_HOME/user_projects/domains/domain_name
         -template=/scratch/temp.jar -template_name=test_install 
         -template_author=myname -log=/scratch/logs/my.log -managed=true
      

      Specifying the -managed=true option packs up only the Managed Servers. If you want to pack the entire domain, omit this option.

    3. Unpack the domain template jar file, using the unpack utility:

      unpack.sh -template=/scratch/aime1/ms.jar
         -domain=MW_HOME/user_projects/domains/domain_name
         -log=/scratch/logs/new.log -log_priority=info
      
    4. Ensure that the application artifacts are accessible from the Managed Server host. That is, if the application artifacts are not on the same server as the Managed Server, they must be in a location accessible by the Managed Server.

      Note:

      • For stage mode applications, the Administration Server takes care of pushing the bits to the stage directories in the Managed Server.

      • For no-stage and external-stage applications, ensure that application files are available in the stage directories of the Managed Server.

      See Oracle Fusion Middleware Deploying Applications to Oracle WebLogic Server for information about stage, no-stage, and external-stage mode applications.

    5. Start the Managed Server. You can use the Oracle WebLogic Server Administration Console, WLST, or the following script:

      DOMAIN_HOME/bin/startManagedWebLogic.sh managed_server_name 
                admin_url 
      

      The Managed Server connects to the Administration Server and updates its configuration changes.

8.3.5 Recovering an Oracle Home

You can recover an Oracle home, which contains binary files for particular components.

To recover your Oracle home for a particular component:

  1. Recover the Oracle home to the original directory from a backup file. For example:

    cd ORACLE_HOME
    tar -xf Oracle_home_backup_061112.tar 
    
  2. Restart the Managed Server to which applications are deployed using the WLST start command. For example:

    wls:/mydomain/serverConfig> start('myserver','Server')
    

8.3.6 Recovering an Oracle Instance

An Oracle instance home contains configuration information for system components, such as Oracle HTTP Server or Oracle Internet Directory. (See Section 2.4.2 for a list of system components.)

The following topics describe how to recover an Oracle instance:

8.3.6.1 Recovering After Oracle Instance Home Deleted from File System

You can recover an Oracle instance home after it was corrupted or erroneously deleted from the file system.

To recover an Oracle instance home that was corrupted or deleted from the file system:

  1. Stop all relevant processes. That is, stop all processes that are related to that Oracle instance.

  2. Recover the Oracle instance home directory from a backup file. For example:

    cd ORACLE_INSTANCE
    (UNIX) tar -xf Instance_home_backup_061112.tar 
    (Windows) jar xtf Instance_home_backup_061112.jar
    
  3. Start all relevant processes. That is, start all processes that are related to that Oracle instance:

    opmnctl startall
    

8.3.6.2 Recovering After Oracle Instance Home Deregistered

You can recover an Oracle instance home after it was erroneously deregistered from the domain.

To recover an Oracle instance home that was deregistered:

  1. Recover the Oracle instance home directory from a backup file. For example, on Linux:

    cd ORACLE_INSTANCE
    tar -xf Instance_home_backup_061112.tar 
    
  2. Register the Oracle instance, along with all of its components, with the Administration Server, using the opmnctl registerInstance command. For example:

    opmnctl registerInstance -adminHost admin_server_host 
         -adminPort admin_server_port -adminUsername username 
         -adminPassword password
         -oracleInstance ORACLE_INSTANCE_dir -oracleHome ORACLE_HOME_dir
         -instanceName Instance_name -wlserverHome Middleware_Home
    

8.3.7 Recovering Components

For most components, the following topics describe how to recover a component:

For information about additional actions you may need to take to recover particular components, see "Recovering Components" in the Oracle Fusion Middleware Administrator's Guide.

8.3.7.1 Recovering After Component's Files Are Deleted or Corrupted

You can restore a component's files if they are deleted or corrupted or if the component cannot be started or is not functioning properly because the component's configuration was changed and committed. You may not be able to ascertain what change is causing the problem and you want to revert to an earlier version.

To recover a component, the steps you take depend on the type of component:

  • For Java components, such as Oracle SOA Suite, you recover the Managed Server, as described in Section 8.3.4.

  • For system components, such as Oracle HTTP Server or Oracle Web Cache:

    1. Stop the component. For example, to stop Oracle HTTP Server:

      opmnctl stopproc ias-component=HTTP_Server
      

      For more information on stopping components, see Section 3.4.4.

    2. Recover the component-specific files from backup. See the Oracle Fusion Middleware Administrator's Guide for a list of the directories and files needed for each component. For example, to recover Oracle HTTP Server files, you recover the following directories:

      ORACLE_INSTANCE/config/OHS/component_name
      ORACLE_INSTANCE/diagnostics/logs/OHS/component_name
      ORACLE_INSTANCE/tmp/OHS/component_name
      
    3. Start the component. For example, to start Oracle HTTP Server:

      opmnctl startproc ias-component=HTTP_Server
      

      For more information on starting components, see Section 3.4.4.

8.3.7.2 Recovering Components After Cluster Configuration Change

You can recover components in a cluster when the components cannot be started or are not functioning properly because the configuration was changed and committed at the cluster level. You may not be able to ascertain what change is causing the problem and you want to revert to an earlier version.

Caution:

Performing a domain-level recovery can impact other aspects of a running system and all of the configuration changes performed after the backup was taken will be lost.

To recover the components:

  1. Stop the cluster:

    stop('cluster_name', 'Cluster')
    
  2. Stop all processes, such as the Managed Servers and the Administration Server. For example, to stop the Administration Server on Linux, use the following script:

    DOMAIN_HOME/bin/stopWeblogic.sh username password admin_url
    
  3. Recover the Administration Server configuration by recovering the domain home backup to a temporary location. Then, restore the config directory to the following location:

    DOMAIN_HOME/config
    
  4. Start the Administration Server. You can use WLST or the following script:

    DOMAIN_HOME/bin/startWebLogic.sh -Dweblogic.management.username=username
     -Dweblogic.management.password=password
     -Dweblogic.system.StoreBootIdentity=true
    
  5. Start the cluster. You can use the Oracle WebLogic Server Administration Console or WLST. For example, to use the WLST start command:

    start('clusterName', 'Cluster')
    

The latest configuration is retrieved from the Administration Server to every member of the cluster.

8.4 Creating a Record of Your Oracle Fusion Middleware Configuration

In the event you need to restore and recover your Oracle Fusion Middleware environment, it is important to have all the necessary information at your disposal. This is especially true in the event of a hardware loss that requires you to reconstruct all or part of your Oracle Fusion Middleware environment on a new disk or host.

You should maintain an up-to-date record of your Oracle Fusion Middleware environment that includes the information listed in this section. You should keep this information both in hardcopy and electronic form. The electronic form should be stored on a host or e-mail system that is completely separate from your Oracle Fusion Middleware environment.

Your Oracle Fusion Middleware hardware and software configuration record should include:

8.5 Learn More

For more information about backing up and recovering Oracle Fusion Middleware, see the following topics in the Oracle Fusion Middleware Administrator's Guide: