3 Troubleshooting Your Upgrade

This chapter describes how to resolve common problems and issues that may occur while you are using the Upgrade Assistant to upgrade to Oracle Fusion Middleware to 12c (12.1.3):

This chapter contains the following topics:

3.1 General Troubleshooting Guidelines

If errors occur while you are running the Upgrade Assistant, use the following steps to troubleshoot the problem:

Note:

Errors that occur (or are detected) during the Examination phase can be corrected and the Upgrade Assistant can be restarted.

Errors that occur during the Upgrade phase, however, require you to restore your environment from your backup copies, correct the errors and then restart the upgrade process from the beginning.

Your corrective course of action will depend on the phase in which the error occurred. For more information, see Investigating Examination Failures and Investigating Upgrade Failures.

  1. Locate and open the Upgrade Assistant log file with a text editor:

    For the location of the log file, see Section 3.2, "Reviewing Log Files".

  2. To verify the correct version of Upgrade Assistant jar file run the following command:

    unzip -p ua.jar META-INF/MANIFEST.MF

    You can run the above command only after you change directory to the jlib directory: cd ORACLE_HOME/oracle_common/upgrade/jlib.

  3. Locate any error messages that are identified by number; for example, UPGAST-00091.

  4. Look up the error in the Oracle Fusion Middleware Error Messages.

    The description of the error in the Oracle Fusion Middleware Error Messages should include a description of the cause of the error, as well as the action you should take to resolve the error.

  5. Based on whether or not you can locate an error message and the error message description, do the following:

    • If, by reviewing the log files and the Oracle Fusion Middleware Error Messages, you are able to identify a solution to the upgrade failure, you can implement your solution and then re-start the Upgrade Assistant and perform the upgrade again.

      When you re-run the Upgrade Assistant, any components that were upgraded successfully during the previous run will not be affected. However, the Upgrade Assistant will attempt to upgrade any components that were not upgraded successfully during a previous run of the utility.

    • Contact Oracle Support for any errors that are not documented or that cannot be resolved by following documented actions. Note that some errors that occur will require the repository to be restored from backup, the problem to be resolved, and another upgrade to be run. Note that Oracle Support requires both the UA.log file and if present, the UA.out file, to be provided for troubleshooting purposes.Provide complete logs and not just excerpts of those files.

3.2 Reviewing Log Files

Should any failures occur when running Upgrade Assistant, log files will be needed to help diagnose and correct the problem; do not delete them. When running the Upgrade Assistant, you can alter the contents of your log files by specifying a different -logLevel from the command line. The default value is -logLevel NOTIFICATION You can alter the location of your log files using the -logDir parameter. For more information see -logLevel in Table 2-2, "Upgrade Assistant Silent Mode Command Line Parameters"

Tip:

To expedite the review process, search for the word "ERROR".

For more information on understanding error messages in your log files, see Resolving Common Upgrade Assistant Errors.

Log files are stored in the following default directory:

On UNIX operating systems:

ORACLE_HOME/oracle_common/upgrade/logs/ua<timestamp>.log

On Windows operating systems:

ORACLE_HOME\oracle_common\upgrade\logs\ua<timestamp>.log

Some components will create a second log file called ua<timestamp>.out, also in the same location.

The timestamp will reflect the actual date and time that Upgrade Assistant was run.

For database schema upgrades of certain components, there can also be an output (.out) file that will contain the screen output of commands that were run in a shell process or as PL/SQL scripts. You can locate these output files in the same default directory.

Note:

In the event that there are questions or issues about an upgrade failure that cannot be resolved with the information in this guide, it will be important to retain the log files. If a service request is needed, the entire Upgrade Assistant .log file and the .out files (if available) should be uploaded to the service request.

3.3 Investigating Examination Failures

To determine the cause of an examination failure:

  1. Note the name of the failed component in the Upgrade Assistant dialog or command-line output.

  2. Open the following Upgrade Assistant log file.

    For the location of the log file, see Section 3.2, "Reviewing Log Files".

  3. In the log file, search for the message Starting to examine component_name.

Note:

To complete the upgrade, resolve the issues and then launch the Upgrade Assistant again, or, if possible, click Back to return to a previous screen and make the necessary changes.

3.4 Investigating Upgrade Failures

To determine the cause of an upgrade failure:

  1. Note the name of the failed component in the Upgrade Assistant dialog or command-line output.

  2. Open the Upgrade log file:

    For the location of the log file, see Section 3.2, "Reviewing Log Files".

  3. Search for the message Starting to upgrade component_name.

Note:

To complete the upgrade, restore the environment using your pre-upgrade backup, resolve the issues and then launch the Upgrade Assistant again. You will have to start the upgrade process from the beginning to ensure a successful (complete) upgrade.

3.5 Resolving Common Upgrade Assistant Errors

If errors occur while you are running the Upgrade Assistant, you must correct the conditions that caused them before you try the upgrade again. The following sections provide some common errors that can occur.

Note:

This section provides descriptions of the most common upgrade errors. For a complete list of Fusion Middleware errors, see the Oracle Fusion Middleware Error Messages.

3.5.1 Ensuring there is sufficient disk space

If an upgrade fails due to the database server running out of disk space, you must restore the database server environment from backups, add sufficient disk space or remove unwanted files (such as temp or trace files) from the database server, and then retry the upgrade.

Note:

Once a database schema upgrade has failed due to this class of error, you cannot simply add more disk space and retry the upgrade. The schemas have been left in an inconsistent state and may have been marked "INVALID". You cannot recover from this error without restoring the original database state from backups.

The following examples show some insufficient disk space errors you may encounter:

ORA-01658: unable to create INITIAL extent for segment in tablespace

Cause: The existing schema tablespace does not have sufficient space to complete the upgrade.

Action: Make sure that the tablespace has sufficient room (space) for a successful upgrade. Oracle recommends that you add more data files to the existing 11g database tablespaces, otherwise the upgrade will fail.

ORA-01114: IO error writing block to file <block number>

Cause: The device on which the file resides is probably offline. If the file is a temporary file, then it is also possible that the device has run out of space. This could happen because disk space of temporary files is not necessarily allocated at file creation time.

Action: Restore access to the device or remove unnecessary files to free up space.

ORA-09945: Unable to initialize the audit trail file

Cause: The system is unable to write header information to the file being used as the audit trail. The audit_trail_dest or audit trail destination is full for generation of audit file.

Action: Free up space and retry the operation.

Linux-x86_64 Error: 28: No space left on device

3.5.2 Resolving Database Connection Problems When Upgrading Schemas

If you have trouble connecting to a database when using the Upgrade Assistant to upgrade a component schema, try connecting to the database using another tool, such as SQL*Plus. This will help you troubleshoot the problem by verifying that the database is up and running and available on the network.

3.5.3 Setting the DISPLAY Environment Variable

When running Upgrade Assistant in GUI mode, you must set the DISPLAY variable properly or you may receive the following errors:

Xlib: connection to ":1.0" refused by server

Xlib: No protocol specified

Cause: These errors indicate that the DISPLAY variable is not set up properly to allow a GUI to be displayed to the screen.

Action: Set the DISPLAY environment variable to the system name or IP address of your local workstation, and re-run Upgrade Assistant.

If you continue to receive these errors after setting the DISPLAY variable, try launching another GUI tool, such as vncconfig. If you see the same errors, your DISPLAY environment variable may not be set correctly.

3.6 Upgrading Unsupported Domains

If you receive an error stating that the specified domain cannot be upgraded, contact Oracle Support for more information. Do not attempt to upgrade schemas or domain configurations in an unsupported domain.

3.7 Restarting the Upgrade Assistant After a Failure

If the Upgrade Assistant fails during the upgrade phase or only partially upgrades your components, try to resolve the and then follow these steps:

  1. Restore your backed-up 11g or 12.1.2 environment.

  2. Start the Upgrade Assistant in GUI or command-line mode.

Note:

If you continue to experience upgrade failures, consider setting the -logLevel to TRACE so that more information will be logged. This will be useful when troubleshooting a failed upgrade, but be sure to reset the -logLevel to NOTIFICATION after the issue has been resolved to avoid performance issues.