9 Troubleshoot Database Installations

Follow the instructions described in this section if any issues are encountered during database installation for Oracle Identity Management or Oracle Fusion Applications. It also describes troubleshooting tips for the Oracle Fusion Middleware Repository Creation Utility (used for Oracle Identity Management) and Oracle Fusion Applications Repository Creation Utility operations.

This section contains the following topics:

9.1 Troubleshoot the Oracle Identity Management Database Installation and Oracle Fusion Middleware RCU Operations

For more information about troubleshooting Oracle Database, see Troubleshooting in the Oracle Database Installation Guide for Linux.

For more information about troubleshooting the Oracle Fusion Middleware RCU, see Troubleshooting Repository Creation Utility in the Oracle Fusion Middleware Repository Creation Utility User's Guide.

9.2 Troubleshoot Oracle Fusion Applications Database Installation and Oracle Fusion Applications RCU Operations

This section contains troubleshooting tips for database installation and the Oracle Fusion Applications RCU operations. It is divided into sections for general tips and sections about log files and cleanup features.

9.2.1 General Troubleshooting Tips

If an error is displayed during the creation of applications schemas and tablespaces, take note of the following:

  • Oracle Fusion Applications release notes may contain additional information about this release, such as mandatory Oracle Database server and client patches that must be applied to the environment.

  • This release of Oracle Fusion Applications relies on specific system requirements that are explained in Certification in the Oracle Fusion Applications Release Notes.

  • If abnormal program termination is encountered and the error log displays: Java Exception: java.lang.StackOverflowError occurred while installing Oracle database, then see document id 1056200.1 at My Oracle Support.

  • Ensure that the database is up and running.

  • If there is a failure in the Oracle Fusion Applications RCU during the creation of the tablespaces and loading of the Fusion schemas, start from the beginning by installing an empty database or using the Database Configuration Assistant. There is no drop option.

  • Clean up these areas before redoing the installation. For information about the log files, see Oracle Fusion Applications RCU Log Files.

    • /tmp

    • old log file directories

    • /oraInventory folder contents

    • ORACLE_HOME (or remove the ORACLE_HOME to re-use the directory)

  • If incorrect information is entered on one of the Oracle Fusion Applications RCU screens, use the navigation pane on the left side of the graphical interface to return to that screen.

  • If an error occurred while the Oracle Fusion Applications RCU was running:

    1. Note the error and review the Oracle Fusion Applications RCU log files.

    2. Correct the issue that caused the error. Depending on the type of error, either continue with the operation or restart the Oracle Fusion Applications RCU.

    3. Continue or restart the Oracle Fusion Applications RCU to complete the desired operation.

9.2.2 Database Installation Log Files

The database installation log file reports what happened during each of the phases in a database installation. Click a log file symbol on the Database Installation Progress screen in the Provisioning Wizard to view the log file for that phase. Log files are located in tmp_directory/dbInstall_time_stamp_provtop/logs/provisioning/host. An example on a Linux platform is /tmp/dbInstall_20120216092937_provtop/logs/provisioning/host123. The tmp directory may differ depending on what is considered to be the temporary directory for various platforms. The location of the plan file for the database flow is to tmp_directory/dbInstall_time_stamp_provtop/dbInstall_time_stamp.plan.

9.2.3 Oracle Fusion Applications RCU Log Files

Log files describe what happened during the schema and tablespace creation process, including any failures that occurred. The main Oracle Fusion Applications RCU log file (rcu.log) is written to (Linux) APP_RCU_HOME/rcu/log/logdir.date_timestamp or (Windows) APP_RCU_HOME\rcu\log\logdir.date_timestamp. For example, the log file on a Linux operating system is:

APP_RCU_HOME/rcu/log/logdir.2010-01-02_03-00/rcu.log

The custom_comp_create_tbs.log file lists the PL/SQL statements that created the tablespaces.

In the fusionapps schema, three types of log files are created:

  • fusionapps.log: Lists the PL/SQL that was run.

  • fusionapps_runimport.log: The Oracle Data Pump import log file.

  • fusionapps_verify.log: Lists verification errors if objects created are not what was expected.

In addition to the general log files, each component writes a log file of its own. The file name is in the form of component_name.log. For example, the log file for the BIAPPS_OTBI_RUNIMPORT component is biapps_otbi_runimport.log. All component log files are written to the same directory as the main log file.

The following list shows the log files in alphabetical order, by component name:

  • biapps_otbi_runimport.log

  • crm_fusion_mds_soa.log

  • crm_fusion_soainfra.log

  • fin_fusion_mds_soa.log

  • fin_fusion_soainfra.log

  • fmw_runtime.log

  • fusion_activities.log

  • fusionapps.log

  • fusion_bia_cloud.log

  • fusion_biplatform.log

  • fusion_discussions.log

  • fusion_edqconfig1.log

  • fusion_edqconfig2.log

  • fusion_edqfusion.log

  • fusion_edqresults1.log

  • fusion_edqresults2.log

  • fusion_ero.log

  • fusion_grc.log

  • fusion_iau.log

  • fusion_intg_current.log

  • fusion_intg_final.log

  • fusion_intg_previous.log

  • fusion_ipm.log

  • fusion_mds_ess.log

  • fusion_mds.log

  • fusion_mds_spaces.log

  • fusion_ocserver11g.log

  • fusion_odi.log

  • fusion_opss.log

  • fusion_ora_ess.log

  • fusion_orasdpls.log

  • fusion_orasdpm.log

  • fusion_orasdpsds.log

  • fusion_orasdpxdms.log

  • fusion_otbi.log

  • fusion_portlet.log

  • fusion_rdf.log

  • fusion_ro.log

  • fusion_social_cef.log

  • fusion_social.log

  • fusion_social_views.log

  • fusion_webcenter.log

  • hcm_fusion_mds_soa.log

  • hcm_fusion_soainfra.log

  • hed_fusion_mds_soa.log

  • hed_fusion_soainfra.log

  • lcm_exp_admin.log

  • lcm_object_admin.log

  • lcm_super_admin.log

  • lcm_user_admin.log

  • odi-seed.log

  • oic_fusion_mds_soa.log

  • oic_fusion_soainfra.log

  • prc_fusion_mds_soa.log

  • prc_fusion_soainfra.log

  • prj_fusion_mds_soa.log

  • prj_fusion_soainfra.log

  • scm_fusion_mds_soa.log

  • scm_fusion_soainfra.log

  • searchsys.log

  • setup_fusion_mds_soa.log

  • setup_fusion_soainfra.log

  • dvacctmgr.log

  • dvowner.log

9.2.4 Oracle Fusion Applications RCU Taking a Long Time

When installing all components, including the FUSIONAPPS component, Oracle Fusion Applications RCU should complete in four hours or less. If it is running more than four hours, you can safely cancel the run and perform the post-cancellation steps described below.

Post-Cancellation Steps

Follow the steps and ensure each step is completed successfully. If any prerequisite is not satisfied, do not proceed to the next step.

  1. Verify that the Oracle Fusion Applications Apps RCU has completed importing the data from the dump files. If you do not see the expected line in the log, do not proceed and wait for this import to complete.

    cd $RCU/rcu/log/logdir.YYYY-MM-DD.HH-MM
    tail fusionapps_runimport.log
    verify that the last line in the log has this:
    > Job "SYS"."FUSIONDBJOB" completed with XX error(s) at
    <day> HH:MM:SS YYYY elapsed 0 HH:MM:SS
    
  2. Verify that Oracle Fusion Applications Apps RCU has completed the import integrity check. If the integrity check log does not exist, do not proceed and wait for the verification process to be completed. If there are errors, Oracle Fusion Applications RCU will display the error in the UI.

    cd $RCU/rcu/log/logdir.YYYY-MM-DD.HH-MM
    ls fusionapps_verify.log
    > fusionapps_verify.log
    
  3. Stop the Oracle Fusion Applications Apps RCU. When you stop this run, the SQL process that performs histogram optimization will be running in the background. Proceed to the next step.

    Click on "Stop" button
    Click on "Yes" button when you get "Repository Creation Utility - Warning"
    dialog with the message "Do you want to stop the current operation?"
    It may take a while for the stop operation to complete
    Uncheck the box for "Cleanup for failed components"
    Click on "Close" button
    
  4. Perform these steps manually on the machine where the database is running. Ensure that $ORACLE_HOME and $ORACLE_SID are set correctly to connect to the database.

    1. Recompile database packages.

      cd $RCU/integration/fusionapps/sql
      $ORACLE_HOME/bin/sqlplus "/as sysdba"  
      > @fusionapps_recompile_packages
      > exit
      
    2. Update FND tablespaces.

      cd $RCU/fusionapps/sql
      $ORACLE_HOME/bin/sqlplus "/as sysdba"
      > @fusionapps_update_fnd_tablespaces  FUSION_TS_TX_DATA  FUSION_TS_TX_DATA  
      FUSION_DYN_TS  FUSION_DYN_TS  FUSION_IAS_ORASDPM_AQ  FUSION_IAS_ORASDPM_AQ  
      FUSION_TS_AQ FUSION_TS_AQ FUSION_IAS_ORASDPM_AQ FUSION_IAS_ORASDPM_AQ FUSION_TS_ARCHIVE FUSION_TS_ARCHIVE
      FUSION_TS_DQ FUSION_TS_DQ FUSION_TS_INTERFACE  FUSION_TS_INTERFACE  
      FUSION_TS_MEDIA FUSION_TS_MEDIA FUSION_TS_NOLOGGING FUSION_TS_NOLOGGING
      FUSION_TS_QUEUES FUSION_TS_QUEUES FUSION_TS_SEED FUSION_TS_SEED
      FUSION_TS_SUMMARY FUSION_TS_SUMMARY FUSION_TS_TOOLS FUSION_TS_TOOLS
      FUSION_TS_TX_IDX  FUSION_TS_TX_IDX  
      > exit
      

      Verify the command completes with a "PL/SQL procedure successfully completed." message. If you get a different message indicating any error, review it and contact Oracle support to resolve the issue.

    3. Register Oracle Fusion Applications RCU components in the database.

      $ORACLE_HOME/bin/sqlplus "/as sysdba"
      > update schema_version_registry set status="VALID" where comp_id in
      > ('FUSIONAPPS', 'FUNCTIONAL_SETUP_MANAGER', 'TOPOLOGY_MANAGER');
      

      Verify the command completes with a "3 rows updated" message. If you get a different message, contact the Oracle support.

    4. Verify that Fusion Oracle Applications RCU components are registered properly.

      $ORACLE_HOME/bin/sqlplus "/as sysdba"
      select comp_id, status from schema_version_registry where comp_id in
      ('FUSIONAPPS', 'FUNCTIONAL_SETUP_MANAGER', 'TOPOLOGY_MANAGER');
      

      Verify the output is:

      COMP_ID                        STATUS
      ------------------------------ -----------
      FUNCTIONAL_SETUP_MANAGER       VALID
      FUSIONAPPS                     VALID
      TOPOLOGY_MANAGER               VALID
      

9.2.5 Preverification Failure (UNIX)

The following errors might be ecountered while running the preverification phase on Unix systems:

PRODUCTFAMILY=dbserver!PRODUCT=RDBMS_11gR2!TASK=Checking pre-req files and directories!TASKID=dbserver.RDBMS_11gR2.NONE.preverify.NONE!MESSAGE=!DETAIL=After validating the patch directory, $REPOSITORY/installers/database/patch', it has been determined that no patches will be applied during this install|Opatch Directory Validation Requires:| (1)$REPOSITORY/installers/database/patch must be a valid DIRECTORY|(2) $REPOSITORY/installers/database/patch must contain NO FILES, only subdirectories|(3) Patches must be unzipped in $REPOSITORY/installers/database/patch

Solution: Ignore the error and proceed with the database installation.

9.2.6 Preverification Failure (Solaris)

During provisioning, the preverify phase (target) may display a message that some of the Solaris operating system (version 9 and 10) patches are missing. On Solaris x86-64, the following preverify failures may be reported:

WARNING: Check:Patches failed.
Checking for 127111-02; Not found.  Failed <<<<
Checking for 137111-04; Not found.  Failed <<<<

These failure messages can be ignored.

9.2.7 Using the Cleanup Feature

If there is a failure in creation of the tablespaces or schemas for any component, the Cleanup for failed components checkbox appears on the Completion Summary screen. Select this option to clean up tablespaces and schemas for the failed components.

If an environment (such as the database server) is running out of space, correct it and rerun the software. Any components that are not applied successfully are still enabled (not grayed out) in the interface. Rerun the Oracle Fusion Applications RCU as described in Run the Oracle Fusion Applications RCU to Create Oracle Fusion Applications Database Objects.

9.3 What to Do Next

Follow the instructions in Oracle Identity Management Provisioning to create a response file and provision an Oracle Identity Management environment.