Oracle® Configuration Manager

Troubleshooting and Support Guide

Release 12.1.2

E49552-04

June 2015

Oracle Configuration Manager Troubleshooting and Support Guide, Release 12.1.2

E49552-04

Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.

Contents

Preface

- Audience

- Documentation Accessibility

- Related Documents

- Conventions

- What's Changed

1 Installation and Configuration Issues

- 1.1 Windows-Specific Issues

- 1.1.1 Upgrade Failure on Windows

- 1.1.2 RAC Installs: OCM Does Not Pick Up Oracle Home Name for Its Service Names

- 1.1.3 OCM Upgrade to 10.3.5 Fails When Using the -distribution Option

- 1.1.4 Nonexistent Directory Error Encountered

- 1.1.5 File Extension Error Encountered While Installing OCM

- 1.1.6 Installing OCM on a Windows NFS Shared Drive Fails

- 1.1.7 OCM Does Not Start Manually After Upgrade to OCM 12.1

- 1.1.8 getHttpClientClasspath Variable Undefined Error After Upgrade to OCM 12.1

- 1.2 Linux, Solaris, Unix Issues

- 1.2.1 OCM Fails During Installation of Oracle Database Release 11.2 on Linux x86-64

- 1.2.2 Error Initializing JVM When Installing Oracle Configuration Manager

- 1.3 All Platforms

- 1.3.1 Enabling Debug Logging

- 1.3.2 Errors You May Encounter

- 1.3.2.1 Insufficient Privileges While Running installCCRSQL collectconfig

- 1.3.2.2 Incorrectly Configured Hostnames are Displayed in My Oracle Support

- 1.3.2.3 Oracle Home Environmental Changes not Detected During Collection

- 1.3.3 Known Issues

- 1.3.3.1 Exadata Target Associations Not Displaying Properly in My Oracle Support

- 1.3.3.2 Cloning an OCM Installation Setup in Compatibility Mode Fails

- 1.3.3.3 OCM Collector Installed in Siebel Server and Gateway Directory Trees

- 1.3.3.4 Stale Associations for Virtual Machines Reflected in My Oracle Support UI

- 1.3.3.5 Termination of setupCCR/configCCR Commands Returns Configuration Message

- 1.3.3.6 Error Encountered Running installCCRSQL.sh

- 1.3.3.7 Start Date, Last Collection and Next Collection Times Inconsistent

- 1.3.3.8 Error When Reviewing Collected Configuration Information with Mozilla

- 1.3.3.9 Fusion MiddleWare 12.1.2: OCM Component

- 1.3.3.10 Java Error When Installing or Issuing OCM Commands

- 1.3.3.11 Could not create service <name> : 1072 during software installation

- 1.3.3.12 Metric Collection Error in Log File

- 1.3.3.13 OCM Fails If No Home Directory is Present

- 1.3.3.14 OCM Fails When Patching Oracle Identity Management and Oracle Web Tier

2 Discovery Issues

- 2.1 How the Oracle Database Target Appears My Oracle Support via OCM

- 2.1.1 Database Instrumentation for OCM Data Collection

- 2.1.2 What We Execute

- 2.1.3 DBMS_JOB.SUBMIT

- 2.1.3.1 Pre-10.0 Databases

- 2.1.3.2 Post-10.0 Databases

3 Collection Issues

- 3.1 Siebel Host Name Returned Twice

- 3.2 Database Previously Collected But Now Dropped

- 3.3 Database Collection Is Not Working

4 Common Errors

- 4.1 Oracle Configuration Manager Synchronization Messages

- 4.2 Errors in Oracle Database Trace File

5 Harvester Issues

- 5.1 Harvester Collection Fails If the state/upload/external Directory Is Missing

- 5.2 CSI Assignment UI Harvester Issues

- 5.2.1 Enterprise Manager Cloud Control Harvester has Never Run

- 5.2.2 Enterprise Manager Cloud Control Harvester has not Run Recently

- 5.3 Enterprise Manager Cloud Control Harvester Upload Problems

- 5.3.1 Collection Files are not Deleted After Upload

- 5.3.2 Errors in the emoms_pbs.trc File

- 5.3.3 Harvester is not E-Business Suite (EBS) Aware

- 5.3.4 Harvester Fails during Scheduled Run

- 5.4 Enterprise Manager Cloud Control Harvester Registration Problems

- 5.4.1 Errors When Assigning a Customer Support Identifier (CSI)

- 5.4.2 CSI Information not Displayed

- 5.4.3 CCR Authentication has Failed

- 5.5 Harvester Installation and Configuration Issues

- 5.5.1 After Upgrade, Duplicate Targets Displayed in My Oracle Support

- 5.5.2 OMS and Agent Not Displaying as a Target After Upgrade or New Install

- 5.5.3 Install OCM in an Agent Home

6 Support Hub/Repeater Issues

- 6.1 Debug Tracing For The Oracle Support Hub

- 6.2 Configuring the OCM Repeater Associated with Enterprise Manager 12c with a Proxy

A ORA Error Messages

- A.1 ORA-04021 Error

- A.2 ORA-01925 Error While Running installCCRSQL

Index

Preface

This guide describes the common installation and usage issues with the Oracle Configuration Manager.

Audience

This document is intended users who install and maintain systems running the Oracle Configuration Manager.

Documentation Accessibility

For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.

Access to Oracle Support

Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.

Related Documents

For more information about OCM, see the following documents:

- Oracle® Configuration Manager Release Notes

- Oracle® Configuration Manager Installation and Administration Guide

- Oracle® Configuration Manager Collection Overview

- Oracle® Configuration Manager Support Hub Guide

Conventions

The following text conventions are used in this document:

Convention

Meaning

boldface

Boldface type indicates graphical user interface elements associated with an action, or terms defined in text or the glossary.

italic

Italic type indicates book titles, emphasis, or placeholder variables for which you supply particular values.

monospace

Monospace type indicates commands within a paragraph, URLs, code in examples, text that appears on the screen, or text that you enter.

What's Changed

This table provides a brief overview of the document changes for the latest publication of the Oracle® Configuration Manager Troubleshooting and Support Guide:

Part Number

Change Summary

E49552-03

Added a known issue for Installing OCM on a Windows NFS Shared Drive Fails.
Added a known issue for OCM Does Not Start Manually After Upgrade to OCM 12.1.

Added a known issue for getHttpClientClasspath Variable Undefined Error After Upgrade to OCM 12.1.

Added a known issue for OCM Fails When Patching Oracle Identity Management and Oracle Web Tier.

E49552-02

Updated to version 12.1

1 Installation and Configuration Issues

This chapter decribes the installation and configuration issues encountered for Oracle Configuration Manager (OCM) for the following platforms:

- Windows-Specific Issues

- Linux, Solaris, Unix Issues

- All Platforms

1.1 Windows-Specific Issues

This section describes the installation and configuration issues encountered that are specific to the Windows platform:

- Upgrade Failure on Windows

- RAC Installs: OCM Does Not Pick Up Oracle Home Name for Its Service Names

- OCM Upgrade to 10.3.5 Fails When Using the -distribution Option

- Nonexistent Directory Error Encountered

- File Extension Error Encountered While Installing OCM

- Installing OCM on a Windows NFS Shared Drive Fails

- OCM Does Not Start Manually After Upgrade to OCM 12.1

- getHttpClientClasspath Variable Undefined Error After Upgrade to OCM 12.1

1.1.1 Upgrade Failure on Windows

When there is an Oracle Configuration Manager upgrade failure on the Windows environment, the log file is named update2.log.

1.1.2 RAC Installs: OCM Does Not Pick Up Oracle Home Name for Its Service Names

For some Real Application Cluster (RAC) installs on Windows platforms, Oracle Configuration Manager cannot consistently get the Oracle Home name from the registry. This causes problems with the name of the service that is created to control the Oracle Configuration Manager scheduler.

To resolve this issue: Prior to running setupCCR or configCCR, remove any existing OCM Service names, set the ORACLE_HOME_NAME environment variable to the name of the home as provided to OUI during installation. This allows the variable to be propagated to the service name. For example, the service name will now become Oracle%ORACLE_HOME_NAME%ConfigurationManager.

(Bug 7243846)

1.1.3 OCM Upgrade to 10.3.5 Fails When Using the -distribution Option

When you execute the command emCCR update_components -distribution=<OCM1035 zip file path>, you may see the following error:

<OCM1035 zip file path> is not a valid OCM distribution.

To resolve this issue, perform the following steps:

1. Unzip the 10.3.5 Oracle Configuration Manager collector kit to a temporary directory, for example, c:\temp-ccr.

2. Copy the contents of the c:\temp-ccr\inventory\pending directory to a different directory, for example, c:\ccr-packages.

3. Run the command emCCR update_components -staged_dir=c:\ccr-packages

4. Once this command completes successfully, both the c:\temp-ccr\ and c:\ccr-packages directories can be deleted.

(Bug 12539188)

1.1.4 Nonexistent Directory Error Encountered

Error: "%ORACLE_CONFIG_HOME% refers to a non-existent directory"

Quoting the directory path specification when setting the ORACLE_CONFIG_HOME environmental variable is not required and will result in the aforementioned error being displayed. Set the ORACLE_CONFIG_HOME variable to the directory specification without quotes.

1.1.5 File Extension Error Encountered While Installing OCM

Error: 'Input Error: There is no file extension in "<directory>"'

When installing Oracle Configuration Manager (OCM), an error occurs indicating there is no file extension in a directory. The directory indicated in the error is a substring corresponding to the current directory, whose path contains spaces.

The condition occurs if the following Windows Registry key is set to 1:

HKLM/SYSTEM/CurrentControlSet/Control/FileSystem/NtfsDisable8dot3NameCreation

If this setting is not critical to your operations, you can set the registry key to 0.

Note:
You must restart Windows if you chose to make changes to the registry key effective.

If this setting is critical to your operations, you must install Oracle Configuration Manager in a directory whose path does not contain spaces.

1.1.6 Installing OCM on a Windows NFS Shared Drive Fails

Installing Oracle Configuration Manager (OCM) on a Windows NFS shared drive fails with an error.

OCM must be installed on a local, non-removable, writeable drive. Unless the drive type is set to fixed, OCM may not be supported. The drive type should be set to fixed before the OCM Collector can be configured.

For instructions for setting the drive type property, see:

https://msdn.microsoft.com/en-us/library/ys4ctaz0%28v=vs.84%29.aspx

(Bug 20234326)

1.1.7 OCM Does Not Start Manually After Upgrade to OCM 12.1

After applying the patch to upgrade to OCM 12.1, OCM does not start manually. Running the emccr start command generates an error.

Workaround: Start the service manually from services.msc. Look for Oracle<oracle_home><oracle_config_home>ConfigurationManger service and start it manually.

(Bug 18107265)

1.1.8 getHttpClientClasspath Variable Undefined Error After Upgrade to OCM 12.1

Error: Variable is undefined: 'getHttpClientClasspath'

The OCM version present in the home might be much higher as compared to the version present being shipped as part of patch. If OCM is installed and configured already, it should not be over-written with another fresh kit during upgrade of the product.

Workaround: Copy the ccr\inventory\*.jar files to the ccr\inventory\pending directory and override when asked for conflict. Once copied, run the ccr\bin\deployPackages command.

(Bug 18004161)

1.2 Linux, Solaris, Unix Issues

This section describes the installation and configuration issues encountered for Linux, Solaris, and Unix platforms:

- OCM Fails During Installation of Oracle Database Release 11.2 on Linux x86-64

- Error Initializing JVM When Installing Oracle Configuration Manager

1.2.1 OCM Fails During Installation of Oracle Database Release 11.2 on Linux x86-64

Installation of the Oracle Configuration Manager fails, with the following error in $ORACLE_HOME/ccr/hosts/<hostname>/log/install-core-*.log, indicating a relink issue. The log file contains records such as:

skipping incompatible /usr/lib64/gcc/x86_64-suse-linux/4.3/libgcc.a when
searching for -lgcc
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld:
cannot find -lgcc
collect2: ld returned 1 exit status

The problem occurs on a 64-bit Linux system where the 32-bit libraries are not installed. Install the native Linux x86_bit kit located on the My Oracle Support site (http://support.oracle.com).

1.2.2 Error Initializing JVM When Installing Oracle Configuration Manager

When you install the Oracle Configuration Manager using a CRON script, you may encounter the following error:

Error occurred during initialization of VM
Unable to load native library: .../jdk/jre/lib/i386/libjava.so: symbol __libc_
wait, version GLIBC_2.0 not defined in file libc.so.6 with link time reference

The problem occurs when Java is invoked without the LD_PRELOAD variable being set. This is normally required on Redhat Advanced server configurations. (Refer to Oracle Patch 3006854).

To resolve this issue, define LD_PRELOAD to the preload module prior to invoking the installation of the Oracle Configuration Manager.

1.3 All Platforms

The following sections apply for all platforms:

- Enabling Debug Logging

- Errors You May Encounter

- Known Issues

1.3.1 Enabling Debug Logging

To enable debug logging by the collector, add the following property to the collector.properties file.

log4j.rootCategory=DEBUG, Rolling

If the $ORACLE_CONFIG_HOME variable was defined for the deployment of the collector, the location of the collector.properties file is the $ORACLE_CONFIG_HOME/ccr/config directory.

Otherwise, add the property to the <INSTALLED_PRODUCT_ROOT>/ccr/hosts/<hostname>/config/collector.properties file.

Note that debug logging of the collector consumes space. Once you have completed the debugging task, remove the added entry from the collector.properties file to resume the previous behavior.

To have Perl scripts generate debugging output when the collection is run, set EMAGENT_PERL_TRACE_LEVEL=1 and run emSnapshotEnv.

To see all the steps performed throughout the execution of the setupCCR, configCCR, and deriveCCR commands, set the CCR_DEBUG environment variable to 1. We suggest that the output be redirected to a file. This enables you to send the file to Oracle Support if needed.

1.3.2 Errors You May Encounter

This section describes some of the errors you may encounter while you work with Oracle Configuration Manager and provides possible solutions to these errors:

- Insufficient Privileges While Running installCCRSQL collectconfig

- Incorrectly Configured Hostnames are Displayed in My Oracle Support

- Oracle Home Environmental Changes not Detected During Collection

1.3.2.1 Insufficient Privileges While Running installCCRSQL collectconfig

When you run the installCCRSQL.sh script, it creates the ORACLE_OCM user and sets up a job to collect database configuration information. The ORACLE_OCM user requires EXECUTE privileges on UTL_FILE and DBMS_SCHEDULER for database versions 10g or higher, and on the DBMS_JOB for pre-10g databases. If these privileges are granted to PUBLIC, the ORACLE_OCM user inherits these privileges, otherwise these privileges are explicitly granted when the installCCRSQL.sh script is executed. If the inherited privileges are revoked, the following errors indicating the lack of privileges will be logged in the alert_log:

ORA-12012: error on auto execute of job 52
ORA-04068: existing state of packages has been discarded
ORA-04063: package body "ORACLE_OCM.<package name>" has errors
ORA-06508: PL/SQL: could not find program unit being called

To resolve these errors, you must grant the missing EXECUTE privilege to the ORACLE_OCM user.

- For database versions 10g and higher, grant EXECUTE privileges on the UTL_FILE and DBMS_SCHEDULER packages to the ORACLE_OCM user by entering the following SQL*PLUS commands:

SQL> grant execute on UTL_FILE to oracle_ocm;
SQL> grant execute on DBMS_SCHEDULER to oracle_ocm;
SQL> ALTER PACKAGE oracle_ocm.MGMT_DB_LL_METRICS compile;
SQL> ALTER PACKAGE oracle_ocm.mgmt_config compile;

- For pre-10g databases, grant EXECUTE privileges on the DBMS_JOB package to the ORACLE_OCM user by entering the following SQL*PLUS commands:

SQL> grant execute on UTL_FILE to oracle_ocm;
SQL> grant execute on DBMS_JOB to oracle_ocm;
SQL> ALTER PACKAGE oracle_ocm.MGMT_DB_LL_METRICS compile;
SQL> ALTER PACKAGE oracle_ocm.mgmt_config compile;

1.3.2.2 Incorrectly Configured Hostnames are Displayed in My Oracle Support

To ensure that hostnames are displayed with their fully qualified names on My Oracle Support, the /etc/hosts file must contain an entry that includes both the hostname and the domain in the following format:

<IP-Address>  <Full-HostName>  <Short-HostName>

For example:

10.10.10.10  myhost.mydomain  myhost

If the /etc/hosts file has not been correctly configured, only the short name is displayed on My Oracle Support.

1.3.2.3 Oracle Home Environmental Changes not Detected During Collection

The installation of Oracle Configuration Manager takes a snapshot of the process environment. These key environmental variables are stored such that scheduled collections behave the same even though they are running as detached processes.

If any of the following variables change, update the snapshot file by running the command: $ORACLE_HOME/ccr/bin/emSnapshotEnv.

Key environmental variables are: ORACLE_HOME, ORACLE_CONFIG_HOME, IAS_CONFIG_HOME, ORACLE_INSTANCE, JAVA_HOME, TZ, TNS_ADMIN, CRS_HOME, CLUSTER_NAME, LD_PRELOAD, ORAINST_LOC, BEA_HOME, and WL_HOME.

1.3.3 Known Issues

The following sections describe the known issues with OCM:

- Exadata Target Associations Not Displaying Properly in My Oracle Support

- Cloning an OCM Installation Setup in Compatibility Mode Fails

- OCM Collector Installed in Siebel Server and Gateway Directory Trees

- Stale Associations for Virtual Machines Reflected in My Oracle Support UI

- Termination of setupCCR/configCCR Commands Returns Configuration Message

- Error Encountered Running installCCRSQL.sh

- Start Date, Last Collection and Next Collection Times Inconsistent

- Error When Reviewing Collected Configuration Information with Mozilla

- Fusion MiddleWare 12.1.2: OCM Component

- Java Error When Installing or Issuing OCM Commands

- Could not create service <name> : 1072 during software installationMetric Collection Error in Log File

- OCM Fails If No Home Directory is Present

- OCM Fails When Patching Oracle Identity Management and Oracle Web Tier

1.3.3.1 Exadata Target Associations Not Displaying Properly in My Oracle Support

Oracle Harvester makes associations based on a target property called DBMachineID and it is the same for all partitions. Therefore, every partition comes under one Database Machine target on Oracle Configuration Manager. (A partitioned DB Machine can be a single rack machine or multi-rack machine.)

Because Oracle Harvester for Oracle Database Machine does not support a partitioned rack, users must set a unique value for the split_no target property.

To enable Oracle Harvester to collect each partition as a separate Database Machine, you need to set a unique value for the split_no target property for each partitioned Exadata target on Enterprise Manager.

Follow these steps to set the target property:

1. Collect all the partitioned Database Machine names of a single Exadata hardware.

2. Add a new target property split_no in the OMS

$ORACLE_HOME/emcli/bin/emcli add_target_property -target_type="oracle_dbmachine" -property="split_no";

3. Set a unique value of the split_no target property for each partitioned Database Machine:

$ORACLE_HOME/emcli/bin/emcli set_target_property_value -property_records="<target name of dbmachine>:oracle_dbmachine:split_no:<unique String>"

For example:

ORACLE_HOME/emcli/bin/emcli set_target_property_value -property_records="DB Machine host.example.com_2:oracle_dbmachine:split_no:2"

(Bug 14697857)

1.3.3.2 Cloning an OCM Installation Setup in Compatibility Mode Fails

Using deriveCCR to clone an Oracle Configuration Manager installation that was originally setup in compatibility mode (ORACLE_HOME environment variable was set to the same directory as ORACLE_CONFIG_HOME), to a cloned home that is NOT setup in compatibility mode fails.

To alleviate this problem, edit the collector.properties file located in <cloned_home>/ccr/hosts/<your_hostname>/config where <cloned_home> is the cloned directory. Change the ccr.binHome property to the full path of the cloned home.

For example, change:

ccr.binHome=/scratch/testocm/original/ccr
to
ccr.binHome=/scratch/testocm/cloned/ccr

1.3.3.3 OCM Collector Installed in Siebel Server and Gateway Directory Trees

If you manually installed the Oracle Configuration Manager collector in the server and gateway server directory trees, you should:

1. Stop and decommission those Oracle Configuration Manager collectors in the gateway server and server directories using the command configCCR -r

2. Login to My Oracle Support and disable the targets that were collected by the collector that was decommissioned.

3. Deploy the collector to the Siebel root directory if not already done.

Note: The current Oracle Configuration Manager release has been certified for 7.7, 7.8, 8.0, and 8.1 releases of CRM.

1.3.3.4 Stale Associations for Virtual Machines Reflected in My Oracle Support UI

When a virtual machine changes from a running state to a halted state or vice versa, it takes a maximum of 24 hours to reflect the updated information in My Oracle Support.

For example, if the running Guest Virtual Machine (GVMA) is halted, then GVMA is no longer associated to a Virtual Server but to the parent Virtual Server Pool. However, while viewing this relationship in My Oracle Support, one might see GVMA associated to the Virtual Server (implying a running GVMA) as well as to Virtual Server Pool (implying halted GVMA). Although Oracle Configuration Manager has collected new configuration data capturing these associations, the new snapshot is ignored because the collection time stamp of the Virtual Server snapshot does not change and is the same as the previous snapshot. Every 24 hours, a new snapshot is uploaded to Oracle with a new collection time stamp. When this new snapshot is uploaded to Oracle, the new data is reflected in My Oracle Support.

Note:
For virtual machine collections, it takes 24 hours to reflect the state change.

1.3.3.5 Termination of setupCCR/configCCR Commands Returns Configuration Message

An abnormal termination such as a Control-C during the invocation of setupCCR or reconfiguration using configCCR may result in subsequent attempts returning the message:

This installation is already configured for OCM. Please remove existing configuration first.

To manually reset the environment, remove the config and state directories located under the $ORACLE_HOME/ccr/hosts/<hostname> directory. If ORACLE_CONFIG_HOME is set, remove the config and state directories in the $ORACLE_CONFIG_HOME/ccr directory.

If this is a fresh installation, delete the ccr directory and unzip the Oracle Configuration Manager distribution into the ORACLE_HOME and issue the setupCCR command again.

1.3.3.6 Error Encountered Running installCCRSQL.sh

When you run the $ORACLE_HOME/ccr/admin/scripts/installCCRSQL.sh script, you may encounter the following error:

An error had occurred
For details, check the log file at /u01/app/oracle/10.2.0/db/ccr/log/collectconfigasmdb.log

The contents of the log file identified contains the output:

SQL*Plus: Release 10.2.0.1.0 - Production on Mon Oct 17 17:54:35 2005
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.

Connected to an idle instance.

The log file ends with the text:

ERROR at line 1:
ORA-01034: ORACLE not available

This error may occur if the $ORACLE_HOME environmental variable used to start the database instance ended with a '/' character.

To resolve this issue, stop the database instance and redefine the ORACLE_HOME so that it does not include the trailing slash and restart the database. Once the database is restarted, re-run the installCCRSQL.sh script.

1.3.3.7 Start Date, Last Collection and Next Collection Times Inconsistent

The emCCR status command displays the state of the scheduler, when a collection was last performed, when the next collection will run and the schedulers start time. These times may appear to be inconsistent.

The system's TZ variable affects how the time stamps are stored. Collections performed manually vs. automatically store their collection times based upon the TZ variable of the process invoking the collection.

To resolve this inconsistency, insure the cron daemon is started with the correct system time zone.

1.3.3.8 Error When Reviewing Collected Configuration Information with Mozilla

Error: Error loading stylesheet: An XSLT stylesheet does not have an XML mimetype

Mozilla contains a bug that does not recognize XSL stylesheets correctly. The aforementioned error is displayed as a result.

Configure Mozilla by selecting the Edit -> Preferences... item from the menu. Add a New Type under the Navigator -> Helper Applications entry for a MIME type of 'text/xml' and an extension of 'xsl'. Click on 'Proceed anyway' when the warning message indicating 'Mozilla can handle this type' is presented.

1.3.3.9 Fusion MiddleWare 12.1.2: OCM Component

For Fusion MiddleWare 12.1.2, a mini version of the OCM collector kit is included. This version of the kit includes a minimal set of components that are needed for initial configuration. During configuration if the OCM is connected, the remaining components are download and installed.

If the OCM is not connected, the remaining components must be installed manually. You can download the full OCM collector kit from My Oracle Support (download the latest version of the patch for bug 5567658):

https://support.oracle.com

Once you have downloaded the patch, update OCM with the following command:

cd $FMW_HOME/oracle_common/ccr/bin/emCCR update_components  -distribution=/scratch/distribution/ccr-Production-10.3.8.0.0-Linux-i386.zip

For more details, refer to Section 6.16, "emCCR update_components" of the Oracle® Configuration Manager Installation and Administration Guide:

http://docs.oracle.com/cd/E37284_01/doc.1038/e37289/manage.htm#CACGEHHJ

(Bug 15875776)

1.3.3.10 Java Error When Installing or Issuing OCM Commands

Error: Java version not able to be identified

When installing Oracle Configuration Manager, an error indicating that the JAVA version was not able to be identified is returned, however, one of the following is true:

- JAVA_HOME is defined and the following command returns the JDK version %JAVA_HOME%\bin\java -version

- The current directory is the install root and the following command also returns the JDK version jdk\bin\java -version

The condition occurs if the TMP environmental variable contains a directory specification containing a space and the Windows Registry key is set to 1:

HKLM/SYSTEM/CurrentControlSet/Control/FileSystem/NtfsDisable8dot3NameCreation

Set the registry key to 0 and the TMP environmental variable for the user to a value that does not contain a space.

Note:
You must restart Windows if you chose to make changes to the registry key effective.

1.3.3.11 Could not create service <name> : 1072 during software installation

This error may come on some Microsoft operating systems especially on Windows 2000 and Windows NT systems. It is primarily due the fact that the service under consideration is marked for deletion. During the upgrade process the Oracle Configuration Manager removes the old service and creates a new one.

If the old service is selected in the Windows Service Control Manager (Services) interface in Control Panel or he/she is looking at the old service properties then service removal fails to completely remove the old service.

To correct this problem, close the Service Control Manager (Services) interface and retry the upgrade operation.

1.3.3.12 Metric Collection Error in Log File

The execution of a collection may result in the error:

Failed to execute command - "<ORACLE_HOME>/ccr/engines/Linux/perl/bin/perl"
<ORACLE_HOME>/ccr/sysman/admin/scripts/hostosfile.pl"

The Oracle Configuration Manager times out individual configuration collections if the collection does not complete within 5 minutes. This specific case manifests itself on LINUX if a NFS mount point is not responding to a df -k command.Identify the failing mount point by iterating through the listed filesystems in /etc/mtab by running the ls command on the location. Resolve the problem with the failing NFS server. Alternatively you can remove the entry from the mtab file and the NFS file service will not be monitored. This file is re-created when a filesystem is mounted.

1.3.3.13 OCM Fails If No Home Directory is Present

If a user home directory does not have write permissions, then OCM returns the following error:

chmodFile.sh returned:1

Workaround:

1. Set the CCR_USER_HOME environment property variable to a directory where user of OCM has write permissions

2. Run the following command to allow the OCM collector to capture the new property:

ccr/bin/emSnapshotEnv

Once you complete the steps above, you should no longer see the error.

(Bug 17895226)

1.3.3.14 OCM Fails When Patching Oracle Identity Management and Oracle Web Tier

If you are upgrading Oracle Identity Management or Oracle Web Tier to 11g Release 1 (11.1.1.9.0) from any release prior to and not including 11g Release 1 (11.1.1.6.0), and you did not previously configure Oracle Configuration Manager (OCM), then OCM will fail if you decide to configure it in 11g Release 1 (11.1.1.9.0).

Workaround:

To work around this issue, follow the steps below prior to running the 11g Release 1 (11.1.1.9.0) configuration wizard:

1. Navigate to the ORACLE_HOME/ccr/bin directory.

2. Run the following commands:

setupCCR
configCCR
emCCR collect
emCCR status

If you choose to skip the OCM configuration when you initially run the 11g Release 1 (11.1.1.9.0) configuration wizard, but then choose to configure it later, then follow the steps below:

1. Navigate to the ORACLE_HOME/ccr/bin directory.

2. Set the ORACLE_CONFIG_HOME environment variable to your instance home directory.

3. Run the following commands:

setupCCR
configCCR
emCCR collect
emCCR status

(Bug 16450488)

2 Discovery Issues

This chapter describes the issues encountered during the Discovery process for the Oracle Configuration Manager (OCM).

2.1 How the Oracle Database Target Appears My Oracle Support via OCM

This chapter explains the logic flow that takes place when a Database target is being collected by the standalone Oracle Configuration Manager (OCM) collector.

It is important to ensure that collections for Oracle Database targets are discovered and consistently updated in My Oracle Support (MOS) so that users can validate, review, and perform futher proactive actions in MOS. For example, gathering patch advice, creating Service Requests, and any other administrative type functions current available in MOS relating to a Database Target.

As an OCM and MOS user, it is equally important to understand the process of how a Database target gets populated into MOS and in some cases, how to troubleshoot when a Database Target information either goes 'stale' (LAST_COLLECTED column not updated) in MOS or simply no longer appears as a Target altogether.

2.1.1 Database Instrumentation for OCM Data Collection

The Database instrumentation for OCM data collection includes several phases:

- Verification that the database is in OPEN READ WRITE status.

- Verification that the Database is supported (version 8.1.7 or later).

- The checking of the utl_file_dir database initialization parameter contains the appropriate directory (the state directory where the output from the instrumentation will go) in versions 9.0.1 and earlier.

- The checking of the 'Compatible' database initialization parameter is 8.1 or higher in version 8.1.7.

- The checking of the 'job_queue_processes' database initialization parameter is greater than zero in versions earlier than 10.0.

- The verification that any existing OCM user and its artifacts are dropped if present.

Note:
If the Database version is 11.0 or greater, the Access Control List (ACL) created for OCM to use (oracle-sysman-ocm-Resolve-Access.xml) is dropped, or the ORACLE_OCM user's privilege to use another ACL for access to the UTL_INADDR package is removed.

2.1.2 What We Execute

If the ORACLE_OCM.MGMT_DB_LL_METRICS PL/SQL package exists, the ORACLE_OCM user is dropped (drop user ORACLE_OCM cascade).

If ORACLE_OCM_CONFIG_DIR or ORACLE_OCM_CONFIG_DIR2 directory objects exist, they are dropped.

Then, we grant:

- EXECUTE on SYS.UTL_FILE

- EXECUTE on SYS.UTL_INADDR

- EXECUTE on SYS.DBMS_SQL

- EXECUTE on SYS.DBMS_JOB (pre-10.0)

- EXECUTE on SYS.DBMS_FLASHBACK (pre-10.0)

- SELECT ANY on lbacsys.lbac$polt (pre-10.0)

- SELECT ANY on odm.odm_mining_model (pre-10.0)

- SELECT ANY on olapsys.dba$olap_cubes (pre-10.0)

- EXECUTE on SYS.DBMS_SCHEDULER (10.0 +)

- CREATE JOB (10.0 +)

- SELECT ANY on DVSYS.DBA_DV_REALM

- SELECT ANY on content.odm_document

- SELECT ANY on content.odm_record

- For 11.0+, access is granted via ACL to UTL_INADDR for resolution of the host name for ”localhost”. If an ACL exists for this access, the ORACLE_OCM user is added to the access list. If not, an ACL is created for this purpose and assigned accordingly.

- SELECT ANY TABLE (8.1.7 only)

- SELECT ANY DICTIONARY (> 8.1.7)

- The ORACLE_OCM_CONFIG_DIR or ORACLE_OCM_CONFIG_DIR2 directory objects are created, and ORACLE_OCM is granted READ and WRITE on both

- The ocmdbd and ocmdbb scripts are run – creating the MGMT_DB_LL_METRICS package under the ORACLE_OCM user

- The ocmjd/ocmjb (pre10.0) or ocmjd10/ocmjb10 (10.0 +) scripts are run to create the MGMT_CONFIG package under the ORACLE_OCM user

- Finally, the job for configuration collection is submitted and immediately started via

- ORACLE_OCM.MGMT_CONFIG.submit_job();

- ORACLE_OCM.MGMT_CONFIG.run_now();

2.1.3 DBMS_JOB.SUBMIT

Let's look at what happens with the submit_job and run_now job runs:

- Pre-10.0 Databases

- Post-10.0 Databases

2.1.3.1 Pre-10.0 Databases

In pre-10.0 databases, two jobs are created:

DBMS_JOB.SUBMIT(l_job, 'ORACLE_OCM.MGMT_CONFIG.collect_config;', TRUNC(SYSDATE) + 1/24, 'TRUNC(SYSDATE) + 1 + 1/24');

DBMS_JOB.SUBMIT(l_job, &rsquor;ORACLE_OCM.MGMT_CONFIG.collect_stats;', TRUNC(LAST_DAY(SYSDATE)) + 1/24,  'TRUNC(LAST_DAY(SYSDATE) + 1) + 1/24');

Here we use the DBMS_JOB.SUBMIT procedure to create the jobs, the first using the current date/time (rounded to the next hour) as the initial run time with an interval of 24 hours, the second using the last day of the current month as the first run date and an interval of one month.

The first job will run the MGMT_CONFIG.collect_config procedure; the second will run MGMT_CONFIG.collect_stats.

In MGMT_CONFIG.collect_config and MGMT_CONFIG.collect_stats, a cursor is created to select all inst_id values from gv$instance. In a loop, we iterate overall the instances and submit a distinct job for each:

DBMS_JOB.SUBMIT(l_job, p_what, SYSDATE, NULL, FALSE, inst_id);

We end up with a job per instance – in a RAC environment we will have one per running member of the RAC configuration. p_what is either &rsquor;MGMT_DB_LL_METRICS.collect_config_metrics' or &rsquor;MGMT_DB_LL_METRICS.collect_stats_metrics' plus a directory path enclosed in parentheses (i.e. a parameter to the procedure) indicating where the procedure's output is to be written. This can cause a problem in that we may not see output for all members of the RAC, since the path may contain the host name where the instrumentation script was run ($ORACLE_HOME/ccr/hosts/<hostname>/state). If a host in the RAC setup does not have read/write access to this path, the job will fail. The end result is that on the hosts where the job failed we will not discover the RAC database, and we end up with inconsistent information.

The workaround for this is to set ORACLE_CONFIG_HOME=$ORACLE_HOME prior to the DataBase instrumentation. This will result in the job output going to the $ORACLE_HOME/ccr/state location.

2.1.3.2 Post-10.0 Databases

In post-10.0 databases, we again create two jobs:

sys.dbms_scheduler.create_job(
job_name => 'MGMT_CONFIG_JOB',
job_type => 'STORED_PROCEDURE',
job_action => 'ORACLE_OCM.MGMT_CONFIG.collect_config',
schedule_name=> 'SYS.MAINTENANCE_WINDOW_GROUP',
enabled => TRUE,
auto_drop => FALSE,
comments => 'Configuration collection job.');

and

sys.dbms_scheduler.create_job(
job_name => 'MGMT_STATS_CONFIG_JOB',
job_type => 'STORED_PROCEDURE',
job_action => 'ORACLE_OCM.MGMT_CONFIG.collect_stats',
start_date=> SYSTIMESTAMP,
repeat_interval => 'freq=monthly;interval=1;bymonthday=1;byhour=01;byminute=01;bysecond=01',
end_date => NULL,
enabled => TRUE,
auto_drop => FALSE,
comments => 'OCM Statistics collection job.');

Again - the first job runs MGMT_CONFIG.collect_config, the second MGMT_CONFIG.collect_stats - but here we use DBMS_SCHEDULER.CREATE_JOB rather than DBMS_JOB.SUBMIT. Note that the collect_config job uses the SYS.MAINTENANCE_WINDOW_GROUP for its schedule rather than a specific schedule, as is the case with the collect_stats job.

The procedures themselves are also slightly different in 10.0+ databases. They still set up a cursor and loop over the instances found in gv$instance, but the underlying logic for collect_config will create two jobs per instance if we determine we are in a RAC configuration (select parallel from v$instance returns &rsquor;YES') and the instance for which we are submitting the jobs is NOT the current instance:

sys.dbms_scheduler.create_job(
job_name => p_job_name || '_' || inst_id,
job_type => 'PLSQL_BLOCK',
job_action => p_job_action,
start_date => NULL,
repeat_interval => NULL,
enabled => FALSE,
auto_drop => TRUE,
comments => 'OCM collection job run for an instance.');

p_job_name here is 'MGMT_CONFIG_JOB', p_job_action is

'BEGIN ORACLE_OCM.MGMT_DB_LL_METRICS.COLLECT_CONFIG_METRICS (''ORACLE_OCM_CONFIG_DIR''); END;'

sys.dbms_scheduler.create_job(
job_name => p_job_name || '_2_' || inst_id,
job_type => 'PLSQL_BLOCK',
job_action => p_job_action2 ,
start_date => NULL,
repeat_interval => NULL,
enabled => FALSE,
auto_drop => TRUE,
comments => 'OCM 2nd job run for RAC instance.');

p_job_name is &rsquor;MGMT_CONFIG_JOB' again, but p_job_action is

'BEGIN ORACLE_OCM.MGMT_DB_LL_METRICS.WRITE_DB_CCR_FILE (''ORACLE_OCM_CONFIG_DIR2'', TRUE); END;'

What this second job accomplishes is creation of a minimal file with the information necessary for discovery, using the secondary directory object created during instrumentation to find the output directory. This directory object points at $ORACLE_HOME/ccr/state, which should exist – and hopefully be writeable – on all hosts in the RAC setup.

The collect_stats job is created like the first collect_config job, i.e.:

sys.dbms_scheduler.create_job(
job_name => p_job_name || '_' || inst_id,
job_type => 'PLSQL_BLOCK',
job_action => p_job_action,
start_date => NULL,
repeat_interval => NULL,
enabled => FALSE,
auto_drop => TRUE,
comments => 'OCM collection job run for an instance.');

p_job_name here is 'MGMT_STATS_CONFIG_JOB', p_job_action is:

'BEGIN ORACLE_OCM.MGMT_DB_LL_METRICS.COLLECT_STATS_METRICS (''ORACLE_OCM_CONFIG_DIR''); END;'

It's OK if the output from this job only shows up on one host in the RAC setup, since this is the feature usage information common to all instances. As long as we discover the database on the host where this output shows up we will have the data in the collection.

As each of these jobs is created, they are handed off to the DBMS_SCHEDULER subsystem with two calls:

DBMS_SCHEDULER.SET_ATTRIBUTE (p_job_name || '_' || inst_id,'instance_id',inst_id);
DBMS_SCHEDULER.ENABLE (p_job_name || '_' || inst_id);

The first call ties the job to the specified instance (as retrieved in the loop over instance IDs).

The second enables the job – we created it in a DISABLED state (enabled=FALSE) so that we could set the instance_id attribute first. Note that in a non-RAC environment the call to SET_ATTRIBUTE will throw an exception – but the procedure catches this exception and discards it. We're running on a single instance in this case anyway.

Here is a listing of some important files that directly relate to the discovery, population, and ultimately the proper listing of a Database Target in MOS:

Files

Purpose

oracledb.pl

Main Database collection file

collectconfig.sh

Calls MGMT_CONFIG.submit_job which then calls MGMT_CONFIG.collect_config

ocmdbb.sql

OCM DB configuration collection package Body

oracle-sysman-ocm-Resolve-Access.xml

The Access Control List (ACL) created for OCM to use

ocmdbd.sql

OCM DB configuration collection package Definition

ocmdbb.sql

OCM DB configuration collection package Body

ocmjb10.sql

OCM db config collection Job package Body for 10g onwards

ocmjb.sql

OCM db config collection Job package Body for pre 10g

ocmjd10.sql

OCM db config collection Job package Definition for 10g onwards

ocmjd.sql

OCM db config collection Job package Definition for pre 10g

3 Collection Issues

This chapter describes the issues encountered with the various collections supported by Oracle Configuration Manager (OCM).

3.1 Siebel Host Name Returned Twice

Customers are seeing the host names containing Siebel services twice: once with the long name and the second with the short name.

The Siebel topology is discovered from the host that runs the Siebel Gateway Server. A host definition is created and uploaded to Oracle services with the intent that the host configuration definition will rendezvous at Oracle when an Oracle Collection Manager collector is installed on the physical host where the Siebel service runs. Due to host network configuration, the name of the host is returned as different strings; one from the gateway server and another from the physical host. It is required that all hosts within the Siebel topology be configured to return full canonic names for hosts. This is true for the Siebel Gateway server, as well as from all hosts containing Siebel services.

Reconfigure your host network to return the full canonic name for the local host. On the Gateway Server, the network configuration must return the full canonic name for all hosts that are known to participate in a Siebel system defined by the Gateway Server.

3.2 Database Previously Collected But Now Dropped

Customers are seeing messages similar to the following message when a database that's been previously collected but is now dropped.

WARN: Oracle Configuration Manager database objects are not in sync with the
installed configuration collection scripts. Refer to the Installation and
Configuration documentation on reloading the SQL collection packages.

Use emCCR disable_target command. The emCCR status does not look at the list of disabled targets to determine whether it conforms to the list of targets for which it should check metadata.

3.3 Database Collection Is Not Working

If a database instance is not discovered even though the Oracle Configuration Manager instrumentation step reports no errors, ensure that the JOB_QUEUE_PROCESSES parameter is set to a value greater than 0 (zero). This ensures that the DBMS jobs script runs and in turn the Oracle Configuration Manager instrumentation works.

You can set the JOB_QUEUE_PROCESSES parameter using the ALTER command:

alter system set job_queue_processes=<n> scope=both;
where <n> represents the number of JOB_QUEUE_PROCESSES

It is suggested that you set the JOB_QUEUE_PROCESSES parameter to 10.

4 Common Errors

This chapter describes the common errors you may encounter when running the Oracle Configuration Manager.

4.1 Oracle Configuration Manager Synchronization Messages

Oracle Configuration Manager does not allow you to run multiple commands simultaneously. If you attempt to do so, the following messages may be displayed:

- Message: Another operation is in progress. Please wait...
Description: There are several Oracle Configuration Manager commands that cannot run concurrently. If you try to run one of these commands while another command is in progress, the second command will not be executed until the first command is completed. A message indicating that another command is in progress is displayed. The second command will automatically be run when the first command is completed.

Commands: emCCR collect, emCCR getupdates, emCCR update_components, and emCCR upload

Action: Initially, take no action, the second command will be executed when the first command is completed. But if the command execution takes too long, a timeout will occur. If a timeout occurs, ensure there is no Oracle Configuration Manager activity by executing emCCR stop command. Delete the ccr/state/collector.lock file and restart the Scheduler by running the emCCR start command. If you are running the command in Disconnected mode, ensure that no collection or update is taking place and then delete the ccr/state/collector.lock file.

- Message: Operation blocked, waiting...
Description: You cannot run the emCCR update_components command if any other emCCR command is running. If you try to run the command, it will be blocked. You also cannot run any emCCR command while emCCR update_components is running as all other commands will be blocked.

Commands: configCCR and most of the emCCR commands

Action: Initially, take no action, the command will get executed when the current command is completed. If a timeout occurs, ensure that there is no Oracle Configuration Manager activity by executing emCCR stop. Delete the ccr/state/semaphore.op* and ccr/state/semaphore.update* files, and restart Oracle Configuration Manager by running emCCR start. If running the command in Disconnected mode, ensure no collection or update is taking place and delete the ccr/state/semaphore.op* and the ccr/state/semaphore.update* files.

- Message: The Scheduler is down for upgrade.
Description: While upgrading Oracle Configuration Manager, you cannot run any of the emCCR commands.

Commands: All emCCR commands

Action: Retry the commands later.

4.2 Errors in Oracle Database Trace File

The following errors in the Oracle Database Trace file denote that the ORACLE_OCM user account does not have Execute Privilege on the UTL_FILE located in the SYS.UTL_FILE package.

ORA-12012: error on auto execute of job xxxxx
ORA-04063: package body "ORACLE_OCM.MGMT_DB_LL_METRICS" has errors
ORA-06508: PL/SQL: could not find program unit being called:
"ORACLE_OCM.MGMT_DB_LL_METRICS"
ORA-06512: at line 1

To give Execute Privilege to the UTL_FILE, run the following SQL statement:

GRANT EXECUTE ON SYS.UTL_FILE TO ORACLE_OCM;

At the time of instrumentation of the database, Oracle Configuration Manager executes the ccr/admin/execocm.sql script. This script tests to see if PUBLIC access is granted to all the routines Oracle Configuration Manager needs access to. If PUBLIC access is not available, Oracle Configuration Manager grants specific access rights.If you specifically revoke PUBLIC access after installation, you will need to rerun at least the execocm.sql script. Rerunning the collecconfig.sql script accomplishes the same result.

5 Harvester Issues

This chapter describes the issues and problem resolutions for supported harvesters in Oracle Configuration Manager. The following problem areas are addressed:

- Harvester Collection Fails If the state/upload/external Directory Is Missing

- CSI Assignment UI Harvester Issues

- Enterprise Manager Cloud Control Harvester Upload Problems

- Enterprise Manager Cloud Control Harvester Registration Problems

5.1 Harvester Collection Fails If the state/upload/external Directory Is Missing

If you are running Enterprise Manager (EM) and the harvester collection fails with the following error in the sysman/log/emoms.log file:

[JobWorker 75210:Thread-61] ERROR gcharvester.GcCollectionMgr initOcm.? - GC OCM
Harvester: Caught GC Harvester exception from GCInit.init(): The installed
version of Oracle Configuration Manager in the ORACLE_HOME
(/scratch/aime/work/midlwre8937/oms11g) is prior to 10.3.1. The Grid Control
Configuration harvesting requires at a minimum, 10.3.1

the required directory named external is missing in either the $ORACLE_HOME/ccr/hosts/<local_host_name>/state/upload directory (EM release 10.2.0.5) or the ORACLE_INSTANCE_HOME/ccr/state/upload directory (EM releases 11.1 and later).

To resolve this issue, create the directory $ORACLE_HOME/ccr/hosts/<local_host_name>/state/upload/external (EM release 10.2.0.5) or ORACLE_INSTANCE_HOME/ccr/state/upload/external (EM releases 11.1 and later).

5.2 CSI Assignment UI Harvester Issues

The following harvester issues are addressed:

- Enterprise Manager Cloud Control Harvester has Never Run

- Enterprise Manager Cloud Control Harvester has not Run Recently

5.2.1 Enterprise Manager Cloud Control Harvester has Never Run

Problem: The Enterprise Manager Cloud Control Harvester has never run.

Cause: Possible causes include:

- The harvester job is disabled.

- The OCM collector is not installed/configured in the OMS_Home location.

Solution: See errors in the emoms_pbs.trc file located in:

<Middleware_Home>/gc_inst/em/EMGC_OMS1/sysman/log

Or, you can run the following command:

egrep -i emoms_pbs.trc |grep harvester

5.2.2 Enterprise Manager Cloud Control Harvester has not Run Recently

Problem: The harvester has not run in the last {0} days.

Cause: The following causes are possible:

- The harvester job was running successfully for a period of time but now its not running.

- The harvester job has been disabled.

- The OCM collector is not installed/configured in the OMS_Home location.

Solution: See errors in the emoms_pbs.trc file or mgmt_system_error_log table (In Enterprise Manager Cloud Control, click Setup, then Management Services and Repository, and finally Repository Operations).

If there are no errors being reported in the trace files, then perhaps the harvester needs to be started and scheduled. In that case, you need to start the harvester using the gcharvester_job.sql script found in the OMS_HOME/sysman/admin/emdrep/sql/core/latest/gcharvester directory as described below:

declare
l_already_registered NUMBER :=0;
l_job_targets MGMT_JOB_TARGET_LIST := MGMT_JOB_TARGET_LIST();
l_params MGMT_JOB_PARAM_LIST := MGMT_JOB_PARAM_LIST();
l_schedule MGMT_JOB_SCHEDULE_RECORD;
l_job_id_out RAW(16);
l_execution_id_out RAW(16);
l_version varchar2(32);
l_days MGMT_JOB_INT_ARRAY := MGMT_JOB_INT_ARRAY(6);

begin
-- first check if job is already registered (possibly in earlier version of EM)
BEGIN
SELECT 1 into l_already_registered
FROM MGMT_JOB
WHERE job_name='GCHARVESTERJOB' AND job_type='GCHarvesterRun';

EXCEPTION
WHEN NO_DATA_FOUND THEN
null;
END;

IF (l_already_registered = 1) THEN
MGMT_JOBS.stop_all_executions(p_job_name => 'GCHARVESTERJOB');
COMMIT  ;
END IF;

-- start a daily job, running at 2 am
l_schedule :=
MGMT_JOBS.get_job_schedule_record(p_frequency_code => MGMT_JOBS.DAILY_FREQUENCY_CODE,
p_start_time => SYSDATE + 1,
p_end_time => null,
p_execution_hours => 2,
p_execution_minutes => 0,
p_interval => 0,
p_months => null,
p_days => null,
p_timezone_info => MGMT_JOBS.TIMEZONE_REPOSITORY,
p_timezone_target_index => 0,
p_timezone_offset => 0,
p_timezone_region => null,
p_start_grace_period => 240);

MGMT_JOBS.submit_job(p_job_name => 'GCHARVESTERJOB',
p_description => 'GC Harvester Job',
p_job_type => 'GCHarvesterRun',
p_job_targets => l_job_targets,
p_job_params => l_params,
p_schedule => l_schedule,
p_job_id_out => l_job_id_out,
p_execution_id_out => l_execution_id_out,
p_owner => null,
p_system_job => MGMT_JOB_ENGINE.SYSTEM_JOB,
p_job_creds => null,
p_job_target_type => null,
p_job_notify_states => null,
p_running_time_limit => -1);

update MGMT_JOB set job_owner= MGMT_USER.GET_REPOSITORY_OWNER where job_name='GCHARVESTERJOB' AND job_type='GCHarvesterRun';
MGMT_LOG.register_logging_module('GCHARVESTER','GC Harvester', NULL);

commit;
end;
/

The two main arguments are hour to run (execution_hours) and minute to run (execution_minutes).

As long as the time entered is not in the past, the script will run fine allowing the harvester to function properly.

5.3 Enterprise Manager Cloud Control Harvester Upload Problems

The following harvester upload problems are addressed:

- Collection Files are not Deleted After Upload

- Errors in the emoms_pbs.trc File

- Harvester is not E-Business Suite (EBS) Aware

- Harvester Fails during Scheduled Run

5.3.1 Collection Files are not Deleted After Upload

Problem: Collection files are not being deleted after an upload.

Background: In Enterprise Manager Cloud Control 12c, the harvester uploads its data directly to the CCR repository. As part of this process flow, the collection files are deleted as soon as they are uploaded. However, in some cases where a collection is failing or there is a concern about targets and their timestamps, Oracle Support may have you set the delete_upload_file parameter to retain those collection files so they can be reviewed.

Solution: With a connection to the repository as the SYSMAN user, the delete_upload_file property in the mgmt_ocm_upl_props table needs to be set to False as follows:

SQL> insert into mgmt_ocm_upl_props(name, str_value) values('delete_upload_file','false');
SQL> commit;

Note:
At this point, the OMS needs to be restarted in order for that parameter to take effect.

By default, the files will be created in the $INSTANCE_HOME/sysman/ocm/harvester/tmp directory. The files themselves will be incremental in nature.

5.3.2 Errors in the emoms_pbs.trc File

Problem: In some cases, the following error may be noticed in the emoms_pbs.trc file:

2012-03-29 02:00:04,261 [JobWorker208183:JobThreadPool:Long-System:Thread-641692] ERROR
gcharvester.HarvesterJobUtils performHistoryBasedUpdates.? - GC OCM Harvester: Failed to populate OCM metrics based on ECM history: ORA-01878:
specified field not found in datetime or interval ORA-06512: at line 358

Cause: This error is a clear indication that you are encountering a problem documented in Bug 13972728.

Workaround:

1. Locate the OMS instance home.

2. In the $ORACLE_HOME/sysman/config/emInstanceMapping.properties file (where ORACLE_HOME is the Oracle Home of the OMS), there is an entry referencing a file called emgc.properties.

3. The directory in which the emgc.properties file is located is the "instance home" of the OMS. In the following example, /u01/app/oracle/product/gc_inst/em/EMGC_OMS1 is the instance home of the OMS:

EMGC_OMS1=/u01/app/oracle/product/gc_inst/em/EMGC_OMS1/emgc.properties

4. Set the environment variable ORACLE_CONFIG_HOME to the directory of this emgc.properties file. For example:

$export ORACLE_CONFIG_HOME=/u01/app/oracle/product/gc_inst/em/EMGC_OMS1

5. Run the emCCR register command:

cd $ORACLE_HOME/ccr/bin
emCCR register

The next scheduled time the harvester job runs, it will trigger a full collection. In a multiple-OMS environment, this emCCR register command will need to be completed in all locations.

5.3.3 Harvester is not E-Business Suite (EBS) Aware

Problem: In an E-Business Suite (EBS) environment, the harvester is not 'EBS' aware.

Cause: EBS is not currently a harvested target from Enterprise Manager Cloud Control.

Workaround: The OCM standalone product will need to be used in order to upload the data to My Oracle Support.

5.3.4 Harvester Fails during Scheduled Run

Problem: During the scheduled harvester run (default is 2:00 a.m.), the job fails and the following error is dumped into the emoms.trc file:

2012-10-20 02:00:30,781 [JobWorker 73854412:Thread-86] ERROR
gcharvester.HarvesterJobUtils prepareTargetsForShipment.? - GC OCM Harvester: Failed to stage targets for shipment to Oracle: ORA-01031: insufficient privileges ORA-06512: at line 9

At this point, we are trying to create a sequence object and failing.

Cause: The SYAMAN user lacks the privilege to create the sequence. This sequence is used to put all harvested targets in mgmt_ocm_upl_metrics_tmp and then process them in batches of 512 targets at a time. First 1 to 512 will be processed, then 513 to 1024 and so on.

Solution: Assign the SYSMAN user the CREATE SEQUENCE privilege:

GRANT CREATE SEQUENCE, DROP ANY SEQUENCE TO SYSMAN;

5.4 Enterprise Manager Cloud Control Harvester Registration Problems

At first run, the harvester performs an Enterprise Manager Cloud Control Default registration using the registration information of the OCM collector in the OMS location. As part of this registration, the EM Repository URL is also sent for bookkeeping purposes on the CCR side of things. All other Oracle_Homes, which are considered to be a proxy uploaded by the harvester, are uploaded using the Default registration (this is similar to 11GC).

When a Target is added, this Target Oracle Home registration is performed after a user goes to UI CSI assignment page and assigns a CSI to it.

The registration process itself has a two-step approach:

1. Authenticate: Done as part of the UI which authenticates with the CCR database.

2. Register: The harvester internal job picks up on any Pending targets and registers itself against the Oracle CCR database.

Note:
If there are multiple OMS locations, each harvester needs to be registered with the same My Oracle Support Single Sign On (SSO) credential.

In Enterprise Manager Cloud Control 12c, the harvester now uploads information to CCR directly and no longer relies on OCM to perform this task. It also deletes the files as soon as they are uploaded.

Figure 5-1 Registration Types

The following problems with harvester registration are addressed:

- Errors When Assigning a Customer Support Identifier (CSI)

- CSI Information not Displayed

- CCR Authentication has Failed

5.4.1 Errors When Assigning a Customer Support Identifier (CSI)

Problem: While assigning a Customer Support Identifier (CSI), the following error is returned in a pop-up window: You don't have the privilege to do this operation.

Cause: The user does not have the Operator privilege on one or more targets within a given Oracle_Home for that target

Solution:

5.4.2 CSI Information not Displayed

Problem: The CSI drop-down listing does not show any CSI information.

Cause: There is a chance that the My Oracle Support connectivity from the OMS location is not successful.

Solution: Review the emoms.trc file for any possible communication issue.

5.4.3 CCR Authentication has Failed

Problem: The CCR authentication has failed.

Cause: The CCR authentication process uses the CSI selected from the UI and the My Oracle Support preferred ID. If they are not correct, authentication will fail.

Solution: Check if the OCM collector can be configured using these same credentials that is being used in the UI under the My Oracle Support > Preferred Credentials > My Oracle Support Preferred Credentials page.

5.5 Harvester Installation and Configuration Issues

The following common installation and configuration issues are addressed:

- After Upgrade, Duplicate Targets Displayed in My Oracle Support

- OMS and Agent Not Displaying as a Target After Upgrade or New Install

- Install OCM in an Agent Home

5.5.1 After Upgrade, Duplicate Targets Displayed in My Oracle Support

Problem: After an upgrade from Enterprise Manager Grid Control 11g to Enterprise Manager Cloud Control 12c, duplicate targets appear in My Oracle Support.

Cause: This can be a result of not supplying the same My Oracle Support credentials that was used in the 11GC OMS Home when OCM was configured.

Solution: Make sure that in the Enterprise Manager Cloud Control 12c OMS home location, the OCM collector is installed with the same credentials that was used in 11GC OCM location so that no duplicate target are created in MOS.

5.5.2 OMS and Agent Not Displaying as a Target After Upgrade or New Install

Problem: The OMS and Agent do not appear as a target after upgrading or installing a new Enterprise Manager Cloud Control 12c environment.

Cause: New in a Enterprise Manager Cloud Control 12c OCM/Harvester installation, the OMS and Agent targets are now collected via the harvester and not OCM directly.

Solution: This new collection process is a feature of Enterprise Manager Cloud Control 12c.

5.5.3 Install OCM in an Agent Home

Problem: OCM no longer needs to be installed in an Agent home in a Enterprise Manager Cloud Control 12c environment.

Cause: As of Enterprise Manager Cloud Control 12c, the Agent is considered to be a converged target. That means that the harvester now treats both the OCM and 12c metric collections for this target type the same. This convergence is to avoid duplication in configuration metrics and piggy-back on the new harvester functionality.

Solution: This is a feature of Enterprise Manager Cloud Control 12c.

6 Support Hub/Repeater Issues

This chapter describes the issues and problems encountered with running OCM's Support Hub (Repeater) feature. The following issues are addressed:

- Debug Tracing For The Oracle Support Hub

- Configuring the OCM Repeater Associated with Enterprise Manager 12c with a Proxy

6.1 Debug Tracing For The Oracle Support Hub

The OCMREPEATER.PROPERTIES file within the ${ORACLE_HOME}/sysman/config directory will allow you to debug certain connection related issues. This section illustrates the contents of this file and how to change certain parameters for debugging the Oracle Support Hub (also known as the OCM Repeater).

Note:
This property file is not the same as the COLLECTOR.PROPERTIES file found for OCM. One sets properties for the Support Hub, the other for the OCM collector.

To enable debug tracing for the Oracle Support Hub:

1. Edit the ${ORACLE_HOME}/sysman/config/OCMREPEATER.PROPERTIES file. From the listing of available parameter below, adjust:

log4j.rootCategory=WARN, Rolling

to be:

log4j.rootCategory=DEBUG, Rolling

For example:

log4j.appender.Rolling=org.apache.log4j.RollingFileAppender
log4j.appender.Rolling.File=${oracle.home}/sysman/log/ocmrepeater.log
log4j.appender.Rolling.Append=true
log4j.appender.Rolling.MaxFileSize=1000000
log4j.appender.Rolling.MaxBackupIndex=3
log4j.appender.Rolling.layout=org.apache.log4j.PatternLayout
log4j.appender.Rolling.layout.ConversionPattern=%d [%t] %-5p - %m%n

log4j.rootCategory=DEBUG, Rolling

ccr.endpoint=https://ccr.oracle.com

2. Stop and restart the Oracle Support Hub to activate the debug mode. The easiest way to stop and restart the Oracle Support Hub is to execute the following command:

<OCM_REPEATER_HOME>/ocm/repeater/bin/ocmrepeaterctl configure

The ${ORACLE_HOME}/sysman/log/ocmrepeater.log file will now contain additional DEBUG messages.

6.2 Configuring the OCM Repeater Associated with Enterprise Manager 12c with a Proxy

Problem

Configuring an OCM Collector to run via the Oracle Support Hub (also known as the OCM Repeater) associated with Enterprise Manager Cloud Control 12c results in the following message:

OCM Installation Response Generator 10.3.7.0.0 - Production
Copyright (c) 2005, 2012, Oracle and/or its affiliates. All rights reserved.

Unable to establish connection to Oracle Configuration Manager server.
Hostname (https://ccr.oracle.com) is unknown.

Unable to establish a network connection to Oracle. Specify the URL for an
Oracle Support Hub in this format:
http[s]://<OracleSupportHubHost>:<OracleSupportHubPort>
If you do not wish to configure OCM through an Oracle Support Hub, enter NONE
Oracle Support Hub URL: http://OMShost:port
HTTP Failure 901
Received Error:
Oracle Support Hub cannot connect to the OCM Server.

Cause

Enterprise Manager Cloud Control 12c is configured with a proxy server to connect to Oracle, while the repeater is not yet set up to use the proxy server. For example:

export ORACLE_HOME=/u01/em12c/middleware12.1.0.2.0/oms
export OCM_REPEATER_HOME=/u01/em12c/middleware12.1.0.2.0/oms
export OCM_DOMAIN_PATH=/u01/em12c/middleware12.1.0.2.0/gc_inst/user_projects/domains/GCDomain
$ORACLE_HOME/ocm/repeater/bin/ocmrepeaterctl status
Oracle Support Hub - Release: 10.3.6.1.0 - Production
Copyright (c) 2005, 2012, Oracle and/or its affiliates.  All rights reserved.
Built 07/15/2012 07:00:51 PM

Install Root          :/u01/em12c/middleware12.1.0.2.0/oms
Proxy Host            :NONE
Proxy Port            :NONE
Proxy User            :NONE
Logging Level         :WARN, Rolling
Log File Location     :/u01/em12c/middleware12.1.0.2.0/oms/ocm/repeater/log/ocmrepeater.log

Solution

Run the $OCM_REPEATER_HOME/ocm/repeater/bin/ocmrepeaterctl command to configure the Oracle Support Hub for use through a proxy by entering the proxy server details in this format:

[<proxy-user>@]<proxy-host>[:<proxy-port>]

A ORA Error Messages

This appendix lists and describes the ORA error messages associated with the Oracle Configuration Manager.

A.1 ORA-04021 Error

There may be cases when the ORACLE_OCM user needs to be granted the required privileges during installation. While granting the privileges, the following error may occur in the <ocm_install_root>/ccr/log/collectconfig<SID>.log:

ORA-04021: timeout occurred while waiting to lock object SYS.<package like UTL_FILE

This error may occur if another procedure is using the package for which the privileges are being granted. To resolve this error, retry the install when the package is not being used. This error may occur while granting privileges on UTL_FILE, DBMS_SCHEDULER, or DBMS_JOB.

A.2 ORA-01925 Error While Running installCCRSQL

This error may occur if the value of the MAX_ENABLED_ROLES initialization parameter has been exceeded. To resolve this error, you must increase the value of the MAX_ENABLED_ROLES parameter and restart the database as follows:

1. Edit the init<sid>.ora file where <sid> is the database system identifier and increase the value of MAX_ENABLED_ROLES. If a server parameter (spfile) has been used, alter the MAX_ENABLED_ROLES parameter by using the following SQL*PLus command:
SQL>alter system set MAX_ENABLED_ROLES=<value> scope=spfile

2. Restart the database.

Once the database has been restarted, re-run the installCCRSQL.sh script.

Index
C  E  H  I  K  L  O  R  S  U  W

C

collection issues, 3

common errors, 4

CSI Assignment UI Harvester Issues, 5.2

E

Enterprise Manager Cloud Control Harvester Registration Problems, 5.4

Enterprise Manager Cloud Control Harvester Upload Problems, 5.3

H

harvester, 5

Harvester Installation and Configuration Issues, 5.5

I

installCCRSQL.sh, 1.3.2.1

K

known issues, 1.3.3

L

Linux issues, 1.2

O

ORA error messages, A

Oracle Support Hub issues

debug tracing, 6.1

R

Repeater issues, 6

S

Sieble host name returned twice, 3.1

Solaris issues, 1.2

Support Hub issue, 6

U

UNIX issues, 1.2

W

Windows-specific issues, 1.1

Oracle Legal Notices

Copyright Notice

Copyright © 1994-2015, Oracle and/or its affiliates. All rights reserved.

Trademark Notice

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

License Restrictions Warranty/Consequential Damages Disclaimer

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

Warranty Disclaimer

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

Restricted Rights Notice

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.

Hazardous Applications Notice

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Third-Party Content, Products, and Services Disclaimer

This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.

Alpha and Beta Draft Documentation Notice

If this document is in preproduction status:

This documentation is in preproduction status and is intended for demonstration and preliminary use only. It may not be specific to the hardware on which you are using the software. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to this documentation and will not be responsible for any loss, costs, or damages incurred due to the use of this documentation.