5 Harvester Issues

This chapter describes the issues and problem resolutions for supported harvesters in Oracle Configuration Manager. The following problem areas are addressed:

5.1 Harvester Collection Fails If the state/upload/external Directory Is Missing

If you are running Enterprise Manager (EM) and the harvester collection fails with the following error in the sysman/log/emoms.log file:

[JobWorker 75210:Thread-61] ERROR gcharvester.GcCollectionMgr initOcm.? - GC OCM
Harvester: Caught GC Harvester exception from GCInit.init(): The installed
version of Oracle Configuration Manager in the ORACLE_HOME
(/scratch/aime/work/midlwre8937/oms11g) is prior to 10.3.1. The Grid Control
Configuration harvesting requires at a minimum, 10.3.1

the required directory named external is missing in either the $ORACLE_HOME/ccr/hosts/<local_host_name>/state/upload directory (EM release 10.2.0.5) or the ORACLE_INSTANCE_HOME/ccr/state/upload directory (EM releases 11.1 and later).

To resolve this issue, create the directory $ORACLE_HOME/ccr/hosts/<local_host_name>/state/upload/external (EM release 10.2.0.5) or ORACLE_INSTANCE_HOME/ccr/state/upload/external (EM releases 11.1 and later).

5.2 CSI Assignment UI Harvester Issues

The following harvester issues are addressed:

5.2.1 Enterprise Manager Cloud Control Harvester has Never Run

Problem: The Enterprise Manager Cloud Control Harvester has never run.

Cause: Possible causes include:

  • The harvester job is disabled.

  • The OCM collector is not installed/configured in the OMS_Home location.

Solution: See errors in the emoms_pbs.trc file located in:

<Middleware_Home>/gc_inst/em/EMGC_OMS1/sysman/log

Or, you can run the following command:

egrep -i emoms_pbs.trc |grep harvester

5.2.2 Enterprise Manager Cloud Control Harvester has not Run Recently

Problem: The harvester has not run in the last {0} days.

Cause: The following causes are possible:

  • The harvester job was running successfully for a period of time but now its not running.

  • The harvester job has been disabled.

  • The OCM collector is not installed/configured in the OMS_Home location.

Solution: See errors in the emoms_pbs.trc file or mgmt_system_error_log table (In Enterprise Manager Cloud Control, click Setup, then Management Services and Repository, and finally Repository Operations).

If there are no errors being reported in the trace files, then perhaps the harvester needs to be started and scheduled. In that case, you need to start the harvester using the gcharvester_job.sql script found in the OMS_HOME/sysman/admin/emdrep/sql/core/latest/gcharvester directory as described below:

declare
    l_already_registered NUMBER :=0;
    l_job_targets MGMT_JOB_TARGET_LIST := MGMT_JOB_TARGET_LIST();
    l_params MGMT_JOB_PARAM_LIST := MGMT_JOB_PARAM_LIST();
    l_schedule MGMT_JOB_SCHEDULE_RECORD;
    l_job_id_out RAW(16);
    l_execution_id_out RAW(16);
    l_version varchar2(32);
    l_days MGMT_JOB_INT_ARRAY := MGMT_JOB_INT_ARRAY(6);
 
begin
    -- first check if job is already registered (possibly in earlier version of EM)
    BEGIN
        SELECT 1 into l_already_registered
        FROM MGMT_JOB
        WHERE job_name='GCHARVESTERJOB' AND job_type='GCHarvesterRun';
 
        EXCEPTION
            WHEN NO_DATA_FOUND THEN
            null;
    END;
    
    IF (l_already_registered = 1) THEN
      MGMT_JOBS.stop_all_executions(p_job_name => 'GCHARVESTERJOB');
      COMMIT  ;
    END IF;
 
-- start a daily job, running at 2 am
l_schedule :=
MGMT_JOBS.get_job_schedule_record(p_frequency_code => MGMT_JOBS.DAILY_FREQUENCY_CODE,
p_start_time => SYSDATE + 1,
p_end_time => null,
p_execution_hours => 2,
p_execution_minutes => 0,
p_interval => 0,
p_months => null,
p_days => null,
p_timezone_info => MGMT_JOBS.TIMEZONE_REPOSITORY,
p_timezone_target_index => 0,
p_timezone_offset => 0,
p_timezone_region => null,
p_start_grace_period => 240);

MGMT_JOBS.submit_job(p_job_name => 'GCHARVESTERJOB',
p_description => 'GC Harvester Job',
p_job_type => 'GCHarvesterRun',
p_job_targets => l_job_targets,
p_job_params => l_params,
p_schedule => l_schedule,
p_job_id_out => l_job_id_out,
p_execution_id_out => l_execution_id_out,
p_owner => null,
p_system_job => MGMT_JOB_ENGINE.SYSTEM_JOB,
p_job_creds => null,
p_job_target_type => null,
p_job_notify_states => null,
p_running_time_limit => -1);

update MGMT_JOB set job_owner= MGMT_USER.GET_REPOSITORY_OWNER where job_name='GCHARVESTERJOB' AND job_type='GCHarvesterRun';
MGMT_LOG.register_logging_module('GCHARVESTER','GC Harvester', NULL);

commit;
end;
/

The two main arguments are hour to run (execution_hours) and minute to run (execution_minutes).

As long as the time entered is not in the past, the script will run fine allowing the harvester to function properly.

5.3 Enterprise Manager Cloud Control Harvester Upload Problems

The following harvester upload problems are addressed:

5.3.1 Collection Files are not Deleted After Upload

Problem: Collection files are not being deleted after an upload.

Background: In Enterprise Manager Cloud Control 12c, the harvester uploads its data directly to the CCR repository. As part of this process flow, the collection files are deleted as soon as they are uploaded. However, in some cases where a collection is failing or there is a concern about targets and their timestamps, Oracle Support may have you set the delete_upload_file parameter to retain those collection files so they can be reviewed.

Solution: With a connection to the repository as the SYSMAN user, the delete_upload_file property in the mgmt_ocm_upl_props table needs to be set to False as follows:

SQL> insert into mgmt_ocm_upl_props(name, str_value) values('delete_upload_file','false');
SQL> commit;

Note:

At this point, the OMS needs to be restarted in order for that parameter to take effect.

By default, the files will be created in the $INSTANCE_HOME/sysman/ocm/harvester/tmp directory. The files themselves will be incremental in nature.

5.3.2 Errors in the emoms_pbs.trc File

Problem: In some cases, the following error may be noticed in the emoms_pbs.trc file:

2012-03-29 02:00:04,261 [JobWorker208183:JobThreadPool:Long-System:Thread-641692] ERROR
gcharvester.HarvesterJobUtils performHistoryBasedUpdates.? - GC OCM Harvester: Failed to populate OCM metrics based on ECM history: ORA-01878:
specified field not found in datetime or interval ORA-06512: at line 358

Cause: This error is a clear indication that you are encountering a problem documented in Bug 13972728.

Workaround:

  1. Locate the OMS instance home.

  2. In the $ORACLE_HOME/sysman/config/emInstanceMapping.properties file (where ORACLE_HOME is the Oracle Home of the OMS), there is an entry referencing a file called emgc.properties.

  3. The directory in which the emgc.properties file is located is the "instance home" of the OMS. In the following example, /u01/app/oracle/product/gc_inst/em/EMGC_OMS1 is the instance home of the OMS:

    EMGC_OMS1=/u01/app/oracle/product/gc_inst/em/EMGC_OMS1/emgc.properties

  4. Set the environment variable ORACLE_CONFIG_HOME to the directory of this emgc.properties file. For example:

    $export ORACLE_CONFIG_HOME=/u01/app/oracle/product/gc_inst/em/EMGC_OMS1

  5. Run the emCCR register command:

    cd $ORACLE_HOME/ccr/bin
emCCR register

The next scheduled time the harvester job runs, it will trigger a full collection. In a multiple-OMS environment, this emCCR register command will need to be completed in all locations.

5.3.3 Harvester is not E-Business Suite (EBS) Aware

Problem: In an E-Business Suite (EBS) environment, the harvester is not 'EBS' aware.

Cause: EBS is not currently a harvested target from Enterprise Manager Cloud Control.

Workaround: The OCM standalone product will need to be used in order to upload the data to My Oracle Support.

5.3.4 Harvester Fails during Scheduled Run

Problem: During the scheduled harvester run (default is 2:00 a.m.), the job fails and the following error is dumped into the emoms.trc file:

2012-10-20 02:00:30,781 [JobWorker 73854412:Thread-86] ERROR
gcharvester.HarvesterJobUtils prepareTargetsForShipment.? - GC OCM Harvester: Failed to stage targets for shipment to Oracle: ORA-01031: insufficient privileges ORA-06512: at line 9

At this point, we are trying to create a sequence object and failing.

Cause: The SYAMAN user lacks the privilege to create the sequence. This sequence is used to put all harvested targets in mgmt_ocm_upl_metrics_tmp and then process them in batches of 512 targets at a time. First 1 to 512 will be processed, then 513 to 1024 and so on.

Solution: Assign the SYSMAN user the CREATE SEQUENCE privilege:

GRANT CREATE SEQUENCE, DROP ANY SEQUENCE TO SYSMAN;

5.4 Enterprise Manager Cloud Control Harvester Registration Problems

At first run, the harvester performs an Enterprise Manager Cloud Control Default registration using the registration information of the OCM collector in the OMS location. As part of this registration, the EM Repository URL is also sent for bookkeeping purposes on the CCR side of things. All other Oracle_Homes, which are considered to be a proxy uploaded by the harvester, are uploaded using the Default registration (this is similar to 11GC).

When a Target is added, this Target Oracle Home registration is performed after a user goes to UI CSI assignment page and assigns a CSI to it.

The registration process itself has a two-step approach:

  1. Authenticate: Done as part of the UI which authenticates with the CCR database.

  2. Register: The harvester internal job picks up on any Pending targets and registers itself against the Oracle CCR database.

Note:

If there are multiple OMS locations, each harvester needs to be registered with the same My Oracle Support Single Sign On (SSO) credential.

In Enterprise Manager Cloud Control 12c, the harvester now uploads information to CCR directly and no longer relies on OCM to perform this task. It also deletes the files as soon as they are uploaded.

Figure 5-1 Registration Types

Surrounding text describes Figure 5-1 .

The following problems with harvester registration are addressed:

5.4.1 Errors When Assigning a Customer Support Identifier (CSI)

Problem: While assigning a Customer Support Identifier (CSI), the following error is returned in a pop-up window: You don't have the privilege to do this operation.

Cause: The user does not have the Operator privilege on one or more targets within a given Oracle_Home for that target

Solution:

5.4.2 CSI Information not Displayed

Problem: The CSI drop-down listing does not show any CSI information.

Cause: There is a chance that the My Oracle Support connectivity from the OMS location is not successful.

Solution: Review the emoms.trc file for any possible communication issue.

5.4.3 CCR Authentication has Failed

Problem: The CCR authentication has failed.

Cause: The CCR authentication process uses the CSI selected from the UI and the My Oracle Support preferred ID. If they are not correct, authentication will fail.

Solution: Check if the OCM collector can be configured using these same credentials that is being used in the UI under the My Oracle Support > Preferred Credentials > My Oracle Support Preferred Credentials page.

5.5 Harvester Installation and Configuration Issues

The following common installation and configuration issues are addressed:

5.5.1 After Upgrade, Duplicate Targets Displayed in My Oracle Support

Problem: After an upgrade from Enterprise Manager Grid Control 11g to Enterprise Manager Cloud Control 12c, duplicate targets appear in My Oracle Support.

Cause: This can be a result of not supplying the same My Oracle Support credentials that was used in the 11GC OMS Home when OCM was configured.

Solution: Make sure that in the Enterprise Manager Cloud Control 12c OMS home location, the OCM collector is installed with the same credentials that was used in 11GC OCM location so that no duplicate target are created in MOS.

5.5.2 OMS and Agent Not Displaying as a Target After Upgrade or New Install

Problem: The OMS and Agent do not appear as a target after upgrading or installing a new Enterprise Manager Cloud Control 12c environment.

Cause: New in a Enterprise Manager Cloud Control 12c OCM/Harvester installation, the OMS and Agent targets are now collected via the harvester and not OCM directly.

Solution: This new collection process is a feature of Enterprise Manager Cloud Control 12c.

5.5.3 Install OCM in an Agent Home

Problem: OCM no longer needs to be installed in an Agent home in a Enterprise Manager Cloud Control 12c environment.

Cause: As of Enterprise Manager Cloud Control 12c, the Agent is considered to be a converged target. That means that the harvester now treats both the OCM and 12c metric collections for this target type the same. This convergence is to avoid duplication in configuration metrics and piggy-back on the new harvester functionality.

Solution: This is a feature of Enterprise Manager Cloud Control 12c.