This chapter describes the issues and problem resolutions for supported harvesters in Oracle Configuration Manager. The following problem areas are addressed:
Harvester Collection Fails If the state/upload/external Directory Is Missing
Enterprise Manager Cloud Control Harvester Registration Problems
If you are running Enterprise Manager (EM) and the harvester collection fails with the following error in the sysman/log/emoms.log
file:
[JobWorker 75210:Thread-61] ERROR gcharvester.GcCollectionMgr initOcm.? - GC OCM Harvester: Caught GC Harvester exception from GCInit.init(): The installed version of Oracle Configuration Manager in the ORACLE_HOME (/scratch/aime/work/midlwre8937/oms11g) is prior to 10.3.1. The Grid Control Configuration harvesting requires at a minimum, 10.3.1
the required directory named external is missing in either the $ORACLE_HOME/ccr/hosts/<local_host_name>/state/upload
directory (EM release 10.2.0.5) or the ORACLE_INSTANCE_HOME/ccr/state/upload
directory (EM releases 11.1 and later).
To resolve this issue, create the directory $ORACLE_HOME/ccr/hosts/<local_host_name>/state/upload/external
(EM release 10.2.0.5) or ORACLE_INSTANCE_HOME/ccr/state/upload/external
(EM releases 11.1 and later).
The following harvester issues are addressed:
Problem: The Enterprise Manager Cloud Control Harvester has never run.
Cause: Possible causes include:
The harvester job is disabled.
The OCM collector is not installed/configured in the OMS_Home
location.
Solution: See errors in the emoms_pbs.trc
file located in:
<Middleware_Home>/gc_inst/em/EMGC_OMS1/sysman/log
Or, you can run the following command:
egrep -i emoms_pbs.trc |grep harvester
Problem: The harvester has not run in the last {0}
days.
Cause: The following causes are possible:
The harvester job was running successfully for a period of time but now its not running.
The harvester job has been disabled.
The OCM collector is not installed/configured in the OMS_Home
location.
Solution: See errors in the emoms_pbs.trc
file or mgmt_system_error_log
table (In Enterprise Manager Cloud Control, click Setup, then Management Services and Repository, and finally Repository Operations).
If there are no errors being reported in the trace files, then perhaps the harvester needs to be started and scheduled. In that case, you need to start the harvester using the gcharvester_job.sql
script found in the OMS_HOME/sysman/admin/emdrep/sql/core/latest/gcharvester
directory as described below:
declare l_already_registered NUMBER :=0; l_job_targets MGMT_JOB_TARGET_LIST := MGMT_JOB_TARGET_LIST(); l_params MGMT_JOB_PARAM_LIST := MGMT_JOB_PARAM_LIST(); l_schedule MGMT_JOB_SCHEDULE_RECORD; l_job_id_out RAW(16); l_execution_id_out RAW(16); l_version varchar2(32); l_days MGMT_JOB_INT_ARRAY := MGMT_JOB_INT_ARRAY(6); begin -- first check if job is already registered (possibly in earlier version of EM) BEGIN SELECT 1 into l_already_registered FROM MGMT_JOB WHERE job_name='GCHARVESTERJOB' AND job_type='GCHarvesterRun'; EXCEPTION WHEN NO_DATA_FOUND THEN null; END; IF (l_already_registered = 1) THEN MGMT_JOBS.stop_all_executions(p_job_name => 'GCHARVESTERJOB'); COMMIT ; END IF; -- start a daily job, running at 2 am l_schedule := MGMT_JOBS.get_job_schedule_record(p_frequency_code => MGMT_JOBS.DAILY_FREQUENCY_CODE, p_start_time => SYSDATE + 1, p_end_time => null, p_execution_hours => 2, p_execution_minutes => 0, p_interval => 0, p_months => null, p_days => null, p_timezone_info => MGMT_JOBS.TIMEZONE_REPOSITORY, p_timezone_target_index => 0, p_timezone_offset => 0, p_timezone_region => null, p_start_grace_period => 240); MGMT_JOBS.submit_job(p_job_name => 'GCHARVESTERJOB', p_description => 'GC Harvester Job', p_job_type => 'GCHarvesterRun', p_job_targets => l_job_targets, p_job_params => l_params, p_schedule => l_schedule, p_job_id_out => l_job_id_out, p_execution_id_out => l_execution_id_out, p_owner => null, p_system_job => MGMT_JOB_ENGINE.SYSTEM_JOB, p_job_creds => null, p_job_target_type => null, p_job_notify_states => null, p_running_time_limit => -1); update MGMT_JOB set job_owner= MGMT_USER.GET_REPOSITORY_OWNER where job_name='GCHARVESTERJOB' AND job_type='GCHarvesterRun'; MGMT_LOG.register_logging_module('GCHARVESTER','GC Harvester', NULL); commit; end; /
The two main arguments are hour to run (execution_hours) and minute to run (execution_minutes).
As long as the time entered is not in the past, the script will run fine allowing the harvester to function properly.
The following harvester upload problems are addressed:
Problem: Collection files are not being deleted after an upload.
Background: In Enterprise Manager Cloud Control 12c, the harvester uploads its data directly to the CCR repository. As part of this process flow, the collection files are deleted as soon as they are uploaded. However, in some cases where a collection is failing or there is a concern about targets and their timestamps, Oracle Support may have you set the delete_upload_file
parameter to retain those collection files so they can be reviewed.
Solution: With a connection to the repository as the SYSMAN user, the delete_upload_file
property in the mgmt_ocm_upl_props
table needs to be set to False as follows:
SQL> insert into mgmt_ocm_upl_props(name, str_value) values('delete_upload_file','false'); SQL> commit;
Note:
At this point, the OMS needs to be restarted in order for that parameter to take effect.By default, the files will be created in the $INSTANCE_HOME/sysman/ocm/harvester/tmp
directory. The files themselves will be incremental in nature.
emoms_pbs.trc
FileProblem: In some cases, the following error may be noticed in the emoms_pbs.trc
file:
2012-03-29 02:00:04,261 [JobWorker208183:JobThreadPool:Long-System:Thread-641692] ERROR gcharvester.HarvesterJobUtils performHistoryBasedUpdates.? - GC OCM Harvester: Failed to populate OCM metrics based on ECM history: ORA-01878: specified field not found in datetime or interval ORA-06512: at line 358
Cause: This error is a clear indication that you are encountering a problem documented in Bug 13972728.
Workaround:
Locate the OMS instance home.
In the $ORACLE_HOME/sysman/config/emInstanceMapping.properties
file (where ORACLE_HOME
is the Oracle Home of the OMS), there is an entry referencing a file called emgc.properties
.
The directory in which the emgc.properties
file is located is the "instance home" of the OMS. In the following example, /u01/app/oracle/product/gc_inst/em/EMGC_OMS1
is the instance home of the OMS:
EMGC_OMS1=/u01/app/oracle/product/gc_inst/em/EMGC_OMS1/emgc.properties
Set the environment variable ORACLE_CONFIG_HOME
to the directory of this emgc.properties
file. For example:
$export ORACLE_CONFIG_HOME=/u01/app/oracle/product/gc_inst/em/EMGC_OMS1
Run the emCCR register
command:
cd $ORACLE_HOME/ccr/bin emCCR register
The next scheduled time the harvester job runs, it will trigger a full collection. In a multiple-OMS environment, this emCCR register
command will need to be completed in all locations.
Problem: In an E-Business Suite (EBS) environment, the harvester is not 'EBS' aware.
Cause: EBS is not currently a harvested target from Enterprise Manager Cloud Control.
Workaround: The OCM standalone product will need to be used in order to upload the data to My Oracle Support.
Problem: During the scheduled harvester run (default is 2:00 a.m.), the job fails and the following error is dumped into the emoms.trc
file:
2012-10-20 02:00:30,781 [JobWorker 73854412:Thread-86] ERROR gcharvester.HarvesterJobUtils prepareTargetsForShipment.? - GC OCM Harvester: Failed to stage targets for shipment to Oracle: ORA-01031: insufficient privileges ORA-06512: at line 9
At this point, we are trying to create a sequence object and failing.
Cause: The SYAMAN user lacks the privilege to create the sequence. This sequence is used to put all harvested targets in mgmt_ocm_upl_metrics_tmp
and then process them in batches of 512 targets at a time. First 1 to 512 will be processed, then 513 to 1024 and so on.
Solution: Assign the SYSMAN user the CREATE SEQUENCE privilege:
GRANT CREATE SEQUENCE, DROP ANY SEQUENCE TO SYSMAN;
At first run, the harvester performs an Enterprise Manager Cloud Control Default registration using the registration information of the OCM collector in the OMS location. As part of this registration, the EM Repository URL is also sent for bookkeeping purposes on the CCR side of things. All other Oracle_Homes
, which are considered to be a proxy uploaded by the harvester, are uploaded using the Default registration (this is similar to 11GC).
When a Target is added, this Target Oracle Home registration is performed after a user goes to UI CSI assignment page and assigns a CSI to it.
The registration process itself has a two-step approach:
Authenticate: Done as part of the UI which authenticates with the CCR database.
Register: The harvester internal job picks up on any Pending targets and registers itself against the Oracle CCR database.
Note:
If there are multiple OMS locations, each harvester needs to be registered with the same My Oracle Support Single Sign On (SSO) credential.In Enterprise Manager Cloud Control 12c, the harvester now uploads information to CCR directly and no longer relies on OCM to perform this task. It also deletes the files as soon as they are uploaded.
The following problems with harvester registration are addressed:
Problem: While assigning a Customer Support Identifier (CSI), the following error is returned in a pop-up window: You don't have the privilege to do this operation.
Cause: The user does not have the Operator privilege on one or more targets within a given Oracle_Home
for that target
Solution:
Problem: The CSI drop-down listing does not show any CSI information.
Cause: There is a chance that the My Oracle Support connectivity from the OMS location is not successful.
Solution: Review the emoms.trc
file for any possible communication issue.
Problem: The CCR authentication has failed.
Cause: The CCR authentication process uses the CSI selected from the UI and the My Oracle Support preferred ID. If they are not correct, authentication will fail.
Solution: Check if the OCM collector can be configured using these same credentials that is being used in the UI under the My Oracle Support > Preferred Credentials > My Oracle Support Preferred Credentials page.
The following common installation and configuration issues are addressed:
After Upgrade, Duplicate Targets Displayed in My Oracle Support
OMS and Agent Not Displaying as a Target After Upgrade or New Install
Problem: After an upgrade from Enterprise Manager Grid Control 11g to Enterprise Manager Cloud Control 12c, duplicate targets appear in My Oracle Support.
Cause: This can be a result of not supplying the same My Oracle Support credentials that was used in the 11GC OMS Home when OCM was configured.
Solution: Make sure that in the Enterprise Manager Cloud Control 12c OMS home location, the OCM collector is installed with the same credentials that was used in 11GC OCM location so that no duplicate target are created in MOS.
Problem: The OMS and Agent do not appear as a target after upgrading or installing a new Enterprise Manager Cloud Control 12c environment.
Cause: New in a Enterprise Manager Cloud Control 12c OCM/Harvester installation, the OMS and Agent targets are now collected via the harvester and not OCM directly.
Solution: This new collection process is a feature of Enterprise Manager Cloud Control 12c.
Problem: OCM no longer needs to be installed in an Agent home in a Enterprise Manager Cloud Control 12c environment.
Cause: As of Enterprise Manager Cloud Control 12c, the Agent is considered to be a converged target. That means that the harvester now treats both the OCM and 12c metric collections for this target type the same. This convergence is to avoid duplication in configuration metrics and piggy-back on the new harvester functionality.
Solution: This is a feature of Enterprise Manager Cloud Control 12c.