Database Status is Shown as Down when the Database is Up

If a database entity availability status is shown as down in Oracle Management Cloud, but the database is actually up, you can perform the following troubleshooting procedures.

  1. Go to the VM where the lama (Cloud) agent is located.
  2. cd $AGENT_HOME ( ps -ef| grep lama)
  3. Check current status of the entities:
    ./omcli status_entity agent
  4. vi $AGENT_HOME/sysman/emd/targets.xml

    Search for the entity name and check capability property is set to monitoring.

  5. Check the metric schedule:
    ./omcli status agent scheduler | grep Response

    The Response metric should come against the impacted entity.

  6. ./omcli getmetric agent <entity name>,omc_oracle_db,Response
  7. Check if the DB is up and check if it's able to connect using SqlPlus:

    If the DB is installed in the same agent host, do the following:

    cd $DB_HOME 
    export ORACLE_HOME=/u01/app/oracle/product/19.0.0.0/dbhome_1
    export ORACLE_SID=DB0221
    ./sqlplus "<user>/<pass>@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(Host=<host>)(Port=<port>))(CONNECT_DATA=(SID=<sid>)))"
    ./sqlplus "<user>/<pass>@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(Host=<host>)(Port=<port>))(CONNECT_DATA=(SERVICE_NAME=<service_name>)))"

    Provide the values <> with exactly the same values used while registering the DB entity.

    If the DB is NOT installed in the same agent host, check if there is any DB home available in the agent home and use the DB home's SqlPlus to connect using the above steps and check the connectivity between agent and the DB.

  8. Set DEBUG mode:
    ./omcli setproperty agent -allow_new -name log4j.rootCategory -value "DEBUG, emlogAppender, emtrcAppender" -type emomslogging
  9. First, create entity.json and then Refresh the entity:
    ./omcli refresh_entity agent <entity.json>
    cat entity.json
    {
        "entities": [
            {
                "name": "<entity name>",
                "type": "<entity type>"
            }
        ]
    }

    Refresh entity command can throw the error messages while connecting to the DB. This will show why the DB is shown down .

If PDB entities are shown down, but are really up:

  1. PDBs have to be in READ WRITE mode in order to be shown as UP in monitoring. You can determine the PDB status from select * from gv$containers or v$pdbs.
  2. ./omcli getmetric agent <CDB entity name>,omc_oracle_db,PDB_Status
  3. Check targets.xml to determine whether the associations are present in both the DB entity and the PDB entity as shown below:

    DB

    <AssocTargetInstance ASSOCIATION_NAME="omc_contains" SOURCE_TARGET_NAME="<DB entity>" SOURCE_TARGET_TYPE="omc_oracle_db" ASSOC_TARGET_NAME="<PDB entity>" ASSOC_TARGET_TYPE="omc_oracle_pdb"/>

    PDB

    <AssocTargetInstance ASSOCIATION_NAME="omc_monitored_by" SOURCE_TARGET_NAME="<PDB entity>" SOURCE_TARGET_TYPE="omc_oracle_pdb" ASSOC_TARGET_NAME="<DB entity>" ASSOC_TARGET_TYPE="omc_oracle_db"/>