9 Receivelets

Receivelet is a library that allows Enterprise Manager to receive external notifications sent by third-party network elements. These are notifications that are asynchronously sent and without any requests from Oracle Management Agent.

Usually, Oracle Management Agent data retrieval mechanism is based on a polling model, that is, modular libraries, called Fetchlets. Fetchlets collect values of various metrics from their managed targets on a regular basis. Oracle Management Agent then compares the gathered data with user-defined thresholds and generates events when the thresholds were met.

Receivelets are a more efficient way of dealing with these metrics. It depends on the ability of the managed target to detect the condition for its own events, and then communicate with Enterprise Manager only when an event occurs. When this communication happens, Oracle Management Agent uses receivelets to receive the information.

You can use Receivelets as a quicker way to get alerts on data that will be eventually collected via Fetchlets. You can also use Receivelets as a way to send both alerts and data, or just alerts for cases where there is no real data chart associated with the alert.

Receivelet is not a substitute for Fetchlet, but it is another way of collecting data. It is more for immediacy of notification compared to periodic polling that Fetchlet offers. Therefore, if you can fetch data, then use Fetchlets to get that data. However, if your server is capable of sending you events or data at a cost lower than that associated with Fetchlets, then use Receivelets.

A receivelet may be tightly coupled to a particular type of managed target, or may be useful to a broad range of potential targets.

The following receivelets are offered with Enterprise Manager:

9.1 SNMP Receivelets

SNMP Receivelets allow you to receive SNMP Traps notifications from third-party network elements, and translate them into a form compatible with Oracle Management Service.

While monitoring third-party entities in your managed environment, if the status of a third-party network element turns unavailable or if its metric severity conditions (metric thresholds) are met or exceeded, the SNMP Agent of that third-party network element sends a notification to Oracle Management Agent. These notifications may be in the form of SNMP Traps that get triggered asynchronously and without any requests from Oracle Management Agent.

Since these traps are based on SNMP, Oracle Management Agent uses SNMP Receivelets to receive and translate these SNMP Traps into a form compatible with Oracle Management Service.

Once the SNMP Traps are received, the SNMP Receivelet extracts information pertaining to only those object identifiers (OIDs) that are defined in the push descriptor section of the metadata.xml configuration file (that is, only for those third-party network elements that need to be monitored by Enterprise Manager). For information about the locations of configuration file and Document Type Declaration (DTD) files, see Chapter 2: 2 Developing a Management Plug-In of this guide.

Whenever an SNMP Trap is sent by the SNMP agent, the SNMP Receivelet receives those traps based on the SNMP Agent configuration, translates them to an Enterprise Manager understandable format (like event or datapoint based on the push descriptor information), and stores that information (in XML files) in the upload directory. The Upload Manager checks for such new files in the upload directory, and then uploads those files onto Oracle Management Service. Enterprise Manager, then, accesses the Oracle Management Service to extract the collected information and display it to the user.

Configuration Required for Receiving SNMP Traps

To receive SNMP traps, you have to make some configuration settings at Oracle Management Agent side and at SNMP target agent side.

This will enable the SNMP targets to send SNMP traps to Oracle Management Agent's SNMP Receivelet. Once the SNMP traps is received, the SNMP Receivelet uses the Push Descriptor properties, such as MatchAgentAddr, MatchEnterprise, and so on, to identify the target and metric for which the traps belongs. Then the SNMP Receivelet uses the Push Descriptor properties, such as Event<metric-column> or Event<metric-column>OID, SeverityCode, and so on, to convert the traps into an event. Once this happens, the SNMP Receivelet uploads the converted event to the Management Repository and in turn is displayed in the console.

Configuration Settings Required at Oracle Management Agent Side

  1. Define SNMP target in the $ORACLE_HOME/sysman/emd/targets.xml file.

  2. Identify the metrics of the target that you want to monitor using the SNMP Receivelet (SNMP Traps).

  3. Define the identified metrics in the target type metadata file ($ORACLE_HOME/sysman/admin/metadata/<target-type.xml) with the SNMP Push Descriptor as shown in the examples.

  4. Identify the columns of the metric for which you want to generate EM events from the SNMP Target's trap.

  5. Define the identified event columns as one of the Push Descriptor properties. For example, Event<metric-column> or Event<metric-column>OID.

  6. After making these configuration settings, the emctl reload agent command reloads the modified configuration to Oracle Management Agent and SNMP Receivelet starts to listen on a UDP port.

    By default, the SNMP Receivelet listens over UDP on the same port as that of Oracle Management Agent. However, if you want to use a different listening port for the SNMP Receivelet, then add the SnmpRecvletListenNIC(=8002) in the emd.properties file.

Configuration Settings Required at SNMP Target Agent Side

  1. Add or modify entries in the 'manager' table of the SNMP target agent for manager IP/host name and manager port.

    Manager host name is the name of the computer where Oracle Management Agent is running. Manager port is the port on which SNMP Receivelet receives SNMP traps.

Input Parameters

Table 9-1 SNMP Receivelet Input Parameters

Parameter Type Description Use

MatchEnterprise

String

OID used to define the trap being sent.

Required

MatchGenericTrap

String

Code for a generic SNMP trap.

Required

MatchSpecificTrap

String

Trap defined in a MIB (not one of the generic traps), the ID assigned in that MIB.

Required

MatchAgentAddr

String

IP address of the generating SNMP agent, as sent in the trap.

Required

Event<metric-column>

String

Specifies that, on receiving this trap, the recvlet should generate a severity on this metric column. The value of the metric-column should be the value of this parameter. (This case is useful where the expected values of the EM metric are not the same as the triggering SNMP variable.)

Required, if events have to be generated. However, if Event<metric-column>OID is provided, then this is not required.

Event<metric-column>OID

String

Specifies that, on receiving this trap, the recvlet should generate a severity on this metric column. The value of the metric column should be taken from the varbind in the trap with OID equal to the value of this parameter.

Required, if events have to be generated. However, if Event<metric-column> is provided, then this is not required.

SeverityCode

String

Specifies the level at which the severity should be generated. The value of this parameter should be one of 'CRITICAL', 'WARNING', or 'CLEAR'.

Required. However, if SeverityCodeOID is provided, then this is not required.

SeverityCodeOID

String

Specifies the level at which the severity should be generated. If the varbind in the trap with OID equal to the value of this parameter is one of the strings 'CRITICAL', 'WARNING', or 'CLEAR', the severity should be generated at that level; otherwise, no severity should be generated. (This parameter would only be used if the integrator were designing a trap exclusively for use with EM, but may be useful in this case.)

Required. However, if SeverityCode is provided, then this is not required.

Data<metric-column>OID

String

Specifies that, on receiving this trap, the recvlet should generate a datapoint on the metric, for which the value of this metric column should be taken from the varbind in the trap with OID equal to the value of this parameter. (An SNMP Push Descriptor may have many Data* parameters, in which case a single row will be returned, with all specified columns populated from the appropriate varbind in the trap. An SNMP PushDescriptor may not have both a Data* parameter and a Severity* one, nor may it have multiple Severity* parameters.)

Required, if datapoints have to be generated.

Key<metric-column>OID

String

Severity or datapoint generated by this PushDescriptor should contain a key-value for this metric column. The key-value should be taken from the varbind in the trap with OID equal to the value of this parameter. For every key-column in the metric definition, there must be a Key* parameter in the PushDescriptor.

Optional

Context<metric-column>OID

String

If the PushDescriptor generates a severity, the severity should contain a value for this metric column in its event context. The value should be take from the varbind in the trap with OID equal to the value of this parameter. If the PushDescriptor generates a datapoint, this parameter is ignored.

Optional. This can be used only for datapoints.


Example

Example 9-1 shows how a trap from a vendor-specific router looks like.

Example 9-1 Trap from a Vendor-Specific Router

ascendLinkDown    TRAP-TYPE 
   ENTERPRISE     ascend 
   VARIABLES     { ifIndex, ifAdminStatus, ifOperStatus, ifType, 
                   ifName } 
   DESCRIPTION    "This trap is in addition to the generic linkDown 
            trap defined in RFC1215.  This trap provides 
            additional information such as ifAdminStatus, 
            ifOerStatus, ifName, slotIfSlotIndex, slotIfItemIndex. 
            This is an Alarm class trap and it can 
            be enabled/disabled via alarmEnabled and/or 
            ascendLinkDownTrapEnabled in trap profile. 
            This trap is sent only if rfc1215 linkDown trap is generated." 
   ::= 50 

Example 9-2 show how the trap will be received by Oracle Management Agent. Note that <x> in this example is the value of ifIndex that identifies the particular interface that's having problems.

Example 9-2 Trap Received by Oracle Management Agent

Message: 
    version: 0 
    community: 'public' 
    Trap-PDU: 
        enterprise: enterprises.ascend (1.3.6.1.4.1.529) 
        agent-addr: 138.2.204.10 
        generic-trap: 6 
        specific-trap: 50 
        time-stamp: <timestamp from router's sysUptime> 
        variable-bindings: 
            Name: ifIndex.<x> (1.3.6.1.2.1.2.2.1.1.<x>) 
            Type: INTEGER 
            Value: <x> 

            Name: ifAdminStatus.<x> (1.3.6.1.2.1.2.2.1.7.<x>) 
            Type: INTEGER 
            Value: up (1) 

            Name: ifOperStatus.<x> (1.3.6.1.2.1.2.2.1.8.<x>) 
            Type: INTEGER 
            Value: down (2) 

            Name: ifType.<x> (1.3.6.1.2.1.2.2.1.3.<x>) 
            Type: INTEGER 
            Value: iso88023-csmacd (7) 

            Name: ifName.<x> (1.3.6.1.2.1.2.2.1.31.<x>) 
            Type: DisplayString 
            Value: 'eth0' 

Example 9-3 shows how the metric can be defined in the metadata.xml file.

Example 9-3 Metric Defined in the metadata.xml File

<Metric NAME="interfaces" TYPE="TABLE"> 
    <TableDescriptor> 
        <ColumnDescriptor NAME="name" TYPE="STRING" IS_KEY="TRUE"/> 
        <ColumnDescriptor NAME="type" TYPE="NUMBER" IS_KEY="FALSE"/> 
        <ColumnDescriptor NAME="status" TYPE="NUMBER" IS_KEY="FALSE"/> 
        <ColumnDescriptor NAME="configured_status" TYPE="NUMBER" IS_KEY="FALSE"/> 
    </TableDescriptor> 
</Metric> 

Example 9-4 shows how the push descriptor can be defined in the metadata.xml file to trigger a severity.

Example 9-4 Push Descriptor in the metadata.xml File For Triggering a Severity

<PushDescriptor RECVLET_ID="SNMPTrap"> 
   <Property NAME="MatchEnterprise" SCOPE="GLOBAL">1.3.6.1.4.1.529</Property> 
   <Property NAME="MatchGenericTrap" SCOPE="GLOBAL">6</Property> 
   <Property NAME="MatchSpecificTrap" SCOPE="GLOBAL">50</Property> 
   <Property NAME="MatchAgentAddr" SCOPE="INSTANCE">AdminAddress</Property> 
   <Property NAME="SeverityStatusOID" SCOPE="GLOBAL">1.3.6.1.2.1.2.2.1.8</Property> 
   <Property NAME="KeyNameOID" SCOPE="GLOBAL">1.3.6.1.2.1.2.2.1.31</Property> 
   <Property NAME="ContextTypeOID" SCOPE="GLOBAL">1.3.6.1.2.1.2.2.1.3</Property> 
   <Property NAME="ContextConfigured_statusOID" SCOPE="GLOBAL">1.3.6.1.2.1.2.2.1.7</Property> 
   <Property NAME="SeverityCode" SCOPE="GLOBAL">CRITICAL</Property> 
</PushDescriptor>

Example 9-5 show how the push descriptor can be defined in the metadata.xml file to trigger a datapoint, which would specify the reporting of data on the same trap, with ifName as the key-column and the other three as data columns.

Example 9-5 Push Descriptor in the metadata.xml File For Triggering a Datapoint

<PushDescriptor RECVLET_ID="SNMPTrap"> 
   <Property NAME="MatchEnterprise" SCOPE="GLOBAL">1.3.6.1.4.1.529</Property> 
   <Property NAME="MatchGenericTrap" SCOPE="GLOBAL">6</Property> 
   <Property NAME="MatchSpecificTrap" SCOPE="GLOBAL">50</Property> 
   <Property NAME="MatchAgentAddr" SCOPE="INSTANCE">AdminAddress</Property> 
   <Property NAME="KeyNameOID" SCOPE="GLOBAL">1.3.6.1.2.1.2.2.1.31</Property> 
   <Property NAME="DataStatusOID" SCOPE="GLOBAL">1.3.6.1.2.1.2.2.1.8</Property> 
   <Property NAME="DataTypeOID" SCOPE="GLOBAL">1.3.6.1.2.1.2.2.1.3</Property> 
   <Property NAME="DataConfigured_statusOID" SCOPE="GLOBAL">1.3.6.1.2.1.2.2.1.7</Property> 
</PushDescriptor> 

Error Handling

There are no exceptions thrown from SNMP Receivelets.

Notes

  • The metadata.xml file can have multiple push descriptor definitions.

  • For event format, ensure that your push descriptor defines only one metric column. Similarly, for other metric columns, you can create separate, distinct push descriptor definitions.

  • For datapoint format, ensure that your push descriptor defines only one metric column. However, you can use the Context<column_name>OID parameter to capture details of other metric columns, in the same push descriptor definition.

9.2 Advanced Queue Receivelets

Advanced Queue Receivelets allow you to receive notifications from Oracle Databases, and translate them into a form compatible with Oracle Management Service.

Oracle Databases allow you to monitor and apply thresholds to many of its own performance metrics internally; this Advanced Queue Receivelet is dedicated to receiving notifications of these metrics. Therefore, Advanced Queue Receivelets apply only to Oracle Database targets.

While monitoring Oracle Databases, every time one of its performance metrics exceeds its threshold, the database posts an alert to the Advanced Queue. When Oracle Management Agent connects to an Oracle Database, it registers itself as a subscriber to the Advanced Queue; thereafter, a copy of each threshold alert is preserved for Oracle Management Agent in the queue.

If Oracle Management Agent is running and connected to the database at the time the alert is enqueued, then the alert is immediately available and passed on to the repository tier. However, if Oracle Management Agent is not running, then the alert is preserved and made available to the receivelet only when they connect to the database next time.

For the Advanced Queue Receivelet to receive notifications, the oracle_database.xml file must be updated to include push descriptor definitions for different metrics. The oracle_database.xml file can be found at $EMDROOT/sysman/admin/metadata/. You can retain the query descriptors defined in this file to ensure that the SQL Fetchlets continue to collect regular data, while the Advanced Queue Receivelets receive immediate notification of alerts as they are generated by Oracle Databases.

The push descriptors in the Advanced Queue Receivelet contain the properties such as QueueName (defined to the constant GLOBAL string 'ALERT_QUE'), MachineName, Port, SID, UserName, and password. The push descriptors may also define a KeyField property, the value of this property names the field in the Advanced Queue alert whose value should be used as the keyvalue parameter in any call to nmercm_RecvletManager_reportEvent().

Input Parameters

Table 9-2 AQ Receivelet Input Parameters

Parameter Type Description Use

QueueName

String

Name of the advanced queue.

Required.

MachineName

String

Database host.

Required.

Port

Integer

Database port.

Required

SID

String

Database SID.

Required

UserName

String

User name.

Required

password

String

User password.

Optional; default is ""


Example

Example 9-6 shows how the push descriptor can be defined in the oracle_database.xml file for the wait_bottlenecks metric that does not have a key value.

Example 9-6 Push Descriptor in the oracle_database.xml File for a Metric without Key Value

    <PushDescriptor RECVLET_ID="AQMetrics"> 
      <PushProperty NAME="QueueName" SCOPE="GLOBAL">ALERT_QUE</PushProperty> 
      <PushProperty NAME="MachineName" SCOPE="INSTANCE">MachineName</PushProperty> 
      <PushProperty NAME="Port" SCOPE="INSTANCE">Port</PushProperty> 
      <PushProperty NAME="SID" SCOPE="INSTANCE">SID</PushProperty> 
      <PushProperty NAME="UserName" SCOPE="INSTANCE">UserName</PushProperty> 
      <PushProperty NAME="password" SCOPE="INSTANCE">password</PushProperty> 
    </PushDescriptor> 

Example 9-7 shows how the push descriptor can be defined in the oracle_database.xml file for the tbspFull metric that has a key value. The key value is the name column. The value, which is the name of the tablespace an alert applies to, will be sent in the OBJECT_NAME field of the alert.

Example 9-7 Push Descriptor in the oracle_database.xml File for a Metric with Key Value

     <PushDescriptor RECVLET_ID="AQMetrics"> 
      <PushProperty NAME="QueueName" SCOPE="GLOBAL">ALERT_QUE</PushProperty> 
      <PushProperty NAME="MachineName" SCOPE="INSTANCE">MachineName</PushProperty> 
      <PushProperty NAME="Port" SCOPE="INSTANCE">Port</PushProperty> 
      <PushProperty NAME="SID" SCOPE="INSTANCE">SID</PushProperty> 
      <PushProperty NAME="UserName" SCOPE="INSTANCE">UserName</PushProperty> 
      <PushProperty NAME="password" SCOPE="INSTANCE">password</PushProperty> 
      <PushProperty Name="KeyField" SCOPE="GLOBAL">OBJECT_NAME</PushProperty> 
    </PushDescriptor> 

Error Handling

There are no exceptions thrown from Advanced Queue Receivelets.

9.3 HTTP Receivelets

HTTP Receivelets function much like other receivelets, except that they are designed to receive notifications from targets that communicate over HTTP or HTTPS protocol.

For example, application servers running in your environment. These application server may have built-in mechanism to trigger notifications every time a threshold is reached or the status is down. HTTP Receivelets allow you to receive these notifications and store that information (in XML files) in the upload directory.

The following are the parameters used when data information is sent as a notification:

  • TargetName

  • TargetType

  • MetricName

  • CollTS

The following are the parameters used when alert information is sent as a notification:

  • TargetName

  • TargetType

  • CollTS

  • MetricName

  • MetricColumn

  • KeyValue

  • Severity

  • DataValue

  • AlertMessage

Example 9-8 shows how a notification (without an access key) from an HTTP-based targets looks. These notifications do not need any translation, and the information passed is simply captured and stored as is in the XML files. Example 9-9 shows how a simple push descriptor can be defined in the metadata.xml file to receive this information.

However, you may choose to generate access keys that can be used for authenticating the metric information coming from an HTTP-based target. This is to ensure that notifications are parsed only if they are from authorized targets.

In general, communications to the Management Agent should be protected to avoid problems of spoofing and denial of service attacks. Understandably, one option is to use HTTPS protocol as the communication mechanism to encrypt the channel, and the other option is to keep the communication restricted to targets that are authorized by the Management Agent. This authorization mechanism involves generation of access keys.

Authorization or access keys can be generated on demand by the Management Agent and passed on to the HTTP-based targets so that they can use them while connecting to the HTTP Receivelet. The Management Agent stores these generated keys in its memory and verifies if every notification has a valid key. If the key has already been generated for a request string, then that value is returned. Example 9-10 shows how a complex push descriptor can be defined in the metadata.xml file to generate access keys.

Note:

securing your communication is only an option. You can also receive notifications without authenticating the information.

Once the access keys are generated, they need to be sent to the HTTP-based targets so that they can use them while connecting to the HTTP Receivelet. This can be done using one of the following methods:

  1. An optional command to run a script as part of the HTTP Receivelet push descriptor. Example 9-10 shows how a push descriptor can be used with this optional command parameter to run the script and send back the generated access keys.

  2. Setting up a polled metric that will start up the process and provide it with the access key. Example 9-11 shows how a query descriptor can be used to poll the HTTP-based target at regular intervals and send back the generated access keys.

Input Parameters

Table 9-3 HTTP Receivelet Input Parameters

Parameter Type Description Use

command

String

Name of the perl script used that will generate the access key for the metric information received from a particular server.

Optional. Used only when authentication using access keys is required.

STDIN

String

Controls the value that is passed on the stdin to the launched command. The value is used inside the launched command file for access key generation logic.

Optional

ENV

String

Controls the value that is passed via the environment. (Note that if security is not a concern, then integrators can also use a ENV prefix to request a value to be passed in the environment to the launched command). The value is used inside the launched command file for access key generation logic.

Optional


Example

Example 9-8 shows how a notification without an access key from an HTTP-based targets looks:

Example 9-8 Notification without an Access Key

 <!-- Connect -->
<HTTPRecvletConnect HOST="%%EMD_HOST" PORT="%%EMD_PORT" ACCESS_KEY_FILE="%%{T_WORK}/tvmayh04.key">
<!-- get database metric and dump the metric data -->
<HTTPRecvletSendAlert TARGET_TYPE="tvmayh04" TARGET_NAME="tvmayh04" METRIC_NAME="metric" METRIC_COLUMN="col2" SEVERITY="CLEAR" KEY_VALUE="1" VALUE="1" MESSAGE="Severity from recvlet" TIMESTAMP="2000-01-01 00:00:00" />
<!-- Disconnect -->
</HTTPRecvletConnect>

Example 9-9 shows how a simple push descriptor in the metadata.xml file looks:

Example 9-9 Simple Push Descriptor in the metadata.xml File

<Metric NAME="metric" TYPE="TABLE">
   <TableDescriptor>
       <ColumnDescriptor NAME="col1" TYPE="NUMBER" IS_KEY="TRUE"/>
       <ColumnDescriptor NAME="col2" TYPE="NUMBER" IS_KEY="FALSE"/>
   </TableDescriptor>
   <PushDescriptor RECVLET_ID="HTTP">
   </PushDescriptor>
 </Metric> 

Example 9-10 shows how a complex push descriptor in the metadata.xml file looks, and how the generated access keys can be sent back to the HTTP-based target:

Example 9-10 Complex Push Descriptor in the metadata.xml File

<PushDescriptor RECVLET_ID="HTTPRecvlet">
  <Property NAME="command" SCOPE="GLOBAL">%perlBin%/perl %scriptsDir%/startHRClient.pl</Property>
  <Property NAME="STDINKey" SCOPE="SYSTEMGLOBAL">_hr_acc_key_%NAME%_%TYPE%_TestMetric</Property>
  <Property NAME="STDINTName" SCOPE="INSTANCE">NAME</Property>
  <Property NAME="STDINTType" SCOPE="INSTANCE">TYPE</Property>
  <Property NAME="STDINMetric" SCOPE="GLOBAL">TestMetric</Property>
</PushDescriptor>

This results in the startHRClient.pl script being run and the following values being sent to stdin.

Key=<AccessKey>
TName=<TargetName>
TType=<TargetType>
Metric=TestMetric

The STDIN method of passing values helps to keep sensitive data out of the command line or in the process environment.

Example 9-11 Polling Method to Send Back the Generated Access Keys

<QueryDescriptor FETCHLET_ID="OSLineToken">
<Property NAME="command" SCOPE="GLOBAL">%perlBin%/perl %scriptsDir%/checkHRClient.pl</Property>
<Property NAME="STDINKey" SCOPE="SYSTEMGLOBAL">_hr_acc_key_%NAME%_%TYPE%_TestMetric</Property>
<Property NAME="ENVTName" SCOPE="INSTANCE">NAME</Property>
<Property NAME="ENVTType" SCOPE="INSTANCE">TYPE</Property>
<Property NAME="ENVMetric" SCOPE="GLOBAL">TestMetric</Property>
</QueryDescriptor>

Error Handling

There are no exceptions thrown from HTTP Receivelets.