Skip Headers
Oracle® Enterprise Manager Extensibility Guide
11g Release 1 (11.1.0.1)

E12135-03
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

8 Fetchlets

Enterprise Manager data retrieval is handled through predefined "fetchlets." A fetchlet is a parametrized data access mechanism that takes arguments (For example, a script, a SQL statement, a target instance's properties) as input and returns formatted data. Each fetchlet handles a specific type of data access. The fetchlets supplied with Enterprise Manager provide data retrieval capability for the most common data access methods, such as SQL, SNMP (Simple Network Management Protocol), HTTP, and DMS (Dynamic Monitoring Service). To handle more complex data access requirements, Enterprise Manager also provides an OS command fetchlet that allows developers to implement custom metric collection methods.

The following fetchlets are supplied with Enterprise Manager:

OS Command Fetchlets

The operating system (OS) command fetchlets allow you to obtain metric data by executing OS commands (either individually or from scripts) that return a standard out (stdout) data stream.

Three OS command fetchlets are available:

OS Fetchlet

The OS Fetchlet executes a given OS command and returns the command's output in a single cell table.

Input Parameters

Table 8-1 OS Fetchlet Input Parameters

Parameter Type Description Use

command

string

Operating system command to be executed.

required

ENVname

string

OSFetchlet parameters starting with "ENV" appear in the execution environment for the command as name environment variables

zero or more

errStartsWith

String

When defined, this property allows you to define a custom prefix for error messages. If this property is not defined, the OSFetchlet defaults to "em_error=" as the message prefix.

Optional

script

string

Specifies the script to be executed if command property only provides an interpreter. For example, command might consist of "perl." script is then used to specify the particular perl script to be run.

Although scripts can be specified directly from the command parameter, using the script parameter adds to stylistic clarity and readability when defining a target type metadata file.

Optional

args

string

A property that is taken as one or more arguments to the command and script properties.

Although args can be specified directly from the command parameter, using the args parameter adds to stylistic clarity and readability when defining a target type metadata file.

Optional

separateErrorStream

boolean

If an error occurs while executing a command, this property instructs the fetchlet whether to return both the stdout and stderr to the user as an error message. When set to TRUE, only stderr output is sent to the user as an error message when there is a command error. When set to FALSE (default value), both the stdout and the stderr are sent to the user as an error message upon command failure.

Optional. (TRUE/FALSE)

em_metric_timeout

integer

Metric timeout period (in seconds). After the timeout period has finished, the Management Agent returns a timeout exception and terminates any child processes that may have been created. The Management Agent DOES NOT kill any of the grandchild processes.

Specify "-1" for no timeout period.

Optional


Example

You want to obtain metric data by executing the UNIX echo command.

Executing the command from the shell environment, you would enter:

echo Line 1|some more|even more\nLine 2\n\nLine 4|a little more|\n|Line 5\n|Line 6|\n|Line 7|again|\nLine 8|the|end 

The echo command produces the following standard output:

Line 1|some more|even more 
Line 2 

Line 4|a little more| 
|Line 5 
|Line 6| 
|Line 7|again|
Line 8|the|end 

Using the OS Fetchlet with the given example command.

The fetchlet returns the following 1 x 1 table:

Figure 8-1 Table Returned by the OS Fetchlet

Description of Figure 8-1 follows
Description of "Figure 8-1 Table Returned by the OS Fetchlet"

The raw output of the OS command is returned. Any standard error output is appended to the standard output.

Error Handling

Any problems encountered launching the command (For example, the command program no longer exists) results in an oracle.sysman.emSDK.emd.fetchlet.MetricSourceException wrapping a java.io.IOException. If the command exits with a non-zero exit value, the fetchlet throws an oracle.sysman.emSDK.emd.fetchlet.MetricSourceException wrapping an oracle.sysmand.emd.fetchlets.CommandFailedException.

Notes

Commands are NOT executed as if they are being run in a shell. This means that common shell symbols do not work, including piping, output redirection, and backgrounding.

Commands cannot read from standard input.

The fetchlet blocks and waits for the command to finish.

OSLines Fetchlet (split into lines)

The OS Lines Fetchlet executes a given OS command and tokenizes the OS command's output. The output is tokenized by lines. The fetchlet returns the tokens in a single column table. The nth row in the table represents the nth line in the OS command's output.

To get the raw, untokenized output of an OS command, use the OS Fetchlet. To get the output of an OS command tokenized by lines and each line tokenized by a given delimiter, see the OS Line Token Fetchlet.

Input Parameters

Table 8-2 OSLines Fetchlet Input Parameters

Parameter Type Description Use

command

string

Operating system command to be executed.

required

startsWith

string

Only lines starting with this string are included in the result.

optional; default = "" (all lines are included)

ENVname

string

Parameters starting with "ENV" appear in the execution environment for the command as name environment variables

zero or more of these

errStartsWith

string

When defined, this property allows you to define a custom prefix for error messages. If this property is not defined, the OSFetchlet defaults to "em_error=" as the message prefix.

Optional

script

string

Specifies the script to be executed if command property only provides an interpreter. For example, command might consist of "perl." script is then used to specify the particular perl script to be run.

Although scripts can be specified directly from the command parameter, using the script parameter adds to stylistic clarity and readability when defining a target type metadata file.

Optional

args

string

A property that is taken as one or more arguments to the command and script properties.

Although args can be specified directly from the command parameter, using the args parameter adds to stylistic clarity and readability when defining a target type metadata file.

Optional

separateErrorStream

boolean

If an error occurs while executing a command, this property instructs the fetchlet to whether to return both the stdout and stderr to the user as an error message. When set to TRUE, only stderr output is sent to the user as an error message when there is a command error. When set to FALSE (default value), both the stdout and the stderr are sent to the user as an error message upon command failure.

Optional. (TRUE/FALSE)

em_metric_timeout

integer

Metric timeout period (in seconds). After the timeout period has finished, the Management Agent returns a timeout exception and terminates any child processes that may have been created. The Management Agent DOES NOT kill any of the grandchild processes.

Specify "-1" for no timeout period.

Optional


Example

Let's take the following Unix command:

echo Line 1|some more|even more\nLine 2\n\nLine 4|a little more|\n|Line 5\n|Line 6|\n|Line 7|again|\nLine 8|the|end 

It produces the following output:

Line 1|some more|even more  Line 2 

Line 4|a little more|  |Line 5  |Line 6|  |Line 7|again|  Line 8|the|end 

Running OSLinesFetchlet with the given example command produces the following single column table.

Figure 8-2 Table Returned by the OS LINES Fetchlet

Description of Figure 8-2 follows
Description of "Figure 8-2 Table Returned by the OS LINES Fetchlet"

Note that without content, "\n" results in a blank line inserted between Line 2 and Line 4.

Notes:

Commands are NOT executed as if they are being run in a shell. This means that common shell symbols do not work, including piping, output redirection, and backgrounding.

Commands cannot read from standard input.

The fetchlet blocks and waits for the command to finish.

The standard output of the command is captured and the standard error is captured and appended to the standard output.

Lines are tokenized using "\n".

OSLineToken Fetchlet (tokenized lines)

The OS Line Token Fetchlet executes a given OS command and tokenizes the OS command's output. The output is tokenized first by lines, and then each line is tokenized by a given delimiter set. The fetchlet returns the tokens in a table. The nth row in the table represents the nth line in the OS command's output. The nth column in the table represents the nth token in a line as determined by the given delimiter set.

To get the raw, untokenized output of an OS command, see the OS Fetchlet.

Input Parameters

Table 8-3 OSLineToken Fetchlet Input Parameters

Parameter Type Description Use

command

string

Operating system command to be executed.

required

delimiter

string

Set of characters that act as delimiters to tokenize the lines

optional; default = "" (just breaks output into lines)

startsWith

string

Only lines starting with this string are included in the result

optional; default = "" (all lines are included)

ENVname

string

Parameters starting with "ENV" appear in the execution environment for the command as name environment variables

zero or more of these

errStartsWith

String

When defined, this property allows you to define a custom prefix for error messages. If this property is not defined, the OSFetchlet defaults to "em_error=" as the message prefix.

Optional

script

string

Specifies the script to be executed if command property only provides an interpreter. For example, command might consist of "perl." The script is then used to specify the particular perl script to be run.

Although scripts can be specified directly from the command parameter, using the script parameter adds to stylistic clarity and readability when defining a target type metadata file.

Optional

args

string

A property that is taken as one or more arguments to the command and script properties.

Although args can be specified directly from the command parameter, using the args parameter adds to stylistic clarity and readability when defining a target type metadata file.

Optional

separateErrorStream

boolean

If an error occurs while executing a command, this property instructs the fetchlet to whether to return both the stdout and stderr to the user as an error message. When set to TRUE, only stderr output is sent to the user as an error message when there is a command error. When set to FALSE (default value), both the stdout and the stderr are sent to the user as an error message upon command failure.

Optional. (TRUE/FALSE)

em_metric_timeout

integer

Metric timeout period (in seconds). After the timeout period has finished, the Management Agent returns a timeout exception and terminates any child processes that may have been created. The Management Agent DOES NOT kill any of the grandchild processes.

Specify "-1" for no timeout period.

Optional


Example

Let's take the following Unix command:

echo Line 1|some more|even more\nLine 2\n\nLine 4|a little more|\n|Line 5\n|Line 6|\n|Line 7|again|\nLine 8|the|end 

It produces the following output:

Line 1|some more|even more 
Line 2 

Line 4|a little more| 
|Line 5 
|Line 6| 
|Line 7|again| 
Line 8|the|end 

Running OSLineTokenFetchlet with the given example command and a single character "|" for the delimiter generates the following table:

Figure 8-3 Table Returned by the OS Token Lines Fetchlet

Description of Figure 8-3 follows
Description of "Figure 8-3 Table Returned by the OS Token Lines Fetchlet"

Error Handling

Any problem launching the command (unable to find the command program) results in a oracle.sysman.emSDK.emd.fetchlet.MetricSourceException wrapping a java.io.IOException.

If the command exits with a non-zero exit value, the fetchlet throws a oracle.sysman.emSDK.emd.fetchlet.MetricSourceException wrapping a oracle.sysmand.emd.fetchlets.CommandFailedException.

Notes

Commands are NOT executed as if they are being run in a shell. This means that common shell symbols do not work, including piping, output redirection, and backgrounding.

The fetchlet promptly closes the input stream to the running command.

The fetchlet blocks and waits for the command to finish.

Lines are tokenized using "\n".

The delimiter can be a single character or a set of characters. For example, it can be "|+_", if the line should be broken up by pipes, pluses, and underscores. If two or more delimiters are together in the output text, such as "||" or "+|+", then it is as if there are empty string tokens between them. These empty strings get columns in the result table. It is NOT considered that there are empty strings preceding a delimiter that starts a line or following a delimiter that ends a line.

In order to express non-printable characters in the delimiter set (such as tabs) in XML, use "&#xHH;" where H is the hexadecimal identifier for the character.

SQL Fetchlet

The SQL Fetchlet executes a given SQL statement on a given database as a given user and returns the table result.

Input Parameters

Table 8-4 SQL Fetchlet Input Parameters

Parameter Type Description Use

Connection Information

MachineName

string

Database host

Required

Port

integer

Database port

Required

SID

string

Database SID

Required unless ServiceName is specified

ServiceName

string

Database ServiceName

Required unless SID is specified

OidRepSchemaName

string

Enterprise Manager Management Repository name. This parameter extracts database connection string and credential information (MachineName, Port, SID, ServiceName, UserName, password) from the Management Repository. Using this parameter eliminates the need to specify the connection and credential parameters in each target type metadata file, or having this target instance information exposed in the targets.xml file.

Required if connection and credential information parameters are not used.

OracleHome

string

Database's Oracle Home (used in conjunction with OidRepSchemaName).

Required when OidRepSchemaName is used.

Credential Information

UserName

string

user name

Required

password

string

user password

Optional; default is ""

Role

string

Role used when connecting to the database (e.g., SYSDBA)

optional; allowed choices are SYSDBA, SYSOPER, and NORMAL (default)

General

 

STATEMENT

string

SQL statement or PL/SQL block

Required unless FILENAME is specified.

FILENAME

string

Full path of the file containing the SQL statement or PL/SQL block

Required unless STATEMENT is specified.

NUMROWS

integer

Maximum number of rows to output.

Required

Bind Parameters

SQLINPARAM<position>

string

Value of input parameter at position <position>

Zero or more, one for each input parameter.

SQLOUTPARAMPOS

integer

Position of output parameter, if it exists

There can be exactly one output parameter, if it exists.

SQLOUTPARAMTYPE

string

Type of the output parameter, if it exists.

There can be exactly one output parameter type, if it exists.

transpose

TRUE/FALE

If TRUE, the result is transposed: rows to columns and columns to rows.

 

Notes

The SQL Fetchlet uses the Oracle Call Interface (OCI) from libclntsh.so.

The SQL statement or PL/SQL block can be specified either through the STATEMENT property, or via a file whose name is provided through the FILENAME property.

The SQL fetchlet supports input parameters. Input and output parameters are indicated in the SQL/PLSQL text in the usual way, by using ":<number>". Input parameters can be used to bind values to both SQL queries and PL/SQL blocks.

Input parameter values are specified as properties of the form SQLINPARAM<position>. There can be any number of input parameters. The input parameters need to be scalar: input parameters of type ARRAY and STRUCT are not supported.

The SQL fetchlet supports the execution of anonymous PL/SQL blocks (which may call other functions or procedures) to retrieve data. When executing a block of PL/SQL, data is returned to the fetchlet by means of an OUT parameter. There can be exactly one out parameter. It must be of type SQL_CURSOR (a PL/SQL REF CURSOR), or it must be a named type that represents an array of objects. In the latter case, each field of the object represents one column of the table; and each object instance in the array represents one complete row in the table. The OUT parameter position and type are specified by means of the properties SQLOUTPARAMPOS and SQLOUTPARAMTYPE. If an OUT parameter is specified, then the fetchlet assumes it is executing PL/SQL and treats the STATEMENT property as an anonymous PL/SQL block.

Note:

When using a SQLOUTPARAMTYPE of type 'ARRAY', you must identify the array as follows:
  • If you create the array type specified in the SQLOUTPARAMTYPE from SQL*Plus or any utility without using double quotation marks to surround the array name, then you must specify the array name using all upper-case letters in the target metadata file for this property. The reason for this because the RDBMS automatically changes the array name to all upper-case.

  • If you create the array type specified in the SQLOUTPARAMTYPE from SQL*Plus or any utility using double-quotation marks to surround the array name, then the RDBMS retains the case specified. For this reason, users must specify the array name using the same case used in the target metadata file.

If no OUT parameter is specified, the fetchlet assumes that it is executing a SQL query.

Note that all input parameters to the SQL fetchlet are strings. This means that all other data types will have to be converted to strings. This is straightforward for data types such as numbers, but not, for example, dates and timestamps. You can pass an absolute date or timestamp by passing a character representation of the value (using a DateFormat class). There is no way currently to pass in a date function, such as SYSDATE or SYSDATE+1. In such case, you could embed the date argument directly in the SQL, for example:

begin func1(:1, :2, SYSDATE); end;

The other caveat is passing null arguments to a procedure. Consider the following SQL:

STATEMENT=begin func1(:1,:2); end;
SQLINPARAM1=null
SQLOUTPARAMPOS=2
SQLOUTPARAMTYPE=fooret

Assume that the first argument is intended to be a varchar2. By parameterizing it and passing 'null' as the first argument, what we are really doing is passing the *string* 'null' to the argument, and not a null value. If you intend to pass a null value, do the following:

STATEMENT=begin func1(null, :1); end;
SQLOUTPARAMPOS=1
SQLOUTPARAMTYPE=fooret

Examples

The following properties execute a query (get all users) with no parameters:

MachineName=skini-pc 
Port=1521 
SID=o817 
UserName=scott 
password=tiger 
STATEMENT=select * from all_users; 
NUMROWS=30 

The following properties execute a query (get the first few objects of a specified type owned by a specified user) with input parameters:

MachineName=skini-pc 
Port=1521 
SID=o817 
UserName=scott 
password=tiger 
STATEMENT=select * from all_objects where owner=:1 and object_type=:2 and rownum<:3tt>
NUMROWS=30 
SQLINPARAM1=SYSTEM 
SQLINPARAM2=INDEX 
SQLINPARAM3=10 

The following example executes a PL/SQL procedure that returns a cursor and has input parameters:

MachineName=skini-pc
Port=1521
SID=o817
UserName=scott
password=tiger
STATEMENT=begin :1 := skini_junk.func1(:2); end;
NUMROWS=30
SQLINPARAM2=SYSTEM
SQLOUTPARAMPOS=1
SQLOUTPARAMTYPE=sql_cursor

The following example specifies a PL/SQL procedure that returns an array of strings:

MachineName=skini-pc
Port=1521
SID=o817
UserName=scott
password=tiger
STATEMENT=begin skini_junk.newproc(:1,:2); end;
NUMROWS=30
SQLINPARAM1=SYSTEM
SQLOUTPARAMPOS=2
SQLOUTPARAMTYPE=my_string_array

The following example specifies a PL/SQL package that returns an array of structures:

MachineName=skini-pc
Port=1521
SID=o817
UserName=scott
password=tiger
STATEMENT=begin :1 := skini_junk.func2(:2,:3,:4,:5,:6); end;
NUMROWS=30
SQLINPARAM2=somename
SQLINPARAM3=someplace
SQLINPARAM4=someanimal
SQLINPARAM5=something
SQLINPARAM6=22
SQLOUTPARAMPOS=1
SQLOUTPARAMTYPE=my_struct_array

The PL/SQL used in the examples above is provided below for reference:

create or replace type my_type as Object (
    name varchar2(128),
    place varchar2(128),
    animal integer,
    thing number,
    thing2 number);
/
create or replace type my_struct_array as table of my_type;
/

create or replace type my_string_array as table of varchar2(3000);
/

create or replace type my_int_array as table of integer;
/

create or replace package skini_junk as

type Jcr is ref cursor;

function func1(username in varchar2) return Jcr;
function func2(name varchar2, place varchar2, animal integer,
             thing number, thing2 number) return my_struct_array;
procedure newproc(name varchar2, outArray OUT my_string_array);
procedure newproc2(numrows in varchar2, outArray OUT my_int_array);

end skini_junk;
/


create or replace package body skini_junk as

function func1(username in varchar2) return Jcr is
cr Jcr;
begin
    open cr for select object_name, object_type, status from all_objects where
         owner=upper(username);

     return cr;
end;

function func2(name varchar2, place varchar2, animal integer,
             thing number, thing2 number) return my_struct_array IS
ret my_struct_array := my_struct_array();

begin
    ret.extend(50);

    for i in 1..50 loop
        ret(i) := my_type(name || i,
                          place || i,
                          animal+i,
                          thing+i,
                          thing2+i);
    end loop;
    return ret;
end;

procedure newproc(name varchar2, outArray OUT my_string_array) IS
begin
    outArray := my_string_array();
    outArray.extend(100);

    for i in 1..100 loop
        outArray(i) := name || i;
    end loop;
end;

procedure newproc2(numrows in varchar2, outArray OUT my_int_array) IS
begin
    outArray := my_int_array();
    outArray.extend(numrows);
    for i in 1..numrows loop
        outArray(i) := i;
    end loop;
end;

end skini_junk;
/

SNMP Fetchlet

An object identifier (OID) corresponds to either a MIB variable instance or a MIB variable with multiple instances. Given a list of (OIDs), the SNMP Fetchlet polls an SNMP agent on a given host for corresponding instances.

Input Parameters

Table 8-5 SNMP Fetchlet Input Parameters

Parameter Type Description Use

hostname

string

Host name of the SNMP agent

Required examples: "bigip.us.oracle.com" "148.87.10.5"

PORT

string

Port of the SNMP agent

Optional. Default is "161"

COMMUNITY

string

SNMP community string

Optional; Default is "public"

TIMEOUT

STRING

SNMP timeout.

Optional. Default is five seconds.

OIDS

string

A list of substrings separated by delimiters. Each substring starts with an OID (in numerical dot notation), and can be optionally ended with *PlacementOID. (See notes for details.)

Required; examples:

"1.3.6.1.2.1.2.1.1.1.0,1.3.6.1.2.1.2.1.1.3.0,1.3.6.1.2.1.2.1.1.5.0"

"1.3.6.1.2.1.2.1.2.2.1.2 1.3.6.1.2.1.2.1.2.2.1.10 1.3.6.1.2.1.2.1.2.2.1.16"

"1.3.6.1.2.1.2.2.1.3 1.3.6.1.2.1.2.2.1.5 1.3.6.1.2.1.4.20.1.1*1.3.6.1.2.1.4.20.1.2 1.3.6.1.2.1.4.20.1.3*1.3.6.1.2.1.4.20.1.2"

DELIM

string

A delimiter to separate individual substrings in OIDS.

Optional; default is whitespace characters .(dot), *(star) and 0-9 (digits) cannot be used as delimiters

TABLE

string

Each OID in OIDS corresponds to a variable with multiple instances if this parameter is "TRUE" and to a single variable instance if it is "FALSE".

Optional; default is "FALSE"


Error Handling

MissingParameterException is thrown if either host name or OIDS is not given. FetchletException is thrown if TABLE is not equal to either TRUE or FALSE, an I/O error occurs while sending/receiving SNMP messages to/from the agent, or the agent responds with an SNMP error.

Notes

The table returned by the fetchlet will contain a column for every OID in OIDS. If input OIDs correspond to single variable instances, the table will have just one row with those instances. On the other hand, if the OIDs correspond to variables with multiple instances, each column in the table will contain instances for its OID and each row will correspond to a different subidentifier. (A subidentifier is an OID extension that uniquely identifies a particular variable instance for some MIB variable.) OIDS must contain either all OIDs with subidentifiers or all OIDs without the subidentifiers.

For example, to request the variable instances for the three OIDs: sysDescr, sysUpTime, and sysName, OIDS would have to be "1.3.6.1.2.1.2.1.1.1.0 1.3.6.1.2.1.2.1.1.3.0 1.3.6.1.2.1.2.1.1.5.0". In this case, all OIDs contain the instance subidentifier, ".0". The return table would appear as follows (the actual values may be different):

Alternatively, assume that some MIB contains the following 3 columns and 4 instances:

Figure 8-5 SNMP Fetchlet: Columns 3 and 4 Content

Description of Figure 8-5 follows
Description of "Figure 8-5 SNMP Fetchlet: Columns 3 and 4 Content"

To construct a table with 3 columns corresponding to ifDescr, ifInOctets, and ifOutOctets, OIDS would have to be defined as follows

"1.3.6.1.2.1.2.1.2.2.1.2 1.3.6.1.2.1.2.1.2.2.1.10 1.3.6.1.2.1.2.1.2.2.1.16"

The fetchlet would return:

Figure 8-6 SNMP Fetchlet:ifDescr, ifInOctets, and ifOutOctets OIDS

Description of Figure 8-6 follows
Description of "Figure 8-6 SNMP Fetchlet:ifDescr, ifInOctets, and ifOutOctets OIDS"

The rows correspond to subidentifiers 1,2,3,4 respectively.

Any OID in OIDS can be appended with another placement OID. The variable instances for the placement OID do not appear in the returned table. Instead, they determine the place for the variable instances of the original OID within a column. In particular, for every variable instance I with subidentifier S in the set of instances for the original OID, (a) there must exist a variable instance X with subidentifier S in the set of instances corresponding to the placement OID, and (b) X is used as the subidentifier for the instance I.

For example, consider a MIB containing the following 3 columns, each with 4 variable instances:

Figure 8-7 SNMP Fetchlet: MIB Content with 4 Variable Instances

Description of Figure 8-7 follows
Description of "Figure 8-7 SNMP Fetchlet: MIB Content with 4 Variable Instances"

To construct a table containing ifDescr and ipAdEntNetMask, OID of ipAdEntIfIndex would have to be used as the placement OID to "align" the columns. Thus, the OIDS input to the fetchlet would be "1.3.6.1.2.1.2.1.2.2.1.2 1.3.6.1.2.1.2.1.4.20.1.3*1.3.6.1.2.1.2.1.4.20.1.2". The fetchlet output will be as follows:

Figure 8-8 SNMP Fetchlet: Table Containing ifDescr and ipAdEntNetMask

Description of Figure 8-8 follows
Description of "Figure 8-8 SNMP Fetchlet: Table Containing ifDescr and ipAdEntNetMask"

If OIDS were "1.3.6.1.2.1.2.1.2.2.1.2 1.3.6.1.2.1.2.1.4.20.1.3" for the previous example, the output would be as follows:

Figure 8-9 SNMP Fetchlet: Alternate OID

Description of Figure 8-9 follows
Description of "Figure 8-9 SNMP Fetchlet: Alternate OID"

URL Timing Fetchlet

The URL Timing Fetchlet gets the contents of a given URL, timing not only the base page source but any frames or images in the page as well.

Input Parameters

Table 8-6 URL Timing Fetchlet Input Parameters

Parameter Description Use

url#

URL(s) to download. "url0" is required but any number of URLs can be specified beyond url0 that following the convention: url0, url1, url2, url3.

Required.

proxy_host

Proxy host used to make a URL connection.

Optional. Specifies the proxy to be used for accessing URLs. If the proxy_host_override value is provided, then that value will be used instead.

proxy_port

Port used by the proxy host used make the URL connection.

Optional.

dont_proxy_for

Domains for which the proxy will not be used.

Optional. For example, .us.oracle.com, .uk.oracle.com

use_proxy

When used in conjunction with the proxy override input parameters, use_proxy specifies a proxy to be used in lieu of the original proxy. When set to false without the proxy override parameters set, no proxy is used.

Optional. Parameter can be set to true or false.

proxy_host_override

Alternate proxy host used to make the URL connection.

Optional. Overrides proxy_host.

proxy_port_override

Alternate proxy port used to make the URL connection.

Optional. Overrides proxy_port.

dont_proxy_override

Do not use the proxy for domains.

Optional. Parameter can be set to true or false.

internet_cert_loc

Path pointing to the location of a certificate to be used to access a secure (HTTPS) URL.

Optional.

auth_realm

Realm for the Basic Authentication log on. If the realm is not specified for the authentication, authentication does not occur and the download of the page fails with a 401 response code.

Optional.

auth_user

User name for Basic Authentication.

Optional.

auth_password

Password for Basic Authentication.

Optional.

retries

Number of times to retry the url if it initially fails.

Optional. Default = 1

connection_timeout

Wait time (in milliseconds) allowed to establish a connection to a server. This time also includes time required for name resolution.

Optional. Default= 60000 milliseconds (1 minute)

read_timeout

Idle time in the read waiting for the server to respond. For example, if no data is received from the server during the specified timeout period, the operation is considered failure.

Optional. Default = 12000 milliseconds (2 minutes)

timeout

Number of milliseconds after which the page download is considered a failure. This will detect if the site is functional but is extremely slow.

Optional. Default = 300000 ms (5 minutes)

status_comparator

When collating the rows to make a single row, the status_comparator parameter will indicate whether all URLs should have been a success (and) or any URLs should have been a success (or).

Optional. Default = and

cache

Indicates whether to use a cache when accessing an URL. Set the parameter to "n" to specify that no cache be used.

Optional. Default = yNote: The scope of the cache is per request. There is no persistent cache across multiple get metric requests.

output_format

Specifies the output format to be used: summary, detailed, repeat_column. For more information on output formats, see Metric Columns and Output Modes.

Required. summary : gives a default set of metrics in a single row for all urlsdetailed: gives a default set of metrics for each url.repeat_column : gives a single row of metric with timing for each of the url.

metrics

Specifies which metric columns need to be returned. For more information on metrics columns returned for each output format, see Table 8-8, "URLTIMING Fetchlet: Metric Columns"

Optional. Allows you to specify of what needs to be returned from the fetchlet and in which order.Example: status, status_description, total_response_time


Metric Columns and Output Modes

The format of information and specific metric information returned are controlled by the "output_format" and "metrics" input parameters. The following table lists the format categories and the metrics (columns) returned by each. For a description of available metric columns, see Table 8-8, "URLTIMING Fetchlet: Metric Columns"

Table 8-7 URLTIMING Fetchlet: Output Formats

Output Format Description Metric Columns

summary

Returns a default set of metrics in a single row for all URLs

If the metrics input parameter is specified, then only the columns specified will be returned.

computed_response_time, status, status_description, dns_time, connect_time, redirect_time, first_byte_time, html_time, content_time, total_response_time, rate, max_response_time, avg_response_time, avg_connect_time, avg_first_byte_time, broken_count, broken_content

detailed

Returns a default set of metrics for each url.

If the metrics input parameter is specified, then only the columns specified will be returned.

url, computed_response_time, status, status_description, dns_time, connect_time, redirect_time, first_byte_time, html_time, content_time, total_response_time, rate, redirect_count, html_bytes, content_bytes, total_bytes, avg_connect_time, avg_first_byte_time, broken_count, broken_details

repeat_column

Returns a single row of metrics with timing for each of the URLs.

If the metrics input parameter is specified, then those columns will be returned for each url followed by overall status and status_description. (Note the output will always be single row).

total_response_time repeated for each URL followed by overall status and status_description.


Metric Columns

The following table shows the metric columns returned by the URLTIMING fetchlet.

Table 8-8 URLTIMING Fetchlet: Metric Columns

Column Name Description

status

The overall status of all URLs. By default AND is used to compute overall status but this can be changed using the status_comparator input parameter.

connect_time

The time to connect to the server and send the request.

first_byte_time

Time taken between sending the request and reading the first byte from the response.

total_response_time

Time taken for fetching ALL urls and associated content (gif, css, javascript etc).

max_response_time

Also referred as Slowest page time. This the time taken by the slowest URL.

avg_response_time

Average response time for URL. Computed as total response time / number of pages (urls).

rate

Kilo Bytes per second. This is computed by total bytes received / total time taken to receive them.

html_time

Total time taken to download the html part of all pages. This time excludes time to fetch images and other contents. (Includes time to fetch frame html).

content_time

Time taken to download the page content (gif, javascripts, css etc.).

redirect_time

Total time taken for all redirects occurring while fetching the set of urls specified.

redirect_count

Number of redirects.

total_bytes

Total number of bytes.

html_bytes

Total number of HTML bytes. (Includes bytes for frame html).

content_bytes

Total number of content bytes.

status_description

This is present only when the status is down. Corresponds to HTTP Status description.

request_count

Number of request made. (Includes all html as well as content requests).

broken_count

Number errors while fetching images or other content elements.

broken_details

List of images or other content elements that could not be fetched. This has format of url[broken list], url[broken list...

computed_response_time

This time approximates the time it would have taken for a client (like browser) to fetch all the pages in the transaction. This number is computed as if the contents of every page (gifs, css etc) were fetched using multiple threads.

avg_connect_time

Total connect_time / total number of connections made.

avg_first_byte_time

Total First Byte Time / Number of requests made (either to fetch HTML or content).

dns_time

Time to resolve host name (not implemented, always returns zero).

url

Returns the url, can only be used in 'detailed' output_format.


Example

Let's take the following URL:

url0=http://www.oracle.com/ 

With the input parameter output_format=summary, the fetchlet returns the following table (minus the headers on the columns):

Figure 8-10 Summary Output Format

Summary Output Format
Description of "Figure 8-10 Summary Output Format"

With output_format = summary and metrics = total_response_time, status, status_description the fetchlet returns the following table (minus the headers on the columns):

Figure 8-11 Summary Output Format with Specified Metric Columns

Description of Figure 8-11 follows
Description of "Figure 8-11 Summary Output Format with Specified Metric Columns"

With output_format = summary and metrics = total_response_time, status, status_description the fetchlet returns the following table (minus the headers on the columns) and the server is giving error:

Figure 8-12 Summary Output Format with Specified Metric Columns and Internal Server Error

Description of Figure 8-12 follows
Description of "Figure 8-12 Summary Output Format with Specified Metric Columns and Internal Server Error"

Let's take the following URL:

url0=http://www.oracle.com/
url1=http://nedc.us.oracle.com/

With the output_format=summary, the fetchlet returns the following table (minus the headers on the columns). Here the numbers are time taken for fetching both the urls.

Figure 8-13 Summary Output Format for Two URLs

Description of Figure 8-13 follows
Description of "Figure 8-13 Summary Output Format for Two URLs"

With the output_format=detailed, the fetchlet returns the following table (minus the headers on the columns):

Figure 8-14 Detailed Output for Two URLs

Description of Figure 8-14 follows
Description of "Figure 8-14 Detailed Output for Two URLs"

With the output_format=repeat_column, the fetchlet returns the following table (minus the headers on the columns):

Figure 8-15 Repeat Column Output Format

Description of Figure 8-15 follows
Description of "Figure 8-15 Repeat Column Output Format"

Error Handling

Metric error if the URL parameter is missing, malformed, or if the metric cannot be computed.

Notes

The time required to perform a retry will be added on to the total time of the page. For example, if two retries are performed and then a success occurs, the total page time will be the time of the page that succeeded plus the time it took for the two retries to fail.

Proposed usage:

Dynamic Monitoring Service (DMS) Fetchlet

The Dynamic Monitoring Service (DMS) fetchlet contacts an Application Server (AS) and then collects the metrics instrumented by the DMS.

The DMS allows application and system developers to measure and export customized, component-specific performance metrics. The Oracle Management Agent allows software components to import runtime performance data into Oracle Enterprise Manager Grid Control.

The DMS Fetchlet is an Oracle Management Agent plug-in module that allows the Management Agent to import the performance data that is exported by the DMS. Using the DMS fetchlet, any component that is instrumented using DMS API calls may share its performance data with Enterprise Manager Grid Control.

Advantages to Using DMS for Oracle Management Agent Integration

With DMS, a component can insulate itself from the operational details of the Management Agent. A component would not need to deploy (or maintain) its own fetchlet or deploy (or maintain) a Tcl script or shell script to plug into one of the existing fetchlets. A component would not need to devise its own new way of measuring or exporting performance metrics. Performance metrics can be measured and reported in a consistent way across components. The DMS fetchlet contacts the remote DMS runtime directly with no need for forking shell scripts or Tcl scripts. Most importantly, DMS automatically produces the long, complicated metadata document for you and thereby saves many hours of tedious and error-prone hand editing.

Input Parameters

Table 8-9 DMS Fetchlet Input Parameters

Name Type Description Use

oraclehome

String

Top directory under which the monitored IAS instance is installed. It is used only for monitoring local IAS processes. For monitoring remote IAS processes, users should give it an empty value and specify property "opmnremoteport" and/or "machine" instead.

Required.

Example:

"/private/oracle/ias"

version

String

AS Version number of the target. It is used to distinguish the version of monitored AS instance.

Optional

Example:

"9.0.4"

opmnport

Integer

Oracle Process Monitoring and Notification (OPMN) port. It is used primarily for monitoring remote AS processes. It should be specified together with property "machine". If it is present and valid, property "oraclehome" and "httpport" are ignored.

Optional

Example:

"6200"

httpport

Integer

HTTP port is used primarily for monitoring stand-alone processes. It should be specified together with property "machine". It will be ignored, if property "opmnport" is present. If it is present and valid, property "oraclehome" is ignored.

Optional

Example:

"7777"

machine

String

Host name where the Internet Application Server (AS) instance runs. It should be specified together with property "opmnport". If it is not present, the local host is assumed.

Optional

Example:

"my-sun.us.oracle.com"

metric

String

Name of the table-type metric.

Required

Example:

"Servlets"

columnOrder

String

A list of metric column names separated by ";". The column names must be specified in same order as they appear in the target type metadata file.

Do not include "name", "host", "process" and "fullname" columns.

Required

Example:

"processTimes;totalRequest;requestRate"

usecache

String

Whether to cache this metric. The legal values are "true", "false" and "refreshall" with "true" being the default. The "refreshall" value tells the DMS to delete its cache data and retrieve the most recent data from all targets.

Optional.

Example: "false"

Setting "usecache" to "false" will bypass DMS caching

proxyHost

String

Proxy host through which to make the HTTP connection

Optional

Example:

"proxy.us.oracle.com"

proxyPort

Integer

Proxy port through which to make the HTTP connection

Optional

Example:

"80"

dontProxyFor

String

Domains for which the proxy will not be used.

Optional

Example:

".us.oracle.com" or "18.219.0"

useDefaultProxy

String

When used in conjunction with the proxy override parameters, this variable specifies a proxy other than the original one. When set to false without the proxy override parameters set, no proxy at all is used.

Optional

Example:

"true" or "false"

proxyHostOverride

String

proxy host through which to make the HTTP connection

Optional

Example:

"www-proxy.us.oracle.com"

proxyPortOverride

Integer

proxy port through which to make the HTTP connection

Optional

Example:

"80"

authrealm

String

Realm for the Basic Authentication logon. If the realm is not specified for the authentication, authentication does not occur and the download of the page fails with a 401 response code.

Optional

Example:

"Please input your flex account login:"

authuser

String

Username for Basic Authentication

Optional

"superuser"

authpwd

String

Password for Basic Authentication

Optional

Example:

"welcome"


Error Handling

DMS Fetchlet throws MissingParameterException if any of the properties "oraclehome", "metric", "columnOrder", "opmnport", or "httpport" is missing. It throws FetchletException if any of the ports given is not valid.

Notes

The first four columns of the metric table returned are always column "name", "fullname", "host" and "process". Therefore, do not include them in columnOrder string. Property "machine" should be specified together with either properties "opmnport" or "httpport". In this case, the property "oraclehome" is ignored.

DMS Fetchlet/Oracle Management Agent Integration Instructions

DMS has been used in several components (such as Apache, JServ, OSE, and Portal) to provide a consistent performance monitoring infrastructure for Oracle 9i Application Server. The Sensors are easy to use and save most of the work related to performance measurement because they hide most of the details related to timing, counting, and categorization. Finally, DMS hides many Management Agent details from component developers and much of the Management Agent integration effort.

Integrating DMS Data with the Management Agent

As mentioned earlier, DMS allows application and system developers to measure and export customized, component-specific performance metrics. The Oracle Management Agent enables software components to import runtime performance data into Enterprise Manager Grid Control. This section describes how to integrate DMS performance metrics with the Management Agent.

Step 1: Install AS

Step 2: Install Enterprise Manager Grid Control

Step 3: Instrument your Component with DMS

To enable DMS metrics for Enterprise Manager Grid Control, you must follow two additional rules:

  • Rule 1: All Nouns exported to the Management Agent must have types Noun types can be set either by specifying the "type" parameter in the Noun.create() methods or by using the Noun.setType(String) method. The idea is that every Noun type will be converted automatically to a management repository table. Every Noun of a given type will become a row in the type's corresponding management repository table. The metrics contained by a Noun become columns in the repository table metric. Any Noun without a type will not be exported to Management Agent.

  • Rule 2: All Nouns of a given type must contain a consistent set of Sensor names Because the metrics contained by a Noun become columns in a management repository table, you must make sure that all Nouns of a given type contain the same Sensors. This ensures that each row of the corresponding repository table has the same set of columns. DMS does not check this constraint for you.

For example, the following Java snippet shows how to create typed Nouns that contain a consistent set of Sensors. DMS will automatically convert these into a repository table named "MyType":

  /* first create the nouns*/
  Noun n1 = Noun.create("/myExample/myComponent/noun1", "MyType");
  Noun n2 = Noun.create("/myExample/myComponent/noun2", "MyType");

  /* next, create the Sensors */
  PhaseEvent pe1 = PhaseEvent.create(n1, "criticalPhase", "a critical interval");
  PhaseEvent pe2 = PhaseEvent.create(n2, "criticalPhase", "a critical interval");
  Event e1 = Event.create(n1, "importantEvt", "an important event");
  Event e2 = Event.create(n2, "importantEvt", "an important event");

  /* here is a third set that shows the use of Noun.setType(String) */
  PhaseEvent pe3 = PhaseEvent.create( 
                               "/myExample/myComponent/noun3/criticalPhase", 
                               "a critical interval");
  Event e3 = PhaseEvent.create( 
                               "/myExample/myComponent/noun3/importantEvt", 
                               "an important event");
  Noun n3 = Noun.get("/myExample/myComponent/noun3");
  n3.setType("MyType");

For this example, the "MyType" table will contain three rows and four columns. Besides the columns corresponding to the two Sensors, there will be a "name" column and a "path" column that will contain the DMS path name including the process name and "/myExample/myCom...".

If these Nouns/Sensors are created in several servlet engines within the AS site, then the AggreSpy will find each of the servlet engines and will aggregate all of the Nouns/Sensors into a single MyType table.

Step 4: Generate your Target Metadata Document

You can generate the Target Metadata Document using your browser. Point your browser to your AS site that you want to monitor using the following URL:

http://YOUR_AS_HOST:YOUR_AS_PORT/YOUR_SERVLET_PATH/AggreSpy?format=targetmetadata 

You should use the actual host, port and servlet path of your AS installation in the above URL. The servlet path usually defaults to "servlet". The XML document you get is the Target Metadata Document for your AS site. The first comment of the XML document explains where you can obtain the Target Metadata Document and instructions telling you what needs to be done to this document.

Step 5: Install the Target Metadata Document

Follow the steps described in the first comment of the XML document. Save the XML document to a file called "oracle_dms.xml" under the "metadata" directory of your Enterprise Manager installation (OMS_ORACLE_HOME/sysman/admin/metadata/). If you want to monitor a subset of the metrics or merge the metrics with the ones in the existing "oracle_dms.xml" file, you should save this new defintion to a separate file called target_name.xml. You will also need to change the Target Type entry in the generated metadata document.

Next, you should add the target instance information of your AS site to your "targets.xml" file residing under the top directory of your Enterprise Manager installation. You can find a block of XML tags in the comment you read. They look like:

<Target Type='oracle_dms' NAME='DMS_YOUR-IAS-HOST_YOUR-IAS-PORT' VERSION='2.0'>
 <Property NAME='host' VALUE='YOUR_IAS_HOST' />
 <Property NAME='port' VALUE='YOUR_IAS_PORT' />
 <Property NAME='dmsPath' VALUE='YOUR_SERVLET_PATH' />
</Target>

Copy this block and paste it to the "targets.xml" file between <targets> and </targets> tags.

Finally, to add the new target metadata file and target instance information from the targets.xml file to Enterprise Manager Grid Control, you must run the following command:

>$ORACLE_HOME/bin/emctl reload

Step 6: View your metrics

You are ready to view your metrics using Enterprise Manager's Metric Browser. See "Activating the Metric Browser" for information on setting up the Metric Browser. First, make sure that AS and your component are still running. Next, restart the Oracle Management Agent. Finally, point your browser to your Management Agent installation using the following URL:

http://<YOUR_AGENT_HOST>:<YOUR_AGENT_PORT>/emd/browser/main 

The Management Agent port information can be found in the $AGENT_HOME/sysman/config/emd.properties file at the EMD_URL line.

You should use the actual host and port of your Management Agent installation in the above URL. You will find your AS site listed as the target "DMS_YOUR-AS-HOST_YOUR-AS-PORT". If you click on the link, you will see a list of metric IDs. You can browse your metrics by clicking on the respective metric IDs.

HTTP Data Fetchlets

The HTTP data fetchlets obtain the contents of a URL and returns the contents of the URL as data. Three fetchlets are available:

URL Fetchlet (raw)

The URL Fetchlet gets the contents of a given URL and returns the contents of the URL in a single cell table.

To get the output of a URL tokenized by lines and each line tokenized by a given delimiter, see the URL Line Token Fetchlet.

Input Parameters

Table 8-10 URL Fetchlet Input Parameters

Name Description Use

url

URL to retrieve the contents of

required

proxyHost

proxy host through which to make the URL connection.

optional

proxyPort

proxy port through which to make the URL connection.

optional


Example

Let's take the following URL:

http://localhost/nhcities.txt 

It has the following contents:

Line 1: Nashua, Keene,

Line 2: Concord

Line 3: , Conway, Manchester, Milford, Brookline,

Line 4:

Line 5: Hollis, Meredith


Now let's run the URL Fetchlet with the given URL. The fetchlet returns the following one-by-one table:

Figure 8-16 URL Fetchet Output

Description of Figure 8-16 follows
Description of "Figure 8-16 URL Fetchet Output"

The raw contents of the URL is returned.

Error Handling

MissingParameterException if URL parameter is missing. FetchletException if the URL is malformed or an I/O error occurs in retrieving the content of the URL.

URL Lines Fetchlet (split into lines)

The URL Fetchlet gets the contents of a given URL and tokenizes the contents of the URL. The output is tokenized by lines. The fetchlet returns the tokens in a single column table. The nth row in the table represents the nth line of the URL contents.

Note:

To get the raw, untokenized contents of a URL, see the URL Fetchlet. To get the contents of a URL tokenized by lines and each line tokenized by a given delimiter, see the URL Line Token Fetchlet.

Table 8-11 URL Lines Fetchlet Input Parameters

Name Description Use

url

URL to retrieve the contents of

required

proxyHost

proxy host through which to make the URL connection.

optional

proxyPort

proxy port through which to make the URL connection.

optional

startsWith

only lines starting with this string are included in the result

optional; default = "" (all lines are included)


Example

Let's take the following URL:

http://localhost/nhcities.txt 

It has the following contents:

Line 1: Nashua, Keene,

Line 2: Concord

Line 3: , Conway, Manchester, Milford, Brookline,

Line 4:

Line 5: Hollis, Meredith


Now let's run the URL Fetchlet with the given URL.

The fetchlet returns the following table:

Figure 8-17 URL LInes Fetchlet Output

Description of Figure 8-17 follows
Description of "Figure 8-17 URL LInes Fetchlet Output"

Error Handling

MissingParameterException if URL parameter is missing.

FetchletException if the URL is malformed or an I/O error occurs in retrieving the content of the URL.

Notes

Lines are tokenized using "\n".

URL Line Token Fetchlet (tokenized lines)

The URL Fetchlet gets the contents of a given URL and tokenizes the contents of the URL. The output is tokenized first by lines, and then each line is tokenized by a given delimiter set. The fetchlet returns the tokens in a table. The nth row in the table represents the nth line of the URL content. The nth column in the table represents the nth token in a line as determined by the given delimiter set.

To get the raw, untokenized contents of a URL, see the URL Fetchlet.

Table 8-12 URL Line Token Fetchlet Input Parameters

Name Description Use

url

URL to retrieve the contents of

required

delimiter

set of characters that act as delimiters to tokenize the lines

optional; default = "" (just breaks output into lines)

proxyHost

proxy host through which to make the URL connection.

optional

proxyPort

proxy port through which to make the URL connection.

optional

startsWith

only lines starting with this string are included in the result

optional; default = "" (all lines are included)


Example

Let's take the following URL:

http://localhost/nhcities.txt 

It has the following contents:

Line 1: Nashua, Keene,

Line 2: Concord

Line 3: , Conway, Manchester, Milford, Brookline,

Line 4:

Line 5: Hollis, Meredith

Now let's run the URL Fetchlet with the given URL and a single character "," for the delimiter.

The fetchlet returns the following table:

Figure 8-18 URL Token Lines Output

Description of Figure 8-18 follows
Description of "Figure 8-18 URL Token Lines Output"

Error Handling

MissingParameterException if URL parameter is missing.

FetchletException if the URL is malformed or an I/O error occurs in retrieving the content of the URL.

Notes

Lines are tokenized using "\n".

The delimiter can be a single character or a set of characters. For example, it can be "|+_", if the line should be broken up by pipes, pluses, and underscores. If two or more delimiters are together in the output text, such as "||" or "+|+", then it is as if there are empty string tokens between them. These empty strings get columns in the result table. It is NOT considered that there are empty strings preceding a delimiter that starts a line or following a delimiter that ends a line.

In order to express non-printable characters in the delimiter set (such as tabs) in XML, use "&#xHH;" where H is the hexadecimal identifier for the character.

URLXML Fetchlet

The URL XML Fetchlet obtains the XML content of a given URL, and extracts information based on a given pattern. A pattern is a list of "chunks" of XML to match against. The return table is a table with a column for each grabber (*) in the pattern in order and a row each time the pattern chunk matches in the XML content.

Input Parameters

Table 8-13 URLXML Fetchlet Input Parameters

Name Description Use

url

URL to retrieve the contents of

Required.

pattern

The pattern used to extract information from XML; this is a list of XML chunks that that is compared against the XML content of the URL. Each chunk contains one or more "grabbers" (*) in the text portion of the elements that define what should the flattened into text and extracted.

Required.

proxyHost

The proxy host through which to make the URL connection.

Optional.

proxyPort

The proxy port through which to make the URL connection.

Optional.

ignoreDtd

If set to TRUE, specifies that the DTD file referenced by the content XML should be ignored. This is useful in cases where the DTD file cannot be accessed.

Optional.

generateKey

If set to true, a unique key will be generated for each row. The key will occupy the first column of the result, and will be numeric.

Optional.

throwConnException

If set to TRUE, a java.net.ConnectException will be thrown. Otherwise, it will be caught and an empty result set will be returned. Setting this property to FALSE provides behavior which is consistent with the DMSFetchlet.

Optional. The default value is TRUE.


Example

Let's take the following URL:

http://localhost/urlxmltestfile.xml

It has the following content:

<?xml version="1.0"?>
<testfile>
    <test>Simple text</test>
    <test><level>A little more complex</level></test>
    <test></test>
    <notatest></notatest>
    <test>Yet more complexity<level>Even a little more complex</level>Will it ever stop?</test>
    <test1>must match<level>extract me!</level></test1>
    <test1>must match here<level>extract me, too!</level></test1>
</testfile>

Running the URL XML Fetchlet with the given URL and the pattern:

<testfile><test>*<level>*</level></test></testfile>

returns the following table:

Figure 8-19 URL XML Fetchlet Output

Description of Figure 8-19 follows
Description of "Figure 8-19 URL XML Fetchlet Output"

Error Handling

MissingParameterException if URL or pattern parameters are missing.

A FetchletException is generated if:

Notes

Setting the proxy host and/or port changes these settings for the java.net package for the whole Java environment and is not thread-safe if the proxy settings are changing.

WBEM Fetchlet

The WBEM fetchlet accesses a CIMOM and retrieves requested information using the specified CIM class. The CIM class is mapped to a Management Repository table metric. The name of the CIM class is the name of the table metric that is returned, and the properties defined for the CIM class are used to name the table columns for the metric. The properties of interest must be specified during metric definition.

The fetchlet returns the instances that have been instantiated for the CIM class as rows of the Management Repository table metric.

Input Parameters

Table 8-14 WBEM Fetchlet Input Parameters

Name Type Description Use

hostname

String

Host name of the CIMOM

Optional; default is "localhost"

port

Integer

Port for the CIMOM

Optional; default is 5988

namespace

String

CIM Namespace

Optional; default is "root/cimv2"

username

String

Username to use for CIMOM authorization on the host where the CIMOM is running

Required

password

String

Password to use for CIMOM authorization on the host where the CIMOM is running

Required

CIMclassname

String

Name of the CIM class whose instances will be returned

Required for all operations except STATUS. STATUS operations just check whether the CIMOM is running, so a class name is not needed.

operation

String

Operation to be performed. Supported operations include COUNT, which returns a count of the number of instances in the class, VALUES, which returns the values of the specified properties for each instance of the class, or STATUS, which provides status information about the CIMOM.

Optional, default is VALUES

properties

String

The property names from the CIM class definition that we are interested in collecting.

Required for VALUES operation. If the operation is VALUES, we can have 1 to N of these, separated by a semicolon. If the operation is VALUES, and no properties are provided, an error is returned. Properties are handed to the EMD in the order that they are specified.


Error Handling

The following types of errors have been identified for the WBEM fetchlet.

MissingParameterException occurs when:

Fetchlet exception occurs when:

Notes

Ports: Some CIMOM client interfaces expose the port that the CIMOM is listening on while some clients do not. To cover both cases, the port is exposed as an optional input parameter that defaults to port 5988. This is the default Pegasus CIMOM listener port. The Java API that is provided through Sun's Wbem Services does not expose the CIMOM port.

Protocols: Most CIMOMs support either an RMI or HTTP protocol for communicating with the CIMOM. The testing that has been done shows that the HTTP protocol is not as stable, and in some cases, not fully implemented in the CIMOM. Because of this, the protocol currently defaults to RMI. The actual parameters for the WBEM Services CIMOM for the protocol are: CIMClient.CIM_RMI or CIMClient.CIM_XML.

Fetchlet Operations: The WBEM APIs are very flexible at allowing clients to traverse the class hierarchies that are defined and their associations. At this point in time, the options on accessing CIM data from an EMD are restricted to counting, getting the properties of classes, and CIMOM status. These are the more important operations that need to be performed for monitoring. As additional requirements come in, we can add new operations to support them if necessary. For the prototype, only the count operation has been implemented.

Authentication: Most CIMOMs provide APIs to support authentication through a user identity mechanism. The majority of the CIMOMs have not implemented the API, so this capability is really a no-op. In any case, we've supplied the capability in the fetchlet so that as CIMOM implementations catch up with the standard, we'll have the necessary support in place.

Examples

The Wbem fetchlet supports three basic operations. At this point, the fetchlet only handles one operation at a time, so you cannot mix count, status, and value operations within a single fetchlet call. The following example shows how to write the metadata for a COUNT operation:

Example 8-1 COUNT Operation Metadata

<Metric NAME="Load" TYPE="TABLE">
    <Display>
       <Label NLSID="wbem_cimom_load">Load</Label>
     </Display>
     <TableDescriptor>
       <ColumnDescriptor NAME="Active Clients" TYPE="NUMBER" IS_KEY="FALSE">
           <Display>
             <Label NLSID="wbem_cimom_active_clients">Active CIMOM Clients</Label>
           </Display>
         </ColumnDescriptor>
     </TableDescriptor>  
    <QueryDescriptor FETCHLET_ID="Wbem">
       <Property NAME="username" SCOPE="GLOBAL">guest</Property>
       <Property NAME="password" SCOPE="GLOBAL">guest</Property>
       <Property NAME="CIMClassname" SCOPE="GLOBAL">EX_SFLProvider</Property>
       <Property NAME="operation" SCOPE="GLOBAL">COUNT</Property> 
    </QueryDescriptor>
  </Metric>

The FETCHLET_ID is identified as Wbem. Property names are passed to the fetchlet for the required parameters user name, password, and CIMClassname. The operation is identified as COUNT.

The following example shows how to implement a Response Status metric to determine whether the CIMOM is running or not. It returns a value of 1 if the connection to the CIMOM is successful, otherwise 0.

Example 8-2 Response Status Metric

  <Metric NAME="Response" TYPE="TABLE">
     <Display>
       <Label NLSID="wbem_cimon_response">Response</Label>
     </Display>
     <TableDescriptor>
       <ColumnDescriptor NAME="Status" TYPE="NUMBER" IS_KEY="FALSE">
           <Display>
             <Label NLSID="wbem_cimom_response_status">Status</Label>
           </Display>
         </ColumnDescriptor> 
    </TableDescriptor> 
    <QueryDescriptor FETCHLET_ID="Wbem">
       <Property NAME="username" SCOPE="GLOBAL">guest</Property>
       <Property NAME="password" SCOPE="GLOBAL">guest</Property>
       <Property NAME="operation" SCOPE="GLOBAL">STATUS</Property>
     </QueryDescriptor>
   </Metric>

The default operation is the VALUES operation. It is used to fetch the values of a class that is defined in the CIMOM.

In the final example, the EX_Teacher class is accessed and fetches the name column. Name is the key of the class and of the new metric being defined, so the IS_KEY property is set to true. The CIM class properties will be mapped to the Enterprise Manager columns in the order that they are specified in the properties property. In this case, there is only 1 property - Name.

Example 8-3 Single Property Fetched for a Class

<Metric NAME="EX_Teacher" TYPE="TABLE">
     <Display>
       <Label NLSID="wbem_EX_Teacher">EX_Teacher Class</Label>
     </Display>
     <TableDescriptor>
       <ColumnDescriptor NAME="Name" TYPE="STRING" IS_KEY="TRUE">
           <Display>
             <Label NLSID="wbem_ex_teacher_name">Name</Label>
           </Display>
         </ColumnDescriptor>
     </TableDescriptor>
     <QueryDescriptor FETCHLET_ID="Wbem">
       <Property NAME="username" SCOPE="GLOBAL">guest</Property>
       <Property NAME="password" SCOPE="GLOBAL">guest</Property>
       <Property NAME="CIMClassname" SCOPE="GLOBAL">EX_Teacher</Property>
       <Property NAME="properties" SCOPE="GLOBAL">Name</Property>
     </QueryDescriptor>
   </Metric>

If multiple properties are fetched for a class, semi-colons should separate them. The properties should be provided in the order that the column descriptors are specified for the metric table definition.

Example 8-4 Multiple Properties Fetched for a Class

<Metric NAME="EX_SFLProvider" TYPE="TABLE">
     <Display>
       <Label NLSID="wbem_EX_SFLProvider">EX_SFLProvider Class</Label>
     </Display>
     <TableDescriptor>
       <ColumnDescriptor NAME="Name" TYPE="STRING" IS_KEY="TRUE">
           <Display>
             <Label NLSID="wbem_ex_sfl_name">Name</Label>
           </Display>
         </ColumnDescriptor>
       <ColumnDescriptor NAME="Win" TYPE="NUMBER" IS_KEY="FALSE">
           <Display>
             <Label NLSID="wbem_ex_sfl_win">Win</Label>
           </Display>
         </ColumnDescriptor>
       <ColumnDescriptor NAME="Lost" TYPE="NUMBER" IS_KEY="FALSE">
           <Display>
             <Label NLSID="wbem_ex_sfl_lost">Lost</Label>
           </Display>
         </ColumnDescriptor>
     </TableDescriptor>
     <QueryDescriptor FETCHLET_ID="Wbem">
       <Property NAME="username" SCOPE="GLOBAL">guest</Property>
       <Property NAME="password" SCOPE="GLOBAL">guest</Property>
       <Property NAME="CIMClassname" SCOPE="GLOBAL">EX_SFLProvider</Property>
       <Property NAME="properties" SCOPE="GLOBAL">Name;Win;Lost</Property>
     </QueryDescriptor>
   </Metric>

JDBC Fetchlet

Call-level interfaces such as JDBC permit external access to SQL database manipulation and update commands. The Java Database Connectivity (JDBC) fetchlet allows you to execute common JDBC commands and obtain their response time for any type of database.

Input Parameters

Table 8-15 JDBC Fetchlet Input Parameters

Name Description Use

Transaction Name

(Standard)

Required.

Beacon Name

(Standard)

Required.

Connect String

Connection string provided by the user. The Connect String must comply with the URL format specified by the vendor of the database to which the user is trying to connect.

Examples:

Format required by Oracle:

jdbc:oracle:thin:@hostname:port

Format required by MySQL:

jdbc:mysql://hostname:port

Required.

Class Name String

The driver class name to be used for connections.

Example:

oracle.jdbc.driver.OracleDriver

You have two options for configuring the Agent to use the .jar file containing the driver:

  1. Place the .jar file in $JAVA_HOME/jre/lib/ext. CLASSPATH does not need to be modified.

  2. Place the .jar file anywhere and update CLASSPATH in emd.properties file with the path to jar. Bounce Agent. This should be scripted and be transparent to user.

Required.

Username

User name to be used when connecting to the database.

Required.

Password

Password to be used when connecting to the database.

Required.

Role

User Role

Required.

Statement

SQL statement to be executed. Use of PL/SQL is possible by using prepareCall() API.

Required.


Table 8-16 Metric Columns Collected

Column Description

Status

Status of the test. Status is 'down' if there is a SQLException generated by the fetchlet.

Total Time

Time required for the fetchlet to execute the test.

Connect Time

Time required for DriverManager.getConnection() to complete.

Prepare Time

Time required for conn.prepareStatement() to complete.

Execute Time

Time required for stmt.executeQuery() to complete.

Fetch Time

Time required for while(rs.next()) { rs.getRow() } to complete.

Close Time

Time required for closing resultset, statement, connection to complete.

Number of rows

Number of rows fetched.

Total time per row

 

Fetch time per row

 

OJMX/SOAP Fetchlet

The OJMX fetchlet communicates with the JMX agent on the managed J2EE server and performs the specified operations.

Input Parameters

Table 8-17 JMX Fetchlet Input Parameters

Name Type Description

metricType

String

Tells the fetchlet that this metric is of type Web Service (GWS).

requestBodyElement

String

Provides the name of the Web Service operation as used in the Web Service request body.

documentType

String

Specifies SOAP encoding. For example: rpc/encoded or doc/literal.

soapVersion

String

Version of SOAP that the fetchlet should use to communicate with the Web Service (default is 1.2)

targetNamespace

String

Target namespace of Web Service.

ColumnOrder

String []

Comma separated list of XPaths to pick the column for a resultant Web Service response.

rowData

String []

Comma separated list of XPath prefixes which, when appended to corresponding columnOrder Xpaths, provide the value of the metric column for each row when arrays are returned in the response from a Web Service.

URI

String

URI of Web Service.

soapAction

String

SOAP action for web service from WSDL.

returnType

String

Type of web service invocation return value.

arguments

XML

Values to pass to the Web Service invocation.


Output

The OJMX fetchlet returns an object of type MetricResult that contains the information retrieved.