5 Enterprise Manager Integration Utility

This chapter describes how to get started with the Enterprise Manager Integration Utility and use it to integrate Enterprise Manager with Oracle Enterprise Repository.

This chapter contains the following sections:

5.1 Overview

Enterprise Manager (EM) is a comprehensive monitoring tool. It includes support for monitoring Oracle's SOA Products, as well as J2EE Servers, Databases, JVMs, and Operating Systems.

The EM Integration utility closes the loop between runtime and design-time by providing a summary of runtime performance metrics from EM into Oracle Enterprise Repository. The EM Integration utility pulls metrics from EM and pushes to Oracle Enterprise Repository. The EM Integration tool currently supports and works with EM Grid Control 10g R3 (10.2.0.5) and EM Grid Control 11g.

EM Grid control (EMGC) is capable of monitoring multiple versions of a product or target simultaneously for one EMGC instance, be it EMGC 10g or EMGC 11g. For example, Oracle Service Bus 10g and 11g can be simulataneously monitorerd by EMGC, and hence metrics for both these instances can be pulled back into a single Oracle Enterprise Repository instance, which has projects harvested from Oracle Service Bus 10g or 11g. So Oracle Enterprise Repository-EM Integration also supports gathering metrics for multiple product version targets into Oracle Enterprise Repository regardless of EM GC 10g or 11g.

This section contains the following topics:

5.1.1 Prerequisites

Before using the EM Integration utility, you must perform the following prerequisites:

  • The EM Integration utility requires the OER-Harvester datapack to be installed in Oracle Enterprise Repository.

  • The assets that receive metrics from EM must already be in Oracle Enterprise Repository. They should have been published to Oracle Enterprise Repository from SOA Suite/BPEL-PM or Oracle Service Bus using the Oracle Enterprise Repository Harvester. The EM Integration utility requires the OER-Harvester datapack to be installed in Oracle Enterprise Repository.

    For more information about importing, see Oracle Fusion Middleware Configuration Guide for Oracle Enterprise Repository.

  • The EM Integration utility needs read access to the Enterprise Manager shared database views.

    For more information about Enterprise Manager Database views, see http://download.oracle.com/docs/cd/B16240_01/doc/em.102/b40007/views.htm#BACCEIBI.

    To create a read-only user in Enterprise Manager (in cases where you do not have a SYSMAN repository owner privilege):

    1. Log in to Enterprise Manager Grid Control (EMGC) console as SYSMAN or any super administrator. The Oracle Enterprise Manager Fusion Middleware Control page is displayed.

    2. In the top-right corner, click Setup, and then select Administrators on the left pane and click Create.

    3. Enter the username and password and accept the defaults for the other fields, and then click Next.

    4. Remove the Public role that is listed in the table on the right hand side.

    5. Under Create Administrator: System Privileges, select VIEW ANY TARGET.

    6. Under Create Administrator: Targets, select all the targets that this new admin should be able to view.

    7. Under Create Administrator: Target Privileges , select BATCH ASSIGN and then click View.

      Note:

      In most cases, the View option might be selected and grayed out.
    8. Click Apply.

    9. Login to SQLPLUS with SYSMAN user to grant additional privileges to this new user, for example, oem_view.

      Grant SELECT on MGMT$METRIC_DAILY to oem_view;
      Grant SELECT on MGMT$METRIC_CURRENT to oem_view;
      Grant SELECT on MGMT$TARGET_PROPERTIES to oem_view;
      Grant SELECT on MGMT_VERSIONS to oem_view;
      Grant SELECT on MGMT_TARGET_PROPERTIES to oem_view;
      Grant SELECT on MGMT_TARGET_ASSOCS to oem_view;
      Grant SELECT on MGMT_TARGETS to oem_view;
      Grant SELECT on MGMT_TARGET_ASSOC_DEFS to oem_view;
      
  • The EM Integration utility requires Java SDK version 6 or higher.

  • Install 10g or 11g EM Grid Control to enable EM Integration tool to harvest metrics into Oracle Enterprise Repository. The Oracle Enterprise Repository EM Integration tool, currently, integrates with EM-Grid control and works with both 10g and 11g version of SOA.

  • When integrating with EM 11g, use the em-integration-settings.xml configuration file. However, when integrating with EM 10g, you have to use the em-integration-settings-em10.xml configuration file.

5.1.2 Obtaining the Enterprise Manager Integration Utility

To obtain the Enterprise Manager Integration utility:

  1. Navigate to this Oracle_HOME\repositoryXXX\core\tools\solutions directory on your computer. A list of all the solution packs for Oracle Enterprise Repository is displayed.

  2. Unzip the 11.1.1.x.x-EM-Integration.zip file to a temporary directory.

  3. Import the 11.1.1.x.x-OER-Harvester-Solution-Pack.zip file into Oracle Enterprise Repository using the Import/Export Tool. For more information, see Oracle Fusion Middleware User's Guide for Oracle Enterprise Repository.

5.1.3 High Level Use Cases

You can use the EM Integration utility to:

  • Publish metrics from Oracle Service Bus Services in EM to existing Oracle Enterprise Repository assets.

  • Publish metrics from BPEL-PM Processes in EM to existing Oracle Enterprise Repository assets.

  • Publish metrics from Web Services in EM to existing Oracle Enterprise Repository assets.

  • Publish Endpoint and Deployment : BPEL assets in Oracle Enterprise Repository from deployment information in EM.

  • Support links back to the detailed metrics in EM, from the Oracle Enterprise Repository Web UI.

  • Support configurability and extensibility, so that new targets in EM can be mapped to Oracle Enterprise Repository assets.

  • Support capturing metrics for Oracle SOA Suite composite targets in EM 11g. EM Integration allows you to publish metrics for composites that are invoking a service and are being monitored in EM 11g. However, unlike the other targets such as Oracle Service Bus, BPEL, WebLogic, if the service endpoint does not exist in Oracle Enterprise Repository, it will not be created during the integration process. If the assets do exist, they will be updated with the metrics.

5.2 Using the Enterprise Manager Integration Utility

This section describes how to use the Enterprise Manager Integration Utility.

This section contains the following topics:

5.2.1 Running from Command Line

The EM Integration utility can be run in the command line using the em-integration.bat utility (for Windows) or em-integration.sh (for Linux and Unix).

Before running em-integration.bat or em-integration.sh, ensure that the environment variables mentioned in Table 5-1 are set. In Windows, from a command window, you can type "set X" to see the value of the variable X, and "set X=abc" to set the value of FOO to "abc".

Table 5-1 Command Line Script

Environment Variable Description

JAVA_HOME

Ensure that the JAVA_HOME environment variable points to an installed java runtime (JRE) or SDK. This must be Java version 5 or higher.

JAVA_OPTS

Ensure that you set your JAVA_OPTS parameter as follows:

set JAVA_OPTS=-Dhttp.proxyPort=80 -Dhttp.proxyHost=www-proxy.us.oracle.com -Dhttp.nonProxyHosts= "*.oracle.com|localhost"

JAVA_OPTS refers to the extra options to the java executable. In normal cases, there is no must set this variable. However, a common exception when you must set this variable is when your computer is inside a firewall, and you must use an HTTP proxy to access external servers.

See Also: http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html


Table 5-2 shows the options that can be specified using the EM Integration command line utility:

Table 5-2 Command Line Options for the EM Integration

EM Integration Options Description

-settings <file>

Refers to the configuration settings specified in the XML file.

-er_url <URL>

Specifies the URL for the Oracle Enterprise Repository instance.

-er_user <User Name>

Specifies the name of the Oracle Enterprise Repository user.

-er_password <Password>

Specifies the password of the Oracle Enterprise Repository user.

To ensure security, the password must be encrypted.

The Oracle Enterprise Repository Web console has a tool to encrypt passwords: http://<host>:<port>/<domain>/diag/encryptstrings.jsp

-em_url <URL>

Specifies the JDBC URL for the EM database.

-em_user <User Name>

Specifies the EM database user name.

-em_password <Password>

Specifies the EM database password.

To ensure security, the password must be encrypted.

The Oracle Enterprise Repository Web console has a tool to encrypt passwords: http://<host>:<port>/<domain>/diag/encryptstrings.jsp

For more information about encrypting passwords from command-line, see Section 5.4, "Encrypting the Configuration File Passwords".

-help

Displays the online help for the EM Integration utility.


Note:

Note none of the above options are mandatory and these can be omitted. If the options are omitted, then the utility makes use of the information in the em-integration-settings.xml file in the current directory where em-integration.bat resides. If the command line parameters are provided, then these override the information in em-integration-settings.xml.

Figure 5-1 shows the command line utility options and online help displayed by the em-integration.bat -help command.

Figure 5-1 Enterprise Manager Integration Command-Line Utility Options

Description of Figure 5-1 follows
Description of "Figure 5-1 Enterprise Manager Integration Command-Line Utility Options "

5.2.2 Scheduling from Enterprise Manager

Enterprise Manager has the ability to schedule command-line applications to run periodically. The EM Integration utility can be scheduled to run this way.

For more information, see "Job System" in Oracle Enterprise Manager Administrator's Guide.

It is recommended that you schedule the EM Integration to run once a day, in the evening or when there is low activity on the system.

5.2.3 Metric Publishing

The section describes the various metric publishing options:

5.2.3.1 Oracle Service Bus

EM Integration updates existing assets in Oracle Enterprise Repository of type "Endpoint", which have an endpointURI attribute that matches the endpointURI metric in EM, for Oracle Service Bus Proxy Services and External Services.

Note:

An External Service is a service, which is invoked by an Oracle Service Bus Business Service.

If no "Endpoint" asset is found with the matching endpointURI, then the tool attempts to find a matching Service asset, by name and Oracle Service Bus service type (BusinessService or ProxyService). If it finds one, then it creates a new "Endpoint" asset and relates it to the service. In the case of Business Services, the endpoint is related to the External Service invoked by the Business Service.

If there are issues collecting metrics for the Oracle Service Bus target in EM, such as the metrics are not being collected or there are errors in EM Grid Control to collect the Oracle Service Bus service metrics, then see the following link for more details:

http://download.oracle.com/docs/cd/E11857_01/install.111/e17558/disc_mon_osb.htm

Note:

As mentioned in the link, the first time the metrics for Oracle Service Bus services is uploaded to the EM, the value is "0". And if no metric is collected, then turn the monitoring Flag on under Oracle Service Bus Console -> Global Settings.

Also, ensure that if there are any additional monitoring flags for each Proxy/Business Service in Oracle Service Bus (within the service Operational Settings), then enable that too to be able to gather metrics for all the services.

5.2.3.2 BPEL PM

EM Integration updates existing assets in Oracle Enterprise Repository of type "Deployment:BPEL", which has the "EntryPoint" relationship to an "Endpoint" asset that has an endpointURI attribute that matches the endpointURI metric in EM. EM stores the WSDL location of the endpoint.

If no "Endpoint" asset is found with the matching endpointURI, then the tool attempts to find a matching Service asset, by qualified name from the WSDL. If it finds one, then it creates a new "Endpoint" asset and relates it to the service.

If no "Deployment:BPEL" asset is found, then the tool attempts to find a maching Business Process: BPEL asset, by process name. If it finds one, then it creates a new "Deployment:BPEL" asset and relates it to the service.

5.2.3.3 Web Services

EM Integration updates existing assets in Oracle Enterprise Repository of type "Endpoint"", which have an endpointURI attribute that matches the endpointURI metric in EM.

If no "Endpoint" asset is found with the matching endpointURI, then the tool attempts to find a matching Service asset, by qualified name from the WSDL. If it finds one, then it creates a new "Endpoint" asset and relates it to the service.

5.2.3.4 Artifacts Creation

For every "Endpoint" that is created, the EM Integration Utility creates a corresponding Artifact:WSDL asset, if it does not already exist in Oracle Enterprise Repository. The EM Integration Utility also creates the Artifact:WSDL and Artifact:XSD assets for any imported file. It follows the fingerprinting rules in Oracle Enterprise Repository Harvester to look for existing artifacts in Oracle Enterprise Repository.

5.2.3.5 Endpoint Creation

It is common for production endpoints to be monitored in EM, but not captured in Oracle Enterprise Repository. The EM Integration utility creates these endpoints, if it can find a matching service.The EM Integration utility creates Endpoint assets, if it can find a matching Service and creates Deployment:BPEL assets, if it can find a matching Business Process:BPEL.

5.2.3.6 Metrics to Update

In its default configuration, EM Integration attempts to update the following metrics in Oracle Enterprise Repository. These are all visible in the Operational tab for the asset in the Oracle Enterprise Repository Web user interface.

Not all of these metrics are gathered in EM for all of the target types mentioned above, so some might be omitted. The em-integration-settings.xml file configures which metrics are gathered for which targets.

  • Daily Average Response Time

  • Weekly Average Response Time

  • Monthly Average Response Time

  • Minimum Response Time

  • Maximum Response Time

  • Daily Requests

  • Weekly Requests

  • Monthly Requests

  • Daily Faults

  • Weekly Faults

  • Monthly Faults

  • Availability - Daily %

  • Availability - Weekly %

  • Availability - Monthly %

  • Availability - Daily %

  • Current Rated Maximum Requests per Second

  • Start Date for Metrics Monitoring

  • Last Updated

  • Link back to detailed metrics in EM

5.3 Configuring the Enterprise Manager Integration Utility

This section describes how to configure the Enterprise Manager Integration Utility. It contains the following topics:

5.3.1 Setting the Repository and Enterprise Manager Connection Information for the Command-line Utility

Open the em-integration-settings.xml file, which is located at <EM Integration Home> and modify the following XML chunk to point to the Oracle Enterprise Repository instance, the EM database, and the web console with the right credentials.

The EM database is used to retrieve the metric information that is sent to Oracle Enterprise Repository. The EM console URL is used as the base URL for links back to EM, that are created within Oracle Enterprise Repository.

The introspectionSettings section is used to configure the properties of any Endpoint assets that the utility creates. The createMissingEndpoints attribute indicates whether Endpoint and Deployment:BPEL assets are created. If false, then the rest of the section in the code below is ignored.

The matchMultipleEndpoints attribute indicates the required action to be taken if the utility finds multiple Endpoints or Deployment:BPEL assets in Oracle Enterprise Repository that match a target in EM. If true, then it updates all the endpoints with metrics (and log a warning). If false, then it skips the target (and log a warning).

The matchMultipleServices attribute indicates the required action to be taken if the utility finds multiple Services or Business Process:BPEL assets in Oracle Enterprise Repository that match a target in EM, during endpoint creation. If true, then it attaches new Endpoints / Deployment:BPELs to each of the matching assets (and log a warning). If false, then it skips the target (and log a warning).

The introspectionDescription, introspectionVersion, and registrationStatus attributes indicate properties that are set on any assets created.

<repository>
  <uri>http://localhost:7131/oer30/services/FlashlineRegistry</uri>
  <credentials>
    <user>admin</user>
    <password>*****</password>
    <enableTransaction>false</enableTransaction>
    <triggerEvent>false</triggerEvent>
  </credentials>
  <timeout>120000</timeout>
</repository>
<introspectionSettings>
  <createMissingEndpoints>true</createMissingEndpoints>
   <introspectionDescription>Found in EM </introspectionDescription>
    <introspectionVersion>1.0</introspectionVersion>
    <namespace>mythings</namespace> 
    <!-- The Valid Registration states are 1)Unsubmitted 2)Submitted - Pending
 Review 3)Submitted - Under Review 4) Registered -->
    <registrationStatus>Submitted</registrationStatus>
  </introspectionSettings>
<emDatabase>
  <driverClass>oracle.jdbc.driver.OracleDriver</driverClass>
  <url>jdbc:oracle:thin@remotehost.there.com:1501:mysid</url>
  <credentials>
    <user>sysman</user>
    <password>*****</password>
  </credentials>
</emDatabase>

Note:

It is recommended that you run the Enterprise Manager Integration Utility as a user with the Basic Access Settings for Assets - View, Edit, Accept, and Register.

Alternatively, the connection information can also be passed as parameters to the command line utility as follows:

C:\test\em-integration> em-integration - er_url http://localhost:7101/oer/services/FlashlineRegistryTr -er_user admin - er_password <password> - em_url jdbc:oracle:thin:@remotehost.there.com:1501:mysid - em_user sysman - em_password <empassword>

5.3.2 Advanced Configuration

This section describes the following advanced configuration options for the Enterprise Manager Integration utility:

5.3.2.1 Metric Mappings

The mapping between metrics in EM and fields in Oracle Enterprise Repository are configured in em-integration-settings.xml, as shown in Figure 5-2. To change these from the default settings, open this file and modify the following:

  • <metricMappingGroup>: Mappings for a set of metrics with the same metric name in the EM database views. EM groups a set of related metrics under the same "metric name". Each individual metric has a different "metric column". This element has the following attribute:

    • metricName: The EM metric name from the EM database views.

  • <metricMapping>: Mapping for a single metric. This element has the following attributes:

    • metricColumn: The EM metric column from the EM database views.

    • rollupPeriod: Indicates which EM database view to query against, and how to aggregate the data. Must be one of {DAILY, WEEKLY, MONTHLY, CURRENT}.

    • viewColumn: Indicates which column in the EM database view to pull the data from. Must be one of {VALUE, AVERAGE, SUM, MINIMUM, MAXIMUM, STANDARD_DEVIATION, SAMPLE_COUNT, COLLECTION_TIMESTAMP, ROLLUP_TIMESTAMP, NOW(), DETAILS_URL}.

      • The CURRENT rollup period does not support columns AVERAGE, SUM, MINIMUM, MAXIMUM, STANDARD_DEVIATION, SAMPLE_COUNT, or ROLLUP_TIMESTAMP.

      • The DAILY, WEEKLY and MONTHLY rollup periods doe not support columns VALUE or COLLECTION_TIMESTAMP.

      • NOW() is a "virtual column" that contains the time of the query against EM.

      • DETAILS_URL is a "virtual column" that contains a URL to the detailed information in EM. This is constructed by the EM Integration code - it's not in the EM database views.

    • oerField: The internal name of the custom data field in Oracle Enterprise Repository where this data is stored.

    • overwrite: Indicates whether to overwrite the custom data field in Oracle Enterprise Repository if it already has data. It represents either true or false, by default the value is true, if not specified.

Figure 5-2 Metric Mappings

Description of Figure 5-2 follows
Description of "Figure 5-2 Metric Mappings"

5.3.2.2 Target Finders

The logic for correlating targets in EM with assets in Oracle Enterprise Repository is different for each target type. EM Integration ships with target finders for the following products: Oracle Service Bus, BPEL-PM, WebLogic.

If you are an advanced user, then you can create your own target finder classes. These must be configured in em-integration-settings.xml, as shown in Figure 5-3.

  • <targetFinder>: Mappings for a target type in EM to a target finder class. This element has the following attributes

    • targetType: The EM target type from the EM database views.

    • targetFinderClass: The fully-qualified classname of the finder class. This must implement com.oracle.oer.integration.em.TargetFinder, which is found in em-integration.jar.

Figure 5-3 Target Finders

Description of Figure 5-3 follows
Description of "Figure 5-3 Target Finders"

5.3.2.3 Logging

EM Integration uses log4j for logging the detailed tasks performed and the log file is placed in the <EM Integration Home>. The logging options can be changed by updating the log4fl.properties file located in the <EM Integration> folder.

5.4 Encrypting the Configuration File Passwords

To ensure security, the passwords in the configuration file must be encrypted before running the Enterprise Manager Integration utility. You can encrypt the configuration file passwords in the EM using either of the following methods:

From the Command Prompt

  1. Navigate to the <em_integration_home> directory.

  2. From a command prompt, run the password encryption tool as follows:

    > encrypt.bat em-integration-settings.xml em-integration-settings.xml

    The password encryption tool, (encrypt.bat/encrypt.sh), which is located in <Oracle_home>/tools/solutions/11.1.1.x.x-OER-PasswordTools.zip, allows you to encrypt the passwords that are stored in the configuration file, em-integration-settings.xml.

From the Oracle Enterprise Repository Web Console

  1. Navigate to the Oracle Enterprise Repository Web console at

    http://<host>:<port>/<domain>/diag/encryptstrings.jsp (replace host, port, and domain to match to your computer credentials)

  2. Scroll down to the Tools section and click the Encrypt Strings for passwords link to launch the Password encryption page.

  3. Enter the clear text password into the String to Encrypt text box.

  4. Click the Submit Query button.

  5. Copy the resulting encrypted password string and paste it into the appropriate context or properties file(s).

5.5 Known Issues

This section describes the following known Enterprise Manager Integration Utility issues:

5.5.1 Using Incorrect Encrypted Password

If you configure the em-integration-settings.xml file with incorrect encrypted password, then a very long stack trace is displayed. An extract from the very long stack trace is as follows:

        at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unk
own Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentConten
Dispatcher.dispatch(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(U
known Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
        at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
        at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
        at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
        at org.apache.axis.encoding.DeserializationContext.parse(Deserializatio
Context.java:227)
        at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696)
        at org.apache.axis.Message.getSOAPEnvelope(Message.java:424)
        at org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnder
tandChecker.java:62)
        at org.apache.axis.client.AxisClient.invoke(AxisClient.java:206)
        at org.apache.axis.client.Call.invokeEngine(Call.java:2765)
        at org.apache.axis.client.Call.invoke(Call.java:2748)
        at org.apache.axis.client.Call.invoke(Call.java:2424)
        at org.apache.axis.client.Call.invoke(Call.java:2347)
        at org.apache.axis.client.Call.invoke(Call.java:1804)
        at com.flashline.registry.openapi.service.v300.FlashlineRegistryTrSoapB
ndingStub.authTokenCreateWithLicense(FlashlineRegistryTrSoapBindingStub.java:74
1)
        at com.oracle.oer.sync.plugin.writer.oer.ALERConnectionCache.getAuthTok
n(ALERConnectionCache.java:128)
        at com.oracle.oer.sync.plugin.writer.oer.ALERAssetQueries.getToken(ALER
ssetQueries.java:82)
        at com.oracle.oer.sync.plugin.writer.oer.ALERAssetQueries.assetTypeQuer
ByUUID(ALERAssetQueries.java:159)
        at com.oracle.oer.sync.framework.MetadataManager.putAssetType(MetadataM
nager.java:204)
        at com.oracle.oer.sync.framework.impl.DefaultPluginManager.processIntro
pector(DefaultPluginManager.java:104)
        ... 5 more