Skip Headers
Oracle® Enterprise Manager Getting Started with Oracle Fusion Middleware Management
11g Release 1 (11.1.0.1)

E17558-05
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

18 Exporting Data

This chapter includes the following export features:

Data Export Modes

There are three different modes to export performance data collected by ADP to external databases and other persistence formats. These modes give you flexibility to choose the best way to extract performance data from ADP.

Export to File

In this mode, ADP exports its raw performance data as several CSV (comma separated value) files.

Export to Database

In this mode, ADP exports its raw performance data as several ANSI SQL statements. These SQL statements allow you to create tables and insert data.

Aggregation Export to File

In this mode, ADP exports its aggregated performance data after it's daily aggregation operation as several CSV files.

ADP Export Configuration

The following sections describe ADP export configuration:

ADP Periodic Export Configuration

ADP stores real-time performance metrics in its internal data repository (Oracle database). If you want to store this data in your historical data repository, ADP provides automatic means for performance data export. You can control the frequency of export runs, the time when the export should run, and the time range of export data within a day.

Example 18-1 ADP Export Configuration

# Setting for integrated export
AggregationManager.IntegratedExport = false
AggregationManager.ExportDataStartHour = 0
AggregationManager.ExportDataEndHour = 0
AggregationManager.ExportDataSetRangeInHour = 4
AggregationManager.ExportDataSetIntervalInHour = 1
AggregationManager.ExportDataSetDelay = 10000
AggregationManager.ExportStartTime = 0
AggregationManager.ExportEndTime = 0
AggregationManager.ExportFilePurgeTime = 10d

By default, automatic data export feature is disabled. To enable it, set AggregationManager.IntegratedExport parameter to true. A pair of parameters, AggregationManager.ExportDataStartHour and AggregationManager.ExportDataEndHour, indicates the time range within a day of the export performance data in which you are interested. By default it is 24 hours.

Parameter AggregationManager.ExportDataSetRangeInHour shows how much data is stored in each export file. By default it is 4 hours worth of data. This means that ADP will create multiple export data files with 4 hours worth of data.

To minimize export query impact on normal ADP performance data collection functionality, spread out lengthy data exporting queries. This is achieved by setting AggregationManager.ExportDataSetIntervalInHour parameter. By default it is 1 hour. This means that export query thread will be running every hour.

AggregationManager.ExportDataSetDelay defines cool down time for consecutive export queries. By default it is 10 seconds. This means that the next export query will happen not earlier than 10 seconds after the previous one.

AggregationManager.ExportFilePurgeTime indicates how many days the export data file will be available before ADP deletes it. By default it is 10 days. Current default values are optimal and this section should not be changed except from enabling the automatic export feature.

Automatic export function relies on the definition of the data to be exported by looking into $CAMM_HOME/config/export.xml file and exports performance data based on the rules defined in this file.

The output directory is specified in export.xml and will be overwritten on each export interval.

Manual Execution of Metric Export

The bin directory on the ADP manager contains scripts called runExportMetric.sh/bat and runExportEvent.sh/bat that export metrics and events (such as alerts) to CSV files, respectively.

Run runExportMetric.sh like:

./runExportMetric.sh <path to export.xml configuration> <start time> <end time>

For example:

C:\oracle\em11g\bin\runExportMetric.bat c:\oracle\em11g\config\export.xml 
"4/1/09 16:06:00" "4/1/09 16:36:00"

The start time and end time are in the machine's local time zone. The output exported csv files' timestamps will be in UTC/GMT.

export.xml File

The export.xml file contains all the directives and filters required to export performance metrics and events. This file is used by the ADP Integrated Automatic export feature as well as manual export scripts.

Example 18-2 Contents of export.xml File

<?xml version="1.0" encoding="UTF-8"?>
<export xmlns="http://www.acsera.com/ns/export" verbose="true" exportMetric="true"
exportEvent="true" metricDataGrain="180s" exportFullMetric="true">
<!--
<output type="jdbc" convertTimeFormat="true"
arguments="access,metric,sun.jdbc.odbc.JdbcOdbcDriver,jdbc:odbc:acsera"/>
-->
<output type="file" convertTimeFormat="false"
arguments="/home/acsera/acsera/export,metric"/>
<entityTypes exportAllTypes="false">
<entityType name="BEA.ProcessNode"/>
<entityType name="J2EE.Dispatcher"/>
<entityType name="J2EE.JDBC.ConnectionPool"/>
<entityType name="J2EE.JVM"/>
</entityTypes>
<!--extra filter -->
<!--
<filters>
<filter key="containerID" values="cgServer"/>
</filters>
-->
<!-- don't modify this -->
<columns>
<column header="Timestamp" type="Timestamp"/>
<column header="EntityID" type="EntityID"/>
<column header="Application" type="Entity" key="applicationID" default=""/>
</columns>
</export>

The following tables explain the various attributes.

Table 18-1 Attributes on <export>

Attribute Description

exportMetric

true/false, whether to export performance metrics

exportEvent

true/false, whether to export SLO events

metricDataGrain

60s, 180s, or 1800s, the metric aggregation tier to export. Note that the population of the 180s tier will be delayed by 1.25 hours and the 1800s tier will be delayed by 7 hours compared to the 60s tier.

exportFullMetric

true/false, whether to include sum, count, min, and max metrics or not.


Table 18-2 Attributes on <output>

Attribute Description

type

file/jdbc, whether to output to csv files or to write data to another database using JDBC. In the case of JDBC output, the necessary tables will be created automatically by the export mechanism.

convertTimeFormat

true/false, whether to convert the metric timestamp to human readable format (in UTC/GMT). If false, the timestamp will be a long integer. Provide the arguments (jdbc) in a comma separated list.

arguments

See Table 18-3 and Table 18-4 for details.


Table 18-3 Value of the "arguments" Attribute of the <output> Element When the "type" Attribute Is "JDBC"

Attribute Description

First Parameter

Database type, an arbitrary string

Second Parameter

Table prefix name, should always be metric

Third Parameter

Fully qualified JDBC driver class

Fourth Parameter

JDBC URL


Table 18-4 Value of the "arguments" Attribute of the <output> Element When the "type" Is "file"

Argument Description

First Parameter

CSV file output directory (this directory will be created, existing files will be overwritten)

Second Parameter

Table prefix name, should always be metric


Table 18-5 Attribute of the <entityTypes> Element

Attribute Description

exportAllTypes

true/false, whether to output all entity types or only the ones specified in <entityType> elements.


Table 18-6 Attribute of the <entityType> Element

Attribute Description

name

Name of the entityType to include in the export if exportAllTypes is set to false


Example of Exported Data for WebLogic

The tables in this section describe the fields in the various export files.

Table 18-7 Export File Name: metricBEA_ChannelInstance.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Fully qualified name of the channel

channelID

Fully qualified name of the channel

serviceID

URL of the service / JPD

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

containerID

Name of the Oracle WebLogic Server instance

nodeID

Name of the physical machine

domainID

Name of the Oracle WebLogic domain

displayNameID

Display name

resourceID

Name of the monitored resource as configured by the user

entityTypeID

Type of the monitored entity

Metric.J2EE.ChannelInstance.MessageCount

JMX metric

Metric.J2EE.ChannelInstance.DeadMessageCount

JMX metric


Table 18-8 Export File Name: metricBEA_ProcessType.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Implementation class name

processID

Display name of the process

serviceID

URL of the service / JPD

projectID

Name of Workshop project / web application module

containerID

Name of the Oracle WebLogic Server instance

nodeID

Name of the physical machine

domainID

Name of the Oracle WebLogic domain

entityTypeID

Type of the monitored entity

applicationID

Name of the Application

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

deploymentID

Unique ID used by Oracle WebLogic to track application deployments

resourceID

Name of the monitored resource as configured by the user

displayNameID

Display name

controlContainerID

Implementation class name of the process

Metric.J2EE.ProcessType.Arrivals

Instrumentation metric -- number of arrivals

Metric.J2EE.ProcessType.Aborts

Instrumentation metric -- number of aborts

Metric.J2EE.ProcessType.ElapsedTime

Instrumentation metric -- average elapsed time

Metric.J2EE.ProcessType.Active

Instrumentation metric -- number of active requests

Metric.J2EE.ProcessType.VisitCount

Instrumentation metric -- number of completed requests

Metric.J2EE.ProcessType.Exceptions

Instrumentation metric -- number of exceptions


Table 18-9 Export File Name: metricBEA_TimerEventGenerator.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Fully qualified name of the channel

channelID

Fully qualified name of the channel

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

channelTxID

Fully qualified name of the channel

domainID

Name of the Oracle WebLogic domain

displayNameID

Display name

resourceID

Name of the monitored resource as configured by the user

entityTypeID

Type of the monitored entity

Metric.J2EE.TimerEventGenerator.MessageCount

JMX metric

Metric.J2EE.TimerEventGenerator.ErrorCount

JMX metric


Table 18-10 Export File Name: metricJ2EE_Dispatcher.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Fully qualified name of the Execute Queue

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

containerID

Name of the Oracle WebLogic Server instance

nodeID

Name of the physical machine

domainID

Name of the Oracle WebLogic domain

executeQueueID

Name of the Execute Queue as configured by user

displayNameID

Display name

resourceID

Name of the monitored resource as configured by the user

entityTypeID

Type of the monitored entity

Metric.J2EE.Dispatcher.ServicedRequestsTotalCount

JMX metric

Metric.J2EE.Dispatcher.IdleThreads

JMX metric

Metric.J2EE.Dispatcher.PendingRequests

JMX metric


Table 18-11 Export File Name: metricJ2EE_EJB_Entity.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

methodID

Name of the EJB method executed

domainID

Name of the Oracle WebLogic domain

entityTypeID

Type of the monitored entity

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

ejbID

Name of the EJB

webApplicationID

Name of the web module

displayNameID

Display name

controlContainerTypeID

Identifies type of control

elementID

Implementation class name

processID

Display name of the process

serviceID

URL of the service / JPD

projectID

Name of Workshop project / web application module

containerID

Name of the Oracle WebLogic Server instance

nodeID

Name of the physical machine

ejbComponentID

Name of Java EE component that contains this EJB

applicationID

Name of the Application

resourceID

Name of the monitored resource as configured by the user

controlContainerID

Implementation class name of the process

Metric.J2EE.EJB.Entity.Locking.LockManagerAccessCount

JMX metric

Metric.J2EE.EJB.Entity.ResponseTime

Instrumentation metric -- response time

Metric.J2EE.EJB.Entity.Cache.BeansCurrentCount

JMX metric

Metric.J2EE.EJB.Entity.Cache.AccessCount

JMX metric

Metric.J2EE.EJB.Entity.Pool.WaiterCurrentCount

JMX metric

Metric.J2EE.EJB.Entity.Transaction.CommittedTotalCount

JMX metric

Metric.J2EE.EJB.Entity.Locking.WaiterTotalCount

JMX metric

Metric.J2EE.EJB.Entity.Transaction.TimedOutTotalCount

JMX metric

Metric.J2EE.EJB.Entity.Cache.HitCount

JMX metric

Metric.J2EE.EJB.Entity.Locking.WaiterCurrentCount

JMX metric

Metric.J2EE.EJB.Entity.Pool.IdleCount

JMX metric

Metric.J2EE.EJB.Entity.Locking.EntriesCurrentCount

JMX metric

Metric.J2EE.EJB.Entity.VisitCount

Instrumentation metric -- invocation count

Metric.J2EE.EJB.Entity.Locking.TimeoutTotalCount

JMX metric

Metric.J2EE.EJB.Entity.Pool.InUseCount

JMX metric

Metric.J2EE.EJB.Entity.Pool.WaiterTotalCount

JMX metric

Metric.J2EE.EJB.Entity.Pool.TimeoutTotalCount

JMX metric

Metric.J2EE.EJB.Entity.Transaction.RolledBackTotalCount

JMX metric

Metric.J2EE.EJB.Entity.Cache.ActivationCount

JMX metric

Metric.J2EE.EJB.Entity.Cache.PassivationCount

JMX metric


Table 18-12 Export File Name: metricJ2EE_EJB_Stateless.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Implementation class name

projectID

Name of Workshop project / web application module

nodeID

Name of the physical machine

containerID

Name of the Oracle WebLogic Server instance

domainID

Name of the Oracle WebLogic domain

ejbComponentID

Name of Java EE component that contains this EJB

entityTypeID

Type of the monitored entity

applicationID

Name of the Application

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

ejbID

Name of the EJB

resourceID

Name of the monitored resource as configured by the user

displayNameID

Display name

Metric.J2EE.EJB.Stateless.Transaction.TimedOutTotalCount

JMX metric

Metric.J2EE.EJB.Stateless.Pool.WaiterTotalCount

JMX metric

Metric.J2EE.EJB.Stateless.Pool.InUseCount

JMX metric

Metric.J2EE.EJB.Stateless.Transaction.CommittedTotalCount

JMX metric

Metric.J2EE.EJB.Stateless.Transaction.RolledBackTotalCount

JMX metric

Metric.J2EE.EJB.Stateless.Pool.IdleCount

JMX metric

Metric.J2EE.EJB.Stateless.Pool.TimeoutTotalCount

JMX metric


Table 18-13 Export File Name: metricJ2EE_JDBC_ConnectionPool.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Name of JDBC connection pool

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

containerID

Name of the Oracle WebLogic Server instance

nodeID

Name of the physical machine

domainID

Name of the Oracle WebLogic domain

displayNameID

Display name

resourceID

Name of the monitored resource as configured by the user

entityTypeID

Type of the monitored entity

Metric.J2EE.JDBC.ConnectionPool.WaitingForConnectionCurrentCount

JMX metric

Metric.J2EE.JDBC.ConnectionPool.WaitingForConnectionHighCount

JMX metric

Metric.J2EE.JDBC.ConnectionPool.ActiveConnectionsHighCount

JMX metric

Metric.J2EE.JDBC.ConnectionPool.ActiveConnectionsCurrentCount

JMX metric

Metric.J2EE.JDBC.ConnectionPool.FailuresToReconnectCount

JMX metric

Metric.J2EE.JDBC.ConnectionPool.WaitSecondsHighCount

JMX metric

Metric.J2EE.JDBC.ConnectionPool.ConnectionDelayTime

JMX metric


Table 18-14 Export File Name: metricJ2EE_JMS_Destination.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Name of JMS destination

nodeID

Name of the physical machine

containerID

Name of the Oracle WebLogic Server instance

jmsServerRuntimeID

Name of the JMS server

domainID

Name of the Oracle WebLogic domain

jmsDistributedQueueMemberID

Name of the JMS distributed queue member

entityTypeID

Type of the monitored entity

jmsQueueID

Name of the JMS queue

jmsRuntimeID

Name of the JMS service

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

jmsDistributedQueueID

Name of the JMS distributed queue

resourceID

Name of the monitored resource as configured by the user.

displayNameID

Display name

Metric.J2EE.JMS.Destination.ConsumersCurrentCount

JMX metric

Metric.J2EE.JMS.Destination.BytesCurrentCount

JMX metric

Metric.J2EE.JMS.Destination.MessagesPendingCount

JMX metric

Metric.J2EE.JMS.Destination.BytesThresholdTime

JMX metric

Metric.J2EE.JMS.Destination.MessagesHighCount

JMX metric

Metric.J2EE.JMS.Destination.BytesReceivedCount

JMX metric

Metric.J2EE.JMS.Destination.MessagesReceivedCount

JMX metric

Metric.J2EE.JMS.Destination.BytesHighCount

JMX metric

Metric.J2EE.JMS.Destination.MessagesCurrentCount

JMX metric

Metric.J2EE.JMS.Destination.ConsumersTotalCount

JMX metric

Metric.J2EE.JMS.Destination.ConsumersHighCount

JMX metric

Metric.J2EE.JMS.Destination.BytesPendingCount

JMX metric


Table 18-15 Export File Name: metricJ2EE_JMS_Service.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Name of the JMS service

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

containerID

Name of the Oracle WebLogic Server instance

nodeID

Name of the physical machine

domainID

Name of the Oracle WebLogic domain

displayNameID

Display name

resourceID

Name of the monitored resource as configured by the user

entityTypeID

Type of the monitored entity

jmsRuntimeID

Name of the JMS service

Metric.J2EE.JMS.Service.ConnectionsHighCount

JMX metric

Metric.J2EE.JMS.Service.ConnectionsCurrentCount

JMX metric

Metric.J2EE.JMS.Service.JMSServersCurrentCount

JMX metric

Metric.J2EE.JMS.Service.JMSServersHighCount

JMX metric

Metric.J2EE.JMS.Service.ConnectionsTotalCount

JMX metric

Metric.J2EE.JMS.Service.JMSServersTotalCount

JMX metric


Table 18-16 Export File Name: metricJ2EE_JVM.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Name of the JVM

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

containerID

Name of the Oracle WebLogic Server instance

nodeID

Name of the physical machine

domainID

Name of the Oracle WebLogic domain

displayNameID

Display name

resourceID

Name of the monitored resource as configured by the user

entityTypeID

Type of the monitored entity

Metric.J2EE.JVM.JRockit.HeapSizeCurrent

JMX metric

Metric.J2EE.JVM.JRockit.HeapFreeCurrent

JMX metric

Metric.J2EE.JVM.JRockit.PhysMemTotal

JMX metric

Metric.J2EE.JVM.JRockit.PhysMemUsed

JMX metric

Metric.J2EE.JVM.JRockit.GarbageCollectionCountTotal

JMX metric

Metric.J2EE.JVM.JRockit.GarbageCollectionTimeTotal

JMX metric

Metric.J2EE.JVM.HeapFreeCurrent

JMX metric

Metric.J2EE.JVM.JRockit.PhysMemFree

JMX metric

Metric.J2EE.JVM.JRockit.NursurySizeTotal

JMX metric

Metric.J2EE.JVM.JRockit.ActiveDaemonThreads

JMX metric

Metric.J2EE.JVM.JRockit.ActiveThreads

JMX metric

Metric.J2EE.JVM.HeapSizeCurrent

JMX metric

Metric.J2EE.JVM.JRockit.HeapUsedCurrent

JMX metric


Table 18-17 Export File Name: metricJ2EE_Server.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Name of the Java EE server instance

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

containerID

Name of the Java EE server instance

nodeID

Name of the physical machine

domainID

Name of the Oracle WebLogic domain

displayNameID

Display name

resourceID

Name of the monitored resource as configured by the user

entityTypeID

Type of the monitored entity

Metric.J2EE.Server.RestartsTotalCount

JMX metric


Table 18-18 Export File Name: metricJ2EE_Servlet.csv

Field Description

StartTime

Clock time (long) at data insertion

EntityID

ADP unique identifier for the monitored entity

elementID

Name of the servlet implementation class

applicationID

Name of the Application

infrastructureID

ID of the monitoring infrastructure. Oracle is the only value at this time.

containerID

Name of the Oracle WebLogic Server instance

nodeID

Name of the physical machine

domainID

Name of the Oracle WebLogic domain

servletID

Name of the servlet

webApplicationID

Name of the web module

displayNameID

Display name

resourceID

Name of the monitored resource as configured by the user

entityTypeID

Type of the monitored entity

Metric.J2EE.Servlet.InvocationTotalCount

JMX metric

Metric.J2EE.Servlet.ExecutionTimeAverage

JMX metric