Configuring Diagnostic Archives

7 Configuring Diagnostic Archives

The Archive component captures and persists all data events, log records, and metrics collected by the WebLogic Diagnostics Framework (WLDF) from server instances and applications running on them. You can subsequently access archived diagnostic data in online mode (that is, on a running server), or in off-line mode using the WebLogic Scripting Tool (WLST).

This chapter explains how to configure the Archive, and also how to configure WLDF to archive diagnostic data to a file store or a Java Database Connectivity (JDBC) data source:

You can also specify when and under what conditions old data will be removed from the archive, as described in Retiring Data from the Archives.

Configuring the Archive
You can configure the diagnostic archive on a per-server basis. The configuration is persisted in the config.xml file for a domain, under the <server-diagnostic-config> element for the server.
Configuring a File-Based Store
WLDF supports the ability to use a file-based store for the Archive. If you choose the use of a file-based store, the only configuration option you must set is the location of the directory where the store is to be maintained. The default directory is DOMAIN_HOME/servers/SERVER_NAME/data/store/diagnostics.
Configuring a JDBC-Based Store
WLDF supports the ability to create the Archive in a JDBC-based store.
Retiring Data from the Archives
To maintain the archived data, you must delete the old archived data periodically. WLDF includes a configuration-based data retirement feature for doing this. The data can be deleted based on the size of the data and time period when it was created.

Configuring the Archive

You can configure the diagnostic archive on a per-server basis. The configuration is persisted in the config.xml file for a domain, under the <server-diagnostic-config> element for the server.

Example configurations for file-based stores and JDBC-based stores are shown in Example 7-1 and Example 7-7.

Note:

Resetting the system clock while diagnostic data is being written to the archive can produce unexpected results. See Resetting the System Clock Can Affect How Data Is Archived and Retrieved.

Parent topic: Configuring Diagnostic Archives

Configuring a File-Based Store

WLDF supports the ability to use a file-based store for the Archive. If you choose the use of a file-based store, the only configuration option you must set is the location of the directory where the store is to be maintained. The default directory is DOMAIN_HOME/servers/SERVER_NAME/data/store/diagnostics.

When you save to a file-based store, WLDF uses the WebLogic Server persistent store. See Using the WebLogic Persistent Store in Administering the WebLogic Persistent Store.

An example configuration for a file-based store is shown in Example 7-1.

Example 7-1 Sample Configuration for File-based Diagnostic Archive (in config.xml)

<domain>
  <!-- Other domain configuration elements -->
  <server>
    <name>myserver</name>
    <server-diagnostic-config>
      <diagnostic-store-dir>data/store/diagnostics</diagnostic-store-dir>
      <diagnostic-data-archive-type>FileStoreArchive
      </diagnostic-data-archive-type>
    </server-diagnostic-config>
  </server>
  <!-- Other server configurations in this domain -->
</domain>

Parent topic: Configuring Diagnostic Archives

Configuring a JDBC-Based Store

WLDF supports the ability to create the Archive in a JDBC-based store.To use a JDBC store, the appropriate tables must exist in a database, and JDBC must be configured to connect to that database. For information about how to configure JDBC using the WebLogic Server Administration Console, see Configure database connectivity in Oracle WebLogic Server Administration Console Online Help. For additional information about JDBC configuration, see Administering JDBC Data Sources for Oracle WebLogic Server.

Note:

If you install multiple WLDF schemas in the same database, you need to provide a way to distinguish among them when accessing the diagnostic archives. You can do this when you configure the diagnostic archive for a server instance by specifying the schema name to use for accessing JDBC-based archive tables in that database. See Configuring JDBC Resources for WLDF.

Parent topic: Configuring Diagnostic Archives

Creating WLDF Tables in the Database

If they do not already exist, you must create the database tables used by WLDF to store data in a JDBC-based store. Two tables are required:

The wls_events table stores data generated from WLDF Instrumentation events.
The wls_hvst table stores data generated from the WLDF Harvester component.

The SQL Data Definition Language (DDL) used to create tables may differ for different databases, depending on the SQL variation supported by the database.

Parent topic: Configuring a JDBC-Based Store

Apache Derby

Example 7-2 shows the DDL that you can use to create the wls_events and wls_hvst tables in Apache Derby.

Example 7-2 DDL Definition of the WLDF Tables for Apache Derby

-- WLDF Instrumentation and Harvester archive DDLs using Derby
 
AUTOCOMMIT OFF;
 
-- DDL for creating wls_events table for instrumentation events
 
DROP TABLE wls_events;
 
CREATE TABLE wls_events (
  RECORDID INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1),
  TIMESTAMP BIGINT default NULL,
  CONTEXTID varchar(128) default NULL,
  TXID varchar(32) default NULL,
  USERID varchar(32) default NULL,
  TYPE varchar(64) default NULL,
  DOMAIN varchar(64) default NULL,
  SERVER varchar(64) default NULL,
  SCOPE varchar(64) default NULL,
  MODULE varchar(64) default NULL,
  MONITOR varchar(64) default NULL,
  FILENAME varchar(64) default NULL,
  LINENUM INTEGER default NULL,
  CLASSNAME varchar(250) default NULL,
  METHODNAME varchar(64) default NULL,
  METHODDSC varchar(4000) default NULL,
  ARGUMENTS clob(100000) default NULL,
  RETVAL varchar(4000) default NULL,
  PAYLOAD blob(100000),
  CTXPAYLOAD VARCHAR(4000),
  DYES BIGINT default NULL,
  THREADNAME varchar(250) default NULL
);
 
-- DDL for creating wls_events table for instrumentation events
 
DROP TABLE wls_hvst;
 
CREATE TABLE wls_hvst (
  RECORDID INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1),
  TIMESTAMP BIGINT default NULL,
  DOMAIN varchar(64) default NULL,
  SERVER varchar(64) default NULL,
  TYPE varchar(64) default NULL,
  NAME varchar(250) default NULL,
  ATTRNAME varchar(64) default NULL,
  ATTRTYPE INTEGER default NULL,
  ATTRVALUE VARCHAR(4000),
  WLDFMODULE VARCHAR(250) default NULL
);
 
COMMIT;

Consult the documentation for your database or your database administrator for specific instructions for creating these tables for your database.

Parent topic: Creating WLDF Tables in the Database

Oracle Database

Example 7-3 shows the DDL that you can use to create the wls_events table in Oracle database.

Example 7-3 DDL Definition of the wls_events Table for Oracle Database

SET SERVEROUTPUT ON;
 
DECLARE
 vCtr     Number;
 vSQL     VARCHAR2(2000);
 vcurr VARCHAR2(256);
BEGIN
 
  SELECT sys_context( 'userenv', 'current_schema' ) into vcurrSchema from dual;
  dbms_output.put_line('Current Schema: '||vcurrSchema);

  SELECT COUNT(*)
  INTO vCtr
  FROM user_tables
  WHERE table_name = 'WLS_EVENTS';
 
  IF vCtr = 0 THEN
    dbms_output.put_line('Creating WLS_EVENTS table');
    vSQL := 'CREATE TABLE "WLS_EVENTS" (
    "RECORDID" NUMBER(20,0) DEFAULT NULL, 
    "TIMESTAMP" NUMBER(20,0) DEFAULT NULL, 
    "CONTEXTID" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "TXID" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "USERID" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "TYPE" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "DOMAIN" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "SERVER" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "SCOPE" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "MODULE" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "MONITOR" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "FILENAME" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "LINENUM" NUMBER(10,0) DEFAULT NULL, 
    "CLASSNAME" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "METHODNAME" VARCHAR2(250 BYTE) DEFAULT NULL, 
    "METHODDSC" VARCHAR2(4000 BYTE) DEFAULT NULL, 
    "ARGUMENTS" CLOB DEFAULT NULL, 
    "RETVAL" VARCHAR2(4000 BYTE) DEFAULT NULL, 
    "PAYLOAD" BLOB DEFAULT NULL, 
    "CTXPAYLOAD" VARCHAR2(4000 BYTE) DEFAULT NULL, 
    "DYES" NUMBER(20,0) DEFAULT NULL, 
    "THREADNAME" VARCHAR2(250 BYTE) DEFAULT NULL
   )';
   EXECUTE IMMEDIATE vSQL;
   vSQL := 'CREATE UNIQUE INDEX WLS_EVENTS_RECORD_IDX ON WLS_EVENTS(RECORDID)';
   EXECUTE IMMEDIATE vSQL;
   vSQL := 'CREATE INDEX WLS_EVENTS_TS_IDX ON WLS_EVENTS(TIMESTAMP)';
   EXECUTE IMMEDIATE vSQL;
  END IF;
  
  SELECT COUNT(*)
  INTO vCtr
  FROM user_tab_columns
  WHERE table_name = 'WLS_EVENTS' AND column_name = 'THREADNAME';
  
  IF vCtr = 0 THEN
    dbms_output.put_line('Creating THREADNAME column in WLS_EVENTS table');
    vSQL := 'ALTER TABLE WLS_EVENTS ADD("THREADNAME" VARCHAR2(250 BYTE) DEFAULT NULL)';
    EXECUTE IMMEDIATE vSQL;  
  END IF;
 
  SELECT COUNT(*) INTO vCtr FROM user_sequences
  WHERE sequence_name = 'SEQ_WLS_EVENTS_RECORDID';
  
  IF vCtr = 0 THEN
    vSQL := 'CREATE SEQUENCE SEQ_WLS_EVENTS_RECORDID MINVALUE 1 MAXVALUE 99999999999999999999 START WITH 1 INCREMENT BY 1 NOCACHE';
    EXECUTE IMMEDIATE vSQL;
  END IF;
 
  SELECT COUNT(*) INTO vCtr FROM user_triggers
  WHERE table_name = 'WLS_EVENTS';
 
  IF vCtr = 0 THEN  
    vSQL := 'CREATE OR REPLACE TRIGGER TRG_WLS_EVENTS_INSERT 
    BEFORE INSERT ON WLS_EVENTS 
    REFERENCING NEW AS newRow 
    FOR EACH ROW 
    BEGIN 
      IF :newRow.RECORDID IS NULL THEN 
        SELECT SEQ_WLS_EVENTS_RECORDID.nextval INTO :newRow.RECORDID FROM DUAL; 
      END IF; 
    END;';
    EXECUTE IMMEDIATE vSQL;    
  END IF;  
  
END;
/

Example 7-4 shows the DDL that you can use to create the wls_hvst table in Oracle database.

Example 7-4 DDL Definition of the wls_hvst Table for Oracle Database

SET SERVEROUTPUT ON;
 
DECLARE
 vCtr     Number;
 vSQL     VARCHAR2(1000); 
 vcurrSchema VARCHAR2(256);
BEGIN
 
  SELECT sys_context( 'userenv', 'current_schema' ) into vcurrSchema from dual;
  dbms_output.put_line('Current Schema: '||vcurrSchema);
 
  SELECT COUNT(*)  
    INTO vCtr  
    FROM user_tables  
    WHERE table_name = 'WLS_HVST';
 
  IF vCtr = 0 THEN
    dbms_output.put_line('Creating WLS_HVST table');
    vSQL := 'CREATE TABLE "WLS_HVST"
    (	
      "RECORDID" NUMBER(20,0) NOT NULL, 
      "TIMESTAMP" NUMBER(20,0) DEFAULT NULL, 
      "DOMAIN" VARCHAR2(250 BYTE) DEFAULT NULL, 
      "SERVER" VARCHAR2(250 BYTE) DEFAULT NULL, 
      "TYPE" VARCHAR2(250 BYTE) DEFAULT NULL, 
      "NAME" VARCHAR2(250 BYTE) DEFAULT NULL, 
      "ATTRNAME" VARCHAR2(250 BYTE) DEFAULT NULL, 
      "ATTRTYPE" NUMBER(10,0) DEFAULT NULL, 
      "ATTRVALUE" VARCHAR2(4000 BYTE) DEFAULT NULL, 
      "WLDFMODULE" VARCHAR2(250 BYTE) DEFAULT NULL
    )';  
   EXECUTE IMMEDIATE vSQL;   
   vSQL := 'CREATE UNIQUE INDEX WLS_HVST_RECORD_IDX ON WLS_HVST(RECORDID)';
   EXECUTE IMMEDIATE vSQL;
   vSQL := 'CREATE INDEX WLS_HVST_TS_IDX ON WLS_HVST(TIMESTAMP)';
   EXECUTE IMMEDIATE vSQL;
  END IF;
 
  SELECT COUNT(*)
    INTO vCtr FROM user_tab_columns
    WHERE table_name = 'WLS_HVST' AND column_name = 'WLDFMODULE';
  
  IF vCtr = 0 THEN
    dbms_output.put_line('Creating WLDFMODULE column in WLS_HVST table');
    vSQL := 'ALTER TABLE WLS_HVST ADD("WLDFMODULE" VARCHAR2(250 BYTE) DEFAULT NULL)';
    EXECUTE IMMEDIATE vSQL;  
  END IF;
  
  SELECT COUNT(*) INTO vCtr FROM user_sequences
  WHERE sequence_name = 'SEQ_WLS_HVST_RECORDID';
  
  IF vCtr = 0 THEN
    vSQL := 'CREATE SEQUENCE SEQ_WLS_HVST_RECORDID MINVALUE 1 MAXVALUE 99999999999999999999 START WITH 1 INCREMENT BY 1 NOCACHE';
    EXECUTE IMMEDIATE vSQL;
  END IF;
 
  SELECT COUNT(*) INTO vCtr FROM user_triggers
  WHERE table_name = 'WLS_HVST';
 
  IF vCtr = 0 THEN  
    vSQL := 'CREATE OR REPLACE TRIGGER TRG_WLS_HVST_INSERT 
    BEFORE INSERT ON WLS_HVST 
    REFERENCING NEW AS newRow 
    FOR EACH ROW 
    BEGIN 
      IF :newRow.RECORDID IS NULL THEN 
        SELECT SEQ_WLS_HVST_RECORDID.nextval INTO :newRow.RECORDID FROM DUAL; 
      END IF; 
    END;';
    EXECUTE IMMEDIATE vSQL;    
  END IF;  
    
END;
/

Consult the documentation for your database or your database administrator for specific instructions for creating these tables for your database.

Parent topic: Creating WLDF Tables in the Database

MySQL

Example 7-5 shows the DDL that you can use to create the wls_events table in MySQL database.

Example 7-5 DDL Definition of the wls_events Table in MySql Database

DROP PROCEDURE if exists create_alter_wls_events
/
 
CREATE PROCEDURE create_alter_wls_events() 
language sql
BEGIN
  CREATE TABLE IF NOT EXISTS WLS_EVENTS
  (
    RECORDID BIGINT AUTO_INCREMENT PRIMARY KEY,
    TIMESTAMP BIGINT NOT NULL,
    CONTEXTID VARCHAR(250) default NULL,
    TXID VARCHAR(250) default NULL,
    USERID VARCHAR(250) default NULL,
    TYPE VARCHAR(250) default NULL,
    DOMAIN VARCHAR(250) default NULL,
    SERVER VARCHAR(250) default NULL,
    SCOPE VARCHAR(250) default NULL,
    MODULE VARCHAR(250) default NULL,
    MONITOR VARCHAR(250) default NULL,
    FILENAME VARCHAR(250) default NULL,
    LINENUM INT UNSIGNED default NULL,
    CLASSNAME VARCHAR(250) default NULL,
    METHODNAME VARCHAR(250) default NULL,
    METHODDSC VARCHAR(4000) default NULL,
    ARGUMENTS TEXT(100000) default NULL,
    RETVAL VARCHAR(4000) default NULL,
    PAYLOAD BLOB(100000),
    CTXPAYLOAD VARCHAR(4000),
    DYES BIGINT UNSIGNED default NULL,
    THREADNAME VARCHAR(250) default NULL,
    INDEX(TIMESTAMP)
  );
 
  IF NOT EXISTS(
    SELECT * FROM `information_schema`.`COLUMNS`
      WHERE COLUMN_NAME='THREADNAME' AND TABLE_NAME='WLS_EVENTS') THEN 
      ALTER TABLE `WLS_EVENTS` ADD `THREADNAME` varchar(250) default NULL;
  END IF;
 
END
/
 
CALL create_alter_wls_events()
/
 
DROP PROCEDURE if exists create_alter_wls_events
/

Example 7-6 shows the DDL that you can use to create the wls_hvst table in MySQL database.

Example 7-6 DDL Definition of wls_hvst Table in MySql Database

DROP PROCEDURE if exists create_alter_wls_hvst
/
 
CREATE PROCEDURE create_alter_wls_hvst() 
language sql
BEGIN
  CREATE TABLE IF NOT EXISTS WLS_HVST
  (
    RECORDID BIGINT AUTO_INCREMENT PRIMARY KEY,
    TIMESTAMP BIGINT NOT NULL,
    DOMAIN VARCHAR(250) default NULL,
    SERVER VARCHAR(250) default NULL,
    TYPE VARCHAR(250) default NULL,
    NAME VARCHAR(250) default NULL,
    SCOPE VARCHAR(250) default NULL,
    ATTRNAME VARCHAR(250) default NULL,
    ATTRTYPE INT default NULL,
    ATTRVALUE VARCHAR(4000) default NULL,
    WLDFMODULE VARCHAR(250) default NULL,
    INDEX(TIMESTAMP)
  );
 
  IF NOT EXISTS(
    SELECT * FROM `information_schema`.`COLUMNS`
      WHERE COLUMN_NAME='WLDFMODULE' AND TABLE_NAME='WLS_HVST') THEN 
      ALTER TABLE `WLS_HVST` ADD `WLDFMODULE` varchar(250) default NULL;
  END IF;
 
END
/
 
CALL create_alter_wls_hvst()
/
 
DROP PROCEDURE if exists create_alter_wls_hvst
/

Consult the documentation for your database or your database administrator for specific instructions for creating these tables for your database.

Parent topic: Creating WLDF Tables in the Database

Configuring JDBC Resources for WLDF

After you create the tables in your database, you must configure JDBC to access the tables. (See Administering JDBC Data Sources for Oracle WebLogic Server.) Then, as part of your server configuration, you specify that JDBC resource as the data store to be used for a server's archive.

If multiple WLDF JDBC archive schemas exist in the same database, you can specify the particular schema to use for accessing JDBC-based archive tables in that database. There is no default value for a schema name: If you do not specify one, no schema name is applied when WLDF validates the runtime table, and no schema name is used for the SQL statements. You specify the schema name in the WLDFServerDiagnosticMBean.DiagnosticJDBCSchemaName attribute, which you can access from the Diagnostic Archives: Configuration page in the WebLogic Server Administration Console. See Configure diagnostic archives in Oracle WebLogic Server Administration Console Online Help.

An example configuration for a JDBC-based store is shown in Example 7-7.

Example 7-7 Sample configuration for JDBC-based Diagnostic Archive (in config.xml)

<domain>
  <!-- Other domain configuration elements -->
  <server>
    <name>myserver</name>
    <server-diagnostic-config>
      <diagnostic-data-archive-type>JDBCArchive
      </diagnostic-data-archive-type>
      <diagnostic-jdbc-resource>JDBCResource</diagnostic-jdbc-resource>
    <server-diagnostic-config>
  </server>
  <!-- Other server configurations in this domain -->
</domain>

If you specify a JDBC resource but it is configured incorrectly, or if the required tables do not exist in the database, WLDF uses the default file-based persistent store.

Parent topic: Configuring a JDBC-Based Store

Retiring Data from the Archives

To maintain the archived data, you must delete the old archived data periodically. WLDF includes a configuration-based data retirement feature for doing this. The data can be deleted based on the size of the data and time period when it was created.

You can configure size-based data retirement at the server level and age-based retirement at the individual archive level, as described in the following sections:

Parent topic: Configuring Diagnostic Archives

Configuring Data Retirement at the Server Level

You can set the following data retirement options for a server instance:

The preferred maximum size of the server instance's data store (<preferred-store-size-limit>) and the interval at which it is checked, on the hour, to see if it exceeds that size (<store-size-check-period>).

When the size of the store is found to exceed the preferred maximum, an appropriate number of the oldest records in the store are deleted to reduce the size below the specified threshold. This is called "size-based data retirement."

Note:

Size-based data retirement can be used only for file-based stores. These options are ignored for database-based stores.
Enable or disable data retirement for the server instance.

For file-based diagnostic stores, this enables or disables the size-based data retirement options discussed above. For both file-based stores and database-based stores, this also enables or disables any age-based data retirement policies defined for individual archives in the store. See Configuring Age-Based Data Retirement Policies for Diagnostic Archives.

Parent topic: Retiring Data from the Archives

Configuring Age-Based Data Retirement Policies for Diagnostic Archives

The data store for a server instance can contain the following types of diagnostic data archives whose records can be retired using the data retirement feature:

Harvested metrics data (logical name: HarvestedDataArchive)
Instrumentation events data (logical name: EventsDataArchive)
Custom data (user-defined name)

Note:

WebLogic Server log files are maintained both at the server level and the domain level. Data is retired from the current log using the log rotation feature. See Configuring WebLogic Logging Services in Configuring Log Files and Filtering Log Messages for Oracle WebLogic Server.

Age-based policies apply to individual archives. The data store for a server instance can have one age-based policy for the HarvestedDataArchive, one for the EventsDataArchive, and one each for any custom archives.

When records in an archive exceed the age limit specified for records in that archive, those records are deleted.

Parent topic: Retiring Data from the Archives

Sample Configuration

Data retirement configuration settings are persisted in the config.xml configuration file for the server's domain, as shown in Example 7-8.

Example 7-8 Data Retirement Configuration Settings in config.xml

<domain>
<!-- other domain configuration settings -->
   <server>
   <name>MedRecServer</name>
   <!-- other server configuration settings -->
   <server-diagnostic-config>
      <diagnostic-store-dir>data/store/diagnostics</diagnostic-store-dir>
      <diagnostic-data-archive-type>FileStoreArchive
         </diagnostic-data-archive-type>
      <data-retirement-enabled>true</data-retirement-enabled>
      <preferred-store-size-limit>120</preferred-store-size-limit>
      <store-size-check-period>1</store-size-check-period>
      <wldf-data-retirement-by-age>
         <name>HarvestedDataRetirementPolicy</name>
         <enabled>true</enabled>
         <archive-name>HarvestedDataArchive</archive-name>
         <retirement-time>1</retirement-time>
         <retirement-period>24</retirement-period>
         <retirement-age>45</retirement-age>
      </wldf-data-retirement-by-age>
      <wldf-data-retirement-by-age>
         <name>EventsDataRetirementPolicy</name>
         <enabled>true</enabled>
         <archive-name>EventsDataArchive</archive-name>
         <retirement-time>10</retirement-time>
         <retirement-period>24</retirement-period>
         <retirement-age>72</retirement-age>
      </wldf-data-retirement-by-age>
   </server-diagnostic-config>
   </server>
</domain>

Parent topic: Retiring Data from the Archives