13 Accessing Diagnostic Data With the Data Accessor

This chapter describes the Data Accessor in WebLogic Server 10.3.6 and describes how to use it online (when a server is running) and offline (when a server is not running).

You use the Data Accessor component of the WebLogic Diagnostics Framework (WLDF) to access diagnostic data from various sources, including log records, data events, and harvested metrics.

Using the Data Accessor, you can perform data lookups by type, component, and attribute. You can perform time-based filtering and, when accessing events, filtering by severity, source, and content. You can also access diagnostic data in tabular form.

This chapter includes the following sections:

Data Stores Accessed by the Data Accessor

The Data Accessor retrieves diagnostic information from other WLDF components. Captured information is segregated into logical data stores that are separated by the types of diagnostic data. For example, server logs, HTTP logs, and harvested metrics are captured in separate data stores.

WLDF maintains diagnostic data on a per-server basis. Therefore, the Data Accessor provides access to data stores for individual servers.

Data stores can be modeled as tabular data. Each record in the table represents one item, and the columns describe characteristics of the item. Different data stores may have different columns. However, most data stores have some of the same columns, such as the time when the data was collected.

The Data Accessor can retrieve the following information about data stores used by WLDF for a server:

  • A list of supported data store types, including:

    • HTTP_LOG

    • HARVESTED_DATA_ARCHIVE

    • EVENTS_DATA_ARCHIVE

    • SERVER_LOG

    • DOMAIN_LOG

    • HTTP_ACCESS_LOG

    • WEBAPP_LOG

    • CONNECTOR_LOG

    • JMS_MESSAGE_LOG

    • CUSTOM_LOG

  • A list of available data store instances

  • The layout of each data store (information that describes the columns in the data store)

You can use the WLDFAccessRuntimeMBean to discover such data stores, determine the nature of the data they contain, and access their data selectively using a query.

For complete documentation about WebLogic logs, see Configuring Log Files and Filtering Log Messages for Oracle WebLogic Server.

Accessing Diagnostic Data Online

You access diagnostic data from a running server by using the Administration Console, JMX APIs, or the WebLogic Scripting Tool (WLST).

Accessing Data Using the Administration Console

You do not use the Data Accessor explicitly in the Administration Console, but information collected by the Accessor is displayed, for example, in the Summary of Log Files page. See "View and Configure Logs" in the Oracle WebLogic Server Administration Console Help.

Accessing Data Programmatically Using Runtime MBeans

The Data Accessor provides the following runtime MBeans for discovering data stores and retrieving data from them:

  • Use the WLDFAccessRuntimeMBean to do the following:

    • Get the logical names of the available data stores on the server.

    • Look up a WLDFDataAccessRuntimeMBean to access the data from a specific data source, based on its logical name. The different data stores are uniquely identified by their logical names.

    See "WLDFAccessRuntimeMBean" in the Oracle WebLogic Server MBean Reference.

  • Use the WLDFDataAccessRuntimeMBean to retrieve data stores based on a search condition, or query. You can optionally specify a time interval with the query, to retrieve data records within a specified time duration. This MBean provides meta-data about the columns of the data set and the earliest and latest timestamp of the records in the data store.

    Data Accessor runtime Mbeans are currently created and registered lazily. So, when a remote client attempts to access them, they may not be present and an InstanceNotFoundException may be thrown.

    The client can retrieve the WLDFDataAccessRuntime's attribute of the WLDFAccessRuntime to cause all known data access runtimes to be created, for example:

      ObjectName objName = 
          new ObjectName("com.bea:ServerRuntime=" + serverName + 
                          ",Name=Accessor," +
                          "Type=WLDFAccessRuntime," +
                          "WLDFRuntime=WLDFRuntime"); 
      rmbs.getAttribute(objName, "WLDFDataAccessRuntimes");
    

    See "WLDFDataAccessRuntimeMBean" in the Oracle WebLogic Server MBean Reference.

Using WLST to Access Diagnostic Data Online

Use the WLST exportDiagnosticDataFromServer command to access diagnostic data from a running server. For the syntax and examples of this command, see "Diagnostics Commands" in the WebLogic Scripting Tool Command Reference.

Using the WLDF Query Language with the Data Accessor

To query data from data stores, use the WLDF query language. For Data Accessor query language syntax, see Appendix A, "WLDF Query Language."

Accessing Diagnostic Data Offline

Use the WLST exportDiagnosticData command to access historical diagnostic data from an offline server. For the syntax and examples of this command, see "Diagnostics Commands" in the WebLogic Scripting Tool Command Reference.

Note:

You can use exportDiagnosticData to access archived data only from the machine on which the data is persisted.

You cannot discover data store instances using the offline mode of the Data Accessor. You must already know what they are.

Accessing Diagnostic Data Programmatically

Example 13-1 shows the source Java code for a utility that uses the Accessor to query the different archive data stores.

Example 13-1 Sample Code to Use the WLDF Accessor

/*
 * WLAccessor.java
 *
 * Demonstration utility that allows query of the different ARCV data stores
 * via the WLDF Accessor.
 *
 */
 
 import javax.naming.Context;
 import weblogic.jndi.Environment;
 import java.util.Hashtable;
 import java.util.Iterator;
 import java.util.Properties;
 import weblogic.management.ManagementException;
 import weblogic.management.runtime.WLDFAccessRuntimeMBean;
 import weblogic.management.runtime.WLDFDataAccessRuntimeMBean;
 import weblogic.diagnostics.accessor.ColumnInfo;
 import weblogic.diagnostics.accessor.DataRecord;
 import java.io.File;
 import java.io.FileInputStream;
 import java.io.FileNotFoundException;
 import java.io.IOException;
 
 import javax.management.MBeanServerConnection;
 import javax.management.remote.JMXConnector;
 import javax.management.remote.JMXConnectorFactory;
 import javax.management.remote.JMXServiceURL;
 import javax.management.ObjectName;
 import weblogic.management.mbeanservers.runtime.RuntimeServiceMBean;
 import weblogic.management.runtime.ServerRuntimeMBean;
 import weblogic.management.jmx.MBeanServerInvocationHandler;
 import weblogic.management.configuration.ServerMBean;
 
 /**
  * Demonstration utility that allows query of the different ARCV data stores
  * via the WLDF Accessor. The class looks up the appropriate accessor and
  * executes the query given the specified query parameters.
  *
  * To see information about it's usage, compile this file and run
  *
  *   java WLAccessor usage
  */
public class WLAccessor {
  
  /** Creates a new instance of WLAccessor */
  public WLAccessor(Properties p) {
  initialize(p);
  }
 
  /**
  * Retrieve the specfied WLDFDataAccessRuntimeMBean instance for querying.
  */
  public WLDFDataAccessRuntimeMBean getAccessor(String accessorType)
  throws Throwable
  {
  // Get the runtime MBeanServerConnection
  MBeanServerConnection runtimeMBS = this.getRuntimeMBeanServerConnection();
 
  // Lookup the runtime service for the connected server
  ObjectName rtSvcObjName = new ObjectName(RuntimeServiceMBean.OBJECT_NAME);
  RuntimeServiceMBean rtService = null;
 
    rtService = (RuntimeServiceMBean) 
      MBeanServerInvocationHandler.newProxyInstance(
        runtimeMBS, rtSvcObjName
        );

    // Walk the Runtime tree to the desired accessor instance.
    ServerRuntimeMBean srt = rtService.getServerRuntime();

    WLDFDataAccessRuntimeMBean ddar = 
      srt.getWLDFRuntime().getWLDFAccessRuntime().
      lookupWLDFDataAccessRuntime(accessorType);

    return ddar;
  }
 
 /**
   * Execute the query using the given parameters, and display the formatted
   * records.
   */
  public void queryEventData() throws Throwable
  {
    String logicalName = "EventsDataArchive";
    WLDFDataAccessRuntimeMBean accessor = getAccessor(accessorType);

    ColumnInfo[] colinfo = accessor.getColumns();
    inform("Query string: " + queryString);

    int recordsFound = 0;
    Iterator actualIt = 
      accessor.retrieveDataRecords(beginTime, endTime, queryString);
    while (actualIt.hasNext()) {
      DataRecord rec = (DataRecord)actualIt.next();
      inform("Record[" + recordsFound + "]: {");
      Object[] values = rec.getValues();
      for (int colno=0; colno < values.length; colno++) {
        inform("[" + colno + "]  "
               + colinfo[colno].getColumnName() + 
               " (" +  colinfo[colno].getColumnTypeName() + "):   " + 
                values[colno]);
      }
      inform("}");
      inform("");
      recordsFound++;
    }
    inform("Found " + recordsFound + " results");
  }

  /**
   * Main method that implements the tool.
   * @param args the command line arguments
   */
  public static void main(String[] args) {
    try {
      WLAccessor acsr = new WLAccessor(handleArgs(args));
      acsr.queryEventData();
    } catch (UsageException uex) {
      usage();
    } catch (Throwable t) {
      inform("Caught exception, " + t.getMessage(), t);
      inform("");
      usage();
    }
  }

  public static class UsageException extends Exception {}

  /**
   * Process the command line arguments, which are provided as name/value pairs.
  */
  public static Properties handleArgs(String[] args) throws Exception
  {
    Properties p = checkForDefaults();
    for (int i = 0; i < args.length; i++) {
      if (args[i].equalsIgnoreCase("usage"))
        throw new UsageException();

      String[] nvpair = new String[2];
      int token = args[i].indexOf('=');
      if (token < 0) 
        throw new Exception("Invalid argument, " + args[i]);
      nvpair[0] = args[i].substring(0,token);
      nvpair[1] = args[i].substring(token+1);
      p.put(nvpair[0], nvpair[1]);
    }
    return p;
  }
  
  /**
   * Look for a default properties file
   */
  public static Properties checkForDefaults() throws IOException {
    Properties defaults = new Properties();
    try {
      File defaultprops = new File("accessor-defaults.properties");
      FileInputStream defaultsIS = new FileInputStream(defaultprops);
      //inform("loading options from accessor-defaults.properties");
      defaults.load(defaultsIS);
      } catch (FileNotFoundException fnfex) {
      //inform("No accessor-defaults.properties found");
    } 
    return defaults;
  }
  public static void inform(String s) {
    System.out.println(s);
  }
  public static void inform(String s, Throwable t) {
    System.out.println(s);
    t.printStackTrace();
  }

  private MBeanServerConnection getRuntimeMBeanServerConnection() 
    throws IOException
  {
    // construct jmx service url
    
    // "service:jmx:[url]/jndi/[mbeanserver-jndi-name]"
    JMXServiceURL serviceURL =
      new JMXServiceURL(
        "service:jmx:" + getServerUrl() + 
        "/jndi/" + RuntimeServiceMBean.MBEANSERVER_JNDI_NAME
        );

    // specify the user and pwd. Also specify weblogic provide package
    inform("user name [" + username + "]");
    inform("password [" + password + "]");
    Hashtable h = new Hashtable();
    h.put(Context.SECURITY_PRINCIPAL, username);
    h.put(Context.SECURITY_CREDENTIALS, password);
    h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES,
          "weblogic.management.remote");
    // get jmx connector 
    JMXConnector connector = JMXConnectorFactory.connect(serviceURL, h);

    inform("Using JMX Connector to connect to " + serviceURL);
    return connector.getMBeanServerConnection();
  }

  private void initialize(Properties p) {
    serverUrl = p.getProperty("url","t3://localhost:7001");
    username = p.getProperty("user","weblogic");
    password = p.getProperty("pass","weblogic");
    queryString = p.getProperty("query","SEVERITY IN
('Error','Warning','Critical','Emergency')");
    accessorType = p.getProperty("type","ServerLog");

    try {
      beginTime = Long.parseLong(p.getProperty("begin","0"));

      String end = p.getProperty("end");
      endTime = (end==null) ? Long.MAX_VALUE : Long.parseLong(end);
    } catch (NumberFormatException nfex) {
      throw new RuntimeException("Error formatting time bounds", nfex);
    }
  }

  private static void usage() {
    inform("");
    inform("");
    inform("Usage: ");
    inform("");
    inform("  java WLAccessor [options]");
    inform("");
    inform("where [options] can be any combination of the following: ");
    inform("");
    inform("    usage                   Prints this text and exits");
    inform("    url=<url>               default: 't3://localhost:7001'");
    inform("    user=<username>         default: 'weblogic'");
    inform("    pass=<password>         default: 'weblogic'");
    inform("    begin=<begin-timestamp> default: 0");
    inform("    end=<end-timestamp>     default: Long.MAX_VALUE");
    inform("    query=<query-string>    default: \"SEVERITY IN ('Error','Warning','Critical','Emergency')\"");
    inform("    type=<accessor-type>    default: 'ServerLog'");
    inform("");
    inform("Example:");
    inform("");
    inform("    java WLAccessor user=system pass=gumby1234 url=http://myhost:8000 \\");
    inform("       query=\"SEVERITY = 'Error'\" begin=1088011734496 type=ServerLog");
    inform("");
    inform("");
    inform("");
    inform("All properties (except \"usage\") can all be specified in a file ");
    inform("in the current working directory. The file must be named: ");
    inform("");
    inform("           \"accessor-defaults.properties\"");
    inform("");
    inform("Each property specified in the defaults file can still be ");
    inform("overridden on the command-line as shown above");
    inform("");
  }

  /** Getter for property serverUrl.
   * @return Value of property serverUrl.
   *
   */
  public java.lang.String getServerUrl() {
    return serverUrl;
  }
  
  /** Setter for property serverUrl.
   * @param serverUrl New value of property serverUrl.
   *
   */
  public void setServerUrl(java.lang.String serverUrl) {
    this.serverUrl = serverUrl;
  }
  
  protected String serverName = null;
  protected String username = null;
  protected String password = null;
  protected String queryString = "";
  private String serverUrl = "t3://localhost:7001";
  private String accessorType = null;

  private long endTime = Long.MAX_VALUE;
  private long beginTime = 0;

  private WLDFAccessRuntimeMBean dar = null;

}

Resetting the System Clock Can Affect How Data Is Archived and Retrieved

Resetting the system clock to an earlier time while diagnostic data is being written to the WLDF Archive or logs can cause unexpected results when you query that data based on a timestamp. For example, consider the following sequence of events:

  1. At 2:00 p.m., a diagnostic event is archived as RECORD_200, with a timestamp of 2:00:00 PM.

  2. At 2:30 p.m., a diagnostic event is archived as RECORD_230, with a timestamp of 2:30:00 PM.

  3. At 3:00 p.m., the system clock is reset to 2:00 p.m.

  4. At 2:15 p.m. (after the clock was reset), a diagnostic event is archived as RECORD_215, with a timestamp of 2:15:00 PM.

  5. You issue a query to retrieve records generated between 2:00 and 2:20 p.m.

The query will not retrieve RECORD_215, because the 2:30:00 PM timestamp of RECORD_230 ends the query.