7 Managing Usage Tracking

This chapter describes how to manage usage tracking for Oracle Business Intelligence. The Oracle BI Server supports the collection of usage tracking data. When usage tracking is enabled, the Oracle BI Server collects usage tracking data for each query and inserts it directly to a database table.

Note:

The Oracle BI Summary Advisor feature works in conjunction with the usage tracking feature. Summary Advisor only works with direct insertion usage tracking.

Oracle BI Summary Advisor is only available when you are running Oracle Business Intelligence on the Oracle Exalytics Machine. See Using Oracle BI Summary Advisor to Identify Query Candidates for Aggregation in Metadata Repository Builder's Guide for Oracle Business Intelligence Enterprise Edition.

This chapter includes the following sections:

About Usage Tracking

The Oracle BI Server supports the accumulation of usage tracking statistics that can be used in a variety of ways such as database optimization, aggregation strategies, or billing users or departments based on the resources that they consume.

The BI Server tracks usage at the detailed query level.

When you enable usage tracking, statistics for every query are inserted into a database table or are written to a usage tracking log file. If you use direct insertion, then the BI Server directly inserts the usage tracking data into a relational database table. It is recommended that you use direct insertion to write statistics to a database table.

When the BI Server starts, it validates the column names in the metadata against the list of valid columns in the usage tracking table. The following events occur:

Column names. If there is a mismatch between the columns in the database table and the columns in the metadata, then it results in a database error on insert.
Varchar length. If the length in the metadata and the set length in the table do not match, then an error is written to the obis1-diagnostic.log file and usage tracking is disabled.

Setting Up Usage Tracking

Specify the database, schema, and connection pool details for usage tracking.

To set up usage tracking:

In the Oracle database, create a schema for usage tracking, and assign all privileges to that schema.
Add the Oracle database, connection pool, and the schema to be used for usage tracking to the Oracle BI Repository (RPD).
Upload the modified RPD file.
Restart BI Server.
In Oracle BI Answers, select RPD subject area, then and run queries.

This creates the usage tracking tables for logical and physical queries, and adds the entries for usage tracking.

Setting Up Direct Insertion to Collect Information for Usage Tracking

Direct insertion is the recommended method for setting up usage tracking.

This section describes how to set up direct insertion, and contains the following topics:

Setting Up the Usage Tracking Statistics Database

Before you can use direct insertion usage tracking, you must set up a database to store the usage tracking statistics.

You must run the Repository Creation Utility (RCU) on the target database to create the required statistics schemas.

Typically, you use the database you installed for use with Oracle Business Intelligence as the statistics database because this database already has the RCU-created schemas. The RCU-created table names for usage tracking are S_NQ_ACCT, S_NQ_DB_ACCT, and S_NQ_INITBLOCK. See Description of the Usage Tracking Data.

You also need to import the database into the Physical layer of the Oracle BI repository.

Run the Repository Creation Utility on an external database of your choice. You can skip this step if you choose to use the database you installed for use with Oracle Business Intelligence for usage tracking statistics, because this database has the RCU-created tables already.
Open the Administration Tool and import the database into the Physical layer. See Metadata Repository Builder's Guide for Oracle Business Intelligence Enterprise Edition.
Save and close the repository.

Setting Direct Insertion Parameters

You can set specific parameters for direct insertion on any new installation.

To set up direct insertion for new (non-upgraded) installations, use a text editor.

To set up direct insertion usage tracking use the steps below.

On the Oracle BI Server computer, open the NQSConfig.INI file in a text editor. You can find this file at:

BI_DOMAIN/config/fmwconfig/biconfig/OBIS

Make a backup copy of the file before editing.
In the [USAGE_TRACKING] section, update the following parameters:
- Set ENABLE to YES.
- Set DIRECT_INSERT to YES.
- Set PHYSICAL_TABLE_NAME to the name of the fully-qualified database table for collecting query statistic information, as it appears in the Physical layer of the Oracle BI repository. For example:
```
PHYSICAL_TABLE_NAME = "My_DB"."DEV_BIPLATFORM"."S_NQ_ACCT";
```
- Set CONNECTION_POOL to the name of the fully-qualified connection pool for the query statistics database, as it appears in the Physical layer of the Oracle BI repository. For example:
```
CONNECTION_POOL = "My_DB"."Usage Connection Pool";
```
- Set INIT_BLOCK_TABLE_NAME to the name of the fully-qualified database table for inserting records that correspond to the initialization block statistics, as it appears in the Physical layer of the Oracle BI repository. For example:
```
INIT_BLOCK_TABLE_NAME = "My_DB"."DEV_BIPLATFORM"."S_NQ_INITBLOCK";
```
- Set INIT_BLOCK_CONNECTION_POOL to the name of the fully-qualified connection pool for the table for inserting records that correspond to the initialization block statistics, as it appears in the Physical layer of the Oracle BI repository. For example:
```
INIT_BLOCK_CONNECTION_POOL = "My_DB"."Usage Connection Pool";
```
Note:
For Usage Tracking insertions to succeed, the connection pool must be configured with a user ID that has write access to the back-end database. Also, it is recommended that the connectivity type supports international data.
Save and close the file.
Restart the Oracle BI Server.

Setting Optional Direct Insert Parameters

The Usage Tracking section of the NQSConfig.INI file has several parameters.

In addition to the setup parameters described previously, you can also update the following optional parameters in the Usage Tracking section of the NQSConfig.INI file:

BUFFER_SIZE. This parameter indicates how much memory the BI Server allocates for buffering the insert statements. Such a buffer lets the BI Server submit multiple insert statements as part of a single transaction, improving Usage Tracking insert throughput. It also means that ordinary analyses do not have to wait on Usage Tracking insertions, which improves average query response time. You might want to adjust this value based on available memory and memory utilization on the server computer.
BUFFER_TIME_LIMIT_SECONDS. This parameter indicates the maximum amount of time that an insert statement remains in the buffer before the Usage Tracking subsystem attempts to issue it. This time limit ensures that the BI Server issues the insert statements quickly, even during periods of extended quiescence.
NUM_INSERT_THREADS. This parameter indicates the number of threads that remove insert statements from the buffer and issue them to the Usage Tracking database. Assuming separate connection pools for readers and inserters, the number of insert threads typically equals the Maximum Connections setting in the connection pool.
MAX_INSERTS_PER_TRANSACTION. This parameter indicates the maximum number of insert statements that the Usage Tracking subsystem attempts to issue as part of a single transaction. The larger this number, the greater potential throughput for UsageMarathon Tracking inserts. However, a larger number also increases the likelihood of transactions failing due to deadlocks. A small value for BUFFER_TIME_LIMIT_SECONDS can limit the number of inserts per transaction.

See NQSConfig.INI File Configuration Settings.

Description of the Usage Tracking Data

The table below describes each column in the S_NQ_ACCT usage tracking table. Where appropriate, the data type and length is also included.

As you review the descriptions in the table below, you might assume that certain of the time-related columns can be added or subtracted to equal exact values. For example, you might assume that TOTAL_TIME_SEC is equal to END_TS minus START_TS. The following list explains why the columns do not provide such exact values:

The various processes run in parallel and their speed depends on the load on the BI Server and on database performance. The server-based operations might be either light or intensive.
If all connections are full, then the query enters a queue and waits to be processed. The timing depends on the load and configuration of the BI Server.

Column	Description
CACHE_IND_FLG	Default is N. Y indicates a cache hit for the query; N indicates a cache miss.
COMPILE_TIME_SEC	The time in seconds that is required to compile the query. The number for COMPILE_TIME_SEC is included in TOTAL_TIME_SEC, as described in this table.
CUM_DB_TIME_SEC	The cumulative time of all queries sent to the database. Queries run in parallel, so the cumulative query time is equal to or greater than the total time connected to the database. For example, if a logical request spawns 4 physical SQL statements sent to the database, and the query time for 3 of the queries is 10 seconds, and for one query is 15 seconds. Since the queries run in parallel, nqsserver is only connected to the database for 15 seconds, but CUM_DB_TIME_SEC will show 45 seconds.
CUM_NUM_DB_ROW	The total number of rows that are returned by the back-end databases.
END_DT	The date the logical query was completed.
END_HOUR_MIN	The hour and minute the logical query was completed.
END_TS	The date and time that the logical query finished. The start and end timestamps also reflect any time that the query spent waiting for resources to become available. If the user submitting the query navigates away from the page before the query finishes, then the final fetch never happens, and a timeout value of 3600 is recorded. However, if the user navigates back to the page before the timeout, then the fetch completes at that time, and this is recorded as the end_ts time.
ERROR_TEXT	Default is Null. Varchar(250) Error message from the back-end database. This column is only applicable if the `SUCCESS_FLG` (for more information, see entry later in this table) is set to a value other than 0 (zero). Multiple messages are concatenated and are not parsed by the BI Server.
ID	The unique row ID.
NODE_ID	Concatenates <hostname>:<component_name> where <component_name> can be overridden by the environment variable COMPONENT_NAME. For example, `examplehost:obis1` (for a single instance). Default value of COMPONENT_NAME is obis1.
NUM_CACHE_HITS	Indicates the number of times that the cache result returned for the query. NUM_CACHE_HITS is a 32-bit integer (or a 10-digit integer). Default is Null.
NUM_CACHE_INSERTED	Indicates the number of times that the query generated a cache entry. Default is Null. NUM_CACHE_INSERTED is a 32-bit integer (or a 10-digit integer).
NUM_DB_QUERY	The number of queries that were submitted to back-end databases to satisfy the logical query request. For successful queries (SuccessFlag = 0) this number is 1 or greater.
PRESENTATION_NAME	Default is Null. Varchar(128) The name of the Oracle BI Presentation Catalog.
QUERY_BLOB	Contains the entire logical SQL statement without any truncation. The QUERY_BLOB column is a long character string.
QUERY_KEY	Default is Null. Varchar(128). An MD5 hash key that is generated by Oracle Business Intelligence from the logical SQL statement.
QUERY_SRC_CD	The source of the request. Note that the requestor can set QUERY_SRC_CD to any string value to identify itself. Possible values include: Report - If the source is an analysis or any export operation. Drill - If the source is a change in dimension caused by drilling up or down. ValuePrompt or DashboardPrompt - If the source is a value drop-down list in a filter dialog or a dashboard prompt. VisualAnalyzer - If the source is a project to visualize data. DisplayValueMap or MemberBrowserDisplayValues or MemberBrowserPath - If the source is a value related to the display of an analysis. SOAP - If the source is a call from web services such as DataSetSvc. rawSQL - If the source is raw SQL. Seed - If the source is an agent that seeds the cache of the analytics server. Null - If the source is a physical table, or column row count, or view data.
QUERY_TEXT	Varchar(1024). The SQL statement that was submitted for the query. You can change the length of this column (using the ALTER TABLE command), but note that the text that is written into this column is always truncated to the size that is defined in the physical layer. It is the responsibility of the repository administrator not to set the length of this column to a value greater than the maximum query length that is supported by the back-end physical database. For example, Oracle Databases enable a maximum Varchar of 4000, but Oracle Databases truncate to 4000 bytes, not 4000 characters. Hence, if you use a multi-byte character set, the actual maximum string size has a varying number of characters, depending on the character set and characters used.
REPOSITORY_NAME	The name of the repository that the query accesses.
ROW_COUNT	The number of rows that are returned to the query client. When a large amount of data is returned from a query, this column is not populated until the user displays all of the data.
IMPERSONATOR_USER_NAME	Default is None. Varchar(128). The user name of the impersonated user. If the request is not run as an impersonated user, then the value is 'None'.
SAW_DASHBOARD	The path name of the dashboard. If the query was not submitted through a dashboard, then the value is NULL.
SAW_DASHBOARD_PG	Default is Null. Varchar(150) The page name in the dashboard. If the request is not a dashboard request, then the value is NULL.
SAW_SRC_PATH	The path name in the Oracle BI Presentation Catalog for the analysis.
START_DT	The date that the logical query was submitted.
START_HOUR_MIN	The hour and minute that the logical query was submitted.
START_TS	The date and time that the logical query was submitted.
SUBJECT_AREA_NAME	The name of the business model that is being accessed.
SUCCESS_FLG	The completion status of the query, as defined in the following list: 0 - The query completed successfully with no errors. 1 - The query timed out. 2 = The query failed because row limits were exceeded. 3 = The query failed due to some other reason.
TOTAL_TIME_SEC	The time in seconds that the BI Server spent working on the query while the client waited for responses to its analyses. TOTAL_TIME_SEC includes the time for COMPILE_TIME_SEC. This setting is the same as the Response time in the nqquery.log file, as described in Setting the Query Logging Level.
USER_NAME	The name of the user who submitted the query.

The table below describes the S_NQ_DB_ACCT table, which supplements the usage tracking table by providing the physical SQL information for the logical queries stored in S_NQ_ACCT. S_NQ_DB_ACCT has a foreign key relationship back to S_NQ_ACCT.

Column	Description
END_DT	The date the physical query was completed.
END_HOUR_MIN	The hour and minute the physical query was completed.
END_TS	The date and time the physical query finished. The start and end timestamps also reflect any time that the query spent waiting for resources to become available.
ID	The unique row ID.
LOGICAL_QUERY_ID	Varchar2(50). Refers to the logical query in the S_NQ_ACCT table.
QUERY_BLOB	Contains the entire physical SQL statement without any truncation. The QUERY_BLOB column is a long character string.
QUERY_TEXT	Varchar(1024). The SQL statement that was submitted for the query.
ROW_COUNT	The number of rows that are returned to the query client.
TIME_SEC	The physical query execution time.
START_DT	The date that the physical query was submitted.
START_HOUR_MIN	The hour and minute that the physical query was submitted.
START_TS	The date and time that the physical query was submitted.

The table below describes each column in the S_NQ_INITBLOCK usage tracking table, which tracks information about initialization blocks.

Column	Description
USER_NAME	Varchar2(128). The name of the user who ran the initialization block.
REPOSITORY_NAME	Varchar2(128). The name of the repository that the query accesses.
TENANT_ID	Varchar2(128). The name of the tenant of the user who ran the initialization block.
SERVICE_NAME	Varchar2(128). The name of the service.
ECID	Varchar2(1024). The system-generated execution context ID.
SESSION_ID	Number(10). The ID of the session.
BLOCK_NAME	Varchar2(128). The name of the initialization block that was executed.
START_TS	The date and time that the initialization block started.
END_TS	The date and time that the initialization block finished. The start and end timestamps also reflect any time that the query spent waiting for resources to become available.
DURATION	Number(13,3). The length of time it took to execute the initialization block.
NOTES	Varchar2(1024). Notes about the initialization block and its execution.