Persisting Behavioral Tracking Data

 

To record how online customers are interacting with your e-commerce site, you can record event information to a database. These kinds of events are called Behavior Tracking events. E-analytics and e-marketing systems can then analyze these events offline to evaluate customer behavior and transactional data. You can use the knowledge gained from analysis to create and optimize personalization rules, set up product offers, and develop interactive marketing campaigns. This section describes the requirements and database schema needed to log event data for analytical use.

This topic includes the following sections:

• Activating Behavior Tracking

• Data Storage

• Constraints and Indexes

• Scripts

 

---

Activating Behavior Tracking

Before Behavior Tracking events can be recorded to a database, you must enable the Behavior Tracking listener. This is accomplished by adding a class to the application-config.xml file.

Warning: For proper operation, the WebLogic Server requires that changes to the application-config.xml file be made using the WebLogic Server Administration Console.

Note: For more information on using the WebLogic Server Administration Console, see the WebLogic Server Documentation Center.

To add the Behavior Tracking listener, take the following steps:

1. In the WebLogic Server Console, navigate to Synchronous or Asynchronous Listeners tab in the node tree for wlcsDomain as follows:

   http://hostname:port/console -> wlcsDomain -> Deployments -> wlcsApp -> Service Configurations -> Event Service -> Configuration Tab -> Synchronous Listeners

2. Add the Behavior Tracking listener (com.bea.p13n.tracking.listeners.BehaviorTrackingListener) to the Listen Class to Add field, and then click the Add button. See Figure 3-1.

   Figure 3-1 WebLogic Server Administration Console—Event Service

   
    

Note: You must configure your database before activating Behavior Tracking. For information on how to do this, see Production Environment Scenario.

Event Properties

This section describes Behavior Tracking properties more fully and details the mechanism that persists Behavior Tracking event data to the database. Each Behavior Tracking event property described here can be configured in the WebLogic Server Administration Console. Configuring the Behavior Tracking Service in WebLogic Server details how to set these properties.

As previously mentioned, Behavior Tracking events are placed in a buffer and then intermittently persisted to the Event tables in the database where they can be analyzed offline. An asynchronous service is used so that long-running event handlers can execute without delaying the application from a Web site visitor's perspective.

The buffered Behavior Tracking events are swept into the database using a pool of data connections. The default Data Source is weblogic.jdbc.jts.commercePool. You can use a different Data Source. To do this, create and configure the new Data Source (see Configuring a Data Source) and substitute the name of the default Data Source with the name of the new Data Source in the WebLogic Server Administration Console.

The particular events that are persisted to the database are specified in the behaviorTracking.persistToDatabase property. You can view and alter the list of the persisted events in the WebLogic Server Administration Console. The types in this list must match the type specified in the event; for example, the SessionBeginEvent has as its type the string "SessionBeginEvent".

The frequency of the sweeping of events from the buffer is controlled by the following properties in the application-config.xml file:

• MaxBufferSize

• SweepInterval

• SweepMaxTime

You should tune these properties to optimize performance. A buffer sweep should be performed often enough that writing to the database is not too time consuming but not so frequent that the operation is wasteful.

Configuring the Behavior Tracking Service in WebLogic Server

To configure the various Behavior Tracking properties, take these steps:

Warning: For proper operation, the WebLogic Server requires that changes to the application-config.xml file be made using the WebLogic Server Administration Console.

Note: For more information on using the WebLogic Server Administration Console, see the WebLogic Server Documentation Center.

1. In the WebLogic Server Console, navigate to the Behavior Tracking Service (shown in Figure 3-1) in the node tree for wlcsDomain, as follows:

   http://hostname:port/console -> wlcsDomain -> Deployments -> wlcsApp -> Service Configurations -> Behavior Tracking

   Figure 3-2 WebLogic Server Administration Console—Behavior Tracking Service

   
    

2. To change the Data Source, enter the fully-qualified name of the Data Source in the Data Source JNDI Name field.

3. To change the sweeping of events from the buffer, enter the new buffer values in the appropriate fields.

4. To specify whether a particular event is persisted, add or remove the event from the Persisted Event Types list box.

Configuring a Data Source

This section provides a brief description about configuring a new Data Source for a connection pool used for persisting events.

To configure a new Data Source, take the following steps.

Note: For more information on using the WebLogic Server Administration Console, see the WebLogic Server Documentation Center.

1. In the WebLogic Server Console, navigate to the Behavior Tracking Service (shown in Figure 3-1) in the node tree for wlcsDomain, as follows:

   http://hostname:port/console -> wlcsDomain -> Services -> JDBC -> Data Sources -> Behavior Tracking

   Figure 3-3 WebLogic Server Administration Console—JDBC Data Sources

   
    

2. In the right pane, click Configure a new JDBC Data Source.

3. Enter the appropriate values for the new Data Source in the appropriate tabs and fields.

 

---

Data Storage

This section provides an overview of relational databases and the database schemas and tables that are required for recording Behavior Tracking events.

Relational Databases

Relational databases have both logical and physical structures. Logically you may define one or more databases. Each database may contain one or more tables and indexes, and each table may have multiple columns and rows. The logical structure of databases is quite similar between vendors. However, the physical structure of a database is very vendor-specific. Essentially, the physical structure defines areas on disk drives where the data is stored. Each database environment uses its own terminology and implementation for storing data at the operating system level. For example, Oracle uses the term tablespace and the Microsoft SQL Server uses the term filegroup.

When a database structure is defined by a database administrator, attention must be paid to the location of specific tables. Some tables are static in that they do not change much; some tables are dynamic in that many rows are being added and deleted; and some tables are read frequently and some rarely. Depending on their behavior, tables should be placed on different physical locations. Some of the most highly-used tables in WebLogic Portal and WebLogic Personalization Server are used for Behavior Tracking. The activity of a single customer moving around your site may generate multiple table entries. Therefore, it is recommended that you place these tables on the fastest drives in the computer. Experienced database administrators are aware of many techniques for monitoring and configuring a database installation for optimal performance. If you do not have a database administrator working with your installation and you have a lot of activity on your site, you should bring in a well-qualified database administer for regular maintenance of your system.

Database Directory Paths

The default database directory paths are:

• %WL_PORTAL_HOME%\db\<db vendor>\<db version>\... (Windows)

• $WL_PORTAL_HOME/db/<db vendor>/<db version>/... (UNIX)

where WL_PORTAL_HOME is the directory in which you installed WebLogic Portal or WebLogic Personalization Server.

For example, if you are using Oracle 8.1.7 on UNIX, the location would be $WL_PORTAL_HOME/db/oracle/817/....

BEA provides scripts to help set up the database schema needed for recording Behavior Tracking events, as well as the schema needed for recording data associated with WebLogic Portal and WebLogic Personalization Server. This data includes information from orders, catalogs, products, portals, and portlets.

For Oracle databases, the tablespaces created for WebLogic Portal and WebLogic Personalization Server data are the WLCS_DATA and WLCS_INDEX.

Note: WLCS_DATA and WLCS_INDEX are tablespace names created by BEA scripts. If you use a particular naming convention, you can rename them.

Behavior tracking uses a tablespace called WLCS_EVENT_DATA. This tablespace stores all Behavior Tracking tables, indexes, and constraints. Because of the potential for high volumes of data, this tablespace should be monitored closely.

Behavior Tracking Database Schema

Three tables are provided for the Behavior Tracking data. The EVENT table stores all event data. The EVENT_ACTION table logs actions used by third-party vendors against the recorded event data, and the EVENT_TYPE table references event types and categories in the EVENT table. Figure 3-4 shows a logical entity-relation diagram for the Behavior Tracking Database.

Figure 3-4 Entity-Relation Diagram for the Behavior Tracking Database

 
 

The EVENT Database Table

Table 3-1 describes the metadata for the EVENT table. This table stores all Behavior Tracking event data. It is an extremely active table.

See the section Constraints and Indexes for information about the constraint defined for this table.

The Primary Key is EVENT_ID.

Table 3-1 The EVENT Table Metadata

Column Name	Data Type	Null Value	Description and Recommendations
EVENT_ID	NUMBER	NOT NULL	A unique, system-generated number used as the record ID. This field is the table's primary key.
EVENT_TYPE	VARCHAR(30)	NOT NULL	A string identifier that shows which event was fired.
EVENT_DATE	DATE	NOT NULL	The date and time of the event.
WLS_SESSION_ID	VARCHAR(254)	NOT NULL	A unique, WebLogic Server-generated number assigned to the session.
XML_DEFINITION	CLOB	NULL	An XML document that contains pertinent event information. It is stored as a CLOB (Character Large Object).
USER_ID	VARCHAR(50)	NULL	The user ID associated with the session and event. If the user has not logged in this column will be null.

 

As shown in Table 3-1, the EVENT table has six columns; each column corresponds to a specific event element. Five of the EVENT table's columns contain data common to every event type. The XML_DEFINITION column contains all information from these five columns plus event data that is unique to each event type. An XML document is created specifically for each event type. The data elements corresponding to each event type are captured in the XML_DEFINITION column of the EVENT table. These elements are listed in Table 3-2.

Table 3-2 XML_DEFINITION Data Elements

Event	Data Element
AddToCartEvent	event_date event_type session_id user_id sku quantity unit_list_price currency
BuyEvent	event_date event_type session_id user_id sku quantity unit_price currency application_name
CampaignUserActivityEvent	event_date event_type session_id user_id campaign_id
ClickCampaignEvent	event_date event_type session_id user_id document_type document_id campaign_id scenario_id application_name
ClickContentEvent	event_date event_type session_id user_id document_type
ClickProductEvent	event_date event_type session_id user_id document_type document_id sku category_id application_name
DisplayCampaignEvent	event_date event_type session_id user_id document_type document_id campaign_id scenario_id application_name
DisplayContentEvent	event_date event_type session_id user_id document_type document_id
DisplayProductEvent	event_date event_type session_id user_id document_type document_id sku category_id application_name
PurchaseCartEvent	session_id user_id event_date event_type total_price order_id currency application_name
RemoveFromCartEvent	event_date event_type session_id user_id sku quantity unit_price currency application_name
RuleEvent	event_date event_type session_id user_id ruleset_name rule_name
SessionBeginEvent	event_date event_type session_id user_id
SessionEndEvent	event_date event_type session_id user_id
SessionLoginEvent	event_date event_type session_id user_id
UserRegistrationEvent	event_date event_type session_id user_id

 

The EVENT_ACTION Database Table

Table 3-3 describes the metadata for the EVENT_ACTION table. This table logs actions used by third-party vendors against the recorded event data. It is a fairly static.

The Primary Key is comprised of EVENT_ACTION and ACTION_DATE.

Table 3-3 EVENT_ACTION Table Metadata

Column Name	Data Type	Null Value	Description and Recommendations
EVENT_ACTION	VARCHAR(30)	NOT NULL	The event action taken such as BEGIN EXPORT or END EXPORT. This field is one of the table's primary keys.
ACTION_DATE	DATE	NOT NULL	The date and time of the event. This field is one of the table's primary keys.
EVENT_ID	NUMBER	NULL	The ID of the event that corresponds with the event action taken.

 

The EVENT_TYPE Database Table

Table 3-4 describes the metadata for the EVENT_TYPE table. This table references event types and categories in the EVENT table. This table is static.

See the section Constraints and Indexes for information about the constraint defined for this table.

The Primary Key is EVENT_TYPE.

Table 3-4 EVENT_TYPE Table Metadata

Column Name	Data Type	Null Value	Description and Recommendations
EVENT_TYPE	VARCHAR(30)	NOT NULL	A unique, system-generated number used as the record ID. This field is the table's primary key.
EVENT_GROUP	VARCHAR(10)	NOT NULL	The event category group associated with the event type.
DESCRIPTION	VARCHAR(50)	NULL	A description of the EVENT_TYPE.

 

Note: To record custom events, you must create an entry in this table. If a custom event does not have a record in this table, you cannot persist it to the EVENT table.

Constraints and Indexes

There is a single foreign key constraint between the EVENT_TYPE columns in the EVENT and EVENT_TYPE tables. As previously mentioned, if a custom event does not have a record in the EVENT_TYPE table, it cannot be persisted to the EVENT table.

Other than Primary Keys on each of the tables, there are only two indexes on the EVENT table. One index is on the EVENT.EVENT_DATE column and the other index is comprised of the EVENT.EVENT_TYPE and EVENT.EVENT_DATE columns.

 

---

Scripts

BEA provides scripts to create the Behavior Tracking database schema and tables for Oracle databases. This section provides information about the structures used in both a development and a production environment.

Development Environment Scenario

In a development environment, you may not want or need separate databases or tablespaces for recording Behavior Tracking events from the databases or tablespaces used for WebLogic Portal and WebLogic Personalization Server. Accordingly, you can include the Behavior Tracking database objects along side the database objects of these products. The easiest way to accomplish this is to execute the create_all script found in the event directory of your database installation.

Log into Oracle using SQL*Plus and execute the create_all.sql script in this location:

%WL_PORTAL_HOME%/db/oracle/817/event/create_all.sql

where WL_PORTAL_HOME is the directory in which you installed WebLogic Portal or WebLogic Personalization Server.

The create_all scripts in the event subdirectory executes the following scripts:

• drop_event.sql: Drops all the Behavior Tracking database objects.

• create_event.sql: Creates all the Behavior Tracking database objects.

• insert_event_type.sql: Populates the EVENT_TYPE table with base data.

Production Environment Scenario

This scenario is intended for use in an Oracle production environment where multiple tablespaces and their corresponding elements, such as tables and indexes, can reside in separate tablespaces and potentially on a different database server than WebLogic Portal or WebLogic Personalization Server database objects.

Before enabling the Behavior Tracking events, complete the following steps:

1. Identify the server and database used for recording Behavior Tracking events.

2. In the WL_PORTAL_HOME/db/oracle/817/event directory where WL_PORTAL_HOME is the directory in which you installed the WebLogic Portal or WebLogic Personalization Server:

   1. Edit the create_event_tablespaces.sql script to properly define the tablespace path and data filenames.

   2. Execute the create_event_tablespaces.sql to create the tablespaces.

   3. Edit the create_event_users.sql to ensure the correct user account will be created when this script is executed (the account name by default is WLCS_EVENT).

   4. Execute the create_event_users.sql.

3. Using SQL*Plus, connect as the user defined in create_event_users.sql and execute the script create_all.sql. This script will call drop_event.sql, create_event.sql, and insert_event_type.sql.

4. Change your Data Source information to point to this host, database instance, and user account. For more information, see Event Properties.

Description of Each Script

The Oracle scripts are described in the following list:

• WL_PORTAL_HOME/db/oracle/817/event/create_all.sql

  Executes the following scripts: drop_event.sql, create_event.sql, and insert_event_type.sql.

• WL_PORTAL_HOME/db/oracle/817/event/create_event.sql

  Creates the tables, indexes, and constraints associated with Behavior Tracking events.

• WL_PORTAL_HOME/db/oracle/817/event/create_event_tablespaces.sql

  Creates tablespaces for storage of Behavior Tracking events information.

• WL_PORTAL_HOME/db/oracle/817/event/create_event_users.sql

  Creates the WLCS_EVENT database user and grants the appropriate privileges for working with the Behavior Tracking event tables.

• WL_PORTAL_HOME/db/oracle/817/event/drop_event.sql

  Drops the Behavior Tracking event tables.

• WL_PORTAL_HOME/db/oracle/817/event/insert_event_type.sql

  Populates the EVENT_TYPE table with base data.