39 Working with Memory-Centric Visitor Tracking

Site visitors who browse Oracle WebCenter Sites: Engage assets typically provide information about themselves in a personal profile. Demographics information is also collected as the visitors browse. To prevent overloading the WebCenter Sites database with large amounts of visitor data, and therefore, to improve performance, memory-centric visitor tracking was developed for Engage assets.

For information about the memory-centric method for tracking visitors and requirements for implementing this method, see these topics:

• About Memory-Centric Visitor Tracking

• Enabling Memory-Centric Visitor Tracking

• How Memory-Centric Visitor Tracking Works

39.1 About Memory-Centric Visitor Tracking

When large numbers of visitors browse a website, storing all of their personal information in a single repository degrades the performance of the delivery system. For this reason, websites that collect demographic information about visitors often require an additional repository to help improve site performance. You can gain additional performance by preventing load on the WebCenter Sites database.

WebCenter Sites supports add-on repositories, enabling Engage developers to implement a repository of their own choice to store visitor scalar attribute values. Custom code must be written to store and retrieve visitor information to and from the repository. In addition, the WebCenter Sites memory-centric visitor tracking method must be enabled. Differences between memory- and database-centric methods are outlined in this overview. Information about enabling memory-centric tracking is provided in the rest of this chapter, followed by diagrams illustrating how memory-centric visitor tracking works.

This section includes the following topics:

• Database-Centric Model

• Memory-Centric Model

39.1.1 Database-Centric Model

In database-centric visitor tracking, performance issues on Engage-enabled websites arise for the following reasons:

• All visitor information is stored in the WebCenter Sites database. Multiple database accesses are required to store and retrieve information.

• When dynamic recommendations are used and a large amount of data is returned, WebCenter Sites logs a greater number of dependencies in cache, which leads to performance degradation. To restore performance, the render:overridedeps tag limits the number of dependencies that are logged in the WebCenter Sites database. This tag applies to both the database and memory-centric models.

39.1.2 Memory-Centric Model

Memory-centric tracking improves system performance by reducing load on the WebCenter Sites database.

• All computations are performed in memory:

  • When custom code retrieves visitor scalar attribute values (such as age and gender) from the add-on repository, the memory-centric model stores the retrieved values in memory and uses them with history attribute values to compute segments to which the visitor belongs.

  • Memory-centric tracking computes statistics on various history attribute values in memory and then caches the statistics. Statistics include sums, counts, oldest, and newest.

• Alias and history attribute values are stored in the WebCenter Sites database. It helps reduces both the volume of visitor information and the number of calls required to access the information.

As in database-centric tracking, developers can use the render:overridedeps tag to specify the number and types of Engage asset dependencies to log in the WebCenter Sites database. Enabling memory-centric tracking requires setting a property in wcs_properties.json and writing supporting template code.

39.2 Enabling Memory-Centric Visitor Tracking

You can enable memory-centric visitor tracking by manually adding the vis.useSessionVisitorConnection property to the wcs_properties.json file on the delivery system. The code that you write should support memory-centric visitor tracking. You can also batch save History attributes to reduce memory usage.

See these topics:

• Visitor Tracking Property

• Supporting Code

• Batch-Saving History Attributes to the Database

39.2.1 Visitor Tracking Property

Memory-centric visitor tracking is enabled by the property vis.useSessionVisitorConnection, which must be manually added (using the Property Management Tool) to the WebCenter Sites wcs_properties.json file on the delivery system:

• Setting vis.useSessionVisitorConnection=true enables memory-centric visitor tracking. Supporting template code must also exist on the delivery system for Engage visitors to be correctly tracked; pre-existing code can be reused. See Supporting Code.

• Setting vis.useSessionVisitorConnection=false, leaving it blank, or omitting it from wcs_properties.json enables the database-centric method.

  Note:

  Visitor attributes are not stored in the database when using the memory mode. Therefore, on all content management systems, vis.useSessionVisitorConnection must be either set to false or omitted, thus enabling the database-centric method, which allows visitor attributes to be created and otherwise managed. (Visitor attribute management is supported only on content management systems that are enabled for database-centric tracking. It is not supported on delivery systems. Visitor attributes must be published to the delivery system.)

39.2.2 Supporting Code

The skeleton template in this section shows the type of code that must be written to support memory-centric tracking.

Note:

How the template code executes depends on the value of the vis.useSessionVisitorConnection property. If the value is set to false, the database method of tracking visitors is used. If the value is set to true, memory-centric tracking takes effect.

vdm:setalias
retrieve visitor scalar attributes from add-on repository
vdm:setscalar
commercecontext:calculatesegments
commercecontext:getrecommendations
render Engage assets
vdm:recordhistory
...
render:overridedeps

See How Memory-Centric Visitor Tracking Works.

39.2.3 Batch-Saving History Attributes to the Database

As of WebCenter Sites 7.5 Patch 2, history attributes are saved first to memory, then to the file system, and finally as a batch to the WebCenter Sites database. Batch-saving prevents excessive memory usage (which would otherwise occur when data is produced faster than it can be saved to the database). The batch-save process batches one HistoryAttributeDef table at a time from the file and saves the tables to the database in bulk.

Batch saving to the database requires Engage to have access to database information. Database access is enabled by adding a file named <dataSourceName>.properties to the classpath to specify the following information: driver, url, user, and password. For example, if the data source name is csDataSource, then a file named csDataSource.properties must be placed in the classpath. In our example, properties in the file are set as follows:

driver=com.jnetdirect.jsql.JSQLDriver
url=jdbc:JSQLConnect://localhost:1433/database=TomcatDB
user=tomcatuser
password=tomcatuser 

The following Java JVM parameter controls the time interval at which threads are spawned for asynchronous batch-saving of history attributes from memory to the WebCenter Sites database through the file system:

-Dvisitor.SyncInterval=<seconds>

If left unspecified, the parameter value defaults to 30 seconds.

39.3 How Memory-Centric Visitor Tracking Works

A first-time visitor creates a personal profile, a tag in the template code logs the visitor's alias to the WebCenter Sites database, memory-centric visitor tracking logs a unique ID per visitor, the existing database-centric method logs a unique ID per visit, and the visitor's personal information such as age and gender are stored to the add-on repository. WebCenter Sites computes the visitor's segments from scaler and History attribute values. It computes the sums and counts and stores the results in memory. WebCenter Sites compares the visitor's scalar values in memory to those in the segment rules, the history attribute values to those in the segment rules, the sums and counts in memory to the sums or counts in the segment rules. From the comparisons, WebCenter Sites determines the visitor's segments. Then it determines which Engage assets are of most interest to the visitor.

Diagrams in these topics show how memory-centric tracking works.

Note:

In the diagrams that follow, the add-on repository is a system of your own choice, used to store and retrieve visitor scalar attribute values. Custom code must be written to store and retrieve the values.

See these topics:

• Visitor Detection

• Retrieval of Scalar Values

• Collection of History Attribute Values

• Computation of Sums and Counts

• Computation of Segments

• Display of Recommended Assets

• Logging of Dependencies

39.3.1 Visitor Detection

A first-time visitor enters the site and creates a personal profile. The following events occur:

1. The vdm:setalias tag in the template code is used to log the visitor's alias to the WebCenter Sites database. In subsequent sessions, the visitor is automatically recognized. As a result, a single ID unique to the visitor is kept in the database, which improves performance.

   Memory-centric visitor tracking logs a unique ID per visitor, whereas the existing database-centric method logs a unique ID per visit even for returning visitors. Extra IDs create extra load on the database and reduce performance.

2. The visitor's scalar values (personal information, such as age and gender) are stored to the add-on repository (by custom code).

In the remaining steps, WebCenter Sites gathers and computes information that helps determine the visitor's segments and Engage assets to display to the visitor.

This figure shows the flow for visitor detection:

Figure 39-1 Visitor Detection Flow

Description of "Figure 39-1 Visitor Detection Flow"

39.3.2 Retrieval of Scalar Values

WebCenter Sites begins computing the visitor's segments, starting with scalar values:

1. WebCenter Sites retrieves scalar values from the add-on repository (using custom code).

2. WebCenter Sites sets retrieved scalar values in memory, using the vdm:setscalar tag.

   The session-based implementation stores visitor information (scalar values) only in the http session. Therefore, when a new session begins, visitor information is made available through the add-on repository, instead of the WebCenter Sites database. A new level of caching has been added to optimize the performance of querying for visitor scalar values.

   This figure shows the flow for scalar value retrieval:

   Figure 39-2 Scalar Value Retrieval Flow

   
   Description of "Figure 39-2 Scalar Value Retrieval Flow "

39.3.3 Collection of History Attribute Values

History attribute values are also required for computing a visitor's segments. During the session, history attribute values are collected cyclically:

1. The vdm:recordhistory tag is used to collect history attribute values into memory at one-minute intervals.

2. The history attribute values are saved to the file system and then written to the WebCenter Sites database in batches. The memory is cleared and the cycle starts again.

   Visitor history attribute values are collected first in memory to avoid the need for acquiring database connections at each call. They are saved to the file system to enable batch saving to the database, which minimizes memory and database connections usage, and increases the availability of history attribute values.

When history attribute values are committed to the database, computation of sums and counts begins, as shown in Computation of Sums and Counts.

The following figure shows the flow for history attribute values:

Figure 39-3 History Attribute Value Flow

Description of "Figure 39-3 History Attribute Value Flow "

39.3.4 Computation of Sums and Counts

After the first set of history attribute values is collected and written to the database, WebCenter Sites reads the database at 1-minute intervals, computes the sums and counts, and stores the results in memory.

Caching of sums and counts significantly reduces database queries, while the 1-minute interval makes the computation of sums and counts accurate to within 1 minute.

The following figure shows the flow for computing sums and counts:

Figure 39-4 Computing Sums and Counts Flow

Description of "Figure 39-4 Computing Sums and Counts Flow "

39.3.5 Computation of Segments

At this point, WebCenter Sites has enough information to compute the visitor's segments. Using the commercecontext:calculatesegments tag, WebCenter Sites does the following:

1. Compares the visitor's scalar values in memory to those in the segment rules.

2. Compares the history attribute values in the database to those in the segment rules.

3. Compares the sums and counts in memory to the sums or counts in the segment rules.

From the comparisons, WebCenter Sites determines the visitor's segments.

This figure shows the flow for computing segments:

Figure 39-5 Computing Segments Flow

Description of "Figure 39-5 Computing Segments Flow "

39.3.6 Display of Recommended Assets

Having determined the visitor's segments, WebCenter Sites uses the commercecontext:getrecommendations tag to determine which Engage assets are of most interest to the visitor. Templates for Engage assets display the recommended assets to the visitor.

Further performance gains are achieved by use of the render:overridedeps tag, as explained in Logging of Dependencies.

The following figure shows the flow to display recommended assets:

Figure 39-6 Displaying Recommended Assets Flow

Description of "Figure 39-6 Displaying Recommended Assets Flow "

39.3.7 Logging of Dependencies

The render:overridedeps tag is used to reduce the number of dependencies logged in cache. The tag must be inserted in the template (or CSElement), just before the </cs:ftcs> tag, to remove all the existing dependencies logged on the page and to log the dependencies that are specified by the render:overridedeps tag. The possible dependencies are:

• Unknown dependencies

• Dependencies on a single specific asset

• Unknown dependencies for a specific asset type

The render:overridedeps tag takes the following parameters:

• cid: ID of the asset of type c. When combined with c, it works like asset, a dependency is logged against the asset determined by c and cid.

• c: Type of asset.

• deptype: Dependency type.

Which parameters are used and how they are set determines the dependency that is logged:

• When deptype alone is specified and set to unknowndeps, render:overridedeps logs an unknown dependency.

• When c and cid are both specified, render:overridedeps logs a dependency on the asset specified by c and cid.

• When c and deptype='unknowndeps' are both specified, render:overridedeps logs an unknown dependency on the asset type specified by c.

  Note:

  Use render:overridedeps carefully. The only way to flush page caches is to edit the asset for which dependencies are logged. For example, when logging dependencies for a single asset (or asset type), flush the cache by editing the asset for which dependencies are logged. Users must know which asset(s) to edit.