41 Memory-Centric Visitor Tracking

Site visitors who browse Engage assets typically provide information about themselves in a personal profile. Demographics information is also collected as visitors browse. To prevent overloading WebCenter Sites's database with large amounts of visitor data, and therefore to improve performance, memory-centric visitor tracking was developed for Engage assets.

This chapter describes the memory-centric method of tracking visitors and outlines requirements for implementing this method.

This chapter contains the following sections:

41.1 Overview of Memory-Centric Visitor Tracking

When large numbers of visitors browse a website, storing all of their personal information in a single repository degrades the performance of the delivery system. For this reason, sites that collect demographic information on visitors often require an additional repository to help improve site performance. Additional performance gain can be achieved by preventing load on WebCenter Sites's database.

WebCenter Sites supports add-on repositories, enabling Engage developers to implement a repository of their own choice to store visitor scalar attribute values. Custom code must be written to store and retrieve visitor information to and from the repository. In addition, WebCenter Sites's memory-centric visitor tracking method must be enabled. Differences between memory- and database-centric methods are outlined in this overview. Information about enabling memory-centric tracking is provided in the rest of this chapter, followed by diagrams illustrating how memory-centric visitor tracking works.

This section contains the following topics:

41.1.1 Database-centric Model

In database-centric visitor tracking, performance issues on Engage-enabled websites arise for the following reasons:

  • All visitor information is stored in WebCenter Sites's database. Multiple database accesses are required to store and retrieve information.

  • When dynamic recommendations are used and a large amount of data is returned, WebCenter Sites logs a greater number of SystemItemCache entries, which leads to performance degradation. To restore performance, the render:overridedeps tag was introduced as a way of limiting the number of dependencies that are logged in WebCenter Sites's database. This tag applies to both the database and memory-centric models.

41.1.2 Memory-centric Model

Memory-centric tracking improves system performance by reducing load on WebCenter Sites's database.

  • All computations are performed in memory:

    • When custom code retrieves visitor scalar attribute values (such as age and gender) from the add-on repository, the memory-centric model stores the retrieved values in memory and uses them with history attribute values to compute segments to which the visitor belongs.

    • Memory-centric tracking computes statistics on various history attribute values in memory and then caches the statistics. Statistics include sums, counts, oldest, and newest.

  • Alias and history attribute values are stored in WebCenter Sites's database, but in a way that reduces both the volume of visitor information and the number of calls required to access the information.

As in database-centric tracking, developers can use the render:overridedeps tag to specify the number and types of Engage asset dependencies to log in WebCenter Sites's database. Enabling memory-centric tracking requires setting a property in visitor.ini and writing supporting template code.

41.2 Enabling Memory-centric Visitor Tracking

This section contains the following topics:

41.2.1 Visitor Tracking Property

Memory-centric visitor tracking is enabled by the property vis.useSessionVisitorConnection, which must be manually added to WebCenter Sites's visitor.ini file on the delivery system:

  • Setting vis.useSessionVisitorConnection=true enables memory-centric visitor tracking. Supporting template code must also exist on the delivery system in order for Engage visitors to be correctly tracked; pre-existing code can be reused. For more information, see Section 41.2.2, "Supporting Code."

  • Setting vis.useSessionVisitorConnection=false, leaving it blank, or omitting it from visitor.ini enables the database-centric method.


    On all content management systems, vis.useSessionVisitorConnection must be either set to false or omitted, thus enabling the database-centric method, which allows visitor attributes to be created and otherwise managed. (Visitor attribute management is supported only on content management systems that are enabled for database-centric tracking. It is not supported on delivery systems; visitor attributes must be published to the delivery system.)

41.2.2 Supporting Code

The skeleton template in this section shows the type of code that must be written to support memory-centric tracking.


How the template code executes depends on the value of the vis.useSessionVisitorConnection property. If the value is set to false, the database method of tracking visitors is used. If the value is set to true, memory-centric tracking takes effect.

retrieve visitor scalar attributes from add-on repository
render Engage assets

For more information about this template, see Section 41.3, "How Memory-centric Visitor Tracking Works."

41.2.3 Batch-Saving History Attributes to the Database

As of WebCenter Sites 7.5 Patch 2, history attributes are saved first to memory, then to the file system, and finally as a batch to WebCenter Sites's database. Batch-saving prevents excessive memory usage (which would otherwise occur when data is produced faster than it can be saved to the database). The batch-save process batches one HistoryAttributeDef table at a time from the file and saves the tables to the database in bulk.

Batch saving to the database requires Engage to have access to database information. Database access is enabled by adding a file named <dataSourceName>.properties to the classpath to specify the following information: driver, url, user, and password. For example, if the data source name is csDataSource, then a file named csDataSource.properties must be placed in the classpath. In our example, properties in the file are set as follows:


The following Java JVM parameter was added to control the time interval at which threads are spawned for asynchronous batch-saving of history attributes from memory to WebCenter Sites's database via the file system:


If left unspecified, the parameter value defaults to 30 seconds.

41.3 How Memory-centric Visitor Tracking Works

Diagrams in the rest of this chapter show how memory-centric tracking works.


In the diagrams that follow, the add-on repository is a system of your own choice, used to store and retrieve visitor scalar attribute values. Custom code must be written to store and retrieve the values.

This section contains the following topics:

41.3.1 Visitor Detection

A first-time visitor enters the site and creates a personal profile. The following events occur:

  1. The vdm:setalias tag in the template code is used to log the visitor's alias to WebCenter Sites's database. In subsequent sessions, the visitor is automatically recognized. As a result, only a single ID unique to the visitor is kept in the database, which improves performance.

    Note that memory-centric visitor tracking logs a unique ID per visitor, whereas the existing database-centric method logs a unique ID per visit even for returning visitors. Extra IDs create extra load on the database and reduce performance.

  2. The visitor's scalar values (personal information, such as age and gender) are stored to the add-on repository (by custom code).

In the remaining steps, WebCenter Sites gathers and computes information that it needs to determine the visitor's segments and therefore which Engage assets to display to the visitor.

Figure 41-1 Visitor Detection Flow

Description of Figure 41-1 follows
Description of "Figure 41-1 Visitor Detection Flow"

41.3.2 Retrieving Scalar Values

WebCenter Sites begins computing the visitor's segments. Starting with scalar values:

  1. WebCenter Sites retrieves scalar values from the add-on repository (using custom code).

  2. WebCenter Sites sets retrieved scalar values in memory, using the vdm:setscalar tag.

    The session-based implementation stores visitor information (scalar values) only in the http session. Therefore, when a new session begins, visitor information is made available through the add-on repository, instead of WebCenter Sites's database. A new level of caching has been added to optimize the performance of querying for visitor scalar values.

    Figure 41-2 Scalar Value Retrieval Flow

    Description of Figure 41-2 follows
    Description of "Figure 41-2 Scalar Value Retrieval Flow"

41.3.3 Collecting History Attribute Values

History attribute values are also required for computing a visitor's segments. During the session, history attribute values are collected cyclically:

  1. The vdm:recordhistory tag is used to collect history attribute values into memory at one-minute intervals.

  2. The history attribute values are saved to the file system and then written to WebCenter Sites's database in batches. The memory is cleared and the cycle starts again.

    Visitor history attribute values are collected first in memory to avoid the need for acquiring database connections at each call. They are saved to the file system to enable batch saving to the database, which minimizes memory and database connections usage, and increases the availability of history attribute values.

When history attribute values are committed to the database, computation of sums and counts begins, as shown in Section 41.3.4, "Computing Sums and Counts."

Figure 41-3 History Attribute Value Flow

Description of Figure 41-3 follows
Description of "Figure 41-3 History Attribute Value Flow"

41.3.4 Computing Sums and Counts

After the first set of history attribute values is collected and written to the database, WebCenter Sites:

  1. Reads the database at 1-minute intervals, computes the sums and counts, and Stores the results in memory.

Caching of sums and counts significantly reduces database queries, while the 1-minute interval makes the computation of sums and counts accurate to within 1 minute.

Figure 41-4 Computing Sums and Counts Flow

Description of Figure 41-4 follows
Description of "Figure 41-4 Computing Sums and Counts Flow"

41.3.5 Computing Segments

At this point, WebCenter Sites has enough information to compute the visitor's segments. Using the commercecontext:calculatesegments tag, WebCenter Sites does the following:

  1. Compares the visitor's scalar values in memory to those in the segment rules.

  2. Compares the history attribute values in the database to those in the segment rules.

  3. Compares the sums and counts in memory to the sums or counts in the segment rules

From the comparisons, WebCenter Sites determines the visitor's segments.

Figure 41-5 Computing Segments Flow

Description of Figure 41-5 follows
Description of "Figure 41-5 Computing Segments Flow"

41.3.6 Determining and Displaying Recommended Assets

Having determined the visitor's segments, WebCenter Sites uses the commercecontext:getrecommendations tag to determine which Engage assets are of most interest to the visitor. Templates for Engage assets display the recommended assets to the visitor.

Further performance gains are achieved by use of the render:overridedeps tag, as explained in Section 41.3.7, "Logging Dependencies."

Figure 41-6 Displaying Recommended Assets Flow

Description of Figure 41-6 follows
Description of "Figure 41-6 Displaying Recommended Assets Flow"

41.3.7 Logging Dependencies

The render:overridedeps tag is used to reduce the number of SystemItemCache entries logged. The tag must be inserted at the end of the template (or CSElement), just before the </cs:ftcs> tag, to remove all the existing dependencies logged on the page and to log the dependencies that are specified by the render:overridedeps tag. The possible dependencies are:

  • Unknown dependencies

  • Dependencies on a single specific asset

  • Unknown dependencies for a specific asset type

The render:overridedeps tag takes the following parameters:

  • cid: ID of the asset of type c. When combined with c, it works like asset, a dependency will be logged against the asset determined by c and cid.

  • c: Type of asset.

  • deptype: Dependency type.

Which parameters are used and how they are set determines the dependency that is logged:

  • When deptype alone is specified and set to unknowndeps, render:overridedeps logs an unknown dependency.

  • When c and cid are both specified, render:overridedeps logs a dependency on the asset specified by c and cid.

  • When c and deptype='unknowndeps' are both specified, render:overridedeps logs an unknown dependency on the asset type specified by c.


    Use render:overridedeps carefully. The only way to flush page caches is to edit the asset for which dependencies are logged. For example, if you are logging dependencies for a single asset (or asset type), you can flush the cache by editing the asset for which dependencies are logged. Users must know which asset(s) to edit.