About sift dimensions

Auto sifting is an extension to autogeneration which positions newly generated dimension values within an existing hierarchy.

The hierarchy must be a sift hierarchy, which is a normal hierarchy that contains dimension values of type Sift. As with autogeneration, the dimension mapping in the property mapper must be configured for auto generation.

Configuring sift dimension values

A sift dimension value must be configured as type Sift and specify a range. In the illustration below, all of the dimension values except the root and Quintana-Roo are sift dimension values. The first layer of dimension values after the root specify ranges that span a set of letters, A to D, E to H, and so on. The set of child dimension values below Q-T are also ranges, although they aren't as obvious. The Q dimension value includes all strings that are greater than Q but less than R. Q, therefore, includes strings such as Q, QWERTY, QUEEN, QQQ, QZZZ but not PEZ or RA.

An example of a sift dimension.

Note: Although the range for a sift dimension value is specified the same way as for a range dimension value, sift and range are separate, and mutually exclusive, features.

Sifting an auto-generated dimension value

When a new dimension value is auto-generated, it sifts down the hierarchy according to the ranges it matches. For example, in the illustration above, the auto-generated Quintana-Roo dimension value would first sift into Q-T, and then into Q. The sifting of auto-generated dimension values is handled according to these rules:

Dimension values that do not match any sift range in the hierarchy are added as children of the dimension root.
Dimension values that match the ranges of multiple children of a sift dimension value are placed under the first child they match.
Dimension values that do not match any child of a sift dimension value are placed under the deepest value they match.

It is a good idea to configure the last child under each sift dimension value as a catch-all, with a range of NEG_INF to POS_INF , inclusive. That way, auto-generated dimension values that do not match any other range can be gathered under an Other dimension value.

Sift dimension values and matching

Sift dimension values do not participate in matching. Even though the range Q-T appears within the dimension above, the property value Quintana-Roo will not match it. Following the match failure, a new Quintana-Roo dimension value is automatically generated and sifted through the hierarchy. Any further properties with the value Quintana-Roo will match this auto-generated dimension value.

Sift hierarchy recommendations

Sift hierarchies are frequently made collapsible. A collapsible hierarchy is a form of conditional hierarchy in which some or all of the internal (non-root and non-leaf) dimension values are flagged as potentially collapsible. The MDEX Engine automatically removes, or collapses, these dimension values when there are only a few leaves available for refinement, creating a more streamlined, user-friendly navigation experience for your users. See About collapsible dimensions for details.

In addition, sift dimension values are typically ranked, to ensure that the ranges appear in order during navigation.

Note: All dimension values in a sift dimension must be specified as type Sift or there may be unexpected behavior.

Saving autosift state information

At the end of each data processing run, the dimension server saves the state of an auto sift dimension, just like any other auto-generated dimension. At the start of the next run, the sift hierarchy may be updated; therefore, all of the previously auto-generated dimension values are re-sifted into the new sift hierarchy. This ensures that they are positioned correctly within the hierarchy, even when the hierarchy has been changed since it was initially generated.

Note: For a common sift dimension use case scenario, see A typical scenario.