Understanding Information Lifecycle Management (ILM)

ILM-Enabled Maintenance Objects: Single Retention Periods

The majority of Maintenance Objects support a single retention period. They can either inherit the system wide retention days that are configured on the ILM Configuration master configuration or they can have a retention period specified via the Maintenance Object option ILM Retention Period in Days.

For these objects either the Maintenance Object specific retention period or the system wide retention period will apply to all transactions.

ILM-Enabled Maintenance Objects: Multiple Retention Periods

For several Maintenance Objects the retention period may vary based on the type of transaction. For example, initial measurement data for non-billed units of measure such as voltage might be archived much more rapidly than measurements that feed into the production of billing determinants.

These maintenance objects are differentiated from the single retention period maintenance objects in a few ways:

• The primary table for the entity has an additional column Retention Period (RETENTION_​PERIOD) that captures the retention period in days for each transaction. This value is populated based on the ILM Configuration - MDM master configuration.
• There is additional configuration available on the ILM Configuration -MDM master configuration to define multiple retention periods by maintenance object specific criteria
• Initial Measurement Data: retention periods can be specified by type of IMD (Interval vs Scalar), and primary UOM of the measuring component
• Device Events: retention periods can be specified by device event type category
• Activities: retention periods can be specified by activity type category
• There is a special ILM Crawler batch that supports crawling transactions by their specific retention periods
• A combination of each transactions ILM Date and Retention Period are considered when identifying records to evaluate for eligibility. Refer to batch controls ILM Crawler - Device Event (D1-DECRL), ILM Crawler - IMD (D1-IMDCL), or ILM Crawler - Activity (D1-ACTCR) for more details.
• There is an additional maintenance object option ILM Date Partition Months which feeds into the ILM Crawler batch to delay processing until all records on a partition would be eligible. This is explained later in this section under the header Multiple Retention Period Eligibility Crawling Strategies.

Adjusting Eligibility of Non-Final Transactions

As delivered Oracle Utilities Meter Data Management and Customer to Meter maintenance objects allow non-final transactions to be eligible for archive. This is a logical approach when transactional data has a very long retention period (e.g. 3 year old non-final Initial Measurement Data is unlikely to have relevance and in many circumstances shouldn't be finalized).

If non-final transactions should not be eligible for archive this can be controlled at either the Maintenance Object (MO) or Business Object (BO) level:

• MO - ILM Restrict By Status (Y/N): Setting this to "Y" opts into the ability to restrict archiving by status, this is required to be set for either of the MO or BO level restrictions to be considered. To enforce the status it requires that either the ILM Restrict By BO Final Status (Y/N) or ILM Final Status Field Value options be configured as well.
• MO - ILM Restrict By BO Final Status (Y/N): Setting this to "Y" indicates the transactions for the MO must be in a final status to be eligible for ILM. A status is considered to be final based on the BO lifecycle configuration. This can be overridden by the BO option Final Status Required for Archive (Y/N). This is only applicable for an MO that is maintained by business objects.
• BO - Final Status Required for Archive (Y/N): Setting this to "Y" indicates the transactions for this particular BO must be in a final status. This setting overrides the MO level ILM Restrict By BO Final Status (Y/N).

The following table helps to illustrate how these three settings impact whether a given transaction must be in a final status:

Archive Eligibility Hierarchy

The table below summarizes details of the eligibility algorithms for each maintenance object supported by ILM. Information on this table includes:

• Cascaded By: Indicates how archiving for transactions for the maintenance object is initiated. For example, device events can be archived by themselves or as part of a related activity.
• ILM Date Computation: Indicates the type(s) of transactions that impact the calculation of the ILM Date for transactions for the maintenance object. "Itself" indicates that the ILM date is not impacted by other transactions.
• Specific Eligibility Considerations: Indicates specific considerations used by the eligibility algorithm.
• Cascaded Transactions: Indicates other transactions that are archived when transactions for the maintenance object are archived. For example, VEE exceptions are archived with initial measurements.

Proactive Archive Eligibility for Transactional Data

Some types of transaction data, including Device Events, Initial Measurement Data, and Usage Transactions can be analyzed for archive eligibility as they are received rather than waiting until their retention period has expired. This means that these records are judged ready to be archived as they are created and do not need to be read, analyzed, and updated during the ILM crawling process. This greatly reduces the number of records to be crawled at the end of the retention period and shortens the window of time required for ILM crawling. This section outlines how the ILM Archive switch (ILM_​ARCH_​SW) is set to “Y” (Yes) for records of these types as part of their initial processing.

• Device Events:

  • The ILM Archive switch is set to “Y” for device events that meet the following criteria:

    • No service issue monitors were created for the device event

    • The device event is not a paired device event

  • The ILM Archive switch is set to “Y” for device events via the following algorithms:

    • Create Service Issue Monitor from Device Event (D1-DVCEVTSIM)

    • Create Service Issue Monitor from Device Event for Paired Event (D1-PRDVEVSIM)

• Initial Measurement Data:

  • The ILM Archive switch is set to “Y” by default for initial measurements (this is a different approach than the other record types).

  • The ILM Archive switch is set to “N” for initial measurements that meet the following criteria:

    • A consumption synchronization activity has been created for the initial measurement

    • A service issue monitor initiated from a VEE exception related to the initial measurement

  • The ILM Archive switch is set to “N” for initial measurements that meet the above criteria via the following algorithms:

    • Update Latest Measurement Date/Time on Scalar MC with Consumption Sync (D1-UDTSCMCRE)

    • Update Latest Measurement Date/Time on MC with Consumption Sync (D1-UPD-DTMC)

    • Create Service Issue Monitor from VEE Exception (D2-VEEEXCSIM)

• Usage Transactions:

  • The ILM Archive switch is set to “Y” when a usage transaction enters the “Sent” or “Discarded” state (or the “Calculated” or “Discarded” states for a sub-usage transaction ) via the following algorithms:

    • Send Usage (D2–SEND-USG)

    • Set ILM Switch to ‘Y’ (D1-SETILMSWY)

Multiple Retention Period Eligibility Crawling Strategies

When multiple retention periods are defined for a maintenance object each record will have its retention period written to the database at the time of record creation. This retention period is then used to identify when the record will be ready for eligibility evaluation (aka crawling).

The system supports two methodologies for doing so:

• Evaluate Individual Records:
• This is our default approach
• This is the standard approach to evaluating ILM retention for all maintenance objects with a single retention period
• The ILM date for each record will be compared against the current date less the retention period for that type of record. Simply stated, any records older than the retention period will be evaluated for eligibility.
• In this option we will process a portion of a partition each day and only once we had processed the entire partition would the DBA be able to take an archiving action on the partition
• This results in many visits to the partition to determine eligibility

• The image below illustrates how the evaluate individual records method will crawl one day of records on a partition and sub-partition each day until the retention period for the last day of the partition had expired. All records would then be archived together when the last day of the partition was marked as eligible. This results in daily processing of small amounts of records but frequent visits to the partition.

  

• Evaluate the Entire Partition
• This option will be enabled by configuring the ILM Date Partition Months in the maintenance object options
• In this option all records on a ILM date partition retention period sub-partition will be evaluated at the same time.
• For example, for a monthly partitioning strategy in the month of January we would not evaluate any records in the January partition until January 31st had reached the end of its retention period.
• This means that we would evaluate the 1st through the 31st all at one time
• This option should typically result in a single visit to the partition to determine eligibility

• The image below illustrates how the evaluate entire partition method will only crawl records from a given partition and sub-partition when all records on that partition are eligible to be evaluated. Therefore, when the last day of the partition is ready to be evaluated the entire partition will be evaluated. This results in periodic processing with, and in many cases, a single visit to the partition for ILM purposes.