Understanding Information Lifecycle Management (ILM)
Information Lifecycle Management (ILM) is a separately licensable component of Oracle Utilities Meter Data Management that works in tandem with Oracle database feature of the same name.
For further background on the overall ILM process see the
Information Lifecycle Management chapter of the Oracle Utilities Application Framework
Administrative User Guide.
This section provides an overview of the components used by the Information Lifecycle Management functionality, including:
• ILM-Enabled Maintenance Objects: Single Retention Period
• ILM-Enabled Maintenance Objects: Multiple Retention Periods
• Adjusting Eligibility of Non-Final Transactions
• Archive Eligibility Hierarchy
• Multiple Retention Period Eligibility Crawling Strategies
Refer to the
Information Lifecycle Management chapter of the Oracle Utilities Application Framework
Administrative User Guide for general information about how ILM works with OUAF applications.
ILM-Enabled Maintenance Objects: Single Retention Periods
The majority of Maintenance Objects support a single retention period. They can either inherit the system wide retention days that are configured on the ILM Configuration master configuration or they can have a retention period specified via the Maintenance Object option ILM Retention Period in Days.
For these objects either the Maintenance Object specific retention period or the system wide retention period will apply to all transactions.
ILM-Enabled Maintenance Objects: Multiple Retention Periods
For several Maintenance Objects the retention period may vary based on the type of transaction. For example, initial measurement data for non-billed units of measure such as voltage might be archived much more rapidly than measurements that feed into the production of billing determinants.
These maintenance objects are differentiated from the single retention period maintenance objects in a few ways:
• The primary table for the entity has an additional column Retention Period (RETENTION_PERIOD) that captures the retention period in days for each transaction. This value is populated based on the ILM Configuration - MDM master configuration.
• There is additional configuration available on the ILM Configuration -MDM master configuration to define multiple retention periods by maintenance object specific criteria
• Initial Measurement Data: retention periods can be specified by type of IMD (Interval vs Scalar), and primary UOM of the measuring component
• Device Events: retention periods can be specified by device event type category
• Activities: retention periods can be specified by activity type category
• There is a special ILM Crawler batch that supports crawling transactions by their specific retention periods
• A combination of each transactions ILM Date and Retention Period are considered when identifying records to evaluate for eligibility. Refer to batch controls ILM Crawler - Device Event (D1-DECRL), ILM Crawler - IMD (D1-IMDCL), or ILM Crawler - Activity (D1-ACTCR) for more details.
• There is an additional maintenance object option ILM Date Partition Months which feeds into the ILM Crawler batch to delay processing until all records on a partition would be eligible. This is explained later in this section under the header Multiple Retention Period Eligibility Crawling Strategies.
Adjusting Eligibility of Non-Final Transactions
As delivered Oracle Utilities Meter Data Management and Customer to Meter maintenance objects allow non-final transactions to be eligible for archive. This is a logical approach when transactional data has a very long retention period (e.g. 3 year old non-final Initial Measurement Data is unlikely to have relevance and in many circumstances shouldn't be finalized).
If non-final transactions should not be eligible for archive this can be controlled at either the Maintenance Object (MO) or Business Object (BO) level:
• MO - ILM Restrict By Status (Y/N): Setting this to "Y" opts into the ability to restrict archiving by status, this is required to be set for either of the MO or BO level restrictions to be considered. To enforce the status it requires that either the ILM Restrict By BO Final Status (Y/N) or ILM Final Status Field Value options be configured as well.
• MO - ILM Restrict By BO Final Status (Y/N): Setting this to "Y" indicates the transactions for the MO must be in a final status to be eligible for ILM. A status is considered to be final based on the BO lifecycle configuration. This can be overridden by the BO option Final Status Required for Archive (Y/N). This is only applicable for an MO that is maintained by business objects.
• BO - Final Status Required for Archive (Y/N): Setting this to "Y" indicates the transactions for this particular BO must be in a final status. This setting overrides the MO level ILM Restrict By BO Final Status (Y/N).
The following table helps to illustrate how these three settings impact whether a given transaction must be in a final status:
• MO: ILM Restrict By Status | • MO: ILM Restrict By BO Final Status | • BO: Final Status Required for Archive | • Final Status Required for Eligibility? |
---|
• N | • Y | • Y | • N |
• N | • Y | • N | • N |
• N | • N | • Y | • N |
• Y | • N | • N | • N |
• Y | • Y | • N | • N |
• Y | • Y | • Y | • Y |
• Y | • N | • Y | • Y |
• Y | • Y | • Not Provided | • Y |
• Y | • N | • Not Provided | • N |
Note:
There are a few MOs that do not support the BO level setting. To best understand how these fields are interpreted for an MO refer to the detailed description of the algorithm plugged into the ILM Eligibility system event for the MO. For example, the generic ILM Eligibility Based on Status (F1-ILMELIG) algorithm does not support the BO level override option.
Archive Eligibility Hierarchy
The table below summarizes details of the eligibility algorithms for each maintenance object supported by ILM. Information on this table includes:
• Cascaded By: Indicates how archiving for transactions for the maintenance object is initiated. For example, device events can be archived by themselves or as part of a related activity.
• ILM Date Computation: Indicates the type(s) of transactions that impact the calculation of the ILM Date for transactions for the maintenance object. "Itself" indicates that the ILM date is not impacted by other transactions.
• Specific Eligibility Considerations: Indicates specific considerations used by the eligibility algorithm.
• Cascaded Transactions: Indicates other transactions that are archived when transactions for the maintenance object are archived. For example, VEE exceptions are archived with initial measurements.
Maintenance Object | Cascaded By | ILM Date Computation | Specific Eligibility Considerations | Cascaded Transactions |
---|
Initial Measurement | | Create date | VEE Exceptions must be eligible Associated Related MC Synchronization activity must be complete | VEE Exceptions |
VEE Exception | Initial Measurement | Earlier of related Initial Measurement or VEE Exception Create Date | Related Service Issue Monitor must be complete | |
Usage Transaction | | The latest create date for all related usage transactions | All related usage transactions must be eligible for archiving | Related Usage Transactions |
Device Event | Activity | Create date Superseded by paired event create date if earlier (where applicable) | Related paired events must be eligible Related service issue monitor must be complete Related activity must be complete | |
Activity | Parent activity | Create date Superseded Device event, child activity, or completion event create date supersede (where applicable) | Cascaded transactions must be eligible Command requests must not be associated to a non-final service issue monitor Command requests must not be associated to an install event on off history entry | Child activities Device events Outbound communications Inbound communications Completion events |
Outbound Communication | Activity | Create Date Superseded by initiating activity's create date (where applicable) | | Inbound communication |
Inbound Communication | Outbound Communication | Create date Superseded by initiating outbound communication create date (where applicable) | | Device events |
Proactive Archive Eligibility for Transactional Data
Some types of transaction data, including Device Events, Initial Measurement Data, and Usage Transactions can be analyzed for archive eligibility as they are received rather than waiting until their retention period has expired. This means that these records are judged ready to be archived as they are created and do not need to be read, analyzed, and updated during the ILM crawling process. This greatly reduces the number of records to be crawled at the end of the retention period and shortens the window of time required for ILM crawling. This section outlines how the ILM Archive switch (ILM_ARCH_SW) is set to “Y” (Yes) for records of these types as part of their initial processing.
• Device Events:
◦ The ILM Archive switch is set to “Y” for device events that meet the following criteria:
▪ No service issue monitors were created for the device event
▪ The device event is not a paired device event
◦ The ILM Archive switch is set to “Y” for device events via the following algorithms:
▪ Create Service Issue Monitor from Device Event (D1-DVCEVTSIM)
▪ Create Service Issue Monitor from Device Event for Paired Event (D1-PRDVEVSIM)
• Initial Measurement Data:
◦ The ILM Archive switch is set to “Y” by default for initial measurements (this is a different approach than the other record types).
◦ The ILM Archive switch is set to “N” for initial measurements that meet the following criteria:
▪ A consumption synchronization activity has been created for the initial measurement
▪ A service issue monitor initiated from a VEE exception related to the initial measurement
◦ The ILM Archive switch is set to “N” for initial measurements that meet the above criteria via the following algorithms:
▪ Update Latest Measurement Date/Time on Scalar MC with Consumption Sync (D1-UDTSCMCRE)
▪ Update Latest Measurement Date/Time on MC with Consumption Sync (D1-UPD-DTMC)
▪ Create Service Issue Monitor from VEE Exception (D2-VEEEXCSIM)
• Usage Transactions:
◦ The ILM Archive switch is set to “Y” when a usage transaction enters the “Sent” or “Discarded” state (or the “Calculated” or “Discarded” states for a sub-usage transaction ) via the following algorithms:
▪ Send Usage (D2–SEND-USG)
▪ Set ILM Switch to ‘Y’ (D1-SETILMSWY)
Multiple Retention Period Eligibility Crawling Strategies
When multiple retention periods are defined for a maintenance object each record will have its retention period written to the database at the time of record creation. This retention period is then used to identify when the record will be ready for eligibility evaluation (aka crawling).
Note:
As specified in the Oracle Utilities Meter Data Management Database Administrator's Guide each unique retention period for the maintenance object will be a sub-partition within a given ILM date partition.
The system supports two methodologies for doing so:
• Evaluate Individual Records:
• This is our default approach
• This is the standard approach to evaluating ILM retention for all maintenance objects with a single retention period
• The ILM date for each record will be compared against the current date less the retention period for that type of record. Simply stated, any records older than the retention period will be evaluated for eligibility.
• In this option we will process a portion of a partition each day and only once we had processed the entire partition would the DBA be able to take an archiving action on the partition
• This results in many visits to the partition to determine eligibility
• The image below illustrates how the evaluate individual records method will crawl one day of records on a partition and sub-partition each day until the retention period for the last day of the partition had expired. All records would then be archived together when the last day of the partition was marked as eligible. This results in daily processing of small amounts of records but frequent visits to the partition.
• Evaluate the Entire Partition
• This option will be enabled by configuring the ILM Date Partition Months in the maintenance object options
• In this option all records on a ILM date partition retention period sub-partition will be evaluated at the same time.
• For example, for a monthly partitioning strategy in the month of January we would not evaluate any records in the January partition until January 31st had reached the end of its retention period.
• This means that we would evaluate the 1st through the 31st all at one time
• This option should typically result in a single visit to the partition to determine eligibility
• The image below illustrates how the evaluate entire partition method will only crawl records from a given partition and sub-partition when all records on that partition are eligible to be evaluated. Therefore, when the last day of the partition is ready to be evaluated the entire partition will be evaluated. This results in periodic processing with, and in many cases, a single visit to the partition for ILM purposes.
Parent topic