Partitioning and Data Removal

• Implement Partitioning: Partitioning is essential to manage the large amounts of data coming into Meter Data Management. Customers will have significant performance problems without partitioning. Partitioning is required for good performance and to organize the data so old data can easily be dropped from the database. Customers need historic partitions (for conversion and replacement reads) and future partitions.

  • Cloud Implementations: Customers are no longer required to run the "Add Partition" (K1-ILMAD) batch process every month. Oracle automatically adds partitions for entities, every month, on desired domains and schemas. However customer are required to run the "ILM Automation - New Subretention Value" (F1-ILMSV) batch process (for sub-partitions, if needed) on a regular basis to ensure that new partitions are created as needed.

  • On-Premise Implementations: Please refer to the Oracle Utilities Meter Data Management Database Administration Guide for partitioning guidelines for on-premises implementations.

• Implement Compression: Compression is essential to reduce the size of the database. For standard hardware we recommend Advanced Compression. For engineered systems that use Oracle Exadata, we highly recommend HCC (Hybrid Columnar Compression) this reduces the overall database by approximately 50%.

  • Cloud Implementations: Customers should use the “ILM Automation – Non-Measurement Compression” (K1-ILMNC) and “ILM Automation – Measurement Compression” (K1-ILMMC) batch processes to change the compression level from Advanced to Hybrid Columnar for non-Measurement data at least two months old, which can result in significant disk space savings. Use the following parameters when running this batch process: Owner, MO (optional), DOP (Degree of Parallelism), Estimated Time Limit (How long job can run until it will not compress another partition). In larger databases this can run for several days a month.

  • On-Premise Implementations: Please refer to the Oracle Utilities Meter Data Management Database Administration Guide for compression guidelines for on-premises implementations.

• Setup Information Lifecycle Management (ILM): – The Information Lifecycle Management processes mark items eligible to be removed and update partitions to segregate the data that can be removed. Customers need to define the “ILM Configuration” default retention periods (via the ILM Master Configuration). The retention period should be defined for all ILM objects. ILM Crawlers are provided for low volume transactions. High Volume transactions are marked as eligible for archiving automatically.

  Note:

  Completed IMDs (Initial Measurement Data records) can be removed shortly after being imported. Removing IMDs after 3 months will reduce the compressed database by approximately 50%. There will be database storages savings by archiving other ILM objects as well, such as Activities and To Do Entries.

  Different types of implementations call for different approaches to using ILM.

  • Cloud Implementations: Run the ILM Crawlers Batch Job (batch controls by object) and once the data is aged, run the "ILM Automation - Drop Partition" (K1-ILMDR)_batch process to remove the old partitions from the cloud service database. Please refer to the detailed description of this batch process for additional details.

  • On-Premise Implementations: Please refer to the Oracle Utilities Meter Data Management Database Administration Guide and Information Lifecycle Management for ILM guidelines for on-premises implementations.