Partitioning and Data Removal
Implement Partitioning: Partitioning is essential to manage the large amounts of data coming into Meter Data Management. Customers will have significant performance problems without partitioning. Partitioning is required for good performance and to organize the data so old data can easily be dropped from the database. Customers need historic partitions (for conversion and replacement reads) and future partitions.
Cloud Implementations: Customers should run 'Add Partition' jobs, F1-ILMAD and F1-ILMSV (for sub-partitions, if needed) on a regular basis to ensure that new partitions are created regularly.
On-Premise Implementations: Please refer to the Oracle Utilities Meter Data Management Database Administration Guide for partitioning guidelines for on-premises implementations.
Implement Compression: Compression is essential to reduce the size of the database. For standard hardware we recommend Advanced Compression. For engineered systems that use Oracle Exadata, we highly recommend HCC (Hybrid Columnar Compression) this reduces the overall database by approximately 50%.
Cloud Implementations: Customers should use the “ILM Automation – Non-Measurement Compression” (F1-ILMNC) batch process to will change the compression level from Advanced to Hybrid Columnar for non-Measurement data at least two months old, which can result in significant disk space savings. Use the following parameters when running this batch process: Owner, MO (optional), DOP (Degree of Parallelism), Estimated Time Limit (How long job can run until it will not compress another partition). In larger databases this can run for several days a month.
On-Premise Implementations: Please refer to the Oracle Utilities Meter Data Management Database Administration Guide for compression guidelines for on-premises implementations.
Setup Information Lifecycle Management (ILM): – The Information Lifecycle Management processes mark items eligible to be removed and update partitions to segregate the data that can be removed. Customers need to define the “ILM Configuration” default retention periods (via the ILM Master Configuration). The retention period should be defined for all ILM objects. Important note, completed IMDs (Initial Measurement Data records) are no longer needed and can be removed shortly after being imported. Removing IMDs after 3 months will reduce the compressed DB by about 50%. There will be savings from other ILM objects too - including Activities and To Do Entries .ILM Crawlers are provided for low volume transactions. High Volume transactions are marked as eligible for archiving automatically.
Cloud Implementations: Run the ILM Crawlers Batch Job (batch controls by object) and once the data is aged request Cloud Operations to move the partitions to Object Storage.
On-Premise Implementations: Please refer to the Oracle Utilities Meter Data Management Database Administration Guide and Information Lifecycle Management for ILM guidelines for on-premises implementations.