1. Administering Oracle Fusion Data Intelligence
  2. Configure Oracle Fusion Data Intelligence Data
  3. Schedule Frequent Data Refreshes
  4. About Scheduling Frequent Data Refreshes

About Scheduling Frequent Data Refreshes

You can refresh specific set of tables currently with plans to support more functional areas and datasets in future releases.

When you refresh certain functional areas, prioritized datasets, and warehouse tables, be sure you understand which tables you can refresh or not because the selective refresh of some tables could lead to functional inconsistencies when combining data from multiple subject areas. This frequent data refresh capability is designed for refresh of base tables that capture the transactional data; it isn't meant for derived datasets that require aggregations, snapshots, or complex transformation logic. Such processing creates data latency that doesn't support high volume of frequent data refresh. For Oracle Fusion Data Intelligence, you can schedule frequent refreshes for functional areas that are visible in the Frequent Data Refresh Schedule tab on the Pipeline Settings page.

To know which tables are available for frequent refreshes, see:

If you've enabled the "Prioritized Data Refresh" preview feature and selected datasets for a prioritized incremental refresh, then those specific datasets are available for a frequent data refresh. See Prioritize Datasets for Incremental Refresh (Preview). If you've enabled the "Frequent Refresh Tables" preview feature and saved your selection of the warehouse tables, then the selected tables are available as "Warehouse Tables" for a frequent data refresh. See Schedule Frequent Refreshes of Warehouse Tables (Preview). If you want to select the warehouse tables created by the custom data configurations that were generated prior to release Platform 23.R4, then you must regenerate the applicable custom data configurations for these tables to be visible for selection. From release Platform 23.R4, the warehouse tables created by the custom data configurations are available for a frequent data refresh under the Frequent Refresh tab.

When you select the functional areas for a frequent refresh, you won’t be able to refresh the data pipeline for the applicable functional area using the Refresh Data option on the Data Configuration page. The frequent data refresh process doesn't refresh data from external sources through the data augmentation connectors. Oracle Fusion Data Intelligence processes the data augmentations as part of the incremental loads. If you change a data augmentation after including it in the frequent data refresh schedule, then you must remove that data augmentation and let the next incremental refresh finish. Otherwise, the frequent data refresh might fail. After the incremental refresh is complete, you can add the updated data augmentation back to the frequent data refresh schedule.

For frequent data refreshes, the semantic model won't be updated. The update jobs for semantic model won't run as part of data augmentations, they run for data augmentations only during incremental loads.

Review and consider the following to ensure that frequent data refreshes work as expected:
  • Performance of frequent data refreshes depends on the:
    • Size of data.
    • Data change such as what data has changed, and which pipeline gets triggered.
    • Number of extracted records that may result in very different number of published records, for example, 44 extracted records resulted in 1060 published records in 70 minutes and 395 extracted records resulted in 55 published records in 35 minutes.
  • The frequent data refresh process doesn’t get executed in the following scenarios:
    • In the 180-minute window before the scheduled start of the daily incremental data refresh.
    • If any release upgrade is in progress.
    • Until the previous frequent data refresh process is completed. You can set 1 hour frequency (maximum), however, in some cases it takes more than 1 hour to complete the refresh; in that case, the next frequent data refresh process starts at the next hour.
  • For dataset-level (warehouse tables) frequent data refresh:
    • You must know which exact datasets to refresh.
    • There is a limit of up to 20 datasets for each run.
    • Dependencies aren’t automatically incorporated. You must determine the dependencies and include the applicable tables.