Upgrade Source Applications

This page describes how to upgrade your source applications to be compatible with Oracle Utilities Analytics Warehouse v25.10.0.0.0.

Note:

Before proceeding with source upgrade, ensure you have a proper backup plan to revert the changes in case of any failure.
Upgrade the source applications to be compatible with Oracle Utilities Analytics Warehouse v25.10.0.0.0.

Upgrade Network Management System (NMS) Source Application

Note:

Perform the following steps only if the upgraded source application is NMS, and you are upgrading to version 2.6.0 or later.
If the upgraded source application is NMS and version is below 2.6.0, perform the steps mentioned in Upgrade Other Source Applications.

Bring down the source application.
Ensure the database is up and running in both the source and the target.
Upgrade the NMS source application only—without executing NMS UTC data conversion migration scripts, and without executing the normal setup process which runs other migrations involving DDL changes.
After a successful upgrade, do not release the source applications to end users. Ensure that you stop the source applications but keep the source databases up and accessible.
If the source database is not in UTC, then do the following:
1. Stop all GoldenGate Processes (EXTRACTS and REPLICATS in the source and the target).
2. Execute the NMS UTC data conversion migration script as part of the upgrade to NMS 2.6.0 or later before changing the source database time zone to UTC.
3. Update the source database time zone to UTC as part of the upgrade to NMS 2.6.0 or later.
If the source database is in UTC and the source Application is not in UTC, perform the following steps:
1. Alter the initial checkpoint of the GoldenGate EXTRACT process using BEGIN NOW in the source. Note: Here the product context code is <NMS1>.
```
cd  < Oracle GoldenGate Microservices Installation Home>/bin
./adminclient
OGG> connect https://<source_goldengate_host>:<service_manager_port> as <goldengate admin user> password <goldengate admin user password> !
OGG> ALTER EXTRACT <NMS1>AAX BEGIN NOW
```
2. Ensure the source application is still down.
3. Start all GoldenGate Processes (EXTRACTS and REPLICATS) in the source and target.
If the source database and source Application are in UTC, ensure the source application is still down, and then start all GoldenGate Processes (EXTRACTS and REPLICATS) in the source and target.
Execute the normal setup process which runs other migrations involving DDL changes as part of upgrade to NMS 2.6.0 or later.
Ensure that the table NMS_ACCOUNTS_HISTORY is created in the source database.
Ensure Incremental Changes (DDL and DML changes during the upgrade) are synced. The Lag for Chkpt and Time Since Chkpt in both the source and target databases should be 0.
Stop all GoldenGate Processes (EXTRACTS and REPLICATS) in the source and the target.

Upgrade Other Source Applications

Note:

Perform the following steps if the upgraded source applications are Meter Data Management (MDM), Customer Care and Billing (CCB), Customer to Meter (C2M), Work and Asset Management (WAM), and NMS (if you are upgrading to a version below 2.6.0).

Bring down the source application.
Ensure the database is up and running in both the source and the target.
Start all Golden Gate Processes (EXTRACTS and REPLICATS) in the source and the target.
Make sure the data in the source is in sync with the replication tables. The Lag for Chkpt and Time Since Chkpt in both the source and target databases should be 0.
Upgrade the source application.
After a successful upgrade, do not release the source applications to end users. Ensure that you stop the source applications but keep the source databases up and accessible.
Ensure Incremental Changes (DDL and DML changes during the upgrade) are synced. Make sure the data in the source is in sync with the replication tables. The Lag for Chkpt and Time Since Chkpt in both the source and target databases should be 0.
Stop all Golden Gate Processes (EXTRACTS and REPLICATS) in the source and the target.

Post-Source Upgrade Steps

Run Post-Source Upgrade Steps

Note:

Steps 1, 5, 9, 10, 11, 12, and 13 are only valid for the Oracle Utilities Network Management System (NMS) Source. Ignore these steps for Oracle Utilities Customer Care and Billing, Meter Data Management, Customer To Meter, and Work and Asset Management.

Perform the steps below in accordance with the upgraded NMS version in ODI Studio.
For NMS Versions 2.6.0.0 and Later:
1. Navigate to Load Plans and Scenarios > Framework > Data correction > Repository Fixes.
2. Right-click the B1_UPD_NMS_SCEN_FROM_260 scenario.
3. Click Run.
4. Set Context as "Global".
5. Click OK.
For NMS Versions after 2.5.0.2 and below 2.6.0.0:
1. Navigate to Load Plans and Scenarios > Framework > Data correction > Repository Fixes.
2. Right-click the B1_UPD_NMS_SCEN_FROM_2502_UNDER_260 scenario.
3. Click Run.
4. Set Context as "Global".
5. Click OK.
For NMS Versions after 2.4.0.1 and below 2.5.0.2:
1. Navigate to Load Plans and Scenarios > Framework > Data correction > Repository Fixes.
2. Right-click the B1_UPD_NMS_SCEN_FROM_2401_UNDER_2502 scenario.
3. Click Run.
4. Set Context as "Global".
5. Click OK.
For NMS Versions below 2.4.0.1:
1. Navigate to Load Plans and Scenarios > Framework > Data correction > Repository Fixes.
2. Right-click the B1_UPD_NMS_SCEN_UNDER_2401 scenario.
3. Click Run.
4. Set Context as "Global".
5. Click OK.
Run the source configuration in REGISTER mode.
Note:
- Context Code should be same with old Context Code which was cleaned during GoldenGate Classic cleanup process.
- Perform the source configuration as per the steps mentioned in Configure ETL Source for OUAW (except all steps from section Set Up Source Database Server for Oracle GoldenGate Microservices and step 1 from section Source Configuration Using OUAW Wizard).
Make sure that all EXTRACT, REPLICAT, Distribution Paths are up and running. Lag at the Chkpt and Time Since Chkpt should be zero (0). This can be confirmed through the info after logging in to Source and Target OGG MS Administration server console and the Source Distribution Server console.
Stop the GoldenGate Microservice REPLICAT processes in the target server.
Check if the tables MERGED_EVENTS and NMS_ACCOUNTS_HISTORY are already synced in Oracle Utilities Analytics Warehouse using the queries below.

Note:
Skip this step if any of the required tables are not present in the NMS Replication schema.
```
SELECT * FROM MDADM.B1_TABLE_SYNC WHERE SRC_TABLE_NAME='MERGED_EVENTS' AND CONTEXT_CD='<NMS_REPLICATION_SCHEMA>';
```
```
SELECT * FROM MDADM.B1_TABLE_SYNC WHERE SRC_TABLE_NAME='NMS_ACCOUNTS_HISTORY' AND CONTEXT_CD='<NMS_REPLICATION_SCHEMA>';
```
If a record exists for a table and the ROWS_EXTRACTED, ROWS_LOADED column values are NOT NULL for a respective table, do not execute the below delete statement for that respective table.

If the record does not exist for a table or if the ROWS_EXTRACTED, ROWS_LOADED values are NULL for a respective table, perform the below delete step for the respective table to purge any data that has got synced.
```
DELETE FROM <NMS_REPLICATION_SCHEMA>.MERGED_EVENTS;
DELETE FROM <NMS_REPLICATION_SCHEMA>.NMS_ACCOUNTS_HISTORY; 
COMMIT;
```
Run B1_SYNC_CONTEXT Version 001 in ODI studio for the configured product context.
Start the GoldenGate Microservice REPLICAT processes in the target server.
Bring up the source application.
For NMS version 2.6.0 and later, verify if the B1_EVENT_MULTI_STORM_VW view was created in the Replication schema. Note: Here the context code is NMS1.
```
Select * from <NMS1REP>.B1_EVENT_MULTI_STORM_VW;
```
If the view was not created, run this scenario:

Note:
Select the appropriate context. If multiple contexts are configured, execute for each context separately.
1. Navigate to Load Plans and Scenarios > Oracle Utilities BI > NMS > Replication.
2. Right click B1_EVENT_MULTI_STORM_VW and select Run.
3. Set Context as the NMS configured context code and click OK.
For NMS version 2.6.0 and later, run the B1_STORM_DATA_FIX_2802 Version 001 scenario for back-filling of storm changes (as part of 2.8.0.2) in NMS Star Schema, based on the current configured source NMS version.

Note:
Select the appropriate context. If multiple contexts are configured, execute for each context separately.
1. Navigate to Load Plans and Scenarios > Framework > Data correction > Data Fixes.
2. Right click B1_STORM_DATA_FIX_2802 Version 001 and select Run.
3. Set Context as the NMS configured context code and click OK.
Perform the following steps to correct the historical data in CD_DAMAGE_ASMT and CF_DAMAGE_ASMT entities from DAMAGE_ASSETS source table.
For NMS Versions 2.4.0.1 and Later:
1. Navigate to ODI Designer > Load Plans and Scenarios > Framework > Data correction > Data Fixes.
2. Right-click B1_UPD_DAMAGE_ASMT_29001.
3. Click Run.
  
  Note:
  Select the appropriate context. If multiple contexts are configured, execute for each context separately.
Disable all NMS facts and dimensions (including B1_BR_SW_PLAN_JOBS).

Note:
Perform this step only if not already performed during the Post OUAW Upgrade Steps.
This can be ensured by checking that there is no record for any entity in B1_READY_VW view in MDADM schema:
```
select distinct entity_name from MDADM.b1_ready_vw;
```
For Account Dimension (NMS) related changes, perform an incremental or complete load of the data from NMS_ACCOUNTS_HISTORY table into Account Dimension.

Note:
Perform this step only if not already performed during the Post OUAW Upgrade and Source Technology Upgrade.

Option 1: Incremental Data Load
CD_ACCT dimension will load only the incremental data from NMS_ACCOUNTS_HISTORY table. The data which is already loaded in the dimension from CES_CUSTOMERS_HISTORY table will not be modified or updated and remains the same. The dependent facts will not be impacted while performing initial load.
1. Ensure the NMS_ACCOUNTS_HISTORY is present in replication layer with data.
2. Stop the GoldenGate REPLICAT.
3. Navigate to Load Plans and Scenarios > Oracle Utilities BI > Shared > Dimensions > Account Dimension to load all the initial and incremental data from CES_CUSTOMERS_HISTORY table into the dimension.
4. On the Definition tab, double-click B1_PKG_NMS_CD_ACCT_UPTO_24013 Version 001 and rename it to B1_PKG_NMS_CD_ACCT Version 001.
5. On the Definition tab, double-click B1_NMS_D_ACCT_VW_UPTO_24013 Version 001 and rename (if not already done) it to B1_NMS_D_ACCT_VW Version 001.
6. Click Save.
7. Run the View generation script for the configured NMS Context.
8. Enable the entity active flag for CD_ACCT dimension and B1_RUN_ALL to load the Account dimension. This loads data from CES_CUSTOMERS_HISTORY until latest checkpoint.
9. Ensure that the CD_ACCT dimension data loads are completed until the checkpoint date. The output of below queries should result in same date:
```
SELECT MAX (SLICE_END_DTTM) FROM MDADM.B1_JOBS_VW WHERE CONTEXT_CD= <NMS_context_code> AND ENTITY_NAME='CD_ACCT';
SELECT LAST_UPDATE_TS FROM MDADM.B1_CHECKPOINT WHERE GROUP_NAME LIKE 'NMS%';
```
10. Revert the changes for scenario names for CD_ACCT dimension upon completion of the job loads by navigating to Load Plans and Scenarios > Oracle Utilities BI > Shared> Dimensions > Account Dimension.
11. On the Definition tab, double-click B1_PKG_NMS_CD_ACCT Version 001 and rename it to B1_PKG_NMS_CD_ACCT_UPTO_24013 Version 001.
12. On the Definition tab, double-click B1_NMS_D_ACCT_VW Version 001 and rename it to B1_NMS_D_ACCT_VW_UPTO_24013 Version 001.
13. Click Save.
14. Stop B1_RUN_ALL from ODI Studio.
15. Start the GoldenGate REPLICAT.
16. Load the incremental data into CD_ACCT dimension from NMS_ACCOUNTS_HISTORY table by navigating to Load Plans and Scenarios > Oracle Utilities BI > Shared> Dimensions > Account Dimension.
17. On the Definition tab, double-click B1_PKG_NMS_CD_ACCT_FROM_24013 Version 001 and rename it to B1_PKG_NMS_CD_ACCT Version 001.
18. On the Definition tab, double-click B1_NMS_D_ACCT_VW_FROM_24013 Version 001 and rename it to B1_NMS_D_ACCT_VW Version 001.
19. Click Save.
20. Run the View generation for the Configured Context.
21. Enable B1_RUN_ALL and load the Account dimension. At this point, the data is loaded from NMS_ACCOUNTS_HISTORY.
22. Enable all other dimensions and facts, and perform incremental loads.
Option 2: Complete Data Load
CD_ACCT dimension will load the complete data from NMS_ACCOUNTS_HISTORY table. The data which is already loaded in the dimension from CES_CUSTOMERS_HISTORY table will be updated. All the dependent facts will also be updated with the latest ACCT_KEY values from CD_ACCT table. Once the loads are done, the older records in the account dimension (all the data loaded from CES_CUSTOMERS_HISTORY table) will be deleted.
1. Stop B1_RUN_ALL from ODI studio. Run this query to ensure no ETL jobs are in the "Running" state:
```
SELECT * FROM   MDADM.B1_JOBS_VW WHERE  STATUS_FLG in ('W','P', 'R');
```
  Note:
  If there are entries found, wait for the jobs to finish.
2. Ensure that the NMS_ACCOUNTS_HISTORY table is present and with data in the replication layer.
3. Ensure the scenario names for the existing CD_ACCT NMS Dimension are renamed (if not already done) with suffix 'UPTO_24013' in ODI by navigating to Load Plans and Scenarios > Oracle Utilities BI > Shared> Dimensions > Account Dimension.
4. On the Definition tab, double-click B1_PKG_NMS_CD_ACCT Version 001 and rename it to B1_PKG_NMS_CD_ACCT_UPTO_24013 Version 001.
5. On the Definition tab, double-click B1_NMS_D_ACCT_VW Version 001 and rename it to B1_NMS_D_ACCT_VW_UPTO_24013 Version 001.
6. Click Save.
7. Rename the scenario name for the new CD_ACCT NMS Dimension by navigating to Load Plans and Scenarios > Oracle Utilities BI > Shared> Dimensions > Account Dimension.
8. On the Definition tab, double-click B1_PKG_NMS_CD_ACCT_FROM_24013 Version 001 and rename it to B1_PKG_NMS_CD_ACCT Version 001.
9. On the Definition tab, double-click B1_NMS_D_ACCT_VW_FROM_24013 Version 001 and rename it to B1_NMS_D_ACCT_VW Version 001.
10. Click Save.
11. Run the View generation for the Configured NMS Context.
12. Run the B1_ACCT_DIM_UPD_28020 scenario for the configured NMS Context by navigating to Load Plans and Scenarios > Framework > Data Correction > Data Fixes > B1_ACCT_DIM_UPD_28020.
13. Enable the entity active flag of CD_ACCT dimension.
14. Enable B1_RUN_ALL and load the Account dimension, where data is loaded from NMS_ACCOUNTS_HISTORY.
15. Run the B1_F_ACCT_KEY_UPD_28020 scenario for the configured NMS Context by navigating to Load Plans and Scenarios > Framework > Data Correction > Data Fixes > B1_F_ACCT_KEY_UPD_28020.
16. Run the B1_ACCT_DIM_DEL_KEYS_28020 scenario for the configured NMS Context by navigating to Load Plans and Scenarios > Framework > Data Correction > Data Fixes > B1_ACCT_DIM_DEL_KEYS_28020.
17. Enable all other dimensions and facts, and perform incremental loads.