Check Upgrade Plan Status and Progress
In software versions 3.0.2-b892153 and later, the Upgrader uses an upgrade plan as a kind of checklist to perform all upgrade operations, which implies full management cluster upgrades as well as individual component upgrades. The Oracle Private Cloud Appliance Concepts Guide describes this approach in more detail as part of the "Upgrade" section in the chapter Appliance Administration Overview.
The upgrade plan is generated when the Upgrader itself is upgraded to the latest version. The plan is based on a comparison of the currently installed components on the rack, and the target component versions and latest packages downloaded to shared storage during the preparation of the upgrade environment. The resulting upgrade plan shows for which components an upgrade procedure needs to be executed in the next phase.
All components must be upgraded in a prescribed order. The upgrade plan will prevent a component upgrade procedure from starting if the preceding upgrades have not been completed. An error message informs the administrator which components need to be upgraded first.
Note:
In general, firmware may be upgraded whenever new versions are made available for your system. Firmware upgrades can be applied in no particular order and independently of other components. The ZFS Storage Appliance is an exception: its firmware must be upgraded before all other components.
This is the order of operations enforced through the upgrade plan:
-
Prepare upgrade environment (Upgrade PreConfig)
-
Upgrade the Upgrader (PreUpgrade)
-
ZFS Storage Appliance firmware
-
Compute nodes
-
Host operating system of management nodes
-
MySQL cluster database
-
Secret service (including Etcd and Vault)
-
Kubernetes container orchestration packages (platform layer)
-
Containerized microservices
-
Oracle Cloud Infrastructure images
Check the Upgrade Plan
After the upgrade environment has been prepared, all upgrade operations required to bring the system up-to-date are listed in the upgrade plan. Whenever an upgrade procedure has been completed successfully, the upgrade plan is updated with the latest status: for upgraded components the source and target versions are identical and the "upgrade required" flag is disabled.
At any point in time you can check how far the system has progressed through the upgrade plan. It indicates which components are already up-to-date and which still require upgrading. The upgrade plan provides a snapshot view, which is accurate and useful at the time you are consulting it. To see detailed progress of upgrade activity, use the job framework instead.
The upgrade plan can be viewed in two ways:
-
In the Service Web UI, go to Maintenance in the navigation menu and select Upgrade Plan.
-
In the Service CLI, enter the following command:
PCA-ADMIN> getUpgradePlan Data: id component currentBuild targetBuild currentVersion targetVersion requireReboot timeEstimation (min) requireUpgrade impactedInfra -- --------- ------------ ----------- -------------- ------------- ------------- -------------------- -------------- ------------- generic zfssa 3.0.2-b1300385 3.0.2-b1300385 2013.06.05.8.73.1-2.73.5701.1 2013.06.05.8.73.1-2.73.5701.1 false 45 false host,compute 100.96.2.64 compute 3.0.2-b1300130 3.0.2-b1300385 oraclelinux-release-7.9-1.0.9 3.0.202.3.11.0.0.0.5.7ea3f4ccbf3e17fc... true 50 true compute 100.96.2.65 compute 3.0.2-b1300130 3.0.2-b1300385 oraclelinux-release-7.9-1.0.9 3.0.202.3.11.0.0.0.5.7ea3f4ccbf3e17fc... true 50 true compute 100.96.2.66 compute 3.0.2-b1300130 3.0.2-b1300385 oraclelinux-release-7.9-1.0.9 3.0.202.3.11.0.0.0.5.7ea3f4ccbf3e17fc... true 50 true compute 100.96.2.33 host 3.0.2-b1300130 3.0.2-b1300385 oraclelinux-release-7.9-1.0.13 3.0.202.3.11.0.0.0.5.7ea3f4ccbf3e17fc... true 40 true host 100.96.2.34 host 3.0.2-b1300130 3.0.2-b1300385 oraclelinux-release-7.9-1.0.13 3.0.202.3.11.0.0.0.5.7ea3f4ccbf3e17fc... true 40 true host 100.96.2.35 host 3.0.2-b1300130 3.0.2-b1300385 oraclelinux-release-7.9-1.0.13 3.0.202.3.11.0.0.0.5.7ea3f4ccbf3e17fc... true 40 true host generic mysql 3.0.2-b1300385 3.0.2-b1300385 8.0.36-1.1 8.0.36-1.1 false 15 false host generic etcd 3.0.2-b1300385 3.0.2-b1300385 3.5.6 3.5.6 false 5 false host generic vault 3.0.2-b1300385 3.0.2-b1300385 v1.7.1-3.10.0.0.0.4.g5f2d5af v1.7.1-3.10.0.0.0.4.g5f2d5af false 5 false host generic kubernetes 3.0.2-b1300385 3.0.2-b1300385 1.25.16-2 1.25.16-2 false 60 false host,compute generic platform 3.0.2-b1300130 3.0.2-b1300385 None None false 40 true host,compute Oracle-Linux-7.9 ociImages 3.0.2-b1300130 3.0.2-b1300385 None 2024.07.31_0 false 5 true host Oracle-Linux8-OKE-1.28.8 ociImages 3.0.2-b1300130 3.0.2-b1300385 None 20241201 false 5 true host Oracle-Solaris-11 ociImages 3.0.2-b1300130 3.0.2-b1300385 None 2024.08.26_0 false 5 true host Oracle-Linux8-OKE-1.27.12 ociImages 3.0.2-b1300130 3.0.2-b1300385 None 20241201 false 5 true host Oracle-Linux8-OKE-1.26.15 ociImages 3.0.2-b1300130 3.0.2-b1300385 None 20241201 false 5 true host Oracle-Linux-9 ociImages 3.0.2-b1300130 3.0.2-b1300385 None 2024.07.31_0 false 5 true host Oracle-Linux-8 ociImages 3.0.2-b1300130 3.0.2-b1300385 None 2024.07.31_0 false 5 true host 100.96.0.33 ilom 3.0.2-b1300130 3.0.2-b1300385 5.1.4.20 5.1.4.25 true 10 true host,compute 100.96.0.34 ilom 3.0.2-b1300130 3.0.2-b1300385 5.1.4.20 5.1.4.25 true 10 true host,compute 100.96.0.35 ilom 3.0.2-b1300130 3.0.2-b1300385 5.1.4.20 5.1.4.25 true 10 true host,compute 100.96.0.64 ilom 3.0.2-b1300130 3.0.2-b1300385 5.1.4.20 5.1.4.25 true 10 true host,compute 100.96.0.65 ilom 3.0.2-b1300130 3.0.2-b1300385 5.1.4.20 5.1.4.25 true 10 true host,compute 100.96.0.66 ilom 3.0.2-b1300130 3.0.2-b1300385 5.1.4.20 5.1.4.25 true 10 true host,compute leaf switch 3.0.2-b1300385 3.0.2-b1300385 10.3.4a 10.3.4a false 60 false host,compute spine switch 3.0.2-b1300385 3.0.2-b1300385 10.3.4a 10.3.4a false 60 false host,compute mgmt switch 3.0.2-b1300385 3.0.2-b1300385 10.3.4a 10.3.4a false 60 false host,compute
Browse the Job Framework
The Upgrader is integrated with the job framework of the appliance controller software. Each operation results in a work request that triggers one or more jobs. These work requests and jobs can be queried. Their status and detailed properties provide insights into the Upgrader activity on the system.
When you enter a Service CLI command, a job ID and work request ID are returned. These allow you to check status, retrieve more details, find related jobs in the same workflow, and so on. When you create an upgrade (or patch) request using the Service Web UI, those IDs are not displayed on screen, but you can find them in the Upgrade Jobs table. From there, you can also drill down into the work requests and jobs.
- Using the Service Web UI
-
To perform an upgrade operation from the Service Web UI, you use the main navigation menu, go to Maintenance, and click Upgrade & Patching. This opens the Upgrade Jobs page, which displays all recent upgrade jobs in a table, and allows you to create a new upgrade (or patch) request by clicking the button in the top-right corner.
The Upgrade Jobs table lets you browse all the Upgrader requests and jobs, so you can use it to track progress of current, past, and scheduled Upgrader activity.
-
The default table view shows jobs with their associated command, workflow request, time stamps, and result. The Job ID and Request ID are hyperlinks to pages with more detailed information.
The Actions menu on the right hand side of each table row has options equivalent to the hyperlinks, and also lets you copy the job or request ID.
-
When filtered by request ID, the Upgrade Jobs table shows only the jobs that were created as part of the selected upgrade request. Use it to track multicomponent workflows and check which jobs are completed or pending. If a failure occurred, you can see which job did not complete as expected.
To return to the default table view, click Clear Filter.
-
To view the details of a job, click its job ID in the Upgrade Jobs table. Besides the information shown in the jobs table, the Upgrade Job detail page contains the log file location, job execution details, and the list of tasks performed as part of the upgrade job.
If the upgrade job has not completed, you can kill it by clicking the button in the top right corner of the page.
-
To view the details of an upgrade request, click the request ID in the Upgrade Jobs table. Besides the information shown in the jobs table, the Upgrade Status detail page contains source and target builds, workflow details, and a list of associated upgrade jobs.
The upgrade job list is the same as the main Upgrade Jobs table filtered by request ID. It also lets you access each job's detail page by clicking the job ID.
In addition to the job list, the Resources section of the Upgrade Status page lets you display for which components the job has been completed, and which components are still pending.
-
- Using the Service CLI
-
When you perform an upgrade operation from the Service CLI, a request is created. The job ID and upgrade request ID are returned immediately. Use these IDs to query the jobs and requests, and find the details you need.
-
When working in the Service CLI, the request IDs from the Upgrader Workflow Service are the primary element to access information about Upgrader activity. In case you did not note the request ID when entering the command, or you are searching for data from previous workflows, use the
getUpgradeRequestscommand to look up the one you want to investigate.PCA-ADMIN> getUpgradeRequests Data: id -- PREUPGRADE-61b56867-7966-4096-9c74-08efe8d2d0ad UWS-496e26aa-2923-46f1-a485-c086a005077a UWS-da5d70ba-09ea-4ef0-b8ea-84d9f44c5e62 PREUPGRADE-a6aa2c0e-ceff-4de4-aa23-de7d21b06668 [...]
-
With the correct upgrade request ID you can check the status and progress of the upgrade workflow. Use the
getUpgradeStatuscommand to display the workflow details, including the job list, and completed and pending components.PCA-ADMIN> getUpgradeStatus requestId=UWS-910dd8fd-28b3-4f97-a909-02c26085266c Data: Request id = UWS-910dd8fd-28b3-4f97-a909-02c26085266c Status = Failed Composition ID = fullrack Type = ISO Previous Build = 3.0.2-b1333709 Target Build = 3.0.2-b1337789 Jobs 1 = 1741252377283-zfssa-81712 Jobs 2 = 1741253904337-compute-72870 [...] Jobs 13 = 1741269765197-platform-663307 Completed Components 1 = zfssa(generic) Completed Components 2 = compute(100.96.2.64) [...] Pending Components 1 = compute(100.96.2.68) Pending Components 2 = host(100.96.2.33) [...]
-
Using the upgrade request ID you can also list the jobs associated with the request, and scan it for status, progress, and potential issues. Use the
getUpgradeJobscommand to display the job list.PCA-ADMIN> getUpgradeJobs requestId=UWS-910dd8fd-28b3-4f97-a909-02c26085266c Data: id Upgrade Request Id Command Name Result -- ------------------ ------------ ------ 1741269765197-platform-663307 UWS-910dd8fd-28b3-4f97-a909-02c26085266c platform Failed 1741269703036-kubernetes-656961 UWS-910dd8fd-28b3-4f97-a909-02c26085266c kubernetes Passed 1741267992739-host-422432 UWS-910dd8fd-28b3-4f97-a909-02c26085266c host Passed 1741266155984-host-161315 UWS-910dd8fd-28b3-4f97-a909-02c26085266c host Passed 1741264266152-host-11547 UWS-910dd8fd-28b3-4f97-a909-02c26085266c host Passed 1741264201476-compute-87720 UWS-910dd8fd-28b3-4f97-a909-02c26085266c compute Passed 1741264136546-compute-81124 UWS-910dd8fd-28b3-4f97-a909-02c26085266c compute Passed 1741264070151-compute-74130 UWS-910dd8fd-28b3-4f97-a909-02c26085266c compute Passed 1741252377283-zfssa-81712 UWS-910dd8fd-28b3-4f97-a909-02c26085266c zfssa Passed
-
To investigate a failure, or display more detailed information about a particular job, use the
getUpgradeJobcommand. The job details include the path to the log file, the complete task list, and error messages captured during the run.PCA-ADMIN> getUpgradeJob upgradeJobId=1741269765197-platform-663307 Data: Upgrade Request Id = UWS-910dd8fd-28b3-4f97-a909-02c26085266c Composition Id = fullrack Name = platform Pid = 663307 Host = pcamn01 Log File = /nfs/shared_storage/pca_upgrader/log/pca-upgrader_platform_services_2025_03_06-14.02.45.log Arguments = {"component_names":null,"diagnostics":false,"display_task_plan":false,"dry_run_tasks":false,"expected_iso_checksum":null,[...]} Status = Failed Execution Time(sec) = 4068 Tasks 1 - Name = Validate Upgrader Version [...] Tasks 31 - Name = Network Controllers Upgrade Tasks 31 - Description = Upgrade Network Controllers. Tasks 31 - Message = [{'name': 'auxiliary-metadata', 'version': 2, [...] Error: UPGRADE FAILED: release pcaswitchmgr failed, and has been rolled back due to atomic being set: pre-upgrade hooks failed: timed out waiting for the condition }] Tasks 31 - Status = Failed Tasks 31 - Reason Phrase = Failed Tasks 31 - Execution Time(sec) = 1834 [...]
-