Patching Oracle Cloud Infrastructure Images

Caution:

Ensure that all preparation steps for system patching have been completed. For instructions, see Prepare for Appliance Patching.

When new Oracle Cloud Infrastructure Images become available and supported for Oracle Private Cloud Appliance, you can pick up these images using the patching process.

Oracle Cloud Infrastructure Images installed using the patching method are stored in the /nfs/shared_storage/oci_compute_images directory on the ZFS storage appliance.

Using the Service Web UI

In the navigation menu, go to the Maintenance section and click Upgrade Plan. This provides an overview of current and target component versions.
Click Upgrade & Patching to display the Upgrade Jobs page.
In the top-right corner of the Upgrade Jobs page, click Create Upgrade or Patch.

The Create Request window appears. Choose Patch as the Request Type.
Select the appropriate patch request type: Patch OCIImages.
If required, fill out the request parameters:
- Advanced Options JSON: Not available.
- Log Level: Optionally, select a specific log level for the upgrade log file. The default log level is "Information". For maximum detail, select "Debug".
- Alternative ULN Channel: This parameter forces the request to use a non-standard ULN channel. Do not use this option unless Oracle explicitly instructs you to do so.
- Verify Only: Enable this option to run the operation in verification only mode.
- Force: Enable this option to force the operation. Use only when instructed by Oracle.
Click Create Request.

The new patch request appears in the Upgrade Jobs table.

Using the Service CLI

Enter the patch command.

PCA-ADMIN> patchOCIImages
Command: patchOCIImages
Status: Success
Time: 2023-01-18 19:33:09,756 UTC
Data:
  Service request has been submitted. Upgrade Job Id = 1641839285475-oci-94665 \
Upgrade Request Id = UWS-778b08bc-f579-492b-993d-915dcf581374

Use the request ID and the job ID to check the status of the patching process.

PCA-ADMIN> getupgradejobs
Command: getupgradejobs
Status: Success
Time: 2023-01-18 22:38:51,764 UTC
Data:
  id                             upgradeRequestId                           commandName   result
  --                             ----------------                           -----------   ------
  1641839285475-oci-94665        UWS-778b08bc-f579-492b-993d-915dcf581374   oci           Passed
  1641838937541-platform-56313   UWS-bc4372ae-8f51-4b40-9306-992fb6459878   platform      Passed
  
PCA-ADMIN> getUpgradeJob upgradeJobId=1680260388058-oci-94665
Command: getUpgradeJob upgradeJobId=1680260388058-oci-94665
Status: Success
Time: 2023-01-18 23:03:22,769 UTC
Data:
  Upgrade Request Id = UWS-778b08bc-f579-492b-993d-915dcf581374
  Name = oci
[...]

Resolving Security Vulnerabilities in OKE Clusters

At the end of the Oracle Cloud Infrastructure images upgrade or patch process, the Upgrader launches a background job to resolve any known CVEs that might affect existing clusters deployed through the Oracle Private Cloud Appliance Kubernetes Engine (OKE). When the new images have been imported, an OKE Service tool ensures that the running control plane nodes receive the latest available CVE fixes delivered with the new images.

The CVE fixes are applied in a fully automated way, but the process could be derailed by timing issues in the appliance upgrade or patching workflow. Thus, it is important for an appliance administrator to monitor the OKE background job and verify that CVE fixes have been applied successfully to all existing OKE clusters. Note that an error in OKE cluster patching will NOT cause the appliance upgrade or patching process to fail.

The status of the Oracle Cloud Infrastructure images upgrade or patch process indicates that the OKE Service tool has been run. It also provides the OCIDs of the OKE clusters found, and any work requests for cluster patching operations that should be tracked.

getUpgradeJob upgradeJobId=1724442488245-oci-35655
Data:
  Log File = /nfs/shared_storage/pca_upgrader/log/pca-upgrader_oci_instance_images_<date>-<time>.log
[...]
  Tasks 12 - Message = OKE Clusters CVE Patching initiated:
  {"ocid1.cluster.<AK01234567>.<mypca>.63f7764a345d4d74a9abd5267ad55a28p6ixuw4ejzr73yugynu4lrwbcaao": "No operations performed", 
  "ocid1.cluster.<AK01234567>.<mypca>.ac198ab8583848e8947501f7061bde16mx17lm2u2rugld6u3ujxthgnygsj": "ocid1.workrequest.<AK01234567>.<mypca>.oke-g8l7kh306zlt59zb9vc4yvo532b1j4jwtffnmel83v1qif0q93lum7er"}

In the example, two active OKE clusters are found. The first cluster is using the latest image and does not need to be patched. The second cluster is out of date and needs to be updated with the latest available image. Use the work request to track the cluster update status from the OCI CLI, using the command: oci ce work-request get --work-request-id <workrequest_OCID>.

# oci ce work-request get --work-request-id ocid1.workrequest.<AK01234567>.<mypca>.oke-g8l7kh306zlt59zb9vc4yvo532b1j4jwtffnmel83v1qif0q93lum7er
{
  "data": {
    "compartment-id": "ocid1.compartment.<AK01234567>.<mypca>.ezbf00rrfc0qnoi8rofk3yzcbq0yeg9ly0gzf6caebv3ugogzm1v3qww5q9f",
    "id": "ocid1.workrequest.<AK01234567>.<mypca>.oke-g8l7kh306zlt59zb9vc4yvo532b1j4jwtffnmel83v1qif0q93lum7er",
    "operation-type": "UNKNOWN_ENUM_VALUE",
    "resources": [
      {
        "action-type": "UPDATED",
        "entity-type": "CLUSTER",
        "entity-uri": null,
        "identifier": "ocid1.cluster.<AK01234567>.<mypca>.ac198ab8583848e8947501f7061bde16mx17lm2u2rugld6u3ujxthgnygsj"
      }
    ],
    "status": "SUCCEEDED",
    "time-accepted": "2024-09-03T11:18:29.750438+00:00",
    "time-finished": "2024-09-03T11:36:19.313926+00:00",
    "time-started": "2024-09-03T11:18:36.451513+00:00"
  },
  "etag": "00fa0a51-a9dd-5455-f390-429a20817d6d"
}

If errors have occurred, and certain clusters were not updated based on the latest available image, first ensure that the cluster is in a good working state, then run the following command from one of the management nodes:

# kubectl exec -it -n oke <oke_pod_name> -c oke -- pca-oke-cluster-tool --action patch-cluster-cve