Patching a Compute Node
Caution:
Ensure that all preparation steps for system patching have been completed. For instructions, see Prepare for Patching.
When patching to appliance software version 3.0.2-b1081557 or later, the ZFS Storage Appliance firmware must be patched before all other components. For more information, see Checking Upgrade Plan Status and Progress.
The compute node patching ensures that the latest Oracle Linux kernel and user space packages are installed,
as well as the ovm-agent
package with appliance-specific optimizations.
Compute nodes must be provisioned and locked, then patched one at a time, concurrent patches
are not supported. After a successful patch, when a compute node has rebooted, the
administrator must manually remove the locks to allow the node to return to normal
operation.
Ensure synchronization of the mirror on the shared storage is complete prior to compute node
patching by issuing the syncUpstreamUlnMirror
command. For more information,
see Prepare for Patching.
Note:
In case the ILOM also needs to be patched, you can integrate it into this procedure by executing the optional steps. The combined procedure eliminates the need to evacuate and reboot the same node twice.
Note:
In software versions 3.0.2-b892153 and later all patch operations are based on the upgrade
plan, which is generated when the pre-upgrade command is executed. For more information, see
Prepare for Patching. When a component is already at the
required version, the patch operation is skipped. However, patching with the same version
can be forced using the Service Web UI or Service CLI command option
(force=True
), if necessary.
Obtaining a Host IP Address
From the Service CLI, compute nodes are patched one at a time, using each one's internal IP address as a command parameter. However, the locking commands use the compute node ID instead. To run all commands for a compute node patch procedure you need both identifiers.
To obtain the host IP address and ID, as well as other information relevant to the patch procedure, use the Service CLI command provided in the following example. You can run the command as often as needed to check and confirm status as you proceed through the upgrade of all compute nodes.
PCA-ADMIN> list computeNode fields hostname,ipAddress,ilomIp,state,firmwareVersion,provisioningLocked,maintenanceLocked orderby hostname ASCENDING Data: id Hostname Ip Address ILOM Ip Address State Firmware Version Provisioning Locked Maintenance Locked -- -------- ---------- --------------- ----- ---------------- ------------------- ------------------ cf488903-fef8-4a51-8a41-c6990e4755c5 pcacn001 100.96.2.64 100.96.0.64 On PCA Hypervisor:3.0.2-681 false false 42a7594d-1173-4dbd-4755-07810cc2d527 pcacn002 100.96.2.65 100.96.0.65 On PCA Hypervisor:3.0.2-681 false false bc0f37d5-ba77-423e-bc11-017704b47e59 pcacn003 100.96.2.66 100.96.0.66 On PCA Hypervisor:3.0.2-681 false false 2e5ac527-01f5-4230-ae41-0522fcb57c9a pcacn004 100.96.2.67 100.96.0.67 On PCA Hypervisor:3.0.2-681 false false 5a6b61cf-7e99-4df2-87e4-b37c5fb0bfb8 pcacn005 100.96.2.68 100.96.0.68 On PCA Hypervisor:3.0.2-681 false false 885f2aa4-f017-41e8-b2bc-e588cc0c6162 pcacn006 100.96.2.69 100.96.0.69 On PCA Hypervisor:3.0.2-681 false false
Using the Service Web UI
-
Set the provisioning and maintenance locks for the compute node you are about to patch. Ensure that no active compute instances are present on the node.
Caution:
Depending on the high-availability configuration of the Compute service, automatic instance migrations can prevent you from successfully locking a compute node. For more information, refer to the following sections in the Hardware Administration chapter of the Oracle Private Cloud Appliance Administrator Guide:
- Migrating instances and locking a compute node: see "Performing Compute Node Operations".
-
Compute service HA configuration: see "Configuring the Compute Service for High Availability".
-
In the navigation menu, click Rack Units. In the Rack Units table, click the name of the compute node you want to patch to display its detail page.
-
In the top-right corner of the compute node detail page, click Controls and select the Provisioning Lock command.
-
When the provisioning lock has been set, click Controls again and select the Migrate All Vms command. The Compute service evacuates the compute node, meaning it migrates the running instances to other compute nodes.
-
When compute node evacuation is complete, click Controls again and select the Maintenance Lock command. This command might fail if instance migrations are in progress. Wait a few minutes and retry.
-
In the navigation menu, click Upgrade & Patching.
-
Optionally, patch the server ILOM first.
-
In the top-right corner of the Upgrade Jobs page, click Create Upgrade or Patch. The Create Request window appears.
-
Choose Patch as the Request Type. Select the appropriate patch request type: Patch ILOM.
Fill out the server's assigned IP address in the ILOM network. This is an IP address in the internal 100.96.0.0/23 range.
-
Click Create Request. The new patch request appears in the Upgrade Jobs table.
-
Wait 5 minutes to allow the ILOM patch job to complete. Then proceed to patching the host.
-
-
In the top-right corner of the Upgrade Jobs page, click Create Upgrade or Patch.
The Create Request window appears. Choose Patch as the Request Type.
-
Select the appropriate patch request type: Patch CN.
-
If required, fill out the request parameters:
-
Host IP: Enter the compute node's assigned IP address in the internal administration network. This is an IP address in the internal 100.96.2.0/23 range.
-
ULN: Enter the fully qualified domain name of the ULN mirror in your data center. This parameter is deprecated in software version 3.0.2-b892153 and later.
-
Log Level: Optionally, select a specific log level for the upgrade log file. The default log level is "Information". For maximum detail, select "Debug".
-
Advanced Options JSON: Not available.
-
-
Click Create Request.
The new patch request appears in the Upgrade Jobs table.
-
When the compute node has been patched successfully, release the provisioning and maintenance locks.
For more information, refer to the section "Performing Compute Node Operations". It can be found in the chapter Hardware Administration of the Oracle Private Cloud Appliance Administrator Guide.
-
Open the compute node detail page.
-
In the top-right corner of the compute node detail page, click Controls and select the Maintenance Unlock command.
-
When the maintenance lock has been released, click Controls again and select the Provisioning Unlock command.
-
Using the Service CLI
-
From the output you obtained with the compute node list command earlier, get the ID and the IP address of the compute node you intend to patch.
-
Set the provisioning and maintenance locks for the compute node you are about to patch.
Caution:
Depending on the high-availability configuration of the Compute service, automatic instance migrations can prevent you from successfully locking a compute node. For more information, refer to the following sections in the Hardware Administration chapter of the Oracle Private Cloud Appliance Administrator Guide:
- Migrating instances and locking a compute node: see "Performing Compute Node Operations".
-
Compute service HA configuration: see "Configuring the Compute Service for High Availability".
-
Disable provisioning for the compute node.
PCA-ADMIN> provisioningLock id=cf488903-fef8-4a51-8a41-c6990e4755c5 Status: Success JobId: 6ee78c8a-e227-4d31-a770-9b9c96085f3f
-
Evacuate the compute node. Wait for the migration job to finish before proceeding to the next step.
PCA-ADMIN> migrateVm id=cf488903-fef8-4a51-8a41-c6990e4755c5 force=true Status: Running JobId: 6f1e94bc-7d5b-4002-ada9-7d4b504a2599 PCA-ADMIN> show Job id=6f1e94bc-7d5b-4002-ada9-7d4b504a2599 Run State = Succeeded
-
Lock the compute node for maintenance.
PCA-ADMIN> maintenanceLock id=cf488903-fef8-4a51-8a41-c6990e4755c5 Status: Success JobId: e46f6603-2af2-4df4-a0db-b15156491f88
-
Optionally, rerun the compute node list command to confirm lock status. For example:
PCA-ADMIN> list computeNode fields hostname,ipAddress,ilomIp,state,firmwareVersion,provisioningLocked,maintenanceLocked orderby hostname ASCENDING Data: id Hostname Ip Address ILOM Ip Address State Firmware Version Provisioning Locked Maintenance Locked -- -------- ---------- --------------- ----- ---------------- ------------------- ------------------ cf488903-fef8-4a51-8a41-c6990e4755c5 pcacn001 100.96.2.64 100.96.0.64 On PCA Hypervisor:3.0.2-681 true true 42a7594d-1173-4dbd-4755-07810cc2d527 pcacn002 100.96.2.65 100.96.0.65 On PCA Hypervisor:3.0.2-681 false false bc0f37d5-ba77-423e-bc11-017704b47e59 pcacn003 100.96.2.66 100.96.0.66 On PCA Hypervisor:3.0.2-681 false false 2e5ac527-01f5-4230-ae41-0522fcb57c9a pcacn004 100.96.2.67 100.96.0.67 On PCA Hypervisor:3.0.2-681 false false 5a6b61cf-7e99-4df2-87e4-b37c5fb0bfb8 pcacn005 100.96.2.68 100.96.0.68 On PCA Hypervisor:3.0.2-681 false false 885f2aa4-f017-41e8-b2bc-e588cc0c6162 pcacn006 100.96.2.69 100.96.0.69 On PCA Hypervisor:3.0.2-681 false false
-
Optionally, patch the server ILOM first.
- Enter the ILOM patch command.
Syntax (entered on a single line):
patchIlom hostIp=<ilom-ip>
Example:
PCA-ADMIN> patchIlom hostIp=100.96.0.64 Data: Service request has been submitted. Upgrade Job Id = 1620921089806-ilom-21480 Upgrade Request Id = UWS-732d6fce-9f06-4329-b972-d093bee40010 PCA-ADMIN> getUpgradeJob upgradeJobId=1620921089806-ilom-21480
- Wait 5 minutes to allow the ILOM patch job to complete. Then proceed to patching the host.
- Enter the ILOM patch command.
-
Enter the compute node patch command.
Syntax (entered on a single line):
patchCN hostIp=<compute-node-ip> [optional] uln=<http|https>://<hostname.domainname>/<sub-directories>
The parameter marked optional is deprecated in software version 3.0.2-b892153 and later. For earlier versions, include the fully qualified domain name of the ULN mirror with the command.
Example:
PCA-ADMIN> patchCN hostIp=100.96.2.64 ULN=http://host.example.com/yum Status: Success Data: Service request has been submitted. Upgrade Job ID = 1685372050358-compute-50568 Upgrade Request ID = UWS-f226d7d2-549d-4902-8614-e1f40bdc9ff6
-
Use the request ID and the job ID to check the status of the patching process.
PCA-ADMIN> getUpgradeJobs Command: getUpgradeJobs Status: Success Time: 2023-01-01 21:09:34.745 UTC Data: id upgradeRequestId commandName result -- ---------------- ----------- ------ 1685372050358-compute-50568 UWS-f226d7d2-549d-4902-8614-e1f40bdc9ff6 compute Passed PCA-ADMIN> getUpgradeJob upgradeJobId=1685372050358-compute-50568 Command: getUpgradeJob upgradeJobId=1685372050358-compute-50568 Status: Success Time: 2023-01-01 21:10:13,804 UTC Data: Upgrade Request Id = UWS-f226d7d2-549d-4902-8614-e1f40bdc9ff6 Name = compute [...]
-
When the compute node patch has completed successfully and the node has rebooted, release the locks.
For more information, refer to "Performing Compute Node Operations" in the Hardware Administration section of the Oracle Private Cloud Appliance Administrator Guide.
-
Release the maintenance lock.
PCA-ADMIN> maintenanceUnlock id=cf488903-fef8-4a51-8a41-c6990e4755c5 Status: Success JobId: 625af20e-4b49-4201-879f-41d4405314c7
-
Release the provisioning lock.
PCA-ADMIN> provisioningUnlock id=cf488903-fef8-4a51-8a41-c6990e4755c5 Status: Success JobId: 523892e8-c2d4-403c-9620-2f3e94015b46
-
-
Proceed to the next compute node and repeat this procedure.
The output from the compute node list command indicates the current status. For example:
PCA-ADMIN> list computeNode fields hostname,ipAddress,ilomIp,state,firmwareVersion,provisioningLocked,maintenanceLocked orderby hostname ASCENDING Data: id Hostname Ip Address ILOM Ip Address State Firmware Version Provisioning Locked Maintenance Locked -- -------- ---------- --------------- ----- ---------------- ------------------- ------------------ cf488903-fef8-4a51-8a41-c6990e4755c5 pcacn001 100.96.2.64 100.96.0.64 On PCA Hypervisor:3.0.2-696 false false 42a7594d-1173-4dbd-4755-07810cc2d527 pcacn002 100.96.2.65 100.96.0.65 On PCA Hypervisor:3.0.2-696 false false bc0f37d5-ba77-423e-bc11-017704b47e59 pcacn003 100.96.2.66 100.96.0.66 On PCA Hypervisor:3.0.2-696 false false 2e5ac527-01f5-4230-ae41-0522fcb57c9a pcacn004 100.96.2.67 100.96.0.67 On PCA Hypervisor:3.0.2-696 false false 5a6b61cf-7e99-4df2-87e4-b37c5fb0bfb8 pcacn005 100.96.2.68 100.96.0.68 On PCA Hypervisor:3.0.2-681 false false 885f2aa4-f017-41e8-b2bc-e588cc0c6162 pcacn006 100.96.2.69 100.96.0.69 On PCA Hypervisor:3.0.2-681 false false