This section describes different post-upgrade steps that may be required after an upgrade.
After an upgrade, it is good practice to refresh your server pools to ensure that the information presented within Oracle VM Manager is up to date and synchronized with the actual state of any servers in a server pool. More importantly, if you are using any third party Oracle VM Storage Connect plug-in for your storage devices, you may be unable to refresh your storage arrays until these plug-ins have been validated. Refreshing server pools allows the servers that have been presented to the storage to validate the Oracle VM Storage Connect plug-in that they are using. Once this has been done, operations on storage arrays can resume normally.
To refresh your server pools, refer to Refresh All in the Oracle VM Manager User's Guide.
If you have upgraded from an earlier release, it is possible that some of the hypervisor
boot parameters for your server may need editing to ensure that newer features are enabled. To
update these boot parameters, on each affected Oracle VM Server, open the grub configuration in an
editor. If you are using UEFI boot, the grub configuration file is located at
/boot/efi/EFI/redhat/grub.cfg
, otherwise the grub configuration file is
located at /boot/grub2/grub.cfg
. Edit the line starting with
multiboot2 /xen.mb.efi
and append the required boot parameters.
For large servers with more than 20 physical CPUs
For Oracle VM Servers with more than 20 physical CPUs, performance can be improved by pinning
virtual CPUs to dom0 and setting a maximum value for the number of virtual CPUs that can be
used by dom0. On a new install where a system has more than 20 physical CPUs, this value is
usually set to 20.
To ensure that
you get the best possible performance on an Oracle VM Server with more than 20 physical CPUs, set the
Xen boot parameters dom0_vcpus_pin
and dom0_max_vcpus=20
.
For example:
multiboot2 /xen.mb.efi dom0_mem=max:6144M dom0_vcpus_pin \ dom0_max_vcpus=20 placeholder ${xen_rm_opts}
Some versions of the Xen 4.4.4 kernel require that you set the
dom0_vcpus_pin
parameter to dom0_vcpus_pin=true
for
the configuration to take effect. Run the following command on Oracle VM Server to find the Xen
version: rpm -qa | grep xen.
If you have previously updated the Xen boot parameters on an earlier Oracle VM 3.4.x release, and then upgraded to Oracle VM Release 3.4.5 or later with Xen kernel version 4.4.4-130 or higher, either syntax for the parameter is accepted.
The standard syntax as of Oracle VM Release 3.4.5 is
dom0_vcpus_pin
; there is no need to append =true
.
For Oracle VM Servers with fewer than 20 physical CPUs, you should ensure
that the number of virtual CPUs specified is reduced accordingly
and does not exceed the number of physical CPUs per socket.
Assigning too many virtual CPUs can cause Non-Uniform Memory
Access (NUMA) latency issues if the virtual CPUs are spread across
sockets. For example, if an instance of Oracle VM Server with
hyper-threading enabled has 2 sockets with 18 cores per socket and
2 CPU threads per core, this results in a total of 36 physical
CPUs. In this case, it is correct to set
dom0_max_vcpus
to 20. However, with
hyper-threading disabled, only 18 physical CPUs are on each socket
and as a result, you should set the
dom0_max_vcpus
parameter to 18.
When you have finished modifying the grub configuration file, for
example, /etc/default/grub
, you must rebuild the
system GRUB2 configuration that is used at boot time. This is done
by running:
# grub2-mkconfig -o /boot/grub2/grub.cfg
You must also reboot the server for these settings to take effect.
For information on performance optimization goals and techniques for Oracle VM Server for x86, see Optimizing Oracle VM Server for x86 Performance, on Oracle Technology Network at: http://www.oracle.com/technetwork/server-storage/vm/ovm-performance-2995164.pdf.
If the kdump service was enabled previously, it is recommended that you check and confirm the settings are correct after the upgrade or enable the kdump service after the upgrade if it was not enabled previously.
For more information, see Manually Configuring kdump for Oracle VM Server in the Oracle VM Administrator's Guide.
On a small number of systems, local physical disks may be allocated an alternate device mapper ID during the upgrade process. This only affects a few different ATA type disks. The change in device mapper ID can result in some local physical disks, their file systems and repositories as being marked as missing within Oracle VM Manager. Although the physical disk is discovered after the upgrade, it is detected as a new disk. The old entry within Oracle VM Manager, for the same disk, is retained but is marked as missing.
If you are affected by this, you may need to take some steps to reconfigure your environment for the change. This includes removing the missing disks and then refreshing the newly discovered disks, their file systems and any hosted repositories. The repository must additionally be presented to the affected Oracle VM Server again, so that it can be used.
If you have virtual machines that use local physical disks for storage, you may need to reconfigure these virtual machines to remove the missing disk and replace it with the newly discovered disk.
This section describes how to identify whether you are affected and the steps that you may need to take to return to a fully functioning environment.
To see whether you are affected by this issue at all, you can use the Oracle VM Manager Web Interface to view the Physical Disks perspective for each Oracle VM Server in your environment. If a disk has been affected by the upgrade, it is displayed with a WARNING status. The event message shows "Physical disk is Offline".
In an environment with many Oracle VM Servers, it may not be practical to use the Oracle VM Manager Web Interface to check the status of the physical disks for every server. You can, however, use the Oracle VM Manager Command Line Interface in combination with the grep utility to quickly view all of the events that contain this event message. An example, where the command is run on the Oracle VM Manager host, follows:
$ ssh -l admin localhost -p 10000 "getEvents severity=WARNING" | grep -i "physical \ disk is offline" id:3190 createTime:timestamp
modTime:timestamp
type:storage.device.offline. severity:Warning summary:Physical disk is Offline acked:No description:OVMEVT_007005D_001 Rescan storage layer on server [ovs237] did not return physical disk [3600605b0057feba01a0e216512d1b7aa] for storage array [Generic Local Storage Array @ ovs237]. assocObjectId:0004fb0000180000833252939ba41cf9
Note that for these events, you must check that the event occurs for a Generic Local Storage Array. The server that is affected is displayed in the output, as is the id for the missing disk.
If this event message exists for any of the local physical disks attached to any of your Oracle VM Servers, then you are most likely affected by this issue and should continue with the rest of the steps described here.
By default, the simple names that Oracle VM Manager assigns to physical disks are based on the disk Page83 ID. If you have not renamed your physical disks, identifying newly discovered disks that map onto missing disks is simple, since the disk names can be compared to find matches. This is illustrated in the following output from the Oracle VM Manager Command Line Interface:
OVM> list physicalDisk
Command: list physicalDisk
Status: Success
Time: timestamp
Data:
id:0004fb00001800007b2cdfbecb973c4d
name:SATA_VBOX_HARDDISK_VBcdfe5359-5146ac3d_
id:0004fb0000180000f9c966f19c14cc95
name:SATA_VBOX_HARDDISK_VBcdfe5359-5146ac3d
In this output, the first physical disk listed is the original disk item, while the second disk is the newly discovered version of the same disk. The first physical disk is marked as missing in Oracle VM Manager. Since the second disk has the same name, with the exclusion of the final underscore character, it is easy to identify that these entries actually point to the same disk.
If you have changed the names for your physical disks, this process is fractionally more complicated since you must compare the Page83 ID of each disk to find the match. Obtaining the Page83 ID of a disk using the Oracle VM Manager Command Line Interface is illustrated below:
OVM> show physicaldisk id=0004fb00001800009eed86fbc36b41bc
Command: show physicaldisk id=0004fb00001800009eed86fbc36b41bc
Status: Success
Time: timestamp
Data:
Page83 ID = 350014ee20137ee44
Server Reserved = No
Shareable = No
Size (GiB) = 465.76
State = UNKNOWN
Thin Provision = No
Type = LUN
Vendor = ATA
File System 1 = 0004fb00000500000809e28f4fab56b1 [fs on 350014ee20137ee44]
Volume Group = 0004fb00003200004dddc710a12aa1b7 [Local Storage Volume Group]
Id = 0004fb00001800009eed86fbc36b41bc [Server5 ATA Disk]
Name = Server5 ATA Disk
Description = WDC WD5001ABYS-0
Locked = false
Note that the Page83 ID is shown as a property of the disk. You must obtain the Page83 ID for each missing disk and then find any physical disks that have their name or Page83 ID set to the same value. These disks are the newly discovered versions of the same physical disk.
The names of any child objects of the missing disk, such as file system names, repository names and the names of any items within the repository are all associated with the missing disk. This information is lost when you delete the missing disk from your environment. If you have named any of these objects within Oracle VM Manager, you should take note of the names of these objects before continuing. Although this is not a necessary step, since the names are only helpful within Oracle VM Manager and do not affect the running of your environment, to return your environment to the same state after you delete any missing disks you may wish to rename these objects once they are associated with the newly discovered version of the same physical disk.
Using the Oracle VM Manager Command Line Interface you can run the following commands to obtain a dump of the names of all of these objects:
OVM> list physicaldisk
OVM> list filesystem
OVM> list repository
OVM> show repository id=0004fb0000030000e0f09683cd2ac72a
Substitute
0004fb0000030000e0f09683cd2ac72a
with
the id for each repository in your environment, and run the same
command for each repository.
The goal of this exercise is to obtain a listing of the names of any of the objects that may be affected when you remove the physical disk that they are associated with. Store the information returned by these commands in a place that you can refer to when you need to rename objects.
If you have made a record of the object names of all affected objects, it is safe to delete the missing disks, as illustrated in the following output from the Oracle VM Manager Command Line Interface:
OVM> delete physicalDisk id=0004fb00001800007b2cdfbecb973c4d
Command: delete physicalDisk id=0004fb00001800007b2cdfbecb973c4d
Status: Success
Time: timestamp
JobId: 1399409030129
These actions can also be performed in the Oracle VM Manager Web Interface, if preferred.
Once you have deleted the missing disk entry, you must perform the following actions:
Identify the newly discovered disk that corresponds to the disk that you have deleted, and refresh it as described in Refresh Physical Disk in the Oracle VM Manager User's Guide.
Now, refresh the file systems on the disk as described in Refresh File System in the Oracle VM Manager User's Guide.
Next, you must refresh the repositories that are hosted on the local physical disk, as described in Refresh Repository in the Oracle VM Manager User's Guide.
Finally, present the repository to the Oracle VM Server where the physical disk is located. See Present or Unpresent Repository in the Oracle VM Manager User's Guide for more information.
At this point, your environment is usable, although the names of various objects have been lost. For instance, any file system names, repository names, or the names of objects such as virtual machines in a repository are likely to have been reset to default values. If you kept a record of the object names affected, as recommended in the previous step, then you can manually rename objects as required.
If you have any virtual machines that were configured to use a local physical disk that is marked as missing, you must remove the original disk mapping and then reconfigure the virtual machine to attach a new physical disk. When you select the new physical disk, you should select the physical disk with the same Page83 ID as the missing disk. To reconfigure these disk mappings, you must edit the virtual machine as described in Edit Virtual Machine in the Oracle VM Manager User's Guide.