Handling Missing Disks, File Systems and Repositories

On a small number of systems, local physical disks may be allocated an alternate device mapper ID during the upgrade process. This only affects a few different ATA type disks. The change in device mapper ID can result in some local physical disks, their file systems and repositories as being marked as missing within Oracle VM Manager. Although the physical disk is discovered after the upgrade, it is detected as a new disk. The old entry within Oracle VM Manager, for the same disk, is retained but is marked as missing.

If you are affected by this, you may need to take some steps to reconfigure your environment for the change. This includes removing the missing disks and then refreshing the newly discovered disks, their file systems and any hosted repositories. The repository must additionally be presented to the affected Oracle VM Server again, so that it can be used.

If you have virtual machines that use local physical disks for storage, you may need to reconfigure these virtual machines to remove the missing disk and replace it with the newly discovered disk.

This section describes how to identify whether you are affected and the steps that you may need to take to return to a fully functioning environment.

Checking For Missing Disks

To see whether you are affected by this issue at all, you can use the Oracle VM Manager Web Interface to view the Physical Disks perspective for each Oracle VM Server in your environment. If a disk has been affected by the upgrade, it is displayed with a WARNING status. The event message shows "Physical disk is Offline".

In an environment with many Oracle VM Servers, it may not be practical to use the Oracle VM Manager Web Interface to check the status of the physical disks for every server. You can, however, use the Oracle VM Manager Command Line Interface in combination with the grep utility to quickly view all of the events that contain this event message. An example, where the command is run on the Oracle VM Manager host, follows:

$ ssh -l admin localhost -p 10000 "getEvents severity=WARNING" | grep -i "physical \
  disk is offline"
id:3190     createTime:May 09, 2014 2:02:00 pm     modTime:May 09, 2014 2:02:00 pm     
type:storage.device.offline.     severity:Warning     summary:Physical disk is 
Offline     acked:No description:OVMEVT_007005D_001 Rescan storage layer on 
server [ovs237] did not return physical disk [3600605b0057feba01a0e216512d1b7aa] 
for storage array [Generic Local Storage Array @ ovs237]. 

Note that for these events, you must check that the event occurs for a Generic Local Storage Array. The server that is affected is displayed in the output, as is the id for the missing disk.

If this event message exists for any of the local physical disks attached to any of your Oracle VM Servers, then you are most likely affected by this issue and should continue with the rest of the steps described here.

Identifying Newly Discovered Disks and Matching Them To Missing Disks

By default, the simple names that Oracle VM Manager assigns to physical disks are based on the disk Page83 ID. If you have not renamed your physical disks, identifying newly discovered disks that map onto missing disks is simple, since the disk names can be compared to find matches. This is illustrated in the following output from the Oracle VM Manager Command Line Interface:

OVM> list physicalDisk
Command: list physicalDisk
Status: Success
Time: 2014-05-06 13:40:49,503 PDT

In this output, the first physical disk listed is the original disk item, while the second disk is the newly discovered version of the same disk. The first physical disk is marked as missing in Oracle VM Manager. Since the second disk has the same name, with the exclusion of the final underscore character, it is easy to identify that these entries actually point to the same disk.

If you have changed the names for your physical disks, this process is fractionally more complicated since you must compare the Page83 ID of each disk to find the match. Obtaining the Page83 ID of a disk using the Oracle VM Manager Command Line Interface is illustrated below:

OVM> show physicaldisk id=0004fb00001800009eed86fbc36b41bc
Command: show physicaldisk id=0004fb00001800009eed86fbc36b41bc
Status: Success
Time: 2014-05-13 13:45:56,732 CEST
  Page83 ID = 350014ee20137ee44
  Server Reserved = No
  Shareable = No
  Size (GiB) = 465.76
  State = UNKNOWN
  Thin Provision = No
  Type = LUN
  Vendor = ATA
  File System 1 = 0004fb00000500000809e28f4fab56b1  [fs on 350014ee20137ee44]
  Volume Group = 0004fb00003200004dddc710a12aa1b7  [Local Storage Volume Group]
  Id = 0004fb00001800009eed86fbc36b41bc  [Server5 ATA Disk]
  Name = Server5 ATA Disk
  Description = WDC WD5001ABYS-0
  Locked = false

Note that the Page83 ID is shown as a property of the disk. You must obtain the Page83 ID for each missing disk and then find any physical disks that have their name or Page83 ID set to the same value. These disks are the newly discovered versions of the same physical disk.

Useful Steps Before Reconfiguring Your Environment

The names of any child objects of the missing disk, such as file system names, repository names and the names of any items within the repository are all associated with the missing disk. This information is lost when you delete the missing disk from your environment. If you have named any of these objects within Oracle VM Manager, you should take note of the names of these objects before continuing. Although this is not a necessary step, since the names are only helpful within Oracle VM Manager and do not affect the running of your environment, to return your environment to the same state after you delete any missing disks you may wish to rename these objects once they are associated with the newly discovered version of the same physical disk.

Using the Oracle VM Manager Command Line Interface you can run the following commands to obtain a dump of the names of all of these objects:

OVM> list physicaldisk
OVM> list filesystem
OVM> list repository
OVM> show repository id=0004fb0000030000e0f09683cd2ac72a

Substitute 0004fb0000030000e0f09683cd2ac72a with the id for each repository in your environment, and run the same command for each repository.

The goal of this exercise is to obtain a listing of the names of any of the objects that may be affected when you remove the physical disk that they are associated with. Store the information returned by these commands in a place that you can refer to when you need to rename objects.

Steps To Deal With Affected Repositories

If you have made a record of the object names of all affected objects, it is safe to delete the missing disks, as illustrated in the following output from the Oracle VM Manager Command Line Interface:

OVM> delete physicalDisk id=0004fb00001800007b2cdfbecb973c4d
Command: delete physicalDisk id=0004fb00001800007b2cdfbecb973c4d
Status: Success
Time: 2014-05-06 13:43:55,253 PDT
JobId: 1399409030129

These actions can also be performed in the Oracle VM Manager Web Interface, if preferred.

Once you have deleted the missing disk entry, you must perform the following actions:

At this point, your environment is usable, although the names of various objects have been lost. For instance, any file system names, repository names, or the names of objects such as virtual machines in a repository are likely to have been reset to default values. If you kept a record of the object names affected, as recommended in the previous step, then you can manually rename objects as required.

Steps To Deal With Affected Virtual Machines

If you have any virtual machines that were configured to use a local physical disk that is marked as missing, you must remove the original disk mapping and then reconfigure the virtual machine to attach a new physical disk. When you select the new physical disk, you should select the physical disk with the same Page83 ID as the missing disk. To reconfigure these disk mappings, you must edit the virtual machine as described in Edit Virtual Machine in the Oracle VM Manager User's Guide.