Scenario 2: Re-imaging the Management Domain and Restoring User Domains from Backups

5.20.3 Scenario 2: Re-imaging the Management Domain and Restoring User Domains from Backups

This procedure re-images the management domain and reconstructs all the user domains.

The following procedure can be used when the management domain is damaged beyond repair and no backup exists for the management domain, but there is a backup available of the storage repository (/EXAVMIMAGES file system) housing all the user domains.

Re-image the management domain with the image used in the other management domains in the rack using the procedure described in Re-Imaging the Oracle Exadata Database Server.

Run the following commands:

# /opt/oracle.SupportTools/switch_to_ovm.sh

# /opt/oracle.SupportTools/reclaimdisks.sh –free –reclaim

If the recovery is on Oracle Exadata Database Machine eighth rack, then perform the procedure described in Configuring Oracle Exadata Database Machine Eighth Rack Oracle Linux Database Server After Recovery.

Rebuild the OCFS2 file system on the /dev/sda3 partition.

# umount /EXAVMIMAGES

# mkfs -t ocfs2 -L ocfs2 -T vmstore --fs-features=local /dev/sda3 --force

Mount the OCFS2 partition /dev/sda3 on /EXAVMIMAGES.
```
# mount -t ocfs2 /dev/sda3 /EXAVMIMAGES
```

Mount the backup NFS server to restore the /EXAVMIMAGES file system which holds the user domain images.

# mkdir -p /remote_FS

# mount -t nfs -o ro,intr,soft,proto=tcp,nolock nfs_ip:/location_of_backup /remote_FS

Restore the /EXAVMIMAGES file system.
```
# tar -Spxvf /remote_FS/backup-of-exavmimages.tar -C /EXAVMIMAGES
```
Note:
The restore process of storage repository restores the user domain specific files (files under /EXAVMINAGES/GuestImages/user_domain/) as regular files and not as OCFS2 reflinks, which is what these files in the storage repository were originally at the time of the user domain creation. Consequently, the space usage in /EXAVMINAGES may go up after the restoration process when compared to the original space usage at the time of the backup.
Manually configure the network bridges.
1. Determine the version of the ovmutils RPM.
```
# rpm -qa|grep ovmutils
```
2. If the version of the ovmutils RPM is earlier than 12.1.2.2.0, perform these steps:
  1. Back up /opt/exadata_ovm/exadata.img.domu_maker. You will need the backup copy later.
```
# cp /opt/exadata_ovm/exadata.img.domu_maker /opt/exadata_ovm/exad
ata.img.domu_maker-orig
```
  2. Open the /opt/exadata_ovm/exadata.img.domu_maker file in a text editor such as vi, and search for g_do_not_set_bridge=yes. This string should be located a few lines below the case statement option network-discovery).
    
    Change the string to g_do_not_set_bridge=no.
    
    Save and exit /opt/exadata_ovm/exadata.img.domu_maker.
  3. Run /opt/exadata_ovm/exadata.img.domu_maker manually for every XML file in the /EXAVMIMAGES/conf directory.
```
# cd /EXAVMIMAGES/conf
# ls -1|while read file; do /opt/exadata_ovm/exadata.img.domu_maker 
network-discovery $file /tmp/netdisc-$file; done
```
  4. Restore /opt/exadata_ovm/exadata.img.domu_maker from the backup copy.
```
# cp /opt/exadata_ovm/exadata.img.domu_maker-orig /opt/exad
ata_ovm/exadata.img.domu_maker
```
3. If the version of the ovmutils RPM is 12.1.2.2.0 or later, then run the following command:
```
# /opt/exadata_ovm/exadata.img.domu_maker add-bonded-bridge-
dom0 vmbondeth0 eth4 eth5
```

For each user domain directory in the /EXAVMIMAGES/GuestImages directory, perform the following steps:

Get the UUID of the user domain.

# grep ^uuid /EXAVMIMAGES/GuestImages/user_domain_hostname/vm.cfg
|awk -F"=" '{print $2}'|sed s/"'"//g|sed s/" "//g

The command returns the uuid value, which is used in the commands below.

Create a sub-directory for the UUID.
```
# mkdir -p /OVS/Repositories/uuid 
```
Create a symbolic link for the vm.cfg file for the user_domain_hostname in the new UUID directory.
```
# ln -s /EXAVMIMAGES/GuestImages/user_domain_hostname/vm.cfg /OVS/Repositories/uuid/vm.cfg
```

Configure autostart for the user_domain_hostname.

# ln -s /OVS/Repositories/uuid/vm.cfg /etc/xen/auto/user_domain_hostname.cfg

Create the VirtualDisks sub-directory.
```
# mkdir VirtualDisks
```
Enter the VirtualDisks directory.
```
# cd VirtualDisks
```

Create four symbolic links in this directory using the four disk image names in the vm.cfg file, pointing to the four *.img files in /EXAVMIMAGES/GuestImages/user_domain_hostname directory.

For example, the following is a sample disk entry in a sample vm.cfg file in a /OVS/Repositories/uuid directory:

disk =  ['file:/OVS/Repositories/6e7c7109c1bc4ebba279f8
4e595e0b27/VirtualDisks/dfd641a1c6a84bd69643da704ff98594.img,xv
da,w','file:/OVS/Repositories/6e7c7109c1bc4ebba279f84e595e0b27/
VirtualDisks/d349fd420a1e49459118e6a6fcdbc2a4.img,xvdb,w','file
:/OVS/Repositories/6e7c7109c1bc4ebba279f84e595e0b27/VirtualDisk
s/8ac470eeb8704aab9a8b3adedf1c3b04.img,xvdc,w','file:/OVS/Repos
itories/6e7c7109c1bc4ebba279f84e595e0b27/VirtualDisks/333e7ed28
50a441ca4d2461044dd0f7c.img,xvdd,w']

You can list the four *.img files in the /EXAVMIMAGES/GuestImages/user_domain_hostname directory:

# ls /EXAVMIMAGES/GuestImages/user_domain_name/*.img
/EXAVMIMAGES/GuestImages/user_domain_name/System.img
/EXAVMIMAGES/GuestImages/user_domain_name/grid12.1.0.2.2.img
/EXAVMIMAGES/GuestImages/user_domain_name/db12.1.0.2.2-3.img
/EXAVMIMAGES/GuestImages/user_domain_name/pv1_vgexadb.img

In this example, the following commands can be used to create the four symbolic links where dbm01db08vm01 is the user domain host name:

# ln -s /EXAVMIMAGES/GuestImages/dbm01db08vm01/System.img
 $(grep ^disk /EXAVMIMAGES/GuestImages/dbm01db08vm01/vm.cfg|awk 
-F":" '{print $2}'|awk -F"," '{print $1}'|awk -F"/" '{print $6}')

# ln -s /EXAVMIMAGES/GuestImages/dbm01db08vm01/grid12.1.0.2.2.img
 $(grep ^disk /EXAVMIMAGES/GuestImages/dbm01db08vm01/vm.cfg|awk 
-F":" '{print $3}'|awk -F"," '{print $1}'|awk -F"/" '{print $6}')

# ln -s /EXAVMIMAGES/GuestImages/dbm01db08vm01/db12.1.0.2.2-3.img
 $(grep ^disk /EXAVMIMAGES/GuestImages/dbm01db08vm01/vm.cfg|awk 
-F":" '{print $4}'|awk -F"," '{print $1}'|awk -F"/" '{print $6}')

# ln -s /EXAVMIMAGES/GuestImages/dbm01db08vm01/pv1_vgexadb.img 
$(grep ^disk /EXAVMIMAGES/GuestImages/dbm01db08vm01/vm.cfg|awk 
-F":" '{print $5}'|awk -F"," '{print $1}'|awk -F"/" '{print $6}')

Restart each user domain.

# xm create /EXAVMIMAGES/GuestImages/user_domain_hostname/vm.cfg

At this point all the user domains should start along with the Oracle Grid Infrastructure and the database instances.

Parent topic: Recovering an Oracle VM Deployment