Oracle^® SuperCluster Quorum Disk Manager

Add Quorum Disk to Database Domains (Global Zones)

The example in this section creates quorum disks for a small rack configuration with two database domains: db01 and db02. On both db01 and db02, there are two InfiniBand interfaces: bondib0 and bondib1.

You can find the network interfaces to be used for communication with the iSCSI devices using the following command:

$ oifcfg getif | grep cluster_interconnect | awk '{print $1}' 

You can find the IP address of each interface using the following command:

ipadm show-addr interface_name 

The InfiniBand IP addresses for this example are as follows:

• On the first database domain (db01):

  • Network interface: bondib0, IP address: 192.168.10.45

  • Network interface: bondib1, IP address: 192.168.10.46

• On the second database domain (db02):

  • Network interface: bondib0, IP address: 192.168.10.47

  • Network interface: bondib1, IP address: 192.168.10.48

The ASM disk group to which the quorum disks will be added is DATAC1. The ASM owner is grid, and the user group is dba.

Initially, the voting files reside on a normal redundancy disk group RECOC1:

$ $GRID_HOME/bin/crsctl query css votedisk
##  STATE    File Universal Id                 File Name  Disk group 
--  -----    -----------------                 ---------  ---------  
1.  ONLINE   21f5507a28934f77bf3b7ecf88b26c47 (o/192.168.76.187;192.168.76.188/RECOC1_CD_00_celadm12) [RECOC1]  
2.  ONLINE   387f71ee81f14f38bfbdf0693451e328 (o/192.168.76.189;192.168.76.190/RECOC1_CD_00_celadm13) [RECOC1]  
3.  ONLINE   6f7fab62e6054fb8bf167108cdbd2f64 (o/192.168.76.191;192.168.76.192/RECOC1_CD_00_celadm14) [RECOC1] 
Located 3 voting disk(s). 

To create the quorum disks:

1. Verify that your SuperCluster system fulfills the proper software requirements and that you have set up the public key authentication for the ZFS storage controllers, if necessary.

   See Software Requirements for Quorum Disk Manager and Set Up Public Key Authentication for ZFS Storage Controllers for more information.

2. Log into the first database domain (db01) and the second database domain (db02) as a root user.

   In this procedure, it is assumed that you have two separate terminal windows where you are able to run steps on the first database domain (db01) in one terminal window and steps on the second database domain (db02) in the second terminal window, and you are able to see when a process has completed in the db01 terminal window before running that same step on the db02 terminal window.

   ---

   Note -  The following commands use --owner=oracle and --group=dba as examples. Replace the owner and group information in the following commands with information that is valid for your setup.

   ---

3. On the first database domain, create quorum disk configurations:

   db01# /opt/oracle.SupportTools/quorumdiskmgr --create --config --owner=oracle  --group=dba --network-iface-list="bondib0, bondib1" 

4. Wait until the process has completed on the first database domain, then, on the second database domain, run the same command to create quorum disk configurations:

   db02# /opt/oracle.SupportTools/quorumdiskmgr --create --config --owner=oracle  --group=dba --network-iface-list="bondib0, bondib1" 

5. Verify that the configurations have been successfully created on the first and second database domains.
   1. Enter the following command on db01:

      db01# /opt/oracle.SupportTools/quorumdiskmgr --list --config 

   2. Enter the same command on db02:

      db02# /opt/oracle.SupportTools/quorumdiskmgr --list --config 

   Output similar to the following appears for both db01 and db02:

   Owner: oracle 
   Group: dba
   ifaces: bondib0,bondib1   
   
   [SUCCESS ] Successfully list quorumdisk config file 

6. On the first database domain, create a target for ASM disk group DATAC1 and make the target visible to both the first and second database domains.

   db01# /opt/oracle.SupportTools/quorumdiskmgr --create --target --asm-disk-group=datac1 --visible-to="db01_bondib0_IPaddress, db01_bondib1_IPaddress, db02_bondib0_IPaddress, db02_bondib1_IPaddress" 

   For example:

   db01# /opt/oracle.SupportTools/quorumdiskmgr --create --target --asm-disk-group=datac1 --visible-to="192.168.10.45, 192.168.10.46, 192.168.10.47, 192.168.10.48" 

7. Wait until the process has completed on the first database domain, then, on the second database domain, run the same command to create a target for ASM disk group DATAC1 and make the target visible to both the first and second database domains.

   db02# /opt/oracle.SupportTools/quorumdiskmgr --create --target --asm-disk-group=datac1 --visible-to="db01_bondib0_IPaddress, db01_bondib1_IPaddress, db02_bondib0_IPaddress, db02_bondib1_IPaddress" 

   For example:

   db02# /opt/oracle.SupportTools/quorumdiskmgr --create --target --asm-disk-group=datac1 --visible-to="192.168.10.45, 192.168.10.46, 192.168.10.47, 192.168.10.48" 

8. Verify the target has been successfully created on both database domains.
   1. Enter the following command on the first database domain:

      db01# /opt/oracle.SupportTools/quorumdiskmgr --list --target

      On the first database domain, output similar to the following appears:

      Name: iqn.1986-03.com.sun:02:080a9aa9-3c1a-cd96-b80e-a812ce744211
      Size: 128 MB
      Lun: QD_datac1_db01
      Host name: DB01
      ASM disk group name: DATAC1
      Visible to: 192.168.10.45, 192.168.10.46, 192.168.10.47, 192.168.10.48
      Discovered by:
      
      [SUCCESS ] List lun(s)

   2. Enter the same command on the second database domain:

      db02# /opt/oracle.SupportTools/quorumdiskmgr --list --target

      On the second database domain, output similar to the following appears:

      Name: iqn.1986-03.com.sun:02:080a9aa9-3c1a-cd96-b80e-a812ce744111
      Size: 128 MB
      Lun: QD_datac1_db02
      Host name: DB02
      ASM disk group name: DATAC1
      Visible to: 192.168.10.45, 192.168.10.46, 192.168.10.47, 192.168.10.48
      Discovered by:
      
      [SUCCESS ] List lun(s)   

9. On the first database domain, create devices from targets on both database domains.

   db01# /opt/oracle.SupportTools/quorumdiskmgr --create --device --target-ip-list="db01_bondib0_IPaddress, db01_bondib1_IPaddress, db02_bondib0_IPaddress, db02_bondib1_IPaddress"

   For example:

   db01# /opt/oracle.SupportTools/quorumdiskmgr --create --device --target-ip-list="192.168.10.45, 192.168.10.46, 192.168.10.47, 192.168.10.48"

10. Wait until the process has completed on the first database domain, then, on the second database domain, run the same command to create devices from targets on both database domains.

    db02# /opt/oracle.SupportTools/quorumdiskmgr --create --device --target-ip-list="db01_bondib0_IPaddress, db01_bondib1_IPaddress, db02_bondib0_IPaddress, db02_bondib1_IPaddress"

    For example:

    db02# /opt/oracle.SupportTools/quorumdiskmgr --create --device --target-ip-list="192.168.10.45, 192.168.10.46, 192.168.10.47, 192.168.10.48"

11. Verify the devices have been successfully created on both database domains.
    1. Enter the following command on the first database domain:

       db01# /opt/oracle.SupportTools/quorumdiskmgr --list --device

    2. Enter the same command on the second database domain:

       db02# /opt/oracle.SupportTools/quorumdiskmgr --list --device

    Output similar to the following appears on both db01 and db02:

    Device path: /dev/exadata_quorum/QD_DATAC1_DB01
    Size: 128 MB
    Host name: DB01
    ASM disk group name: DATAC1
    
    Device path: /dev/exadata_quorum/QD_DATAC1_DB02
    Size: 128 MB
    Host name: DB02
    ASM disk group name: DATAC1
    
    [SUCCESS  ] List devices 

12. Complete the quorum disk configuration process.

    Go to Complete the Quorum Disk Configuration for those instructions.