2.12.6 Prepare the Re-imaged Database Server for the Cluster

This task describes how to ensure the changes made during initial installation are done to the re-imaged, bare metal database server.

Note:

For Oracle VM systems, follow the procedure in Expanding an Oracle RAC Cluster on Oracle VM Using OEDACLI.

1. Copy or merge the contents of the following files using files on a working database server as reference:
   1. Copy the contents of the /etc/security/limits.conf file.
   2. Merge the contents of the /etc/hosts files.
   3. Copy the /etc/oracle/cell/network-config/cellinit.ora file.
   4. Update the /etc/oracle/cell/network-config/cellinit.ora file with the IP_ADDRESS of the ifcfg-bondib0 interface (in case of active/passive bonding) or ib0 and ib1 interfaces (in case of active/active bonding) of the replacement server.
   5. Copy the /etc/oracle/cell/network-config/cellip.ora file.
      The content of the cellip.ora file should be the same on all database servers.
   6. Configure additional network requirements, such as 10 GbE.
   7. Copy the modprobe configuration.

      The contents of the configuration file should be the same on all database servers.

      • Oracle Linux 5 or 6: The file is located at /etc/modprobe.conf.
      • Oracle Linux 7 or later: The file is located at /etc/modprobe.d/exadata.conf.
   8. Copy the /etc/sysctl.conf file.
      The contents of the file should be the same on all database servers.
   9. Update the cellroute.ora.

      Make a copy of the /etc/oracle/cell/network-config/cellroute.ora file. Modify the contents on the replacement server to use the local InfiniBand interfaces on the new node.

   10. Restart the database server so the network changes take effect.
2. Set up the users for the software owners on the replacement database server by adding groups.
   If you are using role-separated management, then the users are usually oracle and grid. If you use a single software owner, then the user is usually oracle. The group information is available on a working database server.
   1. Obtain the current group information from a working database server.

      # id oracle
      uid=1000(oracle) gid=1001(oinstall) groups=1001(oinstall),1002(dba),1003(oper),1004(asmdba)
      

   2. Use the groupadd command to add the group information to the replacement database server.

      # groupadd -g 1001 oinstall
      # groupadd -g 1002 dba
      # groupadd -g 1003 oper
      # groupadd -g 1004 asmdba
      

   3. Obtain the current user information from a working database server.

      # id oracle uid=1000(oracle) gid=1001(oinstall) \
        groups=1001(oinstall),1002(dba),1003(oper),1004(asmdba)
      

   4. Add the user information to the replacement database server.

      # useradd -u 1000 -g 1001 -G 1001,1002,1003,1004 -m -d /home/oracle -s \
        /bin/bash oracle
      

   5. Create the Oracle Base and Grid home directories, such as /u01/app/oracle and /u01/app/12.2.0.1/grid.

      # mkdir -p /u01/app/oracle
      # mkdir -p /u01/app/12.2.0.1/grid
      # chown -R oracle:oinstall /u01/app
      

   6. Change the ownership on the cellip.ora and cellinit.ora files.

      The ownership is usually oracle:oinstall.

      # chown -R oracle:oinstall /etc/oracle/cell/network-config
      

   7. Secure the restored database server.

      # chmod u+x /opt/oracle.SupportTools/harden_passwords_reset_root_ssh
      # /opt/oracle.SupportTools/harden_passwords_reset_root_ssh
      

      The database server restarts. Log in as the root user when prompted by the system. You are prompted for a new password. Set the password to match the root password of the other database servers.

   8. Set the password for the Oracle software owner.
      The owner is usually oracle.

      # passwd oracle
      

3. Set up SSH for the oracle account.
   1. Log in to the oracle account on the replacement database server.

      # su - oracle
      

   2. Create the dcli group file on the replacement database server listing the servers in the Oracle cluster.
   3. Run the following command on the replacement database server.

      $ dcli -g dbs_group -l oracle -k
      

   4. Exit and log in again as the oracle user.

      $ exit
      # su - oracle
      

   5. Verify SSH equivalency.

      $ dcli -g dbs_group -l oracle date
      

4. Set up or copy any custom login scripts from a working database server to the replacement database server.

   In the following command, replacement_server is the name of the new server, such as dm01db01.

   $ scp .bash* oracle@replacement_server:.