1. Login to the Bootstrap Host using the root credentials configured during OS installation of the Bootstrap Host.
2. Create the directories needed on the Installer Bootstrap Host.

   $ mkdir -p /var/occne/cluster/<cluster_short_name>/yum.repos.d

3. Mount the Utility USB.

   Note: Follow the instructions for mounting a USB in Linux are at: Installation of Oracle Linux 7.x on Bootstrap Host.

4. Copy the hosts.ini file (created using procedure: OCCNE Inventory File Preparation) into the /var/occne/cluster/<cluster_short_name>/ directory.

   This hosts.ini file defines the Bastion KVM Host to the Provision Container running the provision image downloaded from the repo.

   $ cp /<path_to_usb>/hosts.ini /var/occne/cluster/<cluster_short_name>/hosts.ini
    
   Example:
   $ cp /media/usb/hosts.ini /var/occne/cluster/rainbow/hosts.ini

5. Edit the /var/occne/cluster/<cluster_short_name>/hosts.ini file to include the ToR host_net (vlan3) VIP for NTP clock synchronization. Use the ToR VIP address (ToRswitch_Platform_VIP ) as defined in procedure: Installation PreFlight Checklist : Complete OA and Switch IP SwitchTable as the NTP source. Update the ntp_server field with the VIP address. Update the occne_repo_host_address to point to this Bootstrap Host internal address. This is being used for PXE booting and accessing the NFS share on the Installer Bootstrap Host (db-1/RMS1).

   Example (from hosts.sample.ini):
    
   ntp_server='172.16.3.1'
   occne_repo_host_address='172.16.3.4'

6. Copy the customer specific .repo file from the Utility USB to the Installer Bootstrap Host.

   This is the .repo file created by the customer that provides access to the onsite (within their network) yum repositories needed to complete the full deployment of OCCNE 1.3 onto the Installer Bootstrap Host. It needs to be in two places, one for the local system, and one for the target systems.

   $ cp /<path_to_usb>/<customer_specific_repo>.repo /var/occne/cluster/<cluster_short_name>/yum.repos.d/.
   $ cp -r /var/occne/cluster/<cluster_short_name>/yum.repos.d /var/occne/.
   $ echo "reposdir=/var/occne/yum.repos.d" >> /etc/yum.conf
    
   Example:
   $ cp /media/usb/ol7-mirror.repo /var/occne/cluster/rainbow/yum.repos.d/
   $ cp -r /var/occne/cluster/rainbow/yum.repos.d /var/occne/
   $ echo "reposdir=/var/occne/yum.repos.d" >> /etc/yum.conf

1. Add an entry to the /etc/hosts file on the Installer Bootstrap Host to provide a name mapping for the Customer central repository.

   $ vi /etc/hosts
    
   Example:
   10.75.200.217 rainbow-reg
   
   To Verify:
   $ ping <central_repo_name>
    
   Example:
   # ping rainbow-reg
   PING reg-1 (10.75.200.217) 56(84) bytes of data.
   64 bytes from reg-1 (10.75.200.217): icmp_seq=1 ttl=61 time=0.248 ms
   64 bytes from reg-1 (10.75.200.217): icmp_seq=2 ttl=61 time=0.221 ms
   64 bytes from reg-1 (10.75.200.217): icmp_seq=3 ttl=61 time=0.239 ms

2. Verify the repo access execute the following command:

   $ yum repolist
   
   Example:
   $ yum repolist
   Loaded plugins: ulninfo
   repo id                repo name                                                                   status
   !UEKR5/x86_64          Unbreakable Enterprise Kernel Release 5 for Oracle Linux 7 (x86_64)             80
   !addons/x86_64         Oracle Linux 7 Addons (x86_64)                                                  91
   !developer/x86_64      Packages for creating test and development environments for Oracle Linux 7     226
   !developer_EPEL/x86_64 Packages for creating test and development environments for Oracle Linux 7  13,246
   !ksplice/x86_64        Ksplice for Oracle Linux 7 (x86_64)                                            393
   !latest/x86_64         Oracle Linux 7 Latest (x86_64)                                               5,401
   repolist: 19,437

The iso file must be accessible from a Customer Site Specific repository. This file should be accessible because the ToR switch configurations were completed in procedure: Configure Top of Rack 93180YC-EX Switches

Copy the OL7 ISO file to the /var/occne directory. The example below uses OracleLinux-7.5-x86_64-disc1.iso. If this file was copied to the Utility USB, it can be copied from there into the same directory on the Bootstrap Host.

Note: If the user copies this ISO from their laptop then they must use an application like WinSCP pointing to the Management Interface IP.

scp <usr>@<site_specific_address>:/<path_to_iso>/OracleLinux-7.5-x86_64-disc1.iso /var/occne/OracleLinux-7.5-x86_64-disc1.iso

$ yum install docker-engine nfs_utils ansible

1. Copy the docker registry certificate to two places on the Bootstrap Host.

   Note: How to obtain the docker registry certificate <source> is not necessarily covered in the procedure. The user can use the instructions at reference 1 to understand this more thoroughly.

   $ mkdir -p /var/occne/certificates
   $ cp <source>.crt /var/occne/certificates/<occne_private_registry>:<occne_private_registry_port>.crt
   $ mkdir -p /etc/docker/certs.d/<occne_private_registry>:<occne_private_registry_port>
   $ cp <source>.crt /etc/docker/certs.d/<occne_private_registry>:<occne_private_registry_port>/ca.crt
    
   Example:
   $ mkdir -p /var/occne/certificates
   $ cp <source>.crt /var/occne/certificates/rainbow_reg:5000.crt
   $ mkdir -p /etc/docker/certs.d/rainbow_reg:5000
   $ cp <source>.crt /etc/docker/certs.d/rainbow_reg:5000/ca.crt 

2. Start the docker daemon.

   $ systemctl daemon-reload
   $ systemctl restart docker
   $ systemctl enable docker
     
   Verify docker is running:
   $ ps -elf | grep docker
   $ systemctl status docker

Run the following commands using sudo (assumes nfs-utils has already been installed in procedure: Installation of Oracle Linux 7.x on Bootstrap Host : Install Additional Packages).

$ echo '/var/occne 172.16.3.4/24(ro,no_root_squash)' >> /etc/exports
$ systemctl start nfs-server
$ systemctl enable nfs-server
  
Verify nfs is running:
$ ps -elf | grep nfs
$ systemctl status nfs-server

$ mkdir -p /var/occne/pxelinux
$ mount -t iso9660 -o loop /var/occne/OracleLinux-7.5-x86_64-disc1.iso /mnt
$ cp /mnt/isolinux/initrd.img /var/occne/pxelinux
$ cp /mnt/isolinux/vmlinuz /var/occne/pxelinux

$ chmod -R 777 /var/occne/pxelinux

$ systemctl stop dhcpd
$ systemctl disable dhcpd
$ systemctl stop tftp
$ systemctl disable tftp

$ getenforce
active
  
If the output of this command displays "active", change it to "permissive" by editing the /etc/selinux/config file.
  
$ vi /etc/selinux/config
  
Change the SELINUX variable to passive: SELINUX=permissive
save the file
  
Reboot the system: reboot

This command generates a private and public key for the cluster. These keys are passed to the Bastion Host and used to communicate to other nodes from that Bastion Host. The public key is passed to each node on OS install. Do not supply a passphrase when it asks for one. Just hit enter.

Note: The private key (occne_id_rsa) must be copied to a server that going to access the Bastion Host because the Bootstrap Host is repaved. This key is used later in the procedure to access the Bastion Host after it has been created.

$ mkdir -m 0700 /var/occne/cluster/<cluster_short_name>/.ssh
$ ssh-keygen -b 4096 -t rsa -C "occne installer key" -f "/var/occne/cluster/<cluster_short_name>/.ssh/occne_id_rsa" -q -N ""

1. Run the docker commands below to perform the OS install and Bastion Host VM creation on the Bastion KVM Host (RMS2):

   $ docker run --rm --network host --cap-add=NET_ADMIN -v /var/occne/cluster/<cluster_short_name>/:/host -v /var/occne/:/var/occne:rw -e "OCCNEARGS=--limit=<bastion_full_name>,<bastion_kvm_host_full_name>,localhost" <image_name>:<image_tag>
     
     
   Example:
     
   $ docker run -it --rm --network host --cap-add=NET_ADMIN -v /var/occne/cluster/rainbow/:/host -v /var/occne/:/var/occne:rw -e "OCCNEARGS=--limit=bastion-2.rainbow.lab.us.oracle.com,db-2.rainbow.lab.us.oracle.com,localhost" winterfell:5000/occne/provision:1.3.0
   

2. Verify that Bastion Host VM is installed by logging into RMS2/db-2 and issuing the following command. The <ansible_host> field (which is an IPv4 address) is derived from the hosts.ini file db-2 entry for host_hp_gen_x groups.

   Note: This command is optional. Had a failure actually occurred, the docker run command would have experienced failures.

   ssh -i /var/occne/cluster/<cluster_short_name>/.ssh/occne_id_rsa  admusr@<oam_host>
    
   $ sudo virsh list
     
   Example:
   ssh -i /var/occne/cluster/rainbow.lab.us.oracle.com/.ssh/occne_id_rsa  admusr@10.75.148.6
   $ sudo virsh list
    
    Id    Name                           State
   ----------------------------------------------------
    11    bastion-2.rainbow.lab.us.oracle.com running

3. Login to Bastion Host from the Boostrap Host as admusr using the generated key from the /var/occne/cluster/<cluster_short_name> directory to confirm the VM is set up correctly. The <oam_host> field (which is an IPv4 address) is derived from the hosts.ini file bastion-2 entry for the host_kernel_virtual group.

   Note: This command is optional. Had a failure actually occurred, the docker run command would have experienced failures.

   ssh -i /var/occne/cluster/<cluster_short_name>/.ssh/occne_id_rsa  admusr@<oam_host>
    
   Example:
   ssh -i /var/occne/cluster/rainbow/.ssh/occne_id_rsa admusr@10.75.148.5