10 Preparing the Oracle Cloud Infrastructure for an Enterprise Deployment

If you plan to deploy Identity and Access Management on Oracle Cloud Infrastructure (OCI) using the Oracle Container Engine for Kubernetes, you have to configure OCI to facilitate the deployment. Create the required OCI components to perform the deployment.

Note:

The instructions provided in this guide are correct at the time of publishing. Due to the evolving nature of the OCI interface, you may find minor changes in the options. See the Oracle Cloud Infrastructure documentation to obtain the latest steps.

This chapter includes the following topics:

• About the OCI Deployment
  This illustration shows all the OCI components that you require to deploy Oracle Identity and Access Management on OCI. It shows the different network requirements and how the OCI components fit into those networks. Each subnet is protected by security lists.
• Creating an SSH Key Pair
  You can configure OCI by using the Oracle Cloud Console and a bastion node. The SSL certificates provide a secure access to the bastion node, compute instances, OKE worker nodes, and database hosts. You have to create an SSL certificate on the host you use to configure OCI. This host could be a laptop or a desktop.
• Creating an OCI Compartment
  Create a container in your OCI tenancy to hold the deployment.
• Creating an OKE Cluster in OCI
  You can create a cluster in OKE in one of two ways: by creating a quick cluster with minimum user input or by manually creating a cluster that provides more flexibility.
• Creating a Bastion Node
  You cannot access the cluster directly because the cluster is in a dedicated subnet. You can use a bastion node to access the cluster. The bastion node will be publicly available.
• Creating Compute Instances for Oracle HTTP Servers
  The web tier resides in its own subnet separated from both the load balancer and the application tier. This section describes the procedures to create two compute instances for the web tier, place them in different availability domains, and set up security lists and route tables to facilitate access.
• Creating File Systems and Mount Targets
  You need to create NFS file systems for Kubernetes Persistent Volumes and Oracle HTTP Server installations.
• Creating Load Balancers
  You need to create two OCI load balancers. One of these load balances is used to direct public traffic and the other for internal call backs. The load balancer used for internal traffic is not available outside the OCI container.
• Creating a Network Load Balancer
  This step is required only if you want to configure a load balancer to route traffic to the Kubernetes worker nodes.
• Creating a Database
  There are several different databases that you can create in OCI. For this example, a bare metal RAC database will be created. You may need to create one or more databases.
• Creating a Vault
  A vault is used to store the credentials of your deployment. At present, the only Oracle Identity and Access Management product using a vault is Oracle Advanced Authentication (OAA). OAA can use either an OCI-based vault (recommended) or a file-based vault.
• Creating a DNS Server
  This is an optional task. It is important that all host names are resolvable, including the load balancer virtual hosts. You can make them resolvable by adding entries to the local hosts files. However, in OCI, using a private DNS server is the simpler method.
• Updating Kubernetes CoreDNS
  
• Validating Your Environment
  Perform the checks described in this section to ensure that your environment is ready for a deployment.
• Preparing a Disaster Recovery Environment
  A disaster recovery environment is a replica of your primary environment located in a different region from the primary region. This environment is a standby environment that you switch over to in the event of the failure of your primary environment.

About the OCI Deployment

This illustration shows all the OCI components that you require to deploy Oracle Identity and Access Management on OCI. It shows the different network requirements and how the OCI components fit into those networks. Each subnet is protected by security lists.

Figure 10-1 An Illustration of the OCI Layout for OKE

When deploying Oracle Identity and Access Management in OCI, you have to set up the OCI environment with the following characteristics:

• VCN: There will be one public Virtual Cloud Network which provides external access to the environment. For security reasons, the VCN is broken down into a several subnets.
• Subnets: The VCN is divided into several subnets to ensure that the network traffic is routed only to the areas requiring it. For instance, traffic to the database subnet will not be available directly from the internet. Traffic is available only to the Application (OKE) tier, which interacts with the database subnet.
• Security Lists: Security lists provide an additional layer of security that allows traffic only into and out of a subnet, based on the ports and protocols permitted.
• Bastion Node: The Bastion node is a compute instance inside the VCN that you can log in to. The Bastion node can communicate with all the components inside the deployment. The Bastion node is used for setting up the environment and for ongoing management. Therefore, you must lock down access to the Bastion node to ensure that it is accessed only by clients on your corporate network who are registered with it using an SSL key pair.
• Load balancer: The two LBaaS services are created within the OCI framework. The public-facing load balancer is used to access the Oracle Identity and Access Management deployment from the internet. The private load balancer is for internal traffic, routing it is not available outside of the VCN. The public load balancer is the only internet-facing part of your deployment (except for the Bastion node).
• Oracle Container Engine for Kubernetes (OKE): This is where your application is deployed inside the Kubernetes containers. It is not visible to the internet directly. The Oracle HTTP servers are not deployed in the OKE cluster. These servers are placed into a separate demilitarized zone (DMZ).
• Compute Instances: You require a minimum of two compute instances to host your Oracle HTTP servers. These are placed into a demilitarized zone (DMZ) below the load balancers. The load balancers send requests to the OHS servers, which pass the traffic onto the application residing within the OKE cluster.
• Database: The database(s) are present in a dedicated subnet below the OKE cluster.
• DNS: The DNS server is optional. It is used internally to provide name resolution. You can achieve name resolution by maintaining entries in the individual host files.
• Availability Zones: The availability zones ensure that your application is deployed on different hardware within the data center. This type of deployment ensures that the system continues to function even if there is a hardware issue.

The following sections describe the procedure to set up the components depicted in this illustration:

Creating an SSH Key Pair

You can configure OCI by using the Oracle Cloud Console and a bastion node. The SSL certificates provide a secure access to the bastion node, compute instances, OKE worker nodes, and database hosts. You have to create an SSL certificate on the host you use to configure OCI. This host could be a laptop or a desktop.

After you create the certificate on the device, share it with the OCI resources to enable access to the resources and to manage them. If you use more than one device, you have to register the SSL keys for all those devices.

If you do not have an SSL certificate for the device you are using, create the cetificate using the following command:

ssh-keygen -t rsa -N "" -b 2048 -f id_rsa

This command creates two files id_rsa and id_rsa.pub in the .ssh directory under the home directory. These are the certificate files you will use to access the OCI resources.

Creating an OCI Compartment

Create a container in your OCI tenancy to hold the deployment.

To create a compartment:

1. Log in to the Oracle Cloud Infrastructure Console, select Identity, and then click Compartments.
2. Click Create Compartment.
3. Specify a Name and Description.
4. Click Create Compartment.

You will create all the OCI objects inside this compartment.

Creating an OKE Cluster in OCI

You can create a cluster in OKE in one of two ways: by creating a quick cluster with minimum user input or by manually creating a cluster that provides more flexibility. If you create a cluster using the quick cluster, you will have minimum configuration options for cluster networking. Networking is important to specify the network subnets you want to use.

• Creating an OKE Cluster Using Quick Cluster
  
• Creating an OKE Cluster Manually
  

Creating an OKE Cluster Using Quick Cluster

The first step in preparing OCI is to create an OKE cluster. This step creates the virtual cloud network, the OKE cluster, and the worker nodes.

To create a quick cluster with default settings:

1. Log in to the Oracle Cloud Infrastructure Console.
2. Select Developer Services (located in Containers and Artifacts) and click Kubernetes Clusters.
3. Click Create Cluster.
4. Select Quick Create.
5. Click Launch Workflow.
   The Create Cluster screen is displayed.
6. Enter the following details:
   • Name: Specify a name for the cluster. For example: IDMEDG.
   • Compartment - Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Version: Select the version of Kubernetes you want to use. Ensure that the version you select is supported for IDM Kubernetes deployments.
   • Kubernetes API End point: Select Private. The Kubernetes cluster will not be exposed directly to the internet.
   • Kubernetes Workers: Select Private. The Kubernetes cluster will not be exposed directly to the internet.
   • Shape: Select the OCI shape you want to use to create the Kubernetes worker nodes. The shape you choose will depend on the number of worker nodes you want to create and the size of those nodes.
   • Number of nodes: Select the number of worker nodes you want to create.
7. Click Show Advanced Options. In the Public SSH Key box, copy the content of the id_rsa.pub file which you created earlier. See Creating an SSH Key Pair.
8. Click Next.
9. Review the summary and click Create Cluster.

   The workflow creates:

   • Virtual Cloud Network (VCN)
   • Route Tables
   • Security Lists
   • Kubernetes Cluster
   • Node Pool
10. Click Close.

Creating an OKE Cluster Manually

To create an OKE cluster manually, you should complete the steps explained in this section. If you want to link two VCNs together, for example, use one VCN for the primary deployment and the other for the DR (Disaster Recovery) deployment, it is essential that the Network CIDRS/IP Addresses do not overlap.

For example, you could use 10.0.0.0/16 for your primary network and 10.1.0.0/16 for your DR network.

• Creating an Oracle Virtual Cloud Network
  
• Adding Additional Security Rules
  
• Creating an API Security List
  
• Creating an API Subnet
  
• Creating the OKE Cluster
  

Creating an Oracle Virtual Cloud Network

To create an Oracle Virtual Cloud Network:

1. Log in to the Oracle Cloud Infrastructure Console.
2. Select Networking Virtual Cloud Networks.
3. Click Start VCN Wizard.
4. Select Create VCN with Internet Connectivity and click Start VCN Wizard.
5. Enter the following information in the wizard:
   • Name: Select a name for the network. For example idm_oke_vcn.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • VCN CIDR Block: Enter the internal CIDR block you want to use for your network. For example: 10.0.0.0/16.
   • Public Subnet CIDR Block: Enter the CIDR of the subnet you want to export to the internet. For example: 10.0.20.0/24.
   • Private Subnet CIDR Block: Enter the CIDR of the subnet you want to use privately (this is where the Kubernetes worker nodes will reside). For example: 10.0.10.0/24.
   • Use DNS Hostnames in this VCN: Select this option.
6. Click Next.
7. Review the summary information of the details specified and click Create.
8. When complete, click View Virtual Cloud Network.

These steps will create a public and private subnet.

Adding Additional Security Rules

To add extra security rules to the default security list for OKE:

1. Log in to the Oracle Cloud Infrastructure Console.
2. Select Networking Virtual Cloud Networks.
3. Select the newly created network idm_oke_vcn. See Creating an Oracle Virtual Cloud Network.
4. Click Security Lists.
5. Click the default security list. For example, security list for the private subnet - idm_oke_vcn.
6. Click Add Ingress Rules to add an Ingress rule as described in Table 10-1.
7. Click Egress in the Resources List
8. Click Add Egress Rules to add an Egress rule as described in Table 10-1.

   Table 10-1 Description of Ingress and Egress Rules

   Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Destination Port Range	Type	Code
   Ingress	CIDR	10.0.10.0/24		All protocols
   Ingress	CIDR	10.0.0.0/28		ICMP		3	4
   Ingress	CIDR	10.0.0.0/28		TCP
   Ingress	CIDR	0.0.0.0.0/0		TCP	22
   Egress	CIDR		10.0.10.0/24	All protocols
   Egress	CIDR		10.0.0.0/28	TCP	6433
   Egress	CIDR		10.0.0.0/28	TCP	12250
   Egress	CIDR		10.0.0.0/28	ICMP		3	4
   Egress	Service		All Services in Oracle Service Network	TCP	443

Creating an API Security List

Kubernetes requires an additional subnet to communicate with the Kubernetes control plane. To enable this communication, you should first create a security list.

1. Log in to the Oracle Cloud Infrastructure Console.
2. Select Networking Virtual Cloud Networks.
3. Click the newly created network idm_oke_vcn. See Creating an Oracle Virtual Cloud Network.
4. Click Security Lists and select Create Security List.
5. Enter the following information:
   • Name: Enter a name for the security list. For example: api-seclist.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
6. Click Add Another Ingress Rule to add an Ingress rule as described in Table 10-2 (repeat for each Ingress rule).
7. Click Add Another Egress Rule to add an Egress rule as described in Table 10-2 (repeat for each Egress rule).

   Table 10-2 Description of Ingress and Egress Rules

   Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Destination Port Range	Type	Code
   Ingress	CIDR	0.0.0.0/0		TCP	6443
   Ingress	CIDR	10.0.10.0/24		TCP	6443
   Ingress	CIDR	10.0.10.0/24		TCP	12250
   Ingress	CIDR	10.0.10.0/24		ICMP		3	4
   Egress	CIDR	10.0.10.0/24		TCP
   Egress	CIDR	10.0.10.0/24		ICMP		3	4
   Egress	Service	All services in the Services Network.		TCP	443

8. Click Create Security List.

Creating an API Subnet

Kubernetes requires an additional subnet to communicate with the Kubernetes control plane. To create an API subnet:

1. Log in to the Oracle Cloud Infrastructure Console.
2. Select Networking Virtual Cloud Networks.
3. Click the newly created network idm_oke_vcn. See Creating an Oracle Virtual Cloud Network.
4. Click Subnets and select Create Subnet.
5. Enter the following information:
   • Name: Enter a name for the subnet. For example: api-subnet.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Subnet Type: Select Regional.
   • CIDR Block: Enter the CIDR of the subnet. For example: 10.0.0.0/28.
   • Route Table: Select Default Route Table for idm_oke_vcn.
   • Subnet Access: Select Private.
   • Use DNS Hostnames in this Subnet - Select this option.
   • Security List: Select the security list you created above: api-seclist. See Creating an API Security List.
6. Click Create Subnet.

Creating the OKE Cluster

Now that you have created the network, create the OKE cluster:

1. Log in to the Oracle Cloud Infrastructure Console.
2. Select Developer Services (located in Solutions and Platform) and click Kubernetes Containers (OKE).
3. Click Create Cluster and select Custom Create.
4. Click Submit.
5. Enter the following information in the wizard:
   • Name: Enter a name for the cluster. For example: idm-oke.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Kubernetes Version: Select the version of Kubernetes you want to create.
6. Click Next.
7. Enter the following information in the Networking Setup page:
   • Network Type: Select Flannel Overlay.
   • VCN: Select the VCN you created earlier. For example: idm_oke_vcn. See Creating an Oracle Virtual Cloud Network.
   • Kubernetes Service LB Subnets: Select the public subnet that was automatically created with the VCN.
   • Kubernetes API Endpoint Subnet: Select the API subnet you created earlier. For example: api-subnet. See Creating an API Subnet.
   • Assign a Public IP Address to the API Endpoint - Do not select this option.
   • Click Next.
8. Enter the following information in the Node Pools page:
   • Name: Specify a name for the pool. For example: Pool1.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Kubernetes Version: Select the version of Kubernetes you want to create. This should be the same as the cluster version.
   • Shape and Image: Enter details of the shape and capacity of the worker nodes you want to create. For example: VM.Standard.E3.Flex.
   • OCPU: 4.
   • RAM: 64GB.
   • Image: Oracle Linux 8.
   • Number of Nodes: The number of worker nodes you want to create.
   • Boot Volume: You can use the default value or increase the size of the boot volume if you anticipate using different container images and versions.
   • Placement Configuration: You should place the worker nodes in different availability domains. To place the worker nodes, create a placement for each availability zone. For example, if you have three or four worker nodes, create a placement for three different availability domains. To create a placement, enter the following information:
     • Availability Domain: Select one.
     • Worker Node Subnet: Select the default private subnet which was created with the VCN.
     • Pod Subnet: Select the default private subnet which was created with the VCN.
9. Click Next.
10. Review the cluster summary, and then click Create Cluster.

Creating a Bastion Node

You cannot access the cluster directly because the cluster is in a dedicated subnet. You can use a bastion node to access the cluster. The bastion node will be publicly available.

Note:

The basion node is the window to your environment. Therefore, access to the bastion node should be strictly controlled.

The creation of a bastion node includes the following steps:

• Creating Security Lists
  
• Creating a Route Table
  
• Creating a Subnet for the Bastion Node
  
• Adding the Security List to the Kubernetes Node Subnet
  
• Creating the Bastion Compute Instance
  
• Connecting to the Bastion Node
  
• Configuring the Bastion Node
  

Creating Security Lists

You need to create security lists which enable the bastion node to communicate with the subnet that the Kubernetes cluster uses. In addition, you need to allow access to the bastion node from the internet. This section describes the minimum steps you need to perform to enable this access. You should harden your security lists to ensure that only certain machines/networks have access to this node. Information about restricting access beyond the SSL key generated earlier is outside the scope of this document.

To create security lists:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From the Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Select Security Lists from the list of resources.

• Creating a Private Security List
  
• Creating a Public Security List
  
• Creating a Setup Security List
  

Creating a Private Security List

To create a private security list:

1. Click Create Security List.
2. Enter the following details:
   • Name: Enter a name for the security list. For example: bastion-private-seclist.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
3. Click Add Another Ingress Rule to add an Ingress rule as described in Table 10-3 (repeat for each Ingress rule).
4. Click Add Another Egress Rule to add an Egress rule as described in Table 10-3 (repeat for each Egress rule).
5. Click Create Security List.

Table 10-3 Description for Ingress and Egress Rules

Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Destination Port Range
Ingress	CIDR	10.0.1.0/29		TCP	22
Ingress	CIDR	10.0.1.0/29		ICMP
Egress	CIDR		0.0.0.0/0	All Protocols

Note:

10.0.1.0 is the subnet you will use for the bastion node. You can change this value if required.

Creating a Public Security List

To create a public security list:

1. Click Create Security List.
2. Enter the following details:
   • Name: Enter a name for the security list. For example: bastion-public-seclist.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
3. Click Add Another Ingress Rule to add an Ingress rule as described in Table 10-4 (repeat for each Ingress rule).
4. Click Add Another Egress Rule to add an Egress rule as described in Table 10-4 (repeat for each Egress rule).
5. Click Create Security List.

Table 10-4 Description for Ingress and Egress Rules

Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Destination Port Range	Type
Ingress	CIDR	0.0.0.0/0		TCP	22
Ingress	CIDR	10.0.1.0/29		ICMP		3
Ingress	CIDR	0.0.0.0/0		ICMP
Egress	CIDR		0.0.0.0/0	All Protocols

Note:

10.0.1.0 is the subnet you will use for the bastion node. You can change this value if required. Unless otherwise stated, leave the values blank.

Creating a Setup Security List

During the set up of Oracle Identity and Access Management, the bastion node requires access to some of the Kubernetes services that get created as part of the build process.

The access is not required after the build process is complete. For manageability reasons, a separate security list is created for this purpose. This way, after the setup is complete, you just have to remove the security list from the subnet. If further setups are required, you can add as needed.

The security list should be added to the following subnets:

• Private subnet for Node Manager

• db-subnet.

To create a setup security list:

1. Click Create Security List.
2. Enter the following details:
   • Name: Enter a name for the security list. For example: bastion-setup-seclist.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
3. Click Add Another Ingress Rule to add an Ingress rule as described in Table 10-5 (repeat for each Ingress rule).
4. Click Create Security List.

Table 10-5 Description for Ingress Rules

Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Source Port Range	Destination Port Range	Comment
Ingress	CIDR	10.0.1.0/29		TCP		30701	OAM Administration Server Kubernetes Service Port
Ingress	CIDR	10.0.1.0/29		TCP		31800
Ingress	CIDR	10.0.1.0/29		TCP		31920

Note:

The destination ports listed above are dependent on the values you provide to your installation. Sample values will be used for consistency within this guide.

The destination port range of 31800 and 31920 are required if you are deploying Elasticsearch.

Creating a Route Table

You should create a route table which enables the bastion node to communicate with the subnet that the Kubernetes cluster uses. In addition, you should also enable access to the bastion node from the internet.

To create a route table:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From the Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Select Route Tables from the list of resources.
4. Click Create Route Table.
5. Enter the following details:
   • Name: Enter a name for the route table. For example: bastion-route-table.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
6. Click Add Another Route Rule.
7. Enter the following information:
   • Target Type - Select Internet Gateway.
   • Destination CIDR Block - Enter 0.0.0.0/0.
   • Compartment - Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Target Internet Gateway - Select the internet gateway. For example: oke-igw-quick-clustername-id.
8. Click Create Route Table.

Creating a Subnet for the Bastion Node

After you create the security rules and route table, you should create a subnet and assign the security rules and route table to it.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From the Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Click Create Subnet.
4. Enter the following details:
   • Name: Enter a name for the subnet. For example: bastion-subnet.
   • Subnet Type: Select Regional.
   • CIDR Block: Select the subnet you want to use for the bastion network. For example: 10.0.1.0/29.
   • Route Table: Select the route table you created earlier. For example: bastion-route-table. See Creating a Route Table.
   • Subnet Access: Select Public Subnet.
   • DNS Resolution: Select Use DNS Hostnames in the subnet.
   • Security List: Select the public security list you created earlier. For example: bastion-public-seclist. See Creating a Public Security List.
5. Click Create Subnet.

Adding the Security List to the Kubernetes Node Subnet

To enable communication between the bastion subnet and the Kubernetes Cluster Subnet, you need to add the private security list to the Kubernetes Node subnet.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From your Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Click the Kubernetes node network that looks similar to oke-nodesubnet-quick-clustername-id-regional.
4. Click Add Security List.
5. Select your compartment.
6. Select the private bastion security list. For example: bastion-private-seclist.
7. Click Add Security List.
8. Repeat the above steps to add the bastion-setup-seclist security list.

Creating the Bastion Compute Instance

After defining the networking details, create the bastion node.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Compute and click Instances.
3. Click Create Instance.
4. Enter the following information:
   • Name: A name for your bastion node. For example: idm_bastion.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Placement: Select an Availability Domain.
   • Image: Select the operating system image you want to use. For example: Oracle Enterprise Linux 8.x.
   • Shape: Select an architecture and shape you want to use. For example: VM.Standard.E4.Flex.
   • Network: Select the VCN that was created when you created the Kubernetes Cluster. See Creating an OKE Cluster in OCI.
   • Subnet: Select the bastion subnet you created earlier. For example: bastion-subnet. See Creating a Subnet for the Bastion Node.
5. Click Assign public IP Address to make this instance available from the internet.
6. In the Add SSH Keys box, select Paste SSH Keys.
7. Copy the contents of the id_rsa.pub file that you created earlier. See Creating an SSH Key Pair.
8. Click Create.

The summary screen displays the public IP address assigned to the bastion node. Make a note of this address. You will need it for connecting to the node.

Connecting to the Bastion Node

You can now connect to the bastion node using the following SSH command:

ssh -i id_rsa opc@BastionIPAddress

Alternatively, if you are using SSH agent forwarding, which enables you to use your local SSH keys instead of leaving the keys (without passphrases) on the server, then you can use the following command:

ssh -A opc@BastionIPAddress

Configuring the Bastion Node

After you create the bastion node, you need to configure it. Perform the following steps to configure the bastion node:

Note:

To perform the steps in this section, you will require the following information from the Oracle Cloud Infrastructure Console:

• User OCID: To obtain your User OCID, click your profile in the OCI Console (top right) and select User Settings to view your OCID.
• Tenancy OCID: To obtain your Tenancy OCID, click your profile in the Oracle Cloud Infrastructure Console (top right) and select your tenancy to view the tenancy OCID.
• Region: The region in which you have deployed the cluster.

• Setting Up the OCI CLI to Download Kubeconfig
  
• Installing Helm
  
• Installing Git
  
• Installing X11 Packages
  
• Installing Other Packages
  
• Enabling X11 Forwarding
  
• Setting Up the Hosts File
  

Setting Up the OCI CLI to Download Kubeconfig

To set up the OCI CLI:

1. Ensure that you are using the latest version of Python by using the following command:

   python -V

   If you are using Python version 3.6, switch to the latest version by using the command:

   sudo alternatives --set python3 /usr/bin/python3.9

   Check the version again.

   Failure to do check the version may result in cryptography errors when you run the kubectl commands.

   If Python 3.9 is not available, then install it using the command:

   sudo yum install -y python39

2. Install OCI CLI.

   bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh)"

3. Respond to the prompts from the installation script.
4. To download kubeconfig later, after the set up, you need to set up the oci config file. Run the following command and enter the details when prompted:

   oci setup config

   Sample Setup:

   $ oci setup config
       This command provides a walk through of creating a valid CLI config file.
     
       The following links explain where to find the information required by this
       script:
     
       User API Signing Key, OCID and Tenancy OCID:
     
           https://docs.cloud.oracle.com/Content/API/Concepts/apisigningkey.htm#Other
     
       Region:
     
           https://docs.cloud.oracle.com/Content/General/Concepts/regions.htm
     
       General config documentation:
     
           https://docs.cloud.oracle.com/Content/API/Concepts/sdkconfig.htm
    
   Enter a location for your config [/home/opc/.oci/config]:
   Enter a user OCID: ocid1.user.oc1..xxxxxxxxxxx
   Enter a tenancy OCID: ocid1.tenancy.oc1..xxxxxxxxx
   Enter a region (e.g. ap-hyderabad-1, ap-melbourne-1, ap-mumbai-1, ap-osaka-1, ap-seoul-1, ap-sydney-1, ap-tokyo-1, ca-montreal-1, ca-toronto-1, eu-amsterdam-1, eu-frankfurt-1, eu-zurich-1, me-jeddah-1, sa-saopaulo-1, uk-gov-london-1, uk-london-1, us-ashburn-1, us-gov-ashburn-1, us-gov-chicago-1, us-gov-phoenix-1, us-langley-1, us-luke-1, us-phoenix-1): us-phoenix-1
   Do you want to generate a new API Signing RSA key pair? (If you decline you will be asked to supply the path to an existing key.) [Y/n]: Y
   Enter a directory for your keys to be created [/home/opc/.oci]:
   Enter a name for your key [oci_api_key]:
   Public key written to: /home/opc/.oci/oci_api_key_public.pem
   Enter a passphrase for your private key (empty for no passphrase):
   Private key written to: /home/opc/.oci/oci_api_key.pem
   Fingerprint: 74:d2:f2:db:62:a9:c4:bd:9b:4f:6c:d8:31:1d:a1:d8
   Config written to /home/opc/.oci/config
     
     
       If you haven't already uploaded your API Signing public key through the
       console, follow the instructions on the page linked below in the section
       'How to upload the public key':
     
           https://docs.cloud.oracle.com/Content/API/Concepts/apisigningkey.htm#How2

5. The above command creates a keyfile called oci_api_key_public.pem in $HOME/.oci. Add this key to the Oracle Cloud Infrastructure Console.
   1. Log in to the Oracle Cloud Infrastructure Console.
   2. Select Profile and click User Settings.
   3. On the User Settings screen, select API Keys .
   4. Click Add API Key.
   5. Click Paste Public Key.
   6. Copy the contents of the oci_api_key_public.pem file to the Public Key block and click Add.
6. You now need to refer the Oracle Cloud Infrastructure Console to get the remaining steps to set up the bastion node. Each deployment is different:
   1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
   2. From the Kubernetes Cluster Summary screen, click Access Cluster. A screen will be displayed with the remaining steps. These steps will include:
      • Creating a directory for the kube file.
      • Accessing the Kubeconfig file for the cluster.
      • Adding an environment variable to point to the cluster. (You should also add this variable to the .bashrc file for persistence.)

      For example:

      Sample Cluster Access Steps on the bastion node

      $ oci -v

      $ mkdir -p $HOME/.kube

      $ oci ce cluster create-kubeconfig --cluster-id ocid1.cluster.oc1.xxxxx --file $HOME/.kube/config --region us-phoenix-1 --token-version 2.0.0

      $ export KUBECONFIG=$HOME/.kube/config

      $ echo "export KUBECONFIG=$HOME/.kube/config" > $HOME/.bashrc

7. Install the kubectl client to access the cluster from the bastion node.

   Enter the following commands to download the kubectl client:

   $ curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.20.8/bin/linux/amd64/kubectl
   $ sudo mv kubectl  /bin/
   $ sudo chmod +x /bin/kubectl

   Note:

   Download the version appropriate to the version of Kubernetes you selected at the time of creating the OKE cluster. See Creating an OKE Cluster in OCI.

   If you are unsure of the Kubernetes version:

   1. Log in to the Oracle Cloud Infrastructure Console.
   2. Select Developer Services (located in Solutions and Platform), and then click Kubernetes Clusters.
8. Validate that kubectl works by using the following command:

   kubectl get nodes

Installing Helm

Helm is required by the WebLogic Operator and Oracle Unified Directory. To install Helm on to the bastion node, run the following commands:

$ curl -fsSL -o get_helm.sh
      https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ ./get_helm.sh
$ helm version
version.BuildInfo{Version:"v3.13.1",
GitCommit:"3547a4b5bf5edb5478ce352e18858d8a552a4110", 
GitTreeState:"clean", GoVersion:"go1.20.8"}

Installing Git

Git contains sample code to deploy Oracle Fusion Middleware on Kubernetes. Install GIT using the following command:

sudo yum install git -y

Installing X11 Packages

For security reasons, the Oracle HTTP Server is not installed inside the Kubernetes cluster. To install the Oracle HTTP Server, you need to install the X11 packages to enable X11 forwarding. Use the following command to install the X11 packages:

sudo yum install -y libXrender libXtst xauth xterm nc

Installing Other Packages

If you are using the automation scripts provided in this guide, you will also need to install other packages. Use the following command to install other packages:

sudo yum install -y openldap*java

For information about using automation scripts, see Automating the Identity and Access Management Enterprise Deployment.

Enabling X11 Forwarding

Configure SSHD to not use localhost for X11:

1. Open /etc/ssh/sshd_config in your preferred editor.

   sudo vi /etc/ssh/sshd_config

2. Search for the line that has "X11UseLocalhost yes" (it is commented out).
3. Remove the comment from the beginning of the line.
4. Change the yes to no.
5. Save the file.
6. Restart SSHD by using the following command:

   sudo systemctl restart sshd

Setting Up the Hosts File

When setting up, you should make curl commands to the load balancer. Because the bastion node uses the private DNS, the IP addresses returned for the load balancer end points is through the internal network that the bastion host does not have access to. To get around this issue, create an entry in the bastion hosts file for each entry point that points to the public IP address of the load balancer.

Note:

You cannot perform this step until you have created the load balancers. See Creating Load Balancers.

For example, if the public IP address of the load balancer is 129.1.1.3, add the following entry to the bastion hosts file:

129.1.1.3 login.example.com prov.example.com iadadmin.example.com igdadmin.example.com

Creating Compute Instances for Oracle HTTP Servers

The web tier resides in its own subnet separated from both the load balancer and the application tier. This section describes the procedures to create two compute instances for the web tier, place them in different availability domains, and set up security lists and route tables to facilitate access.

• Creating a Service Gateway
  
• Creating Security Lists
  
• Creating an OHS Security List
  
• Adding the OHS Security List to the Kubernetes Subnet
  
• Creating a Route Table
  
• Creating a Subnet for Web Nodes
  
• Creating the OHS Compute Instances
  
• Connecting to the OHS Nodes
  
• Configuring the OHS Nodes
  

Creating a Service Gateway

The web tier is not accessible to the internet directly. But it will require access to internal resources to perform operations such as accessing yum for adding the required packages and performing upgrades. To enable access to these internal systems, you need to create a service gateway if the system did not automatically create one for you.

To create a service gateway:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Networking and click Virtual Cloud Networks.
3. Click your Virtual Cloud Network. This is the same network that was created when you created the Kubernetes cluster. See Creating an OKE Cluster in OCI.
4. Select Service Gateways from the list of resources.
5. Click Create Service Gateway.
6. Enter the following information:
   • Name: Specify a name for the service gateway.
   • Compartment: Select the compartment you created earlier. See Creating an OCI Compartment.
   • Services: Select All IAD Services in Oracle Services Network.
7. Click Create Service Gateway.

Creating Security Lists

You need to create security lists which enable web tier nodes to communicate with the subnet that the Kubernetes cluster uses. In addition, you need to enable access to the web tier hosts from the load balancer. This section describes the minimum steps you need to perform to enable this access. You should harden your security lists to ensure that only certain machines/networks have access to this node. This part is outside the scope of this guide.

To create security lists:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From the Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Select Security Lists from the list of resources.

• Creating a Public Security List
  

Creating a Public Security List

To create a public security list:

1. Click Create Security List.
2. Enter the following details:
   • Name: Enter a name for the security list. For example: web-public-seclist.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
3. Click Add Another Ingress Rule to add an Ingress rule as described in Table 10-6 (repeat for each Ingress rule).
4. Click Add Another Egress Rule to add an Egress rule as described in Table 10-6 (repeat for each Ingress rule).
5. Click Create Security List.

Table 10-6 Description for Ingress and Egress Rules

Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Destination Port Range	Type	Code
Ingress	CIDR	0.0.0.0/0		TCP	22
Ingress	CIDR	10.0.2.0/28		TCP	80
Ingress	CIDR	10.0.2.0/28		TCP	443
Ingress	CIDR	0.0.0.0/0		ICMP		3	4
Ingress	CIDR	10.0.2.0/28		ICMP
Ingress	CIDR	10.0.2.0/28		TCP	111
Ingress	CIDR	10.0.2.0/28		TCP	2049-2050
Ingress	CIDR	10.0.2.0/28		UDP	111
Ingress	CIDR	10.0.2.0/28		UDP	2048
Ingress	CIDR	10.0.2.0/28		TCP	7777
Ingress	CIDR	10.0.1.0/29		TCP	7777
Ingress	CIDR	10.0.4.0/24		TCP	7777
Ingress	CIDR	10.0.5.0/24		TCP	7777
Ingress	CIDR	10.0.10.0/24		TCP	443
Ingress	CIDR	10.0.10.0/24		TCP	80
Ingress	CIDR	10.0.10.0/24		TCP	7777
Egress	CIDR		0.0.0.0/0	All Protocols

Note:

10.0.2.0 is the subnet you will use for the web tier nodes. You can change this value, if required.

Creating an OHS Security List

During the running of Oracle Identity Management, the web tier hosts pass through the requests to the Kubernetes services that get created as part of the provisioning process. You need to create a security list to enable this communication to take place.

1. Click Create Security List.
2. Enter the following details:
   • Name: Enter a name for the security list. For example: ohs-seclist.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
3. Click Add Another Ingress Rule to add an Ingress rule as described in Table 10-7 (repeat for each Ingress rule).
4. Click Create Security List.

Table 10-7 Description for Ingress Rules

Rule Type	Type	Source CIDR	Protocol	Destination Port Range	Comment
Ingress	CIDR	10.0.2.0/28	TCP	30701	OAM Administration Server Kubernetes Service Port
Ingress	CIDR	10.0.2.0/28	TCP	30510	OAM Policy Manager Kubernetes Service Port
Ingress	CIDR	10.0.2.0/28	TCP	30410	OAM Server Kubernetes Service Port
Ingress	CIDR	10.0.2.0/28	TCP	30711	OIG Administration Server Kubernetes Service Port
Ingress	CIDR	10.0.2.0/28	TCP	30140	OIM Server Kubernetes Service Port
Ingress	CIDR	10.0.2.0/28	TCP	30801	SOA Server Kubernetes Service Port
Ingress	CIDR	10.0.2.0/28	TCP	30901	OUDSM Server Kubernetes Service Port
Ingress	CIDR	10.0.2.0/28	TCP	30777	Nginx Ingress Controller

Note:

The destination ports listed in this table are dependent on the values you provide to your installation.

Adding the OHS Security List to the Kubernetes Subnet

The security list now needs to be added to the subnet used by Kubernetes.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From the Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Select the Kubernetes sublist from the list of displayed subnets. The subnet will have a name similar to oke-nodesubnet-<ClusterName>-<id>.
4. Click Add Security List.
5. Select the compartment and the security list you created earlier. For example: ohs-seclist. See Creating an OCI Compartment and Creating an OHS Security List.
6. Click Add Security List.

Creating a Route Table

You need to create a route table which enables the web tier nodes to communicate with the subnet that the Kubernetes cluster uses.

To create a route table:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From the Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Select Route Tables from the list of resources.
4. Click Create Route Table.
5. Enter the following details:
   • Name: Enter a name for the route table. For example: web-route-table.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
6. Click Add Another Route Rule .
7. Enter the following information:
   • Target Type - Select Service Gateway.
   • Destination Service - Select All XXX Services in Oracle Services Network.
   • Compartment - Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Target Service Gateway - Select the service gateway. For example: oke-sgw-quick-clustername-id.
8. Click Create Route Table.

Creating a Subnet for Web Nodes

Now that you have created security rules and route table, you can create a subnet and assign the security rules and route table to it.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From the Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Click Create Subnet.
4. Enter the following details:
   • Name: Enter a name for the subnet. For example: web-subnet.
   • Subnet Type: Select Regional.
   • CIDR Block: Select the subnet you want to use for the web nodes network. For example: 10.0.2.0/28.
   • Route Table: Select the route table you created earlier. For example: web-route-table. See Creating a Route Table.
   • Subnet Access: Select Private Subnet.
   • DNS Resolution: Select Use DNS Hostnames in the subnet.
   • Security List: Select the public security list you created earlier. For example: web-public-seclist. See Creating a Public Security List.
5. Click Create Subnet.

Creating the OHS Compute Instances

Now that the networking has been defined, you can create the web tier nodes.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Compute and click Instances.
3. Click Create Instance.
4. Enter the following information:
   • Name: A name for the OHS node. For example: webhost1.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Placement: Select an Availability Domain.
   • Image: Select Oracle Enterprise Linux 8.x.
   • Shape: Select an architecture and shape you want to use. For example: VM.Standard.E4.Flex.
   • Network: Select the VCN that was created when you created the Kubernetes Cluster. See Creating an OKE Cluster in OCI.
   • Subnet: Select the web-subnet you created earlier. See Creating a Subnet for Web Nodes.
5. Click Do not assign public IPv4 Address to make this instance unavailable directly from the internet.
6. In the Add SSH Keys box, select Paste SSH Keys.
7. Copy the contents of the id_rsa.pub file that you created earlier. See Creating an SSH Key Pair.

   Note:

   If you use the id_rsa key file you created on your laptop to connect to the web tier node, you must either copy that key to the web host or use SSH Agent forwarding. Alternatively, create a new key on the web tier node and use that key here.
8. Click Create.

The summary screen displays the private IP address assigned to the web tier node. Make a note of this address. You will need it for connecting to the node.

Repeat the steps for the second node. Ensure that you choose a different availability domain for the second web tier host.

Connecting to the OHS Nodes

You cannot connect to the web tier hosts directly. You must use the bastion node. You can connect to the web tier hosts after you have connected to the bastion node. See Connecting to the Bastion Node.

Note:

When you created the compute instance, you specified the SSH key you would use to connect to the host. If you used the same key as your laptop/desktop, you should use SSH Agent forwarding to connect to the web host. Alternatively, you can also use the SSH key you created on the bastion host.

You can now connect to the web host from the bastion node using the following SSH command:

ssh -i id_rsa opc@webhostIPAddress

Alternatively, if you are using SSH agent forwarding, which enables you to use your local SSH keys instead of leaving the keys (without passphrases) on the server, you can use the same pass through command:

ssh -A opc@webIPAddress

Configuring the OHS Nodes

After you create the web tier nodes, you need to configure them. Perform the following steps to configure the nodes:

• Installing X11 Packages
  
• Installing Additional Packages
  
• Enabling X11 Forwarding
  
• Preparing the Compute Instance for Use by Oracle HTTP Server
  
• Using the Firewall
  
• Creating a Software Owner Account
  
• Preparing the Hosts File
  
• Connecting to the Compute Instances to Install OHS
  

Installing X11 Packages

For security reasons, the Oracle HTTP Server is not installed inside the Kubernetes cluster. To install the Oracle HTTP Server, you have to install the X11 packages to enable X11 forwarding.

Use the following commands to install the X11 packages:

sudo yum repolist

sudo yum install -y libXrender libXtst xauth xterm nc

Installing Additional Packages

The Oracle HTTP Server requires additional packages to be present as part of the installation. Use the following command to install these additional packages:

sudo yum install -y libaio-devel* compat-libstdc++-* compat-libcap* gcc-c++-* ksh* libnsl*

Enabling X11 Forwarding

Configure SSHD to not use localhost for X11:

1. Open /etc/ssh/sshd_config in your preferred editor.
2. Search for the line that has "X11UseLocalhost yes" (it is commented out).
3. Remove the comment from the beginning of the line.
4. Change the yes to no.
5. Save the file.
6. Restart SSHD by using the following command:

   sudo systemctl restart sshd

Preparing the Compute Instance for Use by Oracle HTTP Server

You may also need to install additional Linux packages required to install the Oracle HTTP server, as well as setting the kernel parameters. See Preparing the Kubernetes Host Computers for an Enterprise Deployment.

Using the Firewall

The compute instance is created using an Oracle Linux image. The image comes with a built-in firewall, which is enabled by default. Even though you have security rules defined in your network, the Linux server rejects these requests because of the built-in Linux firewall.

You can decide to use this extra firewall or rely on your OCI security rules.

• Opening the Ports in the Firewall
  
• Disabling the Firewall
  

Opening the Ports in the Firewall

If you decide to use the firewall, you need to add firewall rules that enable every port coming in to the server to be allowed.

1. For every port that needs to be accessed, execute the following command:

   sudo firewall-cmd --permanent --add-port=YOUR PORT/tcp

   For example:

   sudo firewall-cmd --permanent --add-port=7777/tcp

2. Restart the firewall service after you configure all the ports. Use the following command to restart:

   sudo systemctl restart firewalld

3. Validate the firewall configuration by executing the following command:

   sudo firewall-cmd --list-ports

Disabling the Firewall

To disable the firewall, run the following commands:

sudo systemctl stop firewalld
sudo systemctl disable firewalld

Creating a Software Owner Account

It is not good practice to install the Oracle software using the OPC user. It is better to create a custom user to own the software. You can create a custom user by running the following commands:

sudo adduser -u 1001 oracle
sudo groupadd -g 1002 oinstall
sudo usermod -a -G oinstall oracle 
sudo usermod -g oinstall oracle

Preparing the Hosts File

The nature of the networks in an OCI environment means that the Oracle HTTP Server instances will not have access to the public load balancer. This can cause issues when the Oracle HTTP Server tries to access some virtual hosts.

In later sections, you will create a public load balancer for connections from the outside world to your system. See Creating a Public Load Balancer.

You will also create a private load balancer to allow you to route requests from the private subnets to this load balancer. See Creating a Private Load Balancer.

To ensure that the requests from the Oracle HTTP Server are directed to the private load balancer rather than the public, you should create an entry in the /etc/hosts file on the web hosts, which looks as follows:

IP ADDRESS OF PRIVATE LOAD BALANCER login.example.com

For example:

10.0.2.7 login.example.com

Connecting to the Compute Instances to Install OHS

To install the Oracle HTTP Server, you will need access to a graphical display. To get this access, you should use X11 Forwarding.

1. From your desktop/laptop, install an 'X' Window server. For example: XQuartz for MacOS.
2. SSH to the bastion server by using the following command:

   ssh -AX opc@bastionserver

3. SSH to the web server by using the following command:

   ssh -AX oracle@webserver

For detailed instructions for installing OHS, see Installing and Configuring Oracle HTTP Server.

Creating File Systems and Mount Targets

You need to create NFS file systems for Kubernetes Persistent Volumes and Oracle HTTP Server installations.

The filesystems that you have to create are described in Storage Requirements for an Enterprise Deployment.

• Overview of Preparing the File System for an Enterprise Deployment
  
• Summary of File Systems
  
• Creating a File System
  
• Setting the Mount Target Storage Reporting
  
• Creating a PV Security List
  
• Adding the Security List to the Subnet
  
• Mounting File Systems on Hosts
  

Overview of Preparing the File System for an Enterprise Deployment

It is important to set up your storage in a way that makes the enterprise deployment easy to understand, configure, and manage.

This chapter provides an overview of the process of preparing the file system for an enterprise deployment. Oracle recommends setting up your storage according to information in this chapter. The terminology defined in this chapter is used in the diagrams and procedures throughout the guide.

Summary of File Systems

See Table 4-3 for details of the file systems you have to create.

You have to mount the file systems to the bastion node only during the initial set up.

Creating a File System

To create a file system:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Storage and click File Systems.
3. Click Create File System.
4. Select Filesystem for NFS.
5. Click Edit Details in the File System Information section.
6. Enter the following details:
   • Name: Provide a name for the file system. For example: oudpv.
   • Compartment: Select the compartment you created earlier. See Creating an OCI Compartment.
7. Click Edit Details in the Export Information section.
8. Enter the following:
   • Export Path: This is the path you want to export. For example: /exports/IAMPVS/oudpv.
9. Click Edit Details in the Mount Target Information section.
10. Enter the following details:
    • Mount Target Name: Specify a name for the mount target. For example: IAMPV
    • Virtual Cloud Network: Select the VCN.
    • Subnet:
      • For the persistent volumes, select the oke-node subnet.
      • For OHS1, select the subnet you created for the web tier.
11. Click Create.

Note:

Create a new mount target only for the first persistent volume (PV). Subsequent PVs should use the same mount target.

For web tiers, create two mount targets, one in each of the OHS availability domains.

Setting the Mount Target Storage Reporting

When you install Oracle products, the installer checks the available disk storage. This check fails when you use an OCI file system. The system displays a message saying that there is insufficient disk space. To overcome this error, you can configure OCI to report a specified amount of free space.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Storage and click Mount Targets.
3. Select the OHS mount target for Availability Domain 1.
   The mount target is displayed.
4. Click Edit next to the Reported Size (GB) (it looks like a pencil).
5. Set an arbitrary size value. For example: 20.
   This value ensures that the file system, when mounted on the OHS nodes, reports 20GB of free space. This enables the OHS installer to proceed.
6. Click Save.

Creating a PV Security List

Even though you have created the mount point in the same subnet as you want to use it, you still need to create a security list to access it. The web tier entries have already been added. However, you still need to create a security list for the OHS mount target.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From your Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Select Security Lists from the list of resources.
4. Click Create Security List.
5. Enter the following details:
   • Name: Enter a name for the security list. For example: pv-seclist.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
6. Click Add Another Ingress Rule to add an Ingress rule as described in Table 10-8 (repeat for each Ingress rule).
7. Click Add Another Egress Rule to add an Egress rule as described in Table 10-8 (repeat for each Egress rule).
8. Click Create Security List.

Table 10-8 Description for Ingress and Egress Rules

Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Source Port Range	Destination Port Range
Ingress	CIDR	10.0.10.0/24		TCP		111
Ingress	CIDR	10.0.10.0/24		TCP		2048-2050
Ingress	CIDR	10.0.10.0/24		UDP		111
Ingress	CIDR	10.0.10.0/24		UDP		2048
Ingress	CIDR	10.0.1.0/29		TCP		111
Ingress	CIDR	10.0.1.0/29		TCP		2048-2050
Ingress	CIDR	10.0.1.0/29		UDP		111
Ingress	CIDR	10.0.1.0/29		UDP		2048
Ingress	CIDR	10.0.2.0/28		TCP		111
Ingress	CIDR	10.0.2.0/28		TCP		2048-2050
Ingress	CIDR	10.0.2.0/28		UDP		111
Ingress	CIDR	10.0.2.0/28		UDP		2048
Egress	CIDR		10.0.10.0/24	TCP	111
Egress	CIDR		10.0.10.0/24	TCP	2048-2050
Egress	CIDR		10.0.10.0/24	UDP	111
Egress	CIDR		10.0.1.0/29	TCP	111
Egress	CIDR		10.0.1.0/29	TCP	2048-2050
Egress	CIDR		10.0.1.0/29	UDP	111
Egress	CIDR		10.0.2.0/28	TCP	111
Egress	CIDR		10.0.2.0/28	TCP	2048-2050
Egress	CIDR		10.0.2.0/28	UDP	111

Note:

The rules for the bastion subnet are required only for the initial set up/configuration.

Adding the Security List to the Subnet

To add the security list to the subnet:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. From the Kubernetes Cluster Summary screen, click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
3. Select oke-nodesubnet.
4. Click Add Security List.
5. Select the security list you created earlier. For example: pv-seclist. See Creating a PV Security List.
6. Click Add Security List.
7. Repeat Steps 1 to 6 for the subnets web-subnet and the bastion-subnet.

Mounting File Systems on Hosts

Each mount target has a different IP address. To determine how to mount a given file system:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Storage and click File Systems.
3. Select a file system.
4. On the File System screen, select an export from the list of exports.
5. Click Mount Commands at the top of the screen, to view examples of the mount command.
6. For OHS hosts, place the entries in /etc/fstab with the following mount options:

   Sample OHS /etc/fstab entry:

   
   <IP>:/exports/IAMBINARIES/webbinaries1 /u02/private/oracle/products nfs auto,rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768
   <IP>:/exports/IAMCONFIG/webconfig1  /u02/private/oracle/config nfs auto,rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768
   

   Before you can use the file system with the containers, ensure that you can write to the file system. Mount the file system to the bastion node and write to it. If you are unable to write, use the chmod command to enable writing to the file system.

   For example:

   sudo mkdir -p /u02/private/oracle/products /u02/private/oracle/config
   sudo mount -a
   sudo chmod -R 777 /u02/private/oracle

   Table 10-9 Summary of Hosts and the File Systems to be Mounted

   Mount Host	File Systems	Comments
   webhost1	webbinaries1	Mounted as /u02/private/oracle/products.
   webhost2	webbinaries2	Mounted as /u02/private/oracle/products.
   webhost1	webconfig1	Mounted as /u02/private/oracle/config.
   webhost2	webconfig2	Mounted as /u02/private/oracle/config.
   All Kubernetes nodes	images nfs_volumes*	Used as a staging directory to temporarily store container images. Mounted as /images.
   bastion node	oudconfigpv	Mounted as /nfs_volumes/oudconfigpv.
   	oudpv	Mounted as /nfs_volumes/oudpv.
   	oudsmpv	Mounted as /nfs_volumes/oudsmpv.
   	oigpv	Mounted as /nfs_volumes/oigpv.
   	oampv	Mounted as /nfs_volumes/oampv.
   	oiripv	Mounted as /nfs_volumes/oiripv.
   	dingpv	Mounted as /nfs_volumes/dingpv.
   	oaacredpv	Mounted as /nfs_volumes/oaacredpv.
   	oaaconfigpv	Mounted as /nfs_volumes/oaaconfigpv.
   	oaalogpv	Mounted as /nfs_volumes/oaalogpv.
   	oaavaultpv	Mounted as /nfs_volumes/oaavaultpv. Note: Required when using a file-based vault.

   Optionally, mount all PVs. This option lets you delete deployments during the configuration phase, if necessary. Remove these mounts after the system is up and running.

   Note:

   * Alternatively, for these file systems, you can use block volumes.

Creating Load Balancers

You need to create two OCI load balancers. One of these load balances is used to direct public traffic and the other for internal call backs. The load balancer used for internal traffic is not available outside the OCI container.

For more information about load balancers, see Getting Started with Load Balancing.

• Creating a Public Load Balancer
  
• Creating a Private Load Balancer
  

Creating a Public Load Balancer

This load balancer directs traffic from the internet to the Oracle HTTP Servers, which in turn pass on the traffic to the Kubernetes pods.

The public load balancer will send traffic to and from the user via SSL but after the traffic moves inside the OCI Virtual Network, it is sent unencrypted. The decryption occurs due to SSL Termination. You will need to provide your own SSL certificate or create a self-signed certificate for testing purposes.

To create a public load balancer, perform the following steps:

• Creating a Self-Signed Certificate
  
• Creating a Security List
  
• Creating a Route Table
  
• Creating Subnets for the Load Balancer
  
• Creating a Load Balancer
  
• Uploading Load Balancer Certificates
  
• Creating Host Names
  
• Creating Listeners
  
• Updating the Default Listener
  

Creating a Self-Signed Certificate

You can create a self-signed certificate on any host which has access to the openssl packages. The following example is from a Linux box (in this case the bastion server was used).

For more information, see Doc ID 2617046.1.

If you prefer, you can also use a certificate provided by a recognized certificate authority.

To create a self-signed certificate:

1. Create the CA (certificate authority) private key by using the following command:

   openssl genrsa -out ca.key 2048

   Generating RSA private key, 2048 bit long modulus
   ....................+++
   .....+++
   e is 65537 (0x10001)

2. Create the Certificate Signing Request (CSR).

   openssl req -new -key ca.key -out ca.csr

   You are about to be asked to enter information that will be incorporated
   into your certificate request.
   What you are about to enter is what is called a Distinguished Name or a DN.
   There are quite a few fields but you can leave some blank
   For some fields there will be a default value,
   If you enter '.', the field will be left blank.
   -----
   Country Name (2 letter code) [XX]:CR
   State or Province Name (full name) []:SJO
   Locality Name (eg, city) [Default City]:
   Organization Name (eg, company) [Default Company Ltd]:mycompany
   Organizational Unit Name (eg, section) []:
   Common Name (eg, your name or your server's hostname) []:*.example.com
   Email Address []:
   Please enter the following 'extra' attributes
   to be sent with your certificate request
   A challenge password []:
   An optional company name []:

3. Create the CA SIGN certificate that will be used to sign the new certificates.

   openssl x509 -req -in ca.csr -signkey ca.key -out ca.crt

   Signature ok
   subject=/C=CR/ST=SJO/L=Default City/O=mycompany/CN=*.example.com
   Getting Private key

4. Create the private key for the load balancer.

   openssl genrsa -out loadbalancer.key 2048

   Generating RSA private key, 2048 bit long modulus
   ....................+++
   .....+++
   e is 65537 (0x10001)

5. Create the CSR for the load balancer.

   openssl req -new -key loadbalancer.key -out loadbalancer.csr

   You are about to be asked to enter information that will be incorporated
   into your certificate request.
   What you are about to enter is what is called a Distinguished Name or a DN.
   There are quite a few fields but you can leave some blank
   For some fields there will be a default value,
   If you enter '.', the field will be left blank.
   -----
   Country Name (2 letter code) [XX]:CR
   State or Province Name (full name) []:SJO
   Locality Name (eg, city) [Default City]:
   Organization Name (eg, company) [Default Company Ltd]:mycompany
   Organizational Unit Name (eg, section) []:
   Common Name (eg, your name or your server's hostname) []:*.example.com
   Email Address []:
   Please enter the following 'extra' attributes
   to be sent with your certificate request
   A challenge password []:
   An optional company name []:

6. Sign the certificate with the CA certificate.

   openssl x509 -req -in loadbalancer.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out loadbalancer.crt -days 50000

   Signature ok
   subject=/C=CR/ST=SJO/L=Default City/O=mycompany/CN=*.example.com
   Getting CA Private Key

7. Check that the certificate is signed by the CA.

   openssl x509 -in loadbalancer.crt -text

   Certificate:
   Data:
   Version: 1 (0x0)
   Serial Number:
   df:e7:c9:6a:56:e5:e4:c9
   Signature Algorithm: sha256WithRSAEncryption
   Issuer: Issuer: C=CR, ST=SJO, L=Default City, O=mycompany, CN=*.example.com <==== here signed by my ca..
   Validity
   Not Before: Dec 3 16:34:58 2019 GMT
   Not After : Oct 25 16:34:58 2156 GMT
   Subject: =/C=CR/ST=SJO/L=Default City/O=mycompany/CN=*.example.com
   Subject Public Key Info:
   Public Key Algorithm: rsaEncryption
   Public-Key: (2048 bit)
   Modulus:
   00:da:62:ce:69:77:ff:45:b0:84:9f:af:53:44:97:
   13:28:91:44:cd:0b:1d:e5:a1:f6:a3:ef:f8:98:19:
   8d:c2:56:a0:e1:80:1c:e0:0e:ae:34:9a:a8:ae:52:
   d4:71:a4:da:10:8b:fd:df:73:0d:8e:98:ef:d4:7b:
   36:f1:1c:5a:d7:24:88:63:f5:b2:6b:7a:62:50:3a:
   e7:3a:3d:9a:b7:41:db:8e:f5:e8:91:46:48:cf:0c:
   54:da:7b:da:20:76:b6:eb:4b:cb:fa:36:09:f7:94:
   ea:c9:53:3f:b2:bc:66:4c:6d:7f:3f:09:cc:cd:c2:
   10:1f:39:0f:6c:1d:49:7c:db:99:d9:d9:7d:48:dd:
   09:52:50:9d:f5:44:fd:2e:48:f2:78:22:20:3c:07:
   b6:a1:4d:f8:17:82:67:a1:45:52:0a:21:78:ed:1b:
   ca:45:79:16:21:c9:e3:2f:a4:93:d4:bf:67:68:7a:
   b6:d9:8f:e1:53:35:31:a6:17:38:f2:a6:79:b5:12:
   6b:36:f2:2d:69:56:c2:d9:c0:89:d9:31:6b:06:0c:
   1e:ba:a6:30:88:32:7b:92:e4:af:11:ab:37:1a:cb:
   cf:4b:4c:7d:ff:a7:4d:f8:be:cd:98:17:63:83:06:
   cf:e7:ae:4a:d5:6e:6b:e4:0d:f3:6f:70:52:2b:8b:
   12:83
   Exponent: 65537 (0x10001)
   Signature Algorithm: sha256WithRSAEncryption
   d8:36:2e:2e:42:72:76:15:ec:a8:3a:e9:dd:2d:2e:28:42:97:
   48:4e:6f:33:ec:df:3e:a3:11:19:8b:62:d5:89:07:af:b5:ff:
   b6:de:d7:5c:8b:7a:46:37:46:da:b7:44:7f:b6:cc:c8:a9:1e:
   f9:ca:0f:76:2b:29:d2:4c:6a:af:18:9b:1a:62:42:87:e6:21:
   b7:09:15:8d:b3:1d:05:4a:4d:1b:d1:07:00:cd:69:40:92:ed:
   f9:3d:24:c9:b7:b9:00:7e:c3:f9:73:42:7f:13:34:a8:d1:e4:
   32:91:08:51:07:a5:d0:ab:42:fb:83:c4:a7:b5:94:0f:2a:56:
   8b:95:34:1b:63:5b:39:59:88:9b:9f:34:91:98:dc:8c:0a:0e:
   01:f9:b2:6e:fd:2e:95:28:4c:76:dd:fe:a0:3f:f1:16:3b:88:
   cd:e5:0a:f3:dd:52:0d:39:2a:60:2c:f0:5d:79:3b:7e:99:43:
   3b:47:33:85:f9:7c:f1:e8:cb:3d:cd:ab:4c:1f:a2:72:99:70:
   f4:8d:92:4a:24:9e:37:96:ad:24:d5:13:33:05:32:ae:d5:58:
   ed:3e:32:6f:a7:1e:a8:61:a5:fb:73:ea:54:46:b7:07:77:07:
   9a:9d:af:eb:66:5c:55:f1:50:23:fb:da:d9:b7:4b:0b:6d:bb:
   c7:39:18:ae
   -----BEGIN CERTIFICATE-----
   MIIDUDCCAXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXAQsFADBaMQswCQYDVQQGEwJD
   UjEMMAoGA1UECAwDU0pPMRUwEwYDVQQHDAxEZWZhdWx0IENpdHkxDzANBgNVBAoM
   Bm9yYWNsZTEVMBMGA1UEAwwMKi5vcmFjbGUuY29tMCAXDTE5MTIwMzE2MzQ1OFoY
   DzIxNTYxMDI1MTYzNDU4WjB4MQswCQYDVQQGEwJDUjEMMAoGA1UECAwDU0pPMRUw
   EwYDVQQHDAxEZWZhdWx0IENpdHkxHDAaBgNVBAoME0RlZmF1bHQgQ29tcGFueSBM
   dGQxDzANBgNVBAsMBm9yYWNsZTEVMBMGA1UEAwwMKi5vcmFjbGUuY29tMIIBIjAN
   BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA2mLOaXf/RbCEn69TRJcTKJFEzQsd
   5aH2o+/4mBmNwlag4YAc4A6uNJqorlLUcaTaEIv933MNjpjv1Hs28Rxa1ySIY/Wy
   a3piUDrnOj2at0HbjvXokUZIzwxU2nvaIHa260vL+jYJ95TqyVM/srxmTG1/PwnM
   zcIQHzkPbB1JfNuZ2dl9SN0JUlCd9UT9LkjyeCIgPAe2oU34F4JnoUVSCiF47RvK
   RXkWIcnjL6ST1L9naHq22Y/hUzUxphc48qZ5tRJrNvItaVbC2cCJ2TFrBgweuqYw
   iDJ7kuSvEas3GsvPS0x9/6dN+L7NmBdjgwbP565K1W5r5A3zb3BSK4sSgwIDAQAB
   MA0GCSqGSIb3DQEBCwUAA4IBAQDYNi4uQnJ2FeyoOundLS4oQpdITm8z7N8+oxEZ
   i2LViQevtf+23tdci3pGN0bat0R/tszIqR75yg92KynSTGqvGJsaYkKH5iG3CRWN
   sx0FSk0b0QcAzWlAku35PSTJt7kAfsP5c0J/EzSo0eQykQhRB6XQq0L7g8SntZQP
   KlaLlTQbY1s5WYibnzSRmNyMCg4B+bJu/S6VKEx23f6gP/EWO4jN5Qrz3VINOSpg
   LPBdeTt+mUM7RzOF+Xzx6Ms9zatMH6JymXD0jZJKJJ43lq0k1RMzBTKu1VjtPjJv
   px6oYaX7c+pURrcHdweana/rZlxV8VAj+9rZt0sLbbvHORiu
   -----END CERTIFICATE-----

This procedure creates the following files to be used later. See Uploading Load Balancer Certificates.

• ca.crt
• loadbalancer.crt
• loadbalancer.key

Creating a Security List

The security list determines who can access the load balancer and where the load balancer is allowed to send requests.

To create a private security list:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Virtual Cloud Networks.
3. Click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
4. Select Security Lists from the list of resources.
5. Click Create Security List.
6. Enter the following details:
   • Name: Enter a name for the security list. For example: public-lbr-seclist.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
7. Click Add Another Ingress Rule to add an Ingress rule as described in Table 10-10 (repeat for each Ingress rule).
8. Click Add Another Egress Rule to add an Egress rule as described in Table 10-10 (repeat for each Egress rule).
9. Click Create Security List.

Table 10-10 Description for Ingress and Egress Rules

Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Destination Port Range
Ingress	CIDR	0.0.0.0/0		TCP	80
Ingress	CIDR	0.0.0.0/0		TCP	443
Egress	CIDR		10.0.2.0/28	TCP	7777

Note:

10.0.2.0 is the subnet you will use for the web tier.

Creating a Route Table

You need to create a route table which enables the load balancer to communicate with the internet.

To create a route table:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Virtual Cloud Networks.
3. Click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
4. Select Route Tables from the list of resources.
5. Click Create Route Table.
6. Enter the following details:
   • Name: Enter a name for the route table. For example: lbr-route-table.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
7. Click Add Another Route Rule .
8. Enter the following information:
   • Target Type: Select Internet Gateway.
   • Destination CIDR: Enter 0.0.0.0/0.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Target Internet Gateway: Select the internet gateway. For example: oke-igw-quick-clustername-id.
9. Click Create.

Creating Subnets for the Load Balancer

The public load balancer is placed into an isolated subnet. The load balancer is created as a pair so that if one fails, the second one takes on the work load. The load balancers reside in availability domains and a different subnet is created for each load balancer. By using different subnets for the load balancers, you create stricter access rules enabling public access only to the load balancer but not the components for which it load balances.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Virtual Cloud Networks.
3. Click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
4. Click Create Subnet.
5. Enter the following details:
   • Name: Enter a name for the subnet. For example: lbr-subnet1.
   • Subnet Type: Select Availability Domain Specific.
   • Availability Domain: Select an availability domain.
   • CIDR Block: Select the subnet you want to use for the load balancer network. For example: 10.0.4.0/24.
   • Route Table: Select the route table you created earlier. For example: lbr-route-table. See Creating a Route Table.
   • Subnet Access: Select Public Subnet.
   • DNS Resolution: Select Use DNS Hostnames in the subnet.
   • DHCP Options: Select Default DHCP Options for the VCN.
   • Security List: Select the public security list you created earlier. For example: public-lbr-seclist. See Creating a Security List.
6. Click Create Subnet.

Create a second subnet by selecting a different availability domain and a different CIDR block. For example: 10.0.5.0/24.

Creating a Load Balancer

To create a load balancer:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click Create Load Balancer.
4. Select Load Balancer and click Create Load Balancer.
5. Enter the following information:
   • Name: Enter a name for the load balancer. For example: public-loadbalancer.
   • Visibility Type: Select Public.
   • Select Assign a Public IP Address and Ephemeral IP Address unless you want to use a specific IP address, in which case select Reserved IP Address.
   • Shapes: Select Flexible Shape.
   • Bandwidth: Select the anticipated bandwidth.
   • Virtual Cloud Network: Select the Virtual Cloud Network.
   • Subnet: Select both the load balancer subnets you created earlier. See Creating Subnets for the Load Balancer.
6. Click Next.
7. On the Choose Back Ends screen, select your preferred Load Balancing Policy.
8. Click Add Back Ends.
   1. Select Web Server Instances.
   2. Click Add Selected Back Ends.
   3. Change the port to the listen port for Oracle HTTP Server. For example: 7777.
9. In the Specify Health Check Policy screen, change the port to the HTTP server port. For example: 7777.
10. Click Show Advanced Options and enter a name for the back-end set. For example: ohs_servers.
11. Click Next.
12. On the Configure Listener screen, enter the following information:

    Note:

    You will need one listener for each entry point. However, you can add only one listener at this point.

    • Name: Select a name for the listener. For example: iadadmin.
    • Traffic Type: Select the traffic type the listener uses. iadadmin.example.com uses HTTP.
    • Port: Select the load balancer port. iadadmin.example.com uses port 80.
13. Click Next.
14. In the Manage Logging screen, ensure that Create a New Log Group is selected.
15. Change Name. For example: Public_Lbr.
16. Change Log Name. For example: Public_lbr_error.
17. Click Submit.

Uploading Load Balancer Certificates

As the load balancer routes SSL requests, you need to upload the certificates for the load balancer. If you have created a self-signed certificate, add the details of that certificate. If you have your own certificates, upload those.

To upload the certificates:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click the load balancer. For example: public_loadbalancer.
4. Select Certificates from the resource list.
5. Select Load Balancer Managed Certificate.
6. Click Add Certificate.
7. Enter the following information. You can either upload the files directly or paste the contents of the files.
   • Certificate Name: Enter a name for the certificate. For example: Loadbalancer.
   • SSL Certificate: Include the contents of the loadbalancer.crt file.
   • CA Certificate: Select the Specify CA Certificate check box to include the contents of the ca.crt file.
   • Private Key: Select the Specify Private Key check box to include the contents of the loadbalancer.key file.

   See Creating a Self-Signed Certificate.

8. Click Add Certificate.

Creating Host Names

Host names are used to filter the different entry points into the load balancer. You need to create a host name for each load balancer virtual host described in Summary of the Load Balancer Virtual Servers Required for an Enterprise Deployment.

You have to create the following host names:

• iadadmin.example.com
• igdadmin.example.com
• login.example.com
• prov.example.com

To create the load balancer host names:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click the load balancer. For example: public_loadbalancer.
4. Select Hostnames from the resource list.
5. Click Create Hostname.
6. Enter the following information:
   • Name: Enter a name for the host name. For example: iadadmin.
   • Hostname: Enter the fully qualified host name. For example: iadadmin.example.com.
7. Click Create.
8. Repeat for each host name to be created.

Note:

If you want to limit the admin access to users inside the network, you should create the hosts iadadmin.example.com and igdadmin.example.com in the private load balancer.

Creating Listeners

You need to create a listener for each host name you have created earlier. See Creating Host Names. The iadadmin listener has been created at the time of creating the load balancer. See Creating a Load Balancer.

Table 10-11 Summary of Public Load Balancer Listeners

Name	Protocol	Port	SSL	Backend Set	Host Name
iadadmin	http	80		ohs_servers	iadadmin.example.com
igdadmin	http	80		ohs_servers	igdadmin.example.com
login	https	443	Yes	ohs_servers	login.example.com
prov	https	443	Yes	ohs_servers	prov.example.com

To create the load balancer listeners:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click the load balancer. For example: public_loadbalancer.
4. Select Listeners from the resource list.
5. Click Create Listener.
6. Enter the following information:
   • Name: Enter a name for the listener. For example: login.
   • Protocol: Select https.
   • Port: Specify 443.
   • Certificate Name: Ensure that the certificate you created for the load balancer is displayed. If not displayed, select the certificate.

     Note:

     This option will be available only if you use the HTTPS protocol.
   • Hostname: Select login.
   • Backend Set: Select the back end set. For example: ohs_servers.
7. Click Create Listener.
8. Repeat the steps to create the remaining listeners.

Note:

If you want to limit the admin access to users inside the network, create the listeners iadadmin.example.com and igdadmin.example.com in the private load balancer.

Updating the Default Listener

When you created the load balancer, a default listener also gets created. You have to assign the newly created host name to this listener.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click the load balancer. For example: public-loadbalancer.
4. Select Listeners from the resource list.
5. To edit the listener, click the three dots next to the name, and then click Edit.
6. Set the host name to the host name you created earlier. For example: iadadmin. See Creating Host Names.
7. Click Update Listener.

Creating a Private Load Balancer

The private load balancer, which is used to route internal call backs, resides in the same subnet as the Oracle web servers. This load balancer services requests generated from inside the application.

Note:

Web servers issue curl commands to login.example.com. Therefore, you also need to define this on the private load balancer because the web servers do not have direct access to the public load balancer. You can use the same certificates that you used when you created the public load balancer.

To create a private load balancer, perform the following steps:

• Creating a Load Balancer
  
• Creating Host Names
  
• Updating the Default Listener
  
• Uploading Load Balancer Certificates
  
• Creating Listeners
  

Creating a Load Balancer

To create a load balancer:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click Create Load Balancer.
4. Select Load Balancer and click Create Load Balancer.
5. Enter the following information:
   • Name: Enter a name for the load balancer. For example: internal-loadbalancer.
   • Visibility Type: Select Private.
   • Shapes: Select Flexible Shape.
   • Bandwidth: Select the anticipated bandwidth.
   • Virtual Cloud Network: Select the Virtual Cloud Network.
   • Subnet: Select the same subnet as the web servers. For example: web-subnet. See Creating Subnets for the Load Balancer.
6. Click Next.
7. On the Choose Back Ends screen, select your preferred Load Balancing Policy.
8. Click Add Back Ends.
   1. Select Web Server Instances.
   2. Click Add Selected Back Ends.
   3. Change the port to the listen port for Oracle HTTP Server. For example: 7777.
9. In the Specify Health Check Policy screen, change the port to the HTTP server port. For example: 7777.
10. Click Show Advanced Options and enter a name for the back-end set. For example: ohs_servers.
11. Click Next.
12. On the Configure Listener screen, enter the following information:
    • Name: Select a name for the listener. For example: igdinternal.
    • Traffic Type: Select the traffic type the listener uses. igdinternal uses HTTP.
    • Port: Select the load balancer port. igdinternal uses port 7777.
13. Click Next.
14. In the Manage Logging screen, ensure that Create a New Log Group is selected.
15. Click Submit.

Creating Host Names

Host names are used to filter the different entry points into the load balancer. You need to create a host name for each load balancer virtual host described in Summary of the Load Balancer Virtual Servers Required for an Enterprise Deployment.

You have to create the following host names:

• igdinternal.example.com
• login.example.com
• iadadmin.example.com
• igdadmin.example.com

Note:

login.example.com is defined here for internal traffic routing. The EDG uses network segregation. If you do not define it here, calls to login.example.com will attempt to communicate using the public network and fail.

To create the load balancer host name:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click the load balancer. For example: internal_loadbalancer.
4. Select Hostnames from the resource list.
5. Click Create Hostname.
6. Enter the following information:
   • Name: Enter a name for the host name. For example: igdinternal.
   • Hostname: Enter the fully qualified host name. For example: igdinternal.example.com.
7. Click Create.
8. Repeat Steps 5 through 7 to create each of the required host names.

Updating the Default Listener

When you created the load balancer, a default listener also gets created. You have to assign the newly created host name to this listener.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click the load balancer. For example: internal_loadbalancer.
4. Select Listeners from the resource list.
5. To edit the listener, click the three dots next to the name, and then click Edit.
6. Set the host name to the host name you created earlier. For example: igdinternal. See Creating Host Names.
7. Click Update Listener.

Uploading Load Balancer Certificates

As the load balancer routes SSL requests, you need to upload the certificates for the load balancer. If you have created a self-signed certificate, add the details of that certificate. If you have your own certificates, upload those.

To upload the certificates:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click the load balancer. For example: internal_loadbalancer.
4. Select Certificates from the resource list.
5. Click Add Certificate.
6. Enter the following information. You can either upload the files directly or paste the contents of the files.
   • Name: Enter a name for the certificate. For example: Loadbalancer.
   • SSL Certificate: Include the contents of the loadbalancer.crt file.
   • CA Certificate: Select the Specify CA Certificate check box to include the contents of the ca.crt file.
   • Private Key: Select the Specify Private Key check box to include the contents of the loadbalancer.key file.

   See Creating a Self-Signed Certificate.

7. Click Add Certificate.

Creating Listeners

You need to create a listener for each host name you have created earlier. See Creating Host Names.

Table 10-12 Summary of Private Load Balancer Listeners

Name	Protocol	Port	SSL	Backend Set	Host Name
igdinternal	http	7777	No	ohs_servers	igdinternal.example.com
login	https	443	Yes	ohs_servers	login.example.com
iadadmin	http	80	No	ohs_servers	iadadmin.example.com
igdadmin	http	80	No	ohs_servers	igdadmin.example.com

To create the load balancer listeners:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click the load balancer. For example: internal_loadbalancer.
4. Select Listeners from the resource list.
5. Click Create Listener.
6. Enter the following information:
   • Name: Enter a name for the listener. For example: login.example.com.
   • Protocol: Select HTTPS.
   • Port: Specify 443.
   • Certificate Name: Ensure that the certificate you created for the load balancer is displayed. If not displayed, select the certificate.
   • Hostname: Select login.
   • Backend Set: Select the back end set. For example: ohs_servers.
7. Click Create Listener.

Creating a Network Load Balancer

This step is required only if you want to configure a load balancer to route traffic to the Kubernetes worker nodes.

To create a network load balancer:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Load Balancers.
3. Click Create Load Balancer.
4. Select Network Load Balancer and click Create Load Balancer.
5. In the Create Network Loadbalancer section, specify the following information:
   1. Load Balancer Name: Select a name for your load balancer. For example: k8workers.
   2. Visibility Type: Select Private.
   3. Virtual Cloud Network: Select Virtual Cloud Network.
   4. Subnet: Select the same subnet as the Kubernetes worker nodes. For example: one-nodesubnet-quick-<clustername>-<id>.
   5. Compartment: Select the compartment.
6. Click Next.
7. In the Listener screen, specify the following information:
   1. Listener: Select TCP.
   2. Type: Select TCP.
   3. Select Use Any Port.
8. Click Next.
9. In the Backend Set screen, specify the following information:
   1. Backend Set Name: Select K8Workers.
   2. Click Add Backends.
   3. In the Backends screen, ensure that all the worker nodes are selected, and then click Add Backends.
   4. Health Check Policy - Select TCP and set the port to 22. Use the default values for all other values.
10. Click Next.
11. Review the details and click Create.

Creating a Database

There are several different databases that you can create in OCI. For this example, a bare metal RAC database will be created. You may need to create one or more databases.

See Preparing an Existing Database for an Enterprise Deployment for details on the databases and services you should create. This section shows an example of creating one of these databases in OCI.

• Creating a Security List
  
• Creating a Route Table
  
• Creating Subnets for the Database
  
• Creating the Database
  
• Creating a Secondary Pluggable Database
  
• Connecting to the Database Node
  
• Configuring the Database
  

Creating a Security List

To create a private security list:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Virtual Cloud Networks.
3. Click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
4. Click Create Security List.
5. Enter the following details:
   • Name: Enter a name for the security list. For example: db-seclist.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
6. Click Add Another Ingress Rule to add an Ingress rule as described in Table 10-13 (repeat for each Ingress rule).
7. Click Add Another Egress Rule to add an Egress rule as described in Table 10-13 (repeat for each Egress rule).

Table 10-13 Description for Ingress and Egress Rules

Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Destination Port Range
Ingress	CIDR	0.0.0.0/0		TCP	22
Ingress	CIDR	10.0.11.0/24		TCP	1521
Ingress	CIDR	10.0.11.0/24		TCP	6200
Ingress	CIDR	10.0.10.0/24		TCP	1521
Ingress	CIDR	10.0.10.0/24		TCP	6200
Ingress	CIDR	10.0.1.0/29		TCP	1521 Note: Used for set up only.
Egress	CIDR		0.0.0.0/0	All Protocols

Note:

10.0.11.0 is the subnet to use for the database. You can change this value, if required.

Creating a Route Table

You need to create a route table which enables the database to communicate with the OKE cluster.

To create a route table:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Virtual Cloud Networks.
3. Click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
4. Select Route Tables from the list of resources.
5. Click Create Route Table.
6. Enter the following details:
   • Name: Enter a name for the route table. For example: db-route-table.
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
7. Click Add Another Route Rule .
8. Enter the following information:
   • Target Type - Select Service Gateway.
   • Destination Service - Select All XXX Services in Oracle Services Network.
   • Compartment - Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Target Service Gateway - Select the service gateway. For example: oke-sgw-quick-clustername-id.
9. Click Create Route Table.

Creating Subnets for the Database

The database is placed into an isolated subnet.

To create subnets for the database:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking and click Virtual Cloud Networks.
3. Click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
4. Click Create Subnet.
5. Enter the following details:
   • Name: Enter a name for the subnet. For example: db-subnet.
   • Subnet Type: Select Regional.
   • CIDR Block: Select the subnet you want to use for the database network. For example: 10.0.11.0/24.
   • Route Table: Select the route table you created earlier. For example: db-route-table. See Creating a Route Table.
   • Subnet Access: Select Private Subnet.
   • DNS Resolution: Select Use DNS Hostnames in the subnet.
   • DHCP Options: Select Default DHCP Options for the VCN.
   • Security List: Select the security list you created earlier. For example: db-seclist. See Creating a Security List.
6. Click Create Subnet.

Creating the Database

After establishing the network, you can now create the database.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Oracle Database and click Oracle Base Database (VM, BM).
3. Click on Create DB System.
4. Enter the following information:
   • Compartment: Select the name of the compartment you created earlier. See Creating an OCI Compartment.
   • Name: Enter a name for the database infrastructure. For example: Identity_Management_Databases.
   • Availability Domain: Select an availability domain.
   • Shape Type: For this example, select Virtual Machine.
   • Choose a Shape: This value depends on your sizing requirements.
   • Configure DB System: Select a node count greater than 1.
   • Storage Management Software: Select Oracle Grid Infrastructure.
   • Configure Storage: Select the sizing requirements for your storage.
   • In the Add SSH Keys box, select Paste SSH Keys.
   • Copy the contents of the id_rsa.pub file that you created earlier. See Creating an SSH Key Pair.
   • License Type: Select the type of database license you have.
   • Virtual Cloud Network: Click the VCN Name that looks similar to oke-vcn-quick-clustername-id.
   • Client Subnet: Select the DB subnet. For example: db-subnet.
   • Host Name Prefix: Select a host name prefix. For example: db.
   • Database Unique Name Suffix: Set it to a value unique to your system, which is especially important if you are going to create a disaster recovery site. The best practice is to set the suffix to the abbreviated region. For example: lon for London.
   • Database Image: Select the database release you want to use. For example: 21c.
5. Click Next.
6. On the Database Information screen, enter the following information:
   • Database Name: Select a name for your database. For example: iamdb1.
   • PDB Name : Enter a name for the Oracle Access Manager PDB you want to create. For example: iadpdb.
   • Sys Password : Select a password you want to assign to the database SYS account.
   • Workload Type: Select Transaction Processing.
   • Configure Database Backups: Select Enable automatic backups.
   • Backup Retention Period: Specify the period fo which you want to keep the database backups.
   • Backup Scheduling: Specify the preferred time to initiate the backup.
7. Click Create DB System.
8. After the database is created, note the following values for use at a later point:
   • SCAN DNS Name: This is the host name you use to connect to the database.
   • db node 1 and db node 2: To obtain the names/IP addresses of these nodes, click Nodes from the resources list.

Note:

After the database is created, OCI adds a suffix to the database name. Ensure that you use the complete name including this suffix when configuring the database as described in Preparing an Existing Database for an Enterprise Deployment.

Creating a Secondary Pluggable Database

When you create the database, it creates a single pluggable database (PDB). A single PDB may be sufficient for your needs.

However, if you require more PDBs so that OAM, OIG, OIRI, and OAA use different PDBs in the same database, you have to create additional PDBs. See Creating a PDB Using an Existing PDB as a Template.

You can do this either at the database level or, if you are using Oracle OCI, through the OCI console. For adding a PDB at the database level, see Creating a PDB Using an Existing PDB as a Template.

Alternatively, you can create extra pluggable databases by using the OCI Console.

1. Log in to the Oracle Cloud Infrastructure for your tenancy.
2. Navigate to Oracle Database and click Oracle Base Database (VM, BM).
3. Click the DB system hosting the database.
4. Click the Container Database Name from the list of displayed databases.
5. Click Pluggable Databases in the Resources menu.
6. Click Create Pluggable Database.
7. Add a name for the new pluggable database and enter the TDE wallet password for the container database. This password may be the same as the database SYS password if you opt not to set an explicit value.
8. Click Create Pluggable Database.
9. Repeat Step 6 through Step 8 for each additional pluggable database you require.

Connecting to the Database Node

You can now connect to the database node using the following SSH command:

ssh -A opc@databaseNodeIPAdddress

Connect to DB node 1 from the bastion node using the command:

ssh -A opc@dbnode1

After you connect to DB node 1 as opc, connect to the oracle user using the following command:

sudo su - oracle

Configuring the Database

After you create the skeletal database, you should configure the database as described in Preparing an Existing Database for an Enterprise Deployment.

Creating a Vault

A vault is used to store the credentials of your deployment. At present, the only Oracle Identity and Access Management product using a vault is Oracle Advanced Authentication (OAA). OAA can use either an OCI-based vault (recommended) or a file-based vault.

If you are planning to use an OCI-based vault, create the vault using the following steps:

1. Log in to the Oracle Cloud Infrastructure for your tenancy.
2. Select Identity and Security and click Vault.
3. Click Create Vault and specify the following details:
   1. Compartment - Select the compartment you created earlier. See Creating an OCI Compartment.
   2. Name – Enter a name for the vault. For example: oaavault.
   3. Select Make it a private vault.
   4. Click Create Vault.

• Creating the Vault Key
  
• Creating the API Key
  

Creating the Vault Key

To create the vault key:

1. Click the name of the newly created vault. For example: oaavault.
2. Click Create Key.
3. Enter the following information:
   • Compartment - Select the compartment you created earlier. Creating an OCI Compartment.
   • Protections Mode: Software.
   • Name - Enter a name for the key. For example: vaultkey.
4. Click Create Key.

Creating the API Key

To create the API key:

1. Log in to the Oracle Cloud Infrastructure Console.
2. Select Profile and click User Settings.
3. On the User Settings screen, select API Keys.
4. Click Add API Key.
5. Click Download API Key. Keep this file safe.
6. Click Add.
7. Click Close.

Creating a DNS Server

This is an optional task. It is important that all host names are resolvable, including the load balancer virtual hosts. You can make them resolvable by adding entries to the local hosts files. However, in OCI, using a private DNS server is the simpler method.

By default, the compute hosts are configured to use a private DNS server. You have to add the entries only for the local hosts.

• Creating a DNS Zone
  
• Creating DNS Records
  

Creating a DNS Zone

To create a DNS zone:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Navigate to Networking, select DNS Management, and then click Zones.
3. Click Private Zones.
4. Click Create Zone.
5. Enter the following information:
   • Name: Enter a name for the zone. For example: example.com.
   • Select Existing DNS Private View.
   • DNS Private View: Select Virtual Cloud Network.
6. Click Create.

Creating DNS Records

After you create the zone, you can create records in the zone for each host. There are two types of DNS records that have to be created:

• A Record: This is an IP address association with a host name.
• CNAME: This is an alias for the A Record.

If you have multiple hosts using the same IP address, Oracle recommends you to create one 'A Record' and multiple 'CNAME' records.

To create a record:

1. Click Add Record.
2. Select the Record Type: A or CNAME.
3. Specify the name of the host in the domain. For example: loadbalancer.example.com.
4. Specify the Address which is the IP Address of the host. For example: The IP address of the public load balancer.

   OR

   Specify the Target which is the name of the A record with which you want to associate the alias.

5. Set the TTL value to 86400. If the TTL field is disabled, select the lock icon at the end of the row to specify a value.
6. Click Submit.

   Note:

   To continue adding another record, select the ADD ANOTHER RECORD check box. After you click Submit, the Add Record screen remains open to add another record.

   You have to create the following entries:

   Table 10-14 DNS Record Type and the Associated Host Name

   Host Name	Type	Target	Address
   loadbalancer.example.com	A		IP address of the Internal load balancer.
   iadadmin.example.com	CNAME	loadbalancer.example.com
   igdadmin.example.com	CNAME	loadbalancer.example.com
   login.example.com	CNAME	loadbalancer.example.com
   prov.example.com	CNAME	loadbalancer.example.com
   igdinternal.example.com	A		IP address of the Internal load balancer.
   webhost1.example.com	A		IP address of WEBHOST1.
   webhost2.example.com	A		IP address of WEBHOST2.

7. After entering all your entries, click Publish to ensure that they are made available.

Updating Kubernetes CoreDNS

The Kubernetes cluster resolves hostnames using the built-in CoreDNS server. By default, this server will not interact with the corporate DNS server. You must configure this server to either perform local hostname resolution for the application end points or to resolve those end points using the corporate DNS server.

To configure the CoreDNS server:

1. Edit the CoreDNS configmap using the following command:

   kubectl edit configmap/coredns -n kube-system

2. Add a hosts section to the file including one entry for each of the hosts you want to define. For example:

   apiVersion: v1
   data:
     Corefile: |
       .:53 {
           errors
           health {
              lameduck 5s
           }
           ready
           kubernetes cluster.local in-addr.arpa ip6.arpa {
              pods insecure
              fallthrough in-addr.arpa ip6.arpa
              ttl 30
           }
           prometheus :9153
           forward . /etc/resolv.conf {
              max_concurrent 1000
           }
           cache 30
           loop
           reload
           loadbalance}
       example.com:53 {
           errors
           cache 30
           forward . CORPORATE_DNS_IP_ADDRESS
           }
       }
   kind: ConfigMap
   metadata:
     creationTimestamp: "2021-08-13T13:01:56Z"
     name: coredns
     namespace: kube-system
     resourceVersion: "11587286"
     uid: 2facd555-692d-4dfd-80be-5f9e608b0d71

3. Save the file.
4. Restart CoreDNS using the command:

   kubectl rollout restart -n kube-system deploy coredns

   Ensure that the CoreDNS pods restart without any issue, using the command:

   kubectl get pods -n kube-system

   If any errors occur, use the following command to view them:

   kubectl logs -n kube-system coredns--<ID>

   Correct the errors by editing the configmap again.

Validating Your Environment

Perform the checks described in this section to ensure that your environment is ready for a deployment.

For the bastion node

• Check the network connectivity

  ping webhost1.example.com

  ping webhost2.example.com

• Resolve the public address of the load balancer

  ping login.example.com

  ping prov.example.com

• Check that Kubernetes is working

  kubectl get nodes

  Ping each of the worker nodes that are listed as the output of the above command.

From the Web Tier

• Ping the Kubernetes worker nodes

  nslookup k8workers

• Resolve the public address of the load balancer

  ping login.example.com

Preparing a Disaster Recovery Environment

A disaster recovery environment is a replica of your primary environment located in a different region from the primary region. This environment is a standby environment that you switch over to in the event of the failure of your primary environment.

The standby environment will be a separate cluster, ideally in a different data center. If the cluster is dedicated to the application, the second cluster should be a mirror of the primary cluster with the same number and specifications of worker nodes. If your cluster is a multi-purpose cluster that is used by different applications, ensure sufficient spare capacity in the standby site to run the full application workload of the primary cluster.

Each Kubernetes cluster will run the same operating system version and the Kubernetes major release.

Your network will be such that:

• The primary and standby database networks communicate with each other to facilitate the creation of a Data Guard database.
• The primary and standby file system networks communicate with each other to facilitate the replication of the file system data. If you have to run the Rsync process to achieve the replication inside the cluster, then the primary Kubernetes worker network will be able to communicate with the Kubernetes worker network on the standby site.
• A global load balancer will be used to direct the traffic between the primary and standby sites. This load balancer is often independent of the site-specific load balancers used for on-site communication.
• The SSL certificates used in the load balancers must be the same in each load balancer. The traffic should not be aware when the load balancer switches sites.

There are several ways to create a DR environment. This document makes the following assumptions:

• You will create an exact replica of your primary environment.
• The DR environment will reside in the same tenancy as your primary environment.
• The DR environment will reside in the same compartment as your primary environment.
• The DR environment will reside in a different region from your primary environment.

You will create the standby environment using the same steps as the primary environment, followed by the steps to create an OKE cluster, with the following characteristics:

• The VCN must use different CIDRs. For example,10.0.0.0/16 for the primary cluster and 10.1.0.0/16 for the standby cluster.
• Subnets will use different IP address ranges. For example, a primary subnet may be 10.0.4.0/24 and the equivalent standby subnet would be 10.1.4.0/24.
• The load balancer virtual host names will be the same.
• The load balancer SSL certificates will be the same.
• The ports used will be the same.
• The webhost names will be the same.
• Dedicated bastion nodes will exist.

After you create both the environments, complete the following additional steps:

• Creating a Dynamic Routing Gateway
  
• Creating a Dynamic Routing Gateway Attachment
  
• Creating a Remote Peering Connection
  
• Connecting the Site 1 and Site 2 VCNs
  
• Creating Routing Tables for Dynamic Routing Gateways
  
• Creating the Security Lists for Subnets
  
• Checking the Site Connectivity
  

Creating a Dynamic Routing Gateway

A dynamic routing gateway (DRG) is required to ensure that two different virtual cloud networks (VCNs) can communicate with each other. Each site requires a DRG. Therefore, perform the following steps on each site.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Networking and click Dynamic Routing Gateway.
3. Click Create Dynamic Routing Gateway and enter the following information:
   • Name: Provide a meaningful name for the gateway. For example, site1-drg.
   • Compartment: Select the compartment you created earlier. See Creating an OCI Compartment.
4. Click Create Dynamic Routing Gateway.

Creating a Dynamic Routing Gateway Attachment

After you create the DRG, you should attach it to the VCN of the site.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Networking and click Virtual Cloud Networks.
3. Select your Virtual Cloud Network. This is the same network that was created when you created the Kubernetes cluster. See Creating an OKE Cluster in OCI.
4. From the list of resources, select Dynamic Routing Gateways Attachments.
5. Click Create Virtual Cloud Network Attachment and enter the following information:
   • Name: Provide a name for the attachment. For example, DRG-Attachment-Site1.
   • DRG: Select the Dynamic Routing Gateway you created earlier. For example: site1-drg. See Creating a Dynamic Routing Gateway.

   Leave the remaining options at the default values.

6. Click Create Virtual Cloud Network Attachment.

Creating a Remote Peering Connection

To create a Remote Peering Connection (RPC):

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Networking and click Dynamic Routing Gateway.
3. Select the Dynamic Routing Gateway you created earlier. For example, site1-DRG. See Creating a Dynamic Routing Gateway.
4. From the list of resources, select Remote Peering Connections Attachments.
5. Click Create Remote Peering Connection and enter the following information:
   • Name: Provide a meaningful name for the connection. For example, site1-RPC.
   • Compartment: Select the compartment you created earlier. See Creating an OCI Compartment.
6. Click Create Remote Peering Connection.

Connecting the Site 1 and Site 2 VCNs

Before you perform this step, ensure that you have created the Dynamic Routing Gateway (RPG) (see Creating a Dynamic Routing Gateway and the Remote Peering Connection (RPC) (see Creating a Remote Peering Connection) on both the sites. The following steps will help link the two sites.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Networking and click Dynamic Routing Gateway.
3. Select the Dynamic Routing Gateway you created earlier. For example, site1-DRG. See Creating a Dynamic Routing Gateway.
4. From the list of resources, select Remote Peering Connections Attachments.
5. From the Remote Peering Connections Attachments section, select the Remote Peering Connection you created earlier. For example, site1-RPC. See Creating a Remote Peering Connection. Make a note of the Remote Peering Connection OCID displayed at the top of the screen.

Perform the following steps on Site 2:

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Networking and click Dynamic Routing Gateway.
3. Select the Dynamic Routing Gateway you created earlier. For example, site1-DRG. See Creating a Dynamic Routing Gateway.
4. From the list of resources, select Remote Peering Connections Attachments.
5. From the Remote Peering Connections Attachments section, select the Remote Peering Connection you created For Site 2. For example, site2-RPC. For instructions, see Creating a Remote Peering Connection.
6. Click Establish Connection and enter the following information:
   • Region: Select the region that hosts Site 1.
   • Remote Peering Connection OCID: Enter the RPC OCID from Site 1, obtained above.
7. Click Establish Connection.

   The Peering Status changes to Peered. This change may take a few minutes.

Creating Routing Tables for Dynamic Routing Gateways

For subnets in Site 1 to communicate with subnets in Site 2, you should create routing entries in both directions.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Networking and click Virtual Cloud Networks.
3. Select your Virtual Cloud Network. This is the same network that was created when you created the Kubernetes cluster. See Creating an OKE Cluster in OCI.
4. From the list of resources, select Route Table.
5. Select the route table you want to update. For example, db-route-table.
6. From the list of resources, select DRG Route Tables.
7. Click Add Route Rules and enter the following information:
   • Target Type: Dynamic Routing Gateway.
   • Destination Type: CIDR Block
   • Destination CIDR Block: 10.1.11.0/24
   • Description: Traffic to DRG
8. Click Add Route Rules.

   Repeat for each rule in the tables below:

   Table 10-15 Routing Rules for Site 1

   Route Name	Destination CIDR	Next Hop Attachment	Attachment Name
   db-route	10.1.11.0/24	Virtual Cloud Network	DRG-Attachment-Site1
   oke-node-subnet	10.1.10.0/24	Virtual Cloud Network	DRG-Attachment-Site1

   Table 10-16 Routing Rules for Site 2

   Route Name	Destination CIDR	Next Hop Attachment	Attachment Name
   db-route	10.1.11.0/24	Virtual Cloud Network	DRG-Attachment-Site2
   oke-node-subnet	10.1.10.0/24	Virtual Cloud Network	DRG-Attachment-Site2

Creating the Security Lists for Subnets

After creating routes between the subnets, you need to create security lists for each network and add them to the corresponding subnet.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Networking and click Virtual Cloud Networks.
3. From the list of resources, select Security Lists.
4. Select a security list. For example, db-seclist.
5. From the list of resources, select Ingress Rules.
6. Click Add Ingress Rules.
7. Enter the information, as described in Table 10-15 and Table 10-16.
8. Click Add Ingress Rule.
9. From the list of resources, select Egress Rules.
10. Click Add Ingress Rules.
11. Enter the information, as described in Table 10-15 and Table 10-16.
12. Click Add Ingress Rule.

Table 10-17 Security List for Site 1

List	Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Source Port Range	Destination Port Range	Type
db-seclist	Ingress	CIDR	10.1.11.0/24		TCP		1521
db-seclist	Ingress	CIDR	10.1.11.0/24		TCP		6200
Security List for Private Subnet for VCN	Ingress	CIDR	10.1.11.0/24				31444
Pv-seclist	Ingress	CIDR	10.1.11.0/24		TCP		111
Pv-seclist	Ingress	CIDR	10.1.11.0/24		TCP		2048-2050
Pv-seclist	Ingress	CIDR	10.1.11.0/24		UDP		111
Pv-seclist	Ingress	CIDR	10.1.11.0/24		UDP		2048
Pv-seclist	Egress	CIDR	10.1.11.0/24		TCP	111
Pv-seclist	Egress	CIDR	10.1.11.0/24		TCP	2048-2050
Pv-seclist	Egress	CIDR	10.1.11.0/24		UDP	111
Pv-seclist	Egress	CIDR	10.1.11.0/24		UDP	2048

Table 10-18 Security List for Site 2

List	Rule Type	Type	Source CIDR	Destination CIDR	Protocol	Source Port Range	Destination Port Range	Type
db-seclist	Ingress	CIDR	10.0.11.0/24		TCP		1521
db-seclist	Ingress	CIDR	10.0.11.0/24		TCP		6200
Security List for Private Subnet for VCN	Ingress	CIDR	10.0.11.0/24				31444
Pv-seclist	Ingress	CIDR	10.0.11.0/24		TCP		111
Pv-seclist	Ingress	CIDR	10.0.11.0/24		TCP		2048-2050
Pv-seclist	Ingress	CIDR	10.0.11.0/24		UDP		111
Pv-seclist	Ingress	CIDR	10.0.11.0/24		UDP		2048
Pv-seclist	Egress	CIDR	10.0.11.0/24		TCP	111
Pv-seclist	Egress	CIDR	10.0.11.0/24		TCP	2048-2050
Pv-seclist	Egress	CIDR	10.0.11.0/24		UDP	111
Pv-seclist	Egress	CIDR	10.0.11.0/24		UDP	2048

Checking the Site Connectivity

After configuring the Dynamic Gateway and Security Lists, use the Network Path Analyzer to validate inter-region connectivity.

1. Log in to the Oracle Cloud Infrastructure Console for your tenancy.
2. Select Networking and click Network Path Analyzer.
3. Click Create Path Analysis.
4. On the Configure Analysis screen, enter the following information:
   • Name: Enter a name for the test. For example, DB Check.
   • Source IP Address: Enter the IP address of one of the database hosts in Site 1.
   • Destination Address: Enter the IP address of one the database hosts in Site 2.
   • Destination Port: Enter the database port 1521.
5. Click Run Analysis.
6. Ensure that the test is successful before you continue.
7. Repeat the test for each destination port in tables Table 10-17 and Table 10-18.