Installing Oracle Cloud Native Environment

Installation Overview

The high level overview of setting up Oracle Cloud Native Environment is described in this section.

To install Oracle Cloud Native Environment:

Prepare the operator node: An operator node is a host to be used to perform and manage the deployment of environments. The operator node must be set up with the Platform API Server, and the Platform CLI (olcnectl).
Prepare the Kubernetes nodes: The Kubernetes control plane and worker nodes must to be set up with the Platform Agent.
Set up a load balancer: If you're deploying a highly available Kubernetes cluster, set up a load balancer. You can set up an external load balancer, or use the container-based load balancer deployed by the Platform CLI.
Set up X.509 Certificates: X.509 Certificates are used to provide secure communication between the Kubernetes nodes. You must set up the certificates before you create an environment and perform a deployment.
Start the services: Start the Platform API Server and Platform Agent services on nodes using the X.509 Certificates.
Create an environment: Create an environment into which you can install the Kubernetes module and any other optional modules.
Deploy modules: Deploy the Kubernetes module and any other optional modules.

Setting up the Nodes

This section discusses setting up nodes to use in an Oracle Cloud Native Environment. The nodes are used to form a Kubernetes cluster.

An operator node is used to perform the deployment of the Kubernetes cluster using the Platform CLI and the Platform API Server. An operator node might be a node in the Kubernetes cluster, or a separate host. In examples in this book, the operator node is a separate host, and not part of the Kubernetes cluster.

On each Kubernetes node (both control plane and worker nodes) the Platform Agent must be installed. Before you set up the Kubernetes nodes, you must prepare them. For information on preparing the nodes, see Prerequisites.

During the installation of the required packages, an olcne user is created. This user is used to start the Platform API Server or Platform Agent services and has the minimum OS privileges to perform that task. Don't use the olcne user for any other purpose.

Setting up the Operator Node

This section discusses setting up the operator node. The operator node is a host that's used to perform and manage the deployment of environments, including deploying the Kubernetes cluster.

To set up the operator node:

On the operator node, install the Platform CLI, Platform API Server, and utilities.
```
sudo dnf install olcnectl olcne-api-server olcne-utils
```
Enable the olcne-api-server service, but do not start it. The olcne-api-server service is started when you configure the X.509 Certificates.
```
sudo systemctl enable olcne-api-server.service 
```
For information on configuration options for the Platform API Server, see Configuring the Platform API Server.

Setting up Kubernetes Nodes

This section discusses setting up the nodes to use in a Kubernetes cluster. Perform these steps on both Kubernetes control plane and worker nodes.

To set up the Kubernetes nodes:

On each node to be added to the Kubernetes cluster, install the Platform Agent package and utilities.
```
sudo dnf install olcne-agent olcne-utils
```
Enable the olcne-agent service, but do not start it. The olcne-agent service is started when you configure the X.509 Certificates.
```
sudo systemctl enable olcne-agent.service 
```
For information on configuration options for the Platform Agent, see Configuring the Platform Agent.
If the docker service is running, stop, and disable it.
```
sudo systemctl disable --now docker.service
```
If the containerd service is running, stop, and disable it.
```
sudo systemctl disable --now containerd.service
```

Configuring a Proxy Server

If you use a proxy server, configure it with CRI-O on each Kubernetes node.

On each Kubernetes node:

Create a CRI-O systemd configuration directory:

sudo mkdir /etc/systemd/system/crio.service.d

Create a file named proxy.conf in the directory, and add the proxy server information. For example:

[Service]
Environment="HTTP_PROXY=http://proxy.example.com:3128"
Environment="HTTPS_PROXY=https://proxy.example.com:3128"
Environment="NO_PROXY=mydomain.example.com"

If you're also installing Calico (as a module or as the Kubernetes Container Network Interface), or the Multus module, add the Kubernetes service IP (the default is 10.96.0.1) to the NO_PROXY variable:
```
Environment="NO_PROXY=mydomain.example.com,10.96.0.1"
```

Setting up a Load Balancer for Highly Available Clusters

A highly available (HA) cluster needs a load balancer to provide high availability of control plane nodes. A load balancer communicates with the Kubernetes API Server on the control plane nodes.

The methods of setting up a load balancer to create an HA cluster are:

Using an external load balancer instance.
Using a load balancer provided by a cloud infrastructure, for example an Oracle Cloud Infrastructure load balancer.
Using the internal load balancer that can be deployed by the Platform CLI on the control plane nodes.

Setting up an External Load Balancer

To use an external load balancer implementation, it must be set up and ready to use before you perform an HA cluster deployment. The load balancer hostname and port is entered as an option when you create the Kubernetes module. The load balancer must be set up with the following configuration:

The listener listening on TCP port 6443.
The distribution set to round robin.
The target set to TCP port 6443 on the control plane nodes.
The health check set to TCP.

For more information on setting up an external load balancer, see Oracle Linux 9: Setting Up Load Balancing or Oracle Linux 8: Setting Up Load Balancing.

Setting up a Load Balancer on Oracle Cloud Infrastructure

To set up a load balancer on Oracle Cloud Infrastructure:

Sign-in to Oracle Cloud Infrastructure.
Create a load balancer.
Add a backend set to the load balancer using weighted round robin. Set the health check to be TCP port 6443.
Add the control plane nodes to the backend set. Set the port for the control plane nodes to port 6443.
Create a listener for the backend set using TCP port 6443.

For more information on setting up a load balancer in Oracle Cloud Infrastructure, see the Oracle Cloud Infrastructure documentation.

Setting up the Internal Load Balancer

Important:

Using the internal load balancer is not recommended for production deployments. Instead, use a correctly configured load-balancer that's outside the Kubernetes cluster, for example an own external load balancer, or a load balancer provided by a cloud infrastructure, such as an Oracle Cloud Infrastructure load balancer.

To use the internal load balancer deployed by the Platform CLI, you need to perform the following steps to prepare the control plane nodes.

To prepare control plane nodes for the load balancer deployed by the Platform CLI:

Set up the control plane nodes as described in Setting up Kubernetes Nodes.
Use the --virtual-ip option when creating the Kubernetes module to nominate a virtual IP address that can be used for the primary control plane node. This IP address must not be in use on any node and is assigned dynamically to the control plane node assigned as the primary controller by the load balancer. If the primary node fails, the load balancer reassigns the virtual IP address to another control plane node, and that, in turn, becomes the primary node.

Tip:

If you're deploying to Oracle Cloud Infrastructure virtual instances, you can assign a secondary private IP address to the VNIC on a control plane node to create a virtual IP address. Ensure you list this control plane node first when creating the Kubernetes module. For more information on secondary private IP addresses, see the Oracle Cloud Infrastructure documentation.
On each control plane node, open port 6444. When you use a virtual IP address, the Kubernetes API server port is changed from the default of 6443 to 6444. The load balancer listens on port 6443 and receives the requests and passes them to the Kubernetes API server.
```
sudo firewall-cmd --add-port=6444/tcp
sudo firewall-cmd --add-port=6444/tcp --permanent
```
On each control plane node, enable the Virtual Router Redundancy Protocol (VRRP) protocol:
```
sudo firewall-cmd --add-protocol=vrrp
sudo firewall-cmd --add-protocol=vrrp --permanent
```

Setting up Certificates for Kubernetes Nodes

Communication between the Kubernetes nodes is secured using X.509 certificates.

Before you deploy Kubernetes, you need to configure the X.509 certificates used to manage the communication between the nodes. You can use:

Vault: The certificates are managed using the HashiCorp Vault secrets manager. Certificates are created during the deployment of the Kubernetes module. You need to create a token authentication method for Oracle Cloud Native Environment.
CA Certificates: Using certificates signed by a trusted Certificate Authority (CA), and copied to each Kubernetes node before the deployment of the Kubernetes module. These certificates are unmanaged and must be renewed and updated manually.
Private CA Certificates: Using generated certificates, signed by a private CA you set up, and copied to each Kubernetes node before the deployment of the Kubernetes module. These certificates are unmanaged and must be renewed and updated manually. A script is provided to help you set this up.

A software-based secrets manager is recommended to manage these certificates. The HashiCorp Vault secrets manager can be used to generate, assign, and manage the certificates. We recommend you implement an instance of Vault, setting up the appropriate security for the environment.

For more information on installing and setting up Vault, see the Hashicorp documentation.

If you don't want to use Vault, you can use certificates, signed by a trusted CA, and copied to each node. A script is provided to generate a private CA to generate certificates for each node. This script also gives you the commands needed to copy the certificates to the nodes.

Setting up Vault Authentication

To configure Vault for use with Oracle Cloud Native Environment, set up a Vault token with the following properties:

A PKI secret engine with a CA certificate or intermediate, at olcne_pki_intermediary.
A role under that PKI, named olcne, configured to not require a common name, and allow any name.
A token authentication method and policy that attaches to the olcne role and can request certificates.

For information on setting up the Vault PKI secrets engine to generate dynamic X.509 certificates, see:

https://developer.hashicorp.com/vault/docs/secrets/pki

For information on creating Vault tokens, see:

https://developer.hashicorp.com/vault/docs/commands/token/create

Setting up CA Certificates

This section shows you how to use certificates signed by a trusted CA, without using a secrets manager such as Vault. To use certificates, copy them to all Kubernetes nodes, and to the Platform API Server node.

To ensure the Platform Agent on each Kubernetes node, and the Platform API Server have access to certificates, copy them into the /etc/olcne/certificates/ directory on each node. This is the default location of the certificates and is used when setting up the Platform Agent and Platform API Server, and when creating an environment. You can use another location for certificates, but this means you need to specify the location for each certificate and key when you're setting up the services, and the environment.

Tip:

You can use the olcnectl certificates copy command to copy the certificates to the Kubernetes nodes.

The default location for certificates and keys, and the location used in this book, is the /etc/olcne/certificates/ directory on each node.

CA Certificate: /etc/olcne/certificates/ca.cert
Node Key: /etc/olcne/certificates/node.key
Node Certificate: /etc/olcne/certificates/node.cert

Setting up Private CA Certificates

This section shows you how to create a private CA, and use that to generate signed certificates for the nodes. This section also contains information on copying the certificates to the nodes. This section also contains information on generating certificates for nodes that you want to scale into a Kubernetes cluster.

Distribute Node Certificates

Use the Platform CLI to generate and distribute signed certificates to the Kubernetes nodes. You can use a CA Certificate or create one automatically. These certificates are used when starting the Platform API Server and Platform agent on nodes to secure communication between the Kubernetes nodes.

Before you create the certificates, ensure the user on the operator node that's creating the certificates is a member of the olcne group. Use the syntax:

sudo usermod -a -G olcne username

For example, to add the olcne group to the oracle user, on the operator node run:

sudo usermod -a -G olcne oracle

Important:

When you add a group to a user, you must log out of the terminal session, and back in again. This is required to apply the change.

On the operator node, use the olcnectl certificates distribute command to generate and distribute private CA and certificates to the Kubernetes nodes. The syntax to use is:

olcnectl certificates distribute
[--byo-ca-cert certificate-path]
[--byo-ca-key key-path]
[--cert-dir certificate-directory]
[--cert-request-common-name common_name]
[--cert-request-country country]
[--cert-request-locality locality]
[--cert-request-organization organization]
[--cert-request-organization-unit organization-unit]
[--cert-request-state state]
[{-h|--help}]
[{-n|--nodes} nodes]
[--one-cert]
[{-R|--remote-command} remote-command]
[{-i|--ssh-identity-file} file_location]
[{-l|--ssh-login-name} username]
[--timeout minutes]

Provide the nodes for which you want to create certificates using the --nodes option. Create a certificate for each node that runs the Platform API Server or Platform Agent. This means you must create certificates for the operator node, and for each Kubernetes node.

Note:

If you're deploying a highly available Kubernetes cluster using a virtual IP address, you don't need to create a certificate for a virtual IP address.

For example:

olcnectl certificates distribute \
--nodes operator.example.com,control1.example.com,worker1.example.com,worker2.example.com

To set up the information for the private CA, use the --cert-request-* options. The following example also includes options to set the SSH login information for nodes using the --ssh-* options.

olcnectl certificates distribute \
--nodes operator.example.com,control1.example.com,worker1.example.com,worker2.example.com \
--cert-request-common-name cloud.example.com \
--cert-request-country US \
--cert-request-locality "My Town" \
--cert-request-organization "My Company" \
--cert-request-organization-unit "My Company Unit" \
--cert-request-state "My State" \
--ssh-identity-file ~/.ssh/id_rsa \
--ssh-login-name oracle

If you have a CA Certificate and private key to generate certificates, you can provide it using the --byo-ca-cert and --byo-ca-key options, for example:

olcnectl certificates distribute \
--nodes operator.example.com,control1.example.com,worker1.example.com,worker2.example.com \
--byo-ca-cert $HOME/certificates/ca/ca.cert \
--byo-ca-key $HOME/certificates/ca/ca.key

The certificate files for each node are generated and saved in the $HOME/certificates/ directory, in a directory for each hostname. If you used the --cert-dir option, they're saved to the directory you specified.

The CA Certificate and private key are generated to the $HOME/certificates/ca/ directory. If you used the --cert-dir option, they're saved to the directory you specified. If you used a CA Certificate and key using the --byo-ca-* options, they're not created, or saved.

The certificates for each node are copied to the /etc/olcne/certificates/ directory on each Kubernetes node using SSH. This is the default directory for the location of the certificates when you start the Platform API Server and Platform Agent.

Distribute Extra Node Certificates

Use the Platform CLI to generate and distribute signed certificates to the nodes using an existing CA.

This is useful to generate and distribute certificates for any extra nodes that you want to add to a Kubernetes cluster.

On the operator node, use the olcnectl certificates distribute command to generate and distribute private CA and certificates to the Kubernetes nodes. The syntax to use is:

olcnectl certificates distribute
[--byo-ca-cert certificate-path]
[--byo-ca-key key-path]
[--cert-dir certificate-directory]
[--cert-request-common-name common_name]
[--cert-request-country country]
[--cert-request-locality locality]
[--cert-request-organization organization]
[--cert-request-organization-unit organization-unit]
[--cert-request-state state]
[{-h|--help}]
[{-n|--nodes} nodes]
[--one-cert]
[{-R|--remote-command} remote-command]
[{-i|--ssh-identity-file} file_location]
[{-l|--ssh-login-name} username]
[--timeout minutes]

Provide the nodes for which you want to create certificates using the --nodes option.

Provide the location of the existing CA Certificate using the --byo-ca-cert option.

You can use the same CA certificate and private key you used to generate the Kubernetes node certificates by using the --byo-ca-cert and --byo-ca-key options.

For example:

olcnectl certificates distribute \
--nodes worker3.example.com,worker4.example.com \
--byo-ca-cert $HOME/certificates/ca/ca.cert \
--byo-ca-key $HOME/certificates/ca/ca.key

The location of the CA Certificate and private key might be different if you used the --cert-dir option of the olcnectl certificates distribute command when creating the original certificates.

The certificate files for each node are generated and saved in the $HOME/certificates/ directory, in a directory for each hostname.

The certificates for each node are copied to the /etc/olcne/certificates/ directory on each Kubernetes node using SSH. This is the default directory for the location of the certificates when you start the Platform API Server and Platform Agent.

Setting up Certificates for the Platform CLI to the Platform API Server

Communication between the Platform CLI and Platform API Server is secured using X.509 certificates.

We recommend you configure the X.509 certificates used to manage the communication between the Platform CLI and the Platform API Server. You can use:

Vault: The certificates are managed using the HashiCorp Vault secrets manager.
CA Certificates: Using a CA Certificate, signed by a trusted Certificate Authority.
Private CA Certificates: Using generated certificates, signed by a private CA you set up. These certificates are unmanaged and must be renewed and updated manually.

The certificates need to be on the node that contains the Platform CLI (this is most likely the operator node). The name of the directory reflects the name of the host that contains Platform API Server (again, this is most likely the operator node). The format of the directory for the certificates must be:

$HOME/.olcne/certificates/api_server_hostname:port

The api_server_hostname is the hostname of the node where the Platform API Server is installed, and the port is the port to access the Platform API Server (the default is 8091). This is the default directory for the Platform API Server keys, and where the Platform CLI checks for access to the Platform API Server without any extra configuration. For example:

$HOME/.olcne/certificates/operator.example.com:8091

Generate Certificates for the Platform CLI to the Platform API Server

Use the Platform CLI to generate signed certificates for the Platform CLI to access the Platform API Server.

On the operator node, use the olcnectl certificates generate command to generate certificates for the Platform CLI to access the Platform API Server. The syntax to use is:

olcnectl certificates generate
[--byo-ca-cert certificate-path]
[--byo-ca-key key-path]
[--cert-dir certificate-directory]
[--cert-request-common-name common_name]
[--cert-request-country country]
[--cert-request-locality locality]
[--cert-request-organization organization]
[--cert-request-organization-unit organization-unit]
[--cert-request-state state]
[{-h|--help}]
{-n|--nodes} nodes
[--one-cert]

The --cert-dir option sets the location where the certificates are to be saved. We recommend to use the the following format for the directory:

$HOME/.olcne/certificates/api_server_hostname:port

The api_server_hostname is the hostname of the node where the Platform API Server is installed, and the port is the port to access the Platform API Server (the default is 8091). This is the default directory for the Platform API Server keys, and where the Platform CLI checks for access the Platform API Server without any extra configuration.

Important:

This is the path to specify the location of the certificates for the Platform CLI to access the Platform API Server when you create an environment using the olcnectl environment create command.

The --nodes option must be set to the hostname and IP address of the Platform API Server, as shown:

--nodes api_server_hostname,api_server_ip_address

Use the --one-cert option to save the certificates for the hostname and for the IP address to a single file.

You can use the same CA certificate and private key you used to generate the Kubernetes node certificates by using the --byo-ca-cert and --byo-ca-key options.

For example:

olcnectl certificates generate \
--nodes operator.example.com,127.0.0.1 \
--cert-dir $HOME/.olcne/certificates/operator.example.com:8091/  \
--byo-ca-cert $HOME/certificates/ca/ca.cert \
--byo-ca-key $HOME/certificates/ca/ca.key \
--one-cert

In this example, the certificate files (node.cert, node.csr, and node.key) are created and saved in the directory:

$HOME/.olcne/certificates/operator.example.com:8091/

Copy the CA Certificate you used to generate the keys to this directory. For example:

cp $HOME/certificates/ca/ca.cert $HOME/.olcne/certificates/operator.example.com:8091/

The Platform CLI is set up to connect securely to the Platform API Server.

Setting up Certificates for the externalIPs Kubernetes Service

When you deploy Kubernetes, a service is deployed to the cluster that controls access to externalIPs in Kubernetes services. The service is named externalip-validation-webhook-service and runs in the externalip-validation-system namespace. This Kubernetes service requires X.509 certificates be set up before deploying Kubernetes. You can use Vault to generate the certificates, or use existing certificates for this purpose. You can also generate certificates using the Platform CLI. The certificates must be available on the operator node.

The examples in this book use the /etc/olcne/certificates/restrict_external_ip/ directory for these certificates.

Setting up Vault Certificates

You can use Vault to generate a certificates for the externalIPs Kubernetes service. The Vault instance must be configured in the same way as described in Setting up Vault Authentication.

You need to generate certificates for two nodes, named:

externalip-validation-webhook-service.externalip-validation-system.svc

externalip-validation-webhook-service.externalip-validation-system.svc.cluster.local

The certificate information must be generated in PEM format.

For example:

vault write olcne_pki_intermediary/issue/olcne \
    alt_names=externalip-validation-webhook-service.externalip-validation-system.svc,\
externalip-validation-webhook-service.externalip-validation-system.svc.cluster.local \
    format=pem_bundle

The output is displayed. Look for the section that starts with certificate. This section contains the certificates for the node names (set with the alt_names option). Save the output in this section to a file named node.cert. The file looks similar to:

-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAymg8uHy+mpwlelCyC4WrnfLwUmJ5vZmSos85QnIlZvyycUPK
...
X3c8LNaJDfQx1wKfTc/c0czBhHYxgwfau0G6wjqScZesPi2xY0xyslE=
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIID2TCCAsGgAwIBAgIUZ/M/D7bAjhyGx7DivsjBb9oeLhAwDQYJKoZIhvcNAQEL
...
9bRwnen+JrxUn4GV59GtsTiqzY6R2OKPm+zLl8E=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIDnDCCAoSgAwIBAgIUMapl4aWnBXE/02qTW0zOZ9aQVGgwDQYJKoZIhvcNAQEL
...
kV8w2xVXXAehp7cg0BakVA==
-----END CERTIFICATE-----

Look for the section that starts with issuing_ca. This section contains the CA certificate. Save the output in this section to a file named ca.cert. The file looks similar to:

-----BEGIN CERTIFICATE-----
MIIDnDCCAoSgAwIBAgIUMapl4aWnBXE/02qTW0zOZ9aQVGgwDQYJKoZIhvcNAQEL
...
kV8w2xVXXAehp7cg0BakVA==
-----END CERTIFICATE-----

Look for the section that starts with private_key. This section contains the private key for the node certificates. Save the output in this section to a file named node.key. The file looks similar to:

-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAymg8uHy+mpwlelCyC4WrnfLwUmJ5vZmSos85QnIlZvyycUPK
...
X3c8LNaJDfQx1wKfTc/c0czBhHYxgwfau0G6wjqScZesPi2xY0xyslE=
-----END RSA PRIVATE KEY-----

Copy the three files (node.cert, ca.cert and node.key) to the operator node and set the ownership of the files as described in Setting up CA Certificates.

Setting up CA Certificates

If you're using existing certificates, copy them to a directory under /etc/olcne/certificates/ on the operator node. For example:

CA Certificate: /etc/olcne/certificates/restrict_external_ip/ca.cert
Node Key: /etc/olcne/certificates/restrict_external_ip/node.key
Node Certificate: /etc/olcne/certificates/restrict_external_ip/node.cert

Copy these certificates to a different location on the operator node than the certificates and keys used for the Kubernetes nodes as set up in Setting up Certificates for Kubernetes Nodes. This makes sure you don't overwrite those certificates and keys. You need to generate certificates for two nodes, named:

externalip-validation-webhook-service.externalip-validation-system.svc

externalip-validation-webhook-service.externalip-validation-system.svc.cluster.local

Save the certificates for these two nodes as a single file named node.cert.

Ensure the permissions of the directory where the certificates are saved can be read by the user on the operator node that you intend to use to run the olcnectl commands to install Kubernetes.

sudo chown -R username:username /etc/olcne/certificates/restrict_external_ip/

Setting up Private CA Certificates

You can use the Platform CLI to generate the certificates.

Generate Certificates for ExternalIPs Service

Use the Platform CLI to generate signed certificates for the Kubernetes ExternalIPs service.

On the operator node, use the olcnectl certificates generate command to generate certificates for the Kubernetes ExternalIPs service. The syntax to use is:

olcnectl certificates generate
[--byo-ca-cert certificate-path]
[--byo-ca-key key-path]
[--cert-dir certificate-directory]
[--cert-request-common-name common_name]
[--cert-request-country country]
[--cert-request-locality locality]
[--cert-request-organization organization]
[--cert-request-organization-unit organization-unit]
[--cert-request-state state]
[{-h|--help}]
{-n|--nodes} nodes
[--one-cert]

The --cert-dir option sets the location where the certificates are to be saved. We recommend you use the directory:

$HOME/certificates/restrict_external_ip/

The directory you use must have read and write permissions by the user you intend to use use to run the olcnectl commands to install Kubernetes.

Important:

This is the path to specify the location of the ExternalIPs Kubernetes service certificates when you create the Kubernetes module using the olcnectl module create command.

The --nodes option must be set to the name of the Kubernetes service, as shown:

--nodes externalip-validation-webhook-service.externalip-validation-system.svc,externalip-validation-webhook-service.externalip-validation-system.svc.cluster.local

Use the --one-cert option to save the certificates for the two service names to a single file.

You can use the same CA certificate and private key you used to generate the Kubernetes node certificates by using the --byo-ca-cert and --byo-ca-key options.

For example:

olcnectl certificates generate \
--nodes externalip-validation-webhook-service.externalip-validation-system.svc,\
externalip-validation-webhook-service.externalip-validation-system.svc.cluster.local \
--cert-dir $HOME/certificates/restrict_external_ip/ \
--byo-ca-cert $HOME/certificates/ca/ca.cert \
--byo-ca-key $HOME/certificates/ca/ca.key \
--one-cert

In this example, the certificates are created and saved in the directory:

$HOME/certificates/restrict_external_ip/

Copy the CA certificate you used to generate the certificate into to this directory. If you used all the defaults and suggested paths, the following command copies the file:

cp $HOME/certificates/ca/ca.cert $HOME/certificates/restrict_external_ip/

The certificates are set up for the Kubernetes ExternalIPs service.

Starting the Platform API Server and Platform Agent Services

This section discusses using certificates to set up secure communication between the Platform API Server and the Platform Agent on nodes in the cluster. You can set up secure communication using certificates managed by Vault, or using certificates copied to each node. You must configure the Platform API Server and the Platform Agent to use the certificates when you start the services.

For information on setting up the certificates with Vault, see Setting up Certificates for Kubernetes Nodes.

For information on creating a private CA to sign certificates that can be used during testing, see Setting up Private CA Certificates.

Starting the Services Using Vault

This section shows you how to set up the Platform API Server and Platform Agent services to use certificates managed by Vault.

To set up and start the services using Vault:

On the operator node, use the /etc/olcne/bootstrap-olcne.sh script to configure the Platform API Server to retrieve and use a Vault certificate. Use the bootstrap-olcne.sh --help command for a list of options for this script. For example:
```
sudo /etc/olcne/bootstrap-olcne.sh \
--secret-manager-type vault \
--vault-token s.3QKNuRoTqLbjXaGBOmO6Psjh \
--vault-address https://192.0.2.20:8200 \
--force-download-certs \
--olcne-component api-server
```
The certificates are generated and downloaded from Vault.

By default, the certificates are saved to the /etc/olcne/certificates/ directory. You can optionally specify a path for the certificates, for example, by including the following options in the bootstrap-olcne.sh command:
```
--olcne-ca-path /path/ca.cert \
--olcne-node-cert-path /path/node.cert \
--olcne-node-key-path /path/node.key \
```
The Platform API Server is configured to use the certificates, and started. You can confirm the service is running using:
```
systemctl status olcne-api-server.service 
```
On each Kubernetes node, use the /etc/olcne/bootstrap-olcne.sh script to configure the Platform Agent to retrieve and use a certificate. For example:
```
sudo /etc/olcne/bootstrap-olcne.sh \
--secret-manager-type vault \
--vault-token s.3QKNuRoTqLbjXaGBOmO6Psjh \
--vault-address https://192.0.2.20:8200 \
--force-download-certs \
--olcne-component agent
```
The certificates are generated and downloaded from Vault.

By default, the certificates are saved to the /etc/olcne/certificates/ directory. You can optionally specify a path for the certificates, for example, by including the following options in the bootstrap-olcne.sh command:
```
--olcne-ca-path /path/ca.cert \
--olcne-node-cert-path /path/node.cert \
--olcne-node-key-path /path/node.key \
```
The Platform Agent is configured to use the certificates, and started. You can confirm the service is running using:
```
systemctl status olcne-agent.service 
```

Starting the Services Using Certificates

This section shows you how to set up the Platform API Server and Platform Agent services to use certificates which have been copied to each node. This example assumes the certificates are available on all nodes in the /etc/olcne/certificates/ directory.

To set up and start the services using certificates:

On the operator node, use the /etc/olcne/bootstrap-olcne.sh script to configure the Platform API Server to use the certificates. Use the bootstrap-olcne.sh --help command for a list of options for this script. For example:
```
sudo /etc/olcne/bootstrap-olcne.sh \
--secret-manager-type file \
--olcne-component api-server
```
If the certificates for Kubernetes nodes are in a directory other than /etc/olcne/certificates/ (the default), include the location of the certificates. The --olcne-node-cert-path, --olcne-ca-path, and --olcne-node-key-path options set the location of the certificate files. For example:
```
--olcne-node-cert-path /etc/olcne/configs/certificates/production/node.cert \
--olcne-ca-path /etc/olcne/configs/certificates/production/ca.cert \
--olcne-node-key-path /etc/olcne/configs/certificates/production/node.key \
```
The Platform API Server is configured to use the certificates, and started. You can confirm the service is running using:
```
systemctl status olcne-api-server.service 
```
On each Kubernetes node, use the /etc/olcne/bootstrap-olcne.sh script to configure the Platform Agent to use the certificates. For example:
```
sudo /etc/olcne/bootstrap-olcne.sh \
--secret-manager-type file \
--olcne-component agent
```
If the certificates for Kubernetes nodes are in a directory other than /etc/olcne/certificates/ (the default), include the location of the certificates. The --olcne-node-cert-path, --olcne-ca-path, and --olcne-node-key-path options set the location of the certificate files. For example:
```
--olcne-node-cert-path /etc/olcne/configs/certificates/production/node.cert \
--olcne-ca-path /etc/olcne/configs/certificates/production/ca.cert \
--olcne-node-key-path /etc/olcne/configs/certificates/production/node.key \
```
The Platform Agent is configured to use the certificates, and started. You can confirm the service is running using:
```
systemctl status olcne-agent.service 
```