Pre-deployment Configuration

The commands in this procedure are executed from the Bootstrap Host. All terraform commands are executed from the /var/terraform directory.

Table 2-16 Pre-deployment Configuration

Step # Procedure Description
1.

Obtain the TLS Certificate for OpenStack

Depending on the Customer's environment it is very likely that the customer's OpenStack uses certificates for TLS access to the API. Without this certificate, OpenStack commands will not work. Customer's may have to obtain this certificate before using OpenStack client commands.

  1. Ask the OpenStack admin to provide the required TLS certificate to access the client commands (ex. in an Oracle OpenStack system installed with kolla, the certificate will be located at /etc/kolla/certiifcates/haproxy-ca.crt)
  2. Copy the certificate to the Bootstrap Host at location: /etc/pki/<OpenStack_release_name>/haproxy-ca.crt (ex. /etc/pki/kolla/haproxy-ca.crt) (If /etc/pki/<OpenStack_release_name> does not exist, it can be created using command: mkdir -p /etc/pki/<OpenStack_release_name>
  3. Set the environment variable OS_CACERT to the location where the certificate was copied to using the command: export OS_CACERT=/etc/pki/<OpenStack_release_name>/haproxy-ca.crt (ex. export OS_CACERT=/etc/pki/kolla/haproxy-ca.crt)
2.

Get the Openstack RC (API v3) File

This file exports a number of environment variables on the Bootstrap Host for the given user which directs the OpenStack Client commands towards the particular OpenStack Environment. It must be copied to the users home directory on the Bootstrap Host so that the OpenStack Client commands can be executed.

Note: These instructions may be somewhat different on OpenStack Desktops.

  1. From the OpenStack Desktop: go to Project → Compute → Access & Security.

  2. Select the API Access

  3. On the right hand side, select Download OpenStack RC File v3.

    This will download a openrc.sh file prefixed with the OpenStack project name (ex: 5G-openrc.sh). to your PC.

  4. SCP this file (ie. winSCP) to the Bootstrap Host in the /home/admusr directory as .<project_name>-openrc.sh

    Note: In order for SCP/winSCP to work properly, the key mentioned in the Prerequisites above must be used to access the Bootstrap Host. It may also be necessary to add the appropriate Security Group Rules to support SSH (Rule: SSH, Remote: CIDR CIDR: 0.0.0.0/0) under the Network → Security Groups page in the OpenStack Environment. Contact the OpenStack Administrator to get the proper rules added if necessary.

  5. Execute the following command: source .<project_name>-openrc.sh

  6. Execute the following command to verify the OpenStack Client is working: openstack image list
3.

Create SSH Key on Bootstrap Host Create the keys that will be used to access the other VMs. This command generates the private and public keys that are passed to the Bastion Host and used to communicate to other node from that Bastion Host. Do not supply a passphrase when it asks for one. Just hit enter. Also the private key should be copied to a place for safe keeping should the Bootstrap Host be destroyed.
$ ssh-keygen -m PEM -t rsa -b 2048
4.

Add Files to /tmp Directory

These files must be copied to the directories listed using scp or some other means (ie. winSCP).

  1. There are three directories on the Bootstrap Host. These three directories are as follows:

    1. /tmp/yum.repos.d

    2. /tmp/db

    3. /tmp/certificates

  2. Within these three directories the user must supply the following mandatory files:

    1. Add a customer specific central repo .repo file to the /tmp/yum.repos.d directory which allows for access to the customer specific repo (ex: winterfell-mirror.repo).

    2. Add a mysql .zip file. (ex: V980756-01.zip) to the /tmp/db directory. This file is used for installing the ndb MySQL cluster and is downloaded from OSDC.

    3. Add a docker registry certificate to the /tmp/certificates directory for the central docker registry. This file mus tbe copied using the following format:<central_repo_hostname>:<central_repo_port>.crt(ex: winterfell:5000.crt).
5.

Updating the ~.configure/openstack/clouds.yaml File
  1. To obtain the values for the authorization fields need in clouds.yaml below, execute the openstack configuration show command (make sure you have sourced the openrc script before executing this command).
    $ openstack configuration show
    +---------------------------------+-------------------------------------------+
    | Field                           | Value                                     |
    +---------------------------------+-------------------------------------------+
    | api_timeout                     | None                                      |
    | application_catalog_api_version | 1                                         |
    | auth.auth_url                   | http://thundercloud.us.oracle.com:5000/v3 |
    | auth.password                   | <redacted>                                |
    | auth.project_domain_id          | 6c3468f1207c4e00bb441746c2046a90          |
    | auth.project_id                 | 811ef89b5f154ab0847be2f7e41117c0          |
    | auth.project_name               | OCCNE                                     |
    | auth.user_domain_name           | LDAP                                      |
    | auth.username                   | John.Doe                                  |
    | auth_type                       | password                                  |
    | baremetal_api_version           | 1                                         |
    | beta_command                    | False                                     |
    | cacert                          | None                                      |
    | cert                            | None                                      |
    | compute_api_version             | 3                                         |
    | container_api_version           | 1                                         |
    | container_infra_api_version     | 1                                         |
    +---------------------------------+-------------------------------------------+
  2. To get the floating_network_id value for the load balancer configuration in clouds.yaml below, execute the following commands:
    $ openstack network list
    Get the network ID and the subnets id for floating ip address network name:
    Example from the openstack command cli list:
    +--------------------------------------+-------------------+--------------------------------------+
    | ID                                   | Name              | Subnets                              |
    +--------------------------------------+-------------------+--------------------------------------+
    | e4351e3e-81e3-4a83-bdc1-dde1296690e3 | ext-net           | c0e0c185-ed65-4a53-a7a3-418277fb9a20 |
  3. Edit the ~/.configure/openstack/clouds.yaml file (using sudo vi ~/.configure/openstack/clouds.yaml) and update following values for your Openstack Environment using the commands listed above (use double quotes where indicated in the example given below):
    clouds:
      mycloud:
        auth:
          auth_url: <openstack-Identity-api-url>
          username: <openstack-user-name> 
          project_name: <openstack-project-name>
          project_id: <openstack-Project ID>
          user_domain_name: <openstack-Domain Name>
          password: <openstack-user-password>
        region_name: <openstack-region-name-if-available>
        interface: <openstack-intrface- public/private>
        identity_api_version: <openstack-indentity-api-version>
        loadbalancer:
            lbaas_enabled: true
            subnet_id: <openstack_lbaas_subnet_id>
            floating_network_id: <openstack_lbaas_floating_network_id>
            use_octavia: true
            lb_method: ROUND_ROBIN
    Example:
    clouds:
      mycloud:
        auth:
          auth_url: http://thundercloud.us.oracle.com:5000/v3
          username: "john.doe"
          project_name: "OCCNE"
          project_id: 811ef89b5f154ab0847be2f7e41117c0
          user_domain_name: "LDAP"
          password: "johnspw"
        region_name: "RegionOne"
        interface: "public"
        identity_api_version: 3
        loadbalancer:
            lbaas_enabled: true
            subnet_id: c0e0c185-ed65-4a53-a7a3-418277fb9a20
            floating_network_id: e4351e3e-81e3-4a83-bdc1-dde1296690e3
            use_octavia: true
            lb_method: ROUND_ROBIN
6.

Updating cluster.tfvars File The fields in the cluster.tfvars file must be configured to adapt to the current customer Openstack Environment. The steps below detail how to collect and set the fields that must be changed.
  1. From the /var/terraform directory, copy directory occne_example and its contents (cluster.tfvars) using the command below to create a new directory. Change the name of the new directory to include your name to distinguish it from the occne_example directory.

    Note: The directory name is not used for any special purpose other than to distinguish it from the occne_example directory. It can be called anything.

    $ cp -R occne_example occne_<user>
  2. Use the following commands to retrieve the information necessary to configure the cluster.tfvars
    1. The different flavor settings should be set according to the recommendations from Reference 1 in this document. An Admin user of the customer specific OpenStack Environment must add the flavors and provide the UUID of those flavors for configuration into the cluster.tfvars file. The UUID of each specific flavor that is used must be added as the value field of the key/value fields in the cluster.tfvars file.
    2. Once flavors have been added to the OpenStack Environment, the UUID can be retrieved via the following OpenStack Client command from the Bootstrap Host shell:
      $ openstack flavor list | grep <flavor_name>
       
      Example:
       
      $ openstack flavor list | grep medium
      | 43c7b73b-42a7-4f40-b52e-11f6803fc750 | oc-cne.medium        |  16384 |    80 |         0 |     2 | True      |
       
      $ openstack flavor show 43c7b73b-42a7-4f40-b52e-11f6803fc750
      +----------------------------+--------------------------------------+
      | Field                      | Value                                |
      +----------------------------+--------------------------------------+
      | OS-FLV-DISABLED:disabled   | False                                |
      | OS-FLV-EXT-DATA:ephemeral  | 0                                    |
      | access_project_ids         | None                                 |
      | disk                       | 80                                   |
      | id                         | 43c7b73b-42a7-4f40-b52e-11f6803fc750 |
      | name                       | oc-cne.medium                        |
      | os-flavor-access:is_public | True                                 |
      | properties                 |                                      |
      | ram                        | 16384                                |
      | rxtx_factor                | 1.0                                  |
      | swap                       |                                      |
      | vcpus                      | 2                                    |
      +----------------------------+--------------------------------------+
    3. Use the following command to retrieve the floatingip_pool name and external_net UUID.
      $ openstack network list
      +--------------------------------------+-------------------+--------------------------------------+
      | ID                                   | Name              | Subnets                              |
      +--------------------------------------+-------------------+--------------------------------------+
      | 1d25d5ea-77ca-4f56-b364-f53b09292e7b | ext-net2          | f5c5ee71-8688-466d-a79f-4306e2bf3f6a |
      | 668bc488-5307-49ad-9332-24fb0767bb39 | test-network      | 9432b2d5-99c0-43ee-8f8c-4709f38b68d9 |
      | 903155c7-c3ff-4283-bc2b-f34e8b6e76b0 | occne-ebadger-tc1 | ecbadd3e-e239-4830-b8c1-5ff94fa64c3a |
      | 90c160aa-2ef7-47d3-a212-e1790d56c971 | ext-net-ipv6      | 4c0b844f-1557-4454-b561-88fa31f657f3 |
      | c4a7569b-5448-4add-8c4e-006bbdd984ef | cluster1          | 4cf62be3-05e9-4a5b-b2a9-6aceee3c860f |
      | e4351e3e-81e3-4a83-bdc1-dde1296690e3 | ext-net           | c0e0c185-ed65-4a53-a7a3-418277fb9a20 |
      | fc36d63f-b30b-4c7f-979f-9b52b614bbd7 | occne-mkingre     | 7631612f-5d22-49be-975c-6e0a9329339b |
      +--------------------------------------+-------------------+--------------------------------------+
  3. Navigate to occne_<user> directory and edit the contents of the cluster.tfvars file in the newly created directory:
    $ vi occne_<user>/cluster.tfvars
  4. The fields in the cluster.tfvars file must be configured to adapt to the current customer Openstack Provider. Initially the cluster_name and network_name should be set as the same value.
    # <Kubernetes cluster name here>
    cluster_name = "<cluster-name>"
     
    # networking
    network_name = "<cluster-name>"
  5. For setting the ntp_server value in the cluster.tfvars file, use the IP Address of your cloud URL. One way of obtaining this is using the ping command on your Bootstrap Host. (For example: ping thundercloud.us.oracle.com)
    $ ping thundercloud.us.oracle.com
    PING srv-10-75-171-2.us.oracle.com (10.75.171.2) 56(84) bytes of data.
    64 bytes from pc1011601.labs.nc.tekelec.com (10.75.171.2): icmp_seq=1 ttl=63 time=0.283 ms
7.

Updating the clouds.yaml file for Load Balancers

Note: This must be executed after updating the cluster.tfvars file

  1. Get the subnet_id value by running following terrform commands:
    $ terraform init
    $ terraform apply -auto-approve -var-file=occne_<user>/cluster.tfvars
  2. After successful run of terraform apply, run following OpenStack command to get the subnet_id.
     openstack network list
    +--------------------------------------+-------------------+--------------------------------------+
    | ID                                   | Name              | Subnets                              |
    +--------------------------------------+-------------------+--------------------------------------+
    | 1d25d5ea-77ca-4f56-b364-f53b09292e7b | ext-net2          | f5c5ee71-8688-466d-a79f-4306e2bf3f6a |
    | 668bc488-5307-49ad-9332-24fb0767bb39 | test-network      | 9432b2d5-99c0-43ee-8f8c-4709f38b68d9 |
    | 903155c7-c3ff-4283-bc2b-f34e8b6e76b0 | occne-ebadger-tc1 | ecbadd3e-e239-4830-b8c1-5ff94fa64c3a |
    | 90c160aa-2ef7-47d3-a212-e1790d56c971 | ext-net-ipv6      | 4c0b844f-1557-4454-b561-88fa31f657f3 |
    | c4a7569b-5448-4add-8c4e-006bbdd984ef | cluster1          | 4cf62be3-05e9-4a5b-b2a9-6aceee3c860f |
    | e4351e3e-81e3-4a83-bdc1-dde1296690e3 | ext-net           | c0e0c185-ed65-4a53-a7a3-418277fb9a20 |
    | fc36d63f-b30b-4c7f-979f-9b52b614bbd7 | occne-mkingre     | 7631612f-5d22-49be-975c-6e0a9329339b |
    +--------------------------------------+-------------------+--------------------------------------+
    
  3. Copy the value from subnets column for the name that is same as the cluster_name from cluster.tfvars file.
  4. Edit the ~/.configure/openstack/clouds.yaml file (using sudo vi ~/.configure/openstack/clouds.yaml) and update the subnet_id value keeping other values previously configured the same.
    loadbalancer:
         lbaas_enabled: true
         subnet_id: <cluster_name_subnet_id>
     
    Example ((using cluster name: test-network):
       loadbalancer:
            lbaas_enabled: true
            subnet_id: 9432b2d5-99c0-43ee-8f8c-4709f38b68d9