Configure TLS for Replication

You can configure TLS for replication to ensure secure network communication between your replicated TimesTen databases. See Transport Layer Security for TimesTen Replication in the Oracle TimesTen In-Memory Database Security Guide for detailed information.

These sections describe how to configure and use TLS for replication:

Create Metadata Files and Kubernetes Facilities

The /ttconfig/replicationWallet metadata file is required for TLS support for replication. (The /ttconfig directory is located in the containers of your TimesTen databases.) This file must contain the cwallet.sso file (the Oracle wallet) that was generated when you created the TLS certificates. Recall that this file was located in the /scratch/ttuser/instance_dir/instance1/conf/serverWallet directory. See Create TLS Certificates for Replication and Client/Server for information on creating these certificates. This wallet contains the credentials that are used by TimesTen replication for configuring TLS encryption between your active standby pair of TimesTen databases.

In addition to the /ttconfig/replicationWallet metadata file, you may use the other supported metadata files. See About Configuration Metadata Details for information on these supported metadata files.

You can include these metadata files in one or more Kubernetes facilities (for example, in a Kubernetes Secret, in a ConfigMap, or in an init container). This ensures the metadata files are populated in the /ttconfig directory of the TimesTen containers. Note that there is no requirement as to how to get the metadata files into this /ttconfig directory. See Populate the /ttconfig Directory for more information.

The example in the following sections illustrates how to include the replicationWallet metadata file in a Kubernetes Secret. It also creates the db.ini, the adminUser, and the schema.sql metadata files and includes these metadata files in a ConfigMap:

Create a Kubernetes Secret

This section creates the repl-tls Kubernetes Secret. The repl-tls Secret will contain the replicationWallet metadata file.

On your Linux development host:

From the directory of your choice, create an empty subdirectory. This example creates the serverWallet subdirectory. (The serverWallet directory is used in the remainder of this example to denote this directory.)
```
% mkdir -p serverWallet
```
Copy the /scratch/ttuser/instance_dir/instance1/conf/serverWallet/cwallet.sso file into the serverWallet directory that you just created. Recall that this file was generated when you used the ttCreateCerts utility to create the TLS certificates. See "Create TLS Certificates for Replication and Client/Server" for information.
```
% cp /scratch/ttuser/instance_dir/instance1/conf/serverWallet/cwallet.sso 
serverWallet/cwallet.sso
```
Create the Kubernetes Secret.
In this example:
- The name of the Secret is repl-tls. Replace repl-tls with a name of your choosing. (repl-tls is represented in bold.)
- The name of the metadata file required for TLS replication is replicationWallet (represented in bold).
- The location of the wallet directory is serverWallet (in this example, represented in bold). If you use a different directory, replace serverWallet with the name of your directory.
- The name of the Oracle wallet is cwallet.sso (represented in bold).
Use the kubectl create command to create the Secret:
```
% kubectl create secret generic repl-tls 
--from-file=replicationWallet=serverWallet/cwallet.sso
secret/repl-tls created
```

You have successfully created and deployed the repl-tls Kubernetes Secret. The replicationWallet/cwallet.sso file will later be available in the /ttconfig directory of the TimesTen containers. In addition, the file will be available in the /tt/home/timesten/replicationWallet directory of the TimesTen containers.

Create a ConfigMap

This section creates the repl-tls ConfigMap. This ConfigMap contains the db.ini, the adminUser, and the schema.sql metadata files.

These metadata files are not required for TLS, but are included as additional attributes for your TimesTen databases. See "Overview of Configuration Metadata and Kubernetes Facilities" for information on the metadata files and the ConfigMap facility.

On your Linux development host:

From the directory of your choice, create an empty subdirectory for the metadata files. This example creates the cm_replTLS subdirectory. (The cm_replTLS directory is used in the remainder of this example to denote this directory.)
```
% mkdir -p cm_replTLS
```
Navigate to the ConfigMap directory.
```
% cd cm_replTLS
```
Create the db.ini file in this ConfigMap directory (cm_replTLS, in this example). In this db.ini file, define the PermSize and DatabaseCharacterSet connection attributes.
```
vi db.ini

PermSize=200
DatabaseCharacterSet=AL32UTF8
```
Create the adminUser file in this ConfigMap directory (cm_replTLS, in this example). In this adminUser file, create the sampleuser user with the samplepw password.
```
vi adminUser

sampleuser/samplepw
```
Create the schema.sql file in this ConfigMap directory (cm_replTLS, in this example). In this schema.sql file, define the s sequence and the emp table for the sampleuser user. The Operator will automatically initialize your database with these object definitions.
```
vi schema.sql

create sequence sampleuser.s;
create table sampleuser.emp (
  id number not null primary key,
  name char(32)
);
```
Create the ConfigMap. The files in the cm_replTLS directory are included in the ConfigMap and, later, will be available in the TimesTen containers.
In this example:
- The name of the ConfigMap is repl-tls. Replace repl-tls with a name of your choosing. (repl-tls is represented in bold in this example.)
- This example uses cm_replTLS as the directory where the files that will be copied into the ConfigMap reside. If you use a different directory, replace cm_replTLS with the name of your directory. (cm_replTLS is represented in bold in this example.)
Use the kubectl create command to create the ConfigMap:
```
% kubectl create configmap repl-tls --from-file=cm_replTLS
configmap/repl-tls created
```

Use the kubectl describe command to verify the contents of the ConfigMap. (repl-tls, in this example.)

% kubectl describe configmap repl-tls
Name:         repl-tls
Namespace:    mynamespace
Labels:       <none>
Annotations:  <none>
 
Data
====
adminUser:
----
sampleuser/samplepw
 
db.ini:
----
PermSize=200
DatabaseCharacterSet=AL32UTF8
 
schema.sql:
----
create sequence sampleuser.s;
create table sampleuser.emp (id number not null primary key, name char (32));
 
Events:  <none>

You have successfully created and deployed the repl-tls ConfigMap.

Create a TimesTenClassic Object

This section creates the TimesTenClassic object. See "Define and Create a TimesTenClassic Object" and "About the TimesTenClassic Object Type" for detailed information on the TimesTenClassic object.

Perform these steps:

Create an empty YAML file. You can choose any name, but you may want to use the same name you used for the name of the TimesTenClassic object. (In this example, repltls.) The YAML file contains the definitions for the TimesTenClassic object. See "TimesTenClassicSpecSpec" for information on the fields that you must specify in this YAML file as well as the fields that are optional.
In this example, the fields of particular interest for TLS replication are:
- dbSecret: This example uses one Kubernetes Secret (called repl-tls) for the replicationWallet metadata file.
- replicationCipherSuite: This field is required for TLS for replication. In this example, the value is SSL_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256. See Task 3: Configure TLS for Replication in the Oracle TimesTen In-Memory Database Security Guide and see the replicationCipherSuite entry in "Table 17-3" in this book for more information.
- replicationSSLMandatory: This field is optional. In this example, set replicationSSLMandatory equal to 1. See Task 3: Configure TLS for Replication in the Oracle TimesTen In-Memory Database Security Guide and see the replicationSSLMandatory entry in Table 17-3 in this book for more information.
In addition, this example includes:
- name: Replace repltls with the name of your TimesTenClassic object.
- storageClassName: Replace oci-bv with the name of the storage class used to allocate PersistentVolumes to hold TimesTen.
- storageSize: Replace 250Gi with the amount of storage that should be requested for each Pod to hold TimesTen. Note: This example assumes a production environment and uses a value of 250Gi for storageSize. For demonstration purposes, a value of 50Gi is adequate. See the storageSize and the logStorageSize entries in "Table 17-3" for information.
- image: Replace container-registry.oracle.com/timesten/timesten:22.1.1.19.0 with the location and the name of image.
- imagePullSecret: Replace sekret with the image pull secret that Kubernetes should use to fetch the TimesTen image.
- dbConfigMap: This example uses one ConfigMap (called repl-tls) for the db.ini, the adminUser, and the schema.sql metadata files.
```
% vi repltls.yaml

apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
  name: repltls
spec:
  ttspec:
    storageClassName: oci-bv
    storageSize: 250Gi
    image: container-registry.oracle.com/timesten/timesten:22.1.1.19.0
    imagePullSecret: sekret
    dbConfigMap:
    - repl-tls
    dbSecret:
    - repl-tls
    replicationCipherSuite: SSL_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    replicationSSLMandatory: 1
```
Use the kubectl create command to create the TimesTenClassic object from the contents of the YAML file (in this example, repltls.yaml). Doing so begins the process of deploying your active standby pair of TimesTen databases in the Kubernetes cluster.
```
% kubectl create -f repltls.yaml
timestenclassic.timesten.oracle.com/repltls created
```

You have successfully created the TimesTenClassic object in the Kubernetes cluster. The process of deploying your TimesTen databases begins, but is not yet complete.

Monitor Deployment of a TimesTenClassic Object

Use the kubectl get and the kubectl describe commands to monitor the progress of the active standby pair as it is provisioned.

Use the kubectl get command and review the STATE field. Observe the value is Initializing. The active standby pair provisioning has begun, but is not yet complete.
```
% kubectl get ttc repltls
NAME      STATE          ACTIVE   AGE
repltls   Initializing   None     50s
```
Use the kubectl get command again to see if value of the STATE field has changed. In this example, the value is Normal, indicating the active standby pair of databases are now provisioned and the process is complete.
```
% kubectl get ttc repltls
NAME      STATE    ACTIVE      AGE
repltls   Normal   repltls-0   3m45s
```

Use the kubectl describe command to view the active standby pair provisioning in detail.

Note the following have been correctly set in the repltls TimesTenClassic object definition:

The repl-tls Secret has been correctly referenced in the dbSecret field (represented in bold).
The repl-tls Configmap has been correctly referenced in the dbConfigMap field (represented in bold).
The replicationCipherSuite field has been correctly set to SSL_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (represented in bold).
The replicationSSLMandatory field has been correctly set to 1 (represented in bold).

Note: Not all of the output is shown in this example.

% kubectl describe ttc repltls
Name:         repltls
Namespace:    mynamespace
Labels:       <none>
Annotations:  <none>
API Version:  timesten.oracle.com/v1
Kind:         TimesTenClassic
Metadata:
  Creation Timestamp:  2023-04-30T18:51:43Z
  Generation:          1
  Resource Version:    75029797
  Self Link: 
/apis/timesten.oracle.com/v1/namespaces/mynamespace/timestenclassics/repltls
  UID:                 a2915ef3-0fe0-11eb-8b9a-aaa0151611fe
Spec:
  Ttspec:
    Db Config Map:
      repl-tls
    Db Secret:
      repl-tls
    Image:                      container-registry.oracle.com/timesten/timesten:22.1.1.19.0
    Image Pull Policy:          Always
    Image Pull Secret:          sekret
    Replication Cipher Suite:   SSL_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    Replication SSL Mandatory:  1
    Storage Class Name:         oci-bv
    Storage Size:               250Gi
...
Events:
  Type  Reason       Age    From       Message
  ----  ------       ----   ----       -------
  -     Create       4m17s  ttclassic  Secret tta2915ef3-0fe0-11eb-8b9a-aaa0151611fe created
  -     Create       4m17s  ttclassic  Service repltls created
  -     Create       4m17s  ttclassic  StatefulSet repltls created
  -     StateChange  3m10s  ttclassic  Pod repltls-1 Agent Up
  -     StateChange  3m10s  ttclassic  Pod repltls-1 Release 22.1.1.19.0
  -     StateChange  3m10s  ttclassic  Pod repltls-1 Daemon Up
  -     StateChange  2m3s   ttclassic  Pod repltls-0 Agent Up
  -     StateChange  2m3s   ttclassic  Pod repltls-0 Release 22.1.1.19.0
  -     StateChange  2m1s   ttclassic  Pod repltls-0 Daemon Up
  -     StateChange  68s    ttclassic  Pod repltls-0 Database Loaded
  -     StateChange  68s    ttclassic  Pod repltls-0 Database Updatable
  -     StateChange  68s    ttclassic  Pod repltls-0 CacheAgent Not Running
  -     StateChange  68s    ttclassic  Pod repltls-0 RepAgent Not Running
  -     StateChange  67s    ttclassic  Pod repltls-0 RepState IDLE
  -     StateChange  67s    ttclassic  Pod repltls-0 RepScheme None
  -     StateChange  66s    ttclassic  Pod repltls-0 RepAgent Running
  -     StateChange  66s    ttclassic  Pod repltls-0 RepScheme Exists
  -     StateChange  66s    ttclassic  Pod repltls-0 RepState ACTIVE
  -     StateChange  47s    ttclassic  Pod repltls-1 Database Loaded
  -     StateChange  47s    ttclassic  Pod repltls-1 Database Not Updatable
  -     StateChange  47s    ttclassic  Pod repltls-1 CacheAgent Not Running
  -     StateChange  47s    ttclassic  Pod repltls-1 RepAgent Not Running
  -     StateChange  47s    ttclassic  Pod repltls-1 RepScheme Exists
  -     StateChange  47s    ttclassic  Pod repltls-1 RepState IDLE
  -     StateChange  41s    ttclassic  Pod repltls-1 RepAgent Running
  -     StateChange  36s    ttclassic  Pod repltls-1 RepState STANDBY
  -     StateChange  36s    ttclassic  TimesTenClassic was Initializing, now Normal

Your active standby pair of TimesTen databases are successfully deployed (as indicated by Normal.) You are now ready to verify that TLS is being used for replication.

Verify TLS Is Being Used for Replication

To verify TLS is being used for replication, perform the following steps:

Review the active (repltls-0, in this example) pod and the standby pod (repltls-1, in this example).

% kubectl get pods
NAME                                       READY   STATUS    RESTARTS   AGE
repltls-0                                  2/2     Running   0          6m35s
repltls-1                                  2/2     Running   0          6m34s
timesten-operator-f84766548-tch7s          1/1     Running   0          28d

Optional: Use the kubectl exec -it command to invoke the shell in the active Pod (repltls-0, in this example).
```
% kubectl exec -it repltls-0 -c tt -- /bin/bash
```
Optional: From the shell in the active pod, verify the cwallet.sso file is located in the /tt/home/timesten/replicationWallet directory.
```
% ls /tt/home/timesten/replicationWallet
cwallet.sso
 
```
Optional: From the shell in the active pod, verify that the TLS replication-specific values are correct in the timesten.conf configuration file. (This file is located in the /tt/home/timesten/instances/instance1/conf directory.)
In particular, note that:
- replication_wallet is correctly set to /tt/home/timesten/replicationWallet (represented in bold).
- replication_cipher_suite is correctly set to SSL_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (represented in bold).
- replication_ssl_mandatory is correctly set to 1 (represented in bold).
See Task 3: Configure TLS for Replication in the Oracle TimesTen In-Memory Database Security Guide for more information on these timesten.conf attributes.
```
% cat /tt/home/timesten/instances/instance1/conf/timesten.conf
admin_uid=3429
admin_user=timesten
daemon_port=6624
group_name=timesten
hostname=repltls-0
instance_guid=48AC5964-56A1-4C66-AB89-5646A2431EA3
instance_name=instance1
replication_cipher_suite=SSL_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
replication_ssl_mandatory=1
replication_wallet=/tt/home/timesten/replicationWallet
server_port=6625
show_date=1
timesten_release=22.1.1
tns_admin=/ttconfig
verbose=1
```

From the shell in the active pod, run the ttRepAdmin utility with the -showstatus -detail options to verify the replication agent transmitters and receivers are using TLS (as indicated by SSL, represented in bold). See ttRepAdmin in the Oracle TimesTen In-Memory Database Reference for information on this utility.

Note: Not all output is shown in this example.

% ttRepAdmin -showstatus -detail repltls
 
Replication Agent Status as of: 2023-04-30 19:01:55
 
DSN                         : repltls
...
TRANSMITTER thread(s) (TRANSMITTER(M):139870727366400):
 For                     : REPLTLS (track 0) (SSL)
   Start/Restart count   : 1
   Current state         : STATE_META_PEER_INFO
 
RECEIVER thread(s) (RECEIVER:139870719887104):
 For                     : REPLTLS (track 0) (SSL)
   Start/Restart count   : 1
   Current state         : STATE_RCVR_READ_NETWORK_LOOP
...

You have successfully verified that TLS for replication is being used.