Using Configuration Metadata

3 Using Configuration Metadata

This chapter gives an overview of the configuration metadata that is supported in the TimesTen Operator. It also discusses the Kubernetes facilities that you can use to get the configuration metadata into your TimesTen containers. The chapter then discusses additional configuration options. There are examples throughout.

Topics:

Overview of the configuration metadata and the Kubernetes facilities

Configuration metadata lets you define the attributes of your TimesTen database and how that database is to interact with other applications and components. The TimesTen Operator supports several metadata files that contain the configuration metadata. Each metadata file has a specific name. You use a text editor to create the metadata file with the specific name and then add the appropriate metadata to it. For example, the TimesTen Operator supports the db.ini metadata file. You use your editor to create the db.ini file and in it you define attributes for you database.

There is configuration metadata for the following TimesTen Operator deployments:

Active standby pair of TimesTen databases in TimesTen Classic
Active standby pair of TimesTen databases with TimesTen Cache in TimesTen Classic
TimesTen Scaleout grid and the database within the grid

Kubernetes supports various facilities that places the metadata files into the /ttconfig directory of the TimesTen containers. See Populating the /ttconfig directory.

List of the configuration metadata

Table 3-1 lists the metadata files that are supported by the TimesTen Operator. The table provides a description for each of the metadata files and indicates if the metadata file is supported in TimesTen Classic, in TimesTen Scaleout, or in both.

Table 3-1 TimesTen Operator metadata files

Name	Description	TimesTen Classic support	TimesTen Scaleout support
adminUser	Defines an initial user in the database and assigns this user `ADMIN` privileges. Optional.	Yes	Yes
cachegroups.sql	Defines the cache groups in the database. This file is specific to TimesTen Cache. Required if using TimesTen Cache.	Yes	No
cacheUser	Defines the cache administration user in the database. This file is specific to TimesTen Cache. Required if using TimesTen Cache.	Yes	No
csWallet	Defines the credentials that are used for Transport Layer Security (TLS) encryption of client/server communications. Required if using TLS.	Yes	Yes
db.ini	Defines the connection attributes of the database. See List of attributes in the Oracle TimesTen In-Memory Database Reference. Required if using TimesTen Cache. Otherwise, optional.	Yes	Yes
epilog.sql	Performs operations after the replication scheme is created. Optional.	Yes	No
replicationWallet	Defines the credentials that are used for Transport Layer Security (TLS) encryption of replication traffic between the TimesTen databases. Required if using TLS.	Yes	No
schema.sql	Defines an initial schema for the database. Optional.	Yes	Yes
sqlnet.ora	Defines how client applications communicate with an Oracle database. This file is specific to TimesTen Cache. Optional.	Yes	No
tnsnames.ora	Defines the Oracle Database service that TimesTen Cache uses to connect to Oracle Database. Required if using TimesTen Cache.	Yes	No
*.connect	Defines one or more direct connectables for the database in TimesTen Scaleout. Optional.	No	Yes
*.csconnect	Defines one or more client/server connectables for the database in TimesTen Scaleout. Optional.	No	Yes

Configuration metadata details

Metadata files let you specify the attributes and the metadata for your TimesTen database. After you create these files, and you choose a facility to get these files in your TimesTen containers, TimesTen accesses them to determine the attributes and the metadata that is specific to your database.

adminUser

The adminUser file creates an initial user with ADMIN privileges in the TimesTen database. If you provide this file, this user is created after the database is created. This file must contain one line of the form:

user/password

cachegroups.sql

The cachegroups.sql file contains the create cache group definitions and the cache group operations for your database. You can specify the following cache group definitions and cache operations in this file:

(Required): CREATE CACHE GROUP statements to create TimesTen cache groups
(Optional): LOAD CACHE GROUP statements to load data from the Oracle database into your cache groups
(Optional): ttOptUpdateStats or ttOptEstimateStats TimesTen built-in procedures to update statistics on the cache tables

The cachegroups.sql file is required if you are using TimesTen Cache in your TimesTenClassic deployment. This requirement ensures cache groups are created before replication is configured. Note: The instance administrator uses the ttIsql utility to run the cachegroups.sql file.

See:

CREATE CACHE GROUP and LOAD CACHE GROUP in the Oracle TimesTen In-Memory Database SQL Reference
Cache group types in the Oracle TimesTen In-Memory Database Cache Guide
ttOptUpdateStats and ttOptUpdateStats in the Oracle TimesTen In-Memory Database Reference

Here is an example of a cachegroups.sql file. The file defines two cache groups and loads data into one cache group.

CREATE DYNAMIC ASYNCHRONOUS WRITETHROUGH CACHE GROUP writecache
FROM oratt.writetab (
  pk NUMBER NOT NULL PRIMARY KEY,
  attr VARCHAR2(40)
);
 
CREATE READONLY CACHE GROUP readcache
AUTOREFRESH
  INTERVAL 5 SECONDS
FROM oratt.readtab (
  keyval NUMBER NOT NULL PRIMARY KEY,
  str VARCHAR2(32)
);
 
LOAD CACHE GROUP readcache COMMIT EVERY 256 ROWS;

cacheUser

The cacheUser file lets you create the TimesTen cache manager user. This user must have the same name as the cache administration user in the Oracle database, and must already exist in the Oracle database. See Create the Oracle database users and default tablespace in the Oracle TimesTen In-Memory Database Cache Guide.

This file must contain one line of the form,

cacheUser/ttPassword/oraPassword

where cacheUser is the TimesTen cache manager user, ttPassword is the TimesTen password for the TimesTen cacheUser user, and oraPassword is the Oracle database password you specified when you created the cacheUser user in the Oracle database.

For example, assume you have created the cacheuser2 cache administration user in the Oracle Database with password oraclepwd. Assume you designate this cacheuser2 user as the TimesTen cache manager user with a TimesTen password of ttpwd. In this example, the cacheUser metadata file contains this one line:

cacheuser2/ttpwd/oraclepwd

The TimesTen Operator creates the cacheuser2 user with the ttpwd in the TimesTen database. This cacheuser2 user then serves as the cache manager user in your TimesTen database. You do not need to create this TimesTen user. The Operator does it for you.

See Create the TimesTen users in the Oracle TimesTen In-Memory Database Cache Guide.

The Operator grants privileges to the TimesTen cacheUser user (cacheuser2, in this example) that are appropriate for this user's role as the cache manager. These privileges are:

CREATE SESSION
CACHE MANAGER
CREATE ANY TABLE
LOAD ANY CACHE GROUP
REFRESH ANY CACHE GROUP
FLUSH ANY CACHE GROUP
DROP ANY CACHE GROUP
ALTER ANY CACHE GROUP
UNLOAD ANY CACHE GROUP
SELECT ANY TABLE
INSERT ANY TABLE
UPDATE ANY TABLE
DELETE ANY TABLE

csWallet

In a TimesTen Client/Server environment, data is transmitted between your client applications and your TimesTen database unencrypted by default. However, you can configure TLS for Client/Server to ensure secure network communication between TimesTen clients and servers. To encrypt Client/Server traffic, specify the /ttconfig/csWallet file. This file contains the Oracle wallet for the server, which contains the credentials that are used for configuring TLS encryption between your TimesTen database and your Client/Server applications. The file will be available in the containers of your TimesTen databases in the directory /tt/home/timesten/csWallet. You can reference this directory in your db.ini file (by specifying the wallet connection attribute). See Creating TLS certificates for replication and Client/Server and Creating TLS certificates for replication and Client/Server.

The client wallet must also be available to your client applications. See Creating TLS certificates for replication and Client/Server and Configuring TLS for Client/Server.

db.ini

The db.ini file contains the TimesTen DSN definition for your database.

In TimesTen Classic, the db.ini file contains the connection attributes for your database. This file is used to to generate the sys.odbc.ini file for the instances. You can specify data store attributes, first connection attributes, and general connection attributes in the db.ini file. The name of the DSN is the name of the TimesTenClassic object. For example, if your TimesTenClassic object is called sample, the name of your DSN is sample.

In TimesTenScaleout, the db.ini file contains the connection attributes for each element of your database in the grid. The database definition file (dbDef) and its contents are used to create a TimesTen Scaleout database definition. The name of the database definition is the name of the TimesTenScaleout object. For example, if the name of your TimesTenScaleout object is sample, the name of the database definition is sample. The TimesTen Scaleout database is created based on the database definition. For information about creating a database and creating a database definition in TimesTen Scaleout, see Creating a database in the Oracle TimesTen In-Memory Database Scaleout User's Guide.

If you are using TimesTen Cache in your TimesTenClassic deployment, you must specify the OracleNetServiceName and the DatabaseCharacterSet connection attributes in the db.ini file. The DatabaseCharacterSet value must match the value of the database character set in the Oracle Database.

Do not specify the the DataStore or the LogDir connection attributes in the db.ini file. The Operator sets these attributes, placing the database files in Kubernetes Persistent Volumes.

See "List of attributes" in the Oracle TimesTen In-Memory Database Reference for information on the TimesTen connection attributes.

Note:

If the /ttconfig/db.ini file is not present in a TimesTen container, TimesTen creates a default sys.odbc.ini file. For this default sys.odbc.ini, the connection attributes are: Permsize=200 DatabaseCharacterSet=AL32UTF8

This example shows a sample db.ini file that contains various connection attributes for TimesTen databases in TimesTen Classic or TimesTen Scaleout.

PermSize=500
LogFileSize=1024
LogBufMB=1024
DatabaseCharacterSet=AL32UTF8

Here is an example that shows a sample db.ini file that contains the OracleNetServiceName for TimesTen databases that use TimesTen Cache in TimesTen Classic.

PermSize=500
LogFileSize=1024
LogBufMB=1024
DatabaseCharacterSet=AL32UTF8 
OracleNetServiceName=OraCache

epilog.sql

In TimesTen Classic, the epilog.sql file includes operations that occur after the replication scheme has been created and the replication agent has been started. For example, if you want to create replicated bookmarks in XLA, you can include the ttXlaBookmarkCreate TimesTen built-in procedure in this file.

The Operator instructs the instance administrator to run the epilog.sql file using the ttIsql utility.

Here is an example of an epilog.sql file. The example calls the ttXlaBookmarkCreate TimesTen built-in procedure to create XLA bookmarks.

call ttXlaBookmarkCreate('mybookmark',0x01);

For information about replicated bookmarks, see the ttXlaBookmarkCreate TimesTen built-in procedure in the Oracle TimesTen In-Memory Database Reference.

replicationWallet

In TimesTen Classic, TimesTen replication transmits data between your TimesTen databases unencrypted by default. However, you can configure TLS for replication to ensure secure network communication between your replicated TimesTen databases. To do this, specify the /ttconfig/replicationWallet file. This file contains an Oracle wallet, which contains the credentials that are used by TimesTen replication for configuring TLS encryption between your active standby pair of TimesTen databases. See Creating TLS certificates for replication and Client/Server and Configuring TLS for replication.

If you specify this file, you must include the replicationCipherSuite field and optionally include the replicationSSLMandatory field in your TimesTenClassic object definition. See the replicationCipherSuite entry and the replicationSSLMandatory entry in TimesTenClassicSpecSpec and Configuring TLS for replication.

schema.sql

The TimesTen Operator can automatically initialize your database with schema objects, such as users, tables, and sequences. To have the Operator do this, create the schema.sql file.

The Operator directs the instance administrator to use the ttIsql utility to run the schema.sql file immediately after the database is created. This operation occurs before the Operator configures replication or cache in your TimesTen database.

In TimesTen Cache, one or more cache table users own the cache tables. If this cache table user is not the cache manager user, then you must specify the schema.sql file and in it you must include the schema user and assign the appropriate privileges to this schema user. For example, if the oratt schema user was created in the Oracle database, and this user is not the TimesTen cache manager user, you must create the TimesTen oratt user in this file. See Create the Oracle Database users.

Do not include cache definitions in this file. Instead, use the cachegroups.sql metadata file. See cachegroups.sql.

sqlnet.ora

The Oracle Database sqlnet.ora file defines the options for how client applications communicate with the Oracle Database. To use TimesTen Cache or to use tools like ttLoadFromOracle, define a sqlnet.ora file. This file describes how applications, including TimesTen, can connect to your Oracle database. Note: If you define a sqlnet.ora file, you must define a tnsnames.ora file. See tnsnames.ora.

This is an example of a sqlnet.ora file:

NAME.DIRECTORY_PATH= {TNSNAMES, EZCONNECT, HOSTNAME}
SQLNET.EXPIRE_TIME = 10
SSL_VERSION = 1.2

tnsnames.ora

The Oracle Database tnsnames.ora file defines Oracle Net Services to which applications connect. You need to use tnsnames.ora (and perhaps a sqlnet.ora file, described in sqlnet.ora) if you are using:

TimesTen Cache
SQL APIs, such as Pro*C, OCI, or ODPI-C
The ttLoadFromOracle feature

For information about the ttLoadFromOracle TimesTen built-in procedure, see ttLoadFromOracle in the Oracle TimesTen In-Memory Database Reference.

Here is an example of a tnsnames.ora file:

OraTest =   
 (DESCRIPTION = 
   (ADDRESS = (PROTOCOL = TCP)(HOST = database.mynamespace.svc.cluster.local)
     (PORT = 1521))     
   (CONNECT_DATA =       
     (SERVER = DEDICATED)       
     (SERVICE_NAME = OraTest.my.sample.com)))
OraCache =  
 (DESCRIPTION =   
   (ADDRESS = (PROTOCOL = TCP)(HOST = database.mynamespace.svc.cluster.local)
     (PORT = 1521))     
   (CONNECT_DATA =       
     (SERVER = DEDICATED)       
     (SERVICE_NAME = OraCache.my.sample.com)))

*.connect

Files with names that end with the .connect extension define one or more direct connectables for direct mode access to a database in TimesTen Scaleout. You can create as many direct connectables as you like. A direct connectable specifies a set of general connection attribute settings for the database. You can choose any name for the direct connectable as long as it is a valid DSN name. The TimesTen Operator creates one direct connectable for each direct connectable file you create. The .connect extension denotes a direct connectable. Ensure the .connect extension is in lowercase.

The following example creates one direct connectable. The name of the file is sample.connect. The TimesTen Operator creates the sample direct connectable based on the contents of the sample.connect file.

ConnectionCharacterSet=AL32UTF8

For more information about direct connectables, see Connectable operations in the Oracle TimesTen In-Memory Database Reference and Connecting to a database in the Oracle TimesTen In-Memory Database Scaleout User's Guide.

*.csconnect

Files with names that end with the .csconnect extension define one or more client/server connectables for client/server access to a database in TimesTen Scaleout. You can create as many client/server connectables as you like. A client/server connectable specifies a set of general connection attribute settings for the database. You can choose any name for the client/server connectable as long as it is a valid DSN name. The TimesTen Operator creates one client/server connectable for each client/server connectable file you create. The .csconnect extension denotes a client/server connectable. Ensure the .csconnect extension is in lowercase.

The following example creates one client/server connectable. The name of the file is samplecs.csconnect. The TimesTen Operator creates the samplecs client/server connectable based on the contents of the samplecs.csconnect file.

TTC_Timeout=30

For more information about client/server connectables, see Connectable operations in the Oracle TimesTen In-Memory Database Reference and Connecting to a database in the Oracle TimesTen In-Memory Database Scaleout User's Guide.

Populating the /ttconfig directory

You can use different methods to ensure metadata files are placed in the /ttconfig of the TimesTen containers. There is no requirement as to which method to use. Kubernetes provides such facilities as ConfigMaps, Secrets, and init containers for you to consider.

Using ConfigMaps and Secrets

You can use one or more ConfigMaps and one or more Secrets to incorporate metadata files into the TimesTen containers. This lets you specify different TimesTen metadata for different deployments. In addition, you can use Secrets for metadata that contains sensitive data, like passwords and certificates.

The use of a ConfigMap to populate the metadata into Pods is a standard Kubernetes technique. One benefit is that you can modify the ConfigMap after it is created, which results in the immediate update of the files that are in the Pod.

Note:

TimesTen may not immediately notice and act on the changed content of the files.

When you use ConfigMaps and Secrets to hold your metadata and then reference them in the TimesTenClassic object definition, the TimesTen Operator creates a Projected Volume called tt-config. This tt-config volume contains the contents of all the ConfigMaps and all the Secrets specified in the dbConfigMap and the dbSecret fields of your TimesTenClassic or your TimesTenScaleout object. This volume is mounted as /ttconfig in the TimesTen containers.

Note:

You can specify one or more ConfigMaps and/or Secrets in your TimesTenClassic or TimesTenScaleout object using the dbConfigMap and dbSecret datum. The result is that these ConfigMaps and/or Secrets are mounted read-only at /ttconfig. Since such a mount is read-only, you cannot write into it from an init container. Alternatively, you can use an emptydir volume and use an init container to write files into it. However, you cannot combine ConfigMaps and Secrets with an init container. For information about using an init container, see Using an init container.

To use ConfigMaps and Secrets, follow this process:

Decide what facilities will contain what metadata files. For example, you can use one ConfigMap for all the metadata files. Or, for example, you can use one ConfigMap for the db.ini metadata file and one Secret for the adminUser and the schema.sql metadata files. There is no specific requirement.
Create the directory (or directories) that will contain the metadata files.
Use the kubectl create command to create the ConfigMap and the Secrets in the Kubernetes cluster.
Include the ConfigMaps and Secrets in your TimesTenClassic or your TimesTenScaleout object definition.

The following examples illustrate how to use ConfigMaps and Secrets for a TimesTenClassic or a TimesTenScaleout object.

Example using one ConfigMap

This example uses one ConfigMap (called sample) for the db.ini, the adminUser, and the schema.sql metadata files.

You can use this ConfigMap for a TimesTenClassic or a TimesTenScaleout object.

On your development host, from the directory of your choice, create an empty subdirectory for the metadata files. This example creates the cm_sample subdirectory. (The cm_sample directory is used in the remainder of this example to denote this directory.)
```
mkdir -p cm_sample
```
Change to the ConfigMap directory.
```
cd cm_sample
```
Create the db.ini file. In this db.ini file, define the PermSize and DatabaseCharacterSet connection attributes.
```
vi db.ini

PermSize=200
DatabaseCharacterSet=AL32UTF8
```
Create the adminUser file. In this adminUser file, create the sampleuser user with the samplepw password.
```
vi adminUser

sampleuser/samplepw
```
Create the schema.sql file. In this schema.sql file, define the s sequence and the emp table for the sampleuser user. The Operator automatically initializes your database with these object definitions.
```
vi schema.sql

create sequence sampleuser.s;
create table sampleuser.emp (
  id number not null primary key,
  name char(32)
);
```
Create the ConfigMap. The files in the cm_sample directory are included in the ConfigMap. These files are later available in the TimesTen containers.
In this example:
- The name of the ConfigMap is sample. Replace sample with a name of your choosing.
- This example uses cm_sample as the directory where the files that will be copied into the ConfigMap reside. If you use a different directory, replace cm_sample with the name of your directory.
```
kubectl create configmap sample --from-file=cm_sample
```
The output is the following:
```
configmap/sample created
```

Verify the contents of the ConfigMap.

kubectl describe configmap sample

The output is the following:

Name:         sample
Namespace:    mynamespace
Labels:       <none>
Annotations:  <none>
 
Data
====
adminUser:
----
sampleuser/samplepw
 
db.ini:
----
PermSize=200
DatabaseCharacterSet=AL32UTF8
 
schema.sql:
----
create sequence sampleuser.s;
create table sampleuser.emp (
  id number not null primary key,
  name char(32)
);

Events:  <none>

Include the ConfigMap in the object definition. In the dbConfigMap field, specify the name of the your ConfigMap (sample, in this example).

Note this example uses a storageSize of 250Gi (suitable for a production environment). For demonstration purposes, a storageSize of 50Gi is adequate.

This is an example of using the ConfigMap for a TimesTenClassic object.

apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
  name: sample
spec:
  ttspec:
    storageClassName: oci
    storageSize: 250Gi
    image: container-registry.oracle.com/timesten/timesten:22.1.1.9.0
    imagePullSecret: sekret
    dbConfigMap:
    - sample

This is an example of using the ConfigMap for a TimesTenScaleout object.

apiVersion: timesten.oracle.com/v1
kind: TimesTenScaleout
metadata:
  name: sample
spec:
  ttspec:
    storageClassName: oci
    storageSize: 250Gi
    image: container-registry.oracle.com/timesten/timesten:22.1.1.9.0
    imagePullSecret: sekret
    k: 2
    nReplicaSets: 3
    nZookeeper: 3
    dbConfigMap:
    - sample

The sample ConfigMap holds the metadata files. The tt-config volume contains the contents of the sample ConfigMap.

Example using one ConfigMap and one Secret

This example uses one ConfigMap (called myconfig) for the db.ini metadata file and one Secret (called mysecret) for the adminUser and the schema.sql metadata files.

You can use this ConfigMap and Secret for a TimesTenClassic or a TimesTenScaleout object.

On your development host, from the directory of your choice:
- Create one empty subdirectory for the ConfigMap. This example creates the cm_myconfig subdirectory. (The cm_myconfig directory is used in the remainder of this example to denote this directory.) This directory will contain the db.ini metadata file.
- Create a second empty subdirectory for the Secret. This example creates the secret_mysecret subdirectory. (The secret_mysecret directory is used in the remainder of this example to denote this directory.) This directory will contain the adminUser and the schema.sql metadata files.
```
mkdir -p cm_myconfig
mkdir -p secret_mysecret
```
Change to the ConfigMap directory.
```
cd cm_myconfig
```
Create the db.ini file in this ConfigMap directory (cm_myconf, in this example). In this db.ini file, define the PermSize and DatabaseCharacterSet connection attributes.
```
vi db.ini

PermSize=200
DatabaseCharacterSet=AL32UTF8
```
Change to the Secret directory.
```
cd secret_mysecret
```
Create the adminUser file in this Secret directory (secret_mysecret in this example). In this adminUser file, create the sampleuser user with the samplepw password.
```
vi adminUser

sampleuser/samplepw
```
Create the schema.sql file in this Secret directory (secret_mysecret in this example). In this schema.sql file, define the s sequence and the emp table for the sampleuser user. The Operator automatically initializes your database with these object definitions.
```
vi schema.sql

create sequence sampleuser.s;
create table sampleuser.emp (
  id number not null primary key,
  name char(32)
);
```
Create the ConfigMap. The files in the cm_myconfig directory are included in the ConfigMap and, later, will be available in the TimesTen containers.
In this example:
- The name of the ConfigMap is myconfig. Replace myconfig with a name of your choosing.
- This example uses cm_myconfig as the directory where the files that will be copied into the ConfigMap reside. If you use a different directory, replace cm_myconfig with the name of your directory.
Create the ConfigMap.
```
kubectl create configmap myconfig --from-file=cm_myconfig
```
The output is the following:
```
configmap/myconfig created
```

Verify the contents of the ConfigMap.

kubectl describe configmap myconf

The output is the following:

Name:         myconfig
Namespace:    mynamespace
Labels:       <none>
Annotations:  <none>
 
Data
====
db.ini:
----
PermSize=200
DatabaseCharacterSet=AL32UTF8
 
Events:  <none>

Create the Secret. The files in the secret_mysecret directory are included in the Secret and, later, will be available in the TimesTen containers.
In this example:
- The name of the Secret is mysecret. Replace mysecret with a name of your choosing.
- This example uses secret_mysecret as the directory where the files that will be copied into the Secret reside. If you use a different directory, replace secret_mysecret with the name of your directory.
```
kubectl create secret generic mysecret --from-file=secret_mysecret
```
The output is the following:
```
secret/mysecret created
```

Verify the Secret. Note the contents of the adminUser and the schema.sql files are not displayed.

kubectl describe secret mysecret

The output is the following:


Name:         mysecret
Namespace:    mynamespace
Labels:       <none>
Annotations:  <none>
 
Type:  Opaque
 
Data
====
adminUser:   12 bytes
schema.sql:  98 bytes

Include the ConfigMap and the Secret in the object definition.

In the dbConfigMap field, specify the name of the your ConfigMap.
In the dbSecret field, specify the name of the your Secret.

This is an example of using the ConfigMap and the Secret for a TimesTenClassic object.

apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
  name: sample
spec:
  ttspec:
    storageClassName: oci
    storageSize: 250Gi
    image: container-registry.oracle.com/timesten/timesten:22.1.1.9.0
    imagePullSecret: sekret
    dbConfigMap:
    - myconfig
    dbSecret:
    - mysecret

This is an example of using the ConfigMap and the Secret for a TimesTenScaleout object.

apiVersion: timesten.oracle.com/v1
kind: TimesTenScaleout
metadata:
  name: sample
spec:
  ttspec:
    storageClassName: oci
    storageSize: 250Gi
    image: container-registry.oracle.com/timesten/timesten:22.1.1.9.0
    imagePullSecret: sekret
    k: 2
    nReplicaSets: 3
    nZookeeper: 3
    dbConfigMap:
    - myconfig
    dbSecret:
    - mysecret

The myconfig ConfigMap and the mysecret Secret holds the metadata files. The tt-config volume contains the contents of the myconfig ConfigMap and the mysecret Secret.

Example using one ConfigMap for a TimesTenScaleout object

This example shows you how to create a metadata file for a direct connectable and a metadata file for a client/server connectable. It then shows you how to include these connectables in a ConfigMap for a TimesTenScaleout object. The ConfigMap also includes the db.ini, adminUser, and schema.sql metadata files.

On your development host, from the directory of your choice, create an empty subdirectory for the metadata files. This example creates the cm_samplescaleout subdirectory. (The cm_samplescaleout directory is used in the remainder of this example to denote this directory.)
```
mkdir -p cm_samplescaleout
```
Change to the ConfigMap directory.
```
cd cm_samplescaleout
```

Create the direct connectable.

vi samplescaleout.connect

ConnectionCharacterSet=AL32UTF8

Create the client/server connectable.

vi samplecsscaleout.csconnect

TTC_Timeout=30

Create the db.ini file. In this db.ini file, define the PermSize and DatabaseCharacterSet connection attributes.
```
vi db.ini

PermSize=200
DatabaseCharacterSet=AL32UTF8
```
Create the adminUser file. In this adminUser file, create the sampleuser user with the samplepw password.
```
vi adminUser

sampleuser/samplepw
```
Create the schema.sql file. In this schema.sql file, define the s sequence and the emp table for the sampleuser user. The Operator automatically initializes your database with these object definitions.
```
vi schema.sql

create sequence sampleuser.s;
create table sampleuser.emp (
  id number not null primary key,
  name char(32)
);
```
Create the ConfigMap. The files in the cm_samplescaleout directory are included in the ConfigMap. These files are later available in the TimesTen containers.
In this example:
- The name of the ConfigMap is samplescaleout. Replace samplescaleout with a name of your choosing.
- This example uses cm_samplescaleout as the directory where the files that will be copied into the ConfigMap reside. If you use a different directory, replace cm_samplescaleout with the name of your directory.
```
kubectl create configmap samplescaleout --from-file=cm_samplescaleout
```
The output is the following:
```
configmap/samplescaleout created
```

Verify the contents of the ConfigMap.

kubectl describe configmap samplescaleout

The output is the following:

Name:         samplescaleout
Namespace:    mynamespace
Labels:       <none>
Annotations:  <none>
 
Data
====
adminUser:
----
sampleuser/samplepw
 
db.ini:
----
PermSize=200
DatabaseCharacterSet=AL32UTF8

sampleconnectable.connect:
----
ConnectionCharacterSet=AL32UTF8

samplecsconnectable.csconnect:
----
TTC_Timeout=30
 
schema.sql:
----
create sequence sampleuser.s;
create table sampleuser.emp (
  id number not null primary key,
  name char(32)
);

Events:  <none>

Include the ConfigMap in the object definition.

Note this example uses a storageSize of 250Gi (suitable for a production environment). For demonstration purposes, a storageSize of 50Gi is adequate.

apiVersion: timesten.oracle.com/v1
kind: TimesTenScaleout
metadata:
  name: samplescaleout
spec:
  ttspec:
    storageClassName: oci
    storageSize: 250Gi
    image: container-registry.oracle.com/timesten/timesten:22.1.1.9.0
    imagePullSecret: sekret
    k: 2
    nReplicaSets: 3
    nZookeeper: 3
    dbConfigMap:
    - samplescaleout

Using an init container

You can use an init container to place your metadata files into the /ttconfig directory of the TimesTen containers. An init container lets you to create your own scripts to populate the /ttconfig directory. You can use an init container for TimesTenClassic and for TimesTenScaleout objects. For more information about init containers, see:

https://kubernetes.io/docs/concepts/workloads/pods/init-containers

Note:

Here is an example that illustrates how to use an init container for a TimesTenClassic object. The template element is required. This element is applied to Pods that contain the TimesTen Classic instances. The example shows you where to specify the script that populates the /ttconfig directory. It also uses the tt-config volume name in the volumes field of the TimesTenClassic object. If you specify a volume with the tt-config name, it is automatically mounted at /ttconfig in your TimesTen containers.

apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
  name: init1
spec:
  ttspec:
    storageClassName: oci
    storageSize: 250Gi
    image: container-registry.oracle.com/timesten/timesten:22.1.1.9.0
    imagePullSecret: sekret
  template:
    spec:
      imagePullSecrets: 
      - name: sekret
      initContainers:
      - name: initclassic
        image: container-registry.oracle.com/timesten/timesten:22.1.1.9.0
        command:
        - sh
        - "-c"
        - |
          /bin/bash <<'EOF'
          Your script to populate /ttconfig goes here
          EOF
        volumeMounts:
        - name: tt-config
          mountPath: /ttconfig
      volumes:
      - name: tt-config
        emptyDir: {}

When using an init container for a TimesTenScaleout object, the metadata files must be placed in both the dataTemplate and the mgmtTemplate elements.

Additional configuration options

This section discusses additional configuration options. These are optional configurations for your environment:

Persistent storage

When you create a TimesTenClassic object, the Operator automatically creates one or two Persistent Volume Claims (PVCs) per Pod. These PVCs cause Persistent Volumes (PVs) to be allocated by Kubernetes and to be attached to the TimesTen Pods. TimesTen uses the PVs to hold the TimesTen instance and the TimesTen database. If you specify two PVCs, one PV holds the instance and the checkpoint files and the second PV holds the transaction log files.

When you create a TimesTenScaleout object, the same mechanism is used to provision persistent storage for the data instances, the management instance, and the ZooKeeper instances:

For the data instances: The Operator creates one or two PVCs per Pod. TimesTen uses the PVs to hold the TimesTen instance and the TimesTen database. If you specify two PVCs, one PV holds the instance and the checkpoint files and the second PV holds the transaction log files.
For the management instance: The Operator creates one PVC for the Pod that contains the management instance. The PV holds the the TimesTen management instance and the grid database.
For the ZooKeeper instances: The Operator creates one PVC for each Pod that runs a ZooKeeper instance. The PV holds ZooKeeper's persistent data.

When you create a TimesTenClassic object, you must specify storageClassName and you may specify storageSize. These attributes determine the characteristics of the Persistent Volumes. The storageClassName must be one that is provided in your Kubernetes environment. For example, in Oracle Kubernetes Environment (OKE), you may use oci.

The default storage is 50Gi. Use the storageSize attribute to request a different size. A storage size of 50Gi may be adequate for demonstration purposes, but in production environments, you should consider greater storage.

TimesTen places the TimesTen installation, the instance, and the database in this storage. It is mounted in each container, in each Pod, as /tt. The TimesTen instance is located at /tt/home/timesten/instances/instance1.

When you create a TimesTenScaleout object, the following attributes are supported:

Storage class name:
- dataStorageClassName: Name of the storage class that is used to request persistent volumes for the elements of the TimesTen database in the grid. If not specified, the default is the value of storageClassName.
- mgmtStorageClassName: Name of the storage class that is used to request persistent volumes for the database of the management instance. If not specified, the default is the value of storageClassName.
- zookeeperStorageClassName: Name of the storage class that is used to request persistent volumes for ZooKeeper's persistent data. If not specified, the default is the value of storageClassName.
- storageClassName: If the data storage class name, the management storage class name, and the zookeeper storage class name are the same, you can just specify storageClassName.
Storage size:
- dataStorageSize: Amount of storage to be provisioned for each element of the TimesTen database in the grid. The default is 50Gi.
- mgmtStorageSize: Amount of storage to be provisioned for the database of the management instance. The default is 50Gi.
- zookeeperStorageSize: Amount of storage to be provisioned for ZooKeeper's persistent data. The default is 50Gi.
- storageSize: If the data storage size, the management storage size, and the zookeeper storage size are the same, you can just specify storageSize. For example, if the dataStorageSize is 75Gi, and the mgmtStorageSize is 75Gi, and the zookeeperStorageSize is 75Gi, you can specify storageSize with a value of 75Gi. The value for dataStorageSize, for mgmtStorageSize, and for zookeeperStorageSize is set to the value of storageSize.

For the TimesTen databases (using TimesTen Classic) and for the TimesTen database (using TimesTen Scaleout):

TimesTen best practices recommends that the transaction log files associated with a TimesTen database be located on a different storage volume than the checkpoint files for the database. This provides separate paths to storage for the checkpoint and the transaction log operations. For example, you can store the transaction log files in a high performance storage, while storing the checkpoint files in a slower storage. See "Locate checkpoint and transaction log files on separate physical device" in the Oracle TimesTen In-Memory Database Operations Guide for more information.
To locate the checkpoint files and the transaction log files on a separate path of storage, provide a value for a second persistent storage that is used only for the transaction log files. Use the logStorageSize attribute for this and control its placement by using the logStorageClassName attribute. This causes a second PVC to be created for each Pod, which will then be available in each container at /ttlog. (This second storage volume has a /ttlog mount point.)

Here is an example for a TimesTenClassic object. The same example can be used for a TimesTenScaleout object:
```
apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
  name: sample
spec:
  ttspec:
    storageClassName: slower
    storageSize: 750Gi
    logStorageClassName: faster
    logStorageSize: 200G
```

Additional resource specifications

Kubernetes supports affinity and anti-affinity settings that let applications control their placement within the Kubernetes cluster. These settings can be used to ensure all replicas do not reside on a single physical host.

You can specify affinity settings, node selectors, additional containers, tolerations, resource requirements, and other Kubernetes attributes for the TimesTen Pods and the containers within these Pods that are created by the TimesTen Operator.

In a TimesTenClassic deployment, you specify these resource specifications in the TimesTenClassic object's .spec.template datum. The TimesTen Operator passes this template to the StatefulSet. For example, when you deploy a TimesTenClassic object, the Operator configures a replicated pair of TimesTen databases that provide high availability. However, since the Operator does not control the placement of Pods, you can achieve an even greater level of high availability by controlling the placement of the TimesTen Pods. TimesTen Pods can then be available in different availability zones or are on different Kubernetes nodes. To do this, you specify the affinity option in the .spec.template datum for the TimesTenClassic object.

Similar to a TimesTenClassic deployment, you can specify the same resource specifications for a TimesTenScaleout object. The TimesTenScaleout object supports the .spec.mgmtTemplate, .spec.dataTemplate and .spec.zookeeperTemplate attributes. You can use these attributes to pass affinity and other settings to Kubernetes. These are of type PodTemplateSpec:

mgmtTemplate: Applied to the Pod that contains the TimesTen Scaleout management instance. Consists of a single PodTemplateSpec.
dataTemplate: Applied to the Pods that contain the TimesTen Scaleout data instances. Consists of an array of PodTemplateSpec. If specified, there must be one entry in the array for each data space in the grid (k entries in the array). This lets you specify a different placement for each data space. For example, you can have data space one reside in availability zone 1 and data space two reside in availability zone 2.
zookeeperTemplate: Applied to the Pods that contain the ZooKeeper instances used by Scaleout. Consists of a single PodTemplateSpec.

For information about PodTemplateSpec see, https://kubernetes.io/docs/reference/kubernetes-api/

Here is an example of specifying the affinity setting for a TimesTenClassic object.

apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
  name: sample
spec:
  …
  template:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            podAffinityTerm:
             labelSelector:
              matchExpressions: 
               - key: "app"
                 operator: In
                 values:
                  - ds1
             topologyKey: "kubernetes.io/hostname"

Readiness probes

Kubernetes readiness probes lets Kubernetes tell whether a particular application is ready. For example, consider an application that, when it starts, has to perform a lengthy startup procedure. When the application is started by Kubernetes, it is not immediately ready. It cannot handle requests or workloads until the startup procedure is complete.

A readiness probe defines an action that Kubernetes takes in order to determine if the application running in a particular container is ready or not. There are several forms that a readiness probe can take, such as executing a command within the container, or issuing a GET request on a socket into a container. Whichever approach is defined, the end result is that the probe must give a Yes or No answer as to whether the application in the container is ready. The definition of ready is up to the application itself, and varies from application to application. In addition, the logic involved in determining if a given application is ready varies from application to application. See:

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

In TimesTen Classic, the TimesTen Operator supports readiness probes in the following way: A TimesTen instance that is created by the Operator, and is running in the tt container of a Pod, is ready if the database inside of that Pod is loaded, replication is configured, and the database is either a fully functional active or a fully functional standby. A database that is down, or a database that is in the process of being created or duplicated is not ready.

A good use for readiness probes is if you want to replace one or more Nodes in your Kubernetes cluster. In such a case, you can cause Kubernetes to drain the workload from the Node. This causes Kubernetes to delete any Pods that are running on that Node, and create new Pods on other Nodes in the cluster to replace them.

Kubernetes supports the concept of a Pod disruption budget, where you can specify a budget for your application. This budget tells Kubernetes how many evicted Pods in a given Deployment can be tolerated. For example, assume you configure a Deployment with 20 replicas of your application. You could tell Kubernetes to tolerate up to 5 of the replicas being down at a time. When moving a workload from one Node to another, Kubernetes is careful not to delete more than 5 replicas at a time, and waits for their replacements to become ready before deleting more. See:

https://kubernetes.io/docs/concepts/workloads/pods/disruptions/

https://kubernetes.io/docs/tasks/run-application/configure-pdb/

In the case of the TimesTen Kubernetes Operator, if you define a readiness probe in this way and you use a Pod disruption budget of 1 on TimesTen, you can drain the workload from one or more Nodes without creating a total TimesTen outage. Note that when Kubernetes deletes a Pod that is running TimesTen in TimesTen Classic, Kubernetes does not know if the Pod contains an active database or a standby database. Therefore, it may choose to delete the Pod that contains the active database. This causes a failover to the standby, which disrupts applications if performed during normal hours. There is not a way to prevent this. However, Kubernetes will not proceed to delete the other database until the one that was deleted comes back up and is completely in the Healthy state. See "Monitoring the health of each Pod in the active standby pair" for more information on the health of a Pod and the Healthy state.

TimesTen readiness probe

A TimesTen container is defined as ready if the TimesTen database it contains is either a fully functional active or a fully functional standby. If the database is down or is in a transitional state, the container is not ready.

A TimesTen container is ready to Kubernetes by the presence or absence of the /tmp/readiness file in the container's filesystem. If the file exists, the container is ready. You define the readiness probe in the TimesTenClassic object definition.

In the following example, the sample TimesTenClassic object includes a TimesTen readiness probe. Kubernetes executes the cat command in the tt container every 10 seconds. If the command exits with a return code of 0, the container is ready. If the command returns any other value, the container is not ready.

apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
  name: sample
spec:
  ttspec:
…
  template:
    spec:
…
      containers:
      - name: tt
        readinessProbe:
          exec: 
            command:
            - cat
            - /tmp/readiness
          failureThreshold: 1
          periodSeconds: 10
          successThreshold: 1