This chapter describes how to use TimesTen Cache in your Kubernetes environment.
Topics include:
See Appendix B, "TimesTen Cache Example" for a complete example of configuring and using TimesTen Cache in your Kubernetes environment. The example also includes details on setting up the Oracle Database.
You can configure and then use TimesTen Cache in your Kubernetes environment. The TimesTen Operator provides these metadata files for this purpose:
cacheUser: The cacheUser file is required. This file contains one line of the form:
cacheUser/ttPassword/oraPassword
where cacheUser is the user you want to designate as the TimesTen cache manager user. This user must have the same name as the user whom you designated as the cache administration user in the Oracle Database. The cache administration user must already exist in the Oracle Database. Specify ttPassword as the TimesTen password for the TimesTen cacheUser user (the TimesTen cache manager). The oraPassword is the Oracle Database password you specified when you created the cacheUser user in the Oracle Database.
The Operator creates the cacheUser user with the ttpassword in the TimesTen database. This cacheUser user then serves as the cache manager user in your TimesTen database. (Note that you do not need to create this TimesTen user. The Operator does it for you.)
See "Create the Oracle database users" in the Oracle TimesTen Application-Tier Database Cache User's Guide for information on the Oracle Database users. Also see "cacheUser" for more information on the cacheUser metadata file.
cachegroups.sql: The cachegroups.sql file is required. The contents of this file contain the CREATE CACHE GROUP definitions. The file can also contain the LOAD CACHE GROUP statement and the built-in procedures to update statistics on the cache group tables (such as, ttOptEstimateStats and ttOptUpdateStats). See "cachegroups.sql" for more information on this file.
tnsnames.ora: The tnsnames.ora file is required. This file defines Oracle Net Services to which applications connect. For TimesTen Cache, this file configures the connectivity between the TimesTen and the Oracle Database (from which data is being cached). In this context, TimesTen is the application that is the connection to the Oracle Database. See "tnsnames.ora file" for more information on this file.
sqlnet.ora: The sqlnet.ora file is not required. It may be necessary depending on your Oracle Database configuration. The file defines options for how client applications communicate with the Oracle Database. In this context, TimesTen is the application. The tnsnames.ora and sqlnet.ora files together define how an application communicates with the Oracle Database. See "sqlnet.ora file" for information on this file.
db.ini: The db.ini file is required. The contents of this file contain TimesTen connection attributes for your TimesTen databases, which will be included in TimesTen's sys.odbc.ini file. You must specify the OracleNetServiceName and the DatabaseCharacterSet connection attributes in this file. The DatabaseCharacterSet value must match the value of the Oracle database character set value. See "db.ini file" for more information on this file.
schema.sql: The schema.sql file may be required. In TimesTen Cache, one or more cache table users own the cache tables. If this cache table user is not the cache manager user, then you must specify the schema.sql file and in it you must include the schema user and assign the appropriate privileges to this schema user. For example, if the oratt schema user was created in the Oracle Database, and this user is not the TimesTen cache manager user, you must create the TimesTen oratt user in this file. See "Create the Oracle Database users" for more information on the schema users in the Oracle Database. Also see "schema.sql file" for more information on the schema.sql file.
The instance administrator uses the ttIsql utility to run this file immediately after the database is created. This file is run before the Operator configures TimesTen Cache or replication, so ensure there are no cache definitions in this file.
The Operator looks for the presence of the cacheUser and the cachegroups.sql files in the /ttconfig directory of your TimesTen containers to determine if TimesTen Cache should be configured. If these files are present, the Operator, creates the TimesTen cache manager (from the contents of the cacheUser file) and starts the cache agent. The cache manager then uses the ttIsql utility to run the cachegroups.sql file.
The contents of the cachegroups.sql file is run on the active database before it is duplicated to the standby. If there are autorefresh cache groups specified in the cachegroups.sql file, they are paused by the agent prior to duplicating the active database to the standby. After the duplication process completes, these autorefresh cache groups are re-enabled.
Once your active standby pair of TimesTen databases are created and rolled out, the Operator does not monitor or manage TimesTen Cache. Specifically, the Operator does not monitor the health of the cache agents, nor does it take further action to start or stop them. In addition, the Operator does not verify that data is propagating correctly between the TimesTen database and the Oracle Database.
If you delete your databases (by deleting the TimesTenClassic object), the Operator automatically cleans up the Oracle Database metadata. If, however, you want to retain the Oracle Database metadata, specify the cacheCleanUp field in your TimesTenClassic object definition and set its value to false. See "Cleaning up the cache metadata on the Oracle Database" and see the cacheCleanUp entry in Table 11-3, "TimesTenClassicSpecSpec" for more information.
You can include the cacheUser, cachegroups.sql, tnsnames.ora, sqlnet.ora, db.ini and the schema.sql metadata files in one or more Kubernetes facilities (for example, in a Kubernetes Secret, in a ConfigMap, or in an init container). This ensures the metadata files are populated in the /ttconfig directory of the TimesTen containers. Note that there is no requirement as to how to get the metadata files into this /ttconfig directory. See "Populating the /ttconfig directory" for more information.
This example uses the ConfigMap facility to populate the /ttconfig directory in your TimesTen containers.
On your Linux development host:
From the directory of your choice, create an empty subdirectory for the metadata files. This example creates the cm_cachetest subdirectory. (The cm_cachetest directory is used in the remainder of this example to denote this directory.)
% mkdir -p cm_cachetest
Navigate to the ConfigMap directory.
% cd cm_cachetest
Create the cacheUser metadata file in this ConfigMap directory (cm_cachetest, in this example). The cacheUser file must contain one line of the form cacheuser/ttpassword/orapassword, where cacheuser is the user you wish to designate as the cache manager user in the TimesTen database, ttpassword is the TimesTen password you wish to assign to this user, and orapassword is the Oracle Database password that has already been assigned to the Oracle Database cache administration user. Note that the cacheUser name in this file must match the Oracle Database cache administration user.
In this example, the cacheuser2 user with password of oraclepwd was already created in the Oracle Database. Therefore, supply cacheuser2 as the TimesTen cache manager user. You can assign any TimesTen password to this TimesTen cache manager user. This example assigns ttpwd.
vi cacheuser cacheuser2/ttpwd/oraclepwd
Create the cachegroups.sql metadata file in this ConfigMap directory (cm_cachetest, in this example). The cachegroups.sql file contains the cache group definitions. In this example, a dynamic AWT cache group and a read-only cache group are created. In addition, the LOAD CACHE GROUP statement is included to load rows from the oratt.readtab cached table in the Oracle Database into the oratt.readtab cache table in the TimesTen database.
vi cachegroups.sql CREATE DYNAMIC ASYNCHRONOUS WRITETHROUGH CACHE GROUP writecache FROM oratt.writetab ( pk NUMBER NOT NULL PRIMARY KEY, attr VARCHAR2(40) ); CREATE READONLY CACHE GROUP readcache AUTOREFRESH INTERVAL 5 SECONDS FROM oratt.readtab ( keyval NUMBER NOT NULL PRIMARY KEY, str VARCHAR2(32) ); LOAD CACHE GROUP readcache COMMIT EVERY 256 ROWS;
Create the tnsnames.ora metadata file in this ConfigMap directory (cm_cachetest, in this example). See "tnsnames.ora file" for more information on this file.
vi tnsnames.ora
OraTest =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = database.myhost.svc.cluster.local)
(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = OraTest.my.domain.com)))
OraCache =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = database.myhost.svc.cluster.local)
(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = OraCache.my.domain.com)))
Create the sqlnet.ora metadata file in this ConfigMap directory (cm_cachetest, in this example). See "sqlnet.ora file" for more information on this file.
vi sqlnet.ora
NAME.DIRECTORY_PATH= {TNSNAMES, EZCONNECT, HOSTNAME}
SQLNET.EXPIRE_TIME = 10
SSL_VERSION = 1.2
Create the db.ini file in this ConfigMap directory (cm_cachetest, in this example). In this db.ini file, define the PermSize, DatabaseCharacterSet, and the OracleNetServiceName connection attributes. The DatabaseCharacterSet value must match the database character set value in the Oracle Database. See "Create the Oracle Database tables to be cached" for information on how to query the nls_database_parameters system view to determine the Oracle Database database character set. In this example, the value is AL32UTF8.
vi db.ini
PermSize=200 DatabaseCharacterSet=AL32UTF8 OracleNetServiceName=Oracache
Create the schema.sql file in this ConfigMap directory (cm_cachetest, in this example). In this example, create the oratt user. (In this example, also assume this oratt user was previously created in the Oracle Database.) See "Create the Oracle Database users" for information on the oratt user in the Oracle Database.
vi schema.sql create user oratt identified by ttpwd; grant admin to oratt;
Create the ConfigMap. The files in the cm_cachetest directory are included in the ConfigMap and, later, will be available in the TimesTen containers.
In this example:
The name of the ConfigMap is cachetest. Replace cachetest with a name of your choosing. (cachetest is represented in bold in this example.)
This example uses cm_cachetest as the directory where the files that will be copied into the ConfigMap reside. If you use a different directory, replace cm_cachetest with the name of your directory. (cm_cachetest is represented in bold in this example.)
Use the kubectl create command to create the ConfigMap:
% kubectl create configmap cachetest --from-file=cm_cachetest configmap/cachetest created
Use the kubectl describe command to verify the contents of the ConfigMap. (cachetest, in this example.) The metadata files are represented in bold.
% kubectl describe configmap cachetest; Name: cachetest Namespace: mynamespace Labels: <none> Annotations: <none> Data ==== tnsnames.ora: ---- OraTest = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = database.myhost.svc.cluster.local) (PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = OraTest.my.domain.com))) OraCache = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = database.myhost.svc.cluster.local) (PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = OraCache.my.domain.com))) cacheUser: ---- cacheuser2/ttpwd/oraclepwd cachegroups.sql: ---- CREATE DYNAMIC ASYNCHRONOUS WRITETHROUGH CACHE GROUP writecache FROM oratt.writetab ( pk NUMBER NOT NULL PRIMARY KEY, attr VARCHAR2(40) ); CREATE READONLY CACHE GROUP readcache AUTOREFRESH INTERVAL 5 SECONDS FROM oratt.readtab ( keyval NUMBER NOT NULL PRIMARY KEY, str VARCHAR2(32) ); LOAD CACHE GROUP readcache COMMIT EVERY 256 ROWS; db.ini: ---- permSize=200 databaseCharacterSet=AL32UTF8 oracleNetServiceName=Oracache schema.sql: ---- create user oratt identified by ttpwd; grant admin to oratt; sqlnet.ora: ---- NAME.DIRECTORY_PATH= {TNSNAMES, EZCONNECT, HOSTNAME} SQLNET.EXPIRE_TIME = 10 SSL_VERSION = 1.2 Events: <none>
You have successfully created and deployed the cachetest ConfigMap.
This section creates the TimesTenClassic object. See "Defining and creating the TimesTenClassic object" and "The TimesTenClassic object type" for detailed information on the TimesTenClassic object.
Perform these steps:
Create an empty YAML file. You can choose any name, but you may want to use the same name you used for the name of the TimesTenClassic object. (In this example, cachetest.) The YAML file contains the definitions for the TimesTenClassic object. See "TimesTenClassicSpecSpec" for information on the fields that you must specify in this YAML file as well as the fields that are optional.
In this example, note these fields:
name: Replace cachetest with the name of your TimesTenClassic object (represented in bold).
storageClassName: Replace oci with the name of the storage class used to allocate PersistentVolumes to hold TimesTen.
storageSize: Replace 250G with the amount of storage that should be requested for each Pod to hold TimesTen. Note: This example assumes a production environment and uses a value of 250G for storageSize. For demonstration purposes, a value of 50G is adequate. See the storageSize and the logStorageSize entries in the Table 11-3, "TimesTenClassicSpecSpec" for information.
image: Replace phx.ocir.io/youraccount/tt1814110:3 with the location of the image registry (phx.ocir.io/youraccount) and the image containing TimesTen (tt1814110:3).
imagePullSecret: Replace sekret with the image pull secret that Kubernetes should use to fetch the TimesTen image.
dbConfigMap: This example uses one ConfigMap (called cachetest) for the metadata files (represented in bold).
% vi cachetest.yaml apiVersion: timesten.oracle.com/v1 kind: TimesTenClassic metadata: name: cachetest spec: ttspec: storageClassName: oci storageSize: 250G image: phx.ocir.io/youraccount/tt1814110:3 imagePullSecret: sekret imagePullPolicy: Always dbConfigMap: - cachetest
Use the kubectl create command to create the TimesTenClassic object from the contents of the YAML file (in this example, cachetest.yaml). Doing so begins the process of deploying your active standby pair of TimesTen databases in the Kubernetes cluster.
% kubectl create -f cachetest.yaml timestenclassic.timesten.oracle.com/cachetest created
You have successfully created the TimesTenClassic object in the Kubernetes cluster. The process of deploying your TimesTen databases begins, but is not yet complete.
Use the kubectl get and the kubectl describe commands to monitor the progress of the active standby pair as it is provisioned.
Use the kubectl get command and review the STATE field. Observe the value is Initializing. The active standby pair provisioning has begun, but is not yet complete.
% kubectl get ttc cachetest NAME STATE ACTIVE AGE cachetest Initializing None 41s
Use the kubectl get command again to see if value of the STATE field has changed. In this example, the value is Normal, indicating the active standby pair of databases are now provisioned and the process is complete.
% kubectl get ttc cachetest NAME STATE ACTIVE AGE cachetest Normal cachetest-0 3m58s
Use the kubectl describe command to view the active standby pair provisioning in detail.
Note the following:
The cachetest Configmap has been correctly referenced in the dbConfigMap field (represented in bold).
The cache agent is running in the active and the standby Pods (represented in bold).
The cache administration user UID and password have been set in the active and the standby Pods (represented in bold).
Two cache groups have been created in the active and the standby Pods (represented in bold).
The replication agent is running in the active and the standby Pods (represented in bold).
% kubectl describe ttc cachetest
Name: cachetest
Namespace: mynamespace
Labels: <none>
Annotations: <none>
API Version: timesten.oracle.com/v1
Kind: TimesTenClassic
Metadata:
Creation Timestamp: 2020-10-24T03:29:48Z
Generation: 1
Resource Version: 78390500
Self Link: /apis/timesten.oracle.com/v1/namespaces/mynamespace/timestenclassics/cachetest
UID: 2b18d81d-15a9-11eb-b999-be712d29a81e
Spec:
Ttspec:
Db Config Map:
cachetest
Image: phx.ocir.io/youraccount/tt1814110:3
Image Pull Policy: Always
Image Pull Secret: sekret
Storage Class Name: oci
Storage Size: 250G
Status:
Active Pods: cachetest-0
High Level State: Normal
Last Event: 28
Pod Status:
Cache Status:
Cache Agent: Running
Cache UID Pwd Set: true
N Cache Groups: 2
Db Status:
Db: Loaded
Db Id: 30
Db Updatable: Yes
Initialized: true
Pod Status:
Agent: Up
Last Time Reachable: 1603510527
Pod IP: 10.244.7.92
Pod Phase: Running
Replication Status:
Last Time Rep State Changed: 0
Rep Agent: Running
Rep Peer P State: start
Rep Scheme: Exists
Rep State: ACTIVE
Times Ten Status:
Daemon: Up
Instance: Exists
Release: 18.1.4.11.0
Admin User File: true
Cache User File: true
Cg File: true
High Level State: Healthy
Intended State: Active
Name: cachetest-0
Schema File: true
Cache Status:
Cache Agent: Running
Cache UID Pwd Set: true
N Cache Groups: 2
Db Status:
Db: Loaded
Db Id: 30
Db Updatable: No
Initialized: true
Pod Status:
Agent: Up
Last Time Reachable: 1603510527
Pod IP: 10.244.8.170
Pod Phase: Running
Replication Status:
Last Time Rep State Changed: 1603510411
Rep Agent: Running
Rep Peer P State: start
Rep Scheme: Exists
Rep State: STANDBY
Times Ten Status:
Daemon: Up
Instance: Exists
Release: 18.1.4.11.0
Admin User File: true
Cache User File: true
Cg File: true
High Level State: Healthy
Intended State: Standby
Name: cachetest-1
Schema File: true
Rep Create Statement: create active standby pair "cachetest" on
"cachetest-0.cachetest.mynamespace.svc.cluster.local", "cachetest" on
"cachetest-1.cachetest.mynamespace.svc.cluster.local" NO RETURN store
"cachetest" on "cachetest-0.cachetest.mynamespace.svc.cluster.local"
PORT 4444 FAILTHRESHOLD 0 store "cachetest" on
"cachetest-1.cachetest.mynamespace.svc.cluster.local" PORT 4444
FAILTHRESHOLD 0
Rep Port: 4444
Status Version: 1.0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
- Create 5m40s ttclassic Secret
tt2b18d81d-15a9-11eb-b999-be712d29a81e created
- Create 5m40s ttclassic Service cachetest created
- Create 5m40s ttclassic StatefulSet cachetest created
- StateChange 4m28s ttclassic Pod cachetest-0 Agent Up
- StateChange 4m28s ttclassic Pod cachetest-0 Release 18.1.4.11.0
- StateChange 4m28s ttclassic Pod cachetest-0 Daemon Up
- StateChange 3m18s ttclassic Pod cachetest-0 RepScheme None
- StateChange 3m18s ttclassic Pod cachetest-0 RepAgent Not Running
- StateChange 3m18s ttclassic Pod cachetest-0 RepState IDLE
- StateChange 3m18s ttclassic Pod cachetest-0 Database Loaded
- StateChange 3m18s ttclassic Pod cachetest-0 Database Updatable
- StateChange 3m18s ttclassic Pod cachetest-0 CacheAgent Not Running
- StateChange 2m57s ttclassic Pod cachetest-0 CacheAgent Running
- StateChange 2m47s ttclassic Pod cachetest-1 Agent Up
- StateChange 2m47s ttclassic Pod cachetest-1 Release 18.1.4.11.0
- StateChange 2m46s ttclassic Pod cachetest-0 RepAgent Running
- StateChange 2m46s ttclassic Pod cachetest-0 RepScheme Exists
- StateChange 2m46s ttclassic Pod cachetest-0 RepState ACTIVE
- StateChange 2m46s ttclassic Pod cachetest-1 Daemon Up
- StateChange 2m9s ttclassic Pod cachetest-1 CacheAgent Running
- StateChange 2m9s ttclassic Pod cachetest-1 Database Not Updatable
- StateChange 2m9s ttclassic Pod cachetest-1 Database Loaded
- StateChange 2m9s ttclassic Pod cachetest-1 RepAgent Not Running
- StateChange 2m9s ttclassic Pod cachetest-1 RepScheme Exists
- StateChange 2m9s ttclassic Pod cachetest-1 RepState IDLE
- StateChange 2m3s ttclassic Pod cachetest-1 RepAgent Running
- StateChange 118s ttclassic Pod cachetest-1 RepState STANDBY
- StateChange 118s ttclassic TimesTenClassic was Initializing, now
Normal
Your active standby pair of TimesTen databases are successfully deployed (as indicated by Normal.) You are now ready to use TimesTen Cache in your Kubernetes environment.
When you create certain types of cache groups in a TimesTen database, TimesTen stores metadata about that cache group in the Oracle Database. If you later delete that TimesTen database, TimesTen does not automatically delete the metadata in the Oracle Database. As a result, metadata can accumulate on the Oracle Database. See "Dropping Oracle Database objects used by autorefresh cache groups" in the Oracle TimesTen Application-Tier Database Cache User's Guide for more information.
However, in a Kubernetes environment, if you provide a cacheUser metadata file and a cachegroups.sql metadata file when you initially create the TimesTenClassic object, then, by default, the Operator automatically cleans up the Oracle Database metadata if you delete that TimesTenClassic object.
If you do not want the Operator to automatically clean up the Oracle Database, you set the cacheCleanup field in the TimesTenClassic object definition to false. See the cacheCleanup entry in Table 11-3, "TimesTenClassicSpecSpec" for more information. Also see "The supported metadata files" for information on the cacheUser and the cachegroups.sql files.