6 Deploying your TimesTen Grid

The TimesTen Operator supports the deployment of a TimesTen Scaleout grid and associated databases in your Kubernetes cluster.

This chapter provides background information about the deployment process. It summarizes the planning process for configuring a grid and shows you how to apply that plan when configuring a grid in the Kubernetes environment. It describes the steps the TimesTen Operator takes to deploy and provision a grid based on the information you provide. Use this information to gain an understanding of how the Operator functions when deploying your TimesTen Scaleout grid.

The second part of the chapter provides you with an end-to-end example that shows you the steps to deploy your grid in the Kubernetes cluster.

If you want to advance directly to the example, see Deploy your grid.

The TimesTen Operator also supports TimesTen Cache in TimesTen Scaleout. See Working with TimesTen Cache.

Topics:

• About deploying your grid

• Deploy your grid

About deploying your grid

The information in this section provides background information about configuring and deploying a grid in the Kubernetes environment.

Topics:

• About planning your grid

• About configuring your grid

• About provisioning your grid

• About ssh

• About creating your grid

About planning your grid

One of the features of the TimesTen Operator is the ability for it to provision a TimesTen Scaleout grid and its database in the Kubernetes cluster. Just like in any other TimesTen Scaleout environment, you must do some planning for your grid.

Here are some considerations:

• K-Safety (represented by k): How many copies of your TimesTen database? The K-Safety factor determines the number of data spaces in your grid. For example, if you set k to 2, there are two copies of your database: one copy resides in data space one and the second copy in data space two.

• Replica sets: How many replica sets in your grid? A replica set contains k elements, where each element in the replica set is an exact copy of the other elements in the replica set. The value of k, combined with the number of replica sets, determines the number of data instances in the grid. For example, if you set k to 2, and replica sets to 3, then there are six data instances in the grid.

• ZooKeeper instances: How many ZooKeeper instances for the grid?

• Database definition file (DbDef): What data store and first connection attributes are needed for the database in the grid?
• Direct connectable: What general connection attributes are needed for the database when using direct access?
• Client/server connectable: What general connection attributes are needed for the database when using client/server access?

After you define your configuration, you provide that information to the TimesTen Operator. The Operator takes over from there. It automatically does the provisioning and the deployment of the grid and database for you.

About configuring your grid

The TimesTen Operator provides the TimesTenScaleout object type and configuration metadata so that you can define and then deploy your TimesTen Scaleout grid and database. The TimesTenScaleout object type provides the syntax for configuring your grid. The configuration metadata lets you define the connection attributes for your database. Taken together, the Operator has the necessary information to provision your grid and database in the Kubernetes cluster.

Based on your planned configuration in About planning your grid, you can apply that configuration to the Kubernetes environment:

• K-Safety (k): The Operator provides the .spec.ttspec.k element as part of the syntax for the TimesTenScaleout object type. You specify k in the YAML manifest file for your TimesTenScaleout object.

• Replica sets: The Operator provides the .spec.ttspec.nReplicaSets element as part of the syntax for the TimesTenScaleout object type. You specify nReplicaSets in the YAML manifest file for your TimesTenScaleout object.

• ZooKeeper instances: The Operator provides the .spec.ttspec.nZookeeper element as part of the syntax for the TimesTenScaleout object type. You specify nZookeeper in the YAML manifest file for your TimesTenScaleout object.

• Database definition file (DbDef): The Operator creates a DbDef from the contents of the db.ini metadata file. You create this metadata file and then use a Kubernetes facility (or some other means) to place this file in the /ttconfig directory of the tt containers.

• One or more direct connectables: The Operator creates one or more direct connectables from the contents of the *.connect metadata files. You create one or more of these *.connect files and then use a Kubernetes facility (or some other means) to place the files in the /ttconfig directory of the tt containers.

• One or more client/server connectables: The Operator creates one or more client/server connectables from the contents of the *.csconnect metadata files. You create one or more of these *.csconnect files and then use a Kubernetes facility (or some other means) to place the files in the /ttconfig directory of the tt containers.

See TimesTenScaleoutSpecSpec, List of the configuration metadata, and Populating the /ttconfig directory.

After the metadata files are placed in the /ttconfig directory of the tt containers, and you configure and then deploy your TimesTenScaleout object in the Kubernetes cluster, Kubernetes informs the Operator that a TimesTenScaleout object has been created. The Operator begins the process of creating additional Kubernetes objects in order to implement your grid.

About provisioning your grid

The TimesTen Operator gathers the information from the TimesTenScaleout object and begins instantiating the TimesTen Scaleout grid and database. It creates the following StatefulSets:

• One StatefulSet that provides the management instance for the grid. The Operator supports one management instance. The underlying Pod for this management instance is also created.

• One StatefulSet that provides one or more ZooKeeper instances. The Operator determines the number of ZooKeeper instances by the value you specified for the nZookeeper field in your TimesTenScaleout object definition. For example, if you specified a value of 3 for nZookeeper, the Operator creates one StatefulSet with three replicas. The underlying Pods for these ZooKeeper instances are also created. There is one Pod per ZooKeeper instance.

• Additional StatefulSets, the number of which is determined by the value you specified for the k field in your TimesTenScaleout object definition. For example, if you specified a value of 2 for k, the Operator creates two StatefulSets. These StatefulSets provide data instances for the grid. Each of the k StatefulSets provides Pods to implement a single data space in the resultant grid. The StatefulSet has M replicas, the number of which is determined by the value you specified for the nReplicaSets field in your TimesTenScaleout object definition. For example, if you set nReplicaSets to 3, each StatefulSet has three replicas. This number of replicas determines the number of replica sets in the resultant grid.

  In the preceding example, one StatefulSet has three replicas. This one StatefulSet contains three data instances in data space one. A Pod is created for each data instance, so there are three Pods created. The second StatefulSet contains three data instances in data space two. A Pod is created for each data instance, so there are three Pods created. There are a total of six total Pods created for the six data instances.

In addition, the Operator creates the following Kubernetes headless Services:

• One headless Service that provides DNS names for the Pods that contain the management and data instances. This service enables client/server access to the Pods using the TimesTen client/server port 6625.

• One headless Service that provides DNS names for the Pods that contain the ZooKeeper instances. This service enables access to the ZooKeeper internal ports 2888 and 3888, as well as the external port 2181.

There is an example of these StatefulSets and headless Services in Verify the underlying objects.

The Operator also creates Persistent Volume Claims (PVCs) for the Pods. These PVCs cause persistent volumes to be allocated by Kubernetes and attached to the TimesTen Pods and ZooKeeper Pods. See Persistent storage.

Pods that run ZooKeeper consists of a single container called zookeeper. Each Pod that is running TimesTen consists of at least two containers. The tt container runs the TimesTen agent and TimesTen. The daemonlog container writes the TimesTen daemon log to stdout.

As the tt containers in the TimesTen Pods start, the Operator assembles them into a working grid. The grid's model is configured with several objects, including:

• Hosts for the data instances in each of the data space groups

• Each host configured with an installation of TimesTen
• Each host configured with a single TimesTen instance
• A DbDef (the contents of which are from the db.ini file)
• Direct mode connectables, if any, (the contents of which are from the *.connect files)
• Client/server connectables, if any, (the contents of which are from the *.csconnect files)

About ssh

A TimesTen Scaleout grid relies on password-less ssh among the instances of the grid. The TimesTen Operator automatically configures password-less ssh among the tt containers in the grid in your Kubernetes environment. There is no intervention that you need to do.

About creating your grid

When creating the grid, the TimeTen Operator transitions to and from various High Level states. Here is an explanation of these states:

After you create your TimesTenScaleout object in the Kubernetes cluster, the TimesTen Operator creates the StatefulSets and Services that are required to deploy your TimesTenScaleout grid and database. The Operator assigns a High Level state of Initializing to the TimesTenScaleout object.

The Operator periodically polls the status of the StatefulSets' objects and their underlying Pods. When the ZooKeeper Pods are ready, the TimesTenScaleout object transitions from the Initializing state to the ZooKeeperReady state.

When the TimesTen agent in the management Pod starts up, the Operator instructs the agent to create a TimesTen instance and grid. The TimesTenScaleout object transitions to the GridCreated state.

The Operator waits for all of the TimesTen agents in all of the Pods to start. Once all have started, the Operator instructs the TimesTen agent in the management instance to create the hosts, the installations, and the instances in the grid's model for the data instances in the grid.

The DbDef is then created from the contents of the db.ini file. The direct connectables are created from the contents of the *.connect files. The client/server connectables are created from the contents of the *.csconnect files.

The model is applied and the data instances of the grid are created. The TimesTenScaleout object transitions to the InstanceCreated state.

The Operator then instructs the management instance to create the database (by using the TimesTen ttGridAdmin utility with the dbCreate option) and to create the initial distribution map (by using the TimesTen ttGridAdmin utility with the dbDistribute -add all -apply options). The TimesTenScaleout object then transitions to the DbCreated state.

The Operator then instructs the TimesTen agent in one data instance to run the TimesTen ttIsql utility to create the user in the adminUser file and run the schema.sql file (if you provided these files). The TimesTenScaleout object transitions to the Normal state.

The grid and databases are created. The TimesTen agent then opens the database for connections.

The Operator manages and monitors the TimesTenScaleout objects after they are deployed in your Kubernetes cluster. The Operator also detects, repairs, and recovers from failures in your grid and associated databases. See Managing TimesTen Scaleout.

There is an example showing the Operator transitioning to and from these High Level states in Monitor the High Level state of the TimesTenScaleout object.

Deploy your grid

This section provides a step by step walk-through that shows you how to create and deploy a TimesTen Scaleout grid and database in your Kubernetes cluster. The walk-through starts with an example that shows you how to create metadata files and create a Kubernetes ConfigMap. It continues with an example that shows you how to create a YAML file that contains the definitions for your TimesTenScaleout object. It shows you how to deploy that YAML file in your Kubernetes cluster. You learn how to monitor the provisioning of the TimesTen grid and database, and verify the underlying Kubernetes objects were created by the TimesTen Operator. The walk-through completes with one example that shows you how to connect to the TimesTen database and run operations in it. The final example shows you how to connect to the management instance to verify the health of the database and its elements.

• Create the configuration metadata and the Kubernetes ConfigMap for the grid

• Define and deploy the TimesTenScaleout object

• Monitor the High Level state of the TimesTenScaleout object

• Verify the underlying objects

• Connect to the database

• Manage the database

Create the configuration metadata and the Kubernetes ConfigMap for the grid

The following example creates the db.ini, the adminUser, and the schema.sql metadata files for the TimesTen grid and the database. The example also creates a direct and a client/server connectable. The example creates a Kubernetes ConfigMap to place these metadata files into the /ttConfig directory of the TimesTen containers.

Note:

You can use any Kubernetes mechanism to place these metadata files into the /ttConfig directory of the TimesTen containers. See Populating the /ttconfig directory.

On your development host, complete the following steps:

1. From the directory of your choice, create an empty subdirectory for the metadata files. This example creates the cm_scaleout subdirectory. (The cm_scaleout directory is used in the remainder of this example to denote this directory.)

   mkdir -p cm_scaleout

2. Change to this ConfigMap directory.

   cd cm_scaleout

3. Create the db.ini file in this ConfigMap directory. In this example, define the PermSize and the DatabaseCharacterSet connection attributes.

   vi db.ini
   
   PermSize=200
   DatabaseCharacterSet=AL32UTF8

4. Create the adminUser file.

   vi adminUser
   
   adminuser/adminuserpwd

5. Create the schema.sql file. In this schema.sql file, create the sampleuser user, create the s sequence for the sampleuser user, and the emp table for the sampleuser user. The Operator automatically initializes each element of the TimesTen database with these object definitions.

   vi schema.sql
   
   create user sampleuser identified by sampleuserpwd;
   grant admin to sampleuser;
   create sequence sampleuser.s;
   create table sampleuser.emp (id number not null primary key, name char (32));

6. Create the sampledirect direct connectable.

   vi sampledirect.connect
   
   ConnectionCharacterSet=AL32UTF8

7. Create the sampleclient client/server connectable.

   vi sampleclient.csconnect
   
   TTC_Timeout=70

8. Optional: Verify the metadata files are present in the cm_scaleout ConfigMap directory.

   ls

   The output is the following:

   adminUser
   db.ini
   sampleclient.csconnect
   sampledirect.connect
   schema.sql

9. From the cm_scaleout directory, create the samplescaleout ConfigMap. The files in the cm_scaleout directory are included in the ConfigMap. These files are later available in the TimesTen containers. Replace samplescaleout with a name of your choosing.

   kubectl create configmap samplescaleout --from-file .

   The output is the following:

   configmap/samplescaleout created

10. Use the kubectl describe command to verify the contents of the samplescaleout ConfigMap.

    kubectl describe configmap samplescaleout

    The output is the following:

    Name:         samplescaleout
    Namespace:    mynamespace
    Labels:       <none>
    Annotations:  <none>
    
    Data
    ====
    adminUser:
    ----
    adminuser/adminuserpwd
    
    db.ini:
    ----
    PermSize=200
    DatabaseCharacterSet=AL32UTF8
    
    sampleclient.csconnect:
    ----
    TTC_Timeout=70
    
    sampledirect.connect:
    ----
    ConnectionCharacterSet=AL32UTF8
    
    schema.sql:
    ----
    create user sampleuser identified by sampleuserpwd;
    grant admin to sampleuser;
    create sequence sampleuser.s;
    create table sampleuser.emp (id number not null primary key, name char (32));
    
    Events:  <none>
    

You successfully created the metadata files and the ConfigMap.

Define and deploy the TimesTenScaleout object

Defining your environment involves creating your TimesTenScaleout object with attributes customized for your environment. These attributes are described in the steps below. For additional information on defining objects of type TimesTenScaleout, see TimesTen Scaleout.

To define and create the TimesTenScaleout object, complete the following steps:

1. Create a YAML file. You can choose any name for this YAML file, but you may want to use the same name you used for the name of the TimesTenScaleout object. (This example uses samplescaleout.) The YAML file contains the definitions for the TimesTenScaleout object. In this example, the fields that are specific to a TimesTenScaleout object are as follows:

   • k: Set the value of k to the number of copies of data for your TimesTen database. This value determines the number of StatefulSets that the TimesTen Operator creates. A StatefulSet provides the Pods that are used to implement a single data space in the grid. For example, if you set k to 2, the Operator creates two StatefulSets. One StatefulSet provides the Pods for the data instances in data space one. The second StatefulSet provides the Pods for the data instances in data space two.

     This example sets k to 2.

     For information on K-safety and determining an appropriate value for k, see K-safety in the Oracle TimesTen In-Memory Database Scaleout User's Guide.

   • nReplicaSets: Set the value to the number of replica sets in the grid. A replica set contains k elements (where each element is an exact copy of the other elements in the replica set). The nReplicaSets value determines the number of replicas for each StatefulSet. For example, if you set k to 2, the TimesTen Operator creates two StatefulSets for the data instances. If you set nReplicaSets to 3, each StatefulSet contains three replicas, and the total number of replica sets in the database is three.

     This example sets nReplicaSets to 3.

     For information on replica sets, see Understanding replica sets in the Oracle TimesTen In-Memory Database Scaleout User's Guide.

   • nZookeeper: Set the value to the number of ZooKeeper Pods to provision in a StatefulSet. Your options are 1 or 3 or 5.

     This example sets nZookeeper to 3.

   You also need to specify the following fields:

   • name: Replace samplescaleout with the name of your TimesTenScaleout object.

   • storageClassName: Replace oci with the name of the storage class in your Kubernetes cluster that is used to allocate Persistent Volumes to hold the TimesTen database.

   • storageSize: Replace 250Gi with the amount of storage that should be requested for each Pod to hold TimesTen. (This example assumes a production environment and uses 250Gi for storage. For demonstration purposes, you can use 50Gi of storage.) See the storageSize and the logStorageSize entries in Table 15-7.

   • image: Replace container-registry.oracle.com/timesten/timesten:22.1.1.9.0 with the location of the image registry and the name of the image. If you are using the Oracle container-registry.oracle.com/timesten registry as the image registry and the timesten:22.1.1.9.0 image as the container image, no replacement is necessary.

   • imagePullSecret: Replace sekret with the image pull secret that Kubernetes should use to fetch the TimesTen image.

   • dbConfigMap: This example uses one ConfigMap (called samplescaleout) for the db.ini, the adminUser, the schema.sql, the sampledirect.connect, and the sampleclient.csconnect metadata files. This ConfigMap is included in the Projected Volume. The volume is mounted as /ttconfig in the TimesTen containers. See Using ConfigMaps and Secrets.

   vi samplescaleout.yaml
   
   kind: TimesTenScaleout
   metadata:
     name: samplescaleout
   spec:
     ttspec:
       storageClassName: oci
       storageSize: 250Gi
       image: container-registry.oracle.com/timesten/timesten:22.1.1.9.0
       imagePullSecret: sekret
       dbConfigMap:
       - samplescaleout
       k: 2
       nReplicaSets: 3
       nZookeeper: 3
   

2. Create the TimesTenScaleout object from the contents of the YAML file. Doing so begins the process of creating and provisioning a TimesTen grid and database in your Kubernetes cluster.

   kubectl create -f samplescaleout.yaml

   The output is the following:

   timestenscaleout.timesten.oracle.com/samplescaleout created

You successfully created the TimesTenScaleout object in the Kubernetes cluster. The process of provisioning your TimesTen grid and database begins, but is not yet complete.

Monitor the High Level state of the TimesTenScaleout object

Use the kubectl get and the kubectl describe commands to monitor the progress of the grid and database creation.

Note:

For the kubectl get timestenscaleout command, you can alternatively specify kubectl get tts. When used in the kubectl get command, timestenscaleout and tts are synonymous, and return the same results. The first kubectl get examples in this chapter use timestenscaleout. For simplicity, the remaining examples in this book use tts.

1. Review the High Level state of the TimesTenScaleout object. Use the kubectl get command and observe the STATE field. Notice that the value is Initializing. The Operator has created the Kubernetes StatefulSets and Services. The process to deploy and provision your grid and database has begun, but is not yet complete. As you issue additional kubectl get commands, observe how the TimesTenScaleout object transitions to different states. For more information on these states, see About creating your grid.

   kubectl get tts samplescaleout

   The output is similar to the following:

   NAME             OVERALL        MGMT   CREATE   LOAD   OPEN   AGE
   samplescaleout   Initializing                                 20s
   

   kubectl get tts samplescaleout

   The output is similar to the following:

   NAME             OVERALL          MGMT   CREATE   LOAD   OPEN   AGE
   samplescaleout   ZookeeperReady                                 2m48s
   

   kubectl get tts samplescaleout

   The output is similar to the following:

   NAME             OVERALL       MGMT   CREATE   LOAD   OPEN   AGE
   samplescaleout   GridCreated                                 3m58s
   

   kubectl get tts samplescaleout

   The output is similar to the following:

   NAME             OVERALL            MGMT   CREATE   LOAD   OPEN   AGE
   samplescaleout   InstancesCreated                                 6m4s
   

   kubectl get tts samplescaleout

   The output is similar to the following:

   NAME             OVERALL   MGMT     CREATE    LOAD              OPEN   AGE
   samplescaleout   DatabaseCreated                                       6m10s
   

2. Use the kubectl get command again to observe if the High Level state has transitioned from DatabaseCreated to Normal. A Normal state indicates the grid and database are provisioned and the process is complete.

   kubectl get tts samplescaleout

   The output is similar to the following:

   NAME             OVERALL   MGMT     CREATE    LOAD              OPEN   AGE
   samplescaleout   Normal    Normal   created   loaded-complete   open   6m52s
   

Your TimesTen Scaleout grid and database are successfully created and provisioned in your Kubernetes cluster. The database is open for connections.

Verify the underlying objects

The TimesTen Operator creates the following objects for the samplescaleout TimesTenScaleout object.

1. StatefulSets:

   kubectl get statefulsets

   The output is the following:

   NAME                    READY   AGE
   samplescaleout-data-1   3/3     26m
   samplescaleout-data-2   3/3     26m
   samplescaleout-mgmt     1/1     26m
   samplescaleout-zk       3/3     26m

   The Operator creates the samplescaleout-data-1 and the samplescaleout-data-2 StatefulSets. Two StatefulSets are created because the value of k is set to 2. Each StatefulSet provides the Pods for a single data space. There are two data spaces in the grid. Each of the StatefulSets has three replicas (nReplicaSets is set to 3). Therefore, the number of replica sets in the grid is three. There are three data instances in data space one and three data instances in data space two.

   The samplescaleout-mgmt StatefulSet provides the Pod for the management instance. There is one management instance.

   The samplscaleout-zk StatefulSet provides the Pods for the ZooKeeper instances. There are three ZooKeeper instances (nZookeeper is set to 3).

2. Pods:

   kubectl get pods

   The output is the following:

   NAME                                 READY   STATUS    RESTARTS   AGE
   samplescaleout-data-1-0              2/2     Running   0          27m
   samplescaleout-data-1-1              2/2     Running   0          27m
   samplescaleout-data-1-2              2/2     Running   0          27m
   samplescaleout-data-2-0              2/2     Running   0          27m
   samplescaleout-data-2-1              2/2     Running   0          27m
   samplescaleout-data-2-2              2/2     Running   0          27m
   samplescaleout-mgmt-0                2/2     Running   0          27m
   samplescaleout-zk-0                  1/1     Running   0          27m
   samplescaleout-zk-1                  1/1     Running   0          26m
   samplescaleout-zk-2                  1/1     Running   0          25m
   timesten-operator-7677964df9-sp2zp   1/1     Running   0          3d20h
   

   The samplescaleout-data-1 StatefulSet contains the data instances in data space one. This StatefulSet creates the samplescaleout-data-1-0, samplescaleout-data-1-1, and samplescaleout-data-1-2 Pods. The samplescaleout-data-2 contains the data instances in data space two. This StatefulSet creates the samplescaleout-data-2-0, samplescaleout-data-2-1, and samplescaleout-data-2-2 Pods.

   The samplescaleout-mgmt-0 StatefulSet contains the management instance for the grid. This StatefulSet creates the samplescaleout-mgmt-0 Pod.

   Each of the Pods previously mentioned run TimesTen. Each Pod contains at least two containers. In this example, each Pod contains two containers. The tt container runs the TimesTen agent and TimesTen. The daemonlog container writes the TimesTen daemon log to stdout.

   The samplescaleout-zk StatefulSet contains the three ZooKeepers instances. This StatefulSet creates the samplescaleout-zk-1, samplescaleout-zk-2, and samplescaleout-zk-3 Pods. Each of these Pods contain a single container called zookeeper.

   The TimesTen Operator is running in the timesten-operator-554887b4c-48zwk Pod.

3. Headless Services:

   kubectl get services

   The output is the following:

   NAME                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                         AGE
   samplescaleout      ClusterIP   None            <none>        6625/TCP                        28m
   samplescaleout-zk   ClusterIP   None            <none>        2181/TCP,2888/TCP,3888/TCP      28m
   

   The Operator creates the samplescaleout Headless Service. This Service provides the DNS names for the Pods that contain the TimesTen management and data instances. The Service allows client/server access to the Pods that use the client/server port (6625). The DNS names are in the format: podname.samplescaleout.namespace.svc.cluster.local.

   The following example shows the DNS names for the samplescaleout Headless Service:

   samplescaleout-mgmt-0.samplescaleout.mynamespace.svc.cluster.local
   samplescaleout-data-1-0.samplescaleout.mynamespace.svc.cluster.local
   samplescaleout-data-1-1.samplescaleout.mynamespace.svc.cluster.local
   samplescaleout-data-1-2.samplescaleout.mynamespace.svc.cluster.local
   samplescaleout-data-2-0.samplescaleout.mynamespace.svc.cluster.local
   samplescaleout-data-2-1.samplescaleout.mynamespace.svc.cluster.local
   samplescaleout-data-2-2.samplescaleout.mynamespace.svc.cluster.local

   The Operator creates a second Headless Service called samplescaleout-zk. This Service allows access to the ZooKeeper internal ports (2888 and 3888) as well as the external port (2181). The DNS names are in the format: samplescaleout-zk-n.samplescaleout-zk.namespace.svc.cluster.local.

   The following example shows the DNS names for the samplescaleout-zk Headless Service:

   samplescaleout-zk-0.samplescaleout-zk.mynamespace.svc.cluster.local
   samplescaleout-zk-1.samplescaleout-zk.mynamespace.svc.cluster.local
   samplescaleout-zk-2.samplescaleout-zk.mynamespace.svc.cluster.local

4. Persistent Volume Claims (PVCs):

   kubectl get pvc

   The output is the following:

   
   NAME                                    STATUS   VOLUME                                                                              CAPACITY      ACCESS MODES   STORAGECLASS   AGE
   tt-persistent-samplescaleout-data-1-0   Bound    ocid1.volume.oc1.phx.abyhqljtl63wgaxd3nvengilkaxs5h2b23kmtinpzmmqt7bkzjdpnfu2c2fq   53687091200   RWO            oci            14m
   tt-persistent-samplescaleout-data-1-1   Bound    ocid1.volume.oc1.phx.abyhqljthxuwtpqx7rjtwvwsxjjkbgr25amjk7wtk26untbqrealcqhe324q   53687091200   RWO            oci            14m
   tt-persistent-samplescaleout-data-1-2   Bound    ocid1.volume.oc1.phx.abyhqljthtm6frcjtttp6ye7hq4w5vm3jxyay54f4xtcbolv2ercjeca5khq   53687091200   RWO            oci            14m
   tt-persistent-samplescaleout-data-2-0   Bound    ocid1.volume.oc1.phx.abyhqljtxgzm3raxj5sfoe56aonlh2mqqjre4quva4k3q3zbbe7lftwqk3xa   53687091200   RWO            oci            14m
   tt-persistent-samplescaleout-data-2-1   Bound    ocid1.volume.oc1.phx.abyhqljtk3htdair4akll5dfrwpkipv3acjtww5hx3x2fz46af7zdew7gsiq   53687091200   RWO            oci            14m
   tt-persistent-samplescaleout-data-2-2   Bound    ocid1.volume.oc1.phx.abyhqljtiekgulpbadtwigehsuml75sngcuqjgqmx77lfpi3wdeecbecavfa   53687091200   RWO            oci            14m
   tt-persistent-samplescaleout-mgmt-0     Bound    ocid1.volume.oc1.phx.abyhqljtacxwop7r2wqx6hsvun4haaydnco3y6g3rkgbwp5hq35alay7uwaq   53687091200   RWO            oci            14m
   tt-persistent-samplescaleout-zk-0       Bound    ocid1.volume.oc1.phx.abyhqljtyoa5hdchax4sus652jtp665ckaef2cq3lakac2lq52vfbls6kkcq   53687091200   RWO            oci            14m
   tt-persistent-samplescaleout-zk-1       Bound    ocid1.volume.oc1.phx.abyhqljtongpcoggzpg2is25vmumijmah5gustwc3avgnijrjigtqphtrana   53687091200   RWO            oci            13m
   tt-persistent-samplescaleout-zk-2       Bound    ocid1.volume.oc1.phx.abyhqljttmgoljskb2ruawzv365uit7lsln2sbfno5e4vhh6plbgh4tiblfq   53687091200   RWO            oci            10m
   

   The Operator automatically creates one or two Persistent Volume Claims (PVCs) per Pod. These PVCs cause Persistent Volumes (PVs) to be allocated by Kubernetes and attached to the TimesTen Pods. TimesTen uses these PVs to hold the TimesTen instance and database. If you specify two PVCs, one PV holds the TimesTen instance and the checkpoint files and the second PV holds the TimesTen transaction logs. In this example, the Operator creates one PVC for each Pod for a total of six PVCs. Each of the six PVs hold the TimesTen instance and an element of the database.

   The Operator creates one PVC for the Pod that contains the management instance.

   The Operator creates one PVC for each Pod that runs a ZooKeeper instance. This PVC causes a PV to be allocated by Kubernetes and attached to the ZooKeeper Pod. Each PV holds ZooKeeper's persistent data. In this example, since there are three ZooKeeper instances, the Operator created three PVCs.

Connect to the database

You can establish a shell in a TimesTen Pod and connect to the TimesTen database in the grid. You can then run operations in this TimesTen database.

1. Establish a shell in the TimesTen samplescaleout-data-1-0 Pod.

   kubectl exec -it samplescaleout-data-1-0 -c tt -- /bin/bash

2. Connect to the samplescaleout database. Verify the information from the metadata files is in the database. This example connects to the database as the sampleuser user. (This user was created in the schema.sql file). The example then calls the ttConfiguration built-in procedure to check that the PermSize connection attribute has a value of 200 for this element of the database. The example then verifies the sampleuser.emp table exists.

   ttisql -connstr "DSN=samplescaleout;uid=sampleuser;pwd=sampleuserpwd";

   The output is similar to the following:

   Copyright (c) 1996, 2023, Oracle and/or its affiliates. All rights reserved.
   Type ? or "help" for help, type "exit" to quit ttIsql.
   
   
   
   connect "DSN=samplescaleout;uid=sampleuser;pwd=********";
   Connection successful: DSN=samplescaleout;Durability=0;UID=sampleuser;DataStore=/tt/home/timesten/datastore/samplescaleout;
   DatabaseCharacterSet=AL32UTF8;PermSize=200;
   Connections=100;
   (Default setting AutoCommit=1)
   Command> call ttConfiguration ('PermSize');
   < PermSize, 200 >
   1 row found.
   Command> tables;
     SAMPLEUSER.EMP
   1 table found.
   

Manage the database

You can establish a shell in the TimesTen Pod of the management instance. You can then use the ttGridAdmin utility to manage and monitor the grid, including the health of the TimesTen database and its elements.

1. Establish a shell in the TimesTen samplescaleout-management-0 Pod.

   kubectl exec -it samplescaleout-mgmt-0 -c tt -- /bin/bash

2. Run the ttGridAdmin dbStatus -all to check the status of the TimesTen database.

   ttGridAdmin dbStatus -all

   The output is the following:

   Database samplescaleout summary status as of Sun Jan 15 17:00:11 UTC 2023
   
   created,loaded-complete,open
   Completely created elements: 6 (of 6)
   Completely loaded elements: 6 (of 6)
   Completely created replica sets: 3 (of 3)
   Completely loaded replica sets: 3 (of 3)
   
   Open elements: 6 (of 6)
   
   Database samplescaleout element level status as of Sun Jan 15 17:00:11 UTC 2023
   
   Host                    Instance  Elem Status Cache Agent Date/Time of Event  Message
   ----------------------- --------- ---- ------ ----------- ------------------- -------
   samplescaleout-data-1-0 instance1    1 opened stopped     2023-01-15 16:28:57
   samplescaleout-data-1-1 instance1    2 opened stopped     2023-01-15 16:28:57
   samplescaleout-data-1-2 instance1    3 opened stopped     2023-01-15 16:28:56
   samplescaleout-data-2-0 instance1    4 opened stopped     2023-01-15 16:28:57
   samplescaleout-data-2-1 instance1    5 opened stopped     2023-01-15 16:28:56
   samplescaleout-data-2-2 instance1    6 opened stopped     2023-01-15 16:28:56
   
   Database samplescaleout Replica Set status as of Sun Jan 15 17:00:11 UTC 2023
   
   RS DS Elem Host                    Instance  Status Cache Agent Date/Time of Event  Message
   -- -- ---- ----------------------- --------- ------ ----------- ------------------- -------
    1  1    1 samplescaleout-data-1-0 instance1 opened stopped     2023-01-15 16:28:57       
    1  2    4 samplescaleout-data-2-0 instance1 opened stopped     2023-01-15 16:28:57       
    2  1    2 samplescaleout-data-1-1 instance1 opened stopped     2023-01-15 16:28:57       
    2  2    5 samplescaleout-data-2-1 instance1 opened stopped     2023-01-15 16:28:56       
    3  1    3 samplescaleout-data-1-2 instance1 opened stopped     2023-01-15 16:28:56       
    3  2    6 samplescaleout-data-2-2 instance1 opened stopped     2023-01-15 16:28:56       
   
   Database samplescaleout Data Space Group status as of Sun Jan 15 17:00:11 UTC 2023
   
   DS RS Elem Host                    Instance  Status Cache Agent Date/Time of Event  Message
   -- -- ---- ----------------------- --------- ------ ----------- ------------------- -------
    1  1    1 samplescaleout-data-1-0 instance1 opened stopped     2023-01-15 16:28:57       
    1  2    2 samplescaleout-data-1-1 instance1 opened stopped     2023-01-15 16:28:57       
    1  3    3 samplescaleout-data-1-2 instance1 opened stopped     2023-01-15 16:28:56       
    2  1    4 samplescaleout-data-2-0 instance1 opened stopped     2023-01-15 16:28:57       
    2  2    5 samplescaleout-data-2-1 instance1 opened stopped     2023-01-15 16:28:56       
    2  3    6 samplescaleout-data-2-2 instance1 opened stopped     2023-01-15 16:28:56