Additional Configuration Options

This section discusses additional configuration options. These are optional configurations for your environment:

Persistent Storage

When you create a TimesTenClassic object, the Operator automatically creates one or two Persistent Volume Claims (PVCs) per Pod. These PVCs cause Persistent Volumes (PVs) to be allocated by Kubernetes and to be attached to the TimesTen Pods. TimesTen uses the PVs to hold the TimesTen instance and the TimesTen database. If you specify two PVCs, one PV holds the instance and the checkpoint files and the second PV holds the transaction log files.

When you create a TimesTenScaleout object, the same mechanism is used to provision persistent storage for the data instances, the management instance, and the ZooKeeper instances:

For the data instances: The Operator creates one or two PVCs per Pod. TimesTen uses the PVs to hold the TimesTen instance and the TimesTen database. If you specify two PVCs, one PV holds the instance and the checkpoint files and the second PV holds the transaction log files.
For the management instance: The Operator creates one PVC for the Pod that contains the management instance. The PV holds the the TimesTen management instance and the grid database.
For the ZooKeeper instances: The Operator creates one PVC for each Pod that runs a ZooKeeper instance. The PV holds ZooKeeper's persistent data.

When you create a TimesTenClassic object, you must specify storageClassName and you may specify storageSize. These attributes determine the characteristics of the Persistent Volumes. The storageClassName must be one that is provided in your Kubernetes environment. For example, in Oracle Kubernetes Environment (OKE), you may use oci-bv.

The default storage is 50Gi. Use the storageSize attribute to request a different size. A storage size of 50Gi may be adequate for demonstration purposes, but in production environments, you should consider greater storage.

TimesTen places the TimesTen installation, the instance, and the database in this storage. It is mounted in each container, in each Pod, as /tt. The TimesTen instance is located at /tt/home/timesten/instances/instance1.

When you create a TimesTenScaleout object, the following attributes are supported:

Storage class name:
- dataStorageClassName: Name of the storage class that is used to request persistent volumes for the elements of the TimesTen database in the grid. If not specified, the default is the value of storageClassName.
- mgmtStorageClassName: Name of the storage class that is used to request persistent volumes for the database of the management instance. If not specified, the default is the value of storageClassName.
- zookeeperStorageClassName: Name of the storage class that is used to request persistent volumes for ZooKeeper's persistent data. If not specified, the default is the value of storageClassName.
- storageClassName: If the data storage class name, the management storage class name, and the zookeeper storage class name are the same, you can just specify storageClassName.
Storage size:
- dataStorageSize: Amount of storage to be provisioned for each element of the TimesTen database in the grid. The default is 50Gi.
- mgmtStorageSize: Amount of storage to be provisioned for the database of the management instance. The default is 50Gi.
- zookeeperStorageSize: Amount of storage to be provisioned for ZooKeeper's persistent data. The default is 50Gi.
- storageSize: If the data storage size, the management storage size, and the zookeeper storage size are the same, you can just specify storageSize. For example, if the dataStorageSize is 75Gi, and the mgmtStorageSize is 75Gi, and the zookeeperStorageSize is 75Gi, you can specify storageSize with a value of 75Gi. The value for dataStorageSize, for mgmtStorageSize, and for zookeeperStorageSize is set to the value of storageSize.

For the TimesTen databases (using TimesTen Classic) and for the TimesTen database (using TimesTen Scaleout):

TimesTen best practices recommends that the transaction log files associated with a TimesTen database be located on a different storage volume than the checkpoint files for the database. This provides separate paths to storage for the checkpoint and the transaction log operations. For example, you can store the transaction log files in a high performance storage, while storing the checkpoint files in a slower storage. See Locate Checkpoint and Transaction Log Files on Separate Physical Device in the Oracle TimesTen In-Memory Database Operations Guide for more information.
To locate the checkpoint files and the transaction log files on a separate path of storage, provide a value for a second persistent storage that is used only for the transaction log files. Use the logStorageSize attribute for this and control its placement by using the logStorageClassName attribute. This causes a second PVC to be created for each Pod, which will then be available in each container at /ttlog. (This second storage volume has a /ttlog mount point.)

Here is an example for a TimesTenClassic object. The same example can be used for a TimesTenScaleout object:
```
apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
  name: sample
spec:
  ttspec:
    storageClassName: slower
    storageSize: 750Gi
    logStorageClassName: faster
    logStorageSize: 200G
```

Additional Resource Specifications

Kubernetes supports affinity and anti-affinity settings that let applications control their placement within the Kubernetes cluster. These settings can be used to ensure all replicas do not reside on a single physical host.

You can specify affinity settings, node selectors, additional containers, tolerations, resource requirements, and other Kubernetes attributes for the TimesTen Pods and the containers within these Pods that are created by the TimesTen Operator.

In a TimesTenClassic deployment, you specify these resource specifications in the TimesTenClassic object's .spec.template datum. The TimesTen Operator passes this template to the StatefulSet. For example, when you deploy a TimesTenClassic object, the Operator configures a replicated pair of TimesTen databases that provide high availability. However, since the Operator does not control the placement of Pods, you can achieve an even greater level of high availability by controlling the placement of the TimesTen Pods. TimesTen Pods can then be available in different availability zones or are on different Kubernetes nodes. To do this, you specify the affinity option in the .spec.template datum for the TimesTenClassic object.

Similar to a TimesTenClassic deployment, you can specify the same resource specifications for a TimesTenScaleout object. The TimesTenScaleout object supports the .spec.mgmtTemplate, .spec.dataTemplate and .spec.zookeeperTemplate attributes. You can use these attributes to pass affinity and other settings to Kubernetes. These are of type PodTemplateSpec:

mgmtTemplate: Applied to the Pod that contains the TimesTen Scaleout management instance. Consists of a single PodTemplateSpec.
dataTemplate: Applied to the Pods that contain the TimesTen Scaleout data instances. Consists of an array of PodTemplateSpec. If specified, there must be one entry in the array for each data space in the grid (k entries in the array). This lets you specify a different placement for each data space. For example, you can have data space one reside in availability zone 1 and data space two reside in availability zone 2.
zookeeperTemplate: Applied to the Pods that contain the ZooKeeper instances used by Scaleout. Consists of a single PodTemplateSpec.

For information about PodTemplateSpec see, https://kubernetes.io/docs/reference/kubernetes-api/

Here is an example of specifying the affinity setting for a TimesTenClassic object.

apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
  name: sample
spec:
  …
  template:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            podAffinityTerm:
             labelSelector:
              matchExpressions: 
               - key: "app"
                 operator: In
                 values:
                  - ds1
             topologyKey: "kubernetes.io/hostname"