Monitor Progress of an Active Standby Pair Deployment

You can use various kubectl commands to monitor the progress of the active standby pair deployment. After the deployment is complete and successful, you can connect to the database and run operations in it to verify it is working as it should.

Monitor the State of a TimesTenClassic Object

Use the kubectl get and the kubectl describe commands to monitor the progress of the active standby pair as it is provisioned.

Note:

For the kubectl get timestenclassic and kubectl describe timestenclassic commands, you can alternatively specify kubectl get ttc and kubectl describe ttc respectively. timestenclassic and ttc are synonymous when used in these commands, and return the same results. The first kubectl get and the first kubectl describe examples in this chapter use timestenclassic. The remaining examples in this book use ttc for simplicity.

Use the kubectl get command and review the STATE field. Observe the value is Initializing. The active standby pair provisioning has begun, but is not yet complete.
```
% kubectl get timestenclassic sample
NAME     STATE          ACTIVE   AGE
sample   Initializing   None     11s
```

Use the kubectl describe command to view the initial provisioning in detail.

% kubectl describe timestenclassic sample
Name:         sample
Namespace:    mynamespace
Labels:       <none>
Annotations:  <none>
API Version:  timesten.oracle.com/v1
Kind:         TimesTenClassic
Metadata:
  Creation Timestamp:  2023-04-30T15:35:12Z
  Generation:          1
  Resource Version:    20231755
  Self Link: 
/apis/timesten.oracle.com/v1/namespaces/mynamespace/timestenclassics/sample
  UID:                 517a8646-a354-11ea-a9fb-0a580aed5e4a
Spec:
  Ttspec:
    Db Config Map:
      sample
    Image:               container-registry.oracle.com/timesten/timesten:22.1.1.19.0
    Image Pull Policy:   Always
    Image Pull Secret:   sekret
    Storage Class Name:  oci-bv
    Storage Size:        250Gi
Status:
  Active Pods:       None
  High Level State:  Initializing
  Last Event:        3
  Pod Status:
    Cache Status:
      Cache Agent:        Down
      Cache UID Pwd Set:  false
      N Cache Groups:     0
    Db Status:
      Db:            Unknown
      Db Id:         0
      Db Updatable:  Unknown
    Initialized:     true
    Pod Status:
      Agent:                Down
      Last Time Reachable:  0
      Pod IP:
      Pod Phase:            Pending
    Replication Status:
      Last Time Rep State Changed:  0
      Rep Agent:                    Down
      Rep Peer P State:             Unknown
      Rep Scheme:                   Unknown
      Rep State:                    Unknown
    Times Ten Status:
      Daemon:          Down
      Instance:        Unknown
      Release:         Unknown
    Admin User File:   false
    Cache User File:   false
    Cg File:           false
    High Level State:  Down
    Intended State:    Active
    Name:              sample-0
    Schema File:       false
    Cache Status:
      Cache Agent:        Down
      Cache UID Pwd Set:  false
      N Cache Groups:     0
    Db Status:
      Db:            Unknown
      Db Id:         0
      Db Updatable:  Unknown
    Initialized:     true
    Pod Status:
      Agent:                Down
      Last Time Reachable:  0
      Pod IP:
      Pod Phase:            Pending
    Replication Status:
      Last Time Rep State Changed:  0
      Rep Agent:                    Down
      Rep Peer P State:             Unknown
      Rep Scheme:                   Unknown
      Rep State:                    Unknown
    Times Ten Status:
      Daemon:            Down
      Instance:          Unknown
      Release:           Unknown
    Admin User File:     false
    Cache User File:     false
    Cg File:             false
    High Level State:    Unknown
    Intended State:      Standby
    Name:                sample-1
    Schema File:         false
  Rep Create Statement:  create active standby pair "sample" on 
 "sample-0.sample.mynamespace.svc.cluster.local", "sample" on 
 "sample-1.sample.mynamespace.svc.cluster.local" NO RETURN store "sample" on 
 "sample-0.sample.mynamespace.svc.cluster.local" PORT 4444 FAILTHRESHOLD 0 
 store "sample" on "sample-1.sample.mynamespace.svc.cluster.local" PORT 4444 FAILTHRESHOLD  0
  Rep Port:              4444
  Status Version:        1.0
Events:
  Type  Reason  Age   From       Message
  ----  ------  ----  ----       -------
  -     Create  50s   ttclassic  Secret tt517a8646-a354-11ea-a9fb-0a580aed5e4a created
  -     Create  50s   ttclassic  Service sample created
  -     Create  50s   ttclassic  StatefulSet sample created

Use the kubectl get command again to see if value of the STATE field has changed. In this example, the value is Normal, indicating the active standby pair of databases are now provisioned and the process is complete.
```
% kubectl get ttc sample
NAME     STATE    ACTIVE     AGE
sample   Normal   sample-0   3m5s
```

Use the kubectl describe command again to view the active standby pair provisioning in detail.

Note: In this example, the now Normal line displays on its own line. In the actual output, this line does not display as its own line, but at the end of the StateChange previous line.

% kubectl describe ttc sample
Name:         sample
Namespace:    mynamespace
Labels:       <none>
Annotations:  <none>
API Version:  timesten.oracle.com/v1
Kind:         TimesTenClassic
Metadata:
  Creation Timestamp:  2023-04-30T15:35:12Z
  Generation:          1
  Resource Version:    20232668
  Self Link:
/apis/timesten.oracle.com/v1/namespaces/mynamespace/timestenclassics/sample 
  UID:                 517a8646-a354-11ea-a9fb-0a580aed5e4a
Spec:
  Ttspec:
    Db Config Map:
      sample
    Image:               container-registry.oracle.com/timesten/timesten:22.1.1.19.0
    Image Pull Policy:   Always
    Image Pull Secret:   sekret
    Storage Class Name:  oci-bv
    Storage Size:        250Gi
Status:
  Active Pods:       sample-0
  High Level State:  Normal
  Last Event:        35
  Pod Status:
    Cache Status:
      Cache Agent:        Not Running
      Cache UID Pwd Set:  true
      N Cache Groups:     0
    Db Status:
      Db:            Loaded
      Db Id:         26
      Db Updatable:  Yes
    Initialized:     true
    Pod Status:
      Agent:                Up
      Last Time Reachable:  1590939597
      Pod IP:               192.0.2.1
      Pod Phase:            Running
    Replication Status:
      Last Time Rep State Changed:  0
      Rep Agent:                    Running
      Rep Peer P State:             start
      Rep Scheme:                   Exists
      Rep State:                    ACTIVE
    Times Ten Status:
      Daemon:          Up
      Instance:        Exists
      Release:         22.1.1.19.0
    Admin User File:   true
    Cache User File:   false
    Cg File:           false
    High Level State:  Healthy
    Intended State:    Active
    Name:              sample-0
    Schema File:       true
    Cache Status:
      Cache Agent:        Not Running
      Cache UID Pwd Set:  true
      N Cache Groups:     0
    Db Status:
      Db:            Loaded
      Db Id:         26
      Db Updatable:  No
    Initialized:     true
    Pod Status:
      Agent:                Up
      Last Time Reachable:  1590939597
      Pod IP:               192.0.2.2
      Pod Phase:            Running
    Replication Status:
      Last Time Rep State Changed:  1590939496
      Rep Agent:                    Running
      Rep Peer P State:             start
      Rep Scheme:                   Exists
      Rep State:                    STANDBY
    Times Ten Status:
      Daemon:            Up
      Instance:          Exists
      Release:           22.1.1.19.0
    Admin User File:     true
    Cache User File:     false
    Cg File:             false
    High Level State:    Healthy
    Intended State:      Standby
    Name:                sample-1
    Schema File:         true
  Rep Create Statement:  create active standby pair "sample" on 
"sample-0.sample.mynamespace.svc.cluster.local", "sample" on 
"sample-1.sample.mynamespace.svc.cluster.local" NO RETURN store "sample" on 
"sample-0.sample.mynamespace.svc.cluster.local" PORT 4444 FAILTHRESHOLD 0 
store "sample" on "sample-1.sample.mynamespace.svc.cluster.local" PORT 4444 FAILTHRESHOLD 0
  Rep Port:              4444
  Status Version:        1.0
Events:
  Type  Reason       Age    From       Message
  ----  ------       ----   ----       -------
  -     Create       4m43s  ttclassic  Secret tt517a8646-a354-11ea-a9fb-0a580aed5e4a created
  -     Create       4m43s  ttclassic  Service sample created
  -     Create       4m43s  ttclassic  StatefulSet sample created
  -     StateChange  3m47s  ttclassic  Pod sample-0 Daemon Unknown
  -     StateChange  3m47s  ttclassic  Pod sample-0 CacheAgent Unknown
  -     StateChange  3m47s  ttclassic  Pod sample-0 RepAgent Unknown
  -     StateChange  3m47s  ttclassic  Pod sample-1 Daemon Unknown
  -     StateChange  3m47s  ttclassic  Pod sample-1 CacheAgent Unknown
  -     StateChange  3m47s  ttclassic  Pod sample-1 RepAgent Unknown
  -     StateChange  3m26s  ttclassic  Pod sample-0 Agent Up
  -     StateChange  3m26s  ttclassic  Pod sample-0 Release 22.1.1.19.0
  -     StateChange  3m26s  ttclassic  Pod sample-0 Daemon Down
  -     StateChange  3m26s  ttclassic  Pod sample-1 Agent Up
  -     StateChange  3m26s  ttclassic  Pod sample-1 Release 22.1.1.19.0
  -     StateChange  3m26s  ttclassic  Pod sample-1 Daemon Down
  -     StateChange  3m26s  ttclassic  Pod sample-0 Daemon Up
  -     StateChange  3m25s  ttclassic  Pod sample-1 Daemon Up
  -     StateChange  2m13s  ttclassic  Pod sample-0 RepState IDLE
  -     StateChange  2m13s  ttclassic  Pod sample-0 Database Updatable
  -     StateChange  2m13s  ttclassic  Pod sample-0 CacheAgent Not Running
  -     StateChange  2m13s  ttclassic  Pod sample-0 RepAgent Not Running
  -     StateChange  2m13s  ttclassic  Pod sample-0 RepScheme None
  -     StateChange  2m13s  ttclassic  Pod sample-0 Database Loaded
  -     StateChange  2m11s  ttclassic  Pod sample-0 RepAgent Running
  -     StateChange  2m10s  ttclassic  Pod sample-0 RepScheme Exists
  -     StateChange  2m10s  ttclassic  Pod sample-0 RepState ACTIVE
  -     StateChange  113s   ttclassic  Pod sample-1 Database Loaded
  -     StateChange  113s   ttclassic  Pod sample-1 Database Not Updatable
  -     StateChange  113s   ttclassic  Pod sample-1 CacheAgent Not Running
  -     StateChange  113s   ttclassic  Pod sample-1 RepAgent Not Running
  -     StateChange  113s   ttclassic  Pod sample-1 RepScheme Exists
  -     StateChange  113s   ttclassic  Pod sample-1 RepState IDLE
  -     StateChange  106s   ttclassic  Pod sample-1 RepAgent Running
  -     StateChange  101s   ttclassic  Pod sample-1 RepState STANDBY
  -     StateChange  101s   ttclassic  TimesTenClassic was Initializing, now Normal

Your active standby pair of TimesTen databases are successfully deployed (as indicated by Normal.) There are two TimesTen databases, configured as an active standby pair. One database is active. (In this example, sample-0 is the active database, as indicated by Rep State ACTIVE). The other database is standby. (In this example, sample-1 is the standby database as indicated by Rep State STANDBY). The active database can be modified and queried. Changes made on the active database are replicated to the standby database. If the active database fails, the Operator automatically promotes the standby database to be the active. The formerly active database will be repaired or replaced, and will then become the standby.

Verify Underlying Objects Exist

The Operator creates other underlying objects automatically. Verify that these objects are created.

StatefulSet:

% kubectl get statefulset sample
NAME     READY   AGE
sample   2/2     8m21s

Service:

% kubectl get service sample
NAME     TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
sample   ClusterIP   None         <none>        6625/TCP   9m28s

Pods:

% kubectl get pods
NAME                                        READY   STATUS    RESTARTS   AGE
sample-0                                    2/2     Running   0          10m
sample-1                                    2/2     Running   0          10m
timesten-operator-5d7dcc7948-8mnz4          1/1     Running   0          11h

PersistentVolumeClaims (PVCs):

% kubectl get pvc
NAME                         STATUS   VOLUME 
CAPACITY   ACCESS MODES   STORAGECLASS   AGE
tt-persistent-sample-0       Bound
ocid1.volume.oc1.phx.abyhqljrbxcgzyixa4pmmcwiqxgqclc7gxvdnoty367w2qn26tij6kfpx
6qq
250Gi       RWO            oci-bv         10m
tt-persistent-sample-1       Bound
ocid1.volume.oc1.phx.abyhqljtt4qxxoj5jqiskriskh66hakaw326rbza4uigmuaezdnu53qhh
oaa
250Gi       RWO            oci-bv         10m

Verify Connection to Active Database

You can run the kubectl exec command to invoke shells in your Pods and control TimesTen, which is running in those Pods. By default, TimesTen runs in the Pods as the timesten user. Once you have established a shell in the Pod, verify you can connect to the sample database, and that the information from the metadata files is correct. You can optionally run queries against the database or any other operations.

Establish a shell in the Pod.

% kubectl exec -it sample-0 -c tt -- /bin/bash

Connect to the sample database. Verify the information in the metadata files is in the database correctly. For example, attempt to connect to the database as the sampleuser user. Check that the PermSize value of 200 is correct. Check that the sampleuser.emp table exists.

 % ttIsql sample
 
Copyright (c) 1996, 2023, Oracle and/or its affiliates. All rights reserved.
Type ? or "help" for help, type "exit" to quit ttIsql.
 
 
 
connect "DSN=sample";
Connection successful:
DSN=sample;UID=timesten;DataStore=/tt/home/timesten/datastore/sample;
DatabaseCharacterSet=AL32UTF8;ConnectionCharacterSet=AL32UTF8;PermSize=200;
DDLReplicationLevel=3;
(Default setting AutoCommit=1)

Command> connect adding "uid=sampleuser;pwd=samplepw" as sampleuser;
Connection successful:
DSN=sample;UID=sampleuser;DataStore=/tt/home/timesten/datastore/sample;
DatabaseCharacterSet=AL32UTF8;ConnectionCharacterSet=AL32UTF8;PermSize=200;
DDLReplicationLevel=3;
(Default setting AutoCommit=1)
sampleuser: Command> tables;
  SAMPLEUSER.EMP
1 table found.