About Monitoring the Health of an Active Standby Pair of Databases
The Operator monitors and manages the health of each of your active standby pairs. The Operator assigns high level states to the TimesTenClassic object, which you can monitor and review. For example, you can use the kubectl get command to return the high level state of your TimesTenClassic object. Specifically, in this example, the value returned for the STATE field is Normal, indicating that the active and the standby databases are up and running, and working as they should.
% kubectl get ttc sample NAME STATE ACTIVE AGE sample Normal sample-0 15h
The states:
ActiveDown
If the Operator detects that TimesTen in the Pod containing the active database has failed, then the TimesTenClassic object immediately enters the ActiveDown state.
The unreachableTimeout timeout value controls how long the state of the Pod containing the active database can be Unknown before the TimesTenClassic object's state becomes ActiveDown.
When the TimesTenClassic object's state becomes ActiveDown, the standby database immediately becomes the active, and the state of the TimesTenClassic object becomes StandbyDown.
ActiveTakeover
When the TimesTenClassic object is in the Normal state, and the standby database goes down, the state briefly changes to ActiveTakeover.
When AWT cache groups are used, the standby is normally responsible for pushing updates from TimesTen to Oracle Database. However, if the standby fails, the active database takes over this responsibility. This occurs during the ActiveTakeover state.
BothDown
Neither the active nor the standby database is functioning properly. The Operator attempts to bring up the pair of databases.
If both Pods in the active standby pair fail, the Operator uses the information in TimesTenClassicStatus to minimize data loss. See "About the BothDown State" for details.
ConfiguringActive
When the TimesTenClassic object is in the WaitingForActive state, and when the database that should be the active database comes up, the TimesTenClassic object enters the ConfiguringActive state. The Operator then configures this database to be the active. Once the database is configured as the active, the TimesTenClassic object enters the StandbyDown state. See "About the BothDown State" for details.
Failed
If a problem occurs while Initializing a TimesTenClassic object, the object transitions to the Failed state. Once in this state, the Operator does not attempt to repair the object. You must delete it. Use the kubectl describe command to examine the Operator logs to determine the cause of the problem and then recreate the object.
Initializing
This state is reported while the two Pods are starting up for the first time. In your active standby pair configuration, the Pod whose name ends with -0 is initially configured as the active database, and the Pod whose name ends with -1 is initially configured as the standby database. Specifically, if you specified the name for TimesTenClassic as sample, the sample-0 Pod is configured as the active database, and the sample-1 Pod is configured as the standby database. Once the active/standby pair is completely deployed, the TimesTenClassic object transitions to the Normal state.
ManualInterventionRequired
When a TimesTenClassic object enters the ManualInterventionRequired state, the Operator takes no further action for the object. It does not query the TimesTen agents associated with the object to determine the state of TimesTen and it does not command TimesTen to do anything. See "About the ManualInterventionRequired State" and "About Bringing Up One Database" for details.
Reexamine
When the TimesTenClassic object is in the ManualInterventionRequired state, you can specify the .spec.ttspec.reexamine datum to cause the Operator to take over the management of the object again. The Operator moves the object to the Reexamine state. The Operator then examines the state of TimesTen. If you correctly repaired TimesTen, the TimesTenClassic object may then enter the Normal or the StandbyDown state, depending on the nature of your repair. If you did not correctly repair TimesTen, the TimesTenClassic object re-enters the ManualInterventionRequired state. See "About the ManualInterventionRequired State" for details.
StandbyCatchup
This state is entered after the StandbyStarting state. During the StandbyStarting state, the standby copies the active database to the standby Pod. When the duplicate process is complete, the state changes from StandbyStarting to StandbyCatchup. See "StandbyStarting" for more information on the StandbyStarting state. In the StandbyCatchup state, the duplicate process has completed. Transactions that ran during this duplicate process must now be copied over to the standby. Thus the StandbyCatchup state is the state when the newly created standby catches up to any transactions that ran on the active while the duplicate operation was running. Applications can continue to use the active without restriction.
StandbyDown
The active database is functioning properly, but the standby database is not. The Operator automatically attempts to restart and reconfigure the standby database. Applications can continue to use the active database without restriction.
StandbyStarting
The standby is duplicating the database from the active. The StandbyStarting state is complete when the duplicate operation completes. The StandbyCatchup state is then entered. See "StandbyCatchup" for more information on the StandbyCatchup state. Applications can continue to use the active without restriction.
WaitingForActive
When the TimesTenClassic object is in the BothDown state, if the Operator can determine which database contains the most up-to-date data, the TimesTenClassic object enters the WaitingForActive state. The object remains in this state until the Pod that contains the database is running, and the TimesTen agent within the tt container (within that Pod) is responding to the Operator. See "About the BothDown State" for details.