About Monitoring the Health of Each Pod in an Active Standby Pair
The Operator keeps track of the individual health and state of each Pod in the active standby pair. How often the Operator checks the health is defined by the value of the pollingInterval. See "TimesTenClassicSpecSpec" for information on pollingInterval.
Each Pod is assigned a high level state based on the state of various components of Kubernetes and the state of TimesTen. These states are:
CatchingUp
The standby has completed the process of duplicating the database from the active. The newly created standby is catching up to any transactions that ran on the active while the duplicate operation was running.
Down
Either the Pod or the TimesTen components within the Pod (or both) are not functioning properly, given this Pod's role in the active standby pair.
Healthy
The Pod and the TimesTen components within the Pod are in a healthy state, given this Pod's role in the active standby pair.
HealthyActive
When a TimesTenClassic object is in the Reexamine state, the Operator examines the state of both TimesTen instances. The Operator does not know which instance (if any) contains a properly configured active database (or a properly configured standby database). The Operator must examine both instances to see. If a healthy instance is found and that instance contains a properly configured active database, the state of the Pod is reported as HealthyActive.
HealthyStandby
When a TimesTenClassic object is in the Reexamine state, the Operator examines the state of both TimesTen instances. The Operator does not know which instance (if any) contains a properly configured standby database (or a properly configured active database). The Operator must examine both instances to see. If a healthy instance is found and that instance contains a properly configured standby database, the state of the Pod is reported as HealthyStandby.
OtherDown
The Pod and the TimesTen components within the Pod are in a healthy state, but TimesTen in this Pod believes that TimesTen in the other Pod has failed. In particular, the OtherDown state indicates that this Pod contains an active database, and the database's peer has reached the failThreshold. The database in this Pod is no longer keeping transaction logs for its peer, as the peer is too far behind. Recovering the peer requires re-duplicating the active database (which the Operator will perform automatically).
Unknown
The state of this Pod is unknown. Either the Pod is unreachable or the TimesTen agent contained within the Pod has failed.
UpgradeFailed
An automated upgrade was attempted on TimesTen in this Pod and the upgrade failed. See About Upgrading TimesTen Classic.