This chapter describes the process for upgrading to a new patch (or patchset) of the Operator and of TimesTen. It also applies to downgrading. See "Overview of release numbers" in the Oracle TimesTen In-Memory Database Installation, Migration, and Upgrade Guide for information on TimesTen releases.
Topics include:
The upgrade process involves first upgrading the Operator and then upgrading TimesTen.
You must upgrade the Operator manually. This involves:
Updating the crd.yaml and the service_account.yaml files.
Building a new image that contains the new Operator.
Switching to the new Operator
See "Upgrading the Operator" for information on upgrading the Operator.
To upgrade TimesTen, you create a new container image that contains the new TimesTen release. You then update the TimesTenClassic objects associated with the active standby pairs by modifying the value of the TimesTenClassic objects' image CRD syntax element. See "TimesTenClassicSpecSpec" for more information on the image CRD syntax element.
The Operator creates and associates a Kubernetes StatefulSet with each TimesTenClassic object. The StatefulSet causes and controls the creation of the Pods that run TimesTen.
When you modify the image CRD syntax element of a TimesTenClassic object, the Operator notices the change and modifies the corresponding image attribute(s) in the StatefulSet. This controls the future spawning of containers. If a running container terminates, then StatefulSet spawns one to replace it. This replaced container runs the newly specified image.
Similarly, for other containers you provided that are running in these Pods, the Operator examines the value of the each TimesTenClassic object's image value. This value controls which image your provided containers run. If you modify the image value in the TimesTenClassic object, the corresponding attributes in the associated StatefulSet are modified by the Operator.
After the Operator propagates changes from the TimesTenClassic object to the associated StatefulSet, you must then determine if the Operator will perform the steps to upgrade TimesTen or if you will manually perform the steps. This is determined by the value you specified for the TimesTenClassic object's imageUpgradeStrategy CRD syntax element. A value of auto indicates the Operator should perform the upgrade whereas a value of manual indicates you will manually perform the upgrade. If you do not specify a value for imageUpgradeStrategy, the default is auto. See "TimesTenClassicSpecSpec" for more information on the imageUpgradeStrategy CRD syntax element. For automated upgrades, (value of imageUpgradeStrategy is auto), you can also control the amount of time the Operator waits for a Pod to come up (after it has been deleted) by setting the upgradeDownPodTimeout value. See "TimesTenClassicSpecSpec" for more information.
You cannot use this automated upgrade process to upgrade direct mode applications that are running in their own containers. The Operator does propagate the changes from a TimesTenClassic object to the associated StatefulSet, but the changes do not initiate the automated upgrade process. You must manually terminate the applications that are running in their containers. In so doing, StatefulSet then spawns new containers to replace the original containers. These new containers run the newly specified TimesTen image. See "Using direct mode applications" for more information on direct mode applications.
If there are failures in any steps of the upgrade process, the TimesTenClassic object enters the ManualInterventionRequired state. The remaining steps of the upgrade process are cancelled. You must manually fix the active/standby pair to return the pair to management by the Operator. Even when the pair is returned to automatic management, the remaining steps in the upgrade process are not automatically performed. See "Understanding the ManualInterventionRequired state" and "Bringing up one database" for more information.
You must manually upgrade the Operator for both automated and manual upgrades. You can upgrade the current release of the TimesTen Operator to a new release while there are one or more TimesTenClassic objects running in your Kubernetes cluster and while the TimesTen databases that are associated with those TimesTenClassic objects are up and running.
The process of upgrading the Operator involves:
Replacing the crd.yaml and the service_account.yaml files in the /deploy directory
Building the new Operator image
Updating the timestenclassic-operator Deployment
The Operator restarts, and the upgrade to the new Operator becomes effective. During the Operator upgrade, the TimesTen databases continue to run (even while there is no Operator managing them). After the Operator upgrade process is completed, the new Operator continues to manage the current TimesTenClassic objects in your cluster as well as the TimesTen databases associated with those TimesTenClassic objects.
These sections cover the steps for upgrading the Operator:
The new release of the TimesTen Operator is included in the new release of the TimesTen full distribution on Linux 64-bit. (In this example, the new release is 18.1.4.11.0.)
Perform these steps to download the full distribution of TimesTen and then unpack the TimesTen Operator distribution that is embedded within it. Perform all steps from your Linux development host.
From the directory of your choice:
Create one subdirectory into which you will download the new TimesTen full distribution. For example, create the new_installation_dir subdirectory.
Create a second subdirectory into which you will unpack the new TimesTen Operator distribution. For example, create the new_kube_files subdirectory.
% mkdir -p new_installation_dir % mkdir -p new_kube_files
Navigate to new_installation_dir.
% cd new_installation_dir
Download the TimesTen full distribution into this directory. In this example, download the timesten1814110.server.linux8664.zip file (the 18.1.4.11.0 full distribution for Linux 64-bit).
From the new_installation_dir, use the ZIP utility to unpack the TimesTen distribution.
% unzip timesten1814110.server.linux8664.zip Archive: /timesten/installation/timesten1814110.server.linux8664.zip creating: 18.1.4.11.0/ creating: 18.1.4.11.0/ttoracle_home/ ... creating: tt18.1.4.11.0/kubernetes/ ...
Note that the new_installation_dir/tt18.1.4.11.0/kubernetes directory is created. The operator.zip file is located in this directory. For example, this is a sample directory structure after unpacking the distribution (which can change from release to release):
% pwd
new_installation_dir/tt18.1.4.11.0
% dir
3rdparty include lib oraclescripts README.html ttoracle_home
bin info network PERL startup
grid kubernetes nls plsql support
Navigate to the new_kube_files directory and unpack the operator.zip file into it. In this example, unpack the new_installation_dir/tt18.1.4.11.0/kubernetes/operator.zip file.
% cd new_kube_files % unzip new_installation_dir/tt18.1.4.11.0/kubernetes/operator.zip [...UNZIP OUTPUT...]
Review the directory structure. This example shows the most important subdirectories and files, which can change from release to release.
README.md deploy/crd.yaml deploy/operator.yaml deploy/service_account.yaml operator/Dockerfile operator/timestenclassic-operator ttimage/agent2 ttimage/.bashrc ttimage/create1.sql ttimage/create2.sql ttimage/Dockerfile ttimage/get1.sql ttimage/pausecq.sql ttimage/repcreate.sql ttimage/repduplicate.sql ttimage/runsql,sql ttimage/starthost.pl ttimage/.ttdotversion ttimage/.ttdrop
Note:
This directory tree must persist through the lifetime of the TimesTen Operator.In addition, do not delete the TimesTen full distribution file (timesten1814110.server.linux8664.zip, in this example). You need to copy this file into the:
/operator directory to build the new Operator image and push the image to the image registry. See "Build the new Operator image" for details.
/ttimage directory to build the new TimesTen image and push the image to the image registry. See "Build the new TimesTen image" for details.
You must replace the crd.yaml and the service_account.yaml files that reside in the new_kube_files/ttdeploy directory.
Note:
Ensure you do not delete thecrd.yaml file. Doing so deletes the TimesTenClassic objects along with the TimesTen databases associated with them.Perform these steps:
Navigate to the new_kube_files/deploy directory and recreate the crd.yaml file.
% cd new_kube_files/deploy
% kubectl replace -f crd.yaml
customresourcedefinition.apiextensions.k8s.io/timestenclassics.timesten.
oracle.com replaced
While in the new_kube_files/deploy directory, recreate the Kubernetes service account in which the Operator runs. The Operator requires additional privileges to perform the upgrade procedure.
% kubectl replace -f service_account.yaml role.rbac.authorization.k8s.io/timestenclassic-operator replaced serviceaccount/timestenclassic-operator replaced rolebinding.rbac.authorization.k8s.io/timestenclassic-operator replaced
You have successfully replaced the crd.yaml and the service_account.yaml files. You are now ready to build the Operator image.
Before you can run the new Operator, you must build the new Operator image and push it to your image registry.
The files needed to build the new Operator image are provided in the new_kube_files/operator directory (part of the ZIP file you previously unpacked).
To build the new Operator image and push it to your registry, perform these steps:
Navigate to the new_kube_files/operator directory, and copy the TimesTen distribution into it. This example assumes you downloaded the timesten1814110.server.linux8664.zip distribution into the new_installation_dir directory. See "Download the new release of the TimesTen Operator" for information. Then, verify the timesten1814110.server.linux8664.zip file is in the new_kube_files/operator directory.
% cd new_kube_files/operator % cp new_installation_dir/timesten1814110.server.linux8664.zip . % ls -a Dockerfile timesten1814110.server.linux8664.zip timestenclassic-operator
Navigate to the new_kube_files/operator directory (if not already in this directory) and use the docker command to build and tag the new Operator image. When you are tagging the new Operator image, it is recommended that you tag the image with a release number. For example, you can use the naming convention: ttclassic-operator:release (where release is the release you wish to tag). In this example, ttclassic-operator:3 is used to name the new Operator image (represented in bold).
% cd new_kube_files/operator % docker build -t ttclassic-operator:3 . Sending build context to Docker daemon 478.6MB Step 1/7 : FROM container-registry.oracle.com/os/oraclelinux:7 ---> d788eca028a0 Step 2/7 : ARG TT_DISTRO=timesten1814110.server.linux8664.zip ---> Using cache ---> a259a93fe906 Step 3/7 : RUN yum -y install openssl unzip && /usr/sbin/useradd -d /tt-operator -m -u 1001 -s /bin/nologin -U tt-operator ---> Using cache ---> e3f1427246ab Step 4/7 : COPY --chown=tt-operator:tt-operator timestenclassic-operator /usr/local/bin/timestenclassic-operator ---> Using cache ---> 6ccad53230f0 Step 5/7 : COPY --chown=tt-operator:tt-operator $TT_DISTRO /tt-operator/ $TT_DISTRO ---> 5cd31705485a Step 6/7 : USER tt-operator ---> Running in 6a773ddac5dd Removing intermediate container 6a773ddac5dd ---> 875ee38ebc75 Step 7/7 : ENTRYPOINT ["/usr/local/bin/timestenclassic-operator"] ---> Running in fed0f6c94c2f Removing intermediate container fed0f6c94c2f ---> 10dde79e1617 Successfully built 10dde79e1617 Successfully tagged ttclassic-operator:3
Use the docker command to tag the new Operator image.
Replace phx.ocir.io/youraccount with the location of your image registry. (phx.ocir.io/youraccount is represented in bold in this example.)
Replace ttclassic-operator:3 with the name you chose in the previous step. (ttclassic-operator:3 is represented in bold in this example.)
% docker tag ttclassic-operator:3 phx.ocir.io/youraccount/ttclassic-operator:3
Use the docker command to push the new Operator image to your registry.
Replace phx.ocir.io/youraccount with the location of your image registry. (phx.ocir.io/youraccount is represented in bold in this example.)
Replace ttclassic-operator:3 with the name you chose in the previous steps. (ttclassic-operator:3 is represented in bold in this example.)
% docker push phx.ocir.io/youraccount/ttclassic-operator:3 The push refers to repository [phx.ocir.io/youraccount/ttclassic-operator] 46458e9fc890: Pushed 471a399f0540: Pushed 9e51a2b82af3: Pushed 2f915858a916: Layer already exists 3: digest: sha256:9b941f12e3d52298b9b38f7766ddcdfb1d011857a990ff01a8adafd32f3d3e8d size: 1166
You successfully built the new Operator image and pushed it to your image registry.
This section provides the steps to review the current (running) Operator. These steps are not required.
Use the kubectl get command to ensure the current Operator is running (timestenclassic-operator-66bd4bc88b-c8vf, in this example, represented in bold).
% kubectl get pods
NAME READY STATUS RESTARTS AGE
sample-0 2/2 Running 0 168m
sample-1 2/2 Running 0 168m
sample2-0 2/2 Running 0 158m
sample2-1 2/2 Running 0 158m
timestenclassic-operator-66bd4bc88b-c8vfq 1/1 Running 0 3h5m
Use the kubectl describe command to review the current timestenclassic-operator Deployment. Note that the image for the Deployment is the original image (phx.ocir.io/youraccount/ttclassic-operator:2, in this example, represented in bold).
% kubectl describe deployment timestenclassic-operator
Name: timestenclassic-operator
Namespace: mynamespace
CreationTimestamp: Sun, 11 Apr 2021 13:40:36 +0000
Labels: <none>
Annotations: deployment.kubernetes.io/revision: 1
Selector: name=timestenclassic-operator
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0
unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: name=timestenclassic-operator
Service Account: timestenclassic-operator
Containers:
timestenclassic-operator:
Image: phx.ocir.io/youraccount/ttclassic-operator:2
Port: <none>
Host Port: <none>
Command:
timestenclassic-operator
Environment:
WATCH_NAMESPACE: (v1:metadata.namespace)
POD_NAME: (v1:metadata.name)
OPERATOR_NAME: timestenclassic-operator
GODEBUG: x509ignoreCN=0
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: timestenclassic-operator-66bd4bc88b (1/1 replicas created)
Events: <none>
Review the TimesTenClassic objects that are running in the Kubernetes cluster. There are two TimesTenClassic objects running (sample and sample2, in this example).
% kubectl get ttc NAME STATE ACTIVE AGE sample Normal sample-0 3h8m sample2 Normal sample2-0 179m
This section involves updating the current timestenclassic-operator Deployment to use the new Operator container image. After completing the steps in this section, the Operator is restarted and the upgrade to the new Operator becomes effective. If there is more than one Operator running, each Operator is restarted one at a time. This new Operator will continue to manage the TimesTenClassic objects and the TimesTen databases associated with the those TimesTenClassic objects.
Perform these steps:
Navigate to the new_kube_files/deploy directory, and edit the operator.yaml file. This file is provided in the distribution that you previously unpacked. See "Download the new release of the TimesTen Operator" for details.
Update these fields represented in bold (in the operator.yaml file below):
replicas: 1
Replace 1 with the number of copies of the Operator that you would like to run. 1 is acceptable for development and testing. However, you can run more than one replica for high availability purposes.
Replace sekret with the name of the image pull secret that Kubernetes uses to pull images from your registry.
Replace the image line to reference the Operator image you just created. (phx.ocir.io/youraccount/ttclassic_operator:3, in this example).
% cd new_kube_files/deploy % vi operator.yaml apiVersion: apps/v1 kind: Deployment metadata: name: timestenclassic-operator spec: replicas: 1 selector: matchLabels: name: timestenclassic-operator template: metadata: labels: name: timestenclassic-operator spec: serviceAccountName: timestenclassic-operator imagePullSecrets: - name: sekret containers: - name: timestenclassic-operator image: phx.ocir.io/youraccount/ttclassic-operator:3 command: - timestenclassic-operator imagePullPolicy: Always env: - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: OPERATOR_NAME value: "timestenclassic-operator" - name: GODEBUG value: "x509ignoreCN=0"
From within the new_kube_files/deploy directory, use the kubectl replace command to update the timestenclassic-operator Deployment.
% kubectl replace -f operator.yaml deployment.apps/timestenclassic-operator replaced
Use the kubectl get pods command to verify the new Operator is running (timestenclassic-operator-846cb5c97c-sbz22 in this example, represented in bold).
% kubectl get pods NAME READY STATUS RESTARTS AGE sample-0 2/2 Running 0 3h37m sample-1 2/2 Running 0 3h37m sample2-0 2/2 Running 0 3h28m sample2-1 2/2 Running 0 3h28m timestenclassic-operator-846cb5c97c-sbz22 1/1 Running 0 80s
Use the kubectl describe deployment command to view the new timetenclassic-operator Deployment. Note that the Operator is using the phx.ocir.io/youraccount/ttclassic-operator:3 image (represented in bold).
% kubectl describe deployment timestenclassic-operator
Name: timestenclassic-operator
Namespace: mynamespace
CreationTimestamp: Sun, 11 Apr 2021 13:40:36 +0000
Labels: name=timestenclassic-operator
Annotations: deployment.kubernetes.io/revision: 2
Selector: name=timestenclassic-operator
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0
unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: name=timestenclassic-operator
Service Account: timestenclassic-operator
Containers:
timestenclassic-operator:
Image: phx.ocir.io/youraccount/ttclassic-operator:3
Port: <none>
Host Port: <none>
Command:
timestenclassic-operator
Environment:
WATCH_NAMESPACE: (v1:metadata.namespace)
POD_NAME: (v1:metadata.name)
OPERATOR_NAME: timestenclassic-operator
GODEBUG: x509ignoreCN=0
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: timestenclassic-operator-846cb5c97c (1/1 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 4m19s deployment-controller Scaled up replica
set timestenclassic-operator-846cb5c97c to 1
Normal ScalingReplicaSet 3m51s deployment-controller Scaled down replica
set timestenclassic-operator-66bd4bc88b to 0
You have successfully updated the timestenclassic-operator Deployment. The new Operator automatically begins to manage any existing TimesTenClassic objects in your Kubernetes cluster.
After you upgrade the Operator, you must upgrade your active standby pairs of TimesTen databases to a new patch of TimesTen. This involves building a new container image that contains the new TimesTen release and then modifying the image value of each of your TimesTenClassic objects to the name of this new container image.
When you modify the image value, the Operator notices the change and modifies the corresponding image attributes(s) in the StatefulSet.
What happens next is dependent on the value of the TimesTenClassic object's imageUpgradeStrategy element. See "TimesTenClassicSpecSpec" for more information on imageUpgradeStrategy. If the value is:
auto (or not specified): There are no additional steps you need to take. However, you can monitor the progress of the upgrade and then verify the upgrade was successful.
manual: There are additional steps you must complete. These steps include upgrading the standby database and then performing the steps to fail over from the active database to the standby database. You can also verify the upgrade was successful.
These sections cover the steps necessary to upgrade each of your active standby pairs of TimesTen databases. For example purposes, there are two TimesTenClassic objects (sample and sample2) that require upgrading. The sample TimesTenClassic object (with an imageUpgradeStrategy value of auto) is upgraded first. Then the sample2 TimesTenClassic object (with an imageUpgradeStrategy value of manual) is upgraded:
Complete one of the following depending on the upgrade strategy for the TimesTenClassic object. One of these procedures must be done for each TimesTenClassic object you wish to upgrade.
If auto (value of imageUpgradeStrategy is auto or not specified): Perform an automated upgrade
If manual (value of imageUpgradeStrategy is manual): Perform a manual upgrade
This section illustrates how to build TimesTen as a container image and then push the image to your image registry. The files that you need to build the new TimesTen image are provided in the new_kube_files directory tree. See "Download the new release of the TimesTen Operator" for information.
To build the new TimesTen container image, perform these steps:
Navigate to the new_kube_files/ttimage directory, and copy the TimesTen distribution into it. This example assumes you downloaded the timesten1814110.server.linux8664.zip distribution into the new_installation_dir directory. See "Download the new release of the TimesTen Operator" for information. Then, verify the timesten1814110.server.linux8664.zip file is in the new_kube_files/ttimage directory.
% cd new_kube_files/ttimage % cp new_installation_dir/timesten1814110.server.linux8664.zip . % ls *.zip timesten1814110.server.linux8664.zip
Navigate to the new_kube_files/ttimage directory (if not already in this directory). Edit the Dockerfile, replacing timesten1814110.server.linux8664.zip with the name of your TimesTen full distribution. If your TimesTen distribution is timesten1814110.server.linux8664.zip, no modification is necessary. If not, the modification you need to make is represented in bold.
% cd new_kube_files/ttimage % vi Dockerfile # Copyright (c) 2019, 2021, Oracle and/or its affiliates. FROM container-registry.oracle.com/os/oraclelinux:7 ARG TT_DISTRO=timesten1814110.server.linux8664.zip RUN yum -y install tar gzip vim curl unzip libaio util-linux RUN groupadd -g 333 oracle RUN useradd -M -d /tt/home/oracle -s /bin/bash -u 333 -g oracle oracle RUN install -d -m 0750 -o oracle -g oracle /home/oracle COPY --chown=oracle:oracle $TT_DISTRO /home/oracle/ COPY --chown=oracle:oracle .bashrc starthost.pl .ttdrop .ttdotversion agent2 create1.sql create2.sql get1.sql repcreate.sql repduplicate.sql runsql.sql pausecg.sql /home/oracle/ # Uncomment the following line if you are using the optional non-root installation procedure. # USER 333 ENTRYPOINT "/home/oracle/starthost.pl"
Use the docker command to build the new TimesTen container image. Replace tt1814110:3 with a name of your choosing (represented in bold, in the docker build command below). Note that the output may change from release to release.
% docker build -t tt1814110:3 . Sending build context to Docker daemon 445.8MB Step 1/9 : FROM container-registry.oracle.com/os/oraclelinux:7 ---> d788eca028a0 Step 2/9 : ARG TT_DISTRO=timesten1814110.server.linux8664.zip ---> Using cache ---> a259a93fe906 Step 3/9 : RUN yum -y install tar gzip vim curl unzip libaio util-linux ---> Using cache ---> ac676b5376f3 Step 4/9 : RUN groupadd -g 333 oracle ---> Using cache ---> ce16920f085c Step 5/9 : RUN useradd -M -d /tt/home/oracle -s /bin/bash -u 333 -g oracle oracle ---> Using cache ---> 0319814aca1c Step 6/9 : RUN install -d -m 0750 -o oracle -g oracle /home/oracle ---> Using cache ---> c8612b53398a Step 7/9 : COPY --chown=oracle:oracle $TT_DISTRO /home/oracle/ ---> 31cae98b71fd Step 8/9 : COPY --chown=oracle:oracle .bashrc starthost.pl .ttdrop .ttdotversion agent2 create1.sql create2.sql get1.sql repcreate.sql repduplicate.sql runsql.sql pausecg.sql /home/oracle/ ---> e50eb99c9b54 Step 9/9 : ENTRYPOINT "/home/oracle/starthost.pl" ---> Running in 0b41efd38837 Removing intermediate container 0b41efd38837 ---> 171245e546d5 Successfully built 171245e546d5 Successfully tagged tt1814110:3
Use the docker command to tag the new TimesTen container image. Replace the following, represented in bold, in the docker tag command below.
tt1814110:3 with the name you chose in the previous step.
phx.ocir.io/youraccount with the location of your image registry.
% docker tag tt1814110:3 phx.ocir.io/youraccount/tt1814110:3
Use the docker command to push the new TimesTen container image to your registry. Replace the following, represented in bold, in the docker push command below.
phx.ocir.io/youraccount with the location of your image registry.
tt1814110:3 with the name you chose previously.
% docker push phx.ocir.io/youraccount/tt1814110:3 The push refers to repository [phx.ocir.io/youraccount/tt1814110] 97a0f250b2fe: Pushed 650b003a3ad4: Pushed b8de51528854: Pushed 62192d26e325: Pushed 7dfe13e9b5a4: Pushed d8570fce965c: Pushed 2f915858a916: Layer already exists 3: digest: sha256:a6ac313394229eb2256d4a56fbcd8e2eda50ea2cc21991fa76f11701f2299710 size: 1788
You successfully built the new TimesTen container image. It is pushed to your image registry. You are now ready to check the upgrade strategy for each TimesTenClassic object that you wish to upgrade.
The value of each of the TimesTenClassic object's imageUpgradeStrategy element determines the type of upgrade. If the value is auto (or not specified), the upgrade strategy is automated and the Operator does the upgrade for this TimesTenClassic object. If the value is manual, the upgrade strategy is manual and you must manually perform the upgrade for the TimesTenClassic object.
This example shows you how to determine the imageUpgradeStrategy for the TimesTenClassic objects deployed in your Kubernetes cluster.
Review the TimesTenClassic objects that are running in the Kubernetes cluster. There are two TimesTenClassic objects running (sample and sample2, in this example).
% kubectl get ttc NAME STATE ACTIVE AGE sample Normal sample-0 2d3h sample2 Normal sample2-0 2d3h
Use the kubectl describe command to show the sample TimesTenClassic object. Note that the sample TimesTenClassic object has imageUpdateStrategy set to auto (represented in bold). This indicates an automated upgrade strategy. Note also that upgradeDownPodTimeout is set to 900 (represented in bold).
% kubectl describe ttc sample
Name: sample
Namespace: mynamespace
Labels: <none>
Annotations: <none>
API Version: timesten.oracle.com/v1
Kind: TimesTenClassic
Metadata:
Creation Timestamp: 2021-04-11T13:58:09Z
Generation: 1
Resource Version: 150145728
Self Link:
/apis/timesten.oracle.com/v1/namespaces/mynamespace/timestenclassics/sample
UID: f2a16dff-9acd-11eb-86a3-06b2b9dd76bc
Spec:
Ttspec:
Db Config Map:
sample
Image: phx.ocir.io/youraccount/tt181440:2
Image Pull Policy: Always
Image Pull Secret: sekret
Image Upgrade Strategy: auto
Storage Class Name: oci
Storage Size: 250G
Upgrade Down Pod Timeout: 900
...
Use the kubectl describe command to show the sample2 TimesTenClassic object. Note that the sample2 TimesTenClassic object has imageUpdateStrategy set to manual (represented in bold). This indicates a manual upgrade strategy.
% kubectl describe ttc sample2
Name: sample2
Namespace: mynamespace
Labels: <none>
Annotations: <none>
API Version: timesten.oracle.com/v1
Kind: TimesTenClassic
Metadata:
Creation Timestamp: 2021-04-11T14:07:20Z
Generation: 1
Resource Version: 150149654
Self Link:
/apis/timesten.oracle.com/v1/namespaces/mynamespace/timestenclassics/sample2
UID: 3af3e6fb-9acf-11eb-8286-6a1fd5dce8ff
Spec:
Ttspec:
Db Config Map:
sample2
Image: phx.ocir.io/youraccount/tt181440:2
Image Pull Policy: Always
Image Pull Secret: sekret
Image Upgrade Strategy: manual
Storage Class Name: oci
Storage Size: 250G
...
You have successfully checked whether an automated or manual upgrade will be performed for each of the TimesTenClassic objects in your Kubernetes cluster. You are now ready to continue the upgrade process. This example upgrades the sample TimesTenClassic object first. Since the upgrade strategy for the sample TimesTenClassic object is auto, proceed to "Perform an automated upgrade" to continue the upgrade procedure. (Note: You could upgrade the sample2 TimesTenClassic object first. The order does not matter.)
This section describes what you need to do for an automated upgrade. If you wish to do a manual upgrade, see "Perform a manual upgrade" for details.
The automated upgrade process requires you to modify the image value of the TimesTenClassic object to reference the new TimesTen image (18.1.4.11.0, in this example). After you modify the TimesTenClassic object to reference the new TimesTen image, the Operator notices the change and modifies the StatefulSet that it created. The Operator then starts the upgrade process. You can use the kubectl describe command to monitor this upgrade process.
Note:
If you do an automated upgrade, your databases will be taken down, restarted, and failed over immediately. Do not perform this procedure at the busiest time of your production day. Applications will see short outages as a result of the upgrade procedure.You must modify the TimesTenClassic object to reference the new TimesTen image. You do this by modifying the value of the image CRD syntax element(s) for the TimesTenClassic object. See "TimesTenClassicSpecSpec" for more information on the image element.
The value of imageUpgradeStrategy for the sample TimesTenClassic object is auto, which indicates the Operator will perform the upgrade. After you edit the image value of the sample TimesTenClassic object to reference the new TimesTen image, the Operator notices the change, modifies the StatefulSet, and starts the automated upgrade process.
Use the kubectl edit command to edit the sample TimesTenClassic object, changing the .spec.ttspec.image attribute to reference the new TimesTen image (phx.ocir.io/youraccount/tt1814110:3, in this example, represented in bold).
Note: Not all output is shown.
% kubectl edit timestenclassic sample
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this
file will be
# reopened with the relevant failures.
#
apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
...
name: sample
...
spec:
ttspec:
dbConfigMap:
- sample
image: phx.ocir.io/youraccount/tt1814110:3
imagePullPolicy: Always
imagePullSecret: sekret
imageUpgradeStrategy: manual
storageClassName: oci
storageSize: 250G
...
timestenclassic.timesten.oracle.com/sample edited
Use the kubectl describe statefulset command to verify that the Operator has modified the sample StatefulSet and replaced the image with the new image (phx.ocir.io/youraccount/tt1814110:3, in this example, represented in bold).
% kubectl describe statefulset sample
Name: sample
Namespace: mynamespace
CreationTimestamp: Sun, 11 Apr 2021 13:58:10 +0000
Selector: app=sample
Labels: app=sample
Annotations: <none>
Replicas: 2 desired | 2 total
Update Strategy: OnDelete
Pods Status: 1 Running / 1 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=sample
Init Containers:
ttinit:
Image: phx.ocir.io/youraccount/tt1814110:3
Ports: 8443/TCP, 6624/TCP, 6625/TCP, 4444/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP
Command:
perl
/home/oracle/starthost.pl
Environment:
TIMESTEN_HOME: /tt/home/oracle/instances/instance1
LD_LIBRARY_PATH:
/tt/home/oracle/instances/instance1/ttclasses/lib:/tt/home/oracle/instances/
instance1/install/lib:/tt/home/oracle/instances/instance1/install/
ttoracle_home/instantclient_11_2
TT_REPLICATION_TOPOLOGY: activeStandbyPair
TT_INIT_CONTAINER: 1
Mounts:
/tt from tt-persistent (rw)
/ttagent from tt-agent (rw)
/ttconfig from tt-config (rw)
Containers:
tt:
Image: phx.ocir.io/youraccount/tt1814110:3
Ports: 8443/TCP, 6624/TCP, 6625/TCP, 4444/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP
Command:
perl
/home/oracle/starthost.pl
Environment:
TIMESTEN_HOME: /tt/home/oracle/instances/instance1
LD_LIBRARY_PATH:
/tt/home/oracle/instances/instance1/ttclasses/lib:/tt/home/oracle/instances/
instance1/install/lib:/tt/home/oracle/instances/instance1/install/
ttoracle_home/instantclient_11_2
TT_REPLICATION_TOPOLOGY: activeStandbyPair
Mounts:
/tt from tt-persistent (rw)
/ttagent from tt-agent (rw)
/ttconfig from tt-config (rw)
daemonlog:
Image: phx.ocir.io/youraccount/tt1814110:3
Port: <none>
Host Port: <none>
Command:
sh
-c
/bin/bash <<'EOF'
while [ 1 ] ; do tail --follow=name
/tt/home/oracle/instances/instance1/diag/ttmesg.log --max-unchanged-stats=5;
sleep 1; done
exit 0
EOF
Requests:
cpu: 100m
memory: 20Mi
Environment:
TIMESTEN_HOME: /tt/home/oracle/instances/instance1
LD_LIBRARY_PATH:
/tt/home/oracle/instances/instance1/ttclasses/lib:/tt/home/oracle/instances/
instance1/install/lib:/tt/home/oracle/instances/instance1/install/
ttoracle_home/instantclient_11_2
Mounts:
/tt from tt-persistent (rw)
Volumes:
tt-agent:
Type: Secret (a volume populated by a Secret)
SecretName: ttf2a16dff-9acd-11eb-86a3-06b2b9dd76bc
Optional: false
tt-config:
Type: Projected (a volume that contains injected data from
multiple sources)
ConfigMapName: sample
ConfigMapOptional: <nil>
Volume Claims:
Name: tt-persistent
StorageClass: oci
Labels: <none>
Annotations: <none>
Capacity: 250G
Access Modes: [ReadWriteOnce]
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 7s statefulset-controller create Pod sample-1
in StatefulSet sample successful
You have successfully modified the sample TimesTenClassic object to use the new TimesTen image. You are now ready to monitor the automated upgrade process performed by the Operator. Proceed to "Monitor the automated upgrade" to continue.
You can monitor the automated upgrade process performed by the Operator.
Use the kubectl get command to assess the state of the sample TimesTenClassic object.
Note that the state is StandbyDown (represented in bold).
% kubectl get ttc sample
NAME STATE ACTIVE AGE
sample StandbyDown sample-1 2d5h
Wait a few minutes, then run the command again. Note that the state has changed to Normal (represented in bold).
% kubectl get ttc sample
NAME STATE ACTIVE AGE
sample Normal sample-1 2d5h
Use the kubectl describe command to observe how the Operator promoted the standby database (sample-1) to be the active. Note also that both the standby and the active databases have been upgraded to the new release of TimesTen (18.1.4.11.0, in this example).
% kubectl describe ttc sample
Name: sample
Namespace: mynamespace
Labels: <none>
Annotations: <none>
API Version: timesten.oracle.com/v1
Kind: TimesTenClassic
Metadata:
Creation Timestamp: 2021-04-11T13:58:09Z
Generation: 2
Resource Version: 150178771
Self Link:
/apis/timesten.oracle.com/v1/namespaces/mynamespace/timestenclassics/sample
UID: f2a16dff-9acd-11eb-86a3-06b2b9dd76bc
Spec:
Ttspec:
Db Config Map:
sample
Image: phx.ocir.io/youraccount/tt1814110:3
Image Pull Policy: Always
Image Pull Secret: sekret
Image Upgrade Strategy: auto
Storage Class Name: oci
Storage Size: 250G
Upgrade Down Pod Timeout: 900
Status:
Classic Upgrade Status:
Active Start Time: 0
Active Status:
Image Update Pending: false
Last Upgrade State Switch: 0
Prev Reset Upgrade State:
Prev Upgrade State:
Standby Start Time: 0
Standby Status:
Upgrade Start Time: 0
Upgrade State:
Active Pods: sample-1
High Level State: Normal
Last Event: 67
Last High Level State Switch: 1618341354
Pod Status:
Cache Status:
Cache Agent: Not Running
Cache UID Pwd Set: true
N Cache Groups: 0
Db Status:
Db: Loaded
Db Id: 35425
Db Updatable: No
Initialized: true
Last High Level State Switch: ?
Pod Status:
Agent: Up
Last Time Reachable: 1618341923
Pod IP: 10.244.7.46
Pod Phase: Running
Prev High Level State: Healthy
Prev Image: phx.ocir.io/youraccount/tt181440:2
Replication Status:
Last Time Rep State Changed: 1618341175
Rep Agent: Running
Rep Peer P State: start
Rep Scheme: Exists
Rep State: STANDBY
Times Ten Status:
Daemon: Up
Instance: Exists
Release: 18.1.4.11.0
Admin User File: true
Cache User File: false
Cg File: false
Disable Return: false
High Level State: Healthy
Intended State: Standby
Local Commit: false
Name: sample-0
Schema File: true
Using Twosafe: false
Cache Status:
Cache Agent: Not Running
Cache UID Pwd Set: true
N Cache Groups: 0
Db Status:
Db: Loaded
Db Id: 35426
Db Updatable: Yes
Initialized: true
Last High Level State Switch: ?
Pod Status:
Agent: Up
Last Time Reachable: 1618341923
Pod IP: 10.244.6.25
Pod Phase: Running
Prev High Level State: Healthy
Prev Image: phx.ocir.io/youraccount/tt181440:2
Replication Status:
Last Time Rep State Changed: 1618340980
Rep Agent: Running
Rep Peer P State: start
Rep Scheme: Exists
Rep State: ACTIVE
Times Ten Status:
Daemon: Up
Instance: Exists
Release: 18.1.4.11.0
Admin User File: true
Cache User File: false
Cg File: false
Disable Return: false
High Level State: Healthy
Intended State: Active
Local Commit: false
Name: sample-1
Schema File: true
Using Twosafe: false
Prev High Level State: StandbyDown
Prev Reexamine:
Prev Stop Managing:
Rep Create Statement: create active standby pair "sample" on
"sample-0.sample.mynamespace.svc.cluster.local", "sample" on
"sample-1.sample.mynamespace.svc.cluster.local" NO RETURN store "sample" on
"sample-0.sample.mynamespace.svc.cluster.local" PORT 4444 FAILTHRESHOLD 0
store "sample" on "sample-1.sample.mynamespace.svc.cluster.local" PORT 4444
FAILTHRESHOLD 0
Rep Port: 4444
Status Version: 1.0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
- StateChange 15m ttclassic Image updated, automatic upgrade
started
- StateChange 15m ttclassic Deleted STANDBY pod sample-1 per
upgrade
- StateChange 15m ttclassic TimesTenClassic was Normal, now
ActiveTakeover
- StateChange 15m ttclassic TimesTenClassic was ActiveTakeover, now
StandbyDown
- StateChange 13m ttclassic Pod sample-1 Agent Up
- StateChange 13m ttclassic Pod sample-1 Release 18.1.4.11.0
- StateChange 13m ttclassic Pod sample-1 Instance Exists
- StateChange 13m ttclassic Pod sample-1 Daemon Up
- StateChange 13m ttclassic Pod sample-1 Database None
- StateChange 12m ttclassic Pod sample-1 RepState IDLE
- StateChange 12m ttclassic Pod sample-1 Database Loaded
- StateChange 12m ttclassic Pod sample-1 CacheAgent Not Running
- StateChange 12m ttclassic Pod sample-1 RepAgent Not Running
- StateChange 12m ttclassic Pod sample-1 RepScheme Exists
- StateChange 12m ttclassic Pod sample-1 Database Not Updatable
- StateChange 12m ttclassic Pod sample-1 RepAgent Running
- StateChange 12m ttclassic Pod sample-1 RepState STANDBY
- StateChange 12m ttclassic TimesTenClassic was StandbyDown, now
Normal
- StateChange 12m ttclassic Deleted ACTIVE pod sample-0 per upgrade
- StateChange 12m ttclassic TimesTenClassic was Normal, now
ActiveDown
- StateChange 12m ttclassic TimesTenClassic was ActiveDown, now
ActiveTakeover
- StateChange 12m ttclassic Pod sample-1 RepState ACTIVE
- StateChange 12m ttclassic Pod sample-1 Database Updatable
- StateChange 12m ttclassic TimesTenClassic was ActiveTakeover, now
StandbyDown
- StateChange 10m ttclassic Pod sample-0 Agent Up
- StateChange 10m ttclassic Pod sample-0 Instance Exists
- StateChange 10m ttclassic Pod sample-0 Daemon Up
- StateChange 10m ttclassic Pod sample-0 Database None
- StateChange 10m ttclassic Pod sample-0 Release 18.1.4.11.0
- StateChange 9m38s ttclassic Pod sample-0 RepState IDLE
- StateChange 9m38s ttclassic Pod sample-0 Database Loaded
- StateChange 9m38s ttclassic Pod sample-0 Database Not Updatable
- StateChange 9m38s ttclassic Pod sample-0 RepAgent Not Running
- StateChange 9m38s ttclassic Pod sample-0 RepScheme Exists
- StateChange 9m38s ttclassic Pod sample-0 CacheAgent Not Running
- StateChange 9m32s ttclassic Pod sample-0 RepAgent Running
- StateChange 9m32s ttclassic Pod sample-0 RepState STANDBY
- StateChange 9m32s ttclassic Upgrade of ACTIVE complete
- StateChange 9m32s ttclassic Upgrade completed in 385 secs
- StateChange 9m32s ttclassic TimesTenClassic was StandbyDown, now
Normal
The automated upgrade is successful. The active and standby Pods are running the new TimesTen image, which contains the new TimesTen release. If you need to upgrade additional TimesTenClassic objects, see "Perform an automated upgrade" for an automated upgrade or "Perform a manual upgrade" for a manual upgrade. If the upgrade process is complete for all TimesTenClassic objects, see "Verify the active standby pair of databases are upgraded" to verify the upgrade of all TimesTenClassic objects that are running in your Kubernetes cluster.
This example now upgrades the sample2 TimesTenClassic object. Recall that the value of the imageUpgradePolicy is manual for this TimesTenClassic object. Therefore, proceed to "Perform a manual upgrade" to complete the upgrade for the sample2 TimesTenClassic object.
This section describes what you need to do for a manual upgrade. If you wish to do an automated upgrade, see "Perform an automated upgrade" for details.
You must modify the TimesTenClassic object to reference the new TimesTen image. You do this by modifying the value of the image CRD syntax element(s) for the TimesTenClassic object. See "TimesTenClassicSpecSpec" for more information on the image element.
The value of imageUpgradeStrategy for the sample2 TimesTenClassic object is manual, which indicates that you will manually perform the upgrade. After you edit the image value of the sample2 TimesTenClassic object to reference the new TimesTen image, the Operator notices the change, and modifies the StatefulSet. The Operator does not restart the Pods. Rather, it upgrades the image that the Pods should be running. (The later sections entitled "Upgrade the standby database" and "Failover" detail the steps for restarting the Pods.)
Review the original sample2.yaml file. Note that .spec.ttspec.image references the phx.ocir.io/youraccount/tt181440:2 image (represented in bold). Note also that the value of imageUpgradeStrategy is manual (represented in bold).
% cat sample2.yaml
apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
metadata:
name: sample2
spec:
ttspec:
storageClassName: oci
storageSize: 250G
image: phx.ocir.io/youraccount/tt181440:2
imagePullSecret: sekret
imagePullPolicy: Always
dbConfigMap:
- sample2
imageUpgradeStrategy: manual
Use the kubectl edit command to edit the sample2 TimesTenClassic object, changing the .spec.ttspec.image attribute to reference the new TimesTen image (phx.ocir.io/youraccount/tt1814110:3, in this example, represented in bold).
Note: Not all output is shown.
% kubectl edit timestenclassic sample2
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this
file will be
# reopened with the relevant failures.
#
apiVersion: timesten.oracle.com/v1
kind: TimesTenClassic
...
spec:
ttspec:
dbConfigMap:
- sample2
image: phx.ocir.io/youraccount/tt1814110:3
imagePullPolicy: Always
imagePullSecret: sekret
imageUpgradeStrategy: manual
storageClassName: oci
storageSize: 250G
...
timestenclassic.timesten.oracle.com/sample2 edited
Use the kubectl describe statefulset command to verify that the Operator has modified the sample2 StatefulSet and replaced the image with the new image (phx.ocir.io/youraccount/tt1814110:3, in this example, represented in bold).
% kubectl describe statefulset sample2
Name: sample2
Namespace: mynamespace
CreationTimestamp: Sun, 11 Apr 2021 14:07:21 +0000
Selector: app=sample2
Labels: app=sample2
Annotations: <none>
Replicas: 2 desired | 2 total
Update Strategy: OnDelete
Pods Status: 2 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=sample2
Init Containers:
ttinit:
Image: phx.ocir.io/youraccount/tt1814110:3
Ports: 8443/TCP, 6624/TCP, 6625/TCP, 4444/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP
Command:
perl
/home/oracle/starthost.pl
Environment:
TIMESTEN_HOME: /tt/home/oracle/instances/instance1
LD_LIBRARY_PATH:
/tt/home/oracle/instances/instance1/ttclasses/lib:/tt/home/oracle/instances/
instance1/install/lib:/tt/home/oracle/instances/instance1/install/
ttoracle_home/instantclient_11_2
TT_REPLICATION_TOPOLOGY: activeStandbyPair
TT_INIT_CONTAINER: 1
Mounts:
/tt from tt-persistent (rw)
/ttagent from tt-agent (rw)
/ttconfig from tt-config (rw)
Containers:
tt:
Image: phx.ocir.io/youraccount/tt1814110:3
Ports: 8443/TCP, 6624/TCP, 6625/TCP, 4444/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP
Command:
perl
/home/oracle/starthost.pl
Environment:
TIMESTEN_HOME: /tt/home/oracle/instances/instance1
LD_LIBRARY_PATH:
/tt/home/oracle/instances/instance1/ttclasses/lib:/tt/home/oracle/instances/
instance1/install/lib:/tt/home/oracle/instances/instance1/install/
ttoracle_home/instantclient_11_2
TT_REPLICATION_TOPOLOGY: activeStandbyPair
Mounts:
/tt from tt-persistent (rw)
/ttagent from tt-agent (rw)
/ttconfig from tt-config (rw)
daemonlog:
Image: phx.ocir.io/youraccount/tt1814110:3
Port: <none>
Host Port: <none>
Command:
sh
-c
/bin/bash <<'EOF'
while [ 1 ] ; do tail --follow=name
/tt/home/oracle/instances/instance1/diag/ttmesg.log --max-unchanged-stats=5;
sleep 1; done
exit 0
EOF
Requests:
cpu: 100m
memory: 20Mi
Environment:
TIMESTEN_HOME: /tt/home/oracle/instances/instance1
LD_LIBRARY_PATH:
/tt/home/oracle/instances/instance1/ttclasses/lib:/tt/home/oracle/instances/
instance1/install/lib:/tt/home/oracle/instances/instance1/install/
ttoracle_home/instantclient_11_2
Mounts:
/tt from tt-persistent (rw)
Volumes:
tt-agent:
Type: Secret (a volume populated by a Secret)
SecretName: tt3af3e6fb-9acf-11eb-8286-6a1fd5dce8ff
Optional: false
tt-config:
Type: Projected (a volume that contains injected data from
multiple sources)
ConfigMapName: sample2
ConfigMapOptional: <nil>
Volume Claims:
Name: tt-persistent
StorageClass: oci
Labels: <none>
Annotations: <none>
Capacity: 250G
Access Modes: [ReadWriteOnce]
Events: <none>
You have successfully modified the sample2 TimesTenClassic object to use the new TimesTen image. You are now ready to continue the manual upgrade. Proceed to "Upgrade the standby database" to continue this manual upgrade.
Perform these steps to upgrade the standby database.
Note:
Even though you are upgrading the standby database, depending on your replication configuration, this may result in disruption on your active database. This may impact your applications. Perform the upgrade at the appropriate time.Use the kubectl get pods command to review the Pods. The sample2-0 and the sample2-1 Pods are shown in this example.
% kubectl get pods NAME READY STATUS RESTARTS AGE sample2-0 2/2 Running 0 2d6h sample2-1 2/2 Running 0 2d6h
Use the kubectl get ttc command to:
Determine which Pod is the standby. The active Pod is the Pod represented in the ACTIVE column. The standby Pod is the other Pod (not represented in the ACTIVE column). Therefore, for the sample2 TimesTenClassic object, the active Pod is sample2-0, (represented in bold) and the standby Pod is sample2-1.
Ensure the state for the TimesTenClassic object (sample2, in this example) is Normal (represented in bold).
% kubectl get ttc sample2 NAME STATE ACTIVE AGE sample2 Normal sample2-0 2d6h
To upgrade the standby to the new TimesTen image, delete the standby Pod (sample2-1, in this example).
% kubectl delete pod sample2-1 pod "sample2-1" deleted
Kubernetes automatically creates a new sample2-1 Pod to replace the deleted Pod. The Operator configures the new sample2-1 Pod as the standby Pod. This new Pod will now run the newly created TimesTen image.
Use the kubectl get command to verify the standby is up and running and the state is Normal.
Note that the state is StandbyDown (represented in bold).
% kubectl get ttc sample2
NAME STATE ACTIVE AGE
sample2 StandbyDown sample2-0 2d6h
Wait a few minutes, then run the command again. Note that the state has changed to Normal (represented in bold).
% kubectl get ttc sample2
NAME STATE ACTIVE AGE
sample2 Normal sample2-0 2d6h
Use the kubectl describe command to further verify that the standby is up and running again and that the active standby pair health is Normal. During the upgrade of the standby, your applications are not disrupted. Your applications can continue to use the active database.
In this example, note the following:
The image is upgraded to the new release (phx.ocir.io/youraccount/tt1814110:3, represented in bold).
The active database (sample2-0) is not upgraded to the new release. (Release is still 18.1.4.4.0, represented in bold.)
The standby database (sample2-1) is upgraded to the new release. (18.1.4.11.0, represented in bold.)
% kubectl describe ttc sample2
Name: sample2
Namespace: mynamespace
Labels: <none>
Annotations: <none>
API Version: timesten.oracle.com/v1
Kind: TimesTenClassic
Metadata:
Creation Timestamp: 2021-04-11T14:07:20Z
Generation: 2
Resource Version: 150206835
Self Link:
/apis/timesten.oracle.com/v1/namespaces/mynamespace/timestenclassics/sample2
UID: 3af3e6fb-9acf-11eb-8286-6a1fd5dce8ff
Spec:
Ttspec:
Db Config Map:
sample2
Image: phx.ocir.io/youraccount/tt1814110:3
Image Pull Policy: Always
Image Pull Secret: sekret
Image Upgrade Strategy: manual
Storage Class Name: oci
Storage Size: 250G
Status:
Classic Upgrade Status:
Active Start Time: 0
Active Status:
Image Update Pending: true
Last Upgrade State Switch: 0
Prev Reset Upgrade State:
Prev Upgrade State:
Standby Start Time: 0
Standby Status:
Upgrade Start Time: 0
Upgrade State:
Active Pods: sample2-0
High Level State: Normal
Last Event: 41
Last High Level State Switch: 1618347326
Pod Status:
Cache Status:
Cache Agent: Not Running
Cache UID Pwd Set: true
N Cache Groups: 0
Db Status:
Db: Loaded
Db Id: 36351
Db Updatable: Yes
Initialized: true
Last High Level State Switch: ?
Pod Status:
Agent: Up
Last Time Reachable: 1618347435
Pod IP: 10.244.8.199
Pod Phase: Running
Prev High Level State: Healthy
Prev Image: phx.ocir.io/youraccount/tt181440:2
Replication Status:
Last Time Rep State Changed: 0
Rep Agent: Running
Rep Peer P State: start
Rep Scheme: Exists
Rep State: ACTIVE
Times Ten Status:
Daemon: Up
Instance: Exists
Release: 18.1.4.4.0
Admin User File: true
Cache User File: false
Cg File: false
Disable Return: false
High Level State: Healthy
Intended State: Active
Local Commit: false
Name: sample2-0
Schema File: true
Using Twosafe: false
Cache Status:
Cache Agent: Not Running
Cache UID Pwd Set: true
N Cache Groups: 0
Db Status:
Db: Loaded
Db Id: 36351
Db Updatable: No
Initialized: true
Last High Level State Switch: ?
Pod Status:
Agent: Up
Last Time Reachable: 1618347435
Pod IP: 10.244.5.156
Pod Phase: Running
Prev High Level State: Healthy
Prev Image: phx.ocir.io/youraccount/tt181440:2
Replication Status:
Last Time Rep State Changed: 0
Rep Agent: Running
Rep Peer P State: start
Rep Scheme: Exists
Rep State: STANDBY
Times Ten Status:
Daemon: Up
Instance: Exists
Release: 18.1.4.11.0
Admin User File: true
Cache User File: false
Cg File: false
Disable Return: false
High Level State: Healthy
Intended State: Standby
Local Commit: false
Name: sample2-1
Schema File: true
Using Twosafe: false
Prev High Level State: StandbyDown
Prev Reexamine:
Prev Stop Managing:
Rep Create Statement: create active standby pair "sample2" on
"sample2-0.sample2.mynamespace.svc.cluster.local", "sample2" on
"sample2-1.sample2.mynamespace.svc.cluster.local" NO RETURN store "sample2"
on "sample2-0.sample2.mynamespace.svc.cluster.local" PORT 4444
FAILTHRESHOLD 0 store "sample2" on
"sample2-1.sample2.mynamespace.svc.cluster.local" PORT 4444
FAILTHRESHOLD 0
Rep Port: 4444
Status Version: 1.0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
- StateChange 29m ttclassic Image updated, automatic upgrade
disabled
- StateChange 4m35s ttclassic TimesTenClassic was Normal, now
ActiveTakeover
- StateChange 4m29s ttclassic TimesTenClassic was ActiveTakeover, now
StandbyDown
- StateChange 2m47s ttclassic Pod sample2-1 Agent Up
- StateChange 2m47s ttclassic Pod sample2-1 Release 18.1.4.11.0
- StateChange 2m47s ttclassic Pod sample2-1 Daemon Up
- StateChange 2m47s ttclassic Pod sample2-1 Database None
- StateChange 118s ttclassic Pod sample2-1 Database Loaded
- StateChange 118s ttclassic Pod sample2-1 Database Not Updatable
- StateChange 118s ttclassic Pod sample2-1 RepAgent Not Running
- StateChange 118s ttclassic Pod sample2-1 RepState IDLE
- StateChange 113s ttclassic Pod sample2-1 RepAgent Running
- StateChange 113s ttclassic Pod sample2-1 RepState STANDBY
- StateChange 113s ttclassic TimesTenClassic was StandbyDown, now
Normal
You have successfully upgraded the standby database. You are now ready to fail over from the active database to the standby. See "Failover" for details.
You must now fail over from the active database to the standby.
Note:
When you fail over, your active database will be taken down, and failed over immediately. Do not perform this procedure at the busiest time of your production day.Before failing over, quiesce your applications on the active database. (You can also use the ttAdmin -close and the ttAdmin -disconnect commands. See "Opening and closing the database for user connections" and "Disconnecting from a database" in the Oracle TimesTen In-Memory Database Operations Guide for information.)
To avoid potential data loss, use the ttRepAdmin -wait command to wait until replication is caught up, such that all transactions that were executed on the active database have been replicated to the standby database. See "ttRepAdmin" in the Oracle TimesTen In-Memory Database Reference for information.
Once the standby is caught up, fail over from the active database to the standby by deleting the active Pod. When you delete the active Pod, the Operator automatically detects the failure and promotes the standby database to be the active. Client/server applications that are using the active database (sample2-0, in this example) are automatically reconnected to the new active database (sample2-1, in this example). Transactions in flight are rolled back. Prepared SQL statements will need to be re-prepared by the applications. See "Handling failover and recovery" for more information of client/server failover.
Kubernetes automatically creates a new sample2-0 Pod to replace the deleted Pod. The Operator will configure the new Pod as the standby Pod. This new Pod will run the newly created TimesTen image.
Note:
You may want to perform this operation during a scheduled production outage.Use the kubectl delete command to delete the active Pod (sample2-0, in this example).
% kubectl delete pod sample2-0 pod "sample2-0" deleted
Use the kubectl describe command to observe how the Operator recovers from the failure. The Operator promotes the standby database (sample2-1) to be active. Any applications that were connected to the sample2-0 database are automatically reconnected to the sample2-1 database by TimesTen. After a brief outage, the applications can continue to use the database. See "Monitoring the health of the active standby pair of databases" for information on the health and states of the active standby pair.
% kubectl describe ttc sample2
Name: sample2
Namespace: mynamespace
Labels: <none>
Annotations: <none>
API Version: timesten.oracle.com/v1
Kind: TimesTenClassic
Metadata:
Creation Timestamp: 2021-04-11T14:07:20Z
Generation: 2
Resource Version: 150214843
Self Link:
/apis/timesten.oracle.com/v1/namespaces/mynamespace/timestenclassics/sample2
UID: 3af3e6fb-9acf-11eb-8286-6a1fd5dce8ff
Spec:
Ttspec:
Db Config Map:
sample2
Image: phx.ocir.io/youraccount/tt1814110:3
Image Pull Policy: Always
Image Pull Secret: sekret
Image Upgrade Strategy: manual
Storage Class Name: oci
Storage Size: 250G
Status:
Classic Upgrade Status:
Active Start Time: 0
Active Status:
Image Update Pending: true
Last Upgrade State Switch: 0
Prev Reset Upgrade State:
Prev Upgrade State:
Standby Start Time: 0
Standby Status:
Upgrade Start Time: 0
Upgrade State:
Active Pods: sample2-1
High Level State: Normal
Last Event: 57
Last High Level State Switch: 1618348960
Pod Status:
Cache Status:
Cache Agent: Not Running
Cache UID Pwd Set: true
N Cache Groups: 0
Db Status:
Db: Loaded
Db Id: 36623
Db Updatable: No
Initialized: true
Last High Level State Switch: ?
Pod Status:
Agent: Up
Last Time Reachable: 1618349003
Pod IP: 10.244.6.26
Pod Phase: Running
Prev High Level State: Healthy
Prev Image: phx.ocir.io/youraccount/tt181440:2
Replication Status:
Last Time Rep State Changed: 0
Rep Agent: Running
Rep Peer P State: start
Rep Scheme: Exists
Rep State: STANDBY
Times Ten Status:
Daemon: Up
Instance: Exists
Release: 18.1.4.11.0
Admin User File: true
Cache User File: false
Cg File: false
Disable Return: false
High Level State: Healthy
Intended State: Standby
Local Commit: false
Name: sample2-0
Schema File: true
Using Twosafe: false
Cache Status:
Cache Agent: Not Running
Cache UID Pwd Set: true
N Cache Groups: 0
Db Status:
Db: Loaded
Db Id: 36624
Db Updatable: Yes
Initialized: true
Last High Level State Switch: ?
Pod Status:
Agent: Up
Last Time Reachable: 1618349003
Pod IP: 10.244.5.156
Pod Phase: Running
Prev High Level State: Healthy
Prev Image: phx.ocir.io/youraccount/tt181440:2
Replication Status:
Last Time Rep State Changed: 0
Rep Agent: Running
Rep Peer P State: start
Rep Scheme: Exists
Rep State: ACTIVE
Times Ten Status:
Daemon: Up
Instance: Exists
Release: 18.1.4.11.0
Admin User File: true
Cache User File: false
Cg File: false
Disable Return: false
High Level State: Healthy
Intended State: Active
Local Commit: false
Name: sample2-1
Schema File: true
Using Twosafe: false
Prev High Level State: StandbyDown
Prev Reexamine:
Prev Stop Managing:
Rep Create Statement: create active standby pair "sample2" on
"sample2-0.sample2.mynamespace.svc.cluster.local", "sample2" on
"sample2-1.sample2.mynamespace.svc.cluster.local" NO RETURN store "sample2"
on "sample2-0.sample2.mynamespace.svc.cluster.local" PORT 4444
FAILTHRESHOLD 0 store "sample2" on
"sample2-1.sample2.mynamespace.svc.cluster.local" PORT 4444
FAILTHRESHOLD 0
Rep Port: 4444
Status Version: 1.0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
- StateChange 55m ttclassic Image updated, automatic upgrade
disabled
- StateChange 30m ttclassic TimesTenClassic was Normal, now
ActiveTakeover
- StateChange 30m ttclassic TimesTenClassic was ActiveTakeover, now
StandbyDown
- StateChange 28m ttclassic Pod sample2-1 Agent Up
- StateChange 28m ttclassic Pod sample2-1 Release 18.1.4.11.0
- StateChange 28m ttclassic Pod sample2-1 Daemon Up
- StateChange 28m ttclassic Pod sample2-1 Database None
- StateChange 28m ttclassic Pod sample2-1 Database Loaded
- StateChange 28m ttclassic Pod sample2-1 Database Not Updatable
- StateChange 28m ttclassic Pod sample2-1 RepAgent Not Running
- StateChange 28m ttclassic Pod sample2-1 RepState IDLE
- StateChange 27m ttclassic Pod sample2-1 RepAgent Running
- StateChange 27m ttclassic Pod sample2-1 RepState STANDBY
- StateChange 27m ttclassic TimesTenClassic was StandbyDown, now
Normal
- StateChange 3m8s ttclassic TimesTenClassic was Normal, now
ActiveDown
- StateChange 3m1s ttclassic TimesTenClassic was ActiveDown, now
ActiveTakeover
- StateChange 3m1s ttclassic Pod sample2-1 RepState ACTIVE
- StateChange 3m1s ttclassic Pod sample2-1 Database Updatable
- StateChange 2m56s ttclassic TimesTenClassic was ActiveTakeover, now
StandbyDown
- StateChange 113s ttclassic Pod sample2-0 Agent Up
- StateChange 113s ttclassic Pod sample2-0 Release 18.1.4.11.0
- StateChange 113s ttclassic Pod sample2-0 Daemon Up
- StateChange 113s ttclassic Pod sample2-0 Database None
- StateChange 50s ttclassic Pod sample2-0 Database Loaded
- StateChange 50s ttclassic Pod sample2-0 Database Not Updatable
- StateChange 50s ttclassic Pod sample2-0 RepAgent Not Running
- StateChange 50s ttclassic Pod sample2-0 RepState IDLE
- StateChange 44s ttclassic Pod sample2-0 RepAgent Running
- StateChange 44s ttclassic Pod sample2-0 RepState STANDBY
- StateChange 44s ttclassic TimesTenClassic was StandbyDown, now
Normal
You have successfully upgraded to a new release of TimesTen. The active and the standby Pods are running the new TimesTen image, which contains the new TimesTen release. If you need to upgrade additional TimesTenClassic objects, see "Perform an automated upgrade" for an automated upgrade or "Perform a manual upgrade" for a manual upgrade. If the upgrade process is complete for all TimesTenClassic objects, see "Verify the active standby pair of databases are upgraded" to verify the upgrade of all TimesTenClassic objects that are running in your Kubernetes cluster.
In this example, the sample and the sample2 TimesTenClassic objects are now upgraded. The upgrade process is complete. Proceed to "Verify the active standby pair of databases are upgraded" to verify the TimesTenClassic objects are running successfully and have been upgraded.
After the upgrade process is complete for your TimesTenClassic objects, you can verify that each TimesTenClassic object is running successfully and that the active and the standby databases are running the new release of TimesTen.
Use the kubectl get command to verify the Pods are running.
% kubectl get pods NAME READY STATUS RESTARTS AGE sample-0 2/2 Running 0 7h1m sample-1 2/2 Running 0 7h4m sample2-0 2/2 Running 0 4h54m sample2-1 2/2 Running 0 5h22m timestenclassic-operator-846cb5c97c-sbz22 1/1 Running 0 2d8h
Use the kubectl get command to verify the state of each TimesTenClassic object is Normal (represented in bold, in this example).
% kubectl get ttc NAME STATE ACTIVE AGE sample Normal sample-1 2d12h sample2 Normal sample2-1 2d12h
For the sample TimesTenClassic object, use the kubectl exec -it command to invoke a shell in the active Pod (sample-1, in this example). Then, run the ttVersion utility to verify the release is the new release. (18.1.4.11.0, in this example, represented in bold).
% kubectl exec -it sample-1 -c tt -- /usr/bin/su - oracle
% ttVersion
TimesTen Release 18.1.4.11.0 (64 bit Linux/x86_64) (instance1:6624)
2021-04-06T07:34:18Z
Instance admin: oracle
Instance home directory: /tt/home/oracle/instances/instance1
Group owner: oracle
Daemon home directory: /tt/home/oracle/instances/instance1/info
PL/SQL enabled.
Now use the kubectl exec -it command to invoke a shell in the standby Pod (sample-0, in this example). Then, run the ttVersion utility to verify the release is the new release. (18.1.4.11.0, in this example, represented in bold).
% kubectl exec -it sample-0 -c tt -- /usr/bin/su - oracle
% ttVersion
TimesTen Release 18.1.4.11.0 (64 bit Linux/x86_64) (instance1:6624)
2021-04-06T07:34:18Z
Instance admin: oracle
Instance home directory: /tt/home/oracle/instances/instance1
Group owner: oracle
Daemon home directory: /tt/home/oracle/instances/instance1/info
PL/SQL enabled.
For the sample2 TimesTenClassic object, use the kubectl exec -it command to invoke a shell in the active Pod (sample2-1, in this example). Then, use the ttVersion utility to verify the release is the new release (18.1.4.11.0, in this example).
% kubectl exec -it sample2-1 -c tt -- /usr/bin/su - oracle % ttVersion TimesTen Release 18.1.4.11.0 (64 bit Linux/x86_64) (instance1:6624) 2021-04-06T07:34:18Z Instance admin: oracle Instance home directory: /tt/home/oracle/instances/instance1 Group owner: oracle Daemon home directory: /tt/home/oracle/instances/instance1/info PL/SQL enabled.
Now use the kubectl exec -it command to invoke a shell in the standby Pod (sample2-0, in this example). Then, use the ttVersion utility to verify the release is the new release (18.1.4.11.0, in this example).
% kubectl exec -it sample2-0 -c tt -- /usr/bin/su - oracle
% ttVersion
TimesTen Release 18.1.4.11.0 (64 bit Linux/x86_64) (instance1:6624)
2021-04-06T07:34:18Z
Instance admin: oracle
Instance home directory: /tt/home/oracle/instances/instance1
Group owner: oracle
Daemon home directory: /tt/home/oracle/instances/instance1/info
PL/SQL enabled.
The upgrade to a new release of TimesTen is successful for the sample and the sample2 TimesTenClassic objects. The active and the standby Pods for each TimesTenClassic object are running the new TimesTen image, which contains the new TimesTen release.