Setup Steps
Topics in this section:
- Set up your Kubernetes cluster
- Prepare the environment
- Create a database and the appropriate database services
- Install and configure Oracle HTTP Server in the DMZ
- Configure a front-end load balancer
- Apply operating system changes for Coherence
- Run the Repository Creation Utility to set up your database schemas
- Deploy WebLogic Kubernetes Operator and Oracle SOA Suite
- Configure a custom keystore for Oracle SOA Suite domains
- Using Custom Certificates in Oracle HTTP Servers
- Configure a redundant persistent volume
- Set front-end addresses
- Enable FAN for GridLink data sources
- Configure ASM
- Configure CoreDNS allocation
- Adjust the liveness probe for server pods
Set up your Kubernetes cluster
Prepare the environment for the Kubernetes control plane (master nodes)
-
Create the L4/TCP listener for the load balancer (LBR).
-
Create the LBR backend pool with the list of control plane nodes that will be added (use hostnames rather than IP addresses).
Note:
We recommend maintaining the values of the followingkube-apibackend pool parameters within the prescribed range to minimize downtime while restarting the Kubernetes control plane or performing maintenance operations.- Health check interval: Within 1000 milliseconds
- Health check timeout: Within 900 milliseconds
-
Enable the L4 LBR to route to the backend set/pool.
Note:
It is important that this is an L4/TCP listener, not an HTTP/HTTPS listener. -
Make sure that the nodes are in the ready state.
-
Create an SSH key (use a common SSH key to enable access from the node executing the setup to the control plane nodes).
-
Allow traffic through intermediate firewalls between control plane nodes and the front-end LBR. See Ports and Protocols for the required ports.
- Create the HA cluster with 3 master and 3 minion nodes. For setup instructions, see Creating Highly Available Clusters with kubeadm using the Stacked etcd Topology.
Prepare the environment
Configure firewalls and network
- Allow traffic from the load balancer (LBR) to the Oracle HTTP Server (OHS) port that will be configured (7777 by default for OHS).
- Allow traffic from the OHS to the node ports that will be configured on the worker nodes for the Administration Server (30701), SOA cluster (30801), and Service Bus cluster (30901).
- Allow traffic from the worker nodes to the control plane front-end
kube-apivirtual server port and also to the front end for Oracle SOA Suite. - Allow traffic from worker nodes to the database listener and ONS port (1521 and 6200 by default, respectively).
You can use Preparing the Load Balancer and Firewalls for an Enterprise Deployment in the on-premises Enterprise Deployment Guide for Oracle SOA Suite as a reference.
Load Oracle SOA Suite images on all the worker nodes
See Obtain the Oracle SOA Suite Docker image for instructions on loading the images on each worker node and tagging them appropriately.
Enable a shared storage location for the persistent volume
The worker nodes must use a shared storage device. This storage hosts the Oracle SOA Suite domain directory. Initially, a single storage location is used to create a persistent volume that will host the Oracle SOA Suite domain. Mount this shared storage (NFS/NAS) from all the worker nodes using the same mount path on all of them.
For example, mount NFS1 (10.10.0.21:/k8nfs) on all the worker nodes at the shared directory /k8nfs:
$ grep "k8nfs nfs" /etc/fstab
10.10.0.21:/k8nfs /k8nfs nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.0.21,mountvers=3,mountport=2048,mountproto=udp,local_lock=none,addr=10.10.0.21
Later, steps are provided to configure a second storage location for high availability.
Create a database and the appropriate database services
Installing and creating the RAC database is outside the scope of this document. Once the database is configured, the appropriate services must be created to access the schemas from the middle tier. It is critical that a dedicated non-default/administration service is created for Oracle SOA Suite. See Preparing the Database for an Enterprise Deployment in the Enterprise Deployment Guide for Oracle SOA Suite 14.1.2.0.0.
Install and configure Oracle HTTP Server in the DMZ
Follow the steps in Installing and Configuring Oracle HTTP Server to create two Oracle HTTP Server (OHS) instances on nodes separate from the worker nodes. To configure OHS with the back-end Kubernetes Oracle SOA Suite or Service Bus servers, you must use a port in the range of 30000 - 32767. You will use this port in the Oracle SOA Suite configuration scripts later on.
In this Kubernetes Enterprise Deployment Guide, OHS routes to node ports configured for each separate Oracle SOA Suite/Service Bus cluster in the SOA domain. OHS then routes to these node ports, which redirect to the corresponding server pods. The OHS configuration must disable DynamicServerList because the node ports are not WebLogic listeners, and the node port configuration maintains an intelligent list of available WebLogic servers. The OHS directive for the soa-infra mount looks like this:
<Location /soa-infra>
WLSRequest ON
WebLogicCluster workernode1:30801,workernode2:30801,workernode3:30801
</Location>
Similarly, directives for other paths should use the corresponding node port addresses.
Configure a front-end load balancer
You can use either an F5 BIG-IP LBR or another standard LBR, such as Cisco. For the required virtual servers, see Preparing the Load Balancer and Firewalls for an Enterprise Deployment. The on-premises Enterprise Deployment Guide provides a detailed list of virtual servers/listeners that can be used for optimum isolation of services and traffic. For Kubernetes, at a minimum, you should have a virtual server/listener for Oracle SOA Suite using the OHS listeners as the back-end pool.
-
Create the load balancer's L7/HTTPS listener.
-
Create a back-end pool with the list of OHS nodes/ports that will be used by Oracle SOA Suite (use hostnames rather than IP addresses).
-
Enable the load balancer's L7/HTTPS listener to route to the OHS back-end set/pool.
-
Configure the front-end load balancer to route to the OHS pool.
Note:
Oracle recommends that you configure the LBR for cookie-based persistence because session persistence is required for some SOA web applications, such as BPM Worklist (/integration/worklistapp) and SOA Composer (/soa/composer).
Apply operating system changes for Coherence
Coherence requires specific settings to create clusters in a Kubernetes environment. See Coherence Requirements in the WebLogic Kubernetes Operator documentation.
Run the Repository Creation Utility to set up your database schemas
To create the database schemas for Oracle SOA Suite, run the create-rcu-schema.sh script with SOA_PROFILE_TYPE=LARGE. For example:
./create-rcu-schema.sh \
-s SOA1 \
-t soaosb \
-d oracle-db.default.svc.cluster.local:1521/devpdb.k8s \
-n default \
-c oracle-rcu-secret \
-b EBR \
-i soasuite:release-version \
-r SOA_PROFILE_TYPE=LARGE,HEALTHCARE_INTEGRATION=NO
Deploy WebLogic Kubernetes Operator and Oracle SOA Suite
Follow the steps in Install the WebLogic Kubernetes Operator, Prepare the environment for Oracle SOA Suite domains, and Create Oracle SOA Suite domains.
After creating the Oracle SOA Suite domain and starting the servers, check the pods and the services created. Once the Oracle SOA Suite managed servers reach the RUNNING state (the pods are ready), check typical Oracle SOA Suite URLs using the front-end load balancer.
Create NodePort services for the Administration Server, SOA cluster, and OSB cluster with
sessionAffinity set to ClientIP. The following example shows a NodePort
service for the SOA cluster:
$ cat soa-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
name: soainfra-soa-nodeport-service
namespace: soans
labels:
weblogic.domainUID: soainfra
weblogic.clusterName: soa_cluster
spec:
type: NodePort
sessionAffinity: ClientIP
ports:
- port: 7004
protocol: TCP
targetPort: 7004
nodePort: 30234
selector:
weblogic.domainUID: soainfra
weblogic.clusterName: soa_cluster
SOA pods and services deployed and ready:
$ kubectl get all -n soans
NAME READY STATUS RESTARTS AGE
pod/soaedgdomain-adminserver 1/1 Running 0 47h
pod/soaedgdomain-create-soa-infra-domain-job-6pq9z 0/1 Completed 0 68d
pod/soaedgdomain-soa-server1 1/1 Running 0 2d2h
pod/soaedgdomain-soa-server2 1/1 Running 0 2d2h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/soaedgdomain-adminserver ClusterIP None <none> 30012/TCP,7001/TCP 2d4h
service/soaedgdomain-adminserver-ext NodePort 10.104.20.22 <none> 30012:30012/TCP,7001:30701/TCP 31d
service/soaedgdomain-cluster-osb-cluster ClusterIP 10.100.97.127 <none> 8002/TCP 68d
service/soaedgdomain-cluster-soa-cluster ClusterIP 10.101.101.113 <none> 7003/TCP 68d
service/soaedgdomain-cluster-soa-cluster-node-port NodePort 10.105.51.223 <none> 7003:30801/TCP 68d
service/soaedgdomain-osb-server1 ClusterIP 10.110.81.153 <none> 8002/TCP 2d4h
service/soaedgdomain-osb-server2 ClusterIP 10.103.220.112 <none> 8002/TCP 2d4h
service/soaedgdomain-osb-server3 ClusterIP 10.97.50.117 <none> 8002/TCP 2d4h
service/soaedgdomain-osb-server4 ClusterIP 10.98.48.247 <none> 8002/TCP 2d4h
service/soaedgdomain-osb-server5 ClusterIP 10.102.137.176 <none> 8002/TCP 2d4h
service/soaedgdomain-soa-server1 ClusterIP None <none> 7003/TCP 2d4h
service/soaedgdomain-soa-server2 ClusterIP None <none> 7003/TCP 2d4h
service/soaedgdomain-soa-server3 ClusterIP 10.105.108.74 <none> 7003/TCP 2d4h
service/soaedgdomain-soa-server4 ClusterIP 10.109.191.102 <none> 7003/TCP 2d4h
service/soaedgdomain-soa-server5 ClusterIP 10.107.2.99 <none> 7003/TCP 2d4h
NAME COMPLETIONS DURATION AGE
job.batch/soaedgdomain-create-soa-infra-domain-job 1/1 4m24s 68d
Configure a custom keystore for Oracle SOA Suite domains
Run the helper script custom-keystore.sh available at
${WORKDIR}/custom-keystore to configure a custom keystore for
Oracle SOA Suite domains.
See Replacing Demo CA Certificates With Domain CA Signed Certificates.
Using Custom Certificates in Oracle HTTP Servers
Copy trust.p12 (created with custom-keystore.sh) to OHS. The following commands manage keys and wallets in OHS.
Export a certificate from a Java KeyStore (JKS) or PKCS12 keystore file.
$ keytool -export -keystore trust.p12 -storepass trustKeyStorePassword -alias trustcert1 -file trust.crt
Certificate stored in file <trust.crt>
Create a wallet for OHS:
$ orapki wallet create -wallet WLSSLWalletEDG -auto_login -pwd *****
Oracle PKI Tool Release 23.0.0.0.0 - Production
Version 23.0.0.0.0
Copyright (c) 2004, 2024, Oracle and/or its affiliates. All rights reserved.
Operation is successfully completed.
Add the trusted certificate to the wallet:
$ orapki wallet add -wallet WLSSLWalletEDG -trusted_cert -cert trust.crt -pwd *****
Oracle PKI Tool Release 23.0.0.0.0 - Production
Version 23.0.0.0.0
Copyright (c) 2004, 2024, Oracle and/or its affiliates. All rights reserved.
Operation is successfully completed.
Modify $WEB_DOMAIN_HOME/config/fmwconfig/components/OHS/ohs1/mod_wl_ohs.conf to include the appropriate WLSSLWallet file (required to route over SSL to the WLS backends) as follows:
# NOTE : This is a template to configure mod_weblogic.
LoadModule weblogic_module "${PRODUCT_HOME}/modules/mod_wl_ohs.so"
# This empty block is needed to save mod_wl related configuration from EM to this file when changes are made at the Base Virtual Host Level
<IfModule weblogic_module>
WLIOTimeoutSecs 900
KeepAliveSecs 290
FileCaching OFF
WLSocketTimeoutSecs 15
ErrorPage http://www.oracle.com/splash/cloud/index.html
WLRetryOnTimeout NONE
WLForwardUriUnparsed On
SecureProxy On
WLSSLWallet "/u02/oracle/config/keystores/WLSSLWalletEDG"
</IfModule>
Configure a redundant persistent volume
To increase flexibility when moving Oracle SOA Suite or Service Bus pods within the Kubernetes cluster, use node selectors so that odd server pods (soa_server1, soa_server3, soa_server5, and so on) are assigned to node selector 1 and even server pods (soa_server2, soa_server4, soa_server6, and so on) are assigned to node selector 2. The resulting configuration is:
Figure 9-2 Configure a redundant persistent volume

To use this configuration, follow these steps:
-
Stop the Oracle SOA Suite domain. See Scripts to start and stop a domain.
-
Mount NFS1 on all even worker nodes and NFS2 on all odd worker nodes as in the diagram above. For example:
MOUNT ON ODD NODE [opc@olk8-w1 ~]$ grep "k8nfs nfs" /etc/fstab 10.10.0.21:/k8nfs /k8nfs nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.0.21,mountvers=3,mountport=2048,mountproto=udp,local_lock=none,addr=10.10.0.21 MOUNT ON EVEN NODE [opc@olk8-w2 ~]$ grep "k8nfs nfs" /etc/fstab 10.10.0.27:/k8nfs2 /k8nfs nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.0.27,mo untvers=3,mountport=2048,mountproto=udp,local_lock=none,addr=10.10.0.27 MOUNT ON ODD NODE [opc@olk8-w3 ~]$ grep "k8nfs nfs" /etc/fstab 10.10.0.21:/k8nfs /k8nfs nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.0.21,mountvers=3, mountport=2048,mountproto=udp,local_lock=none,addr=10.10.0.21 - Copy the domain mount to an NFS replica, NFS2 (this can be done through a
snapshot or through SFTP/secure copy).
For example, if the domain is deployed in /k8nfs hosted by NFS1, stop the domain, then copy the data securely from /k8nfs on NFS1 to /k8nfs2 on NFS2:
$ cd /k8nfs $ scp -R * user@[NFS2]:/k8nfs2 - Label the odd nodes for NFS1 and the even nodes for NFS2.
For example, add the label diskgroup=dg1 for NFS1 and diskgroup=dg2 for NFS2:
$ kubectl label nodes olk8-w1 diskgroup=dg1 $ kubectl label nodes olk8-w2 diskgroup=dg2 $ kubectl label nodes olk8-w3 diskgroup=dg1Verify the added labels using the following command:
$ kubectl get nodes --show-labelsSample output:
NAME STATUS ROLES AGE VERSION LABELS olk8-m1 Ready master 10d v1.XX.X beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=olk8-m1,kubernetes.io/os=linux,node-role.kubernetes.io/master= olk8-m2 Ready master 10d v1.XX.X beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=olk8-m2,kubernetes.io/os=linux,node-role.kubernetes.io/master= olk8-m3 Ready master 10d v1.XX.X beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=olk8-m3,kubernetes.io/os=linux,node-role.kubernetes.io/master= olk8-w1 Ready <none> 10d v1.XX.X beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,diskgroup=dg1,kubernetes.io/arch=amd64,kubernetes.io/hostname=olk8-w1,kubernetes.io/os=linux,name=admin olk8-w2 Ready <none> 10d v1.XX.X beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,diskgroup=dg2,kubernetes.io/arch=amd64,kubernetes.io/hostname=olk8-w2,kubernetes.io/os=linux,name=wls1 olk8-w3 Ready <none> 10d v1.XX.X beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,diskgroup=dg1,kubernetes.io/arch=amd64,kubernetes.io/hostname=olk8-w3,kubernetes.io/os=linux,name=wls2 - To assign the appropriate selectors in the domain:
- Edit the domain file
(domain.yaml). - Update the managed servers section for all managed servers configured
in the cluster. The following example shows soa_server1 and soa_server2:
managedServers: - serverName: soa_server1 serverPod: nodeSelector: diskgroup: dg1 - serverName: soa_server2 serverPod: nodeSelector: diskgroup: dg2 - Apply the domain.yaml
changes:
$ kubectl apply -f domain.yamlNote:
Once this redundant PV configuration is in use, all changes outside the config directory in the domain must be copied/synchronized to the secondary NAS mount manually. The managed servers using NFS2 in the diagram above replicate only configuration changes that modify files/artifacts under the$DOMAIN_HOME/configdirectory. Other changes are not copied automatically by the WebLogic infrastructure.For example, if you deploy an
earand specify an upload or stage directory outside theconfigdirectory, theearfiles will not be copied by WebLogic.Fileadapter compositesplace output files in mounts accessible from the pods. The mount point and PV/PVC for the different Oracle SOA Suite server file locations must be the same and must differ from those used for the$DOMAIN_HOMElocation.For adapter configuration, use a separate shared location accessible to all server pods. This location must differ from $DOMAIN_HOME. See Persist Adapter Customizations.
For example, create a new NFS share, create the required PV/PVC, and update the domain YAML so that the share is available to all server pods.
serverPod: volumeMounts: - mountPath: /u01/oracle/user_projects name: weblogic-domain-storage-volume - mountPath: /u01/dev/testing/adapters name: soainfra-adapters-pv-volume volumes: - name: weblogic-domain-storage-volume persistentVolumeClaim: claimName: soainfra-pvc - name: soainfra-adapters-pv-volume persistentVolumeClaim: claimName: soainfra-adapters-pvc
- Edit the domain file
Set front-end addresses
For each of the Oracle SOA Suite (soa_cluster) and Service Bus (osb_cluster) clusters, set the appropriate front-end address as follows:
-
Sign in to the WebLogic Remote Console.
-
Navigate to Edit Tree -> environment -> clusters.
-
Click the cluster name.
-
Click the HTTP tab.
-
Enter the Frontend Host and Frontend Port details.
Note:
Set the front-end details for each cluster and for the Administration Server.
Enable FAN for GridLink data sources
Data sources in the default deployment for Oracle SOA Suite on Kubernetes are generic data sources. With ONS auto-registration in Database 12.2 and later, you only need to enable FAN for the data sources to convert them to GridLink (GLDS). You can do this by using the WebLogic Administration Console or the following command, then restarting the Administration Server and the managed servers:
grep -L fan-enabled $domain_home/config/jdbc/*.xml | xargs sed -i "s/<\/jdbc-data-source>/<jdbc-oracle-params><fan-enabled>true<\/fan-enabled><\/jdbc-oracle-params><\/jdbc-data-source>/g"
After the change is applied, verify that all data sources are marked as GridLink Data Sources in the WebLogic Remote Console.
Configure ASM
See Using Whole Server Migration and Service Migration in an Enterprise Deployment in the on-premises Enterprise Deployment Guide for steps to configure Automatic Service Migration.
Configure CoreDNS allocation
Note:
This step applies to any Kubernetes system usingcoredns, regardless of whether Oracle SOA Suite is deployed. However,
worker node creation is implicit in this setup, so this step is performed
after Oracle SOA Suite deployment.
Configure replicas for coredns to run on both the control plane and
master plane. If a restore operation is performed on the control plane, worker nodes
may stop working properly. If all coredns pods run on
worker nodes, the control plane may not function correctly if those nodes are brought down
for maintenance. Place at least two coredns pods on the control
plane and another two on the worker nodes. The coredns footprint is
low.
VIRT RES SHR S %CPU %MEM TIME+ COMMAND
146268 41684 29088 S 0.3 0.1 25:44.04 coredns
According to Scaling CoreDNS, you can estimate the amount of memory required for a CoreDNS instance (using default settings) with the following formula:
MB required (default settings) = (Pods + Services) / 1000 + 54
First, label nodes in both the control and worker planes.
$ kubectl label nodes olk8-m1 area=dnsarea
$ kubectl label nodes olk8-m2 area=dnsarea
$ kubectl label nodes olk8-m3 area=dnsarea
$ kubectl label nodes olk8-w1 area=dnsarea
$ kubectl label nodes olk8-w2 area=dnsarea
$ kubectl label nodes olk8-w3 area=dnsarea
Then update the coredns deployment to use topology spread constraints.
Note:
Topology spread constraints are beta starting in Kubernetes v1.18.First, enable the feature gate in the kube-api server and in
kube-scheduler. Then, modify the coredns
deployment to distribute pods appropriately across worker and control plane
nodes.
The following example shows the coredns topology spread configuration:
Example of the updated coredns deployment YAML:
$ kubectl get deployment coredns -n kube-system -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "7"
creationTimestamp: "2021-01-15T13:15:05Z"
generation: 8
labels:
area: dnsarea
k8s-app: kube-dns
managedFields:
- apiVersion: apps/v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:labels:
.: {}
f:k8s-app: {}
f:spec:
f:progressDeadlineSeconds: {}
f:revisionHistoryLimit: {}
f:selector:
f:matchLabels:
.: {}
f:k8s-app: {}
f:strategy:
f:rollingUpdate:
.: {}
f:maxSurge: {}
f:maxUnavailable: {}
f:type: {}
f:template:
f:metadata:
f:labels:
.: {}
f:k8s-app: {}
f:spec:
f:containers:
k:{"name":"coredns"}:
.: {}
f:args: {}
f:image: {}
f:imagePullPolicy: {}
f:livenessProbe:
.: {}
f:failureThreshold: {}
f:httpGet:
.: {}
f:path: {}
f:port: {}
f:scheme: {}
f:initialDelaySeconds: {}
f:periodSeconds: {}
f:successThreshold: {}
f:timeoutSeconds: {}
f:name: {}
f:ports:
.: {}
k:{"containerPort":53,"protocol":"TCP"}:
.: {}
f:containerPort: {}
f:name: {}
f:protocol: {}
k:{"containerPort":53,"protocol":"UDP"}:
.: {}
f:containerPort: {}
f:name: {}
f:protocol: {}
k:{"containerPort":9153,"protocol":"TCP"}:
.: {}
f:containerPort: {}
f:name: {}
f:protocol: {}
f:readinessProbe:
.: {}
f:failureThreshold: {}
f:httpGet:
.: {}
f:path: {}
f:port: {}
f:scheme: {}
f:periodSeconds: {}
f:successThreshold: {}
f:timeoutSeconds: {}
f:resources:
.: {}
f:limits:
.: {}
f:memory: {}
f:requests:
.: {}
f:cpu: {}
f:memory: {}
f:securityContext:
.: {}
f:allowPrivilegeEscalation: {}
f:capabilities:
.: {}
f:add: {}
f:drop: {}
f:readOnlyRootFilesystem: {}
f:terminationMessagePath: {}
f:terminationMessagePolicy: {}
f:volumeMounts:
.: {}
k:{"mountPath":"/etc/coredns"}:
.: {}
f:mountPath: {}
f:name: {}
f:readOnly: {}
f:dnsPolicy: {}
f:nodeSelector:
.: {}
f:kubernetes.io/os: {}
f:priorityClassName: {}
f:restartPolicy: {}
f:schedulerName: {}
f:securityContext: {}
f:serviceAccount: {}
f:serviceAccountName: {}
f:terminationGracePeriodSeconds: {}
f:tolerations: {}
f:volumes:
.: {}
k:{"name":"config-volume"}:
.: {}
f:configMap:
.: {}
f:defaultMode: {}
f:items: {}
f:name: {}
f:name: {}
manager: kubeadm
operation: Update
time: "2021-01-15T13:15:05Z"
- apiVersion: apps/v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:labels:
f:area: {}
f:spec:
f:replicas: {}
f:template:
f:metadata:
f:annotations:
.: {}
f:kubectl.kubernetes.io/restartedAt: {}
f:labels:
f:foo: {}
f:spec:
f:topologySpreadConstraints:
.: {}
k:{"topologyKey":"area","whenUnsatisfiable":"DoNotSchedule"}:
.: {}
f:labelSelector:
.: {}
f:matchLabels:
.: {}
f:foo: {}
f:maxSkew: {}
f:topologyKey: {}
f:whenUnsatisfiable: {}
manager: kubectl
operation: Update
time: "2021-01-28T16:00:21Z"
- apiVersion: apps/v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.: {}
f:deployment.kubernetes.io/revision: {}
f:status:
f:availableReplicas: {}
f:conditions:
.: {}
k:{"type":"Available"}:
.: {}
f:lastTransitionTime: {}
f:lastUpdateTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Progressing"}:
.: {}
f:lastTransitionTime: {}
f:lastUpdateTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
f:observedGeneration: {}
f:readyReplicas: {}
f:replicas: {}
f:updatedReplicas: {}
manager: kube-controller-manager
operation: Update
time: "2021-01-28T16:00:39Z"
name: coredns
namespace: kube-system
resourceVersion: "2520507"
selfLink: /apis/apps/v1/namespaces/kube-system/deployments/coredns
uid: 79d24e61-98f4-434f-b682-132625b04c49
spec:
progressDeadlineSeconds: 600
replicas: 4
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kube-dns
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
annotations:
kubectl.kubernetes.io/restartedAt: "2021-01-28T15:29:48Z"
creationTimestamp: null
labels:
foo: bar
k8s-app: kube-dns
spec:
containers:
- args:
- -conf
- /etc/coredns/Corefile
image: k8s.gcr.io/coredns:1.6.7
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 5
httpGet:
path: /health
port: 8080
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: coredns
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
- containerPort: 9153
name: metrics
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /ready
port: 8181
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
limits:
memory: 170Mi
requests:
cpu: 100m
memory: 70Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- NET_BIND_SERVICE
drop:
- all
readOnlyRootFilesystem: true
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/coredns
name: config-volume
readOnly: true
dnsPolicy: Default
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-cluster-critical
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: coredns
serviceAccountName: coredns
terminationGracePeriodSeconds: 30
tolerations:
- key: CriticalAddonsOnly
operator: Exists
- effect: NoSchedule
key: node-role.kubernetes.io/master
topologySpreadConstraints:
- labelSelector:
matchLabels:
foo: bar
maxSkew: 1
topologyKey: area
whenUnsatisfiable: DoNotSchedule
volumes:
- configMap:
defaultMode: 420
items:
- key: Corefile
path: Corefile
name: coredns
name: config-volume
status:
availableReplicas: 4
conditions:
- lastTransitionTime: "2021-01-21T19:08:12Z"
lastUpdateTime: "2021-01-21T19:08:12Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
- lastTransitionTime: "2021-01-28T15:29:48Z"
lastUpdateTime: "2021-01-28T16:00:39Z"
message: ReplicaSet "coredns-84b49c57fd" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
observedGeneration: 8
readyReplicas: 4
replicas: 4
updatedReplicas: 4
The labels and topology spread changes are:
labels:
foo: bar
k8s-app: kube-dns
topologySpreadConstraints:
- labelSelector:
matchLabels:
foo: bar
maxSkew: 1
topologyKey: area
whenUnsatisfiable: DoNotSchedule
This ensures an even distribution across the master and worker nodes so that if the control plane is restored, the worker pods continue without issues, and vice versa.
Example of the resulting coredns distribution:
$ kubectl get pods -A -o wide | grep coredns
kube-system coredns-84b49c57fd-4fz4g 1/1 Running 0 166m 10.244.1.20 olk8-m2 <none> <none>
kube-system coredns-84b49c57fd-5mrkw 1/1 Running 0 165m 10.244.4.76 olk8-w2 <none> <none>
kube-system coredns-84b49c57fd-5zm88 1/1 Running 0 165m 10.244.2.17 olk8-m3 <none> <none>
kube-system coredns-84b49c57fd-nqlwb 1/1 Running 0 166m 10.244.4.75 olk8-w2 <none> <none>
Adjust the liveness probe for server pods
By default, the liveness probe checks liveness every 45
seconds, which might cause requests to be routed to backend pods that are no longer
available during outages. We recommend adjusting the liveness probe values so
that pods are marked as down faster after hard node failures. To configure a more
aggressive probe, edit the domain and change the
serverPods.livenessProbe values to the following:
livenessProbe:
failureThreshold: 1
initialDelaySeconds: 60
periodSeconds: 40
successThreshold: 1
timeoutSeconds: 5
See Liveness Probe Customization in the WebLogic Kubernetes Operator documentation for details on how to customize the liveness probe.