Creating Solution Designer Cloud Native Instances

4 Creating Solution Designer Cloud Native Instances

This chapter describes how to create a Solution Designer cloud native instance in your cloud environment using the operational scripts and the base configuration provided in the Solution Designer cloud native toolkit.

This procedure is intended to validate that you are able to create a Solution Designer instance in your environment. Before you create an instance, you must download the Solution Designer cloud native tar file and extract the ocscd-cntk.tgz file.

Installing the Solution Designer Artifacts and the Toolkit

Build container image for the Solution Designer application using the Solution Designer cloud native Image Builder.

You must create a private Docker repository for this image, ensuring that all nodes in the cluster have access to the repository. See "About Container Image Management" for more details.

Download the Solution Designer cloud native toolkit archive and do the following:

On Oracle Linux: Where Kubernetes is hosted on Oracle Linux, download and extract the tar archive to each host that has connectivity to the Kubernetes cluster.
On OKE: For an environment where Kubernetes is running in OKE, extract the contents of the tar archive on each OKE client host. The OKE client host is the bastion host/s that is set up to communicate with the OKE cluster.

Set the variable for the installation directory by running the following command, where ocscd_cntk_path is the installation directory of the Solution Designer cloud native toolkit:

export OCSCD_CNTK=ocscd_cntk_path

Solution Designer uses Helm and Ansible to provision the service. To prepare the ocscd_cntk for Ansible and create a Python Virtual Environment with Ansible setup, run the following command:

To create a Python Virtual Environment with Ansible setup:
```
$OCSCD_CNTK/scripts/bootstrap.sh
```
To create a Python Virtual Environment with Ansible setup using proxy or firewall:
```
$OCSCD_CNTK/scripts/bootstrap.sh -p http://my-corporate-proxy
```

Creating the Namespace

After you set the environment variables, you create the namespace that Solution Designer is installed. The namespace requires a namespace administrator service account so Solution Designer services and manages the secrets and configuration maps.

To create namespace, run the following:

kubectl create namespace ocscd

To create the namespace administrator. You can modify the administrator service account by modifying the details in secrets-admin.yaml file.

kubectl -n ocscd apply -f $OCSCD_CNTK/samples/rbac/secrets-admin.yaml

Installing Kafka

Solution Designer requires asynchronous messaging to interact with Solution Designer services. Kafka is used for asynchronous messaging. You can configure a native Kafka broker.

Setup Strimzi Kafka

You can use a native Kafka broker for asynchronous messaging. Strimzi operator provides a way to run an Apache Kafka cluster on Kubernetes in various deployment configurations.

The CNTK provides a sample Strimzi Kafka which is located at $OCSCD_CNTK/samples/strimzi-kafka. Follow the instructions in the README file located at $OCSCD_CNTK/samples/strimzi-kafka/README.md to install sample Strimzi Kafka.

Note:

This sample does not use the PVC backend storage and must be considered for production use.

Setup OCI Streams as Kafka

You can use OCI Streams as Kafka for asynchronous messaging. When setting up OCI Streams, you must setup the following items when creating a stream pool:

Create a private endpoint.
Under advanced options, select auto create topics = true

After you create the Stream Pool, go to Stream Pool Detail and copy the data in Kafka Connection Settings page.

The user account that has created the Stream Pool is the owner of the Pool and you must get an OCI Access Token for that account to use the Stream Pool.

You can use the copied information to update the Kafka details by running the manage-instance-credentials.sh script.

$OCSCD_CNTK/scripts/manage-instance-credentials.sh -i scd update kafka
Kafka End-point (hostname:port): host_name:port
Kafka security protocol (PLAINTEXT,SASL_PLAINTEXT, SASL_SSL, SSL): Security Protocol
SASL mechanism: SASL_Mechanism
SASL connection string: Copy_SASL_Connection_string;

Where

host_name:port is the host name and port address of the Kafka that you copied from the Bootstrap Servers information in the Kafka Connection Settings page.
Security Protocol is the security protocol that you copied in the Kafka Connection Settings page.
SASL_Mechanism is the security mechanism that you copied in the Kafka Connection Settings page.
Copy_SASL_Connection_string is the SASL Connection strings that you copied in the Kafka Connection Settings page. For example:
```
org.apache.kafka.common.security.plain.PlainLoginModule required username="tenancy/user/poolId" password="AUTH_TOKEN";
```

Note:

Replace AUTH_TOKEN with your access token.

Installing UIM Participant PVC

The UIM Participant requires storage space as scratch space for compiling the UIM artifacts and storing the artifacts for download. A basic template example is in $OCSCD_CNTK/samples/pvc.

Perform the following steps:

Create a PVC that references read/write storage which could be block storage or other technology like NFS, Oracle OCI FSS, or the nfs-subdir-external-provisioner. Record the name of the PVC you created.
For NFS, this can be a static PV (Persistent Volume) as provided in the sample README or a dynamic provisioner like Kubernetes nfs-subdir-external-provisioner.
Update the instance specification file to reference the same PVC name that you created:

uim-participant:  
    storageVolume:    
        pvc: pvc_name

Installing OpenID Connect (OIDC) Relying Party

A Relying Party, is an application or website that outsources its user authentication function to an IDP. There are several commercial and non-commercial offerings. The CNTK provides a sample Apache Rely Party which is located at $OCSCD_CNTK/samples/apache-rely-party. Follow the instructions in the README file located at $OCSCD_CNTK/samples/apache-rely-party/README.md to install sample Apache Rely Party.

Installing OpenSearch

Solution Designer uses OpenSearch which is a distributed search and analytics engine. OpenSearch is an elastic data store that is used to search the entities used in the Solution Designer application.

The CNTK provides a sample OpenSearch which is located at $OCSCD_CNTK/samples/opensearch. Follow the instructions in the README file located at $OCSCD_CNTK/samples/opensearch/README.md to install sample OpenSearch.

Note:

This sample does not use the PVC backend storage and must be considered for production use. The samples include an example using Oracle OCI oci-bv storage.

Creating a Solution Designer Cloud Native Instance

This section describes how to create a Solution Designer cloud native instance.

Setting Environment Variables

Solution Designer cloud native relies on access to certain environment variables to run seamlessly. Ensure that the path to your private specification repository variable is set in your environment:

To set the environment variables:

Create a directory that serves as your specification repository, by running the following command, where spec_repo_path is the path to your private specification repository:
```
$ export SPEC_PATH=spec_repo_path
```

Creating Secrets

You must store sensitive data and credential information in the form of Kubernetes Secrets that the scripts and Helm charts in the toolkit consume. Managing secrets is out of the scope of the toolkit and must be implemented while adhering to your organization's corporate policies.

As a prerequisite to using the toolkit for installing the Solution Designer database or creating an instance, you must create secrets for access to the following:

Solution Designer database
Kafka
Authentication
Object Store

The toolkit provides sample scripts for this purpose. However, they are not pipeline-friendly. The scripts should be used for creating an instance manually and quickly, but not for any automated process for creating instances. The scripts also illustrate both the naming of the secret and the layout of the data within the secret that Solution Designer cloud native requires. You must create secrets prior to running the install-db.sh or create-instance.sh scripts.

The following is an example of running the script to create the secrets that you need:

$OCSCD_CNTK/scripts/manage-instance-credentials.sh -i ocscd create db,db-schema,kafka,ocscd-oidc,s3

Where

db specifies the database connectivity details and the administrator credentials for connecting to the Solution Designer database. This is consumed by the Solution Designer database installer and Oracle Communications Service Catalog and Design- Solution Designer.
db-schema specifies the schema credentials that the Solution Designer services use to persist data.
kafka specifies the connectivity details and the credentials for connecting to Kafka provider.
ocscd-oidc specifies OpenID Connect (OIDC) connectivity and credentials for connecting to the OIDC provider for Service Catalog and Design - Solution Designer.
s3 specifies the connectivity details and the credentials for connecting to an S3 Object Store provider.

Assembling the Specifications

To assemble the specifications:

Copy the instance specification to your $SPEC_PATH and rename:

cp $OCSCD_CNTK/samples/instance.yaml $SPEC_PATH/ocscd.yaml

Note:

The file name must match the instance name or the namespace name.

You edit this file as per the instructions described in the sections that follow.

Configuring the Instance Specification

Edit the instance specification to set the container image location and tag in each microservice location. The following is an instance of a few entries:

initiative-manager: {}
#  env:
#    elasticsearch:
#      host: opensearch-cluster-master
#  replicaCount: 1
#  image:
#    repository: ocscd/initiative-manager
#    tag: latest
#    pullSecrets: []
#  db: 
#    type: "STANDARD" # Acceptable values are STANDARD and ADB
workspace-manager: {}
#  replicaCount: 1
#  image:
#    repository: ocscd/workspace-manager
#    tag: latest
#    pullSecrets: []
#  db: 
#    type: "STANDARD" # Acceptable values are STANDARD and ADB
landing-page-api: {}
#  replicaCount: 1
#  image:
#    repository: ocscd/landing-page-api
#    tag: latest
#    pullSecrets: []
#  db: 
#    type: "STANDARD" # Acceptable values are STANDARD and ADB

Note:

YAML formatting is case-sensitive. While the next step uses vi editor for editing, if you are not familiar with editing YAML files, use a YAML editor to ensure that you do not make any syntax errors while editing. Follow the indentation guidelines for YAML, as incorrect spacing can lead to errors.

If your environment requires a password to download the container images from your repository, create a Kubernetes secret with the image pull credentials. See the Kubernetes documentation for details. Reference the secret name in the instance specification.

# uncomment and set if required
#  image:
#    pullSecrets: []
#      - name: my-regcred

By default, Solution Designer is configured to use a PDB for its database.


#  db: 
#   type: "STANDARD" # Acceptable values are STANDARD and ADB

If you choose a tablespace name that is different from the default name USERS, then you must enter the new name in the instance specification.

#  db:
    #  type: "STANDARD"
    #  defaultTablespace: "USERS"
    #  tempTablespace: "TEMP"

Installing the Schemas

This procedure configures an empty PDB or ADB. Depending on the database strategy for your team, you may have already performed this procedure as described in "Provisioning Oracle Multitenant Container Database (CDB)". Before continuing, ensure that the PDB or ADB being used for creating the Solution Designer instance includes the schema installation. If the database already has the schema installed, skip this procedure.

After the PDB or ADB is created, it is configured with the Solution Designer schema.

Run the following to install the Solution Designer schema.

$OCSCD_CNTK/scripts/install-db.sh -i ocscd -s $SPEC_PATH -c 4

Create a Solution Designer Instance

This procedure describes how to create a Solution Designer instance in your environment using the scripts that are provided with the toolkit.

To create a Solution Designer instance:

Run the following command:
```
$OCSCD_CNTK/scripts/create-instance.sh -i ocscd -s $SPEC_PATH
```
The create-instance.sh script creates and deploys the microservices that are specified in the instance specification file. If the scripts fails, see "Troubleshooting Issues with the Scripts", before you make additional attempts.
If you query the status of the pods, the READY state of the microservices may display 0/1 for several minutes while the Solution Designer application is starting.
When the READY state shows 1/1, your Solution Designer instance is up and running. You can then validate the instance by running the Oracle Communications Service Catalog and Design - Solution Designer application and creating a sample PSR model.

Validating the Solution Designer Instance

After creating an instance, you can validate it by checking the domain configuration and the client UIs.

Run the following command to display the deployment details of the Solution Designer instance that you have created:

kubectl get deployment -n ocscd

The command displays the domain configuration information.

To verify the client UIs, log into the Service Catalog and Design - Solution Designer Web client user interface with the service catalog administrator login credentials created in IAM using the URL:

http://hostname:port/apps/scd/

Where

hostname is the Solution Designer host name.
port is the port number where Solution Designer is installed.

Note:

After a Solution Designer instance is created, it may take a few minutes for the user interface to become active.

Scaling the Solution Designer Application Cluster

You can create multiple instances of the microservices components of Solution Designer by setting the replicaCount accordingly in the MANIFEST.yaml file while creating the Solution Designer instance and during the upgrade. By default, the replicaCount is set to 1. You cannot modify the replicaCount for UIM-Participant component which is set to 1, by default.

To scale the Solution Designer instances, edit the MANIFEST.YAML file.

initiative-manager:  
        replicaCount: 1

workspace-manager:   
        replicaCount: 1

landing-page-api:   
        replicaCount: 1

The following guidelines are for the high availability of the third party deployments such as OpenSearch, Kafka. These guidelines can be altered depending on the desired resiliency.

Guidelines for third party deployments:

OpenSearch can have minimum 3 replicas and have backend PVC storage.
Kafka can have minimum 3 replicas and backend PVC storage.
For database, you can create snapshots, backups, or standby databases for high availability.
For OIDC Relying Party, a single replica is sufficient. If the Relying Party implementation supports replicas, you can add more replicas based on your business needs and the technology chosen.

Deleting and Recreating Your Solution Designer Instance

Deleting Your Solution Designer Instance

To delete your Solution Designer instance, run the following command:

$OCSCD_CNTK/scripts/delete-instance.sh -i ocscd

Recreating Your Solution Designer Instance

When you delete a Solution Designer instance, the database state for that instance still remains unaffected. You can re-create the Solution Designer instance by using the same container image.

To re-create an instance, run the following command:

$OCSCD_CNTK/scripts/create-instance.sh -i ocscd -s $SPEC_PATH

Cleaning Up the Environment

To clean up the environment:

Delete the instance:

$OCSCD_CNTK/scripts/delete-instance.sh -i ocscd

Delete OpenSearch, Kafka, Rely Party, PVC/PV
Delete the namespace, which in turn deletes the Kubernetes namespace and the secrets:
```
kubectl delete namespace ocscd
```
Drop the PDB or ADB.

Troubleshooting Issues with the Scripts

This section provides information about troubleshooting some issues that you may come across when running the scripts.

Timeout Issue

While running install-db.sh script, the install fails with a timeout.

fatal: [local]: FAILED! => {"changed": true, "duration": 124, "method": "create", "msg": "\"Job\" \"pre-deploy-initiative-manager-semele\": Timed out waiting on resource"

To resolve this issue, observe the Kubernetes Pod logs for the Kubernetes Job pre-deploy-initiative-manager-semele mentioned in the error message to determine the database error. Mostly, it is related to the connectivity from the Pod to the database or incorrect credentials. Check the connectivity from the pod to the database and provide correct credentials.

Recreating an Instance

If you face issues when creating an instance, do not try to re-run the create-instance.sh script as this will fail. Instead, perform the cleanup activities and then run the following command:

$OCSCD_CNTK/scripts/create-instance.sh -i ocscd -s $SPEC_PATH

Cleanup Failed Instance

When a create-instance script fails, you must clean up the instance before making another attempt at instance creation.

To clean up the failed instance, delete the instance:

$OCSCD_CNTK/scripts/delete-instance.sh -i ocscd

Pre-deploy Initiative Manager Failure

The pre-deploy-initiative-manager-semele fails with the following log:

com.oracle.communications.semele.cli.SemeleCli getExitCode   
SEVERE: Execution failed:
com.oracle.communications.semele.DbManagerException: java.sql.SQLException: ORA-28009: connection as SYS should be as SYSDBA or SYSOPER

You must check if the database admin user has the required permissions.

When running manage-instance-credentials to give the database details, add 'as sysdba' to the database admin username.

Example:

OCSCD DB Admin(sys) Username: sys as sysdba

Post-Installation Failure

You may receive the following error in the post-installation process:

Error: failed post-install: job failed: BackoffLimitExceeded

When you receive the errors, check the microservice in the Kubernetes namespace.

To see what pod has failed, run kubectl get pods.

Run kubectl describe PodName on the pod.

Run kubectl logs PodNameon the pod.

Post-Installation Workspace Manager Failure

The post-install-workspace-manager may fail with the error,

Participants could not be configured. HTTP return code 401.

You can use one of the following two methods to fix the error:

Remove any self-signed certificates as self-signed certificates are not supported in Solution Designer.

If there is an issue with the OIDC parameters or the OIDC provider, connect to the workspace-manager-pod and find out what error occurs.

Run the following command to connect to the workspace-manager-pod:

kubectl -n namespace exec -ti workspace-manager-pod -- bash

After you connect to the workspace-manager-pod, run the following script to find out what error has occurred:

real_scope=${LCM_OPERATION_SCOPE}
base64_data=$(echo -n "$OIDC_CLIENT_ID:$OIDC_CLIENT_SECRET" | base64 -w 0)
header1="Authorization: Basic "$base64_data
header2="Content-Type: application/x-www-form-urlencoded;charset=UTF-8"
data1="grant_type=client_credentials"
curl -s -H "$header1" -H "$header2" --request POST "$OIDC_TOKEN_ENDPOINT" -d "$data1" -d "scope=$real_scope"

Creating New Connection

After you create the Solution Designer instance, in the Solution Designer Web client user interface, create a new connection in the Workspace application to publish the initiatives to the UIM Participant.

See "Creating New Connection" in Solution Designer User's Guide on how to create a new connection.

Next Steps

The Solution Designer instance is ready for use. To understand details on infrastructure setup and structuring of the instances for your organization, proceed to Planning Infrastructure.