50 Quick Start

Deploy Coherence RAG using a release-matched coherence-rag-server container image to Kubernetes using Coherence Operator.

For example, to deploy a 3-member Coherence RAG cluster that uses a built-in all-MiniLM-L6-v2 embedding model, and Open AI gpt-4o-2024-08-06 chat model you could use the following deployment YAML:

apiVersion: coherence.oracle.com/v1
kind: Coherence
metadata:
  name: coherence-rag-demo
spec:
  replicas: 3
  # Replace with the approved 26.1.0.0.0 image built/published for your deployment.
  image: registry.example.com/coherence-rag-server:26.1.0.0.0
  cluster: coherence-rag-demo
  env:
    - name: MODEL_EMBEDDING
      value: -/all-MiniLM-L6-v2
    - name: MODEL_CHAT
      value: OpenAI/gpt-4o-2024-08-06
    - name: OPENAI_API_KEY
      valueFrom:
        secretKeyRef:
          name: openai-api-key
          key: key
  jvm:
    memory:
      heapSize: 16g
  ports:
    - name: server
      port: 7001

The preceding example exposes Coherence RAG REST API on port 7001 on each pod. It also creates coherence-rag-demo-server Kubernetes service that maps to that port on all the pods, allowing you to expose REST API using ingress, or to forward local port to it for testing.

Note:

We have to pass OpenAI API key as an environment variable to be able to use the chat model, which references Kubernetes secret, for security reasons. To create the secret, you would need to run the following command and specify your own OpenAI API key within the from-literal argument:

kubectl create secret generic openai-api-key --from-file=key=/path/to/protected/openai-api-key

Now that we have the secret configured we can deploy our demo cluster:

kubectl apply -f coherence-rag-demo.yaml

Finally, we can forward local port 7001 to the coherence-rag-demo-server service, which lets you make REST API calls described in the following sections to ingest documents, perform vector searches, and augment chat conversations with the results of those searches:

kubectl port-forward service/coherence-rag-demo-server 7001:7001 -n default

Security Prerequisites

Configure the RAG service’s authenticated identity, model locations and administrative policy before startup or ingestion. Model download and model/configuration administration are separate permissions from document search. Supply credentials through the supported provider configuration and make external downloads an explicit deployment decision.

For the complete configuration and policy, see Securing Coherence RAG, RAG Configuration and Model Download Policy in Securing Oracle Coherence.

Deployment Prerequisites

This manifest is a deployment template. Install a compatible Coherence Operator, build or obtain the approved 26.1.0.0.0 RAG server image, push it to your registry and replace the example image reference. The illustrative chat model selector must be enabled for your provider account; choose the deployment’s approved model rather than treating that historical model name as a current recommendation.

Configure the hosting RAG REST authentication provider and intended admin identity before making configuration or ingestion requests. The OpenAI provider key is for outbound model calls; it does not authenticate callers to the RAG API. The API accepts the configured bearer-token/API-key identity form and rejects Basic authentication at this layer. Keep credentials in the deployment’s supported secret sources.

The built-in JAR-wired embedding model and externally downloaded HuggingFace models follow different setup paths. For an externally downloaded model, configure the exact approved model download allowlist before startup in hardened mode. The relevant security configuration and import restrictions are described in the Security Guide links above.

Save the manifest as coherence-rag-demo.yaml. The remaining kubectl commands apply the deployment and forward a local port for testing; port forwarding does not establish API authentication. Supply the authenticated admin identity when using the document-store and ingestion examples.