5.2 Configure Models

Configure the chat model and embedding model separately.

The chat model is used for answer generation. The embedding model is used for document ingestion and semantic retrieval. Use the same embedding-model configuration for ingestion and retrieval. The chat model is used when retrieved context is supplied to an AI model to generate a response.

5.2.1 Configure the Chat Model

The chat-model configuration identifies the OCI Generative AI chat model that can be used for answer generation.

Syntax

MODEL CONFIG CHAT-MODEL -model-name <model-id>
-serving-type <ON-DEMAND|DEDICATED> -compartment-id <compartment-ocid>
[-config-path <oci-config-file>] [-debug] [-verbose]

SQLcl determines the chat-model integration from the specified model name. Model names containing cohere use the OCI Cohere integration. Other model names use the generic OCI Generative AI integration.

Examples

  • Configure an on-demand chat model:
    SQL> MODEL CONFIG CHAT-MODEL -compartment-id "ocid1.compartment.oc1..example" 
-model-name "cohere.command-r-plus" -serving-type ON-DEMAND
  • For a dedicated deployment, specify the dedicated endpoint identifier and set the serving type to DEDICATED: 
    SQL> MODEL CONFIG CHAT-MODEL -compartment-id "ocid1.compartment.oc1..example" 
-model-name "<dedicated-endpoint-ocid>" -serving-type DEDICATED

5.2.2 Configure an Embedding Model

The embedding model generates vectors during document ingestion and semantic retrieval.

The embedding model converts document content and retrieval queries into vector embeddings.

Note:

Use the same embedding-model configuration for both document ingestion and semantic retrieval to ensure that document content and retrieval queries are represented consistently.

Syntax 

MODEL CONFIG EMBEDDING-MODEL -model-name <model-id>
-serving-type <ON-DEMAND|DEDICATED> -compartment-id <compartment-ocid>
[-config-path <oci-config-file>] [-debug] [-verbose]

Example

  • To configure an embedding model

    SQL> MODEL CONFIG EMBEDDING-MODEL
-model-name "openai.text-embedding-3-large"
-serving-type ON-DEMAND
-compartment-id "ocid1.compartment.oc1..example"

5.2.3 View the Saved Model Configuration

Model configuration is stored in:

~/.dbtools/sqlcl/model_config.json

The file is created automatically by the first successful MODEL CONFIG command.

To display the saved configuration, use the following command:

SQL> MODEL CONFIG -list