4.1.1 Create an LLM Definition

Create an LLM connector to define the model provider, model names, and connection details used by the AI features for MicroTx Workflows. You can reference the connector from workflow tasks, AI agent profiles, and other AI-driven workflow configurations.

Prerequisites

Before creating an LLM connector, complete the setup required for the model provider that you want to use.

To use OpenAI as the model provider, complete the following prerequisites:

  1. Create a new API key in the API Keys page of the OpenAI Developer Platform or use the OpenAI API.
  2. Copy the name and value of the created key and save it safely. You will need to provide this information later.
  3. Identify the model names that you want to use.
  4. Use https://api.openai.com/ as the base URL unless your environment requires a different endpoint.

To use OCI as the model provider, complete the following tasks:

  • Create or identify the OCI user that will call OCI Generative AI APIs.
  • Generate an OCI API signing key pair in PEM format. See Required Keys and OCIDs.
  • Upload the public key to the OCI user and note the public key value and fingerprint.
  • Keep the private key value available; you will paste it into the LLM connector as the API key.
  • Note down the OCID of the user, the tenancy's OCID, OCI region, and the compartment OCID.
  • Confirm the serving mode for the model. You can reach the pretrained foundational models in Generative AI through two modes: on-demand and dedicated.

To use Ollama as the model provider, complete the following tasks:

  • Ensure the Ollama endpoint is reachable from MicroTx Workflows.
  • Identify the model names that you want to use.
  • Copy the base URL for the Ollama service. For example http://localhost:11434 if it is running locally.
  • Create and copy the value of an API key if your Ollama endpoint requires authentication.

To use Oracle Database, create a database profile in MicroTx Workflows. Ensure the database has the embedding model available. See Create a Database Profile. Use this provider when you want Oracle Database to generate embeddings. Oracle Database embedding support uses vector embedding models loaded into the database, such as ONNX-format embedding models. You can use this connector in GENAI_INGESTION task. For information about using Oracle Database as embedding model provider, see https://blogs.oracle.com/machinelearning/use-our-prebuilt-onnx-model-now-available-for-embedding-generation-in-oracle-database-23ai.

To create an LLM definition:
  1. Open the navigation menu and click Connectors.
  2. Click LLM.

    The LLM Definitions list page opens. All the LLMs that you have defined are displayed in a table.

  3. Click add a new building block.
    The New LLM Definition dialog box appears.
  4. Enter the following information.
    • Name: Enter a unique and descriptive name to identify this LLM definition in workflows. The name can be up to 128-characters long. Use only letters, numbers, underscores (_), and hyphens (-). Spaces and other special characters are not supported.
    • Model Provider: Select one of the following models: OCI, OpenAI, Ollama, Oracle Database.
    • Models: List the model names you intend to use for the selected model provider, separated by commas. For example, you may want to enter gpt-4o, gpt-4o-mini if you select OpenAI as the model provider. Refer to documentation specific to the selected model provider for the list of supported models.
    • Description: Optional. Enter a description for the LLM definition.
  5. Enter the following additional configuration information based on the selected model provider.
    • If you select OCI as the Model Provider, provide the following information to access the selected LLM model which is hosted on OCI.
      • API Key: Paste the value of the private RSA key in PEM format.
      • User ID: Enter the OCID of the OCI user whose API signing key is used.
      • Tenant ID: Enter the tenancy's OCID where the models are hosted.
      • Region: Enter the OCI region. For example, us-phoenix-1.
      • Fingerprint: Enter the fingerprint of the uploaded public key.
      • Pass Phrase: Optional. Enter the pass phrase, if you had provided one, while creating the private key.
      • Compartment ID: Specify the OCID of the compartment that contains or has access to the model.
      • Serving Mode: Optional. Specify the serving mode for the model, according to your deployment. The default value is on demand. See On-Demand and Dedicated Modes for OCI Generative AI Models in Oracle Cloud Infrastructure Documentation.
    • If you select OpenAI as the model provider, additionally provide the following configuration information.
      • API Key: Paste your OpenAI API key, which authenticates your requests.
      • Base URL: Specify a custom base URL to route requests to the OpenAI API endpoint. For example, https://api.openai.com/.
    • If you select Ollama as the model provider, additionally provide the following configuration information.
      • API Key: Enter the API key if your Ollama endpoint requires one.
      • Base URL: Enter the Ollama service endpoint.
    • If you select ORACLEDB as the model provider, you must select a database profile that contains the connection details for the Oracle Database used to generate embeddings.

      Note:

      Oracle Database is used as an embedding model provider. It does not function as a general-purpose chat or text-generation LLM in this connector flow.
  6. Click Test to ensure that you can establish a connection with the provided details.
    If you have entered multiple models, test is performed only for the first model that is specified in the comma-separated list.
    If there are any issues, an error message is displayed. Note down the error message so that you can troubleshoot the issue, and then click Close.
  7. Click Test to ensure that you can establish a connection with the provided details.
    If there are any issues, an error message is displayed. Note down the error message so that you can troubleshoot the issue, and then click Close.
  8. Click Submit.
Your new definition appears in the list of available LLMs. Reference this name in your workflows or tasks to utilize the specified model and provider.

Parent topic: Manage LLM Connector