Creating an AI Assistant with OpenSearch and OCI Generative AI

• The tenancy must be subscribed to the US Midwest (Chicago) region or the Germany Central (Frankfurt) region. You don't need to create the cluster in either of those regions, just ensure that the tenancy is subscribed to one of the regions.
• To use AI assistant toolkit with OpenSearch, you need a cluster with OpenSearch image version 2.15 or later. By default, new clusters are configured to use version 2.15. To create a cluster, see Creating an OpenSearch Cluster.

  For existing clusters configured for version 2.3 or version 2.11, you can perform an inline upgrade to version 2.15, for more information, see OpenSearch Cluster Software Upgrades.

  To upgrade existing clusters configured for version 1.2.3 to 2.15, you need to use the upgrade process described in Manually Upgrading an OpenSearch Cluster.

• Create a policy to grant access to Generative AI resources. The following policy example includes the required permissions:

  ALLOW ANY-USER to manage generative-ai-family in tenancy WHERE ALL {request.principal.type='opensearchcluster', request.resource.compartment.id='<cluster_compartment_OCID>'}

  For information on creating a policy, see Creating a Policy.

  If you're new to policies, see Getting Started with Policies and Common Policies.

• Use the settings operation of the Cluster APIs to configure the recommended cluster settings that let you to create a connector. The following example includes the recommended settings:

  PUT _cluster/settings
  {
    "persistent": {
      "plugins": {
        "ml_commons": {
          "only_run_on_ml_node": "false",
          "model_access_control_enabled": "true",
          "native_memory_threshold": "99",
          "rag_pipeline_feature_enabled": "true",
          "memory_feature_enabled": "true",
          "allow_registering_model_via_local_file": "true",
          "allow_registering_model_via_url": "true",
          "model_auto_redeploy.enable":"true",
          "model_auto_redeploy.lifetime_retry_times": 10
        }
      }
    }
  }

  Example response:

  {
    "acknowledged": true,
    "persistent": {
      "plugins": {
        "ml_commons": {
          "rag_pipeline_feature_enabled": "true",
          "memory_feature_enabled": "true",
          "only_run_on_ml_node": "false",
          "model_access_control_enabled": "true",
          "native_memory_threshold": "99",
          "allow_registering_model_via_local_file": "true",
          "model_auto_redeploy": {
            "lifetime_retry_times": "10",
            "enable": "true"
          },
          "allow_registering_model_via_url": "true"
        }
      }
    },
    "transient": {}
  }

To use the Generative AI Assistant, you need to create a Generative AI chat model to interpret the user's instructions and then respond with the requested information. Creating a Generative AI model includes the following steps:

1. Register the model group.
2. Create the connector.
3. Register the model.
4. Deploy the model.

After you create the Generative AI model, you can use it to create a conversational agent. This agent combines the Generative AI model with user-specified tools, allowing it to interpret user text into invocations of tools, returning its interpretation of the results to the user. The model then returns the interpreted results. See Build your own chatbot and Agents and tools for more information.

This example uses the built-in opensearch_dashboards_sample_data_ecommerce index, which you can add to your OpenSearch cluster through the Dashboard. In this example, the agent is given a PPLTool and CatIndexTool. The PPLTool translates user instructions into Piped Processing Language (PPL), which allows the agent to run requests against index data. The CatIndexTool allows the agent to list indexes in the cluster. This tool is necessary for any useful assistant. Use the OpenSearch documentation on AI Agents for information on configuring your own usecase.

Select the following links for more information on these OpenSearch features:

• PPLTool
• VisualizationTool
• VectorDBTool
• CatIndexTool

POST _plugins/_ml/agents/_register
{
  "name": "Chat Agent with Llama",
  "type": "conversational",
  "description": "this is a test agent",
  "app_type": "os_chat",
  "llm": {
    "model_id": "<genAI_model_ID>",
    "parameters": {
      "max_iteration": 5,
      "message_history_limit": 5,
      "disable_trace": false
    }
  },
  "memory": {
    "type": "conversation_index"
  },
  "tools": [
    {
      "type": "PPLTool",
      "description": "Use this tool to transfer natural language to generate PPL and execute PPL to query inside. Use this tool after you know the index name, otherwise, call IndexRoutingTool first. The input parameters are: {index:IndexName, question:UserQuestion}.",
      "parameters": {
        "model_id": "<genAI_model_ID>",
        "model_type": "OPENAI",
        "execute": true
      },
      "include_output_in_agent_response": true
    },
    {
      "type": "CatIndexTool",
      "description": "Use this tool to get OpenSearch index information: (health, status, index, uuid, primary count, replica count, docs.count, docs.deleted, store.size, primary.store.size). \nIt takes 2 optional arguments named `index` which is a comma-delimited list of one or more indices to get information from (default is an empty list meaning all indices), and `local` which means whether to return information from the local node only instead of the cluster manager node (default is false)."
    }
  ]
}

A response similar to the following is returned. Make note of the agent_id returned.

{
  "agent_id": "<conversational_agent_ID>"
}

<genAI_model_ID>

After you create the conversational agent, you create the chatbot agent and configure it as the root chatbot agent in OpenSearch Dashboard for the cluster. The chatbot agent in this example uses the conversational agent to interpret and answer questions, and also uses the MLModelTool to suggest new questions.