Build an AI Powered Chat Router Using Oracle Cloud Infrastructure AI Agent Platform

Introduction

This tutorial demonstrates how to build an Oracle Cloud Infrastructure (OCI) AI agent which will route the user input based on its context to the LLMs hosted in OCI Generative AI service using the Oracle Cloud Infrastructure Agent Development Kit (OCI ADK). You will learn how to package OCI service logic as tools, create an AI agent that interprets natural language, and effectively gets answers based on the context. This gets the best out of the LLMs hosted inside OCI using AI agents.

As part of this tutorial we will be using only two LLMs in the backend. If the user’s input is based on content generation or text summarization, then that input will be sent to Llama 3.3 70B; if the user’s input is about coding or math, then it will be sent to Llama 4 Maverick hosted on the OCI Generative AI services.

Note:

• Familiarity with both basic OCI operations and Python development is helpful but not required.
• Make sure your OCI user and service account has permissions for accessing OCI Generative AI services and agent endpoints inside OCI.
• Once you set up your first AI agent, you can easily extend it with new tools to cover additional OCI services by defining and registering them using the OCI ADK toolkit interface.

Objectives

• Understand the architecture and workflow of OCI AI agents and OCI ADK.

• Develop custom toolkits.

• Configure, register, and run an AI agent that answers the best based on the context.

• Use AI agent to use the OCI Generative AI chat services.

• Reuse this learning for developing similar agents with OCI services using OCI ADK.

Prerequisites

• Install uv: Install uv using the following command as per your OS.

  • macOS/Linux:

    curl -LsSf https://astral.sh/uv/install.sh | sh
    

  • Windows:

    powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
    

• Create a Virtual Environment: Navigate to your project directory and create a virtual environment using the uv command.

  uv venv
  

• Activate the Virtual Environment: Activate the virtual environment.

  • macOS/Linux:

    source .venv/bin/activate
    

  • Windows:

    .venv\Scripts\activate
    

• Install Required Packages: Install the necessary packages using the uv command.

  uv pip install oci "oci[adk]" rich pydantic docstring_parser
  

• Ensure you have:

  • Valid OCI credentials and configurations. For more information, see Setting up the Configuration File.

  • Oracle Cloud Infrastructure Identity and Access Management (OCI IAM) policies permitting access to OCI Generative AI services and to register and use agent endpoints.

  • An available OCI agent endpoint or the ability to create one.

Task 1: Create an OCI AI Agent from the OCI Console

The Oracle Cloud Infrastructure AI Agent Platform (OCI AI Agent Platform) enables you to create intelligent assistants that interpret user questions, plan actions, and call custom or built-in tools (skills) against OCI APIs. Each tool is packaged as a Python class with specific methods, confirming to the OCI ADK’s tool registration system.

1. Log in to the OCI Console, navigate to Generative AI Agents and click Create agent.

   

2. In the Basic information section, enter a Name and the Welcome message. The Routing instruction will be kept blank as part of this tutorial. Click Next.

   

3. In the Add tool section, do not create any tools. We will use OCI ADK to create it.

4. In the Setup agent endpoint section, select Guardrails.

   

5. Click Create Agent.

Key concepts:

• Agent: The AI powered interpreter and orchestrator.
• Tool/toolkit: A class with one or more operations exposed to the agent (@tool).
• Agent endpoint: The OCI resource providing runtime and connectivity.
• Agent client: Python client for agent-endpoint communication.

Note: An agent can route user requests across multiple toolkits (for example, Oracle Cloud Guard, compartments) and compose multi-step operations as needed.

Task 2: Setup the OCI ADK Locally

You will write Python classes defining tools to expose Oracle Cloud Guard operations. Each function is annotated with @tool and can be called with natural language by the agent.

Update the Authentication in the Agent File

• Update the authentication type (auth_type), select the config profile and the region in which we have the AI agent endpoint created.

• Update the endpoint OCID (agent_endpoint_id) as well in get_config.py.

Task 3. Run the AI Agent

1. Download the following file and save it in the same directory.

   • compartments.py.
   • get_config.py.
   • oci_ai_models.py.
   • oci_ai_route.py.
   • oci_ai_text_generation.py.

2. Update the input in the oci_ai_route.py file as per your requirement.

   

3. Run the oci_ai_route.py file using the following command.

   uv run oci_ai_route.py 
   

4. Based on the context the required model is invoked and content is generated.

   

Note: You can use the agent.run() from the oci_ai_route.py added to your webapp or application to invoke this agent.

Related Links

• Pretrained Foundational Models in Generative AI

• OCI Agent Development Kit

• Generative AI Agents

Acknowledgments

• Author - Mukund Murali (Principal Cloud Architect)

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.

---

Title and Copyright Information

Build an AI Powered Chat Router Using Oracle Cloud Infrastructure AI Agent Platform

G38654-01

Copyright ©2025, Oracle and/or its affiliates.