About Building and Managing a Multi-Agent with Oracle Digital Assistant

As enterprises increasingly adopt Generative AI (GenAI) agents to streamline operations and enhance customer experiences, the need for a comprehensive platform to manage these agents becomes paramount. Organizations seek intuitive user interfaces where they can design, deploy, and orchestrate multiple AI agents, seamlessly integrating them with APIs, workflows, and GenAI capabilities as needed.

Oracle is planning to rollout the OCI Generative AI Agents Platform. While we are looking forward for the next release, customer projects proved that Oracle Digital Assistant (ODA) can fit very well as orchestrator for AI agents. With its robust API connectivity, multi-channel deployment, flow designer, and LLM blocks, ODA enables businesses to efficiently manage the entire lifecycle of AI agents.

So, lets explore how ODA provides a structured approach to building, managing, and scaling AI agents.

Oracle Digital Assistant provides a complete ecosystem for building, managing, and scaling AI agents with seamless API integrations, multi-channel deployment, LLM capabilities, and workflow automation. By leveraging ODA, organizations can efficiently manage the entire lifecycle of AI agents, ensuring agility, scalability, and enhanced user experiences. This also applies for AI Agents developed with non OCI technologies and there is a public example of ODA + OpenAI at Bosch.

As enterprises continue their AI-driven transformation, ODA stands as a powerful enabler, simplifying the deployment of intelligent digital assistants while ensuring control, security, and business value.

Understand the Benefits of Using Oracle Digital Assistant to Build and Manage Multi-Agents

  • Seamless API Integrations ODA provides powerful API service integration capabilities, enabling agents to connect with any backend system. Supports REST APIs, database interactions, and external cloud services. Facilitates easy data retrieval and transactional workflows without custom backend development.
  • Multi-Channel Deployment Agents built using ODA can be deployed across multiple channels, including web, mobile, messaging platforms (WhatsApp, Slack, Teams), and voice interfaces. This ensures consistent user experience across all interaction touchpoints.
  • Visual Flow Designer for No-Code Agent Development The Flow Designer provides a low-code/no-code environment for designing AI-driven conversations and workflows. Allows business users and developers to quickly build, test, and deploy agents without extensive coding efforts.
  • LLM Blocks for Generative AI Integration ODA supports Large Language Model (LLM) integration, enabling AI agents to leverage GenAI capabilities when required. Organizations can call any LLM (Oracle GenAI, Cohere, Meta etc.) within an agent's workflow. Support for Prompt builder
  • Centralized Prompt Management: Admins can manage and update multiple prompts from a single location without modifying the Flow Designer, ensuring seamless updates and consistency across AI interactions.
  • Workflow Automation & Orchestration AI agents can trigger and manage workflows within ODA or external workflow engines. Supports event-driven automation, enabling dynamic and responsive agent interactions. Enables HR, IT, and customer support automation through AI-driven workflows.
  • Advanced Analytics & Monitoring ODA provides built-in analytics and reporting to track agent performance and user interactions. Enables optimization based on real-time insights and user feedback.
  • Security & Governance Enterprise-grade security features ensure compliance with data protection regulations. Role-based access control (RBAC) and audit logs for governance.

Architecture

Using ODA with an LLM block allow you to create AI agents fpr very simple to complex architectures (for example, supervisor one). AI agents can call tools, a knowledge base, and other agents, deciding autonomously which tool to use based on the user request.

While your specific architecture might differ from the one presented in this playbook, this example represents a typical implementation of a multi-agent service developed in ODA. In this example architecture, chat originates in the application layer, which comprises an instance of Microsoft Teams and a custom app. The chat content, or query, is directed into ODA's Channels component, and then on to a Skills chatbot. By using ODA, you can call any API; for example in this case, Oracle Fusion HCM APIs. Traffic then moves from the Skills chatbot to a GenAI router agent, which, depending on the the subject of the chat, directs it either to the appropriate HRMS agent or, if the query is unresolved or general, to a RAG agent.

The API can use any backend service from Fusion or from EBS suite, whether by using Oracle Integration or not. This allow the ODA AI agents to use nearly all types of API available in the Oracle Cloud. If the query moves on to one of the HRMS agents, it is processed and then sent through Oracle Integration middleware to one of the appropriate Oracle Fusion ERP services; for example, Oracle E-Business Suite,Oracle Procurement, Oracle Fusion Cloud Human Capital Management, or Oracle Cloud ERP. These services process the query and pass the necessary information back through the HRMS agents, then through the router agent, the Skills chatbot, then the Channels component, and exits ODA, bound for the application layer from which it originated. When logged in, the agent can call the API of the backends by using rights granted to the logged-in user. Note that each agent is itself a code that makes background calls to the GenAI service. In this example, you can log in using Fusion or MS Teams.

This diagram illustrates this architecture:


Description of multi-agent-oda-arch.png follows
Description of the illustration multi-agent-oda-arch.png

multi-agent-oda-arch-oracle.zip

This sample architecture contains these components:
  • Region

    An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, hosting availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).

  • Oracle Digital Assistant

    Oracle Digital Assistant provides a complete ecosystem for building, managing, and scaling AI agents with seamless API integrations, multi-channel deployment, LLM capabilities, and workflow automation. By leveraging ODA, organizations can efficiently manage the entire lifecycle of AI agents, ensuring agility, scalability, and enhanced user experiences.

  • Channels

    Channels carry the chat back and forth from users on various messaging platforms to the digital assistant and its various skills. There are also channels for user agent escalation and testing. You can expose your digital assistants and standalone skills to users by configuring channels in ODA.

  • Skills

    A skill is a chatbot geared toward a specific set of tasks or cater to a user request.

  • Router Agent

    A router agent is an AI agent that directs user queries to the relevant and most appropriate AI agent based on the nature of the query. A router agent relies on LLMs to dynamically analyze and route queries based on context, eliminating the need for predefined intents or extensive training data while achieving zero-shot functionality

  • RAG Agent

    A RAG agent combines the power of Retrieval-Augmented Generation (RAG) and AI agents to enhance the accuracy, adaptability, and complexity of information retrieval and generation tasks.

  • Leaves Agent

    The Leaves agent allows a user to book holidays or away time in the HCM backend. It calls the necessary API based on the natural language request of the user.

  • Letter Agent

    The letter agent is an AI component that assists in writing letters, whether personal, professional, or otherwise. It leverages AI capabilities to generate drafts, personalize content, and even suggest appropriate language or tone.

  • Expense Agent

    The Expense agent allows you to manage and report your expenses.

  • Claims Agent

    The Claims agent allows you to manage and report your claims. It calls the needed API based on the natural language request of the user.

  • OIC Integration Middleware

    OCI integration services connects any application and data source, including Salesforce, SAP, Shopify, Snowflake, and Workday, to automate end-to-end processes and centralize management. The broad array of integrations, with prebuilt adapters and low-code customization, simplify migration to the cloud while streamlining hybrid and multicloud operations.

  • OCI GenAI Service

    OCI Generative AI (GenAI) is a fully managed service for seamlessly integrating various language models into a wide range of use cases, including writing assistance, summarization, analysis, and chat.

About Additional LLM Block Features in ODA

In addition to the listed components, Oracle Digital Assistant provides advanced LLM block features that enhance the control and customization of AI agent responses:

These features include:
  • Enforce JSON-formatted LLM Response: Ensures LLM responses follow a predefined JSON schema for structured outputs. If needed, an event handler can transform the JSON into a user-friendly format, like a structured table or form.

    Note:

    Set Use Streaming to False when applying JSON formatting.
  • Number of Retries: Defines the maximum retry attempts when validation errors (entity or JSON) occur. The retry prompt highlights errors and requests the LLM to correct them. If retries exceed the limit, the dialog follows the error transition.
  • Retry Message: Notifies users when an LLM retry occurs due to validation errors; for example, enhancing the response.
  • Validation Customization Handler: Allows specialized validation via a custom handler deployed in the skill. It can:
    • Further process LLM responses.
    • Assess user requests for inappropriate content.
    • Apply interdependent entity validation, ensuring certain values require or exclude others.
    You must preconfigure the handler in the skill settings.
  • Analyze the calls to the LLMs and the responses.

About Managing the AI Agent Lifecycle in Oracle Digital Assistant

Managing the AI agent lifecycle can be summed up in the five stages described below.

  1. Plan and Design
    1. Define the use case and scope of the AI agent.
    2. Identify required API integrations and workflows.
    3. Design the conversation flow using ODA’s Flow Designer.
  2. Develop and Integrate
    1. Configure API services and backend connectivity.
    2. Implement LLM blocks for generative responses.
    3. Set up intent recognition and training models for better accuracy.
  3. Deploy and Expose
    1. Deploy the AI agent across multiple channels (web, mobile, WhatsApp, Teams, and so on).
    2. Ensure secure access and authentication mechanisms.
  4. Monitor and Optimize
    1. Continuously monitor agent interactions by using ODA analytics.
    2. Improve accuracy by refining intents and training models.
    3. Optimize workflows based on real-time user feedback.
  5. Scale and Maintain
    1. Add new capabilities as business needs evolve.
    2. Ensure periodic security and compliance updates.
    3. Expand to additional business units or geographies as required.