Best Practices

This section presents the essential best practices for deploying enterprise AI agents into production.

1. Define the Agent's Scope

Every production agent needs a narrowly defined purpose.

Define the following for each agent:

  • Business objectives
  • Success metrics
  • Allowed actions
  • Forbidden actions
  • Human approval requirements
  • Escalation paths

Design agents to solve specific business problems rather than serve as general-purpose assistants. Smaller, specialized agents reduce the complexity of validation, security, monitoring, and maintenance compared with a single monolithic agent.

Example bank in practice: The Loan Servicing Agent is scoped to exactly five activities: answering loan balance and payment-history questions, explaining refinancing eligibility, generating payment reminders, creating servicing tickets, and routing hardship requests to a human specialist. It cannot approve a loan modification, waive a fee, change a payment due date, or release funds. Keeping the scope this narrow reduces operational and regulatory risk while improving accuracy.


Description of scope-example.png follows
Description of the illustration scope-example.png

Production implementation: LangGraph defines dedicated nodes for Intent Classification, Account Retrieval, Policy Lookup, Ticket Creation, and Human Handoff. The Oracle Enterprise AI Response API performs reasoning, while every state-changing action is delegated to a governed enterprise API rather than executed directly by the model.

2. Build Modular Agent Architectures

Enterprise agents use modular architectures that separate reasoning from execution.

Typical components include the following:

  • User interface
  • Planning and reasoning layer
  • Tool orchestration
  • Enterprise connectors
  • Memory services
  • Retrieval services (RAG)
  • Policy engine
  • Observability layer
  • Security layer

Separating responsibilities helps teams improve components independently without affecting the overall system and simplifies testing and debugging.

Example bank in practice: The workflow uses independent stages instead of one large prompt: extract the customer's request, retrieve their loan and account records, check applicable servicing policies, calculate any relevant figures (payoff amount, next due date), draft a response, and request human approval when required. The fraud-and-hardship detection module can be upgraded independently of the customer-communication module.


Description of architecture-example.png follows
Description of the illustration architecture-example.png

Production implementation: LangGraph orchestrates each stage as an isolated node. LangChain OCI supplies reusable connectors to the bank's loan servicing platform, while the Oracle Enterprise AI Response API handles reasoning within each stage independently.

3. Implement Robust Tool Governance

An agent is only as safe as the tools it can access.

Define the following for each tool:

  • Explicit input schemas
  • Output validation
  • Authentication requirements
  • Authorization rules
  • Timeout limits
  • Retry behavior
  • Rate limits
  • Idempotency guarantees

Never expose unrestricted APIs directly to a large language model (LLM). Instead, wrap enterprise APIs with controlled tool interfaces that validate every request before execution.

Example bank in practice: The agent can initiate a "payment reminder" or "servicing ticket," but it never accesses the core banking ledger directly. Every action passes through a Servicing Tool that validates the customer's identity, confirms the account is in good standing, checks that the requested action is within policy limits, and only then calls the core banking API.


Description of governance-example.png follows
Description of the illustration governance-example.png

Production implementation: LangChain OCI exposes the Servicing Tool as a structured function with a strict input/output schema. Oracle Identity verifies the caller's authorization before the tool executes, and every invocation, whether successful or rejected, is logged for audit.

4. Enforce Strong Identity and Access Management

Production agents must operate under enterprise identity policies.

Best practices include:

  • Least privilege access
  • Short-lived credentials
  • OAuth or enterprise identity providers
  • Role-based access control (RBAC)
  • Secret management through secure vaults
  • Complete audit logging

Agents must never receive broader permissions than the human users they represent.

Example bank in practice: A loan officer using the agent can view and act only on accounts within their assigned portfolio. When the agent authenticates on behalf of that officer, it inherits the same permissions and no more. If the officer asks about an account outside their portfolio, the underlying tool rejects the request before the model ever generates a response.


Description of iam-example.png follows
Description of the illustration iam-example.png

Production implementation: Oracle Identity Cloud Service (IDCS) authenticates every user session. Each tool call is checked against the user's identity, role, and resource ownership. This check ensures that the agent never exceeds the requesting employee's own access rights.

5. Use Retrieval-Augmented Generation (RAG) for Enterprise Knowledge

Enterprise agents use retrieval rather than relying solely on model training.

Use the following practices:

  • Store enterprise knowledge in searchable indexes
  • Retrieve relevant documents dynamically
  • Apply document permissions during retrieval
  • Cite sources whenever possible
  • Version enterprise knowledge

A well-designed RAG architecture improves accuracy, reduces hallucinations, and ensures that responses remain current without retraining models.

Example bank in practice: When a customer asks about refinancing eligibility, the agent retrieves the current servicing policy manual, the applicable rate sheet, and relevant regulatory disclosures rather than relying on the underlying model's training data. Every answer cites the specific policy section that supports it. When the bank updates its refinancing terms, the agent reflects the change.


Description of rag-example.png follows
Description of the illustration rag-example.png

Production implementation: LangGraph includes a dedicated Retrieval Node. Policy manuals and rate sheets are chunked and embedded directly in Oracle AI Database 26ai using the native VECTOR data type, with HNSW indexes for fast similarity search and hybrid vector indexes that blend semantic and full-text matching for precise policy lookups. Row-level security filters results before retrieval to include only documents that the user is permitted to see, and LangChain OCI queries this store so the Oracle Enterprise AI Response API combines the permission-filtered passages with the customer's question to produce a grounded, citable response.

6. Design for Human-in-the-Loop Approval

Some decisions require human approval.

Require explicit human approval for high-risk operations, including:

  • Financial transactions
  • Customer communications
  • Infrastructure modifications
  • Production deployments
  • Data deletion
  • Legal or compliance actions

Approval workflows reduce operational risk while allowing automation for lower-risk tasks.

Example bank in practice: The agent can automatically send a payment reminder or generate a payoff quote, but any request that implies a hardship accommodation, such as a payment deferral, a fee waiver, or a modified repayment schedule, pauses and routes to a human loan specialist. The specialist receives a summary of the customer's situation, the account history, and the agent's recommendation before deciding.


Description of hitl-exampe.png follows
Description of the illustration hitl-exampe.png

Production implementation: Approval steps are implemented as LangGraph checkpoints. Workflow state is persisted until a human responds. The persisted state allows the process to resume where it paused, even if that takes hours or days.

7. Prioritize Reliability and Resilience

Reliable enterprise deployments anticipate failures rather than assume ideal conditions.

Design agents to handle the following failures gracefully:

  • API failures
  • Network outages
  • Partial tool failures
  • Rate limiting
  • Context window limitations
  • Model unavailability
  • Invalid responses

Recommended engineering practices include circuit breakers, retry policies with exponential backoff, request timeouts, fallback models, queue-based processing, dead-letter queues, and checkpointing long-running workflows. Resilient systems recover automatically without requiring operator intervention.

Example bank in practice: During a routine servicing request, the core banking API times out and the identity service briefly becomes unavailable. Instead of ending the customer's session, the agent retries with exponential backoff, falls back to a cached read-only view of the account for informational questions, and resumes full functionality automatically when the services recover. This approach minimizes disruption to the customer.


Description of reliability-example.png follows
Description of the illustration reliability-example.png

Production implementation: LangGraph checkpointing preserves workflow state across failures. Oracle Enterprise AI Deployment automatically scales inference endpoints under load, and enterprise API connectors implement retry policies and circuit breakers by default.

8. Establish Comprehensive Observability

Enterprise AI systems require the same level of observability as distributed software systems.

Capture prompts, responses, tool calls, latency, token consumption, cost, success rates, error rates, human approvals, user feedback, and execution traces. Distributed tracing across agent workflows simplifies troubleshooting and performance optimization.

Example bank in practice: Customers begin reporting slower responses from the servicing agent. Rather than guessing, engineers inspect distributed traces and find the breakdown: model response 1.1 seconds, policy retrieval 140 ms, core banking lookup 4.6 seconds, identity check 250 ms. The traces identify the bottleneck as a slow core banking service call, not the agent itself.


Description of observability-example.png follows
Description of the illustration observability-example.png

Production implementation: Every LangGraph node emits traces, metrics, and logs. OpenTelemetry correlates the full request path across LangGraph, the Oracle Enterprise AI Response API, enterprise connectors, the retrieval index, and Oracle Enterprise AI Deployment. This correlation gives operations teams complete end-to-end visibility.

9. Evaluate Continuously

Production agents require continuous evaluation.

Maintain evaluation datasets covering business scenarios, edge cases, failure cases, adversarial prompts, security attacks, and regression tests. Incorporate automated evaluations into CI/CD pipelines so that every prompt, workflow, or model update is validated before release.

Example bank in practice: Before any model or prompt update reaches the Loan Servicing Agent, it is run against a fixed evaluation suite: 2,000 historical servicing conversations, 500 regulatory edge cases (for example, disclosures required under lending law), 200 prompt-injection attempts embedded in uploaded documents, and a compliance-accuracy benchmark. Only versions that meet or exceed the current production baseline are promoted.


Description of evaluate-example.png follows
Description of the illustration evaluate-example.png

Production implementation: CI/CD pipelines run the evaluation suite automatically on every change. Prompt edits, workflow changes, and model upgrades all require passing these regression benchmarks before deployment is authorized.

10. Secure Against Prompt Injection and Data Leakage

Prompt injection remains one of the most significant risks in agentic systems.

Mitigation strategies include input validation, prompt sanitization, tool isolation, context separation, output filtering, sensitive data redaction, and policy enforcement before execution. Agents must never follow instructions retrieved from external documents or web content without validation.

Example bank in practice: A customer uploads a supporting document for a refinancing request. The document's metadata contains hidden text that instructs the system to approve the request immediately. Because the system treats uploaded documents as untrusted data, the agent ignores the embedded instruction and continues to evaluate the request only against the bank's actual servicing policies.


Description of secure-example.png follows
Description of the illustration secure-example.png

Production implementation: LangGraph keeps system instructions, user input, retrieved documents, and tool results in separate channels. Policy validation runs before every tool invocation. This layered defense prevents injected instructions from influencing a sensitive action. As an additional control below the application layer, Oracle Deep Data Security enforces declarative row-level, column-level, and cell-level policies directly in Oracle AI Database 26ai. Even if an injected instruction causes the agent to issue an unauthorized query, the database uses the officer's propagated identity and runtime context to return only the data that officer is entitled to see and masks everything else as NULL.

11. Manage Context and Memory

Treat memory as enterprise data.

Best practices include separating session memory from long-term memory, encrypting stored memory, defining retention policies, supporting memory deletion, versioning memory updates, and avoiding storage of unnecessary personal information. Only persist information that improves future task performance while respecting privacy regulations.

Example bank in practice: The Loan Servicing Agent initializes a single OracleAgentMemory instance backed by the bank's Oracle AI Database connection pool. Each customer interaction becomes a thread, scoped to that customer's user_id.

  • When a customer says something such as “I'd prefer to be contacted by text going forward,” the agent calls thread.add_messages(...), which writes the raw conversation into the database and automatically extracts a durable Fact, the preferred-contact-channel preference, without a separate extraction step.
  • For information the agent retains outside the normal conversational flow (for example, “customer's hardship case was escalated to a specialist on July 3”), it writes explicitly by using thread.add_memory(...).
  • On a later call, the agent retrieves relevant history with memory.search(query=..., scope=SearchScope(user_id=...)). This scope keeps memory partitioned per customer. A loan officer's agent session can search only within the scope that it is authorized for (agent-, user-, or team-level), with permissions inherited from the officer's own enterprise identity.

This design maps to the bank's required split between session memory and long-term memory: raw message history functions as session or working memory, while extracted Facts and State function as durable, cross-session memory. Durable memory helps the agent recognize a returning customer's preferences without asking again. The agent can also use Similarity to flag a new request that resembles a prior hardship case without retaining the account numbers or authentication data that the bank does not want an agent to persist.


Description of memory-example.png follows
Description of the illustration memory-example.png

Production implementation: Session and long-term memory both live in Oracle AI Database 26ai. Extracted Facts and State are stored using the native VECTOR data type and indexed with HNSW vector indexes. Elastic Vector Memory keeps retrieval fast as memory grows. Hybrid vector indexes blend semantic and full-text search to power Similarity lookups, with search accuracy tunable as a percentage. Governed Access uses Oracle's built-in database security, including encryption, data masking, and activity auditing. Permissions are inherited from the same identity system described in Section 4. Retention runs as a scheduled lifecycle job that purges memory past the bank's regulatory window.

12. Build Responsible AI Guardrails

Enterprise deployments require multiple layers of AI governance.

Use guardrails to address harmful content, bias detection, toxicity filtering, compliance requirements, privacy protection, intellectual property, and regulatory obligations. Maintain human oversight for exceptional situations.

Example bank in practice: Lending is a heavily regulated activity, so the agent runs every customer-facing response through a compliance guardrail before it is sent. The guardrail checks for required regulatory disclosures (such as fair-lending language) and screens for language that could be interpreted as a discriminatory statement about a protected class. It also blocks the agent from making any statement that resembles a binding credit decision and redacts personally identifiable information before the information is logged or cited in a response.


Description of gurdrails-example.png follows
Description of the illustration gurdrails-example.png

Production implementation: A policy engine sits between the reasoning layer and the customer-facing output and applies compliance and responsible-AI rules as a final checkpoint. The policy engine automatically blocks violations and routes them to a compliance reviewer rather than presenting them to the customer.

13. Optimize Cost and Performance

Production agents balance capability with operational efficiency.

Optimization techniques include model routing, response caching, semantic caching, token optimization, context compression, batch inference, streaming responses, and dynamic model selection. Not every request requires the largest available model; selecting the appropriate model for each task can reduce latency and operational costs.

Example bank in practice: A simple "what's my next payment due date" question is routed to a smaller, low-latency model backed by a semantic cache, because the answer is a straightforward lookup. A nuanced refinancing eligibility question, which requires multi-step reasoning across policy documents, is routed to a larger model. This dynamic routing cut the agent's average response cost by more than half without degrading accuracy on complex requests.


Description of optimize-example.png follows
Description of the illustration optimize-example.png

Production implementation: Oracle Enterprise AI Deployment supports multiple concurrently deployed model tiers, and LangGraph's routing node selects the appropriate tier based on intent classification, with a semantic cache in front of the highest-volume, lowest-complexity request types.

14. Implement Safe Deployment Practices

Treat agent deployments as software releases.

Recommended practices include versioning prompts and workflows, canary deployments, blue/green deployments, feature flags, rollback mechanisms, configuration management, and Infrastructure as Code (IaC). Review and test prompt changes using the same processes as application code.

Example bank in practice: When the bank introduces a new refinancing-recommendation capability, it doesn't release the capability bank-wide at once. The bank deploys the capability in stages: to internal loan officers only in week 1, to 5% of customers in week 2, to 25% of customers in week 3, and to all customers in week 4. The bank monitors error rates at each stage and automatically rolls back the capability if the rates exceed a defined threshold.


Description of deployment-example.png follows
Description of the illustration deployment-example.png

Production implementation: Oracle Enterprise AI Deployment manages versioned model endpoints, IaC provisions consistent environments across development, testing, and production, and feature flags control the rollout independently of the underlying model deployment.

15. Measure Business Outcomes

Technical metrics alone are insufficient.

Track business-focused KPIs such as task completion rate, human intervention rate, customer satisfaction, resolution time, operational cost savings, productivity improvements, accuracy, and compliance rate. These metrics demonstrate business value and guide continuous improvement efforts.

Example bank in practice: Six months after launch, the bank tracks measurable outcomes rather than model accuracy alone.

KPI Before Agent After Agent
Average resolution time 18 minutes 5 minutes
First contact resolution 62% 87%
Human escalations 45% 14%
Customer satisfaction 4.1 / 5 4.7 / 5
Cost per servicing interaction $8.40 $2.75

These outcomes demonstrate measurable business value, not just technical performance.

Production implementation: OpenTelemetry metrics are combined with business KPIs pulled from the bank's CRM and loan servicing systems. Engineering dashboards correlate technical indicators such as latency, token usage, and tool failure rate with business outcomes such as resolution time and customer satisfaction. This correlation enables continuous optimization based on real operational impact.