Support for ONNX Models with Oracle GoldenGate

As organizations adopt AI-powered features, many are looking for ways to keep inference local for security compliance and performance. Oracle GoldenGate makes this possible by supporting local ONNX models, allowing you to leverage AI capabilities without relying on an external model provider.

ONNX Model to allow AI capabilities

Note: Local ONNX model support is available beginning with Oracle GoldenGate 23.26.3.

By supporting local ONNX models, Oracle GoldenGate gives organizations the flexibility to choose where AI inference happens. Whether you’re operating in a secure enterprise environment, deploying at the edge, or working in an air-gapped network, local models provide a practical way to integrate AI while keeping data under your control.

This capability complements GoldenGate’s growing AI integration by providing the freedom to choose between cloud-hosted models and locally deployed ONNX models based on their security, performance, and operational requirements.

Why Use a Local ONNX Model?

While cloud-hosted large language models are convenient, they aren’t always the right fit. Running a model locally allows GoldenGate deployments to: • Keep sensitive data within your environment • Support air-gapped or disconnected deployments • Reduce inference latency • Eliminate dependency on external AI services • Avoid configuring and managing provider credentials This makes local models an excellent option for enterprise and regulated environments.

Deploying ONNX as the Local AI Model

To enable a local ONNX model, copy your ONNX model and its corresponding tokenizer into the following directory:

ServiceManager/var/lib/models

For example:

ServiceManager/var/lib/models/my-model.onnx/tokenizer.json

The tokenizer.json file must reside alongside the .onnx model file.

Automatic Model Discovery

After the files are in place, Oracle GoldenGate automatically discovers the local model.

The Service Manager GUI scans the models directory and recognizes:

Service Manager automatically detects the ONNX model file and tokenizer and allows you to add the model from the Service Manager.

One of the biggest advantages of using a local ONNX model is its simplicity. Unlike cloud-based AI services, local models do not require an AI provider configuration. You don’t require any manual registration or AI Provider Configuration for setting up the ONNX model in Oracle GoldenGate. You also don’t need to specify: