External Initializers and In-Memory Sharing

A model that uses very large initializers can reach the 2 GB size limit. To solve this problem, ONNX can store initializers as external files referred as external initializers or external data.

Advanced models may use large constant tensor values (multi-dimensional arrays) whose cumulated size may be larger than 2GB and prevents serializing these models into protocol buffers.

Embedding models, crucial for tasks like vector similarity search, have initializers that often account for 95% of their file size and may range from dozens of megabytes to several gigabytes. Until this release, the approach was to load a private copy of these initializers into each session's process memory (PGA), leading to high total memory usage for concurrent workloads.

In-Memory Sharing

Oracle's ONNX integration supports the sharing and in-memory population of external initializers, enabling the same model’s initializers to be loaded once into global memory and accessed by all database sessions needing that model. This improves memory efficiency and scalability.

Only ONNX models imported with external initializers are eligible for in-memory population. When enabled, the model's initializers are loaded into a shared global area for use by all qualifying processes.

See Enable and Use External Initializers and on how to enable and use shared external initilizers.