About Oracle Private AI Services Container

The Private AI Services Container is a lightweight, containerized web service that allows you to offload expensive AI computation, such as vector embedding generation, vector index creation, and hosting LLMs outside of the database. This container can run in your data center or on compute nodes in the public cloud.

Offloading these computations can free up database compute resources that can then be used for other tasks, such as similarity search. Depending on the container image that you choose, you can perform inference using either CPU or GPU hardware, or vector index creation on GPU hardware.

Inference Services

With the inference service, you can generate vector embeddings, host LLMs, rerank query results, and perform text classification. A single container image allows you to use multiple of the inference services described here. For a list of available images, along with the services they support, see Install the Private AI Services Container.

• Vector Embedding Service

  The vector embedding service allows you to securely generate vector embeddings outside the database while still storing them directly in your Oracle AI Database. You can use the same embedding models that you would use inside the database, allowing you to seamlessly integrate the use of embeddings generated in the container with those generated in the database for similarity search operations.

  To implement the embedding service, the container can be called from the database using the UTL_TO_EMBEDDING and UTL_TO_EMBEDDINGS procedures of the DBMS_VECTOR PL/SQL package. The container can also be called by REST clients such as curl, or clients that use the OpenAI SDK. Internet access is not needed to use the container.

  For information about the implementation of the vector embedding service, see Use the Vector Embedding Service.

• Private Large Language Model Service

  The Private Large Language Model service allows you to host LLMs for Chat, RAG, and Agentic workloads. The service uses Llama.cpp or vLLM as the LLM runtime, meaning that most LLMs that work on Llama.cpp or vLLM can be used. The container can use x86-64 CPUs or modern NVIDIA GPUs to power the LLMs.

  LLM inference is handled through either a vLLM or Llama.cpp server, which is then managed by the container to facilitate the loading of LLMs and access to LLM-related APIs, such as for chat completion.

  For more information about the Private Large Language Model Service and its implementation, see Use the Private Large Language Model Service.

• Reranking Service

  The reranking service reorders a set of documents according to their relevance to a query. Applications can use reranking after an initial search or retrieval step to improve the ordering of candidate results before presenting them to a user or passing them to another AI workflow.

  The service is available through the /v1/rerank REST endpoint. A model can serve reranking requests only when it is configured with the TEXT_RERANK capability.

  For more information about the reranking service and its implementation, see Use the Reranking Service.

• Text Classification Service

  The text classification service classifies one or more text inputs by using a configured classification model. Applications can use text classification for tasks, such as sentiment analysis, topic labeling, intent detection, and other workflows that assign labels to text.

  The service is available through the /classify REST endpoint. A model can serve classification requests only when it is configured for text classification.

  For more information about the text classification service and its implementation, see Use the Text Classification Service.

Vector Index Service

The vector index service gives you the ability to create Hierarchical Navigable Small World (HNSW) vector indexes faster by offloading the creation to an NVIDIA GPU. The high memory bandwidth and parallel processing of a GPU can produce significant time savings.

The vector index service is implemented using parameters of the CREATE VECTOR INDEX DDL to define the REST endpoint and the API KEY for the Private AI Services Container.

For more information about the implementation of the vector index service, see Use the Vector Index Service.

Note: User data is not stored and is only processed transiently. All requests to the container are stateless and the data is not stored.

Note:

Support may be available for ancillary use of this program in conjunction with a supported Oracle product, to the extent described in that product’s documentation. If support is provided it will be in accordance with Oracle’s technical support policies which may be found at https://www.oracle.com/support/policies/.

For licensing information related to Oracle AI Private Services Container, see Licensing Information User Manual for Oracle Private AI Services Container.