5 Use the Vector Embedding Service

The Private AI Services Container embedding service supports generation of vector embeddings outside of the database. You can generate embeddings using ONNX, vLLM, and llama.cpp runtimes.

The embedding service generates vector embeddings from input data by using models deployed in the container. Applications can use these embeddings for semantic search, similarity comparison, retrieval-augmented generation (RAG), and other AI workflows that need to represent text or images as vectors.

The container enables multiple concurrent users. The effective number of concurrent users is determined by the number of CPU cores and the embedding model.

When using the ONNX runtime engine, a fixed set of Oracle Machine Learning (OML) model types and mining functions are supported. The container automatically determines the model type based on the data type and shape of the model inputs. The supported model types are as follows:

Model Type Model Function Goal
ONNX_TXT EMBEDDING

Generate text embeddings.

The model takes text as input and produces embeddings as output. Example models include sentence transformers and CLIP (text).
ONNX_IMG EMBEDDING

Generate image embeddings.

The model takes an image as input and produces embeddings as output. Example models include Vision Transformers and CLIP (image).

When using the ONNX runtime, only Oracle ONNX Pipeline formatted models that are produced by OML4Py 2.1.1 are supported for deployment in the Private AI Services Container and in the database, as of Oracle AI Database 26ai. Both text and image embedding pipelines are supported.

For more information about ONNX pipeline models, see Oracle Machine Learning for Python User’s Guide. For more information about importing pretrained models in ONNX format, see Oracle AI Database AI Vector Search User's Guide.

The vLLM and llama.cpp runtimes support only text embeddings and require a medium or large container image. For a list of available images along with information about their size, capabilities, and included models, see Install the Private AI Services Container.

The following are examples of model configuration and embedding generation using curl requests. The curl requests are identical among the runtime options except for the model names.

  • This example uses the vLLM runtime engine to generate an embedding from text:
    {
  "models": [
    {
      "name": "granite-embedding-97m-multilingual-r2",
      "path": "ibm-granite/granite-embedding-97m-multilingual-r2",
      "capabilities": ["TEXT_EMBEDDINGS"],
      "runtime": "vllm",
      "startup_timeout": 1200
     }
  ]
}
curl -X POST "http://localhost:9091/v1/embeddings" -H "Content-Type: application/json" 
  -d '{"model": "granite-embedding-97m-multilingual-r2", "input": ["The world of information 
  technology has undergone tremendous change over the past few decades, reimagining the way 
  businesses operate and how individuals interact with data. Digital transformation refers 
  to the integration of digital technology."]}'
  • This example uses the llama.cpp to generate an embedding from the same text input:
    {
  "models": [
    {
      "name": "embeddinggemma-300m-GGUF",
      "path": "unsloth/embeddinggemma-300m-GGUF",
      "capabilities": ["TEXT_EMBEDDINGS"],
      "runtime": "llamacpp",
      "startup_timeout": 1200
     }
  ]
}
curl -X POST "http://localhost:9091/v1/embeddings" -H "Content-Type: application/json" 
  -d '{"model": "embeddinggemma-300m-GGUF", "input": ["The world of information technology 
  has undergone tremendous change over the past few decades, reimagining the way 
  businesses operate and how individuals interact with data. Digital transformation refers 
  to the integration of digital technology."]}'

See Also:

For additional information that is helpful to understanding the use and implementation of the Private AI Services Container embedding service, see the following: