11 Import Pretrained Models in ONNX Format

Oracle AI Database 26ai includes an ONNX runtime engine for running embedding models directly inside the database. This section covers the process of importing existing pretrained embedding models into Oracle AI Database, including converting those models into the ONNX format if they are not already converted.

Open Neural Network Exchange or ONNX is an open standard format of machine learning models. Consider the following use cases:

• Using complex media input such as text or image for similarity search
• Perform text classification
• Perform reranking

You need vector embedding of the media input to perform all the tasks mentioned above. ONNX pipeline models allows you to convert text and/or image models into ONNX format if they are not already in ONNX format, import the ONNX format models into Oracle AI Database, and generate embeddings from your data within the database. The ONNX format models imported to the database could be used for tasks such as search and classification.

Pretrained models are models that are already trained on a media data (text, image, etc.) and saved to a storage format for future use. Hugging Face is the most popular platform that hosts pretrained models typically created with PyTorch.

This table lists machine learning models, along with a brief description, model size, and a PAR download link for each.

Model Name	Description	Dimensions	Size (MB)	Last Updated	Download
all_MiniLM_L12_v2	This is a SentenceTransformers model suitable for semantic similarity search and clustering use cases.	384	116.92	7/15/2024	Link
multilingual_e5_base	This is a multilingual sentence-transformer model that supports 100 languages. Supports text embedding, translation, and multilingual understanding. Features 12 layers with 768-dimensional embeddings for multilingual tasks.	768	1040	2/17/2024	Link
multilingual_e5_small	This is a multilingual sentence-transformer model that supports 100. Supports text embedding, translation, or multilingual understanding. Optimized for efficiency, it can handle multiple languages while maintaining a smaller size for faster inference and reduced computational requirements.	384	76.8	9/9/2024	Link
clip_vit_base_patch32_txt	This is a text encoder that produces embeddings compatible with the CLIP image encoder. Enables text-to-image search and similarity matching between text descriptions and images.	512	243.57	1/29/2021	Link
clip_vit_base_patch32_img	This is an image encoder that produces embeddings compatible with the CLIP text encoder. Enables image-to-text search and similarity matching between images and text descriptions.	512	335.38	1/29/2021	Link

The Python package oml.utils contains three classes: ONNXPipeline, ONNXPipelineConfig, and MiningFunction. The package handles ONNX pipeline generation and export, while also incorporating the necessary pre- and post-processing steps.

• ONNXPipeline : Allows you to import a pretrained model (Your own pretrained model or one of the supported models from Hugging Face)
• ONNXPipelineConfig : Allows you to configure the attributes of pretrained model (Your own pretrained model or one of the supported models from Hugging Face)
• MiningFunction: Allows you to choose from one of the following mining options:
  • EMBEDDING : Corresponds to text and image embedding
  • CLASSIFICATION : Corresponds to text classification
  • REGRESSION : Corresponds to re-ranking

WARNING:

EmbeddingModel and EmbeddingModelConfig are deprecated. Instead, please use ONNXPipeline and ONNXPipelineConfig respectively. The details of the deprecated classes can be found in Python Classes to Convert Pretrained Models to ONNX Models (Deprecated). If a you choose to use a deprecated class, a warning message will be shown indicating that the classes will be removed in the future and advising the user to switch to the new class.

The ONNX pipeline models are available for text embedding, image embedding, multi-modal embedding, text classification and re-ranking.

• Convert Pretrained Models to ONNX Model: End-to-End Instructions for Text Embedding
  This section provides end-to-end instructions from installing the OML4Py client to downloading a pretrained embedding model in ONNX-format using the Python utility package offered by Oracle.
• Python Classes to Convert Pretrained Models to ONNX Models
  
• ONNX Pipeline Models: Text Embedding
  ONNX pipeline models provides text embedding models that accepts text as input and produces embeddings as output. The pipeline models also provide the necessary pre-processing and post-processing needed for the text.
• ONNX Pipeline Models: Image Embedding
  ONNX pipeline models provides image embedding models that accepts image as an input and produces embeddings as output. The pipeline models also provide the necessary pre-processing.
• ONNX Pipeline Models: CLIP Multi-Modal Embedding
  ONNX pipeline models provides multi-modal embedding models that accepts both image and text as an input and produces embeddings as output. The pipeline models also provide the necessary pre-processing needed.
• ONNX Pipeline Models: Text Classification
  ONNX pipeline models provides text classification models that accepts text strings as input and produces the probablity of the input string belonging to a specific label. The pipeline models also provide the necessary pre-processing and post-processing.
• ONNX Pipeline Models: Reranking Pipeline
  ONNX pipeline models provide a reranking pipeline that calculates similarity score for a given pair of texts.
• Load Custom Models from The Local Filesystem
  Custom models that have been fine-tuned from pre-trained Hugging Face models and reside on the local filesystem can be converted to the ONNX format using the new local_model_path setting. This setting can also be useful in cases where networking is unavailable and a pre-trained model has previously been downloaded.