11 Convert Pretrained Models to ONNX Format
OML4Py enables the use of text transformers from Hugging Face by converting them into ONNX format models. OML4Py also adds the necessary tokenization and post-processing. The resulting ONNX pipeline is then imported into the database and can be used to generate embeddings for AI Vector Search.
Note:
-
This feature will only work on OML4Py client. It is not supported on the OML4Py server.
-
In-database embedding models must include tokenization and post-processing. Providing only the core ONNX model to
DBMS_VECTOR.LOAD_ONNX_MODEL
is insufficient because you need to handle tokenization externally, pass tensors into the SQL operator, and convert output tensors into vectors. - Oracle is providing a Hugging Face
all-MiniLM-L12-v2 model in ONNX format, available to download
directly to the database using
DBMS_VECTOR.LOAD_ONNX_MODEL
. For more information, see the blog post Now Available! Pre-built Embedding Generation model for Oracle Database 23ai.
If you do not have a pretrained embedding model in ONNX-format to
generate embeddings for your data, Oracle offers a Python utility package that
downloads pretrained models from an external source, converts the model to ONNX
format augmented with pre-processing and post-processing steps, and imports the
resulting ONNX-format model into Oracle Database. Use the
DBMS_VECTOR.LOAD_ONNX_MODEL
procedure or OML4Py's
export2db()
function to import the file as a mining
model..
Then leverage the in-database ONNX Runtime with the ONNX model to produce vector
embeddings.
- Downloads the pretrained model from external source to your system
- Augments the model with pre-processing and post-processing steps and creates a new ONNX model
- Validates the augmented ONNX model
- Loads into the database as a mining model or optionally exports to a file
The Python utility can take any of the models in the preconfigured list as input. Alternatively, you can use the built-in template that contains common configurations for certain groups of models such as text-based models. To understand what a preconfigured list, what is a built-in template is, and how to use them, read further.
Limitations
This table describes the limitations of the Python utility package.
Note:
This feature is available with the OML4Py client only.Parameter | Description |
---|---|
Transformer Model Type |
Currently supported only for text transformers. |
Model Size |
Model size should be less than 1GB. Quantization can help reduce the size. |
Tokenizers |
Must be either BERT ,
GPT2 , SENTENCEPIECE , or
ROBERTA .
|
Preconfigured List of Models
Preconfigured list of models are common models from external resource
repositories that are provided with the Python utility. The preconfigured models
have an existing specification. Users can create their own specification using the
text template as a starting point. To get a list of all model names in the
preconfigured list, you can use the show_preconfigured
function.
Templates
The Python utility package provides built-in text template for you to configure the pretrained models with pre-processing and post-processing operations. The template has a default specification for the pretrained models. This specification can be changed or augmented to create custom configurations. The text template uses Mean Pooling and Normalization as post-processing operations by default.
-
EmbeddingModelConfig
-
EmbeddingModel
To learn more about the Python classes, their properties, and to configure the properties, see Python Classes to Convert Pretrained Models to ONNX Models.
-
OML4Py Client running on Linux X64 for On-Premises Databases
-
Python 3.12 (the earlier versions are not compatible)
DBMS_VECTOR.LOAD_ONNX_MODEL
is only needed if
export2file
was used to save the ONNX model file to the local
system instead of using export2db
to save the model in the database.
The DBMS_VECTOR.LOAD_ONNX_MODEL
imports the ONNX format model into the
Oracle Database to leverage the in-database ONNX Runtime to produce vector embeddings
using the VECTOR_EMBEDDING
SQL operator.
See Also:
- Oracle Database SQL
Language Reference for information about the
VECTOR_EMBEDDING
SQL function - Oracle Database PL/SQL
Packages and Types Reference for information about the
IMPORT_ONNX_MODEL
procedure - Oracle Database PL/SQL
Packages and Types Reference for information about the
LOAD_ONNX_MODEL
procedure - Oracle Machine Learning for SQL Concepts for more information about importing pretrained embedding models in ONNX format and generating vector embeddings
- https://onnx.ai/onnx/intro/ for ONNX documentation
- Convert Pretrained Models to ONNX Model: End-to-End Instructions
This section provides end-to-end instructions from installing the OML4Py client to downloading a pretrained embedding model in ONNX-format using the Python utility package offered by Oracle. - Python Classes to Convert Pretrained Models to ONNX Models
Explore the functions and attributes of theEmbeddingModelConfig
class andEmbeddingModel
class within Python. These classes are designed to configure pretrained embedding models.