11.2 Python Classes to Convert Pretrained Models to ONNX Models
Explore the functions and attributes of the
EmbeddingModelConfig
class and EmbeddingModel
class
within Python. These classes are designed to configure pretrained embedding
models.
EmbeddingModelConfig
The EmbeddingModelConfig
class contains the properties
required for the package to perform downloading, exporting, augmenting, validation,
and storing of an ONNX model. The class provides access to configuration properties
using the dot operator. As a convenience, well-known configurations are provided as
templates.
Parameters
This table describes the functions and properties of the
EmbeddingModelConfig
class.
Functions | Parameter Type | Returns | Description |
---|---|---|---|
from_template(name,**kwargs) |
|
Instance of
EmbeddingModelConfig |
A static function that creates an
EmbeddingModelConfig object based on a
predefined template given by the name parameter. You can use named
arguments to override the template properties.
|
show_templates() |
NA | List of existing templates | A static function that returns a list of existing templates by name. |
show_preconfigured() |
|
A list of preconfigured model names or properties. | Shows a list of preconfigured model names, or
properties. By default, this function returns a list of names only.
If the properties are required, pass the
include_properties parameter as
True . The returned list will contain a single
dict where each key of the dict is the name of a preconfigured model
and the value is the property set for that model. Finally, if only a
single set of properties for a specific model is required, pass the
name of the model in the model_name parameter (the
include_properties parameter should also be
True ). This will return a list of a single dict
with the properties for the specified model.
|
Template Properties
The text template has configuration properties shown below:
"do_lower_case": true,
"post_processors":[{"name":"Pooling","type":"mean"},{"name":"Normalize"}]
Note:
All other properties in the Properties table will take the default
values. Any property without a default value must be provided when creating the
EmbeddingModelConfig
instance.
Properties
This table shows all properties that can be configured. preconfigured
models already have these properties set to specific values. Templates will use the
default values unless a user overrides it when using the
from_template
function on
EmbeddingModelConfig
.
Property | Description |
---|---|
|
An array of post_processors that will be loaded after the model is loaded or initialized. The list of known and supported post_processors is provided later in this section. Templates may define a list of post_processors for the types of models they support. Otherwise, an empty array is the default. |
|
This property is applicable for text-based models only. The maximum length of input to the model as number of tokens. There is no default value. Specify this value for models that are not preconfigured. |
|
Specifies whether or not to lowercase the input when
tokenizing. The default value is True .
|
|
Perform quantization on the model. This could
greatly reduce the size of the model as well as speed up the
process. It may however result in different results for the
embedding vector (against the original model) and possibly small
reduction in accuracy. The default value is
False .
|
distance_metrics |
An array of names of suitable distance metrics for
the model. The names must be name of distance metrics used for
Oracle vector distance operator. Only used when exporting the model
to the database. Supported list is
["EUCLIDEAN ","COSINE ","MANHATTAN ","HAMMING ","DOT ","EUCLIDEAN_SQUARED "].
The default value is an empty array.
|
languages |
A array of language (Abbreviation) supported in the Database. Only used when exporting the model to the database. For a supported list of languages, see Languages. The default value is an empty array. |
use_float16 |
Specifies whether or not to convert the exported onnx
model to float16. The default value is
False .
|
Properties of post_processors
This table describes the built-in post_processors and their configuration parameters.
post_processor | Parameters | Description |
---|---|---|
|
|
The Pooling post_processor summarizes the output of the transformer model into a fixed-length vector. |
|
|
The Normalize post_processor bounds the vector values to a range using L2 normalization. |
|
|
Applies transformation to the incoming data. |
Example: Configure post_processors
Max
Pooling post_processor
followed by Normalization.
config = EmbeddingModelConfig.from_template("text")
config.post_processors = [{"name":"Pooling","type":"max"},{"name":"Normalize"}]
EmbeddingModel
Use the EmbeddingModel
class to convert transformer
models to the ONNX format with post_processing steps embedded into the final
model.
Parameters
This table describes the signature and properties of the
EmbeddingModel
class.
Functions | Parameters | Description |
---|---|---|
EmbeddingModel(model_name,configuration=None,settings={}) |
|
Creates a new instance of the
EmbeddingModel class.
|
Settings
The settings object is a dictionary passed to the
EmbeddingModel
class. It provides global properties for the
EmbeddingModel
class that are used for non-model-specific
operations, such as logging.
Property | Default Value | Description |
---|---|---|
cache_dir |
$HOME/.cache/OML |
The base directory used for downloads. Model files
will be downloaded from the repository to directories relative to
the cache_dir . If the cache_dir
does not exist at time of execution, it will be created.
|
logging_level |
ERROR |
The level for logging. Valid values are [‘DEBUG’,
‘INFO’, ‘WARNING’, ‘ERROR’, ‘CRITICAL’].
Note: This log level is also applied globally to all python packages and is also mapped to the ONNXRuntime libraries. |
force_download |
False |
Forces download of model files instead of reloading from cache. |
ignore_checksum_error |
False |
Ignores any errors caused by mismatch in checksums when using preconfigured models. |
Functions
This table describes the function and properties of the
EmbeddingModel
class.
Function | Parameters | Description |
---|---|---|
export2file(export_name,output_dir=None) |
|
Exports the model to a file. |
export2db(export_name) |
|
Exports the model to the database. |
Example: Preconfigured Model
This example illustrates the preconfigured embedding model that comes with the Python package. You can use this model without any additional configurations.
"sentence-transformers/distiluse-base-multilingual-cased-v2": {
"max_seq_length": 128,
"do_lower_case": false,
"post_processors":[{"name":"Pooling","type":"mean"},{"name":"Dense","in_features":768, "out_features":512, "bias":true, "activation_function":"Tanh"}],
"quantize_model":true,
"distance_metrics": ["COSINE"],
"languages": ["ar", "bg", "ca", "cs", "dk", "d", "us", "el", "et", "fa", "sf", "f", "frc", "gu", "iw", "hi", "hr", "hu", "hy", "in", "i", "ja", "ko", "lt", "lv", "mk", "mr", "ms", "n", "nl", "pl", "pt", "ptb", "ro", "ru", "sk", "sl", "sq", "lsr", "s", "th", "tr", "uk", "ur", "vn", "zhs", "zht"]
}
Parent topic: Convert Pretrained Models to ONNX Format