Python Classes to Convert Pretrained Models to ONNX Models

This topic provides the functions, attributes and example usage of the python classes ONNXPipelineConfig and ONNXPipeline.

Functions and Attributes of ONNXPipelineConfig

The ONNXPipelineConfig class contains the properties required for the package to perform downloading, exporting, augmenting, validation, and storing of an ONNX model. The class provides access to configuration properties using the dot operator. As a convenience, well-known configurations are provided as templates.

Parameters

This table describes the functions and properties of the ONNXPipelineConfig class.

Functions	Parameter Type	Returns	Description
`from_template(name,**kwargs)`	`name` (`String`): The name of the template `**kwargs`: template properties to override or add	Instance of `ONNXPipelineConfig`	A static function that creates an `ONNXPipelineConfig` object based on a predefined template given by the name parameter. You can use named arguments to override the template properties.
`show_templates()`	NA	List of existing templates	A static function that returns a list of existing templates by name.
`show_preconfigured()`	`include_properties` (`bool,optional`): A flag indicating whether properties should be included in the results. Defaults to `False` so only names will be included by default. `model_name` (`str,optional`): A model name to filter by when including properties. This argument will be ignored if `include_properties` is `False`. Otherwise only the properties of this model will be included in the results.	A list of preconfigured model names or properties.	Shows a list of preconfigured model names, or properties. By default, this function returns a list of names only. If the properties are required, pass the `include_properties` parameter as `True`. The returned list will contain a single dict where each key of the dict is the name of a preconfigured model and the value is the property set for that model. Finally, if only a single set of properties for a specific model is required, pass the name of the model in the `model_name` parameter (the `include_properties` parameter should also be `True`). This will return a list of a single dict with the properties for the specified model.

Template Properties

The text template has configuration properties shown below:

"do_lower_case": true,
"post_processors":[{"name":"Pooling","type":"mean"},{"name":"Normalize"}]

Note: All other properties in the Properties table will take the default values. Any property without a default value must be provided when creating the ONNXPipelineConfig instance.

Properties

This table shows all properties that can be configured. preconfigured models already have these properties set to specific values. Templates will use the default values unless a user overrides it when using the from_template function on ONNXPipelineConfig.

Property	Description
`post_processors`	An array of post_processors that will be loaded after the model is loaded or initialized. The list of known and supported post_processors is provided later in this section. Templates may define a list of post_processors for the types of models they support. Otherwise, an empty array is the default.
`max_seq_length`	This property is applicable for text-based models only. The maximum length of input to the model as number of tokens. There is no default value. Specify this value for models that are not preconfigured.
`do_lower_case`	Specifies whether or not to lowercase the input when tokenizing. The default value is `True`.
`quantize_model`	Perform quantization on the model. This could greatly reduce the size of the model as well as speed up the process. It may however result in different results for the embedding vector (against the original model) and possibly small reduction in accuracy. The default value is `False`.
`distance_metrics`	An array of names of suitable distance metrics for the model. The names must be name of distance metrics used for Oracle vector distance operator. Only used when exporting the model to the database. Supported list is [“`EUCLIDEAN`”,”`COSINE`”,”`MANHATTAN`”,”`HAMMING`”,”`DOT`”,”`EUCLIDEAN_SQUARED`”, “`JACCARD`”]. The default value is an empty array.
`languages`	An array of language (Abbreviation) supported in the Database. Only used when exporting the model to the database. For a supported list of languages, see Languages. The default value is an empty array.
`use_float16`	Specifies whether or not to convert the exported onnx model to float16. The default value is `False`.

Properties of post_processors

This table describes the built-in post_processors and their configuration parameters.

post_processor	Parameters	Description
`Pooling`	`name`: `Pooling`. `type`: Valid values should be `mean`(Default), `max`, `cls`	The Pooling post_processor summarizes the output of the transformer model into a fixed-length vector.
`Normalize`	`name`: Specify `Normalize`	The Normalize post_processor bounds the vector values to a range using L2 normalization.
`Dense`	`name`: Dense `in_features`: Input feature size `out_features`: Output feature size `bias`: Whether to learn an additive bias. The default value is `True`. `activation_function`: Activation function of the dense layer. Currently only supports `Tanh` as the activation function.	Applies transformation to the incoming data.

Example: Configure post_processors

In this example, you override the post_processors in the sentence-transformers template with a Max Pooling post_processor followed by Normalization.

config = ONNXPipelineConfig.from_template("text")
config.post_processors = [{"name":"Pooling","type":"max"},{"name":"Normalize"}]

Functions and Attributes of ONNXPipeline

Use the ONNXPipeline class to convert transformer models to the ONNX format with post_processing steps embedded into the final model.

Parameters

This table describes the signature and properties of the ONNXPipeline class.

Functions	Parameters	Description
`ONNXPipeline(model_name,configuration=None,settings={})`	`model_name(str)`: Name of the model to be used. For example, `medicalai/ClinicalBERT`. `configuration (ONNXPipelineConfig, optional)`: An initialized `ONNXPipelineConfig` object. This parameter must be specified when using a template. If not specified, the model is assumed to be a preconfigured model. `settings (dict, optional)`: A dictionary of global settings that control operations such as logging levels, download behavior, and file locations.	Creates a new instance of the `ONNXPipeline` class.

Settings

The settings object is a dictionary passed to the ONNXPipeline class. It provides global properties used for non-model-specific operations, such as logging and model download behavior.

For example:

pipe = ONNXPipeline(model_name, config, settings={"ignore_safetensors_error": True})

where: ignore_safetensors_error=True allows execution to continue even if the requested model does not provide a safetensors version.

Property	Type	Default Value	Description
`cache_dir`	`str`	`$HOME/.cache/OML`	Directory where Hugging Face models are downloaded and cached. Model files will be downloaded to directories relative to the `cache_dir`. If the directory does not exist at execution time, it will be created automatically.
`logging_level`	`str`	`ERROR`	The level of severity for log messages. Valid values are `['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']`. Note: This log level is also applied globally to all Python packages and is also mapped to the ONNX Runtime libraries.
`force_download`	`bool`	`False`	Forces downloading model files from Hugging Face even if they already exist in the local cache.
`local_model_path`	`str`	`None`	Path to a local copy of a model to be exported instead of downloading the model from Hugging Face. Must be a valid directory path.
`cleanup`	`bool`	`True`	Controls whether intermediate files generated during model export are removed after execution. If set to `False`, intermediate export files are retained.
`ignore_checksum_error`	`bool`	`False`	Ignores any errors caused by mismatch in checksums when using preconfigured models.
`ignore_safetensors_error`	`bool`	`False`	Ignores errors when the requested model does not have a `safetensors` version available.

Functions

This table describes the function and properties of the ONNXPipeline class.

Function	Parameters	Description
`export2file(export_name,output_dir=None)`	`export_name(string)`: Name of the exported file. The file is saved with the `.onnx` extension. `output_dir(string)`: Output directory where the file will be saved. If not specified, the file is saved in the current working directory.	Exports the model to a local `.onnx` file.
`export2db(export_name)`	`export_name`(`string`): Name used for the mining model object in the database. The name must comply with database object naming rules.	Exports the model to the database.

Example: Preconfigured Model

This example illustrates the preconfigured embedding model that comes with the Python package. You can use this model without any additional configurations.

"sentence-transformers/distiluse-base-multilingual-cased-v2": {
        "max_seq_length": 128,
        "do_lower_case": false,
        "post_processors":[{"name":"Pooling","type":"mean"},{"name":"Dense","in_features":768, "out_features":512, "bias":true, "activation_function":"Tanh"}],
        "quantize_model":true,
        "distance_metrics": ["COSINE"],
        "languages": ["ar", "bg", "ca", "cs", "dk", "d", "us", "el", "et", "fa", "sf", "f", "frc", "gu", "iw", "hi", "hr", "hu", "hy", "in", "i", "ja", "ko", "lt",
                      "lv", "mk", "mr", "ms", "n", "nl", "pl", "pt", "ptb", "ro", "ru", "sk", "sl", "sq", "lsr", "s", "th", "tr", "uk", "ur", "vn", "zhs", "zht"]
    }