Container Input Validation

Inputs to the Private AI Services Container can come from a variety of sources, each of which are validated using different methods.

Inputs to the container include HTTP request input from clients, files from the host system provided by the container admin, and environment variables set by the container admin when starting the container.

HTTP Request Input (REST API)

The REST API accepts the media type application/json for all requests that have a body (payload), and returns application/json for all responses. Input is parsed and deserialized to an Object using the serialization framework provided by Micronaut. Any errors during this process will be caught and returned as HTTP 400 to the client along with a message describing the problem. After deserialization, the Request Objects are validated against rules that check for the following:

• Null or empty
• That numeric parameters are numbers of the required type and within the required range
• That text, URI, and UUID parameters match expected values or regular expressions corresponding to the required type

Any failures in validation at this point will also be handled and an HTTP 400 error is returned with a message describing the problem. All regular expressions used in the validation of input data are evaluated to confirm that they are not susceptible to Regular Expression denial-of-service (DOS) vulnerabilities.

User Models

The container admin can specify user models as part of the config.json file. These models can be on the host file system or specified using a Pre-Authenticated Request (PAR) link. Model files can be .onnx files or .zip files that contain a single .onnx file and metadata. In all cases, the container expects the ONNX format model to be a pipeline that matches a specific signature based on metadata. The metadata attributes used to determine the signature of the ONNX format model are modelType and function.

Given the metadata, the container will introspect the model to ensure that the inputs and outputs of the ONNX graph match the expected signature. The following table shows the relationship between metadata, signature, and input/output shape:

Signature	Model Type	Model Function	Input Shape	Output Shape
Image Embedding	ONNX_IMG	embedding	Single-dimensional array of UINT8	Single-dimensional array of embeddings
Text Embedding	ONNX_TXT	embedding	2-dimensional array of String	2-dimensional array of embeddings

ONNX Files

The container checks the validity of ONNX files by loading them with the onnx library and verifying their signatures using the methods described previously. For standalone ONNX files, you specify the location of the ONNX file in the config.json file along with the metadata, for example:

{
    "modelname":"bert_tiny",
    "modelfile":"bert_tiny.onnx",
    "modelfunction": "EMBEDDING",
    "properties":{
        "PRIVATE_AI_ONNX_INTRA_OP_NUM_THREADS":4
    }
}

In this example, the ONNX file is bert_tiny.onnx, which will be loaded from /privateai/models/bert_tiny.onnx on the container file system. This path must be mounted to a directory on the host system that contains the ONNX file. The property modelfunction, along with the input shape and data type from the ONNX model file, are used to validate the model before deployment.

Zip Files

For large models, typically those over 2GB, you may use OML4Py to generate a compatible ONNX format model. Because ONNX relies on a protobuffer that does not support files larger than 2GB, the ONNX Runtime supports loading such models using external data. The OML4Py tool has built in support for generating external data for such models that result in multiple files, including a metadata file. For portability and convenience, these files are zipped and the zip file can be provided to the container for loading.

In addition to the validation of ONNX files as described previously, zip files go through specific validation to prevent malicious Zip Bomb attacks. The files that are expected to be in the zip are fixed and each have maximum allowable sizes that are checked. The container determines a zip file by checking its binary signature.