Oracle Cloud Infrastructure Documentation

Model Deployment Environment Variables

When you create a model deployment resource, you can provide custom environment variables as part of the default configuration. These environment variables are available for use inside the container at runtime.

Format constraint

  • Key must be under 32 characters.
  • Key must contain only letters, digits, and underscore (_).
  • Key must start with a letter.
  • Key must have at least 2 characters.
  • Key must not end with underscore. For example, TEST_.
  • Key, if added, can't be empty. Value can be empty.
  • Value could be any character.
  • Key can't be service-provided environment variables.
  • If ports are passed as an environment variable, then limit them between 1024 and 65535 and also exclude ports 24224, 8446, 8447.

Size constraint

Individual values don't have specific size limits. The overall environment variables including key value pairs is limited to 2048 bytes. If the size is more than 2048 bytes, specify the environment variables by using the model artifact or use OCI Object Storage to pull the data.

Service Provided Environment Variables

The service has reserved environment variables inside the container at runtime for logging or other functional use. These environment variables can't be overridden.

Name Custom Container Default Container
MODEL_DEPLOYMENT_OCID Model deployment resource OCID. Model deployment resource OCID.
MODEL_SERVING_PYTHON_VERSION None. The Python version from runtime.yaml used for model serving.
TENANCY_OCID Tenancy OCID. Tenancy OCID.
CONTAINER_CUSTOM_IMAGE Container image path (with tag or digest) that's provided through the API environment configuration object. None.

Application Environment Variables

These environment variables functionally control how the model deployment is created and can be overridden or configured by the customers by passing in the default configuration.

Name Custom Container Managed Container
WEB_CONCURRENCY Based on available memory and size of model artifact, a heuristic method calculates the number of workers needed to run inside the custom container. Customers can also override the variable or pass a custom concurrency variable as required. Based on available memory and size of model artifact, a heuristic method calculates the number of workers needed to run inside the custom container. Customers can also override the variable or pass a custom concurrency variable as required.
CONTAINER_TYPE Type of container viz TRITON. Only TRITON is supported. None.