Private AI Environment Variables
Environment variables can be used to set configuration properties, such as
file locations, logging settings, and others. They can be passed to the container when
running the container using the standard -e flag.
Typically, the JSON configuration file is the preferred method to
specify most configuration settings, as it centralizes the settings and simplifies
management. However, certain properties can only be set as environment variables
using the -e flag and cannot be specified in the configuration
file. This is because these environment variables are processed before the JSON file
is loaded. The exceptions are listed in the following table:
| Variable Name | Type | Default Value | Value Range | Description |
|---|---|---|---|---|
PRIVATE_AI_CONFIG_FILE |
String | None | None |
Specifies the location of a config.json file (may have a different file name) that will be used as the configuration file. This variable is required if
|
PRIVATE_AI_DRYRUN |
Boolean | FALSE |
|
This optional variable specifies whether to run the container in dry run mode. Information about dry run mode can be found at the end of this section. |
PRIVATE_AI_LOG_STDOUT_ENABLED |
Boolean | FALSE |
|
This optional variable specifies whether to display logs in the console or write to the log directory. |
The following list of environment variables (all optional) can be set
either in the JSON configuration file or using the -e flag when
running the container.
| Variable Name | Type | Default Value | Value Range | Description |
|---|---|---|---|---|
PRIVATE_AI_ONNX_INTRA_OP_NUM_THREADS |
Number | None | From 1 to the number of available processors on the host | The number of intra-op threads for the ONNX Runtime. This environment variable is used as the global default when using multiple models. The property can be set on individual models using the configuration file. |
PRIVATE_AI_MODEL_SCORING_TIMEOUT |
Number | 120 | 1 to 3600 | The maximum time (in seconds) that an ONNX model scoring thread may execute before an attempt is made to cancel it. |
PRIVATE_AI_MODEL_CACHE_TTL |
Number | 5400 | 1 to 7200 | The maximum idle time (in seconds) before a cache entry is expired and removed from the model cache. |
PRIVATE_AI_MODEL_CACHE_MAXENTRIES |
Number | 16 | 1 to 64 | The maximum number of entries that may exist in the model cache before the LRU entry is evicted. |
PRIVATE_AI_MAX_FILE_SIZE |
Number | 800,000,000 | 1 or greater | The maximum model input byte length |
PRIVATE_AI_MAX_IMG_SIZE |
Number | 20,000,000 | 1 or greater | The maximum image byte length for an ONNX model. |
PRIVATE_AI_LOG_LEVEL |
String | INFO |
INFO, OFF,
SEVERE, WARNING,
FINE, FINER,
FINEST, ALL |
The logging level. |
PRIVATE_AI_MAX_MODEL_COUNT |
Number | 64 | 1 or greater | The maximum number of models allowed in one configuration file. |
PRIVATE_AI_MAX_SCORE_PAYLOAD |
Number | 20,000,000 | 1 or greater | The maximum payload for scoring requests. |
PRIVATE_AI_MAX_BATCHSIZE |
Number | 256 | 1 or greater | The maximum batch size. |
The PRIVATE_AI_DRYRUN environment variable enables dry run mode,
which allows Kubernetes operators and automation tools to verify container readiness
and environmental correctness without launching the full application server. Dry run
mode is especially useful for validating dynamic secrets, environment variables, and
configuration files prior to exposing application endpoints or initiating expensive
processes.
In dry run mode, the container performs all validation and initialization steps normally performed during setup, except it does not start the Micronaut server. This approach helps detect misconfigurations early and integrates seamlessly into DevOps and Kubernetes workflows. The container process will exit immediately after successful validation or upon first failure.
The result of a dry run is indicated by one of two files in the
container at /privateai/app/:
dryrun.suc: indicates validation was successful.dryrun.dif: indicates validation failed; the file contains the reason(s) for the failure.