Private AI Services Container API Reference

models

Use GET requests to print a list of all currently deployed models.

Syntax

/v1/models

Returns

A list of deployed models.

Example Output

{
    "data": [
        {
            "id": "all_minilm_l12",
            "modelDeployedTime": "2026-02-09T00:35:11.365558079Z",
            "modelSize": "127.13M",
            "capabilities": [
                "TEXT_EMBEDDINGS"
            ]
        },
        {
            "id": "Phi-3.5-mini-instruct",
            "modelDeployedTime": "2026-02-09T00:26:58.875399477Z",
            "modelSize": "7.12G",
            "capabilities": [
                "TEXT_GENERATION"
            ]
        }
    ]
}

models/{id}

Use GET requests to print information about a specific model.

Syntax

/v1/models/{id}

Parameters

id (string): A unique model name. This parameter is required.

Returns

Information about the model specified by model ID.

Example Output

Model information:

{
  "id": "L55808652807957200809612118083123839056757756025",
  "modelSize": "string",
  "modelDeployedTime": "2025-12-22T16:52:28.365Z",
  "capabilities": [
    "TEXT_EMBEDDINGS"
  ]
}

404: Model not found.

embeddings

Use POST requests to get embeddings against a model.

Syntax

/v1/embeddings

Parameters

x-convert-images (boolean): Indicates whether images in the input list require conversion to JPG. The default value is false.

Example Input

Note that input can be a string or an array of strings.

{
  "input": "string",
  "model": "string"
}

Example Output

Embedding results:

{
  "data": [
    {
      "embedding": [
        0
      ],
      "index": 0
    }
  ],
  "model": "string"
}

400: Error processing the request data.
404: Model not found.
500: An error occurred during the score operation for this model.

health

Use GET requests to verify that the container is ready to use.

Syntax

/health

Example Output

200: Private AI Services Container is up and running.
401: Unauthorized
500: Internal server error

metrics

Use GET requests to return a list of metric names exposed by the application.

Syntax

/metrics

Returns

Returns a list of metric names exposed by the application.

Example Output

Successful response with metric names:

{
  "names": [
    "embeddings_call_error_total",
    "embeddings_call_latency",
    "embeddings_call_success_total",
    "embeddings_call_total",
    "embeddings_last_latency",
    "http.server.requests",
    "jvm.memory.used",
    "process.cpu.usage",
    "system.cpu.usage"
  ]
}

401: Unauthorized
500: Internal server error

metrics/{metricName}

Use GET requests to return detailed information for a metric, including measurements and available tags. Supports optional tag filters using repeated tag query params in the form key:value.

Syntax

/metrics/{metricName}

Parameters

metricName (string): Metric name as returned by GET /metrics. This parameter is required.

tag (array<string>): Tag filter(s) in the form key:value. Repeat for multiple tags.

Example Output

Metric details:

{
  "name": "embeddings_call_error_total",
  "description": "Total number of errors from embeddings calls.",
  "baseUnit": "count",
  "measurements": [
    {
      "statistic": "COUNT",
      "value": 3
    }
  ],
  "availableTags": [
    {
      "tag": "model"
    },
    {
      "tag": "status",
      "values": [
        "success",
        "error"
      ]
    }
  ]
}

400: Invalid tag filter
404: Metric not found
500: Internal server error

api

Use GET requests to return the OpenAI specification for this API in YAML format.

Syntax

/v1/api

Returns

Returns the OpenAPI (YAML) document as a string.

chat/completions

Use POST requests to accept a chat completion request and return a generated response.

Syntax

/v1/chat/completions

Parameters

model (string): Model identifier to use for the completion. Can be the name of the model zip file. Required parameter.
messages (array of object): Conversation messages for the completion. This parameter is required. Each message content can be plain text or a list of typed content parts for multimodal requests. The following objects are included:
- role (string): The role of the message author. This parameter is required. The possible values are:
  - system
  - user
  - assistant
  - developer
  - tool
  - function
- content (string or array of object): The message content as plain text or as a list of typed content parts. In case of an array, the following properties are included:
  - type (string): Content part type, which can be one of the following:
    - text (string): Text payload for content parts with type text.
    - image_url (object): Image payload for content parts with type image_url. A url (string) field is included that provides base64-encoded image data.
- tool_calls (array of object): Tool calls produced by an assistant message.
- tool_call_id (string): Identifier of the tool call answered by a tool message.
- name (string): Function or tool name associated with the message.
- function_call (object): Legacy function call payload.
tools (array of object): Tool definitions to pass through to the chat runtime.
tool_choice (string or object): Tool selection policy to pass through to the chat runtime.
parallel_tool_calls (boolean): Whether the chat runtime may emit parallel tool calls.
temperature (number): Sampling temperature. Default value is 1.0.
stream (boolean): Whether to stream tokens back as they are generated.
stream_options (object): Options that apply when stream is set to true. Includes the following properties:
- include_obfuscation (boolean): Include obfuscation metadata in stream responses (if supported).
- include_usage (boolean): Include usage metrics in stream responses (if supported).
max_tokens (integer): Maximum number of tokens to generate. Default value is 512.

Example Output

Response payload for a chat completion endpoint, as an object including the following properties:
- id (string)
- object (string)
- created (integer): Unix epoch seconds.
- model (string)
- choices (array of object), including the following properties:
  - index (integer)
  - message (object), including the following properties:
    - role (string): Role of the message author.
    - content (string)
    - reasoning_content (string)
    - tool_calls (array of object): Tool call payloads (untyped in your model).
  - delta (object), with the following properties:
    - role (string): Role of the message author.
    - content (string)
    - reasoning_content (string)
    - tool_calls (array of object): Tool call payloads (untyped in your model).
  - finish_reason (string): Reason generation stopped (if provided).
- usage (object), with the following properties:
  - prompt_tokens (integer)
  - total_tokens (integer)
  - completion_tokens (integer)
- error (object): Error details returned by the API when a request fails, including the following properties:
  - message (string): Human-readable error message.
  - type (string): Error category, or type.
  - param (string): Parameter associated with the error (if applicable).
  - code (integer): Numeric error code.

rerank

Use POST requests to get rerankings against a model.

Syntax

/v1/rerank

Parameters

query (string): Text to be compared with the list of documents. Required parameter.

documents (array of string): List of documents that will get sorted according to their relevance to the query. Required parameter.

model (string): A unique model name. Required parameter.

top_n (integer): Number of results with bigger relevance score to return.

max_tokens_per_doc (integer): Maximum tokens to consider per document.

Example Output

Score results, as an object with the following properties:
- id (string)
- model (string)
- results(array of object), with the following properties:
  - index (number)
  - relevance_score (number)
- usage (object), with the following property:
  - total_tokens (integer)
- error (object): Error details returned by the API when a request fails, including the following properties:
  - message (string): Human-readable error message.
  - type (string): Error category, or type.
  - param (string): Parameter associated with the error (if applicable).
  - code (integer): Numeric error code.
400: Error processing the request data.
404: Model not found.
500: An error occurred during the score operation for this model.

classify

Use POST requests to classify one or more text inputs using the specified model. Use this endpoint with models configured for TEXT_CLASSIFICATION, such as an ONNX sentiment model.

Syntax

/classify

Parameters

model (string): Model identifier to use for classification. Required.

input (array): List of input texts to classify. Required.

Example Output

Response payload for text classification with the following fields:
- id (string): A unique identifier for the response.
- created (integer): Creation timestamp (epoch seconds).
- model (string): Model identifier used.
- data (array): Per-input classification results, including the following properties:
  - index (integer): Index of the input item in the request.
  - label (string): Predicted label.
  - probs (array): Probabilities corresponding to each class.
  - num_classes (integer): Total number of classes.
- usage (object): Token usage accounting, including the following properties:
  - prompt_tokens (integer)
  - total_tokens (integer)
  - completion_tokens (integer)
  - prompt_tokens_details (object or array): Additional prompt token breakdown (structure may vary).
- error (object): Error details returned by the API when a request fails, including the following properties:
  - message (string): Human-readable error message.
  - type (string): Error category, or type.
  - param (string): Parameter associated with the error (if applicable).
  - code (integer): Numeric error code.
400: Error processing the request data.
404: Model not found.
500: An error occurred during the classification operation for this model.

A Private AI Services Container API Reference