Guardrails

Guardrails can be used with LLMs to determine whether user prompts should be accepted or rejected, guarding against prompt injection.

A classification model is used to label prompt input to the LLM as either malicious or benign, allowing only benign inputs to be sent to the target LLM. Requests are forwarded to the target model, which must be an LLM with the TEXT_GENERATION capability loaded in the same container.

The Private Large Language Model Service includes PromptGuard2 as the default guardrail model for LLMs using both the vLLM and llama.cpp runtime engines. Guardrails are enabled by default for any model with the TEXT_GENERATION capability. The guardrail property is a JSON object with the following properties:

Property Name	Default Value	Description
permit_threshold	0.90	An optional threshold value for the probability of the permit label (the label that determines if a prompt should be allowed to pass through to the LLM). Values below this threshold will still be rejected even if the guardrail model predicts a benign label.
enabled	true	An optional flag indicating whether or not the guardrail should be enabled.

Note:

If the "system" role is provided in the PRIVATE_AI_ALLOWED_MESSAGE_ROLES property, the guardrail may block messages associated with this role. If the application needs to pass messages with the system role, the guardrail may need to be disabled.

Guardrails are configured by the container administrator and are not visible to clients, including the database. Failure to pass a guardrail results in an HTTP 400 error.

In the following example, permit_threshold is changed to 0.95 for a vLLM model:

"models": [
  { 
    "name": "Llama-3.1-8B-Instruct",
    "path": "meta-llama/Llama-3.1-8B-Instruct",
    "runtime": "vllm",
    "capabilities": ["TEXT_GENERATION"],
    "guardrail":{
       "enabled":true,
       "permit_threshold": 0.95
    },
    "startup_timeout": 1200,
    "runtime_arguments": [
      "--max-model-len", "4096"
    ]
  }
]

In this example, the guardrail is disabled for a llama.cpp model:

"models": [
  {
    "name": "gpt-oss-120b-GGUF",
    "path": "unsloth/gpt-oss-120b-GGUF",
    "runtime": "llamacpp",
    "capabilities": ["TEXT_GENERATION"],
    "guardrail":{
       "enabled":false,
       "permit_threshold": 0.95
    },
    "startup_timeout": 1200,
    "runtime_arguments": [
      "-c", "32768"
    ],
    "properties": {
      "primary_gguf": "gpt-oss-120b-F16.gguf"
    }
  }
]