Oracle Cloud Infrastructure Documentation / Release Notes

Add Image Moderation to OCI Generative AI Guardrails

You can now use the ApplyGuardrails API in OCI Generative AI to moderate image inputs through the existing content moderation capability. Image moderation is available with Guardrails system version 1.1.0.

This feature helps identify unsafe content in standalone images or multimodal requests that include both text and images. You can use it to evaluate user-uploaded images, generated images, screenshots, and images that contain embedded text.

To use image moderation, submit the request with multimodalInput, include an item with "type": "IMAGE", enable contentModerationConfig, and specify Guardrails system version 1.1.0 using guardrailVersionConfig.

Supported image formats include:

  • JPEG
  • PNG
  • WebP

A single request can include up to five images, and each image can contain up to 170 megapixels. Image inputs are also subject to image-token throttling, with a default limit of about 200,000 image tokens per minute. If you need a larger image-token limit, you can request a service limit increase.

Image moderation doesn't introduce a separate response object. Results are returned as part of the existing contentModeration result, including the OVERALL score. The new flaggedModalities field identifies which modality contributed to the moderation result. Supported values are TEXT and IMAGE.

Example response excerpt:

{
                "name": "OVERALL",
                "score": 1.0,
                "flaggedModalities": ["TEXT", "IMAGE"]
                }

For API examples and setup details, see About OCI Generative AI Guardrails.

Important

Disclaimer

Our Content Moderation (CM) and Prompt Injection (PI) guardrails have been evaluated on a range of multilingual benchmark datasets. However, actual performance may vary depending on the specific languages, domains, data distributions, and usage patterns present in customer-provided data as the content is generated by AI and may contain errors or omissions. Accordingly, it is intended for informational purposes only, should not be considered professional advice and OCI makes no guarantees that identical performance characteristics will be observed in all real-world deployments. The OCI Responsible AI team is continuously improving these models.

Our content moderation capabilities have been evaluated against RTPLX, one of the largest publicly available multilingual benchmarking datasets, covering more than 38 languages. However, these results should be interpreted with appropriate caution as the content is generated by AI and may contain errors or omissions. Multilingual evaluations are inherently bounded by the scope, representativeness, and annotation practices of public datasets, and performance observed on RTPLX may not fully generalize to all real-world contexts, domains, dialects, or usage patterns. Accordingly, the findings are intended to be informational purposes only and should not be considered professional advice.