Use Private Endpoints for On-Demand Mode in OCI Generative AI

You can now access pretrained models through OCI Generative AI private endpoints for the on-demand mode.

An OCI Generative AI private endpoint is a private IP address within an OCI virtual cloud network (VCN). Generative AI service sets up the private endpoint in a private subnet of the VCN. Think of the private endpoint as another VNIC within the VCN. You control access to it similar to any other VNIC, using security rules. The service creates this VNIC, and you manage the subnet and its security rules.

To access a Generative AI model through a private endpoint for the on-demand serving mode, select Allow Usage In On-Demand Mode when you create or edit a private endpoint. When you create a private endpoint, you receive a fully qualified domain name (FQDN) for it, which you can use to access on-demand models through a Compute instance that you create in the private subnet.

Learn about Private Endpoints in Generative AI. For information about the service, see the Generative AI documentation.