17 OCI Generative AI (Pretrained Foundation Models)

Generative AI is a fully managed Oracle Cloud Infrastructure service that provides a set of state-of-the-art, customisable large language models (LLMs) that cover a wide range of use cases, including chat, text generation, summarization, and creating text embeddings.

AI Data Platform users can access Generative AI models if they have the requisite permissions and the pre-trained model is hosted in the same region as the data lake. For more information on permissions, see Getting Access to Generative AI. For more information on where Generative AI models are hosted, see Regions with Generative AI.

You can use Generative AI models in AI Data Platform for the following use cases:
  • Use the pre-trained chat models to create text for any purpose.
  • Extract specific pieces of data from text.
  • Generate executive summaries for documents that are too long to read, or summarize any type of text.
  • Classify text into predefined categories.

You can also run batch inferences on Spark Data Frames using the pre-trained models in a language of your choice, like SQL or Python. For more information on pretrained models, see Pretrained Foundational Models in Generative AI.

Prerequisites for Generative AI

You must meet the following prerequisites to use Generative AI in AI Data Platform:
  • User must have USE permissions on the base models
  • AI Data Platform is in the same region where the Generative AI models are hosted

If the prerequisites are met, the models are listed in the default.oci_ai_models schema. You can then list the models in the catalog explorer while working in a notebook and drag drop the models to generate sample code or use the model for batch inference. Alternatively, you can choose to write your code in an AI Data Platform notebook to invoke the model.

You can use the following methods to invoke a Generative AI model:

SQL
select *, query_model(model_name, concat("What is the sentiment for this review: ", review)) as sentiment from <<catalog_name>>.<<schema_name>>.<<table_name>>

Where:

  • model_name is the generative AI model you want to invoke: default.oci_ai_models.<model_name>
  • review is the column name that is used to create the prompt
  • sentiment is the output column name
  • <<catalog_name>>.<<schema_name>>.<<table_name>> is the table in 3-part name pattern
PySpark
df.withColumn("sentiment", query_model(model_name, "What is the sentiment for this review: "+review))
Where:
  • model_name is the generative AI model you want to invoke: default.oci_ai_models.<model_name>
  • review is the column name that is used to create the prompt
  • sentiment is the output column name
  • df is the input data frame

Request Limit

Description Limit Name Service Limit
Maximum number of chat requests per minute allowed per compartment for on-demand inferencing max-on-demand-chat-request-per-minute-count 500