Generative Ai Inference¶
|
Asynchronous client for GenerativeAiInferenceClient. |
OCI Generative AI is a fully managed service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover a wide range of use cases for text generation, summarization, and text embeddings. |
|
|
This class provides a wrapper around |
Models¶
An annotation attached to the assistant's message, used to represent additional metadata such as citations. |
|
Details for applying guardrails to the input content. |
|
The result of applying guardrails to the input content. |
|
To refine search results based on geography, you can specify an approximate user location using any of the following: - city and region are free-text strings, like "Minneapolis" and "Minnesota". |
|
Represents a single instance of assistant message. |
|
Represents a single instance of chat audio content. |
|
Provide a base64 encoded audio or an audio uri if it's supported. |
|
The base class to use for the chat inference request. |
|
The base class that creates the chat response. |
|
A category with its score. |
|
Represents a single instance of the chat response. |
|
The base class for the chat content. |
|
Details of the conversation for the model to respond. |
|
The response to the chat conversation. |
|
Represents a single instance of the generated text. |
|
A section of the generated response which cites the documents that were used for generating the response. |
|
Options for controlling citation generation. |
|
A message that represents a single chat dialog as CHATBOT role. |
|
A message that represents a single chat dialog as CHATBOT role. |
|
Details for the chat request for Cohere models. |
|
Details for the chat request for Cohere models. |
|
The response to the chat conversation. |
|
The response to the chat conversation. |
|
A citation source referencing a document. |
|
A citation source object. |
|
A citation source referencing a tool result. |
|
A source reference or citation for a piece of content. |
|
The base class for the chat content. |
|
Represents a single instance of Chat Document content |
|
Represents a single instance of Chat Image content |
|
Provide a base64 encoded image or an image uri if it's supported. |
|
|
Details for the text generation request for Cohere models. |
|
The generated text result to return. |
A message that represents a single chat dialog. |
|
A message that represents a single message dialog. |
|
|
A definition of tool parameter. |
Specify the format the model output is guaranteed to be of |
|
The json object format for the model structured output |
|
The text format for cohere model response |
|
A message that represents a single chat dialog as SYSTEM role. |
|
A message that represents a single chat dialog as SYSTEM role. |
|
Represents a single instance of text in the chat content. |
|
Represents a single instance of Chat Doument content |
|
Configuration for reasoning features. |
|
A definition of tool (function). |
|
A tool call generated by the model. |
|
A tool call generated by the model. |
|
A message that represents a single chat dialog as TOOL role. |
|
A message that represents a single chat dialog as TOOL role. |
|
The result from invoking tools recommended by the model in the previous chat turn. |
|
A definition of tool (function). |
|
A message that represents a single chat dialog as USER role. |
|
A message that represents a single chat dialog as USER role. |
|
Breakdown of tokens used in a completion. |
|
|
Configuration for content moderation. |
The result of content moderation. |
|
The model's serving mode is dedicated serving and has an endpoint on a dedicated AI cluster. |
|
Developer-provided instructions that the model should follow, regardless of messages sent by the user. |
|
The input of the document to rerank. |
|
Represents a single instance of chat document content. |
|
An object that contains a relevance score, an index and the text for a document. |
|
The base64 encoded document data or a document uri if it's supported. |
|
The base class for the embed content. |
|
Represents a single instance of embed image content. |
|
Represents a single instance of text in the embed content. |
|
Details for the request to embed texts. |
|
The generated embedded result to return. |
|
The function to be executed. |
|
The function call generated by the model. |
|
A function the model may call. |
|
Details for the request to generate text. |
|
The generated text result to return. |
|
The text generated during each run. |
|
Details for the chat request. |
|
The response for a chat conversation. |
|
object containing the source. |
|
Grounding metadata. |
|
chunk to connect model response text to the source in groundingChunk |
|
segment within groundingSupport. |
|
object containing the web source. |
|
Additional configuration for each guardrail. |
|
Details of a guardrail version. |
|
|
The response containing a list of guardrail system versions. |
The version of the guardrail to use. |
|
The guardaril version applied on the given content. |
|
Details of a guardrail version. |
|
Represents a array of image in the guardrails input. |
|
Provide a base64 encoded image or an image uri if it's supported. |
|
The input data for applying guardrails. |
|
The results of applying each guardrail. |
|
Represents a single instance of text in the guardrails input. |
|
Represents a single instance of chat image content. |
|
Provide a base64 encoded image or an image uri if it's supported. |
|
Enables JSON mode, which ensures the message the model generates is valid JSON. |
|
Enables Structured Outputs which ensures the model will match your supplied JSON schema. |
|
Details for the text generation request for Llama models. |
|
|
The generated text result to return. |
The base class for the inference requests. |
|
The base class for inference responses. |
|
The log probabilities of the generated tokens. |
|
Includes the logarithmic probabilities for the most likely output tokens and the chosen tokens. |
|
A message that represents a single chat dialog. |
|
The model's serving mode is on-demand serving on a shared infrastructure. |
|
|
Configuration for personally identifiable information detection. |
|
An item of personally identifiable information. |
Configuration for a Predicted Output, which can greatly improve response times when large parts of the model response are known ahead of time. |
|
|
Configuration for prompt injection |
|
The result of prompt injection protection. |
Breakdown of tokens used in the prompt. |
|
Details required for a rerank request. |
|
The rerank response to return to the caller. |
|
An object specifying the format that the model must output. |
|
The JSON schema definition to be used in JSON_SCHEMA response format. |
|
Contains the HTML and CSS to render the required Search Suggestions. |
|
The generated search query. |
|
The model's serving mode, which is either on-demand serving or dedicated serving. |
|
Static predicted output content, such as the content of a text file that is being regenerated. |
|
Options for streaming response. |
|
Details for the request to summarize text. |
|
Summarize text result to return to caller. |
|
Represents a single instance of system message. |
|
Represents a single instance of text in the chat content. |
|
Enables TEXT mode. |
|
An object that contains the returned token and its corresponding likelihood. |
|
The tool call generated by the model, such as function call. |
|
The tool choice for a tool. |
|
The model can pick between generating a message or calling one or more tools. |
|
The tool choice for a function. |
|
The model will not call any tool and instead generates a message. |
|
The model must call one or more tools. |
|
A tool the model may call. |
|
Represents a single instance of tool message. |
|
Contains metadata for a cited URL included in the assistant’s response. |
|
Usage statistics for the completion request. |
|
Represents a single instance of user message. |
|
Represents a single instance of chat video content. |
|
The base64 encoded video data or a video uri if it's supported. |
|
Options for performing a web search to augment the response. |