GenericChatRequest¶

class oci.generative_ai_inference.models.GenericChatRequest(**kwargs)¶

Bases: oci.generative_ai_inference.models.base_chat_request.BaseChatRequest

Details for the chat request.

Attributes

`API_FORMAT_COHERE`	str(object=’’) -> str
`API_FORMAT_GENERIC`	str(object=’’) -> str
`REASONING_EFFORT_HIGH`	A constant which can be used with the reasoning_effort property of a GenericChatRequest.
`REASONING_EFFORT_LOW`	A constant which can be used with the reasoning_effort property of a GenericChatRequest.
`REASONING_EFFORT_MEDIUM`	A constant which can be used with the reasoning_effort property of a GenericChatRequest.
`REASONING_EFFORT_MINIMAL`	A constant which can be used with the reasoning_effort property of a GenericChatRequest.
`VERBOSITY_HIGH`	A constant which can be used with the verbosity property of a GenericChatRequest.
`VERBOSITY_LOW`	A constant which can be used with the verbosity property of a GenericChatRequest.
`VERBOSITY_MEDIUM`	A constant which can be used with the verbosity property of a GenericChatRequest.
`api_format`	[Required] Gets the api_format of this BaseChatRequest.
`frequency_penalty`	Gets the frequency_penalty of this GenericChatRequest.
`is_echo`	Gets the is_echo of this GenericChatRequest.
`is_parallel_tool_calls`	Gets the is_parallel_tool_calls of this GenericChatRequest.
`is_stream`	Gets the is_stream of this GenericChatRequest.
`log_probs`	Gets the log_probs of this GenericChatRequest.
`logit_bias`	Gets the logit_bias of this GenericChatRequest.
`max_completion_tokens`	Gets the max_completion_tokens of this GenericChatRequest.
`max_tokens`	Gets the max_tokens of this GenericChatRequest.
`messages`	Gets the messages of this GenericChatRequest.
`metadata`	Gets the metadata of this GenericChatRequest.
`num_generations`	Gets the num_generations of this GenericChatRequest.
`prediction`	Gets the prediction of this GenericChatRequest.
`presence_penalty`	Gets the presence_penalty of this GenericChatRequest.
`reasoning_effort`	Gets the reasoning_effort of this GenericChatRequest.
`response_format`	Gets the response_format of this GenericChatRequest.
`seed`	Gets the seed of this GenericChatRequest.
`stop`	Gets the stop of this GenericChatRequest.
`stream_options`	Gets the stream_options of this GenericChatRequest.
`temperature`	Gets the temperature of this GenericChatRequest.
`tool_choice`	Gets the tool_choice of this GenericChatRequest.
`tools`	Gets the tools of this GenericChatRequest.
`top_k`	Gets the top_k of this GenericChatRequest.
`top_p`	Gets the top_p of this GenericChatRequest.
`verbosity`	Gets the verbosity of this GenericChatRequest.

Methods

`__init__`(**kwargs)	Initializes a new GenericChatRequest object with values from keyword arguments.
`get_subtype`(object_dictionary)	Given the hash representation of a subtype of this class, use the info in the hash to return the class of the subtype.

API_FORMAT_COHERE = 'COHERE'¶

API_FORMAT_GENERIC = 'GENERIC'¶

REASONING_EFFORT_HIGH = 'HIGH'¶: A constant which can be used with the reasoning_effort property of a GenericChatRequest. This constant has a value of “HIGH”

REASONING_EFFORT_LOW = 'LOW'¶: A constant which can be used with the reasoning_effort property of a GenericChatRequest. This constant has a value of “LOW”

REASONING_EFFORT_MEDIUM = 'MEDIUM'¶: A constant which can be used with the reasoning_effort property of a GenericChatRequest. This constant has a value of “MEDIUM”

REASONING_EFFORT_MINIMAL = 'MINIMAL'¶: A constant which can be used with the reasoning_effort property of a GenericChatRequest. This constant has a value of “MINIMAL”

VERBOSITY_HIGH = 'HIGH'¶: A constant which can be used with the verbosity property of a GenericChatRequest. This constant has a value of “HIGH”

VERBOSITY_LOW = 'LOW'¶: A constant which can be used with the verbosity property of a GenericChatRequest. This constant has a value of “LOW”

VERBOSITY_MEDIUM = 'MEDIUM'¶: A constant which can be used with the verbosity property of a GenericChatRequest. This constant has a value of “MEDIUM”

__init__(**kwargs)¶

Initializes a new GenericChatRequest object with values from keyword arguments. The default value of the api_format attribute of this class is GENERIC and it should not be changed. The following keyword arguments are supported (corresponding to the getters/setters of this class):

Parameters:

api_format (str) – The value to assign to the api_format property of this GenericChatRequest. Allowed values for this property are: “COHERE”, “GENERIC”
messages (list[oci.generative_ai_inference.models.Message]) – The value to assign to the messages property of this GenericChatRequest.
reasoning_effort (str) – The value to assign to the reasoning_effort property of this GenericChatRequest. Allowed values for this property are: “MINIMAL”, “LOW”, “MEDIUM”, “HIGH”
verbosity (str) – The value to assign to the verbosity property of this GenericChatRequest. Allowed values for this property are: “LOW”, “MEDIUM”, “HIGH”
metadata (object) – The value to assign to the metadata property of this GenericChatRequest.
is_stream (bool) – The value to assign to the is_stream property of this GenericChatRequest.
stream_options (oci.generative_ai_inference.models.StreamOptions) – The value to assign to the stream_options property of this GenericChatRequest.
num_generations (int) – The value to assign to the num_generations property of this GenericChatRequest.
seed (int) – The value to assign to the seed property of this GenericChatRequest.
is_echo (bool) – The value to assign to the is_echo property of this GenericChatRequest.
top_k (int) – The value to assign to the top_k property of this GenericChatRequest.
top_p (float) – The value to assign to the top_p property of this GenericChatRequest.
temperature (float) – The value to assign to the temperature property of this GenericChatRequest.
frequency_penalty (float) – The value to assign to the frequency_penalty property of this GenericChatRequest.
presence_penalty (float) – The value to assign to the presence_penalty property of this GenericChatRequest.
stop (list[str]) – The value to assign to the stop property of this GenericChatRequest.
log_probs (int) – The value to assign to the log_probs property of this GenericChatRequest.
max_tokens (int) – The value to assign to the max_tokens property of this GenericChatRequest.
max_completion_tokens (int) – The value to assign to the max_completion_tokens property of this GenericChatRequest.
logit_bias (object) – The value to assign to the logit_bias property of this GenericChatRequest.
prediction (oci.generative_ai_inference.models.Prediction) – The value to assign to the prediction property of this GenericChatRequest.
response_format (oci.generative_ai_inference.models.ResponseFormat) – The value to assign to the response_format property of this GenericChatRequest.
tool_choice (oci.generative_ai_inference.models.ToolChoice) – The value to assign to the tool_choice property of this GenericChatRequest.
is_parallel_tool_calls (bool) – The value to assign to the is_parallel_tool_calls property of this GenericChatRequest.
tools (list[oci.generative_ai_inference.models.ToolDefinition]) – The value to assign to the tools property of this GenericChatRequest.

api_format¶

[Required] Gets the api_format of this BaseChatRequest. The API format for the model’s family group. COHERE is for the Cohere family models such as the cohere.command-r-16k and cohere.command-r-plus models. GENERIC is for other model families such as the meta.llama-3-70b-instruct model.

Allowed values for this property are: “COHERE”, “GENERIC”

Returns:	The api_format of this BaseChatRequest.
Return type:	str

frequency_penalty¶

Gets the frequency_penalty of this GenericChatRequest. To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far. Values > 0 encourage the model to use new tokens and values < 0 encourage the model to repeat tokens. Set to 0 to disable.

Returns:	The frequency_penalty of this GenericChatRequest.
Return type:	float

static get_subtype(object_dictionary)¶: Given the hash representation of a subtype of this class, use the info in the hash to return the class of the subtype.

is_echo¶

Gets the is_echo of this GenericChatRequest. Whether to include the user prompt in the response. Applies only to non-stream results.

Returns:	The is_echo of this GenericChatRequest.
Return type:	bool

is_parallel_tool_calls¶

Gets the is_parallel_tool_calls of this GenericChatRequest. Whether to enable parallel function calling during tool use.

Returns:	The is_parallel_tool_calls of this GenericChatRequest.
Return type:	bool

is_stream¶

Gets the is_stream of this GenericChatRequest. Whether to stream back partial progress. If set to true, as tokens become available, they are sent as data-only server-sent events.

Returns:	The is_stream of this GenericChatRequest.
Return type:	bool

log_probs¶

Gets the log_probs of this GenericChatRequest. Includes the logarithmic probabilities for the most likely output tokens and the chosen tokens.

For example, if the log probability is 5, the API returns a list of the 5 most likely tokens. The API returns the log probability of the sampled token, so there might be up to logprobs+1 elements in the response.

Returns:	The log_probs of this GenericChatRequest.
Return type:	int

logit_bias¶

Gets the logit_bias of this GenericChatRequest. Modifies the likelihood of specified tokens that appear in the completion.

Example: ‘{“6395”: 2, “8134”: 1, “21943”: 0.5, “5923”: -100}’

Returns:	The logit_bias of this GenericChatRequest.
Return type:	object

max_completion_tokens¶

Gets the max_completion_tokens of this GenericChatRequest. An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.

Returns:	The max_completion_tokens of this GenericChatRequest.
Return type:	int

max_tokens¶

Gets the max_tokens of this GenericChatRequest. The maximum number of tokens that can be generated per output sequence. The token count of your prompt plus maxTokens must not exceed the model’s context length. For on-demand inferencing, the response length is capped at 4,000 tokens for each run.

Returns:	The max_tokens of this GenericChatRequest.
Return type:	int

messages¶

Gets the messages of this GenericChatRequest. The series of messages in a chat request. Includes the previous messages in a conversation. Each message includes a role (USER or the CHATBOT) and content.

Returns:	The messages of this GenericChatRequest.
Return type:	list[oci.generative_ai_inference.models.Message]

metadata¶

Gets the metadata of this GenericChatRequest. Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.

Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

Returns:	The metadata of this GenericChatRequest.
Return type:	object

num_generations¶

Gets the num_generations of this GenericChatRequest. The number of of generated texts that will be returned.

Returns:	The num_generations of this GenericChatRequest.
Return type:	int

prediction¶

Gets the prediction of this GenericChatRequest.

Returns:	The prediction of this GenericChatRequest.
Return type:	oci.generative_ai_inference.models.Prediction

presence_penalty¶

Gets the presence_penalty of this GenericChatRequest. To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far. Values > 0 encourage the model to use new tokens and values < 0 encourage the model to repeat tokens.

Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they’ve appeared. Set to 0 to disable.

Returns:	The presence_penalty of this GenericChatRequest.
Return type:	float

reasoning_effort¶

Gets the reasoning_effort of this GenericChatRequest. Constrains effort on reasoning for reasoning models. Currently supported values are minimal, low, medium, and high. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response.

Allowed values for this property are: “MINIMAL”, “LOW”, “MEDIUM”, “HIGH”

Returns:	The reasoning_effort of this GenericChatRequest.
Return type:	str

response_format¶

Gets the response_format of this GenericChatRequest.

Returns:	The response_format of this GenericChatRequest.
Return type:	oci.generative_ai_inference.models.ResponseFormat

seed¶

Gets the seed of this GenericChatRequest. If specified, the backend will make a best effort to sample tokens deterministically, so that repeated requests with the same seed and parameters yield the same result. However, determinism cannot be fully guaranteed.

Returns:	The seed of this GenericChatRequest.
Return type:	int

stop¶

Gets the stop of this GenericChatRequest. List of strings that stop the generation if they are generated for the response text. The returned output will not contain the stop strings.

Returns:	The stop of this GenericChatRequest.
Return type:	list[str]

stream_options¶

Gets the stream_options of this GenericChatRequest.

Returns:	The stream_options of this GenericChatRequest.
Return type:	oci.generative_ai_inference.models.StreamOptions

temperature¶

Gets the temperature of this GenericChatRequest. A number that sets the randomness of the generated output. A lower temperature means a less random generations.

Use lower numbers for tasks with a correct answer such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.

Returns:	The temperature of this GenericChatRequest.
Return type:	float

tool_choice¶

Gets the tool_choice of this GenericChatRequest.

Returns:	The tool_choice of this GenericChatRequest.
Return type:	oci.generative_ai_inference.models.ToolChoice

tools¶

Gets the tools of this GenericChatRequest. A list of tools the model may call. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

Returns:	The tools of this GenericChatRequest.
Return type:	list[oci.generative_ai_inference.models.ToolDefinition]

top_k¶

Gets the top_k of this GenericChatRequest. An integer that sets up the model to use only the top k most likely tokens in the generated output. A higher k introduces more randomness into the output making the output text sound more natural. Default value is -1 which means to consider all tokens. Setting to 0 disables this method and considers all tokens.

If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20, but the probabilities of the top 10 add up to .75, then only the top 10 tokens are chosen.

Returns:	The top_k of this GenericChatRequest.
Return type:	int

top_p¶

Gets the top_p of this GenericChatRequest. If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.

To eliminate tokens with low likelihood, assign p a minimum percentage for the next token’s likelihood. For example, when p is set to 0.75, the model eliminates the bottom 25 percent for the next token. Set to 1 to consider all tokens and set to 0 to disable. If both k and p are enabled, p acts after k.

Returns:	The top_p of this GenericChatRequest.
Return type:	float

verbosity¶

Gets the verbosity of this GenericChatRequest. Constrains the verbosity of the model’s response. Lower values will result in more concise responses, while higher values will result in more verbose responses.

Allowed values for this property are: “LOW”, “MEDIUM”, “HIGH”

Returns:	The verbosity of this GenericChatRequest.
Return type:	str