Class GenericChatRequest
- java.lang.Object
-
- com.oracle.bmc.http.client.internal.ExplicitlySetBmcModel
-
- com.oracle.bmc.generativeaiinference.model.BaseChatRequest
-
- com.oracle.bmc.generativeaiinference.model.GenericChatRequest
-
@Generated(value="OracleSDKGenerator", comments="API Version: 20231130") public final class GenericChatRequest extends BaseChatRequest
Details for the chat request.
Note: Objects should always be created or deserialized using theGenericChatRequest.Builder
. This model distinguishes fields that are null because they are unset from fields that are explicitly set to null. This is done in the setter methods of theGenericChatRequest.Builder
, which maintain a set of all explicitly set fields calledGenericChatRequest.Builder.__explicitlySet__
. ThehashCode()
andequals(Object)
methods are implemented to take the explicitly set fields into account. The constructor, on the other hand, does not take the explicitly set fields into account (since the constructor cannot distinguish explicit null from unset null).
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
GenericChatRequest.Builder
static class
GenericChatRequest.ReasoningEffort
Constrains effort on reasoning for reasoning models.static class
GenericChatRequest.Verbosity
Constrains the verbosity of the model’s response.-
Nested classes/interfaces inherited from class com.oracle.bmc.generativeaiinference.model.BaseChatRequest
BaseChatRequest.ApiFormat
-
-
Constructor Summary
Constructors Constructor Description GenericChatRequest(List<Message> messages, GenericChatRequest.ReasoningEffort reasoningEffort, GenericChatRequest.Verbosity verbosity, Object metadata, Boolean isStream, StreamOptions streamOptions, Integer numGenerations, Integer seed, Boolean isEcho, Integer topK, Double topP, Double temperature, Double frequencyPenalty, Double presencePenalty, List<String> stop, Integer logProbs, Integer maxTokens, Integer maxCompletionTokens, Object logitBias, Prediction prediction, ResponseFormat responseFormat, ToolChoice toolChoice, Boolean isParallelToolCalls, List<ToolDefinition> tools)
Deprecated.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static GenericChatRequest.Builder
builder()
Create a new builder.boolean
equals(Object o)
Double
getFrequencyPenalty()
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far.Boolean
getIsEcho()
Whether to include the user prompt in the response.Boolean
getIsParallelToolCalls()
Whether to enable parallel function calling during tool use.Boolean
getIsStream()
Whether to stream back partial progress.Object
getLogitBias()
Modifies the likelihood of specified tokens that appear in the completion.Integer
getLogProbs()
Includes the logarithmic probabilities for the most likely output tokens and the chosen tokens.Integer
getMaxCompletionTokens()
An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.Integer
getMaxTokens()
The maximum number of tokens that can be generated per output sequence.List<Message>
getMessages()
The series of messages in a chat request.Object
getMetadata()
Set of 16 key-value pairs that can be attached to an object.Integer
getNumGenerations()
The number of of generated texts that will be returned.Prediction
getPrediction()
Double
getPresencePenalty()
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far.GenericChatRequest.ReasoningEffort
getReasoningEffort()
Constrains effort on reasoning for reasoning models.ResponseFormat
getResponseFormat()
Integer
getSeed()
If specified, the backend will make a best effort to sample tokens deterministically, so that repeated requests with the same seed and parameters yield the same result.List<String>
getStop()
List of strings that stop the generation if they are generated for the response text.StreamOptions
getStreamOptions()
Double
getTemperature()
A number that sets the randomness of the generated output.ToolChoice
getToolChoice()
List<ToolDefinition>
getTools()
A list of tools the model may call.Integer
getTopK()
An integer that sets up the model to use only the top k most likely tokens in the generated output.Double
getTopP()
If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.GenericChatRequest.Verbosity
getVerbosity()
Constrains the verbosity of the model’s response.int
hashCode()
GenericChatRequest.Builder
toBuilder()
String
toString()
String
toString(boolean includeByteArrayContents)
Return a string representation of the object.
-
-
-
Constructor Detail
-
GenericChatRequest
@Deprecated public GenericChatRequest(List<Message> messages, GenericChatRequest.ReasoningEffort reasoningEffort, GenericChatRequest.Verbosity verbosity, Object metadata, Boolean isStream, StreamOptions streamOptions, Integer numGenerations, Integer seed, Boolean isEcho, Integer topK, Double topP, Double temperature, Double frequencyPenalty, Double presencePenalty, List<String> stop, Integer logProbs, Integer maxTokens, Integer maxCompletionTokens, Object logitBias, Prediction prediction, ResponseFormat responseFormat, ToolChoice toolChoice, Boolean isParallelToolCalls, List<ToolDefinition> tools)
Deprecated.
-
-
Method Detail
-
builder
public static GenericChatRequest.Builder builder()
Create a new builder.
-
toBuilder
public GenericChatRequest.Builder toBuilder()
-
getMessages
public List<Message> getMessages()
The series of messages in a chat request.Includes the previous messages in a conversation. Each message includes a role (USER or the CHATBOT) and content.
- Returns:
- the value
-
getReasoningEffort
public GenericChatRequest.ReasoningEffort getReasoningEffort()
Constrains effort on reasoning for reasoning models.Currently supported values are minimal, low, medium, and high. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response.
- Returns:
- the value
-
getVerbosity
public GenericChatRequest.Verbosity getVerbosity()
Constrains the verbosity of the model’s response.Lower values will result in more concise responses, while higher values will result in more verbose responses.
- Returns:
- the value
-
getMetadata
public Object getMetadata()
Set of 16 key-value pairs that can be attached to an object.This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.
Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
- Returns:
- the value
-
getIsStream
public Boolean getIsStream()
Whether to stream back partial progress.If set to true, as tokens become available, they are sent as data-only server-sent events.
- Returns:
- the value
-
getStreamOptions
public StreamOptions getStreamOptions()
-
getNumGenerations
public Integer getNumGenerations()
The number of of generated texts that will be returned.- Returns:
- the value
-
getSeed
public Integer getSeed()
If specified, the backend will make a best effort to sample tokens deterministically, so that repeated requests with the same seed and parameters yield the same result.However, determinism cannot be fully guaranteed.
- Returns:
- the value
-
getIsEcho
public Boolean getIsEcho()
Whether to include the user prompt in the response.Applies only to non-stream results.
- Returns:
- the value
-
getTopK
public Integer getTopK()
An integer that sets up the model to use only the top k most likely tokens in the generated output.A higher k introduces more randomness into the output making the output text sound more natural. Default value is -1 which means to consider all tokens. Setting to 0 disables this method and considers all tokens.
If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20, but the probabilities of the top 10 add up to .75, then only the top 10 tokens are chosen.
- Returns:
- the value
-
getTopP
public Double getTopP()
If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.To eliminate tokens with low likelihood, assign p a minimum percentage for the next token's likelihood. For example, when p is set to 0.75, the model eliminates the bottom 25 percent for the next token. Set to 1 to consider all tokens and set to 0 to disable. If both k and p are enabled, p acts after k.
- Returns:
- the value
-
getTemperature
public Double getTemperature()
A number that sets the randomness of the generated output.A lower temperature means a less random generations.
Use lower numbers for tasks with a correct answer such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.
- Returns:
- the value
-
getFrequencyPenalty
public Double getFrequencyPenalty()
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far.Values > 0 encourage the model to use new tokens and values < 0 encourage the model to repeat tokens. Set to 0 to disable.
- Returns:
- the value
-
getPresencePenalty
public Double getPresencePenalty()
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far.Values > 0 encourage the model to use new tokens and values < 0 encourage the model to repeat tokens.
Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they've appeared. Set to 0 to disable.
- Returns:
- the value
-
getStop
public List<String> getStop()
List of strings that stop the generation if they are generated for the response text.The returned output will not contain the stop strings.
- Returns:
- the value
-
getLogProbs
public Integer getLogProbs()
Includes the logarithmic probabilities for the most likely output tokens and the chosen tokens.For example, if the log probability is 5, the API returns a list of the 5 most likely tokens. The API returns the log probability of the sampled token, so there might be up to logprobs+1 elements in the response.
- Returns:
- the value
-
getMaxTokens
public Integer getMaxTokens()
The maximum number of tokens that can be generated per output sequence.The token count of your prompt plus maxTokens must not exceed the model’s context length. For on-demand inferencing, the response length is capped at 4,000 tokens for each run.
- Returns:
- the value
-
getMaxCompletionTokens
public Integer getMaxCompletionTokens()
An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.- Returns:
- the value
-
getLogitBias
public Object getLogitBias()
Modifies the likelihood of specified tokens that appear in the completion.Example: '{"6395": 2, "8134": 1, "21943": 0.5, "5923": -100}'
- Returns:
- the value
-
getPrediction
public Prediction getPrediction()
-
getResponseFormat
public ResponseFormat getResponseFormat()
-
getToolChoice
public ToolChoice getToolChoice()
-
getIsParallelToolCalls
public Boolean getIsParallelToolCalls()
Whether to enable parallel function calling during tool use.- Returns:
- the value
-
getTools
public List<ToolDefinition> getTools()
A list of tools the model may call.Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.
- Returns:
- the value
-
toString
public String toString()
- Overrides:
toString
in classBaseChatRequest
-
toString
public String toString(boolean includeByteArrayContents)
Return a string representation of the object.- Overrides:
toString
in classBaseChatRequest
- Parameters:
includeByteArrayContents
- true to include the full contents of byte arrays- Returns:
- string representation
-
equals
public boolean equals(Object o)
- Overrides:
equals
in classBaseChatRequest
-
hashCode
public int hashCode()
- Overrides:
hashCode
in classBaseChatRequest
-
-