com.oracle.bmc.generativeaiinference.model

Class CohereLlmInferenceRequest.Builder

java.lang.Object

com.oracle.bmc.generativeaiinference.model.CohereLlmInferenceRequest.Builder

Enclosing class:

CohereLlmInferenceRequest

public static class CohereLlmInferenceRequest.Builder
extends Object

Constructor Summary

Constructors

Constructor and Description
Builder()

Method Summary

Modifier and Type	Method and Description
CohereLlmInferenceRequest	build()
CohereLlmInferenceRequest.Builder	copy(CohereLlmInferenceRequest model)
CohereLlmInferenceRequest.Builder	frequencyPenalty(Double frequencyPenalty) To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far.
CohereLlmInferenceRequest.Builder	isEcho(Boolean isEcho) Whether or not to return the user prompt in the response.
CohereLlmInferenceRequest.Builder	isStream(Boolean isStream) Whether to stream back partial progress.
CohereLlmInferenceRequest.Builder	maxTokens(Integer maxTokens) The maximum number of tokens to predict for each response.
CohereLlmInferenceRequest.Builder	numGenerations(Integer numGenerations) The number of generated texts that will be returned.
CohereLlmInferenceRequest.Builder	presencePenalty(Double presencePenalty) To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far.
CohereLlmInferenceRequest.Builder	prompt(String prompt) Represents the prompt to be completed.
CohereLlmInferenceRequest.Builder	returnLikelihoods(CohereLlmInferenceRequest.ReturnLikelihoods returnLikelihoods) Specifies how and if the token likelihoods are returned with the response.
CohereLlmInferenceRequest.Builder	stopSequences(List<String> stopSequences) The generated text is cut at the end of the earliest occurrence of this stop sequence.
CohereLlmInferenceRequest.Builder	temperature(Double temperature) A number that sets the randomness of the generated output.
CohereLlmInferenceRequest.Builder	topK(Integer topK) An integer that sets up the model to use only the top k most likely tokens in the generated output.
CohereLlmInferenceRequest.Builder	topP(Double topP) If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.
CohereLlmInferenceRequest.Builder	truncate(CohereLlmInferenceRequest.Truncate truncate) For an input that’s longer than the maximum token length, specifies which part of the input text will be truncated.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder()

Method Detail

prompt

public CohereLlmInferenceRequest.Builder prompt(String prompt)

Represents the prompt to be completed. The trailing white spaces are trimmed before completion.

Parameters:

prompt - the value to set

Returns:

this builder

isStream

public CohereLlmInferenceRequest.Builder isStream(Boolean isStream)

Whether to stream back partial progress. If set, tokens are sent as data-only server-sent events as they become available.

Parameters:

isStream - the value to set

Returns:

this builder

numGenerations

public CohereLlmInferenceRequest.Builder numGenerations(Integer numGenerations)

The number of generated texts that will be returned.

Parameters:

numGenerations - the value to set

Returns:

this builder

isEcho

public CohereLlmInferenceRequest.Builder isEcho(Boolean isEcho)

Whether or not to return the user prompt in the response. This option only applies to non-stream results.

Parameters:

isEcho - the value to set

Returns:

this builder

maxTokens

public CohereLlmInferenceRequest.Builder maxTokens(Integer maxTokens)

The maximum number of tokens to predict for each response. Includes input plus output tokens.

Parameters:

maxTokens - the value to set

Returns:

this builder

temperature

public CohereLlmInferenceRequest.Builder temperature(Double temperature)

A number that sets the randomness of the generated output. A lower temperature means a less random generations.

Use lower numbers for tasks with a correct answer such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.

Parameters:

temperature - the value to set

Returns:

this builder

topK

public CohereLlmInferenceRequest.Builder topK(Integer topK)

An integer that sets up the model to use only the top k most likely tokens in the generated output. A higher k introduces more randomness into the output making the output text sound more natural. Default value is 0 which disables this method and considers all tokens. To set a number for the likely tokens, choose an integer between 1 and 500.

If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20, but the probabilities of the top 10 add up to .75, then only the top 10 tokens are chosen.

Parameters:

topK - the value to set

Returns:

this builder

topP

public CohereLlmInferenceRequest.Builder topP(Double topP)

If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.

To eliminate tokens with low likelihood, assign p a minimum percentage for the next token's likelihood. For example, when p is set to 0.75, the model eliminates the bottom 25 percent for the next token. Set to 1.0 to consider all tokens and set to 0 to disable. If both k and p are enabled, p acts after k.

Parameters:

topP - the value to set

Returns:

this builder

frequencyPenalty

public CohereLlmInferenceRequest.Builder frequencyPenalty(Double frequencyPenalty)

To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens. Set to 0 to disable.

Parameters:

frequencyPenalty - the value to set

Returns:

this builder

presencePenalty

public CohereLlmInferenceRequest.Builder presencePenalty(Double presencePenalty)

To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens.

Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they've appeared. Set to 0 to disable.

Parameters:

presencePenalty - the value to set

Returns:

this builder

stopSequences

public CohereLlmInferenceRequest.Builder stopSequences(List<String> stopSequences)

The generated text is cut at the end of the earliest occurrence of this stop sequence. The generated text will include this stop sequence.

Parameters:

stopSequences - the value to set

Returns:

this builder

returnLikelihoods

public CohereLlmInferenceRequest.Builder returnLikelihoods(CohereLlmInferenceRequest.ReturnLikelihoods returnLikelihoods)

Specifies how and if the token likelihoods are returned with the response.

Parameters:

returnLikelihoods - the value to set

Returns:

this builder

truncate

public CohereLlmInferenceRequest.Builder truncate(CohereLlmInferenceRequest.Truncate truncate)

For an input that’s longer than the maximum token length, specifies which part of the input text will be truncated.

Parameters:

truncate - the value to set

Returns:

this builder

build

public CohereLlmInferenceRequest build()

copy

public CohereLlmInferenceRequest.Builder copy(CohereLlmInferenceRequest model)