Class CohereLlmInferenceRequest

Details for the text generation request for Cohere models.

Inheritance

object

LlmInferenceRequest

CohereLlmInferenceRequest

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Oci.GenerativeaiinferenceService.Models

Assembly: OCI.DotNetSDK.Generativeaiinference.dll

Syntax

public class CohereLlmInferenceRequest : LlmInferenceRequest

Properties

FrequencyPenalty

Declaration

[JsonProperty(PropertyName = "frequencyPenalty")]
public double FrequencyPenalty { get; set; }

Property Value

Type	Description
double	To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens. Set to 0 to disable.

IsEcho

Declaration

[JsonProperty(PropertyName = "isEcho")]
public bool? IsEcho { get; set; }

Property Value

Type	Description
bool?	Whether or not to return the user prompt in the response. This option only applies to non-stream results.

IsStream

Declaration

[JsonProperty(PropertyName = "isStream")]
public bool? IsStream { get; set; }

Property Value

Type	Description
bool?	Whether to stream back partial progress. If set, tokens are sent as data-only server-sent events as they become available.

MaxTokens

Declaration

[JsonProperty(PropertyName = "maxTokens")]
public int? MaxTokens { get; set; }

Property Value

Type	Description
int?	The maximum number of tokens to predict for each response. Includes input plus output tokens.

NumGenerations

Declaration

[JsonProperty(PropertyName = "numGenerations")]
public int? NumGenerations { get; set; }

Property Value

Type	Description
int?	The number of generated texts that will be returned.

PresencePenalty

Declaration

[JsonProperty(PropertyName = "presencePenalty")]
public double PresencePenalty { get; set; }

Property Value

Type	Description
double	To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they've appeared in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens. Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they've appeared. Set to 0 to disable.

Type

Description

double

To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they've appeared in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens.
Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they've appeared. Set to 0 to disable.

Prompt

Declaration

[Required(ErrorMessage = "Prompt is required.")]
[JsonProperty(PropertyName = "prompt")]
public string Prompt { get; set; }

Property Value

Type	Description
string	Represents the prompt to be completed. The trailing white spaces are trimmed before completion.

Remarks

Required

ReturnLikelihoods

Declaration

[JsonProperty(PropertyName = "returnLikelihoods")]
[JsonConverter(typeof(StringEnumConverter))]
public CohereLlmInferenceRequest.ReturnLikelihoodsEnum? ReturnLikelihoods { get; set; }

Property Value

Type	Description
CohereLlmInferenceRequest.ReturnLikelihoodsEnum?	Specifies how and if the token likelihoods are returned with the response.

StopSequences

Declaration

[JsonProperty(PropertyName = "stopSequences")]
public List<string> StopSequences { get; set; }

Property Value

Type	Description
List<string>	The generated text is cut at the end of the earliest occurrence of this stop sequence. The generated text will include this stop sequence.

Temperature

Declaration

[JsonProperty(PropertyName = "temperature")]
public double Temperature { get; set; }

Property Value

Type	Description
double	A number that sets the randomness of the generated output. A lower temperature means a less random generations. Use lower numbers for tasks with a correct answer such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.

Type

Description

double

A number that sets the randomness of the generated output. A lower temperature means a less random generations.
Use lower numbers for tasks with a correct answer such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.

TopK

Declaration

[JsonProperty(PropertyName = "topK")]
public int? TopK { get; set; }

Property Value

Type	Description
int?	An integer that sets up the model to use only the top k most likely tokens in the generated output. A higher k introduces more randomness into the output making the output text sound more natural. Default value is 0 which disables this method and considers all tokens. To set a number for the likely tokens, choose an integer between 1 and 500. If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20, but the probabilities of the top 10 add up to .75, then only the top 10 tokens are chosen.

Type

Description

int?

An integer that sets up the model to use only the top k most likely tokens in the generated output. A higher k introduces more randomness into the output making the output text sound more natural. Default value is 0 which disables this method and considers all tokens. To set a number for the likely tokens, choose an integer between 1 and 500.
If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20, but the probabilities of the top 10 add up to .75, then only the top 10 tokens are chosen.

TopP

Declaration

[JsonProperty(PropertyName = "topP")]
public double TopP { get; set; }

Property Value

Type	Description
double

Truncate

Declaration

[JsonProperty(PropertyName = "truncate")]
[JsonConverter(typeof(StringEnumConverter))]
public CohereLlmInferenceRequest.TruncateEnum? Truncate { get; set; }

Property Value

Type	Description
CohereLlmInferenceRequest.TruncateEnum?	For an input that's longer than the maximum token length, specifies which part of the input text will be truncated.