Class CohereLlmInferenceRequest
Details for the text generation request for Cohere models.
Inherited Members
Namespace: Oci.GenerativeaiinferenceService.Models
Assembly: OCI.DotNetSDK.Generativeaiinference.dll
Syntax
public class CohereLlmInferenceRequest : LlmInferenceRequest
Properties
FrequencyPenalty
Declaration
[JsonProperty(PropertyName = "frequencyPenalty")]
public double FrequencyPenalty { get; set; }
Property Value
Type | Description |
---|---|
double | To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens. Set to 0 to disable. |
IsEcho
Declaration
[JsonProperty(PropertyName = "isEcho")]
public bool? IsEcho { get; set; }
Property Value
Type | Description |
---|---|
bool? | Whether or not to return the user prompt in the response. This option only applies to non-stream results. |
IsStream
Declaration
[JsonProperty(PropertyName = "isStream")]
public bool? IsStream { get; set; }
Property Value
Type | Description |
---|---|
bool? | Whether to stream back partial progress. If set, tokens are sent as data-only server-sent events as they become available. |
MaxTokens
Declaration
[JsonProperty(PropertyName = "maxTokens")]
public int? MaxTokens { get; set; }
Property Value
Type | Description |
---|---|
int? | The maximum number of tokens to predict for each response. Includes input plus output tokens. |
NumGenerations
Declaration
[JsonProperty(PropertyName = "numGenerations")]
public int? NumGenerations { get; set; }
Property Value
Type | Description |
---|---|
int? | The number of generated texts that will be returned. |
PresencePenalty
Declaration
[JsonProperty(PropertyName = "presencePenalty")]
public double PresencePenalty { get; set; }
Property Value
Type | Description |
---|---|
double | To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they've appeared in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens.
|
Prompt
Declaration
[Required(ErrorMessage = "Prompt is required.")]
[JsonProperty(PropertyName = "prompt")]
public string Prompt { get; set; }
Property Value
Type | Description |
---|---|
string | Represents the prompt to be completed. The trailing white spaces are trimmed before completion. |
Remarks
Required
ReturnLikelihoods
Declaration
[JsonProperty(PropertyName = "returnLikelihoods")]
[JsonConverter(typeof(StringEnumConverter))]
public CohereLlmInferenceRequest.ReturnLikelihoodsEnum? ReturnLikelihoods { get; set; }
Property Value
Type | Description |
---|---|
CohereLlmInferenceRequest.ReturnLikelihoodsEnum? | Specifies how and if the token likelihoods are returned with the response. |
StopSequences
Declaration
[JsonProperty(PropertyName = "stopSequences")]
public List<string> StopSequences { get; set; }
Property Value
Type | Description |
---|---|
List<string> | The generated text is cut at the end of the earliest occurrence of this stop sequence. The generated text will include this stop sequence. |
Temperature
Declaration
[JsonProperty(PropertyName = "temperature")]
public double Temperature { get; set; }
Property Value
Type | Description |
---|---|
double | A number that sets the randomness of the generated output. A lower temperature means a less random generations.
|
TopK
Declaration
[JsonProperty(PropertyName = "topK")]
public int? TopK { get; set; }
Property Value
Type | Description |
---|---|
int? | An integer that sets up the model to use only the top k most likely tokens in the generated output. A higher k introduces more randomness into the output making the output text sound more natural. Default value is 0 which disables this method and considers all tokens. To set a number for the likely tokens, choose an integer between 1 and 500.
|
TopP
Declaration
[JsonProperty(PropertyName = "topP")]
public double TopP { get; set; }
Property Value
Type | Description |
---|---|
double |
Truncate
Declaration
[JsonProperty(PropertyName = "truncate")]
[JsonConverter(typeof(StringEnumConverter))]
public CohereLlmInferenceRequest.TruncateEnum? Truncate { get; set; }
Property Value
Type | Description |
---|---|
CohereLlmInferenceRequest.TruncateEnum? | For an input that's longer than the maximum token length, specifies which part of the input text will be truncated. |