meta/meta-llama-3.1-405b-instruct

The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering).

A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751).

Prompt

The maximum number of tokens the model should generate as output.

The minimum number of tokens the model should generate as output.

The value used to modulate the next token probabilities.

A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of 'end' or '<stop>'.

A template to format the prompt with. If not provided, the default prompt template will be used.

Presence penalty

Frequency penalty