01-ai/yi-6b

The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering).

A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751).

The value used to modulate the next token probabilities.

The maximum number of tokens the model should generate as output.

The template used to format the prompt. The input prompt is inserted into the template using the `{prompt}` placeholder.

Presence penalty

Frequency penalty