What is Presence Penalty?

A decoding setting that reduces the probability of tokens proportionally to how often they've appeared so far.

A decoding setting that lowers the probability of tokens the model has already produced to reduce loops.

A decoding strategy that samples only from the k most likely next tokens.

A generation strategy that constrains output tokens to valid structures or policy-compliant choices.

An inference acceleration method where a small draft model proposes tokens that a larger model verifies in parallel.

Stored key and value tensors from previous tokens that let transformers generate new tokens without recomputing past attention.