What is Top-k Sampling?

A decoding strategy that samples from the smallest token set whose probabilities sum to p.

A generation strategy that constrains output tokens to valid structures or policy-compliant choices.

Learning or adapting behavior from only a small number of examples.

A language-model metric measuring how surprised the model is by true next tokens.

An inference acceleration method where a small draft model proposes tokens that a larger model verifies in parallel.

Stored key and value tensors from previous tokens that let transformers generate new tokens without recomputing past attention.