AI Glossary Term
What does “Top-k Sampling” mean?
Definition
A decoding strategy that samples only from the k most likely next tokens.
How Top-k Sampling works
Example
With k set to 5, the model considers only its five most likely next words and picks among them by probability. Strange low-probability words cannot be chosen.
Why it matters
It limits randomness without making the output fully predictable. A fixed k can be too tight or too loose depending on context.
Common questions
What does k = 1 do?
It always picks the single most likely token, which is called greedy decoding.
Top-k vs top-p?
Top-k keeps a fixed number of tokens. Top-p keeps however many it takes to reach a probability total.
Used with temperature?
Yes. Both settings are often combined.