AI Glossary Term
What does “Top-p (Nucleus) Sampling” mean?
Definition
A decoding strategy that samples from the smallest token set whose probabilities sum to p.
How Top-p (Nucleus) Sampling works
Example
When the next word is obvious, a few tokens may already cover 90 percent of the probability, so the pool is tiny. When many words fit, the pool grows. This adapts to how confident the model is.
Why it matters
It trims unlikely words while staying flexible. Many applications use it to balance coherence and variety.
Common questions
Who introduced it?
Holtzman and colleagues described nucleus sampling in a 2019 paper.
What value should p be?
Values like 0.9 to 0.95 are common, but they depend on the task.
Should I change both top-p and temperature?
Many providers advise adjusting one at a time to keep behavior easy to understand.