Beam Search
Beam search is a decoding strategy that keeps the several most promising partial sequences at each step instead of greedily committing to one.
Overview
It matters because it produces higher-quality, more coherent text for tasks like translation and summarization than picking the single best word every time.
Deep Dive
When a language model generates text, it predicts a probability for the next token, then repeats. Greedy decoding always takes the single highest-probability token, but that can paint you into a corner — an early locally-best choice may lead to an overall worse sentence. Beam search hedges by maintaining the top-k partial sequences (the 'beam width', often 4-10). At each step it expands every beam with possible next tokens, scores all candidates by their cumulative log-probability, and keeps only the top k. The result is the highest-scoring complete sequence. It became the standard for machine translation and remains common where a faithful, high-probability output matters more than creativity.
Technical Insight
Beam search scores sequences by summing log-probabilities of tokens, which biases it toward shorter sequences (each extra token adds a negative term). To counteract this, systems apply length normalization, dividing the score by sequence length (sometimes raised to a power). A larger beam width explores more candidates but costs more compute and, counterintuitively, can sometimes yield blander or degenerate text — a well-documented effect in neural machine translation.
Strategic Impact
Speed and scale
Language workflows can move faster without sacrificing consistency.
Access and reach
It expands access across languages and communication styles.
Clearer decisions
Teams can spend more time on judgment while automation handles repetition.
The Future of Beam Search
For open-ended, creative generation, beam search is increasingly replaced by sampling methods (top-k, nucleus) because beams tend to produce repetitive, generic text. But for constrained tasks — translation, speech recognition, code generation, structured output — beam search and its variants (diverse beam search, constrained beam search that forces required words) remain valuable. Expect continued hybrid approaches that combine beam-style exploration with sampling, plus task-aware decoding that adapts strategy to whether faithfulness or diversity is the priority.
Real-World Implementation
Neural machine translation systems choosing the most fluent rendering of a sentence across many candidate phrasings
Automatic speech recognition decoding the most likely transcript from acoustic model probabilities
Image captioning models producing a single coherent caption rather than a random plausible one
Constrained generation that forces specific keywords or terminology to appear in the output using constrained beam search
Risks & Guardrails
Hallucinated facts can quietly enter reports, support flows, or research outputs.
Prompt sensitivity can create inconsistent results across similar requests.
Sensitive text data may be exposed if access controls are weak.
Implementation Roadmap
Define output format, tone, and quality standards before rollout.
Ground responses with trusted sources whenever accuracy matters.
Keep a human review checkpoint for high-stakes outputs.
Track failure patterns and retrain prompts or workflows regularly.
Keep Exploring
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Beam Search quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Next guide
Guided Beam Search with Constraints
Frequently asked questions
What is Beam Search?
Beam search is a decoding strategy that keeps the several most promising partial sequences at each step instead of greedily committing to one. It matters because it produces higher-quality, more coherent text for tasks like translation and summarization than picking the single best word every time.
What does the 'beam width' (k) control in beam search?
The beam width is the number of top partial sequences retained and expanded at every step; larger widths explore more candidates.
How does beam search differ from greedy decoding?
Greedy decoding takes only the top token each step; beam search hedges by tracking several promising sequences simultaneously.
Why is length normalization commonly applied in beam search?
Each additional token adds a negative log-probability, so without normalization shorter sequences score higher unfairly.
For which task is beam search especially well-suited?
Beam search excels when the goal is a single accurate, fluent rendering, which is why it became standard in translation.
What is a known downside of using a very large beam width for open-ended generation?
Counterintuitively, wider beams in neural text generation often favor generic, repetitive sequences rather than interesting ones.