Language AI GUIDE
Beam Search
Beam search is a decoding algorithm that retains a fixed number of high-scoring partial sequences at each step.
Listen to this guideAI-narrated · ~2 min
On this page2 min read
Overview
It is used in sequence generation tasks, with results shaped by model scores and any normalization or penalties.
Deep Dive
At each decoding step, beam search expands the current partial candidates and retains the top-scoring continuations. A wider beam explores more options but increases search cost and does not necessarily improve perceived quality. GNMT uses length normalization to mitigate short-output bias and a coverage penalty to encourage source-word coverage in translation. Diverse beam search modifies group selection to reduce redundant candidates.
Real-World Implementation
A beam of width k keeps up to k partial sequences rather than only one greedy continuation.
GNMT used length normalization and a coverage penalty in its translation system.
Diverse beam search adds a penalty to reduce overlap among candidate sequences from beam groups.
For open-ended text, compare beam outputs with sampling because likelihood-ranked outputs can be repetitive or bland in the evaluated settings.
The Future of Beam Search
Select decoding criteria for the task and measure both search cost and output quality. For translation, report source coverage and reference-based metrics; for open-ended generation, include human evaluation where appropriate. The cited sources describe particular methods and experiments rather than a single universally best beam configuration.
Putting it into practice
Why it matters
- Speed and scale. Language workflows can move faster without sacrificing consistency.
- Access and reach. It expands access across languages and communication styles.
- Clearer decisions. Teams can spend more time on judgment while automation handles repetition.
Watch out for
- Hallucinated facts can quietly enter reports, support flows, or research outputs.
- Prompt sensitivity can create inconsistent results across similar requests.
- Sensitive text data may be exposed if access controls are weak.
Next steps
- Define output format, tone, and quality standards before rollout.
- Ground responses with trusted sources whenever accuracy matters.
- Keep a human review checkpoint for high-stakes outputs.
- Track failure patterns and retrain prompts or workflows regularly.
Keep Exploring
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Beam Search quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Frequently asked questions
What is Beam Search?
Beam search is a decoding algorithm that retains a fixed number of high-scoring partial sequences at each step. It is used in sequence generation tasks, with results shaped by model scores and any normalization or penalties.
What does the 'beam width' (k) control in beam search?
The beam width is the number of top partial sequences retained and expanded at every step; larger widths explore more candidates.
How does beam search differ from greedy decoding?
Greedy decoding takes only the top token each step; beam search hedges by tracking several promising sequences simultaneously.
Why is length normalization commonly applied in beam search?
Each additional token adds a negative log-probability, so without normalization shorter sequences score higher unfairly.
For which task is beam search especially well-suited?
Beam search excels when the goal is a single accurate, fluent rendering, which is why it became standard in translation.
What is a known downside of using a very large beam width for open-ended generation?
Counterintuitively, wider beams in neural text generation often favor generic, repetitive sequences rather than interesting ones.
Keep learning
Related guides
More guides picked for this topic