概述
It can improve results when the model sometimes produces a good candidate and the scorer identifies it reliably. More samples cost more generation and can amplify errors in an imperfect scorer.
深入探讨
Best-of-N samples N candidate responses from a model and returns the one with the highest score under a reward model, verifier, or heuristic. Self-consistency is a special case that aggregates final answers by majority vote. If each independent sample has probability p of success and a perfect verifier can recognize success, the chance at least one candidate is correct is 1−(1−p)^N. Real outputs are not necessarily independent and real scorers are imperfect. Research on reward-model overoptimization finds that optimizing against a proxy can eventually reduce gold-standard performance, including in best-of-N selection.
现实世界的实施
Sampling several math solutions and selecting one that passes a verifier
Generating code candidates and ranking them with unit-test results
Using majority vote when multiple reasoning chains reach the same answer
Comparing quality gains against the additional generated tokens and scorer cost
Best-of-N 采样和重新排序的未来
Inference-time scaling research studies adaptive sample counts, verifiers, and reward-model calibration. The right N depends on the task, sample quality, scorer reliability, and compute budget. Evaluations should report both the selected-answer metric and the added generation/scoring cost.
付诸实践
为什么这很重要
- 速度与规模. 语言工作流程可以在不牺牲一致性的情况下更快地移动。
- 交通与覆盖范围. 它扩展了跨语言和沟通方式的访问。
- 更清晰的判决. 团队可以花更多时间进行判断，而自动化则可以处理重复。
留意
- 幻觉的事实可以悄悄地进入报告、支持流程或研究成果。
- 及时的敏感性可能会在类似的请求中产生不一致的结果。
- 如果访问控制薄弱，敏感文本数据可能会暴露。
后续步骤
- 在推出之前定义输出格式、语气和质量标准。
- 当准确性很重要时，请使用可信来源进行地面响应。
- 为高风险输出保留人工审查检查点。
- 跟踪故障模式并定期重新训练提示或工作流程。
不断探索
常见问题
什么是 Best-of-N 采样和重新排序？
Best-of-N is an inference-time selection method: sample multiple outputs and use a scorer or verifier to choose one. It can improve results when the model sometimes produces a good candidate and the scorer identifies it reliably. More samples cost more generation and can amplify errors in an imperfect scorer.
best-of-N采样的核心思想是什么？
Best-of-N 抽取多个候选响应，然后对它们重新排名，返回得分最高的响应。
When is best-of-N most likely to help?
More candidates help only if a good answer sometimes appears and the scorer can recognize it; neither condition is guaranteed.
是什么在很大程度上决定了“N 中最佳”是否能够真正改善结果？
选择的好坏取决于重新排序的好坏；弱者或有偏见的评分者可能会选择错误的答案。
What can happen when N becomes large and the reward model is an imperfect proxy?
Research measures reward-model overoptimization in best-of-N: selecting extreme proxy scores can eventually lower gold-reward performance.
哪种简单的重新排名策略也称为自我一致性？
自洽性对许多推理链进行采样，并选择大多数推理链都同意的最终答案。