概述
It improved results on several studied benchmarks, but accuracy gains depend on the task and samples can share correlated errors; the method also adds inference cost.
深入探讨
A model can produce different reasoning paths for the same prompt. Greedy decoding follows one most-likely path, while self-consistency samples several paths, extracts an answer from each, and selects the most frequent answer. The original study reported improvements on several arithmetic and commonsense benchmarks under its tested models, prompts, and sampling budgets. On suitable tasks, correct paths may converge more consistently than incorrect paths, but this is not guaranteed: paths come from the same model and errors may be correlated. Answer extraction and normalization also matter because equivalent answers can have different surface forms. The method increases inference work because it generates multiple paths. Treat the vote as an aggregation heuristic and use external checks when correctness matters.
现实世界的实施
通过对许多解决方案路径进行采样并对最终数字进行投票，提高小学数学应用题 (GSM8K) 的准确性。
提高多步骤常识性问题回答的可靠性，其中单个链可能会因一个推论而出错。
通过检查样本中哪个输出最一致，提高对代码生成答案的信心。
加强符号或逻辑推理任务，其中不同的推导应汇聚成一个正确的结论。
自一致性解码的未来
自一致性是推理时间扩展的一个基本示例，它的后代现在为推理模型提供动力，这些模型花费额外的计算来更深入地思考。未来的方向包括通过学习验证者或置信度分数对投票进行加权，而不是平均计数，根据问题难度自适应地选择要抽取的样本数量，以及将投票与思想树等搜索框架相结合。当正确性比延迟更重要时，期望它仍然是一个廉价的、免培训的基线，任何系统都可以分层。
付诸实践
为什么这很重要
- 速度与规模. 语言工作流程可以在不牺牲一致性的情况下更快地移动。
- 交通与覆盖范围. 它扩展了跨语言和沟通方式的访问。
- 更清晰的判决. 团队可以花更多时间进行判断，而自动化则可以处理重复。
留意
- 幻觉的事实可以悄悄地进入报告、支持流程或研究成果。
- 及时的敏感性可能会在类似的请求中产生不一致的结果。
- 如果访问控制薄弱，敏感文本数据可能会暴露。
后续步骤
- 在推出之前定义输出格式、语气和质量标准。
- 当准确性很重要时，请使用可信来源进行地面响应。
- 为高风险输出保留人工审查检查点。
- 跟踪故障模式并定期重新训练提示或工作流程。
不断探索
常见问题
What is Self-Consistency Decoding?
Self-consistency samples multiple reasoning paths and aggregates their extracted answers, often by majority vote. It improved results on several studied benchmarks, but accuracy gains depend on the task and samples can share correlated errors; the method also adds inference cost.
自一致性解码如何从语言模型中选择最终答案？
自我一致性对多个不同的推理链进行采样，并选择大多数推理链都同意的最终答案。
Under what condition can self-consistency improve answer accuracy?
Self-consistency can help on tested tasks when correct samples converge and mistakes vary; correlated errors can still make a wrong majority.
自一致性取代了什么解码方法？
与其贪婪地致力于一条最有可能的路径，不如在多条路径上进行自我一致性采样和聚合。
每个采样链的哪一部分实际用于最终投票？
中间推理被丢弃；仅通过多数票汇总最终答案。
What is the main cost of increasing the number of self-consistency samples?
The paper reports gains in its evaluated tasks, but more samples do not guarantee higher accuracy because errors can be correlated.