āļāđāļāđāļāļāļģāđāļāļ°āļāļģāļāđāļāđāļ
When Chain-of-Thought Hurts
āļ āļēāļĐāļēāđāļāđāļ
āļāļđāđāļĄāļ·āļ AI āļ āļēāļĐāļē
Chain-of-thought prompting elicits intermediate generated steps before a final answer in the cited research setups.
āļāļąāļāļāļģāđāļāļ°āļāļģāļāļĩāđāļāļĢāļĢāļĒāļēāļĒāđāļāļĒ AI · ~3 min
It improved some evaluated reasoning tasks, but a plausible explanation is neither a proof of correctness nor a guaranteed account of the modelâs actual decision process.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models studies prompts containing worked examples with intermediate steps. These examples show how to connect a question and final answer through an expressed solution, without changing the modelâs weights during the prompt itself. The reported gains depend on the tasks, models, and scales tested, and do not establish that every small model or every problem benefits. Large Language Models Are Zero-Shot Reasoners studies eliciting steps with a short instruction without worked demonstrations. This is a prompting distinction rather than a new supervised training procedure. Self-Consistency Improves Chain of Thought Reasoning samples diverse reasoning paths and aggregates their final answers instead of selecting one greedy path. Agreement can help in tested settings, but several samples can also share the same error. Intermediate generated text can become context for later tokens in an autoregressive setup, allowing a solution to be expressed over several stages. That mechanism does not guarantee correct decomposition or explain every observed gain. The original DeepSeek-R1 report studies reinforcement learning and staged training for reasoning behavior, including R1-Zeroâs RL experiment and R1âs additional stages. Training a reasoning model and prompting a fixed model with examples should therefore be distinguished. Finally, Language Models Donât Always Say What They Think demonstrates explanations that omit biasing influences and rationalize affected answers in particular model tests. A fluent solution should be checked against the task, and faithfulness to the modelâs underlying decision process is a separate evaluation question.
Comparing worked-example prompting with answer-only exemplars on a math task.
Testing the zero-shot phrase studied by Kojima and colleagues.
Sampling several solution paths and aggregating their final answers.
Checking whether a generated explanation omits a known biasing influence on the answer.
Reasoning evaluations can compare prompts, sampling budgets, model training, and verification methods on tasks with known answers or independently checkable solutions. A useful report should record quality, latency, token cost, and repeated failure patterns rather than treating longer explanations as intrinsically better. Studies of faithfulness can test known input influences and whether explanations acknowledge them. Training and inference procedures may yield different improvements, so their contributions need separate comparisons. An expressed reasoning path can help a reader inspect a proposed answer when it is checked, while its fluency alone does not establish reliability.
āļāļāļŦāļĄāļēāļĒāļāđāļēāļ§āļāļĢāļĩ
āđāļĢāļ·āđāļāļāļĢāļēāļ§ AI āļāļĩāđāļāļĢāļ§āļāļŠāļāļāđāļĨāđāļ§ 3 āđāļĢāļ·āđāļāļāļāļļāļāļ§āļąāļāļāļģāļāļēāļ āđāļāđāļē, āđāļāļĩāļĒāļāđāļāđāļāļ āļēāļĐāļēāļāļąāļāļāļĪāļĐāļāļĢāļĢāļĄāļāļē āļāļĢāļĩāļāļĨāļāļāđāļ āđāļĄāđāļĄāļĩāđāļāļĐāļāļē
āļāļĩāđāļĄāļĨāļŦāļāļķāđāļāļāļāļąāļāļāļļāļāļ§āļąāļāļāļģāļāļēāļ āļĒāļāđāļĨāļīāļāļāļēāļĢāļŠāļĄāļąāļāļĢāđāļāļāļĨāļīāļāđāļāļĩāļĒāļ§ āđāļĢāļēāđāļĄāđāđāļāļĒāļāļēāļĒāļŦāļĢāļ·āļāđāļāļĢāđāļāļĩāđāļāļĒāļđāđāļāļĩāđāļĄāļĨāļāļāļāļāļļāļ
āļāļāļŠāļāļāļāļąāļ§āđāļāļ
āļāđāļāđāļŠāļāļāđāļāļ°āļāļąāļāļāļĩāļŠāļģāļŦāļĢāļąāļāļāļļāļāļāļģāļāļāļ āđāļĨāļ°āđāļāļĢāļąāļāļĢāļāļāļāļĩāđāļŠāļēāļĄāļēāļĢāļāđāļāļĢāđāđāļāđāļāļĢāđāļāļĄ ID āļāļĩāđāļāļĢāļ§āļāļŠāļāļāđāļāđāđāļĄāļ·āđāļāļāļļāļāļāđāļēāļāļŦāļĨāļąāļāļŠāļđāļāļĢ
āļŠāļāļąāļāļŠāļāļļāļāļāļēāļĢāļĻāļķāļāļĐāļē AI āļāļĢāļĩ AI Understanding āđāļāđāļāļāļāļāđāļāļĢāđāļĄāđāđāļŠāļ§āļāļŦāļēāļāļĨāļāļģāđāļĢ 501(c)(3) â āđāļĄāđāļĄāļĩāđāļāļĐāļāļē āđāļĄāđāļĄāļĩāļāļģāđāļāļāđāļāļīāļ, āļāļĨāļāļāđāļ āļāļĢāļīāļāļēāļ
Chain-of-thought prompting elicits intermediate generated steps before a final answer in the cited research setups. It improved some evaluated reasoning tasks, but a plausible explanation is neither a proof of correctness nor a guaranteed account of the modelâs actual decision process.
The method elicits an expressed solution path, whose correctness still requires evaluation.
āđāļāļāļīāļĄāļ° āđāļĨāļ°āļāļāļ° (2022) āļāļāļ§āđāļēāļāļēāļĢāļāđāļāļāđāļēāļĒ 'Let's think step by step' āļāđāļ§āļĒāđāļŦāđāđāļŦāđāđāļŦāļāļļāļāļĨāđāļĄāđāļ§āđāļēāļāļ°āđāļĄāđāļĄāļĩāļāļąāļ§āļāļĒāđāļēāļāļāļĩāđāđāļāđāļāļĨāļāđāļāļēāļĄ
The empirical size-dependent results should not become a universal rule for all models.
The method aggregates answer evidence across samples; agreement can still accompany shared mistakes.
R1-Zero and the fuller R1 procedure have different stages; both should be distinguished from prompting alone.
āđāļĢāļĩāļĒāļāļĢāļđāđāļāđāļāđāļ
āļāļģāđāļāļ°āļāļģāđāļāļīāđāļĄāđāļāļīāļĄāļāļĩāđāđāļĨāļ·āļāļāļŠāļģāļŦāļĢāļąāļāļŦāļąāļ§āļāđāļāļāļĩāđ
āļāđāļāđāļāļāļģāđāļāļ°āļāļģāļāđāļāđāļ
When Chain-of-Thought Hurts
āļ āļēāļĐāļēāđāļāđāļ