À suivreGuide suivant
When Chain-of-Thought Hurts
IA linguistique
GUIDE IA du langage
Chain-of-thought prompting elicits intermediate generated steps before a final answer in the cited research setups.
Écoutez ce guideNarré par l'IA · ~3 min
It improved some evaluated reasoning tasks, but a plausible explanation is neither a proof of correctness nor a guaranteed account of the model’s actual decision process.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models studies prompts containing worked examples with intermediate steps. These examples show how to connect a question and final answer through an expressed solution, without changing the model’s weights during the prompt itself. The reported gains depend on the tasks, models, and scales tested, and do not establish that every small model or every problem benefits. Large Language Models Are Zero-Shot Reasoners studies eliciting steps with a short instruction without worked demonstrations. This is a prompting distinction rather than a new supervised training procedure. Self-Consistency Improves Chain of Thought Reasoning samples diverse reasoning paths and aggregates their final answers instead of selecting one greedy path. Agreement can help in tested settings, but several samples can also share the same error. Intermediate generated text can become context for later tokens in an autoregressive setup, allowing a solution to be expressed over several stages. That mechanism does not guarantee correct decomposition or explain every observed gain. The original DeepSeek-R1 report studies reinforcement learning and staged training for reasoning behavior, including R1-Zero’s RL experiment and R1’s additional stages. Training a reasoning model and prompting a fixed model with examples should therefore be distinguished. Finally, Language Models Don’t Always Say What They Think demonstrates explanations that omit biasing influences and rationalize affected answers in particular model tests. A fluent solution should be checked against the task, and faithfulness to the model’s underlying decision process is a separate evaluation question.
Comparing worked-example prompting with answer-only exemplars on a math task.
Testing the zero-shot phrase studied by Kojima and colleagues.
Sampling several solution paths and aggregating their final answers.
Checking whether a generated explanation omits a known biasing influence on the answer.
Reasoning evaluations can compare prompts, sampling budgets, model training, and verification methods on tasks with known answers or independently checkable solutions. A useful report should record quality, latency, token cost, and repeated failure patterns rather than treating longer explanations as intrinsically better. Studies of faithfulness can test known input influences and whether explanations acknowledge them. Training and inference procedures may yield different improvements, so their contributions need separate comparisons. An expressed reasoning path can help a reader inspect a proposed answer when it is checked, while its fluency alone does not establish reliability.
Newsletter gratuite
Trois histoires d'IA vérifiées chaque matin de la semaine, écrites dans un anglais simple. Gratuit pour toujours, sans publicité.
Un e-mail chaque jour de la semaine. Désabonnez-vous en un clic. Nous ne vendons ni ne partageons jamais votre adresse.
Testez-vous
Commentaires instantanés sur chaque réponse et un certificat partageable avec une pièce d'identité vérifiable une fois que vous avez réussi un cours.
Soutenez l’enseignement gratuit de l’IA. AI Understanding est une organisation à but non lucratif 501(c)(3) : pas de publicité, pas de paywall, jamais. Faire un don
Chain-of-thought prompting elicits intermediate generated steps before a final answer in the cited research setups. It improved some evaluated reasoning tasks, but a plausible explanation is neither a proof of correctness nor a guaranteed account of the model’s actual decision process.
The method elicits an expressed solution path, whose correctness still requires evaluation.
Kojima et coll. (2022) ont constaté que l'ajout de « Pensons étape par étape » incite au raisonnement même sans exemples concrets.
The empirical size-dependent results should not become a universal rule for all models.
The method aggregates answer evidence across samples; agreement can still accompany shared mistakes.
R1-Zero and the fuller R1 procedure have different stages; both should be distinguished from prompting alone.
Continuez à apprendre
Plus de guides sélectionnés pour ce sujet
À suivreGuide suivant
When Chain-of-Thought Hurts
IA linguistique