Guide-linked quiz · Medium Level
When Chain-of-Thought Hurts Quiz
Tests the definition of chain-of-thought, bounded research results, model-specific guidance, and task-based comparisons including latency.
Question 1 of 8
What does chain-of-thought prompting ask a model to produce?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- What does chain-of-thought prompting ask a model to produce?
- Intermediate reasoning text before its answer
- Only an image input
- A guaranteed proof of correctness
- A list of training examples
- What did the 2025 ICML study report across its six selected tasks?
- The study tested every task category
- CoT reduced performance on every task
- Performance drops on three tasks and mixed effects on the others
- CoT always improved performance
- What does that study establish about CoT across all AI tasks?
- It guarantees better performance for reasoning models
- It applies only to image generation
- It is universally harmful
- It can reduce performance in some settings but does not establish a universal rule
- Why might an explicit rationale add operational cost?
- It reduces all input context
- It automatically stops tool use
- It eliminates evaluation needs
- It can use additional output tokens and increase latency
- How should a model-generated explanation be treated as evidence?
- As proof the answer is correct
- As insufficient by itself to prove correctness or faithfully trace the internal process
- As confirmation the model used a source
- As a guarantee of transparency
- What does OpenAI’s current guidance recommend for its API reasoning models?
- Never state constraints
- Always demand a long chain of thought
- Generally avoid explicit “think step by step” requests
- Use identical prompts across every model
- How should a team decide whether CoT helps its own application?
- Compare different models and prompts at once
- Judge by which answer is longer
- Compare it with direct prompting on representative cases using defined scoring
- Assume the research paper covers its task
- Which factors should a team include when comparing prompt variants?
- Output quality and relevant costs such as latency or token use
- Only model marketing claims
- Whether the explanation sounds confident
- Only response length
Keep testing yourself
More quizzes picked for your level.