Guide-linked quiz · Medium Level

When Chain-of-Thought Hurts Quiz

Tests the definition of chain-of-thought, bounded research results, model-specific guidance, and task-based comparisons including latency.

Related guide pathsWhen Chain Of Thought Hurts
Question 1 of 8

What does chain-of-thought prompting ask a model to produce?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. What does chain-of-thought prompting ask a model to produce?
    • Intermediate reasoning text before its answer
    • Only an image input
    • A guaranteed proof of correctness
    • A list of training examples
  2. What did the 2025 ICML study report across its six selected tasks?
    • The study tested every task category
    • CoT reduced performance on every task
    • Performance drops on three tasks and mixed effects on the others
    • CoT always improved performance
  3. What does that study establish about CoT across all AI tasks?
    • It guarantees better performance for reasoning models
    • It applies only to image generation
    • It is universally harmful
    • It can reduce performance in some settings but does not establish a universal rule
  4. Why might an explicit rationale add operational cost?
    • It reduces all input context
    • It automatically stops tool use
    • It eliminates evaluation needs
    • It can use additional output tokens and increase latency
  5. How should a model-generated explanation be treated as evidence?
    • As proof the answer is correct
    • As insufficient by itself to prove correctness or faithfully trace the internal process
    • As confirmation the model used a source
    • As a guarantee of transparency
  6. What does OpenAI’s current guidance recommend for its API reasoning models?
    • Never state constraints
    • Always demand a long chain of thought
    • Generally avoid explicit “think step by step” requests
    • Use identical prompts across every model
  7. How should a team decide whether CoT helps its own application?
    • Compare different models and prompts at once
    • Judge by which answer is longer
    • Compare it with direct prompting on representative cases using defined scoring
    • Assume the research paper covers its task
  8. Which factors should a team include when comparing prompt variants?
    • Output quality and relevant costs such as latency or token use
    • Only model marketing claims
    • Whether the explanation sounds confident
    • Only response length

Keep testing yourself

More quizzes picked for your level.

View all quizzes