Emergent Abilities of Large Language Models Quiz

Answers and explanations appear as you take the quiz.

1. What did early emergence reports describe?
   • Some measured tasks appeared to jump from near-chance to strong performance at larger scale
   • Every model skill improved at an identical rate
   • All capabilities appeared at one fixed parameter count
   • Performance declined smoothly with scale
2. What alternative did Schaeffer and colleagues argue?
   • Model scaling never changes performance
   • Benchmarks cannot measure language models
   • Only larger models can use continuous metrics
   • Metric choice can make gradual changes look like sudden emergence
3. Why can exact-match scoring make a gradual skill look abrupt?
   • It measures partial correctness in detail
   • It removes sampling noise
   • An answer receives no credit until every part matches exactly
   • It records only model size
4. What might happen if partial-credit or continuous metrics are used?
   • Every curve becomes flat
   • Some apparently sharp curves may look smoother
   • The model’s outputs change retroactively
   • All abilities become emergent
5. What is needed before concluding a new ability appeared at a scale threshold?
   • Only compare model parameter counts
   • Use one large checkpoint
   • Rely on a product announcement
   • Examine the task, metric, model family, and statistical evidence
6. What is the relationship between measured emergence and underlying capability growth?
   • The observed curve can reflect both model behavior and the scoring metric
   • The metric has no role
   • The capability claim is always false
   • Only model size matters
7. Why test multiple model sizes and multiple metrics?
   • To make one score look larger
   • To see whether the apparent jump persists under alternative measurements
   • To avoid reporting uncertainty
   • To guarantee a threshold exists
8. What is a careful takeaway from the emergence debate?
   • All sudden capability gains are artifacts
   • Large models have no novel abilities
   • Claims depend on task and measurement; report the metric and scope
   • A single benchmark predicts all future abilities