गाइड से जुड़ी प्रश्नोत्तरी · मध्यम स्तर

Inverse Scaling in Language Models Quiz

Interpret inverse scaling with task scores, model-series controls, memorized patterns, distractors, uncertainty, and possible trend reversals.

संबंधित मार्गदर्शक पथInverse Scaling In Language Models
प्रश्न 1 का 8

Three increasingly large models in a comparable series score 80%, 70%, and 60% on the same accuracy task. What does this show?

प्रत्येक प्रश्न का पूर्वावलोकन करें8 प्रश्न

जैसे ही आप प्रश्नोत्तरी लेते हैं उत्तर और स्पष्टीकरण दिखाई देते हैं।

  1. Three increasingly large models in a comparable series score 80%, 70%, and 60% on the same accuracy task. What does this show?
    • An inverse trend on that task over the tested sizes
    • That every capability declines with model size
    • That all three models have identical training loss
    • That a larger fourth model must score below 60%
  2. Why can better average next-token loss coexist with worse performance on an instruction-following task?
    • The two measures always evaluate the same objective
    • Predicting familiar text and following a changed instruction can favor different outputs
    • Lower next-token loss always means worse language modeling
    • Task accuracy is independent of the model’s output
  3. A prompt supplies an unusual ending to a familiar phrase, but the model uses the familiar ending. Which proposed failure pattern does this illustrate?
    • An increase in accelerator memory
    • A reduction in vocabulary size
    • A memorized continuation taking precedence over the supplied instruction
    • A guaranteed improvement in task accuracy
  4. A fourth, larger model scores 85% after the sequence 80%, 70%, and 60%. How should the evaluation report this?
    • Discard the fourth result to preserve the original claim
    • Declare that the earlier measurements never happened
    • Predict all future models will score exactly 85%
    • Report the reversal and restrict the earlier inverse claim to its measured range
  5. Why does comparing unrelated small and large models complicate a claim that size caused a score change?
    • Training data, objectives, and other choices may also differ
    • Different model names make evaluation impossible
    • The smaller model must have seen less data
    • Parameter counts determine every training choice
  6. What can an easier distractor task cause in an inverse-scaling evaluation?
    • The evaluation code must stop running
    • A model may become better at the distracting pattern instead of the intended task
    • All answer options become equally correct
    • The true task is automatically solved
  7. Which information should be preserved so another team can check the evaluation?
    • Only the largest model’s score
    • Only a summary claiming bigger is worse
    • Prompts, answer format, scoring code, model identifiers, and tested sizes
    • Only the number of generated tokens
  8. If a model’s loss falls as scale increases, can that decline alone be called inverse scaling?
    • Yes, every downward curve is inverse scaling
    • Yes, lower loss means fewer correct predictions by definition
    • Yes, model size determines the direction of every metric
    • No; lower loss normally means improvement, so metric direction matters