Maswali yanayounganishwa na mwongozo · Kati Kiwango

Maswali ya Grokking na Kucheleweshwa kwa Ujumla

Gundua ujumuishaji uliocheleweshwa kwenye data ya algoriti, kwa majaribio ya kupunguza uzito, uchanganuzi wa mzunguko wa Fourier, kuongeza moduli na madoido ya ukubwa wa seti ya data.

Njia za mwongozo zinazohusianaGrokking Na Kucheleweshwa Ujumla
Swali 1 ya 8

Which performance pattern characterizes the studied grokking phenomenon?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. Which performance pattern characterizes the studied grokking phenomenon?
    • Held-out accuracy must improve before any training example is learned
    • Training accuracy can never become high
    • Every model loses all training information at once
    • Held-out generalization improves substantially after the training examples have already been fit
  2. Which tasks did the original grokking paper use as a controlled setting?
    • Only large-scale photorealistic image generation
    • Small algorithmically generated tasks such as modular arithmetic
    • Only real-time online advertising auctions
    • Only unlabelled random filenames
  3. Which regularization intervention was among those examined in the original grokking experiments?
    • Weight decay
    • Mandatory deletion of every activation function
    • A proof that all test labels must be randomized
    • Automatic enlargement of the vocabulary on every step
  4. What computation did the cited modular-addition mechanistic study identify in its trained small transformers?
    • A universal rule for every large deployed model
    • Only a table containing every held-out answer
    • A Fourier-based algorithm using trigonometric relationships
    • No identifiable structured computation
  5. Which phase ordering does Progress Measures for Grokking describe in the studied training dynamics?
    • Cleanup, deletion of all data, then no computation
    • Memorization, circuit formation, then cleanup
    • Perfect held-out performance before any circuit formation
    • A mandatory switch to image-only inputs
  6. Je! ni tahadhari gani ya vitendo ambayo grokking inapendekeza kwa watendaji?
    • Acha kila wakati kutoa mafunzo wakati usahihi wa mafunzo unafikia 100%
    • Usahihi wa uthibitishaji hauna umuhimu
    • Uwanda wa uthibitisho wa mapema haimaanishi kila wakati kwamba mtindo umeshindwa kujifunza
    • Regularization haipaswi kamwe kutumika
  7. For the modular-addition task, what is (6+5) modulo 7?
    • 11
    • 7
    • 1
    • 4
  8. What dataset-size trend did the original grokking paper report in its studied setups?
    • Smaller training datasets required more optimization to reach generalization
    • Smaller datasets always generalized immediately with no optimization
    • Dataset size was never varied
    • Every larger dataset prevented all training accuracy