Ikibazo gihuza ibibazo · Hagati Urwego

Ikibazo cya Grokking kandi cyatinze Kubazwa

Shakisha gutinda muri rusange kubijyanye namakuru ya algorithmic, hamwe nubushakashatsi bwangirika bwibiro, isesengura ryumuzunguruko wa Fourier, kongeramo modular, ningaruka za dataset.

Inzira zijyanye no kuyoboraGukubita no Gutinda muri rusange
Ikibazo 1 Bya 8

Ni ubuhe buryo bwo gukora buranga ubushakashatsi bwakozwe?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Ni ubuhe buryo bwo gukora buranga ubushakashatsi bwakozwe?
    • Held-out accuracy must improve before any training example is learned
    • Training accuracy can never become high
    • Every model loses all training information at once
    • Held-out generalization improves substantially after the training examples have already been fit
  2. Nibihe bikorwa urupapuro rwumwimerere rwakoresheje nkurwego rugenzurwa?
    • Only large-scale photorealistic image generation
    • Small algorithmically generated tasks such as modular arithmetic
    • Only real-time online advertising auctions
    • Only unlabelled random filenames
  3. Ni ubuhe buryo bwo kwishora mubikorwa byari byasuzumwe mubushakashatsi bwambere bwo gufata?
    • Weight decay
    • Mandatory deletion of every activation function
    • A proof that all test labels must be randomized
    • Automatic enlargement of the vocabulary on every step
  4. Ni ubuhe buryo bwo kubara bwatanzwe bwa modular-yongeyeho ubukanishi bwerekanye muri transformateur ntoya yatojwe?
    • A universal rule for every large deployed model
    • Only a table containing every held-out answer
    • A Fourier-based algorithm using trigonometric relationships
    • No identifiable structured computation
  5. Ni ikihe cyiciro cyo gutondekanya ingamba ziterambere kuri Grokking zisobanura mubikorwa byamahugurwa yize?
    • Cleanup, deletion of all data, then no computation
    • Memorization, circuit formation, then cleanup
    • Perfect held-out performance before any circuit formation
    • A mandatory switch to image-only inputs
  6. Ni ubuhe bwitonzi bufatika butanga ibisobanuro byerekana abimenyereza?
    • Buri gihe uhagarike imyitozo mugihe imyitozo yukuri igera 100%
    • Kwemeza neza ntaho bihuriye
    • Kwemeza hakiri kare ntabwo bivuze ko icyitegererezo cyananiwe kwiga
    • Ibisanzwe ntibigomba gukoreshwa
  7. For the modular-addition task, what is (6+5) modulo 7?
    • 11
    • 7
    • 1
    • 4
  8. What dataset-size trend did the original grokking paper report in its studied setups?
    • Smaller training datasets required more optimization to reach generalization
    • Smaller datasets always generalized immediately with no optimization
    • Dataset size was never varied
    • Every larger dataset prevented all training accuracy