Ikibazo gihuza ibibazo · Hagati Urwego
Ikibazo cya Grokking kandi cyatinze Kubazwa
Shakisha gutinda muri rusange kubijyanye namakuru ya algorithmic, hamwe nubushakashatsi bwangirika bwibiro, isesengura ryumuzunguruko wa Fourier, kongeramo modular, ningaruka za dataset.
Ikibazo 1 Bya 8
Ni ubuhe buryo bwo gukora buranga ubushakashatsi bwakozwe?
Reba buri kibazo8 ibibazo
Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.
- Ni ubuhe buryo bwo gukora buranga ubushakashatsi bwakozwe?
- Held-out accuracy must improve before any training example is learned
- Training accuracy can never become high
- Every model loses all training information at once
- Held-out generalization improves substantially after the training examples have already been fit
- Nibihe bikorwa urupapuro rwumwimerere rwakoresheje nkurwego rugenzurwa?
- Only large-scale photorealistic image generation
- Small algorithmically generated tasks such as modular arithmetic
- Only real-time online advertising auctions
- Only unlabelled random filenames
- Ni ubuhe buryo bwo kwishora mubikorwa byari byasuzumwe mubushakashatsi bwambere bwo gufata?
- Weight decay
- Mandatory deletion of every activation function
- A proof that all test labels must be randomized
- Automatic enlargement of the vocabulary on every step
- Ni ubuhe buryo bwo kubara bwatanzwe bwa modular-yongeyeho ubukanishi bwerekanye muri transformateur ntoya yatojwe?
- A universal rule for every large deployed model
- Only a table containing every held-out answer
- A Fourier-based algorithm using trigonometric relationships
- No identifiable structured computation
- Ni ikihe cyiciro cyo gutondekanya ingamba ziterambere kuri Grokking zisobanura mubikorwa byamahugurwa yize?
- Cleanup, deletion of all data, then no computation
- Memorization, circuit formation, then cleanup
- Perfect held-out performance before any circuit formation
- A mandatory switch to image-only inputs
- Ni ubuhe bwitonzi bufatika butanga ibisobanuro byerekana abimenyereza?
- Buri gihe uhagarike imyitozo mugihe imyitozo yukuri igera 100%
- Kwemeza neza ntaho bihuriye
- Kwemeza hakiri kare ntabwo bivuze ko icyitegererezo cyananiwe kwiga
- Ibisanzwe ntibigomba gukoreshwa
- For the modular-addition task, what is (6+5) modulo 7?
- 11
- 7
- 1
- 4
- What dataset-size trend did the original grokking paper report in its studied setups?
- Smaller training datasets required more optimization to reach generalization
- Smaller datasets always generalized immediately with no optimization
- Dataset size was never varied
- Every larger dataset prevented all training accuracy