ELECTRA Pretraining Quiz

Answers and explanations appear as you take the quiz.

1. What pretraining task does ELECTRA use instead of masked language modeling?
   • Replaced token detection
   • Next sentence prediction
   • Autoregressive next-word prediction
   • Permutation language modeling
2. Why is ELECTRA more compute-efficient than BERT?
   • It uses a much smaller vocabulary
   • It computes a learning signal over all tokens, not just masked ones
   • It only trains on the masked 15% of tokens
   • It skips the attention mechanism entirely
3. What role does the small generator network play in ELECTRA?
   • It ranks the final outputs by quality
   • It compresses the model after training
   • It produces plausible replacement tokens for the discriminator to judge
   • It translates text between languages
4. What happens to the generator after pretraining is complete?
   • It is fine-tuned for the downstream task
   • It is merged with the discriminator
   • It becomes the decoder for generation
   • It is discarded; only the discriminator is used
5. ELECTRA was developed primarily by researchers from which organizations?
   • Google and Stanford
   • Meta AI and NYU
   • OpenAI and Microsoft
   • DeepMind and Oxford
6. Which later model extended ELECTRA's replaced-token-detection idea?
   • GPT-2
   • DeBERTa-v3
   • T5
   • Word2Vec
7. What happens if ELECTRA’s generator proposes the original token at a position?
   • It is always labeled replaced because the generator proposed it
   • The position is removed from the input
   • That token is treated as original by the discriminator’s replaced-token label
   • The whole sequence is discarded
8. When comparing ELECTRA with another pretraining method, what makes a compute claim meaningful?
   • Compare model names alone
   • Assume every ELECTRA run uses one fixed compute ratio
   • Ignore generator training cost
   • Specify model size, data, training budget, and evaluation setting