RoBERTa Training Recipe Quiz

Answers and explanations appear as you take the quiz.

1. What was RoBERTa's main insight about the original BERT?
   • BERT's architecture was fundamentally flawed
   • BERT used too many parameters
   • BERT was significantly undertrained
   • BERT needed a decoder added
2. Which BERT objective did RoBERTa remove?
   • Next sentence prediction
   • Masked language modeling
   • Token classification
   • Replaced token detection
3. What is dynamic masking in RoBERTa?
   • Masking only the rarest words in the corpus
   • Masking entire sentences at once
   • Never masking the same token twice in a corpus
   • Generating a new mask pattern each time a sequence is seen
4. How did RoBERTa's training data compare to BERT's?
   • It used the same 16GB of text
   • It used roughly 160GB, about 10x more
   • It used less data but cleaner labels
   • It used only Wikipedia
5. Which organization released RoBERTa?
   • Facebook AI (Meta)
   • Google AI
   • OpenAI
   • Microsoft Research
6. What is the multilingual version of RoBERTa called?
   • mBERT
   • RoBERTa-XL
   • XLM-R (XLM-RoBERTa)
   • PolyRoBERTa
7. Which text encoding did the RoBERTa recipe use according to the guide?
   • A fixed vocabulary of complete words only
   • Byte-level byte-pair encoding
   • One image patch per token
   • A phoneme-only vocabulary
8. A researcher credits a new layer for beating BERT but also changes the data volume and training duration. Which lesson from RoBERTa matters?
   • Architecture changes always explain every improvement
   • Training duration can be ignored if the benchmark is the same
   • Data volume cannot affect an encoder
   • Training choices can account for gains, so they must be considered when comparing models