BERT and Encoder Models Quiz

Answers and explanations appear as you take the quiz.

1. Which contextual access does bidirectional attention permit in the original BERT encoder input?
   • Information from both left and right within the allowed input
   • Only the previous token in every layer
   • Only a second unrelated language
   • A guarantee of two correct answers to every question
2. Which original BERT pretraining objective predicts selected token identities using surrounding context?
   • Only next-token prediction under a causal mask
   • Only human-written class labels for complete documents
   • Only a fixed image reconstruction objective
   • Masked language modeling
3. Which transformer component provides the original BERT representation architecture?
   • Only the autoregressive decoder
   • The encoder
   • A recurrent-only sequence network
   • A convolution-only image decoder
4. What sequence of training stages is illustrated by BERT pretraining followed by downstream fine-tuning?
   • Always train a completely new network from random parameters for every example
   • Apply the task head before creating any learned representation
   • Learn a corpus-based starting representation, then adapt it to a defined task
   • Guarantee that every target task uses no labeled data
5. What output can the original BERT encoder provide to classification or span-prediction heads?
   • Only a long autoregressive continuation in every task
   • Raw audio waveforms
   • Pixel-level image output
   • Contextual text representations
6. Which comparison accurately distinguishes the parameter-related designs of DistilBERT and ALBERT?
   • DistilBERT uses distillation; ALBERT uses factorized embeddings and cross-layer sharing
   • Both are guarantees of identical speed and accuracy on every task
   • Both replace all text input with image pixels
   • Neither retains any learned encoder representation
7. Which label distinction is used in the original BERT next-sentence prediction task?
   • A positive versus negative product review
   • A token's verb tense versus noun number
   • An actual next sentence versus a randomly sampled alternative
   • A source sentence's font size versus text color
8. Which three representations are added in the original BERT input embedding?
   • Only a single corpus-frequency number
   • Token, position, and segment embeddings
   • Decoder probabilities, image pixels, and audio pitch
   • Only one fixed vector shared by every input position