Encoder-Decoder Architectures Quiz

Answers and explanations appear as you take the quiz.

1. What is the main job of the encoder in an encoder-decoder model?
   • To read the input and turn it into rich internal representations
   • To generate the output one token at a time
   • To rank candidate outputs by probability
   • To store the model's training data
2. Why does the decoder use masked (causal) self-attention?
   • To hide the input from the encoder
   • To prevent each position from seeing future tokens during generation
   • To compress the output into fewer tokens
   • To randomly drop tokens for regularization
3. What mechanism connects the decoder to the encoder's representations?
   • Dropout
   • Batch normalization
   • Cross-attention
   • Positional encoding
4. Which of these models is an encoder-decoder (sequence-to-sequence) architecture?
   • GPT-3
   • BERT
   • Word2Vec
   • T5
5. Why is the encoder-decoder design a natural fit for translation?
   • Because the input and output are different sequences, requiring full comprehension then fresh generation
   • Because the input and output are always the same length
   • Because it never needs to generate new text
   • Because it avoids using attention entirely
6. In the original Transformer architecture, what attention does the encoder use over its input sequence?
   • Causal attention that hides all earlier source tokens
   • Bidirectional self-attention over the full source sequence
   • Cross-attention only to an image sequence
   • No attention mechanism
7. What training objective does BART use for pretraining?
   • It predicts only the next pixel in an image
   • It classifies each word into a fixed sentiment label
   • It corrupts text and trains a sequence-to-sequence denoising model to reconstruct the original
   • It performs speech recognition from raw audio
8. How does T5 frame many NLP tasks for one encoder-decoder model?
   • As image classification with a shared label set
   • As speech waveforms with no text tokens
   • As separate model architectures for each task
   • As text-to-text inputs and outputs with task descriptions in the input text