Transformers Quiz

Answers and explanations appear as you take the quiz.

1. Which 2017 paper introduced the transformer architecture?
   • Sequence to Sequence Learning with Neural Networks
   • Attention Is All You Need
   • Neural Machine Translation by Jointly Learning to Align and Translate
   • Convolutional Sequence to Sequence Learning
2. What does self-attention let a transformer do?
   • Retrieve missing facts from an external dictionary
   • Process positions only after the previous position is complete
   • Combine information from other positions in the same sequence into representations
   • Remove the need for position information in every architecture
3. Which comparison matches the guide’s description of transformer architecture variants?
   • The original was encoder-only; later versions all use recurrent decoders
   • The original used an encoder-decoder design; later models may be encoder-only, decoder-only, or encoder-decoder
   • The original was decoder-only; later versions use only convolutions
   • The original omitted attention; later versions introduced it
4. Why do transformers need positional encodings?
   • Because sequence order can matter
   • To map each token ID to its vocabulary embedding
   • To prevent future tokens from being used during next-token prediction
   • To set the maximum number of tokens a system accepts
5. In attention, what are query, key, and value vectors used for?
   • Queries rank possible output tokens, while keys contain the values to copy
   • Values are compared with queries to create weights, and keys are summed
   • Queries encode positions, keys encode token identities, and values set the mask
   • Query-key compatibility determines weights used to combine value vectors
6. What restriction does causal attention impose during next-token prediction?
   • It prevents a position from using earlier tokens
   • It lets each position attend only to the immediately previous token
   • It prevents a position from attending to future tokens
   • It makes each position attend only to itself
7. What does an advertised context limit establish about a model’s performance on a long document?
   • That it will use every part of the document reliably
   • Only the amount of context the system can accept; task performance still needs testing
   • That it will answer every question about the document
   • That the model trained on documents of that length
8. What do attention weights establish about why a model produced its final answer?
   • They are internal calculations, not automatically a faithful explanation
   • They reveal a complete causal account of the answer
   • They prove the answer is factually correct
   • They verify every source used in the response