ELMo Contextual Embeddings Quiz

Answers and explanations appear as you take the quiz.

1. What is the key difference between ELMo and earlier embeddings like Word2Vec?
   • ELMo gives the same word different vectors depending on its sentence context
   • ELMo uses smaller vectors
   • ELMo only works on single words, not sentences
   • ELMo requires no training data
2. What neural architecture does ELMo use to read text?
   • A Transformer with self-attention
   • A deep bidirectional LSTM language model
   • A convolutional image network
   • A simple feedforward network
3. How does ELMo combine its internal layers for a downstream task?
   • It uses only the final top layer
   • It averages all layers equally and fixes the result
   • It uses learned, task-specific weighted combinations of all layers
   • It picks one layer at random each time
4. What does ELMo use as its input units to handle unseen words?
   • Whole-word lookup only
   • Fixed dictionary IDs
   • Byte-pair-encoded subwords from BERT
   • Character-level CNN representations
5. Roughly when was ELMo introduced and by whom?
   • 2018, by Allen Institute for AI researchers
   • 2013, by Google's Word2Vec team
   • 2020, by OpenAI
   • 2015, by Facebook AI
6. Which corpus was used to train the bidirectional language model described in the ELMo paper?
   • The One Billion Word Benchmark
   • The Penn Treebank only
   • A corpus of labeled images
   • The GloVe Common Crawl vectors
7. What language-model prediction does ELMo's forward LSTM learn?
   • Predicting the previous word from future context
   • Assigning a fixed vector to every document
   • Predicting the next word from the preceding context
   • Predicting a sentence's image resolution
8. What linguistic information do ELMo's lower and higher LSTM layers tend to capture?
   • Lower layers encode image color, while higher layers encode audio pitch
   • Every layer stores only word frequency and ignores context
   • Higher layers are always limited to punctuation prediction
   • Lower layers tend toward syntax, while higher layers capture more context-dependent semantics