Long Short-Term Memory Cells Quiz

Answers and explanations appear as you take the quiz.

1. In a common modern three-gate LSTM variant, which gates regulate cell information and exposed output?
   • Read, write, and execute gates
   • Start, stop, and pause gates
   • Forget, input, and output gates
   • Encode, decode, and attend gates
2. What training difficulty was LSTM designed to mitigate?
   • Too many parameters to store
   • Vanishing gradients during learning over long sequences
   • Inability to use a GPU
   • Lack of an activation function
3. Who introduced the LSTM, and in what year?
   • Vaswani and colleagues in 2017
   • Rumelhart and Hinton in 1986
   • LeCun and Bengio in 2012
   • Hochreiter and Schmidhuber in 1997
4. In a common LSTM cell update, why can the additive cell-state path help gradient flow?
   • It reduces reliance on repeated multiplicative transformations across time steps
   • It multiplies the state by a large number each step
   • It deletes the state every step
   • It freezes all weights during training
5. What activation function do LSTM gates typically use to act as soft on/off switches?
   • ReLU, outputting 0 to infinity
   • Softmax over the vocabulary
   • A step function outputting exactly 0 or 1
   • Sigmoid, outputting 0 to 1
6. Why have Transformers largely replaced LSTMs for large-scale language modeling?
   • Transformers cannot handle text at all
   • LSTMs require no training
   • Transformers parallelize across a sequence and capture long-range context via attention
   • Transformers use far fewer numbers in total
7. If an LSTM forget gate is near one and its input gate near zero, what happens to the cell state?
   • It largely retains the previous state
   • It is forced to zero
   • It becomes the raw input token
   • It must reverse the sequence
8. Which gate controls how much of an LSTM cell’s stored state is exposed as output?
   • The forget gate alone
   • The output gate
   • The optimizer momentum term
   • The positional encoding gate