Logit Lens and Tuned Lens Quiz

Answers and explanations appear as you take the quiz.

1. What does the logit lens do?
   • Applies the model's final unembedding to intermediate hidden states to read predictions per layer
   • Adds new layers to a transformer
   • Removes attention from the model
   • Translates between human languages
2. What key improvement does the tuned lens add over the raw logit lens?
   • It uses a larger vocabulary
   • It trains a small per-layer affine map to translate hidden states into the final basis
   • It removes the need for any unembedding
   • It only works on image models
3. What structure in transformers makes these lenses possible?
   • The convolutional filters
   • The tokenizer
   • The additive residual stream that the unembedding later projects to logits
   • The optimizer's learning rate
4. Who introduced the original logit lens, and roughly when?
   • DeepMind in 2015
   • OpenAI in 2018
   • Google in 2012
   • nostalgebraist in 2020
5. Where does the logit lens often show the final answer 'crystallizing'?
   • In middle-to-late layers
   • Only in the very first layer
   • Never, the answer appears all at once
   • Outside the model entirely
6. Why might the raw logit lens be unreliable in early layers?
   • Early layers have no neurons
   • Their hidden states are not yet aligned with the final output basis the unembedding expects
   • Early layers are never used in prediction
   • The unembedding matrix is random
7. What does the model’s final unembedding map a residual-stream vector into?
   • A new set of transformer layers
   • A similarity graph over training examples
   • Vocabulary logits that can be converted into token probabilities
   • A gradient clipping threshold
8. Why can an early-layer raw logit-lens prediction be misleading?
   • The layer has already generated the full answer
   • The tokenizer has been removed
   • The unembedding only works on images
   • The intermediate state may not yet be aligned with the final output basis