Activation Steering and Representation Engineering Quiz

Answers and explanations appear as you take the quiz.

1. At what point in the model's operation does activation steering intervene?
   • At inference time, by modifying hidden activations
   • During pretraining only
   • During data collection
   • Only after the output text is printed
2. How is a steering vector commonly computed?
   • By randomly initializing a matrix
   • As the mean difference between activations on contrasting positive and negative prompts
   • By deleting layers from the model
   • By counting tokens in the prompt
3. Which hypothesis underlies why activation steering works?
   • The idea that models store data as plain text
   • The belief that all neurons are identical
   • The linear representation hypothesis, that many features are roughly linear directions
   • The claim that models cannot represent concepts
4. What does the scaling coefficient on a steering vector control?
   • The model's vocabulary size
   • The number of layers in the model
   • The training learning rate
   • How strongly the behavior is pushed in the chosen direction
5. What is a known risk of steering a model too aggressively?
   • Output coherence and capabilities can degrade
   • The model trains itself automatically
   • The file size grows uncontrollably
   • It permanently rewrites the weights
6. Which technique is often paired with steering to find interpretable features to clamp?
   • JPEG compression
   • Sparse autoencoders that decompose activations into features
   • Domain name registration
   • Disk defragmentation
7. How does activation steering differ from fine-tuning the model weights?
   • It retrains the model on a new labeled dataset
   • It changes the tokenizer vocabulary
   • It adds a vector to hidden activations during inference without updating the base weights
   • It replaces the attention layers with a classifier
8. A steering vector works on the prompts used to compute it but not on a different topic. Which limitation does this illustrate?
   • The vector must be an optimizer state
   • Steering always requires fine-tuning
   • A residual stream contains no features
   • The direction found from one prompt set may not generalize to other contexts