Linear Probing and Frozen Feature Evaluation Quiz

Answers and explanations appear as you take the quiz.

1. In a standard linear probe, which parameters are updated during training?
   • Only the newly added linear classifier on top
   • The entire backbone plus the linear layer
   • Only the backbone, leaving the classifier fixed
   • The embedding layer and the final attention block
2. What does high linear-probe accuracy most directly show?
   • The backbone causally uses the feature for every prediction
   • The labels are decodable by a linear classifier from the frozen features under that dataset and protocol
   • The representation is robust to every distribution shift
   • The probe has identified a human-interpretable mechanism
3. Compared with full fine-tuning, linear probing is primarily attractive because it is:
   • More accurate on every task
   • Guaranteed to generalize better out of distribution
   • Cheaper in compute and memory and gives a cleaner read on feature quality
   • The only way to use a pretrained model
4. A common empirical finding when probing transformers is that:
   • The final layer always gives the best probe accuracy
   • Probing only works on convolutional networks
   • Layer choice never affects probe results
   • Intermediate layers often outperform the last layer for transfer
5. Beyond benchmarking, linear probes are increasingly used in interpretability to:
   • Detect whether a concept like truthfulness is encoded in hidden states
   • Compress models for mobile deployment
   • Replace the optimizer during pretraining
   • Generate new training data automatically
6. A key methodological risk when interpreting a high probe accuracy is that:
   • The backbone secretly keeps training
   • The probe might be exploiting a spurious correlation rather than the true concept
   • Linear probes cannot overfit
   • Cross-entropy loss cannot be used
7. To compare two backbone layers with linear probes, what should remain consistent?
   • The feature vectors from each layer
   • The layer being evaluated
   • The task labels, data split and probe-training protocol
   • The activation values themselves
8. Why might feature normalization and learning-rate sweeps be used in a probe evaluation?
   • To update the frozen backbone
   • To add hidden nonlinear layers to the probe
   • To guarantee the concept is causally used
   • To reduce optimization and scale effects that could distort the representation comparison