Influence Functions for Training Data Attribution Quiz

Answers and explanations appear as you take the quiz.

1. What counterfactual question do influence functions approximate?
   • How the loss on a test point would change if a training example were removed or up-weighted
   • How accurate the model is on average
   • How fast the model trains
   • How many parameters the model has
2. Which mathematical object makes exact influence computation intractable for large models?
   • The softmax function
   • The inverse Hessian of the loss
   • The learning rate schedule
   • The batch size
3. Who adapted influence functions to modern deep learning in a well-known 2017 paper?
   • Goodfellow and Bengio
   • Vaswani et al.
   • Koh and Liang
   • Hinton and LeCun
4. Under Koh and Liang’s standard I_up,loss convention, what does a negative value indicate?
   • The point was not in the training data
   • Upweighting the point is estimated to increase test loss
   • The point has no relationship to the test example
   • Upweighting the point is estimated to lower test loss, so it is helpful
5. Which approximation did Anthropic use to scale influence functions to large language models?
   • EK-FAC (Kronecker-factored curvature)
   • Dropout
   • Beam search
   • Mixup
6. What did Anthropic's scaled study find about influential examples?
   • They are always exact copies of the output
   • They often share abstract patterns rather than identical wording
   • They are randomly distributed
   • Only the first training example matters
7. In the influence-function approximation, what connects a training-example gradient to the effect on a test loss?
   • The test set’s token count
   • A random forest over training labels
   • The inverse Hessian of the training loss
   • The model’s output vocabulary size
8. If removing one training example is estimated to increase a test point’s loss, what does that suggest about the example?
   • It was guaranteed to be a duplicate
   • It had no relationship to the model
   • It necessarily caused the model to overfit
   • It had a helpful influence on that test point under the approximation