Reinforcement Fine-Tuning with Graders Quiz

Answers and explanations appear as you take the quiz.

1. How does reinforcement fine-tuning differ from supervised fine-tuning?
   • It trains the model to imitate exact target outputs
   • It rewards sampled outputs a grader scores highly instead of copying examples
   • It needs no training prompts
   • It only changes the system prompt
2. Which grader type would you use to check that generated code actually works?
   • A programmatic grader that runs unit tests
   • A string-match grader comparing to one reference function
   • A grader that counts words
   • No grader is needed for code
3. Which task is the poorest fit for RFT according to the guide?
   • Classifying items into defined category codes
   • Solving maths problems with known answers
   • Judging subjective quality where expert graders disagree
   • Structured data extraction with a reference answer
4. In Reinforcement Fine-Tuning with Graders: what is reward hacking?
   • Stealing another team's grader
   • The model learning to satisfy the grader rather than the real intent
   • Using too little compute during training
   • A grader giving partial credit
5. Why do prompts where every sample gets the same score provide little learning signal in group-relative methods like GRPO?
   • Because the grader crashes on them
   • Because learning comes from score differences within the group
   • Because they are too long
   • Because GRPO ignores scores entirely
6. What is the purpose of a KL penalty against the original model during RFT?
   • To speed up grading
   • To limit how far the model drifts from its starting behaviour
   • To increase output length
   • To replace the grader
7. A model grader over-rewards confident, lengthy replies. What is the likely outcome of training against it?
   • Outputs drift toward longer, more confident wording
   • The model stops producing output
   • Accuracy is guaranteed to improve
   • The grader becomes stricter automatically
8. Which is the best way to check that rising training reward reflects real improvement?
   • Trust the training reward curve
   • Increase the learning rate
   • Spot-check outputs by hand or with an independent grader on held-out data
   • Remove the validation set