Kuiz berkaitan panduan · Keras Tahap

Penalaan Halus Pengukuhan dengan Kuiz Penggred

Menguji cara penggred menjaringkan output model, tugas yang sesuai dengan penalaan halus tetulang, dan cara penggodaman ganjaran dan isyarat latihan yang lemah timbul.

Laluan panduan yang berkaitanPenalaan Halus Pengukuhan Dengan Penggred
soalan 1 daripada 8

How does reinforcement fine-tuning differ from supervised fine-tuning?

Pratonton setiap soalan8 soalan

Jawapan dan penjelasan muncul semasa anda mengambil kuiz.

  1. How does reinforcement fine-tuning differ from supervised fine-tuning?
    • It trains the model to imitate exact target outputs
    • It rewards sampled outputs a grader scores highly instead of copying examples
    • It needs no training prompts
    • It only changes the system prompt
  2. Which grader type would you use to check that generated code actually works?
    • A programmatic grader that runs unit tests
    • A string-match grader comparing to one reference function
    • A grader that counts words
    • No grader is needed for code
  3. Which task is the poorest fit for RFT according to the guide?
    • Classifying items into defined category codes
    • Solving maths problems with known answers
    • Judging subjective quality where expert graders disagree
    • Structured data extraction with a reference answer
  4. In Reinforcement Fine-Tuning with Graders: what is reward hacking?
    • Stealing another team's grader
    • The model learning to satisfy the grader rather than the real intent
    • Using too little compute during training
    • A grader giving partial credit
  5. Why do prompts where every sample gets the same score provide little learning signal in group-relative methods like GRPO?
    • Because the grader crashes on them
    • Because learning comes from score differences within the group
    • Because they are too long
    • Because GRPO ignores scores entirely
  6. What is the purpose of a KL penalty against the original model during RFT?
    • To speed up grading
    • To limit how far the model drifts from its starting behaviour
    • To increase output length
    • To replace the grader
  7. A model grader over-rewards confident, lengthy replies. What is the likely outcome of training against it?
    • Outputs drift toward longer, more confident wording
    • Model berhenti menghasilkan output
    • Accuracy is guaranteed to improve
    • The grader becomes stricter automatically
  8. Which is the best way to check that rising training reward reflects real improvement?
    • Trust the training reward curve
    • Increase the learning rate
    • Spot-check outputs by hand or with an independent grader on held-out data
    • Remove the validation set