Bài kiểm tra có liên kết hướng dẫn · Cứng Cấp độ

Tinh chỉnh tăng cường với bài kiểm tra của học sinh

Kiểm tra xem học sinh chấm điểm kết quả đầu ra của mô hình như thế nào, nhiệm vụ nào phù hợp với việc tinh chỉnh tăng cường cũng như cách phát sinh các tín hiệu đào tạo yếu và hack phần thưởng.

Đường dẫn hướng dẫn liên quanTinh chỉnh cốt thép với máy chấm điểm
Câu hỏi 1 của 8

How does reinforcement fine-tuning differ from supervised fine-tuning?

Xem trước mọi câu hỏi8 câu hỏi

Câu trả lời và giải thích xuất hiện khi bạn làm bài kiểm tra.

  1. How does reinforcement fine-tuning differ from supervised fine-tuning?
    • It trains the model to imitate exact target outputs
    • It rewards sampled outputs a grader scores highly instead of copying examples
    • It needs no training prompts
    • It only changes the system prompt
  2. Which grader type would you use to check that generated code actually works?
    • A programmatic grader that runs unit tests
    • A string-match grader comparing to one reference function
    • A grader that counts words
    • No grader is needed for code
  3. Which task is the poorest fit for RFT according to the guide?
    • Classifying items into defined category codes
    • Solving maths problems with known answers
    • Judging subjective quality where expert graders disagree
    • Structured data extraction with a reference answer
  4. In Reinforcement Fine-Tuning with Graders: what is reward hacking?
    • Stealing another team's grader
    • The model learning to satisfy the grader rather than the real intent
    • Using too little compute during training
    • A grader giving partial credit
  5. Why do prompts where every sample gets the same score provide little learning signal in group-relative methods like GRPO?
    • Because the grader crashes on them
    • Because learning comes from score differences within the group
    • Because they are too long
    • Because GRPO ignores scores entirely
  6. What is the purpose of a KL penalty against the original model during RFT?
    • To speed up grading
    • To limit how far the model drifts from its starting behaviour
    • To increase output length
    • To replace the grader
  7. A model grader over-rewards confident, lengthy replies. What is the likely outcome of training against it?
    • Outputs drift toward longer, more confident wording
    • Mô hình ngừng sản xuất đầu ra
    • Accuracy is guaranteed to improve
    • The grader becomes stricter automatically
  8. Which is the best way to check that rising training reward reflects real improvement?
    • Trust the training reward curve
    • Increase the learning rate
    • Spot-check outputs by hand or with an independent grader on held-out data
    • Remove the validation set