Kuiz berkaitan panduan · Sederhana Tahap

Kuiz Penalaan Halus Persampelan Penolakan

Meliputi penjanaan dan penapisan calon, ganjaran berasaskan pengesah, latihan semula yang diselia dan risiko daripada mengoptimumkan penjaring yang cacat.

Laluan panduan yang berkaitanPenalaan Halus Pensampelan Penolakan
soalan 1 daripada 8

Which loop best describes how Rejection Sampling Fine-Tuning improves a model?

Pratonton setiap soalan8 soalan

Jawapan dan penjelasan muncul semasa anda mengambil kuiz.

  1. Which loop best describes how Rejection Sampling Fine-Tuning improves a model?
    • Hasilkan banyak jawapan calon, simpan jawapan yang terbaik dan latih semula
    • Padamkan lapisan model berprestasi paling teruk
    • Tolak semua data latihan dan mulakan semula
    • Sampel sekali dan sentiasa terima output
  2. Dalam domain yang boleh disahkan seperti matematik atau kod, apakah yang boleh berfungsi sebagai ganjaran?
    • Model ganjaran yang dipelajari sahaja
    • Sama ada jawapan itu betul atau ujian lulus
    • Panjang jawapan
    • Bilangan token yang dijana
  3. Which model family does the cited paper describe using rejection sampling in its alignment pipeline?
    • GPT-4o
    • BERT Base
    • Meta’s Llama 2-Chat
    • Whisper Large
  4. Why might a team choose rejection-sampling fine-tuning over a policy-gradient approach for some tasks?
    • It never needs a candidate generator or scoring rule
    • It guarantees no reward hacking
    • It always requires less compute than every RL method
    • It can use supervised learning on selected generations instead of updating through policy-gradient RL
  5. Latihan mengenai sampel ganjaran maksimum berkesan melakukan apa?
    • Menyuling kualiti terbaik N ke dalam output sampel tunggal model
    • Meningkatkan saiz perbendaharaan kata
    • Mengalih keluar keupayaan model untuk menjana teks
    • Menggandakan kiraan parameter
  6. What should teams monitor when repeatedly fine-tuning on filtered model-generated outputs?
    • Only the number of model parameters
    • Output diversity alongside held-out correctness
    • Whether every answer uses the same wording
    • Only the training loss on selected samples
  7. What is a limitation of rejection-sampling fine-tuning’s candidate selection?
    • It can only be used with image labels
    • It always changes the tokenizer
    • It cannot train on a high-quality answer that was never generated or retained
    • It makes the scoring function unnecessary
  8. What risk arises if the scoring model used to filter candidates is flawed?
    • The policy becomes unable to generate any text
    • The tokenizer automatically changes vocabulary
    • The reference labels become unnecessary
    • The policy may learn to exploit the scorer rather than improve the intended quality