Idanwo ti o sopọ mọ itọsọna · Alabọde Ipele

Awoṣe adanwo ère

Test reward-model scores, preference data, pairwise ranking loss, reward hacking, and best-of-N use in RLHF. Check assumptions and scope.

Jẹmọ itọsọna awọn ọnaAwoṣe Ere
Ibeere 1 ti 8

Kini abajade akọkọ ti awoṣe ere kan fun bata-idahun kiakia?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Kini abajade akọkọ ti awoṣe ere kan fun bata-idahun kiakia?
    • A nikan scalar Dimegilio
    • Idahun ọrọ ni kikun
    • A iṣeeṣe pinpin lori awọn fokabulari
    • Atokọ ti awọn ami ti o fẹ
  2. What feedback format did the InstructGPT paper collect to train its reward model?
    • A guaranteed correctness label for every possible answer
    • Human rankings of several model responses, converted into pairwise preferences
    • A single sentiment score for each training prompt
    • Only binary safety labels with no response comparisons
  3. Kini isonu ere-awoṣe boṣewa mu dara si?
    • Awọn idi ere iye ti gbogbo esi
    • Ipeye asọtẹlẹ ami atẹle
    • Wipe idahun ti o yan jẹ ti o ga ju eyi ti a kọ lọ
    • Awọn ipari ti awọn idahun
  4. Kini 'fijipa ere'?
    • Jiji awoṣe ere oludije
    • Ọna kan fun iyara ikẹkọ
    • Encrypting ifihan ere
    • Eto imulo ti nlo awọn abawọn ninu awoṣe ere lati gba awọn ikun giga laisi didara tootọ
  5. Bawo ni awoṣe ẹsan ṣe jẹyọ ni ọna ayaworan lati inu awoṣe ede kan?
    • Nipa rirọpo awọn ami-asọtẹlẹ ori pẹlu kan nikan scalar o wu jade ori
    • Nipa fifi awọn ipele akiyesi diẹ sii
    • Nipa yiyọ gbogbo awọn Layer transformer
    • Nípa sísọ ọ̀rọ̀ ìlọ́po méjì
  6. Bawo ni awoṣe ẹsan ṣe lo ni akoko idawọle ni iṣapẹẹrẹ-ti-N ti o dara julọ?
    • Ikẹkọ N awọn awoṣe ere ni afiwe
    • Ṣiṣẹda awọn idahun lọpọlọpọ ati lilo awoṣe ere lati yan ọkan ti o ga julọ
    • Yiyan awọn N wọpọ àmi
    • Apapọ N awoṣe checkpoints
  7. Why might an RLHF study use an ensemble of reward models during training?
    • To guarantee that reward hacking is impossible
    • To replace the human preference data with exact correctness labels
    • To estimate disagreement or uncertainty across learned preference scorers
    • To ensure every candidate receives the same score
  8. Why should reward-model evaluations include held-out comparisons?
    • To ensure training comparisons are never used
    • To make every output identical
    • To guarantee the reward is unbiased
    • To test whether preferences generalize beyond the training comparisons