Guide-yakabatana mibvunzo · Pakati Nhanho
Reward Modelling Quiz
Test reward-model scores, preference data, pairwise ranking loss, reward hacking, and best-of-N use in RLHF. Check assumptions and scope.
Mubvunzo 1 ye 8
Ndeipi yekutanga kubuda kwemubairo modhi kune yakapihwa kukurumidza-mhinduro vaviri?
Tarisa mubvunzo wese8 mibvunzo
Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.
- Ndeipi yekutanga kubuda kwemubairo modhi kune yakapihwa kukurumidza-mhinduro vaviri?
- A single scalar score
- Mhinduro yemavara akazara
- Mukana wekugovera pamusoro pemazwi
- Rondedzero yematokeni anodiwa
- What feedback format did the InstructGPT paper collect to train its reward model?
- A guaranteed correctness label for every possible answer
- Human rankings of several model responses, converted into pairwise preferences
- A single sentiment score for each training prompt
- Only binary safety labels with no response comparisons
- Ko chiyero chemubairo-modhi yekurasikirwa chinogonesa chii?
- Mhedziso yemubairo kukosha kwemhinduro yega yega
- Iyo inotevera-chiratidzo kufanotaura chokwadi
- Kuti mhinduro yakasarudzwa inokwira kupfuura yarambwa
- Hurefu hwemhinduro
- Chii chinonzi 'kupa mubayiro'?
- Kubira mukwikwidzi wemubayiro modhi
- Nzira yekukurumidza kudzidzira
- Kunyora mubairo chiratidzo
- Iyo mutemo inoshandisa kukanganisa mumuenzaniso wemubairo kuti uwane zvibodzwa zvepamusoro pasina mhando chaiyo
- Ko muenzaniso wemubairo unotorwa sei kubva mumuenzaniso wemutauro?
- Nekutsiva iyo token-prediction head ine imwechete scalar inobuda musoro
- Nekuwedzera mamwe matipi ekutarisisa
- Nekubvisa ese ma transformer layers
- Nokupeta mashoko
- Muenzaniso wemubairo unoshandiswa sei panguva yekufungidzira mune yakanakisa-ye-N sampling?
- Kudzidzira N mibayiro modhi mune zvakafanana
- Kugadzira mhinduro dzakawanda uye kushandisa iyo mubairo modhi kusarudza yepamusoro-yepamusoro-zvibodzwa
- Kusarudza maN akajairika tokeni
- Avhareji N modhi yekutarisa nzvimbo
- Why might an RLHF study use an ensemble of reward models during training?
- To guarantee that reward hacking is impossible
- To replace the human preference data with exact correctness labels
- To estimate disagreement or uncertainty across learned preference scorers
- To ensure every candidate receives the same score
- Why should reward-model evaluations include held-out comparisons?
- To ensure training comparisons are never used
- To make every output identical
- To guarantee the reward is unbiased
- To test whether preferences generalize beyond the training comparisons