Isọdọtun Ere ti a ṣe akojọpọ ni adanwo RLHF

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

1. Ninu isọdọtun ere akojọpọ, kini ere idahun kọọkan ni akawe si?
   • Itumọ ati iyapa boṣewa ti ẹgbẹ itọsi tirẹ
   • A ti o wa titi agbaye ibakan
   • Awọn ere lati išaaju epoch
   • Iwọn paramita awoṣe
2. Iru algoridimu wo ni o ni nkan ṣe pẹlu isọdọtun ere akojọpọ?
   • Sitokasitik gradient iran
   • GRPO (Imudara eto imulo ibatan ẹgbẹ)
   • K-tumo si iṣupọ
   • Itankalẹ afẹyinti
3. Apakan wo ni PPO boṣewa ṣe GRPO ni pataki imukuro?
   • Awoṣe ere
   • Nẹtiwọọki nkankikan patapata
   • Nẹtiwọọki iye lọtọ (alariwisi)
   • Awọn data ikẹkọ
4. Kini yoo ṣẹlẹ si idahun ti ere rẹ wa labẹ itumọ ẹgbẹ rẹ?
   • O gba anfani ti o dara ati pe o ni fikun
   • O ti wa ni paarẹ lati awọn dataset
   • Ere rẹ jẹ ilọpo meji
   • O gba anfani odi ati irẹwẹsi
5. What does within-prompt reward normalization provide to a GRPO update?
   • A relative, scaled advantage for each response compared with others for the same prompt
   • A guarantee that all prompt difficulty differences disappear
   • A higher learning rate
   • A way to make every response receive the same reward
6. What happens when every response in a prompt group receives the same reward?
   • Every response gets a positive advantage
   • Centering gives no relative advantage; zero spread also requires numerical handling
   • The group mean doubles the reward
   • The model stops needing a policy objective
7. Which PPO-style update element does GRPO retain?
   • A separately learned value network
   • K-means clustering over tokens
   • A clipped policy objective
   • A language-model tokenizer
8. What can a KL penalty relative to a reference policy help constrain in GRPO?
   • The number of prompts in each group
   • The length of the tokenizer vocabulary
   • The resolution of generated images
   • How far the updated policy drifts from the reference policy