Isunmọ Ilana adanwo

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

1. Isoro wo ni 'pipin' PPO n koju ni akọkọ?
   • Awọn imudojuiwọn eto imulo ti o tobi pupọ ti o mu ikẹkọ bajẹ
   • Ikojọpọ data ti o lọra
   • Awọn ifilelẹ iwọn fokabulari
   • Lilo iranti lakoko itọkasi
2. Ni awoṣe ede-ede RLHF pẹlu PPO, nibo ni ifihan ere nigbagbogbo ti wa?
   • Ifimaaki eniyan taara ti gbogbo ami
   • A oṣiṣẹ ère awoṣe
   • Oṣuwọn ẹkọ ti o dara julọ
   • Awọn iwọn ti awọn fokabulari
3. Kini ijiya iyatọ-KL ṣe ni RLHF ti o da lori PPO?
   • Iyara soke àmi iran
   • Mu awọn ere awoṣe ká išedede
   • Ṣetọju awoṣe aifwy ti o dara lati sẹsẹ pupọ ju awoṣe itọkasi
   • Yọ awọn nilo fun a iye nẹtiwọki
4. Kini ipa ti nẹtiwọọki iye (alariwisi) ni PPO?
   • Lati ṣe ipilẹṣẹ abajade ọrọ ikẹhin
   • Lati tọju data ikẹkọ
   • Lati rọpo awoṣe ere patapata
   • Lati ṣe iṣiro ipadabọ ti o nireti ki awọn anfani le ṣe iṣiro
5. Kini 'anfani' ṣe aṣoju ninu PPO?
   • Elo ni iṣe ti o dara julọ ju ipilẹ ti a reti lọ
   • Iwọn paramita lapapọ ti awoṣe
   • Iwọn gige
   • Olusọdipúpọ ijiya KL
6. What distinguishes GRPO from PPO in the DeepSeekMath paper?
   • It removes all policy sampling and optimizes only supervised labels
   • It estimates relative advantages from groups of sampled responses without a separate value critic
   • It uses a trained reward model as a value network
   • It guarantees zero policy drift after one update
7. What probability ratio does PPO clip in its surrogate objective?
   • The reward divided by the number of prompts
   • The value estimate divided by the KL coefficient
   • The new policy’s action probability divided by the old policy’s action probability
   • The sequence length divided by batch size
8. Why is PPO called an on-policy method in its standard form?
   • It trains only on a static image dataset
   • It never samples actions
   • It updates only a reward model
   • It collects trajectories using the current policy being optimized