Guide-yakabatana mibvunzo · Pakati Nhanho

Proximal Policy Optimization Quiz

Review PPO clipping, policy ratios, reward sources, KL penalties, critics, advantages, and language-model RLHF limits. Check assumptions and scope.

Nzira dzekutungamira dzinoenderanaProximal Policy Optimization
Mubvunzo 1 ye 8

Idambudziko ripi rinonyanya kugadziriswa nePPO?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Idambudziko ripi rinonyanya kugadziriswa nePPO?
    • Yakakura yakawandisa magadzirirwo emitemo anovhiringa kudzidziswa
    • Kutakura data zvishoma
    • Mazwi saizi miganhu
    • Kushandisa Memory panguva yekufungidzira
  2. Mumutauro-modhi RLHF nePPO, chiratidzo chemubairo chinowanzobva kupi?
    • Kunanga zvibodzwa zvemunhu zvechiratidzo chese
    • Muenzaniso wakadzidziswa mubayiro
    • The optimizer's kudzidza mwero
    • Hukuru hwemazwi
  3. Chii chinoitwa neKL-divergence chirango muPPO-based RLHF?
    • Inomhanyisa kugadzirwa kwechiratidzo
    • Inowedzera kurongeka kwemuenzaniso wemubairo
    • Inochengetedza iyo modhi yakanyatso kukwenenzverwa kubva kukukurwa zvakanyanya kure nereferenzi modhi
    • Inobvisa kudiwa kwe network yakakosha
  4. Ndeipi basa rekukosha (kutsoropodza) network muPPO?
    • Kugadzira chinyorwa chekupedzisira chinobuda
    • Kuchengeta data rekudzidziswa
    • Kutsiva modhi yemubayiro zvachose
    • Kufungidzira kudzoka kunotarisirwa saka zvakanakira zvinogona kuverengerwa
  5. Chii chinonzi 'chakanaka' muPPO?
    • Chiito chaive nani sei pane chinotarisirwa kutanga
    • Iyo modhi yakazara parameter kuverenga
    • The clipping range
    • Iyo KL chirango coefficient
  6. What distinguishes GRPO from PPO in the DeepSeekMath paper?
    • It removes all policy sampling and optimizes only supervised labels
    • It estimates relative advantages from groups of sampled responses without a separate value critic
    • It uses a trained reward model as a value network
    • It guarantees zero policy drift after one update
  7. What probability ratio does PPO clip in its surrogate objective?
    • The reward divided by the number of prompts
    • The value estimate divided by the KL coefficient
    • The new policy’s action probability divided by the old policy’s action probability
    • The sequence length divided by batch size
  8. Why is PPO called an on-policy method in its standard form?
    • It trains only on a static image dataset
    • It never samples actions
    • It updates only a reward model
    • It collects trajectories using the current policy being optimized