Ikibazo gihuza ibibazo · Biragoye Urwego

Itsinda ryigihembo gisanzwe mubibazo bya RLHF

Wige uburyo GRPO igereranya ibisubizo byatanzwe mubitsinda ryihuse, ikuraho uwunegura, kandi ikoresha intego ya PPO muburyo bwo kuvugurura politiki.

Inzira zijyanye no kuyoboraItsinda Ryigihembo Cyibisanzwe Muri Rlhf
Ikibazo 1 Bya 8

Mubihembo byitsinda risanzwe, ibihembo bya buri gisubizo ugereranije nibiki?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Mubihembo byitsinda risanzwe, ibihembo bya buri gisubizo ugereranije nibiki?
    • Gutandukanya no kugereranya gutandukana kwitsinda ryayo bwite
    • Ihoraho rihoraho
    • Igihembo cyo mu bihe byashize
    • Icyitegererezo cyo kubara
  2. Niyihe algorithm ifitanye isano cyane no guhembera amatsinda?
    • Kumanuka gahoro gahoro
    • GRPO (Optimisation ya Politiki Yerekeye Itsinda)
    • K-bisobanura guhuriza hamwe
    • Gusubira inyuma
  3. Niki kintu kigize PPO gisanzwe GRPO ikuraho?
    • Icyitegererezo
    • Urusobe rw'imitsi rwose
    • Umuyoboro w'agaciro utandukanye (kunegura)
    • Amakuru y'amahugurwa
  4. Bigenda bite kubisubizo ibihembo biri munsi yitsinda ryayo?
    • Yakira inyungu nziza kandi irashimangirwa
    • Yasibwe muri dataset
    • Igihembo cyacyo cyikubye kabiri
    • Yakiriye inyungu mbi kandi yacitse intege
  5. What does within-prompt reward normalization provide to a GRPO update?
    • A relative, scaled advantage for each response compared with others for the same prompt
    • A guarantee that all prompt difficulty differences disappear
    • A higher learning rate
    • A way to make every response receive the same reward
  6. What happens when every response in a prompt group receives the same reward?
    • Every response gets a positive advantage
    • Centering gives no relative advantage; zero spread also requires numerical handling
    • The group mean doubles the reward
    • The model stops needing a policy objective
  7. Which PPO-style update element does GRPO retain?
    • A separately learned value network
    • K-means clustering over tokens
    • A clipped policy objective
    • Ururimi-rwerekana ibimenyetso
  8. What can a KL penalty relative to a reference policy help constrain in GRPO?
    • The number of prompts in each group
    • The length of the tokenizer vocabulary
    • The resolution of generated images
    • How far the updated policy drifts from the reference policy