Idanwo ti o sopọ mọ itọsọna · Lile Ipele

Ẹgbẹ ojulumo Afihan Iṣapeye adanwo

Apply GRPO group reward normalization, critic removal, and reference-policy regularization, including what relative advantages mean within one prompt.

Jẹmọ itọsọna awọn ọnaẸgbẹ ibatan Afihan Iṣapeye
Ibeere 1 ti 8

Which separately learned estimator does the original GRPO approach avoid training?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Which separately learned estimator does the original GRPO approach avoid training?
    • The policy that generates completions
    • A value or critic model
    • Every possible reward function
    • The reference policy’s vocabulary
  2. Bawo ni GRPO ṣe iṣiro anfani fun ipari kan?
    • Nipa bibeere ori iye ti o kọ ẹkọ ni ami-ami kọọkan
    • Nipa lilo awọn aise ere pẹlu ko si normalization
    • Nipa ifiwera si awọn ipari lati itọsi ti o yatọ
    • Nipa iwọntunwọnsi ere rẹ lodi si iwọn ati iyapa boṣewa ti ẹgbẹ ti awọn ipari arakunrin rẹ
  3. Awọn awoṣe wo ni o jẹ ki GRPO mọ daradara?
    • DeepSeek's DeepSeekMath ati awọn awoṣe ero DeepSeek-R1
    • OpenAI's GPT-2 jara
    • BERT atilẹba ti Google
    • Awọn awoṣe ipilẹ Llama 2 Meta
  4. Kini ipa wo ni KL-divergence oro ni GRPO?
    • O ṣe ipilẹṣẹ ẹgbẹ ti awọn ipari
    • O rọpo awoṣe ere
    • O ṣe itọju eto imulo ikẹkọ lati ma lọ jinna si awoṣe itọkasi tutunini
    • O ṣe iṣiro idiyele iye-fun-àmi
  5. Kini idi ti GRPO paapaa wuyi fun awọn awoṣe ero ikẹkọ lori iṣiro tabi koodu?
    • Ko nilo ifihan ere rara
    • O ṣiṣẹ nikan pẹlu awọn aami ayanfẹ eniyan
    • Ko le lo awọn ipari ti a ṣe ayẹwo
    • O ṣiṣẹ nipa ti ara pẹlu ijẹrisi, awọn ere ti o da lori ofin (fun apẹẹrẹ, ṣe ibaamu idahun tabi ṣe awọn idanwo kọja) ti o gba wọle kọja ẹgbẹ kan ti awọn ipinnu ayẹwo
  6. Kini aiṣedeede ti a mọ pe iṣẹ atẹle (bii Dokita GRPO) gbiyanju lati ṣe atunṣe ni fanila GRPO?
    • Iwa lati foju kọ ere naa patapata
    • Gigun ati awọn aibikita iṣoro ti a ṣafihan nipasẹ bii awọn ere ṣe jẹ deede
    • Ailagbara lati ṣe ayẹwo diẹ ẹ sii ju ọkan lọ
    • Ibeere fun nẹtiwọọki alariwisi nla kan
  7. Three GRPO completions for one prompt receive rewards 2, 4, and 6. Which completion has zero centered advantage?
    • The completion rewarded 2
    • The completion rewarded 6
    • The completion rewarded 4
    • All completions because rewards differ
  8. Why compare GRPO reward statistics among completions of the same prompt?
    • They provide a relative baseline for alternative answers to that prompt
    • They require unrelated prompts to have equal difficulty
    • They replace checking whether an answer is correct
    • They make the highest reward identical across all tasks