Ikibazo gihuza ibibazo · Hagati Urwego

Ikibazo cyo Kwerekana Ingero

Ikizamini cyo guhemba-icyitegererezo amanota, amakuru yatoranijwe, gutakaza amanota abiri, gutakaza ibihembo, no gukoresha-N gukoresha muri RLHF. Reba ibitekerezo n'ibipimo.

Inzira zijyanye no kuyoboraKwerekana Ingororano
Ikibazo 1 Bya 8

Nibihe bisohoka byibanze byigihembo cyicyitegererezo cyatanzwe-igisubizo?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Nibihe bisohoka byibanze byigihembo cyicyitegererezo cyatanzwe-igisubizo?
    • Amanota imwe ya scalar
    • Igisubizo cyuzuye
    • Ikwirakwizwa rishoboka hejuru yamagambo
    • Urutonde rwibimenyetso byatoranijwe
  2. What feedback format did the InstructGPT paper collect to train its reward model?
    • A guaranteed correctness label for every possible answer
    • Human rankings of several model responses, converted into pairwise preferences
    • A single sentiment score for each training prompt
    • Only binary safety labels with no response comparisons
  3. Niki igihombo gisanzwe-icyitegererezo cyatakaye?
    • Igihembo cyuzuye cya buri gisubizo
    • Ibikurikira-ibimenyetso byerekana neza
    • Ko igisubizo cyatoranijwe gifite amanota arenze ayo yanze
    • Uburebure bwibisubizo
  4. 'Kwiba ibihembo' ni iki?
    • Kwiba icyitegererezo cyigihembo cyumunywanyi
    • Uburyo bwo kwihutisha amahugurwa
    • Guhisha ibimenyetso byigihembo
    • Politiki ikoresha inenge muburyo bwo guhemba kugirango ibone amanota menshi idafite ireme ryukuri
  5. Nigute icyitegererezo cyigihembo cyubatswe muburyo bw'ururimi?
    • Mugusimbuza token-guhanura umutwe hamwe na scalar imwe isohoka umutwe
    • Mugushyiramo ibitekerezo byinshi
    • Mugukuraho ibice byose bya transformateur
    • Mugukuba kabiri amagambo
  6. Nigute icyitegererezo cyo guhemba gikoreshwa mugihe cyo gufata umwanzuro muburyo bwiza-bwa N?
    • Amahugurwa N ibihembo byicyitegererezo
    • Kubyara ibisubizo byinshi no gukoresha ibihembo byigihembo kugirango uhitemo amanota menshi
    • Guhitamo N ibimenyetso bisanzwe
    • Kugereranya N icyitegererezo cyicyitegererezo
  7. Why might an RLHF study use an ensemble of reward models during training?
    • To guarantee that reward hacking is impossible
    • To replace the human preference data with exact correctness labels
    • To estimate disagreement or uncertainty across learned preference scorers
    • To ensure every candidate receives the same score
  8. Why should reward-model evaluations include held-out comparisons?
    • To ensure training comparisons are never used
    • To make every output identical
    • To guarantee the reward is unbiased
    • To test whether preferences generalize beyond the training comparisons