Ikibazo gihuza ibibazo · Biragoye Urwego

Gushimangira Kwiga Biturutse kubitekerezo byabantu

Witoze amakuru ya RLHF, ibihembo byigihembo, hamwe nibisanzwe, harimo n'impamvu imanza zidakwiye kwemeza ukuri.

Ikibazo 1 Bya 8

Niyihe ntego nyamukuru ya RLHF kurugero rwururimi?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Niyihe ntego nyamukuru ya RLHF kurugero rwururimi?
    • Guhuza ibisubizo byayo nibyifuzo byabantu kugirango bafashe numutekano
    • Kugirango wongere icyitegererezo cyibara
    • Gucomeka icyitegererezo kugirango byihuse
    • Guhindura icyitegererezo mu zindi ndimi
  2. Ni ubuhe buryo bwo guhemba muri RLHF bwiga gukora?
    • Kora ibisubizo byanyuma kubakoresha
    • Hindura ibisobanuro mumafaranga yigihembo
    • Shiraho amanota ya scalar yerekana uburyo abantu bahitamo igisubizo runaka
    • Hitamo igipimo cyo kwiga
  3. What does a KL-divergence penalty against a reference policy aim to regularize in an RLHF objective?
    • Every factual error with a complete correctness guarantee
    • How far the optimized policy departs from the reference
    • The tokenizer vocabulary’s spelling
    • The number of human annotators automatically
  4. Nigute amakuru yatoranijwe yakusanyirijwe muburyo bwo guhemba?
    • Abantu bandika ibisubizo bishya uhereye kubusa
    • Icyitegererezo cyerekana icyizere cyacyo
    • Ibirango bisanzwe byashizweho mu buryo bwikora
    • Abantu bagereranya ibisubizo bibiri byicyitegererezo kubibazo bimwe hanyuma bagahitamo icyiza
  5. Ni izihe nyungu DPO (Optimisation itaziguye) itanga hejuru ya RLHF ya kera?
    • Ntabwo bisaba amakuru na gato
    • Iremeza ko icyitegererezo kidashobora na rimwe kwibeshya
    • Ihindura politiki muburyo butaziguye kubiri, gusimbuka ibihembo bitandukanye hamwe na PPO loop
    • Bikuraho gukenera kwitegura
  6. Muri RLAIF na AI Itegekonshinga, ibitekerezo byinshi biva he?
    • Uhereye kuri moderi ya AI itanga cyangwa kunegura ibisubizo, kugabanya ibirango byabantu
    • Gusa uhereye kubantu bahembwa abantu
    • Kuva kumashanyarazi atunguranye
    • Kuva kuri moderi yo kwitegura corpus itaziguye
  7. An RLHF response receives a high learned reward but contains a false claim. What does this show?
    • High reward proves the claim is true
    • The response must have been generated without tokens
    • The reference model’s vocabulary disappeared
    • A preference-based reward is a proxy and can be mistaken
  8. Why should an RLHF team examine whose preference comparisons were collected?
    • Annotator identity determines the GPU clock rate
    • The judgments and task distribution shape what the model is rewarded for
    • Every population has identical response preferences
    • Preference data affect only file compression