Guide-yakabatana mibvunzo · Pakati Nhanho

Iterative DPO uye Online Preference Tuning Quiz

Dzidza kuti iterative DPO inozorodza sei data yekuda, inoshandisa vatongi, uye inodzokororwa inodzokororwa kutenderera ichipesana nemutengo wakawedzerwa uye kusarura.

Nzira dzekutungamira dzinoenderanaIterative Dpo Uye Online Preference Tuning
Mubvunzo 1 ye 8

Chii chinodzivirira DPO icho chinyakare RLHF (PPO) chinoda?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Chii chinodzivirira DPO icho chinyakare RLHF (PPO) chinoda?
    • Kudzidzira mumwe wakajeka mubairo modhi
    • Chero data yekudzidziswa
    • Kushandisa neural network
    • Zvido zvevanhu zvachose
  2. Nei iterative (pamhepo) DPO kazhinji iri nani pane kumhanya DPO kamwe chete pane yakatarwa dataset?
    • Hazvidi mavara ekuda
    • Iyo inochengeta data rekudzidzisa pa-policy uye nyowani sezvo modhi inochinja
    • Inobvisa referensi modhi
    • Inoshanda chete kumifananidzo
  3. Ndeipi basa rinoitwa nereferenzi modhi mukurasikirwa kweDPO?
    • Iyo inoburitsa yekupedzisira mushandisi kuburitsa
    • Inonyora zvaunofarira
    • Inosimbisa mutemo kuburikidza ne-log-probability ratio, inodzora kutsauka
    • Inotsiva iyo tokenizer
  4. Mune imwechete iteration yeDPO yepamhepo, chii chakajairika kutevedzana?
    • Bvisa modhi, wozodzidzira kubva pakutanga
    • Chengetedza zvekare data rekare pasina sampling
    • Dzidzisa CNN pane mhinduro
    • Enzanisa mhinduro nyowani, zvido zvemutongi, wobva wamhanyisa DPO yekuvandudza
  5. Chii chinonzi beta hyperparameter muDPO kutonga?
    • Iyo mutemo inotsauka zvakanyanya sei kubva kune referenzi modhi
    • Saizi yemazwi
    • Nhamba yeGPUs
    • Chisarudzo chemufananidzo
  6. Ndeipi pfungwa yakakosha kuseri kwe 'kuzvipa mubairo' iterative DPO?
    • Iyo modhi haishandisi zvaunofarira zvachose
    • Iyo modhi inogadzira uye inotonga yayo yega mhinduro pairi kugadzira data rekudzidzisa
    • Vanhu vanofanira kunyora mbiri imwe neimwe
    • Inongoshandisa mibairo isina kurongeka
  7. Who can label response preferences in an online DPO iteration?
    • Only the tokenizer
    • Only the reference policy’s embedding layer
    • Human raters or an AI judge such as a reward model
    • A discriminator that never sees responses
  8. What additional work does iterative DPO require compared with using a fixed preference dataset once?
    • Training no model parameters
    • Removing all evaluation from the pipeline
    • Storing every possible response in advance
    • Repeated response generation and preference labeling between updates