Ikibazo gihuza ibibazo · Hagati Urwego

Ikibazo Cyibanze Cyiza

Ongera usuzume DPO ikunda, politiki yerekana, ibihembo bitaziguye, beta, ibitandukanye, nimbibi zamahugurwa akunda.

Inzira zijyanye no kuyoboraGuhitamo neza
Ikibazo 1 Bya 8

Niki DPO ikuraho ugereranije na RLHF gakondo?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Niki DPO ikuraho ugereranije na RLHF gakondo?
    • Icyitegererezo cyigihembo cyihariye hamwe no gushimangira imyigire
    • Gukenera amakuru ayo ari yo yose akunda abantu
    • Urufatiro rwateganijwe mbere yururimi
    • Gukoresha log-bishoboka
  2. Ni ayahe makuru urugero rumwe rwa DPO rugizwe?
    • A amanota hamwe numubare wigihembo
    • Ikibazo hamwe nicyatoranijwe kandi igisubizo cyanze
    • Igisubizo nigisubizo kimwe cyukuri
    • Gusa ibihembo by'icyitegererezo
  3. Ni uruhe ruhare icyitegererezo cyerekana muri DPO?
    • Itanga ibisubizo byanze
    • Itanga ibisubizo hamwe nibihembo bigaragara
    • Nibice byahagaritswe politiki yapimwe kugirango igabanye drift
    • Isimbuza amakuru yamahugurwa
  4. How does the DPO loss use a labeled preference pair?
    • It regresses both responses to the same scalar target
    • It trains a separate value critic to predict token-level returns
    • It maximizes the rejected response without a reference
    • It increases the chosen response’s policy-versus-reference log-ratio relative to the rejected response
  5. Niki ibipimo bya beta (ubushyuhe) muri DPO igenzura?
    • Nigute politiki ikomeza kuba hafi yicyitegererezo
    • Igipimo cyo kwiga cya optimizer
    • Umubare wibyifuzo byombi
    • Ingano yamagambo
  6. Which alignment objective can learn from binary feedback that an output is desirable or undesirable?
    • PPO, which requires a value critic for every label
    • KTO, a prospect-theoretic objective for binary desirability feedback
    • DARTS, which searches differentiable network cells
    • Beam search, which changes model weights from rankings
  7. In a standard DPO preference example, which response is preferred?
    • The response with more tokens by definition
    • The response with the lowest reference likelihood
    • The response selected by the preference labeler for the given prompt
    • A random response unrelated to the prompt
  8. In DPO, what expression serves as the implicit reward for a response, up to an input-dependent constant?
    • A separately trained scalar reward-model head
    • A human-written point score for each token
    • The average sequence length of preference data
    • Beta times the policy/reference log-probability ratio