Maswali yanayounganishwa na mwongozo · Kati Kiwango

Maswali ya Kuboresha Upendeleo wa Moja kwa Moja

Kagua jozi za mapendeleo ya DPO, sera ya marejeleo, zawadi kamili, beta, vibadala na vikomo vya mafunzo ya upendeleo wa moja kwa moja.

Njia za mwongozo zinazohusianaUboreshaji wa Mapendeleo ya Moja kwa moja
Swali 1 ya 8

Je, DPO huondoa nini ikilinganishwa na RLHF ya jadi?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. Je, DPO huondoa nini ikilinganishwa na RLHF ya jadi?
    • Muundo tofauti wa zawadi na kitanzi cha mafunzo ya kuimarisha
    • Haja ya data yoyote ya upendeleo wa mwanadamu
    • Muundo wa lugha uliofunzwa awali
    • Matumizi ya uwezekano wa logi
  2. Mfano mmoja wa mafunzo ya DPO una data gani?
    • Alama ya zawadi ya haraka na nambari
    • Kidokezo chenye jibu lililochaguliwa na lililokataliwa
    • Jibu la haraka na moja sahihi
    • Toleo la mfano wa zawadi pekee
  3. Mfano wa marejeleo una jukumu gani katika DPO?
    • Hutoa majibu yaliyokataliwa
    • Hupata majibu kwa zawadi dhahiri
    • Ni msingi uliogandishwa ambao sera inapimwa ili kupunguza mteremko
    • Inachukua nafasi ya data ya mafunzo
  4. How does the DPO loss use a labeled preference pair?
    • It regresses both responses to the same scalar target
    • It trains a separate value critic to predict token-level returns
    • It maximizes the rejected response without a reference
    • It increases the chosen response’s policy-versus-reference log-ratio relative to the rejected response
  5. Je, kigezo cha beta (joto) katika DPO kinadhibiti nini?
    • Jinsi sera inavyowekwa karibu na muundo wa marejeleo
    • Kiwango cha kujifunza cha kiboreshaji
    • Idadi ya jozi za upendeleo
    • Saizi ya msamiati
  6. Which alignment objective can learn from binary feedback that an output is desirable or undesirable?
    • PPO, which requires a value critic for every label
    • KTO, a prospect-theoretic objective for binary desirability feedback
    • DARTS, which searches differentiable network cells
    • Beam search, which changes model weights from rankings
  7. In a standard DPO preference example, which response is preferred?
    • The response with more tokens by definition
    • The response with the lowest reference likelihood
    • The response selected by the preference labeler for the given prompt
    • A random response unrelated to the prompt
  8. In DPO, what expression serves as the implicit reward for a response, up to an input-dependent constant?
    • A separately trained scalar reward-model head
    • A human-written point score for each token
    • The average sequence length of preference data
    • Beta times the policy/reference log-probability ratio