InoteveraGaidhi rinotevera
Odds Ratio Preference Optimization
Mutauro AI
Basics GUIDE
Kureba Kwekugadzirisa muKuda Kugadzirisa
Kureba kwakajairwa kunogadzirisa zvido-tuning zvinangwa kuitira kuti mamodheru arege kuhwina mvumo nekunyora mhinduro refu.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
Izvo zvine basa nekuti masaini emubairo asina kurongeka anosundira chatbots akananga kune verbose, mhinduro dzakatenderedzwa panzvimbo peidzo dziri nani zvechokwadi.
Kudzika Kwakadzika
When models are aligned with comparisons, human or reward-model labelers may prefer longer answers even when the extra text is not better. In standard DPO, the implicit reward sums per-token log-probability differences, so it can grow with response length. SimPO uses average per-token log probability, removes the need for a separate reference model, and adds a target reward margin between preferred and rejected responses. The SimPO paper reports chat-based evaluations on AlpacaEval 2, MT-Bench, and Arena-Hard; these results are tied to its tested models and training setups. Length-normalized objectives can reduce verbosity bias, but aggressive normalization can also encourage under-informative answers. Evaluation should assess completeness and correctness alongside length-controlled preference results.
Kuitwa muNyika yeChokwadi
Kugadzirisa mutengi-rutsigiro mubatsiri neSimPO saka inopa crisp, mhinduro dzakaringana pachinzvimbo chendima dzakapetwa dzinongotaridzika chaizvo.
Kushuma 'kureba-inodzorwa kuhwina mwero' paAlpacaEval 2 kuratidza modhi yakagadziridzwa zvechokwadi pane kungowedzera kutaura.
Kuwedzera kureba kuDPO kana uchinyatso-tuta modhi yekukodha saka inodzosera zvidiki zvakaringana snippets, kwete bloated boilerplate.
Kuongorora mubairo wemodhi iyo yakarongeka inoverengera zvinyorwa zvakareba, wozoibvisa pamberi pekuishandisa kurongedza mubatsiri wekunyora.
Ramangwana Rekureba Kugadziriswa muPreference Optimization
Tarisira kureba kwekutonga kuve chiyero knob pane kungofunga mushure. Vatsvaguri vari kubatanidza kureba kwehurefu nezvirango zvehurefu hwakajeka, mibairo yakarebesa, uye masutu ekuongorora anobata hurefu hwemhinduro nguva dzose kuyera chokwadi chemhando inowanikwa. Sezvo mhando dzemubairo dzichiita zvirinani pakuona verbosity bias, mapaipi ekumisikidza angangotaura kureba-debiased kuhwina mareti nekusarudzika, uye vashandisi vanozowana kutonga kwakanaka pamusoro pekuti dzakapfupika kana dzakadzama mhinduro dzemodhi.
Kuzvishandisa
Nei zvichikosha
- Sarudzo dzakajeka. Inokubatsira kuparadzanisa zvakajeka zvichemo zvehunyanzvi kubva mumutauro wekushambadzira.
- Mutengo uye bhajeti. Iwe unogona kubvunza zvirinani kuita mibvunzo usati washandisa mari kana nguva.
- Team uye workflow. Zvikwata zvine nzwisiso yakagovaniswa inoita zvirinani chigadzirwa, mutemo, uye sarudzo dzekudzidza.
Chenjera
- Zvikwata zvakasiyana zvinogona kushandisa izwi rimwechete zvakasiyana, saka tsanangura nzvimbo nekukurumidza.
- Benchmarks inogona kutaridzika yakasimba nepo chaiyo-yenyika kuita isina kuenzana.
- Kuregeredza mhando yedata uye zvirongwa zvekuongorora zvinowanzogadzira mhedzisiro isina kusimba.
Matanho anotevera
- Tanga netsanangudzo yemutauro wakajeka yemhedzisiro yaunoda.
- Sarudza metric imwe yekubudirira uye imwe yekutadza mamiriro usati waedzwa.
- Mhanya mutyairi mudiki ane data remumiriri, kwete demo rakakwenenzverwa.
- Gwaro uko Kureba Kujairwa muPreference Optimization kunobatsira uye uko nzira dzakareruka dziri nani.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Length Normalization in Preference Optimization quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
Chii chinonzi Length Normalization muPreference Optimization?
Kureba kwakajairwa kunogadzirisa zvido-tuning zvinangwa kuitira kuti mamodheru arege kuhwina mvumo nekunyora mhinduro refu. Izvo zvine basa nekuti masaini emubairo asina kurongeka anosundira chatbots akananga kune verbose, mhinduro dzakatenderedzwa panzvimbo peidzo dziri nani zvechokwadi.
Ndeupi hunhu husingafadzi hunoita kuti kurebesa kurebesa muDPO kunonyanya kuvavarira kudzivirira?
Mubairo wakajeka weDPO unokura nehuwandu hwechiratidzo, saka pasina mamodhiyo emhando yepamusoro dzidza kuti kungoita verbose kunowanzo kuhwina kuenzanisa kwekuda.
Sei mubairo wakakwana weDPO uchiwedzera nekureba kwemhinduro?
Mubairo wakajeka unoverengera log-mukana mareshiyo pachiratidzo chega chega, saka tokeni dzakawanda dzinowanzoreva mubairo wakakura.
SimPO inogadzirisa sei kureba kwekurebesa?
SimPO inowana mhinduro neavhareji yavo (kureba-yakajairwa) log-mukana uye inowedzera mubairo wechinangwa, ichibvisa iyo mechanic kureba mukana.
Ndeipi maitiro ekuongorora anobatsira kusimbisa modhi yakavandudzwa mumhando kwete kungoreba?
Kureba-inodzorwa kuhwina mwero (sepaAlpacaEval 2) inogadzirisa kureba kwemhinduro kuitira kuti mibairo iratidze mhando, kwete verbosity.
Which chat-based benchmarks did the SimPO paper use in its reported evaluations?
The SimPO paper reports chat-based evaluations on AlpacaEval 2, MT-Bench, and Arena-Hard; these benchmark results apply to the tested models and setups.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi