Up tókànItọsọna atẹle
Iṣapeye Ipin Ipin Awọn aidọgba
Èdè AI
Awọn ipilẹ Itọsọna
Iṣe deede Gigun ni Iṣapeye Iyanfẹ
Iṣe deede gigun n ṣatunṣe awọn ibi-afẹde-tuntunwọn ki awọn awoṣe dẹkun gbigba ifọwọsi nikan nipa kikọ awọn idahun to gun.
Gbọ si itọsọna yiiAI-ṣe afihan · ~2 min
Lori iwe yi2 min ka
Akopọ
O ṣe pataki nitori awọn ifihan agbara ẹbun ti a ko ṣe atunṣe Titari awọn chatbots si verbose, awọn idahun padded dipo awọn ti o dara julọ.
Jin Dive
When models are aligned with comparisons, human or reward-model labelers may prefer longer answers even when the extra text is not better. In standard DPO, the implicit reward sums per-token log-probability differences, so it can grow with response length. SimPO uses average per-token log probability, removes the need for a separate reference model, and adds a target reward margin between preferred and rejected responses. The SimPO paper reports chat-based evaluations on AlpacaEval 2, MT-Bench, and Arena-Hard; these results are tied to its tested models and training setups. Length-normalized objectives can reduce verbosity bias, but aggressive normalization can also encourage under-informative answers. Evaluation should assess completeness and correctness alongside length-controlled preference results.
Real-World imuse
Ṣiṣatunṣe oluranlọwọ atilẹyin alabara pẹlu SimPO nitorinaa o funni ni agaran, awọn idahun deede dipo awọn oju-iwe padded ti o kan wo ni kikun.
Ijabọ 'oṣuwọn win iṣakoso gigun' lori AlpacaEval 2 lati ṣafihan awoṣe ti ilọsiwaju nitootọ ju ki o kan ni iwiregbe.
Ṣafikun deede ipari gigun si DPO nigba titọ-tunse awoṣe ifaminsi kan nitoribẹẹ o da awọn snippets to peye pada, kii ṣe igbomikana bloated.
Ṣiṣayẹwo awoṣe ẹsan kan ti o ṣe iṣiro awọn arosọ gigun ni ọna ti o ga julọ, lẹhinna aibikita rẹ ṣaaju lilo rẹ lati ṣe deede oluranlọwọ kikọ.
Ojo iwaju ti Iṣe deede Gigun ni Iṣapeye Iyanfẹ
Reti iṣakoso gigun lati di koko boṣewa kuku ju ironu lẹhin. Awọn oniwadi n ṣajọpọ deede gigun gigun pẹlu awọn ijiya gigun ti o fojuhan, awọn ere ipari-ipari, ati awọn suites igbelewọn ti o mu gigun ipari idahun mu igbagbogbo lati wiwọn awọn anfani didara otitọ. Bii awọn awoṣe ere ṣe dara julọ ni iranran ojuṣaaju iṣojuuwọn, awọn opo gigun titotọ yoo ṣe ijabọ awọn oṣuwọn bori-ipari nipasẹ aiyipada, ati pe awọn olumulo yoo ni iṣakoso ti o dara julọ lori bii terse tabi ṣe alaye awọn idahun awoṣe yẹ ki o jẹ.
Fifi o sinu iwa
Kini idi ti o ṣe pataki
- Awọn ipinnu diẹ sii. O ṣe iranlọwọ fun ọ lati ya sọtọ awọn iṣeduro imọ-ẹrọ lati ede tita.
- Iye owo ati isuna. O le beere awọn ibeere imuse to dara julọ ṣaaju lilo owo tabi akoko.
- Ẹgbẹ ati ṣiṣan iṣẹ. Awọn ẹgbẹ pẹlu oye pinpin ṣe ọja to dara julọ, eto imulo, ati awọn ipinnu ikẹkọ.
Ṣọra fun
- Awọn ẹgbẹ oriṣiriṣi le lo ọrọ kanna ni oriṣiriṣi, nitorinaa ṣalaye iwọn ni kutukutu.
- Awọn aṣepari le wo lagbara lakoko ti iṣẹ-aye gidi ko ṣe deede.
- Aibikita didara data ati awọn ero igbelewọn nigbagbogbo ṣẹda awọn abajade ẹlẹgẹ.
Next awọn igbesẹ
- Bẹrẹ pẹlu itumọ-ede itele ti abajade ti o nilo.
- Mu metiriki aṣeyọri kan ati ipo ikuna kan ṣaaju idanwo.
- Ṣiṣe awakọ kekere kan pẹlu data aṣoju, kii ṣe eto demo didan.
- Iwe-ipamọ nibiti Iṣe deede Gigun ni Iṣapeye Ifẹ ṣe iranlọwọ ati nibiti awọn ọna ti o rọrun dara julọ.
Tesiwaju Ṣiṣawari
Ìwé ìròyìn ọfẹ
Gba ìkànìyàn AI ojoojúmọ́
Ìtàn AI mẹta tó jẹ́rìí ní gbogbo owurọ́ ọjọ́ iṣẹ́, kọ́wé ní Gẹ̀ẹ́sì tó rọrùn. Ọfẹ́ fún gbogbo àkókò, kò sí ìpolówó.
Ìmeèlì kan ní gbogbo ọjọ́ iṣẹ́. Yọkúrò ní ìkànìyàn ní ìkànìyàn kan. A kò ta tàbí pín àdírẹ́sì rẹ.
Ṣe idanwo fun ara rẹ
Take the Length Normalization in Preference Optimization quiz
Ìdáhùn lẹ́sẹ̀kẹsẹ̀ lórí gbogbo ìdáhùn, àti iwe-ẹ̀rí tó lè pin pẹ̀lú ID tó lè jẹ́rìí lẹ́yìn tí o bá kọ́ ẹ̀kọ́.
Ṣe àtìlẹ́yìn ẹ̀kọ́ AI ọfẹ́. AI Understanding jẹ́ àjọ àìní èrè 501(c)(3) — kò sí ìpolówó, kò sí àkúnya owó, ní gbogbo àkókò. Ṣe ẹbun kan
Awọn ibeere ti a beere nigbagbogbo
Kí ni Length Normalization nínú Ààyò Optimization?
Iṣe deede gigun n ṣatunṣe awọn ibi-afẹde-tuntunwọn ki awọn awoṣe dẹkun gbigba ifọwọsi nikan nipa kikọ awọn idahun to gun. O ṣe pataki nitori awọn ifihan agbara ere ti ko ni atunṣe Titari awọn iwiregbe si ọna ọrọ-ọrọ, awọn idahun fifẹ dipo awọn ti o dara julọ nitootọ.
Iwa aifẹ wo ni isọdọtun gigun ni DPO ni akọkọ ṣe ifọkansi lati ṣe idiwọ?
Ẹsan alaiṣedeede DPO dagba pẹlu kika ami, nitorinaa laisi awọn awoṣe deede kọ ẹkọ pe nirọrun jijẹ ọrọ-ọrọ diẹ sii duro lati ṣẹgun awọn afiwera ayanfẹ.
Kini idi ti ẹsan alaiṣedeede DPO maa n pọ si pẹlu ipari esi?
Ẹsan ti ko tọ ni akopọ awọn ipin iṣeeṣe log-iṣeeṣe kọja gbogbo aami, nitorinaa awọn ami diẹ sii ni gbogbogbo tumọ si ere lapapọ lapapọ.
Bawo ni SimPO ṣe koju irẹjẹ gigun?
Awọn idahun SimPO nipasẹ apapọ wọn (ipari-deede) iṣeeṣe log-iṣeeṣe ati ṣafikun ala ẹsan ibi-afẹde kan, yiyọ anfani gigun ẹrọ kuro.
Iṣe igbelewọn wo ni o ṣe iranlọwọ jẹrisi awoṣe ti o ni ilọsiwaju ni didara ju gigun nikan lọ?
Oṣuwọn win iṣakoso gigun (bii AlpacaEval 2) ṣatunṣe fun ipari idahun nitorina awọn anfani ṣe afihan didara, kii ṣe ọrọ-ọrọ.
Which chat-based benchmarks did the SimPO paper use in its reported evaluations?
The SimPO paper reports chat-based evaluations on AlpacaEval 2, MT-Bench, and Arena-Hard; these benchmark results apply to the tested models and setups.
Tesiwaju kikọ
Jẹmọ awọn itọsọna
Awọn itọsọna diẹ sii ti a yan fun koko yii