Ndozi ogologo na nkwalite mmasị
Ogologo ịdị ogologo na-edozi ebumnobi imegharị mmasị ka ụdị kwụsị inwe nkwado naanị site n'ịde ogologo azịza.
Nchịkọta
It matters because uncorrected reward signals push chatbots toward verbose, padded responses instead of genuinely better ones.
Ime miri emi
Mgbe ụdị dabara na ụzọ dị ka RLHF ma ọ bụ DPO, ha na-amụta site na ntụnyere ebe mmadụ (ma ọ bụ ihe atụ ụgwọ ọrụ) họọrọ 'kamma' nke azịza abụọ. Nsogbu na-adịgide adịgide bụ na azịza ogologo oge na-amasịkarị ọbụna mgbe ha na-akaghị mma, yabụ ihe nlereanya ahụ na-amụta ụzọ mkpirisi: bụrụ okwu. Ogologo normalization na-emegide nke a. N'ime DPO ụgwọ ọrụ nke ezoro ezo bụ nchikota nke iche-iche log-token nke ọ bụla, nke na-eto ogologo oge. Ụdị dị iche iche dị ka DPO ogologo-nkịtị na SimPO kesaa ụgwọ ọrụ ahụ site na ọnụ ọgụgụ nke token, na-enye akara na nkezi otu-token kama. Nsonaazụ bụ ụdị ndị na-adị nkenke na n'ebe dị n'elu kama iwelite nzaghachi maka ebumnuche egwuregwu ahụ.
Nghọta nka nka
Ụgwọ ọrụ DPO ezoro ezo bụ log-ratio n'etiti atumatu edeziri na ntụnye aka, chịkọtara n'elu akara ọ bụla na nzaghachi. N'ihi na akara ngosi nke ọ bụla na-agbakwụnye okwu ọzọ (na-adịkarị mma), ụgwọ ọrụ a na-akwụghị ụgwọ na-eji ogologo usoro dị ogologo, na-eleda anya maka mmecha ogologo. SimPO tụfuru ụkpụrụ ntụaka wee jiri nkezi log-probability kwa akara dị ka ụgwọ ọrụ, gbakwunyere oke ụgwọ ọrụ ebumnuche. Nkeji n'ogologo na-ewepu uru ogologo igwe, yabụ gradients mmasị na-egosipụta ogo kama ịgụta okwu.
Mmetụta atụmatụ
Mkpebi doro anya
Ọ na-enyere gị aka ikewapụta nkwupụta ọrụ aka doro anya na asụsụ ahịa.
Ọnụ ego na mmefu ego
Ị nwere ike ịjụ ajụjụ mmejuputa iwu ka mma tupu itinye ego ma ọ bụ oge.
Team na usoro ọrụ
Ndị otu nwere nghọta na-eme ka ngwaahịa, amụma na mkpebi mmụta ka mma.
Ọdịnihu nke nhazi nke ogologo na nkwalite mmasị
Na-atụ anya njikwa ogologo ka ọ bụrụ mkpịsị ọkọlọtọ kama ịtụgharị uche. Ndị na-eme nchọpụta na-ejikọta ogologo ịdị ogologo na ntaramahụhụ ogologo doro anya, ụgwọ ọrụ ogologo, na ụlọ nyocha nke na-ejide ogologo azịza mgbe niile iji tụọ ezigbo uru uru. Ka ụdị ụgwọ ọrụ na-akawanye mma n'ịhụta nkwutọ okwu, pipeline nwere ike na-akọpụta ọnụego mmeri ogologo oge na ndabara, ndị ọrụ ga-enwetakwa njikwa kacha mma n'otú azịza nke ihe nlereanya kwesịrị ịdị.
Mmejuputa n'ezie n'ụwa
Idozi onye na-enyere ndị ahịa aka na SimPO ka ọ na-enye azịza kpụ ọkụ n'ọnụ, nke ziri ezi kama ịdebe paragraf ndị na-ele anya nke ọma.
Ịkọ akụkọ 'ọnụego mmeri ejiri ogologo oge' na AlpacaEval 2 iji gosi ihe nlereanya emelitere n'ezie karịa inwe mkparịta ụka.
Na-agbakwunye ogologo ịdị ogologo na DPO mgbe ọ na-emezigharị usoro koodu ka ọ na-eweghachi obere snippets ziri ezi, ọ bụghị efere mmiri ọkụ.
Na-achọpụta ụdị ụgwọ ọrụ nke na-ahazi ogologo edemede dị elu karịa, wee mebie ya tupu iji ya dozie onye enyemaka ide.
Ihe ize ndụ & okporo ụzọ nche
Otu dị iche iche nwere ike iji otu okwu ahụ mee ihe n'ụzọ dị iche, yabụ kọwapụta oge n'oge.
Ihe nrịbama nwere ike ịdị ike ebe arụmọrụ ụwa na-adaghị adaba.
Ileghara ogo data na atụmatụ nyocha anya na-emepụtakarị nsonaazụ na-adịghị mma.
Map mmejuputa
Malite na nkọwa asụsụ dị larịị nke nsonaazụ ịchọrọ.
Họrọ otu metrik ịga nke ọma na otu ọnọdụ ọdịda tupu nnwale.
Gbaa obere onye na-anya ụgbọ elu nwere data nnọchite anya, ọ bụghị ihe ngosi ngosi na-egbu maramara.
Detuo ebe nzizi ogologo na nkwalite mmasị na-enyere aka yana ebe ụzọ ndị dị mfe dị mma.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Length Normalization in Preference Optimization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nwelite Mmasị Ratio ọdịghị mma
Ajụjụ a na-ajụkarị
What is Length Normalization in Preference Optimization?
Ogologo ịdị ogologo na-edozi ebumnobi imegharị mmasị ka ụdị kwụsị inwe nkwado naanị site n'ịde ogologo azịza. Ọ dị mkpa n'ihi na akara nkwụghachi ụgwọ akwụghị ụgwọ na-akwali chatbots gaa n'okwu verbose, azịza padded kama nke kacha mma.
Kedu omume na-adịghị mma ka ogologo ịdị ogologo na DPO bu n'obi igbochi?
Ụgwọ ọrụ DPO ezoro ezo na-eto site na ọnụ ọgụgụ token, yabụ na-enweghị ụdị nhazigharị, mụta na naanị ikwu okwu ọnụ na-eme ka ị nweta ntụnyere mmasị.
Kedu ihe kpatara ụgwọ ọnwa DPO ọkọlọtọ ji agbasawanye na ogologo nzaghachi?
Ụgwọ ọrụ ahụ ezoro ezo na-achịkọta ọnụ ọgụgụ nke puru omume log n'ofe akara ọ bụla, yabụ ọtụtụ token na-apụtakarị ụgwọ ọrụ ka ukwuu.
Kedu ka SimPO ga-esi lebara anya ogologo oge?
SimPO na-aza nzaghachi site na nkezi ha (ogologo-normalized) log-probability ma na-agbakwunye oke ụgwọ ọrụ ebumnuche, na-ewepụ uru ogologo ọrụ.
Kedu omume nyocha na-enyere aka kwado ihe nlereanya emelitere n'ịdị mma karịa naanị ogologo?
Ọnụego mmeri a na-achịkwa ogologo (dị ka ọ dị na AlpacaEval 2) na-agbanwe maka ogologo azịza ka uru na-egosipụta ịdị mma, ọ bụghị ikwu okwu.