Nchọpụta ihe dị iche iche
Ndozi ngbanwe dị iche iche na-ewepụta ederede dị elu site n'iwepụ ọchịchọ nke obere asụsụ na-adịghị ike site na nke nnukwu, siri ike.
Nchịkọta
It amplifies what the expert knows and the amateur misses, reducing repetition and bland output.
Ime miri emi
Mgbe ụdị asụsụ na-ahọrọ okwu na-esote, ọ na-ewepụta ihe gbasara omume karịa okwu ya. Ndozi mgbanwe (nke Li et al. webatara na 2022) na-eme ụdị abụọ n'otu ọnọdụ: nnukwu 'ọkachamara' na obere 'amu amu'. Kama ịtụkwasị obi na ihe gbasara nke puru omume nke onye okachamara, ọ na-enyocha akara nke onye ọ bụla n'afọ site na ọdịiche dị n'etiti log-probability nke ọkachamara na onye na-amu amu. Token ndị ọkachamara nwere mmasị mana onye na-amu amu anaghị ebuli elu; mkpụrụokwu ọnyà ma ụdị ịhụnanya (dị ka 'nke'' ma ọ bụ nkebi ahịrịokwu ugboro ugboro) na-ebelata, ebe ọ bụ na onye na-amu amu hụkwara ha n'anya. Ihe nzacha plausibility na-ebu ụzọ tụfuo akara ndị ọkachamara chere na ọ gaghị ekwe omume, yabụ ọdịiche ahụ anaghị akwalite nzuzu. Nsonaazụ a na-ekwu okwu nke ọma, na-emekọrịta ihe, na nke na-adịchaghị emegharị ogologo oge karịa nlele anyaukwu ma ọ bụ nucleus, na-enweghị ọzụzụ ọzọ achọrọ.
Nghọta nka nka
Isi akara bụ log p_expert(token) mwepu ọnụọgụ oge log p_amateur(token). N'ihi na onye na-amu amu na-ekerịta njehie nhazi nke ọkachamara (na-akwado akara ngosi ugboro dị elu, looping, nkwughachi azụ), iwepụ ihe gbasara log-probabilities ya na-akagbu ụdị ọdịda ọdịda ndị ahụ na-echekwa ma na-echekwa ezi ihe ọmụma ọkachamara. Mgbochi ngbanwe na-eme mgbanwe na-edobe naanị akara n'elu obere ntakiri (alpha) nke puru omume ọkachamara kachasị elu, na-egbochi ọdịiche ahụ site n'ịkwalite okwu ndị na-adịghị ahụkebe.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke idebe ihe dị iche
Ndozi ngbanwe dị iche iche agbagoro ezinụlọ nke 'ụzọ dị iche na ntinye', gụnyere DoLa (na-emegide ihe nlere mbụ nke ya n'ụdị oge iji bepụ ihe nhụsianya) yana ụdị dị iche iche maara nke ọma nke dị iche na na enweghị akwụkwọ ewepụtara. Na-atụ anya mwekota siri ike na iweghachite, akara eziokwu, na nhụsianya nke obere onye na-amu amu, gbakwunyere nchikota na nhọpụta ntule ka onye na-amu amu na-eduzi ịdịmma ma na-emewanye ọgbọ n'otu oge.
Mmejuputa n'ezie n'ụwa
Na-emepụta ogologo akụkọ na-adịghị emegharị ma ọ bụ akụkọ na-aga n'ihu ebe nucleus sampling na-abanye n'ime loops.
Ijikọta ọkachamara 65B na onye na-amu amu 1.5B iji kwalite ọgbọ mepere emepe na-enweghị ezigbo nhazi.
Mbelata nkwughachi nkwughachi na nchikota na nsonso mkparịta ụka
Na-eje ozi dị ka ihe ndabere maka ụdị DoLa dị iche iche iji wedata echiche efu
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Contrastive Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nchọpụta ihe ize ndụ Bayes kacha nta
Ajụjụ a na-ajụkarị
What is Contrastive Decoding?
Ndozi ngbanwe dị iche iche na-ewepụta ederede dị elu site n'iwepụ ọchịchọ nke obere asụsụ na-adịghị ike site na nke nnukwu, siri ike. Ọ na-amụba ihe ọkachamara maara na onye na-amu amu na-atụfu, na-ebelata ugboro ugboro na mmepụta adịghị mma.
Na ngbanwe dị iche, kedu ọrụ obere ihe nlere anya 'amu amu' na-arụ?
A na-ewepụ ihe gbasara ndekọ nke onye na-amu amu n'aka ndị ọkachamara, na-akagbu mmejọ usoro (dị ka ịkwado token ugboro ugboro) nke ụdị abụọ ahụ na-ekerịta.
Kedu ihe kpatara ibelata ihe puru omume nke onye na-amu amu ji ebelata nkwughachi na mputa na-adighi nma?
Modelsdị abụọ ahụ na-achọkarị akara ngosi dị elu na ugboro ugboro; iwepụ onye na-amu amu na-ewepụ ajọ mbunobi ahụ, na-eweda nhọrọ ndị na-adịghị mma.
Gịnị ka mmachi plausibility na-eme mgbanwe na-eme?
Ihe nzacha plausibility na-atụfu akara ngosi nwere ike dị ala n'okpuru ọkachamara, yabụ ọdịiche ahụ enweghị ike ịkwalite okwu na-enweghị isi ma ọ bụ enweghị isi.
Kedu ihe bụ usoro akara n'ichepụta ọdịiche?
A na-enweta akara nke onye ọ bụla n'ime ya dị ka ihe gbasara ndekọ ndekọ nke ọkachamara na-ewepụ ihe puru omume nke onye amu amu, akara na-akwụghachi ụgwọ nke ọkachamara masịrị ya.
Kedu usoro nke mechara gbatịa echiche dị iche na ọkwa nke otu ụdị?
DoLa na-atụnyere amụma akaghi aka nke ihe nlereanya (nke mbụ) na tozuru oke (mbubreyo) amụma iji belata nhụsianya, na-etinye echiche dị iche n'ime otu ụdị.