Dekodaas buy wuutale
Dekodaas bu wuute dafay defar mbind mu gëna baax ndax dafay dindi tendaas yu modelu làkk bu ndaw te néew doole ci yu yaatu te am doole.
Résumé
It amplifies what the expert knows and the amateur misses, reducing repetition and bland output.
Plongeur bu xóot
Su benn modelu làkk tànnee baat bi ci topp, dafay defar probabilite ci vocabulaire bi. Decodage contrastif (bi Li et al. dugal ci 2022) dafay doxal ñaari xeetu mbir ci benn muy: 'expert' bu mag ak 'amateur' bu ndaw. Duñu wóolu probabilite yu eksper bi, waaye dafay jox poñ ci token bu nekk ci wuute gi nekk ci diggante probabilite log eksper bi ak bu amateur bi. Jetons yi expert yi bëgg waaye amateur yi duñu am doole; baat generic ñaari model yi bëgg (lu melni 'the' wala mbind yu ñuy baamtu) dañuy supprimé, ndax amateur bi itam bëgg na leen. Filtre plausibilité dafay njëkka sànni token yi eksper bi jàpp ni duñu am, kon wuute gi du musa fësal lu ñàkk njariñ. Lépp soo ko boolee mu gëna neex, gëna dëppoo, te gëna néew luñu koy baamtu, moo gën sampling greedy wala nucleus, te soxlawul beneen tàggat.
Gis-gis xarala
Njortu bu mag bi mooy log p_expert (jeton) dindi ci koeffisien yoon log p_amateur (jeton). Ndax amateur bi dafa bokk njuumti sistemik yu eksper bi (gëna bëgg ay token yu bari, looping, baamtu bu baaxul), dindi log-probabilité yi dafay fomm xeetu njuumte yooyu ñu bokk, boole ci denc xam-xam eksper bi dëgg. Benn plausibilité buy méngoo dafay tëye ay jeton yu ëpp benn fraction (alpha) ci probabilite expert bi gëna mag, di tere contraste bi yokk kàddu yu bari te wuute.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu dekodaas bu wuute
Decodage contrastif inspiré na famiy 'contrast ci inference' pexe, lu ci melni DoLa (kontras model bi teel ak mujjentalu layers ngir dagg hallucination) ak xeetu context-aware yu wuute ak ak amul këyitu jël. Xaarandil lëkkaloo bu gëna dëgër ak seetlu, joxe poñ ci dëgg, ak distilaasioŋ bu ndaw-amateur, boole ci boole ak dekodaas speculatif suko defee amateur bi di doxal kalite bi ba noppi gaaw defar benn yoon.
Doxal ci àdduna dëgg
Defar jaar-jaar yu gudd te duñu baamtu wala wéyal xët fu sampling nucleus di dem ci ay boucles
boole ab expert 65B ak ab amateur 1.5B ngir gëna suqaliku
Wàññi baamtu degenerate ci résumé ak ci njariñu waxtaan
Di nekk royuwaay ci DoLa-style self-contrast ngir wàññi gis-gis yu dëggu yi
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Contrastive Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Dekodage Risk Bayes bi gëna ndaw
Laaj yi ñuy faral di laaj
What is Contrastive Decoding?
Dekodaas bu wuute dafay defar mbind mu gëna baax ndax dafay dindi tendaas yu modelu làkk bu ndaw te néew doole ci yu yaatu te am doole. Dafay yokk li eksper bi xam ak li amateur bi namm, wàññi baamtu ak génne gu neexul.
Ci dekodaas bu wuute, ban cër la xeetu 'amateur' bu ndaw bi di def?
Log-probabilités amateur yi dañu leen di dindi ci expert yi, ñu fomm njuumti sistemik yi (lu melni bëgg token yu bari) yu ñaari model yi bokk.
Lan moo waral dindi probabilite amateur yi di wàññi baamtu yi ak génne gu neex gi?
Ñari xeetu mbir yooyu dañu taamu jeton yu bari frequency ak yuy baamtu; dindi amateur bi dafay dindi bët bu ñu bokk, di wàññi tànneef yu baaxul yi.
Lan mooy tënk plausibilite biy méngoo?
Filtre plausibilité bi dafay sànni token yu am probabilite yu néew ci suufu eksper bi, kon kontrast bi mënul fësal kàddu yu amul benn njariñ wala yu amul njariñ.
Ci gàttal lan mooy formul biy joxe poñ ci dekodaas bu wuute?
Kandidaa bu nekk ñu ngi koy jox poñ ci probabilite log eksper bi dindi ci probabilite log amateur biñ pondéree, ñuy neexal token yi eksper bi taamu.
Ban njuréef ci ginaaw mooy yokk xalaat bu wuute bi ci benn model boppam?
DoLa dafay wuutale xeetu xam-xam bu teel (couche bu njëkk) ak bu màgget (couche bu mujj) ngir wàññi hallucination, jëfandikoo konsepti wuutale ci benn model.