I-Contrastive Decoding
Ukuhlukanisa amakhodi kwakha umbhalo wekhwalithi ephezulu ngokususa ukuthambekela kwemodeli yolimi oluncane, olubuthakathaka kuleso esikhulu, esiqinile.
Uhlolojikelele
It amplifies what the expert knows and the amateur misses, reducing repetition and bland output.
I-Deep Dive
Uma imodeli yolimi ikhetha igama elilandelayo, iveza amathuba phezu kwamagama ayo. Ukukhipha amakhodi okuqhathanisayo (okwethulwa ngu-Li et al. ngo-2022) kusebenzisa amamodeli amabili kumongo ofanayo: 'uchwepheshe' omkhulu kanye 'nomfundamakhwela' omncane. Esikhundleni sokuthembela okungenzeka ukuthi uchwepheshe, ithola ithokheni ngayinye yekhandidethi ngomehluko phakathi kwamathuba okungena ochwepheshe kanye nemfundamakhwela. Amathokheni uchwepheshe athanda kodwa i-amateur ayikhushulwa; amagama ajwayelekile womabili amamodeli ayawathanda (njengokuthi 'the' noma imishwana ephindaphindwayo) ayacindezelwa, njengoba umfundamakhwela ewathanda nawo. Isihlungi esizwakalayo siqala silahla amathokheni uchwepheshe acabanga ukuthi mancane kakhulu, ngakho umehluko awukhuthazi umbhedo. Umphumela uba umbhalo oshelelayo, ohambisanayo, futhi ongaphindaphindi kakhulu kunesampula esihahayo noma se-nucleus, ngaphandle kokuqeqeshwa okwengeziwe okudingekayo.
I-Technical Insight
Iphuzu eliyinhloko ilogu elithi p_expert(ithokheni) khipha i-coefficient yezikhathi zokungena ethi p_amateur(ithokheni). Ngoba umfundamakhwela wabelana ngamaphutha esistimu kachwepheshe (ukuthanda amathokheni emvamisa ephezulu, ukujiya, ukuphindaphinda okuwohlokayo), ukususa amathuba okungena kwelogi kukhansela lezo zindlela zokwehluleka okwabiwe kuyilapho kugcinwa ulwazi lwangempela lochwepheshe. Umkhawulo wokuba nokwenzeka ovumelana nezimo ugcina kuphela amathokheni ngaphezu kwengxenye (alpha) yamathuba aphezulu ochwepheshe, okuvimbela ukugqama ekukhuliseni amagama ayivelakancane, angahlangani.
I-Strategic Impact
Isivinini nesikali
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Finyelela futhi ufinyelele
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Izinqumo ezicacile
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Ikusasa Lokuqanjwa Ngokuphikisanayo
Ukukhipha amakhodi okuqhathanisayo kugqugquzele umndeni wezindlela 'zokuqhathaniseka ekuqondeni', okuhlanganisa i-DoLa (eqhathanisa izendlalelo zokuqala zemodeli nezakamuva ukuze usike ukubona izinto ezingekho) kanye nezinhlobonhlobo eziqaphela umongo ezingafani namadokhumenti abuyisiwe nangaphandle kwawo. Lindela ukuhlanganiswa okuqinile nokubuyiswa, amaphuzu weqiniso, kanye ne-small-amateur distillation, kanye nokuhlanganiswa nokuqopha okucatshangelwayo ukuze umuntu ofunda esikoleni aqondise ikhwalithi futhi asheshise ukukhiqiza ngesikhathi esisodwa.
Ukuqaliswa Komhlaba Wangempela
Ikhiqiza indaba ende, engaphindaphindi noma ukuqhubeka kwe-athikili lapho isampula ye-nucleus ikhukhuleka ibe yizihibe
Ukumatanisa uchwepheshe we-65B nomuntu oyimfundamakhwela we-1.5B ukuthuthukisa isizukulwane esivulekile ngaphandle kokulungisa kahle
Ukunciphisa ukuphindaphinda okonakele ekufinyezweni nasekuphumeni kwengxoxo
Isebenza njengesisekelo sokuziqhathanisa kwesitayela se-DoLa ukuze kwehliswe imibono engelona iqiniso
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Contrastive Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ubuncane be-Bayes Risk Decoding
Imibuzo evame ukubuzwa
What is Contrastive Decoding?
Ukuhlukanisa amakhodi kwakha umbhalo wekhwalithi ephezulu ngokususa ukuthambekela kwemodeli yolimi oluncane, olubuthakathaka kuleso esikhulu, esiqinile. Ikhulisa lokho okwaziyo uchwepheshe kanye nemfundamakhwela egejayo, inciphisa ukuphindaphinda nokukhishwayo okuphuphile.
Ekuhlukaniseni amakhodi, iyiphi indima edlalwa imodeli encane 'yemfundamakhwela'?
Amathuba okungena kwemfundamakhwela asuswa kochwepheshe, ukukhansela amaphutha esistimu (njengokuthanda amathokheni avamile) lawo womabili amamodeli abelana ngawo.
Kungani ukususa amathuba e-ateur kunciphisa ukuphindaphinda nokukhishwa okuphuphile?
Womabili amamodeli athambekele ekukhetheni amathokheni aphezulu-frequency kanye nokuphindaphinda; ukususa ongamfundanga mkhuba kususa lokho kubandlulula okwabiwe, kwehlisa ukukhetha okuphuphile.
Yenzani ukucindezela kokuvumelana okuguquguqukayo?
Isihlungi esizwakalayo silahla amathokheni amathuba aphansi ngaphansi kochwepheshe, ngakho ukugqama akukwazi ukuphromotha amagama angahambisani noma angenangqondo.
Cishe ithini ifomula yokunikeza amaphuzu ekubhalweni okuhlukile?
Ikhandidethi ngalinye linikezwa amaphuzu njengamathuba elogi yengcweti kukhishwe amathuba okulogi angenayo isisindo, amathokheni avuzayo akhethwa uchwepheshe ngokukhethekile.
Iyiphi indlela yakamuva enweba umqondo ophambene uye ezendlalelo zemodeli eyodwa?
I-DoLa iqhathanisa ukuqagela kwangaphambi kwesikhathi kwemodeli (ungqimba lwangaphambi kwesikhathi) kanye nezibikezelo ezivuthiwe (zesendlalelo sekwephuzile) ukuze kuncishiswe ukubona izinto ezingekho, kusetshenziswa umqondo wokugqama phakathi kwemodeli eyodwa.