Ingxube Yokujula
Ingxube Yokujula (i-MoD) ivumela isiguquli ukuthi sichithe amanani ahlukene wokubala kumathokheni ahlukene, sihambisa amathokheni 'abalulekile' kuphela ekubalweni esindayo kwesendlalelo ngasinye.
Uhlolojikelele
It cuts the cost of processing easy tokens while keeping a fixed, predictable compute budget.
I-Deep Dive
Ama-transformer ajwayelekile afaka zonke izendlalelo kuwo wonke amathokheni, ngisho neziwubala ezifana nezimpawu zokubhala. Ingxube Yokujula, eyethulwe yi-Google I-DeepMind ngo-2024, yengeza irutha encane kubhulokhi ngayinye ekhetha ingxenye egxilile ye-top-k yamathokheni ukuze ikwazi ukuzinaka ngokugcwele nokubala kwe-MLP; okunye kweqa ibhulokhi ngoxhumano oluyinsalela. Ngenxa yokuthi amathokheni angu-k kuphela acutshungulwa isendlalelo ngasinye, isamba sekhompiyutha (i-FLOPs) simbozwe futhi saziwa kusengaphambili, ngokungafani nezindlela zangaphambili zokujula okuguquguqukayo ezahluka ngokungalindelekile. Lokhu kwenza i-batching kanye nokusetshenziswa kwehadiwe kusebenze kahle. Amamodeli aqeqeshwe yi-MoD angakwazi ukufanisa ikhwalithi yesiguquli esiyisisekelo esebenzisa ama-FLOP ambalwa ngokudlula ukuya phambili, noma afinyelele ikhwalithi ephakeme ngekhompyutha efanayo, futhi umqondo uhlanganiswa ngokwemvelo Ne-Mixture-of-Experts ukunikeza amamodeli e-'MoDE' leyo ndlela kukho kokubili ukujula nobubanzi.
I-Technical Insight
Kubhulokhi ngayinye ye-MoD, irutha yomugqa efundiwe ithola wonke amathokheni futhi igcina i-top-k ngesikolo; amathokheni akhethiwe adlula ukunakwa kanye ne-MLP, kuyilapho amathokheni angakhethiwe aqhutshelwa phambili angashintshiwe yindlela esele. Ukusebenzisa i-top-k engashintshi (kunomkhawulo wethokheni ngayinye) kwenza igrafu yekhompyutha imile futhi imimo ye-tensor ingashintshi, evumelana nehadiwe. Umzila uqeqeshwe nayo yonke inethiwekhi, futhi isizukulwane sembangela sisebenzisa izibikezelo eziyisiza ukuze izinqumo zomzila zingabheki amathokheni azayo.
I-Strategic Impact
Isivinini nesikali
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Finyelela futhi ufinyelele
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Izinqumo ezicacile
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Ikusasa Lengxubevange Yokujula
Ukubala okunemibandela kuyisici esikhulu sokusebenza kahle njengesikali samamodeli, futhi i-MoD iyisibonelo sakuqala, esihlanzekile. Lindela ukuhlanganiswa okujulile Nengxube-Yochwepheshe (umzila kukho kokubili ukujula nochwepheshe), ibhajethi eguquguqukayo eshwabana ukuze uthole okokufaka okulula, namarutha afundiwe abona kangcono ukuthi imaphi amathokheni adinga ngempela ukucutshungulwa okujulile. Njengoba izindleko zokucatshangelwa zibusa ezomnotho zokusatshalaliswa, amasu avumela amamodeli 'acabange kakhudlwana' kuphela lapho kudingeka khona, kuyilapho egcina ukubambezeleka okubikezelwayo, kungenzeka abe indinganiso kuzakhiwo ezinkulu.
Ukuqaliswa Komhlaba Wangempela
Ukunciphisa ama-FLOP adingekayo ukuze kucutshungulwe amadokhumenti amade ngokweqa ukubala okujulile kumathokheni okugcwalisa
Ukuqeqesha imodeli ehambisana nekhwalithi yesisekelo ngekhompyutha ephansi, ukwehlisa izindleko zokuphakela
Ihlanganisa ne-Mixture-of-Experts (I-MoDE) ukuze uhambise kukho kokubili ukujula kwesendlalelo kanye nokukhetha kochwepheshe
Ukugcina ukubambezeleka okubikezelwe, okugxilile ngethokheni ngayinye ngoba ibhajethi yekhompiyutha yesendlalelo ngasinye ilungiswa kusenesikhathi
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixture of Depths quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukuhlanganiswa Kwama-ejenti
Imibuzo evame ukubuzwa
What is Mixture of Depths?
Ingxube Yokujula (i-MoD) ivumela isiguquli ukuthi sichithe amanani ahlukene wokubala kumathokheni ahlukene, sihambisa amathokheni 'abalulekile' kuphela ekubalweni esindayo kwesendlalelo ngasinye. Inciphisa izindleko zokucubungula amathokheni alula kuyilapho igcina ibhajethi yekhompiyutha engaguquki, ebikezelwayo.
Inhlanganisela yokujula ihluka ngani kumathokheni?
Imizila ye-MoD kuphela amathokheni athile ekubalweni esindayo kwesendlalelo ngasinye, ngakho amathokheni ahlukene athola ukujula okuhlukile ngempumelelo.
I-MoD igcina kanjani isabelomali sayo sekhompiyutha sibikezelwa?
Ukukhetha i-top-k egxilile yamathokheni ngebhulokhi ngayinye ivala ama-FLOP futhi kugcina umumo we-tensor ungashintshi, okuvumelana nehadiwe.
Kwenzekani kumathokheni i-router ENGAWUYIkhethi endaweni ethile?
Amathokheni angakhethiwe adlula ukubala kwebhulokhi futhi aqhutshelwa phambili angashintshiwe yindlela eyinsalela.
Ubani owethule Ingxube Yokujula?
Ingxube yokujula yethulwe ngu-Google DeepMind ephepheni lango-2024 lekhompuyutha ye-dynamic transformer.
Kukhiqiza ini ukuhlanganisa i-MoD ne-Mixture-of-Experts?
Ukuhlanganisa umzila ojulile nochwepheshe bomzila kukhiqiza amamodeli we-'MoDE' anesimo sokubala kuzo zombili izendlalelo nochwepheshe.