Cakudawar Masana
Cakudar Kwararru (MoE) ƙirar ƙira ce wacce ke raba hanyar sadarwa zuwa manyan cibiyoyin sadarwa na musamman kuma tana kunna kaɗan ne kawai a kowace shigarwa.
Dubawa
It lets models hold enormous knowledge while keeping each prediction fast and cheap.
Zurfafa nutsewa
Ma'auni mai canzawa yana gudanar da kowane shigarwa ta cikin yadudduka masu yawa iri ɗaya, don haka yin ƙirar mafi wayo yawanci yana nufin sanya kowane ƙididdiga ya fi tsada. Cakudar Masana ta karya wannan hanyar. Yana maye gurbin babban layin gaba da ciyarwa tare da ƙwararrun cibiyoyin sadarwa da yawa tare da ƙaramin 'na'ura mai ba da hanya tsakanin hanyoyin sadarwa' wanda ke yanke shawarar wane ƙwararrun ke sarrafa kowace alama. Yawanci ƙwararrun 1 ko 2 ne kawai ke yin wuta, don haka ƙirar zata iya samun ɗaruruwan biliyoyin jimillar sigogi amma kawai kunna ƙaramin juzu'i kowace alama. Wannan shine dalilin da ya sa samfura kamar Mixtral 8x7B da jita-jita na gine-ginen da ke bayan GPT-4 sun isa babban inganci ba tare da ƙimar ƙimar ƙima ba. Cinikin ciniki yana da rikitarwa: duk ƙwararrun dole ne su dace da ƙwaƙwalwar ajiya, kuma na'ura mai ba da hanya tsakanin hanyoyin sadarwa na iya yin ɓarna ko kuma yin lodin wasu ƙwararru, don haka horo yana buƙatar daidaitawa a hankali.
Fahimtar Fasaha
Zuciyar MoE ita ce hanyar sadarwar gating, ƙaramin koyo wanda ke ƙima kowane ƙwararru don alamar mai shigowa kuma yana bibiyar alamar zuwa manyan-k mafi yawan maki (sau da yawa k=1 ko 2). Don dakatar da na'ura mai ba da hanya tsakanin hanyoyin sadarwa daga aika komai zuwa wasu ƴan ƙwararrun ƙwararrun da aka fi so, horo yana ƙara ƙarin 'asara mai daidaitawa' wanda ke azabtar da rashin daidaituwa. Domin k ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun Ƙwararru waɗanda kawai ke gudanar da kowace alama, ƙididdigewa (FLOPs) suna tsayawa kusan ko da yaushe yayin da kuke ƙara ƙarin ƙwararru, don haka jimillar ma'auni da ma'auni na kowane-token da kansa.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Cakudawar Masana
MoE yana zama kayan aiki na asali don ƙira-ƙira mai iyaka saboda yana kawar da ƙarfi daga farashi. Yi tsammanin ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararrun ƙwararru waɗanda ke yin la'akari da ƙarin mahallin, da ingantattun dabaru don hidimar manyan ƙira mai ƙima akan ƙayyadaddun kayan aiki. Har ila yau, bincike yana magance matsalar ƙwaƙwalwar ajiya, tun da duk masana dole ne a ɗora su duk da cewa kaɗan ne ke gudana, ta hanyar ƙwararrun ƙwararru da ƙididdigewa. Kamar yadda buɗaɗɗen samfura kamar Mixtral da DeepSeek-MoE balagagge, ƙananan gine-ginen za su iya ƙarfafa mataimaka masu inganci akan ƙananan kasafin GPU.
Aiwatar da Gaskiyar Duniya
Mixtral 8x7B yana amfani da ƙwararru 8 kuma yana kunna 2 akan kowace alama, yana ba da kusan jumlolin 47B amma kawai ~ 13B yana aiki kowace alama don sauri, ƙima mai rahusa.
DeepSeek da Qwen suna jigilar manyan samfuran Harshen MoE waɗanda suka dace da ƙira masu yawa akan maƙasudai yayin da suke gudana tare da ƙananan ƙididdiga ta kowace alama.
Masu samar da Cloud LLM suna amfani da MoE don haka babban samfuri guda ɗaya zai iya yiwa masu amfani da yawa hidima da araha, tunda kowace buƙata tana haskaka ƴan ƙwararru ne kawai.
Google's A baya Canja Mai Canjawa ya kai sama da mitoci tiriliyan ta hanyar amfani da manyan hanyoyi 1 don ci gaba da sarrafa lissafin horo.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixture of Experts quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Cakudar Masana LoRA
Tambayoyin da ake yawan yi
What is Mixture of Experts?
Cakudar Kwararru (MoE) ƙirar ƙira ce wacce ke raba hanyar sadarwa zuwa manyan cibiyoyin sadarwa na musamman kuma tana kunna kaɗan ne kawai a kowace shigarwa. Yana ƙyale samfura su riƙe ilimi mai girma yayin kiyaye kowane tsinkaya cikin sauri da arha.
A cikin Cakudar Kwararru, menene zai yanke shawarar waɗanne ƙwararrun ke aiwatar da alamar da aka bayar?
Ƙaramar hanyar sadarwa ta gating tana koyon yin maki kowane ƙwararru don alamar kuma ta tura shi zuwa waɗanda suka fi cin nasara. Ana koyan hanyar tafiya, ba gyarawa ko bazuwar ba.
Me yasa samfurin MoE zai iya samun ƙarin jumlolin jumloli fiye da ƙima mai yawa ba tare da haɓakar madaidaicin ƙimar kowace alama ba?
Domin ƙwararrun ƙwararrun manyan-k ne kaɗai ke kunna wuta a kowace alama, ƙididdigewa mai aiki yana tsayawa kusan koyaushe koda jimlar adadin siga ya girma ta ƙara ƙarin masana.
Wace matsala ce ke magance asarar nauyi-daidaita (taimakon) yayin horon MoE?
Ba tare da daidaitawa ba, mai amfani da na'ura mai ba da hanya tsakanin hanyoyin sadarwa yana son fifita ƙwararrun masana. Asarar ƙarin tana ladabtar amfani da rashin daidaituwa don haka duk ƙwararru za su sami horo.
Mixtral 8x7B yana kunna 2 daga cikin ƙwararrunsa 8 kowace alama. Menene wannan ke nufi game da lissafinsa da girmansa?
Tare da 2 kawai na 8 ƙwararrun masu aiki, sigogi masu aiki a kowace alama (~ 13B) sun fi ƙanƙanta fiye da ~ 47B duka, rage farashin ƙima.
Wanne ne ainihin ƙarancin ƙirar MoE idan aka kwatanta da ƙima mai yawa daidai gwargwado?
Duk da cewa ƙwararrun ƙwararrun ƙwararru ne kawai ke ƙididdige kowane alamar, kowane ma'aunin ƙwararru dole ne a ɗora shi a cikin ƙwaƙwalwar ajiya, yana mai da ƙirar MoE don ƙwaƙwalwar ajiya.