Cakuda Zurfi
Cakudar Zurfafawa (MoD) yana ƙyale mai taswira ya kashe ƙididdige ƙididdiga daban-daban akan alamu daban-daban, yana sarrafa alamun 'mahimmanci' kawai ta kowane nau'i mai nauyi.
Dubawa
It cuts the cost of processing easy tokens while keeping a fixed, predictable compute budget.
Zurfafa nutsewa
Madaidaitan gidajen wuta suna amfani da kowane Layer zuwa kowane alama, har ma da ƙananan abubuwa kamar alamar rubutu. Cakuɗen Zurfafawa, wanda Google DeepMind ya gabatar a cikin 2024, yana ƙara ƙaramin na'ura mai ba da hanya tsakanin hanyoyin sadarwa a kowane shinge wanda ke zaɓar ƙayyadadden juzu'in top-k na alamun don samun cikakkiyar kulawar kai da lissafin MLP; Sauran sun tsallake shingen ta hanyar haɗin da ya rage. Saboda kawai alamomin k da ake sarrafa su a kowane Layer, jimlar lissafin (FLOPs) an rufe su kuma an san su a gaba, sabanin hanyoyin zurfin zurfin zurfin da suka bambanta ba tare da tabbas ba. Wannan yana sa batching da kuma amfani da kayan aiki da kyau. Samfuran da aka horar da MoD na iya dacewa da ingancin gidan wuta na asali ta amfani da ƴan FLOPs a kowane wucewar gaba, ko isa mafi inganci a lissafin guda ɗaya, kuma ra'ayin ya haɗa ta halitta tare da Mixture-of-Experts don ba da ƙirar 'MoDE' waɗanda ke tafiya akan zurfin da faɗi.
Fahimtar Fasaha
A kowane toshe MoD, koyo na'ura mai ba da hanya tsakanin hanyoyin sadarwa na layi yana ƙididdige kowane alama kuma yana kiyaye saman-k da maki; Alamu da aka zaɓa suna wucewa ta hankali da MLP, yayin da alamun da ba a zaɓa ba ana ɗaukar su gaba ba canzawa ta hanyar saura. Yin amfani da kafaffen saman-k (maimakon madaidaicin kofa) yana sa lissafin jadawali ya tsaya tsayin daka da sifofin tensor akai-akai, wanda ke da abokantaka na hardware. An horar da na'ura mai ba da hanya tsakanin hanyoyin sadarwa tare da sauran hanyoyin sadarwa, kuma tsararru na yin amfani da na'urori masu aunawa don haka yanke shawara ba za su kalli alamun gaba ba.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Cakuda Zurfi
Ƙididdigar ƙididdigewa babban lefa ne don inganci azaman sikelin ƙira, kuma MoD farkon, misali ne mai tsabta. Yi tsammanin haɗin kai mai zurfi tare da Cakuda-na-Kwararru (gudanar da kan zurfafawa da ƙwararru), kasafin kuɗi masu daidaitawa waɗanda ke raguwa don abubuwan shigar cikin sauƙi, da kuma hanyoyin da aka koya waɗanda suka fi gano waɗanne alamun da gaske ke buƙatar aiki mai zurfi. Kamar yadda farashin ƙima ke mamaye tattalin arziƙin turawa, fasahohin da ke barin ƙira su yi tunani tuƙuru kawai a inda ake buƙata, yayin da ake kiyaye latti, mai yuwuwa su zama ma'auni a cikin manyan gine-gine.
Aiwatar da Gaskiyar Duniya
Rage FLOPs da ake buƙata don aiwatar da dogayen takardu ta hanyar tsallake ƙididdige ƙididdiga mai zurfi akan alamun filler
Horar da samfurin da ya dace da ingancin tushe a ƙananan ƙididdigewa, rage farashin hidima
Haɗuwa tare da Mixture-of-Experts (MoDE) zuwa hanya akan zurfin Layer biyu da zaɓin gwani
Tsayar da tsinkaya, ƙayyadaddun latency ga kowane alama saboda an daidaita kasafin kuɗin kowane Layer a gaba
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixture of Depths quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Haɗin-Agents
Tambayoyin da ake yawan yi
What is Mixture of Depths?
Cakudar Zurfafawa (MoD) yana ƙyale mai taswira ya kashe ƙididdige ƙididdiga daban-daban akan alamu daban-daban, yana sarrafa alamun 'mahimmanci' kawai ta kowane nau'i mai nauyi. Yana rage farashin sarrafa alamu masu sauƙi yayin kiyaye ƙayyadaddun kasafin ƙididdige ƙididdiga.
Menene Cakudar Zurfafawa ya bambanta a tsakanin alamu?
MoD yana ba da wasu alamu ne kawai ta hanyar ƙididdige nauyi na kowane Layer, don haka alamu daban-daban suna samun zurfi daban-daban yadda ya kamata.
Ta yaya MoD ke kiyaye lissafin lissafin kasafin kudin sa?
Zaɓin ƙayyadadden top-k na alamomin kowane toshe iyakoki FLOPs kuma yana riƙe da sifofin tensor akai-akai, wanda ke dacewa da hardware.
Me zai faru ga alamun da mai ba da hanya tsakanin hanyoyin sadarwa baya zaɓa a wani shingen da aka bayar?
Alamu waɗanda ba zaɓaɓɓu ba suna ƙetare lissafin toshe kuma ana aiwatar da su gaba ba canzawa ta hanyar saura.
Wanene ya gabatar da Cakudar Zurfi?
Google DeepMind ne ya gabatar da cakuɗen Zurfafawa a cikin takarda na 2024 akan lissafin taswira mai ƙarfi.
Menene hada MoD tare da Mixture-of-Experts ke samarwa?
Haɗa zurfafa zirga-zirga tare da ƙwararrun ƙwararru yana haifar da samfuran 'MoDE' waɗanda ke ƙididdige ƙididdigewa akan nau'i biyu da masana.