Uruvange rwimbitse
Uruvange rwimbitse (MoD) rutuma transformateur ikoresha amafaranga atandukanye ya compte kubimenyetso bitandukanye, ikayobora gusa ibimenyetso 'byingenzi' binyuze muri buri cyiciro kiremereye.
Incamake
It cuts the cost of processing easy tokens while keeping a fixed, predictable compute budget.
Kwibira cyane
Impinduka zisanzwe zikoresha buri cyiciro kuri buri kimenyetso, ndetse ntoya nkutumenyetso. Kuvanga Ubujyakuzimu, bwatangijwe na Google DeepMind muri 2024, yongeraho router ntoya kuri buri gice gihitamo top-k igice cyagenwe cyerekana ibimenyetso kugirango ubashe kwiyitaho no kubara MLP; ahasigaye gusimbuka guhagarika ukoresheje ihuza risigaye. Kuberako k ibimenyetso gusa bitunganyirizwa kumurongo, compte yose (FLOPs) irafatwa kandi izwi mbere, bitandukanye nuburyo bwambere bwimbaraga-bwimbitse bwatandukanye muburyo butateganijwe. Ibi bituma gukoresha no gukoresha ibikoresho neza. Moderi yatojwe na MoD irashobora guhuza ubwiza bwibanze bwa transformateur ukoresheje FLOP nkeya kuri pass yimbere, cyangwa ikagera kumurongo wo hejuru kuri compte imwe, kandi igitekerezo gihimbano gisanzwe hamwe na Mixture-y-Impuguke kugirango itange moderi ya 'MoDE' inzira yuburebure n'ubugari.
Ubushishozi
Kuri buri gice cya MoD, yize umurongo wa router wize amanota buri kimenyetso kandi agakomeza top-k kumanota; Ibimenyetso byatoranijwe binyura mubitekerezo na MLP, mugihe ibimenyetso bitatoranijwe bitwarwa imbere bidahindutse n'inzira isigaye. Gukoresha top-k ihamye (aho kuba kuri buri kimenyetso) bituma compte igishushanyo gihamye hamwe na tensor ishusho ihoraho, ikaba yorohereza ibyuma. Router yatojwe hamwe nabandi basigaye, kandi ibisekuruza bitera gukoresha abahanuzi bafasha kugirango ibyemezo byo guhitamo ntibireba ibimenyetso bizaza.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Igihe kizaza cyo kuvanga ubujyakuzimu
Ibarura risanzwe ni leveri yingenzi yo gukora neza nkurugero rwikigereranyo, kandi MoD ni urugero rwambere, rusukuye. Tegereza kwishyira hamwe kwimbitse hamwe nuruvange-rwinzobere (kugendana ubujyakuzimu ninzobere), ingengo yimishinga yo guhuza n'imihindagurikire igabanuka ku nyungu zoroshye, hamwe na marike yize yerekana neza ibimenyetso bikeneye gutunganywa byimbitse. Nkuko ibiciro byibanze byiganjemo ubukungu bwoherejwe, tekinike ireka icyitegererezo 'gitekereza cyane' gusa aho bikenewe, mugihe gikomeza gutinda, birashoboka ko bizaba bisanzwe mububiko bunini bunini.
Gushyira mu bikorwa Isi
Kugabanya FLOPs ikenewe mugutunganya inyandiko ndende usibye kubara byimbitse kubimenyetso byuzuza
Guhugura icyitegererezo gihuye nubuziranenge bwibanze kuri compte yo hasi, kugabanya ibiciro bya serivisi
Gukomatanya hamwe nuruvange-rwinzobere (MoDE) kugirango rugere kumurongo wimbitse zombi no guhitamo abahanga
Kugumya guhanura, gutinda guteganijwe kuri buri kimenyetso kuko ingengo yimibare ya compte igenwa mbere
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixture of Depths quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Uruvange-rw-abakozi
Ibibazo bikunze kubazwa
What is Mixture of Depths?
Uruvange rwimbitse (MoD) rutuma transformateur ikoresha amafaranga atandukanye ya compte kubimenyetso bitandukanye, ikayobora gusa ibimenyetso 'byingenzi' binyuze muri buri cyiciro kiremereye. Igabanya ikiguzi cyo gutunganya ibimenyetso byoroshye mugihe ubitse ingengo yimibare ihamye, iteganijwe.
Niki Kuvanga Ubujyakuzimu bitandukana kubimenyetso?
Inzira ya MoD yerekana ibimenyetso gusa binyuze muri buri cyiciro kiremereye, bityo ibimenyetso bitandukanye bigera kubwimbitse butandukanye.
Nigute MoD ituma ingengo yimari ya compte iteganijwe?
Guhitamo hejuru-k yikimenyetso kuri buri kashe ya FLOPs kandi igakomeza imiterere ya tensor ihoraho, ikaba yorohereza ibyuma.
Bigenda bite kubimenyetso router idahitamo kumurongo runaka?
Ibimenyetso bitatoranijwe byirengagiza kubara kandi bigatwarwa imbere bidahindutse n'inzira isigaye.
Ninde watangije Uruvange rwimbitse?
Uruvange rwimbitse rwatangijwe na Google DeepMind mu mpapuro 2024 kuri compte ya transformateur ya dinamike.
Guhuza MoD nuruvange-rw-Impuguke bitanga iki?
Gukomatanya ubujyakuzimu bwimbitse hamwe ninzobere zuyobora zitanga umusaruro 'MoDE' imiterere igereranya ibice byombi ninzobere.