Umwanya wa Leta ya Mamba
Mamba nicyitegererezo cyubatswe cyubatswe kumwanya wa leta (SSMs) itunganya inyandiko mugihe cyumurongo, itanga ubundi buryo bwihuse kubitekerezo bya Transformer.
Incamake
Its key trick is making the model selectively decide what to remember and forget based on the input itself.
Kwibira cyane
Mamba, yatangijwe na Albert Gu na Tri Dao mu mpera za 2023, yubatswe ku cyitegererezo cya leta cyubatswe. SSM isanzwe ikomatanya amateka yose yurukurikirane muburyo bugaragara-bwihishe leta ikanayivugurura intambwe ku yindi, nkumuyoboro uhoraho usubiramo. Iterambere ni uguhitamo: Mamba ikora ibipimo bya SSM (uko ugomba kubika, amafaranga yo kureka) biterwa nikimenyetso kigezweho, bityo icyitegererezo gishobora kwibanda kumagambo ajyanye no kwirengagiza uwuzuza. Ibi bireka imwe ihamye-ya leta ikora nkibintu-byibukwa kwibuka. Kuberako irinda kugereranya buri kimenyetso nikindi kimenyetso, umunzani wa Mamba ugereranya umurongo hamwe nuburebure bwikurikiranya kandi ukaguma byihuse kubintu byinjira cyane nka genome, amajwi, cyangwa inyandiko-ndende y'ibitabo.
Ubushishozi
Ikibanza cya leta cyerekana ikarita yinjiza ikurikiranye ibisohoka binyuze mumurongo uhoraho ugenwa na matrices A, B, C hamwe nintambwe ya delta. Mbere SSMs yagumije ibyo ikosora, itanga ibitekerezo byihuse. Mamba ikora B, C, na delta imikorere yinjiza, isenya ihuzabikorwa ryihuse, bityo ikoresha ibyuma-bimenyekanisha ibyuma bisanganywe bibitswe muri GPU SRAM byihuse kugirango bigarure umuvuduko mugihe byungutse byinjira mububiko.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Kazoza ka Mamba hamwe na Leta Yatoranijwe
Mamba nuwasimbuye Mamba-2 barimo gusunika mubwubatsi bwa Hybrid buhuza ibice bike byitondewe hamwe na SSM nyinshi, bifata imbaraga zombi. Tegereza SSMs mubufasha-burebure-burebure, ku bikoresho byerekana ibikoresho aho kwibuka bibujijwe, hamwe na domaine itari inyandiko nka ADN n'amajwi. Ubushakashatsi burimo gusuzuma niba SSMs nziza ishobora guhuza na Transformers kumirimo ikeneye kwibukwa neza, kandi niba igereranya ubunini bunini bw'icyitegererezo.
Gushyira mu bikorwa Isi
Kwerekana urugero rurerure cyane rwa ADN aho miliyoni-token Transformers ihenze cyane
Guha imbaraga imvugo ndende-abafasha mu ncamake ibitabo byose bitagabanije
Ibihe nyabyo byamajwi hamwe no kwerekana imvugo itunganya imivumba mbisi neza
Kuri-igikoresho cyangwa impande zoherejwe aho ntoya ihamye-ingano isubiramo leta ibika kwibuka hamwe nubwitonzi bukura
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mamba and Selective State Spaces quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Ikirere cya Leta Icyitegererezo na Mamba
Ibibazo bikunze kubazwa
What is Mamba and Selective State Spaces?
Mamba nicyitegererezo cyubatswe cyubatswe kumwanya wa leta (SSMs) itunganya inyandiko mugihe cyumurongo, itanga ubundi buryo bwihuse kubitekerezo bya Transformer. Amayeri yingenzi ni ugukora icyitegererezo guhitamo icyo kwibuka no kwibagirwa ukurikije ibyinjijwe ubwabyo.
Ni izihe nyungu nyamukuru zo kubara za Mamba kurenza Transformer isanzwe?
Mamba yirinda byose-bigereranya ibimenyetso, bityo igiciro cyacyo gikura neza hamwe nuburebure bwikurikiranya aho kuba inshuro enye nko kwitabwaho.
Ijambo 'guhitamo' risobanura iki muri Mamba?
Guhitamo bisobanura ibipimo nka B, C, na delta bihinduka ibikorwa byinjiza byubu, bitanga ububiko-bwibuke.
Nigute icyitegererezo cya leta cyerekana amateka yurukurikirane?
SSMs ni moderi isubirwamo-yuburyo bugenda bwuzuza ibyahise muburyo bunini, busa na RNN.
Kuki Mamba adashobora gukoresha shortcut yihuta ya SSMs yakoresheje?
Kwemeza kureba bisaba ibipimo bihamye (igihe-bidahinduka); Iyinjiza-ishingiye ku bipimo bimena ibyo, Mamba rero ikoresha parallel scan aho.
Nubuhe buryo Mamba akoresha kugirango agume vuba nubwo yatakaje shortcut?
Mamba ikoresha ibyuma-bimenyesha guhitamo scan ikomeza leta hagati muri SRAM yihuse kugirango igarure ibyinjira.