Mamba ak barabi réew yuñ tànn
Mamba xeetu toppalante la buñ tabax ci kaw xeetu espaas yu réew mi (SSMs) yuy doxal mbind ci diir bu ligneer, di joxe beneen pexe bu gaaw ci bàyyi xel ci Transformer bi.
Résumé
Its key trick is making the model selectively decide what to remember and forget based on the input itself.
Plongeur bu xóot
Mamba, Albert Gu ak Tri Dao ñoo ko dugal ci 2023, ñu tabax ko ci kaw xeetu espaas buñ yamale. SSM bu yàgg dafay tënk jaar-jaaru benn toppalante ci benn anam bu nëbbu bu am dayo bu takku ba noppi diko yeesal jéego ak jéego, melni reso bu xarañ buy baamtu. Li gëna doywaat mooy tànneef: Mamba dafa def parametru SSM (ba ñaata nga wara denc, ba ñaata nga wara bàyyi) mu aju ci token bi fi nekk, suko defee model bi mëna bàyyi xelam ci kàddu yu am solo te baña bàyyi xel ci filler. Loolu dafay tax benn etaa bu am dayo bu takku nekk mémoire buy xam ëmbiit li. Ndax dafay moytu méngale bépp token ak bépp token, Mamba dafay eskale lineaire ak guddaayu toppalante bi ba noppi di gaaw ci dugal yu gudd lool yu melni genomes, audio, wala bind bu am guddaayu téere.
Gis-gis xarala
Modèlu espace état dafay màndargaal ab duggal ak genn ci sistem ligneer bu wéy bu matris A, B, C ak delta bu tollu ci jéego di fësal. SSM yu njëkk ya dañu daan tëye mbir yooyu, loolu tax ñu mëna gis convolution bu gaaw. Mamba dafay defar B, C, ak delta ci fonction yi ci duggal bi, loolu mooy dog gaawaayu convolution bi, moo tax mu jëfandikoo ab scanner parallèle bu hardware-aware buñu denc ci GPU SRAM bu gaaw ngir defaraat gaawaay bi ci di am mémoire bu aju ci duggal bi.
njeextalu pexe
Gaawaay ak yaatuwaay
Liggéeyukaay yi ci làkk yi mën nañu gëna gaaw te duñu yàq deggoo gi.
Dugg ak yegg
Dafay yaatal jëfandikoo gi ci làkk yi ak ci anam yi ñuy jokkoo.
dogal yu gëna leer
Ekip yi mën nañu gëna yàgg ci àtte ci jamono ji otomatisation di liggéey ci baamtu.
Ëlëgu Mamba ak barabi réew yuñ tànn
Mamba ak ki ko donnu ci Mamba-2 ñu ngi push ci architecture hybrid yuy boole ay couche yu ndaw ak ay couche SSM yu bari, di jàpp dooley ñoom ñaar. Xaarandi SSMs ci assistant yu am contexte bu gudd, ci modelu aparey fu memory bi tënk, ak domen yu amul mbind yu melni ADN ak audio. Gëstu mingi xool ndax SSM yu sell yi mën nañu méngoo ak Transformers ci liggéey yi soxla fàttaliku bu jaar yoon, ak ndax ñu ngi jëm ci tolluwaayu model bi gëna mag.
Doxal ci àdduna dëgg
Modeling ay xeetu ADN yu gudd lool, fu ay Transformer yu am ay milioŋ ciy jeton seer lool
Doole assistant làkk yu am contexte bu gudd yuy tënk téere yépp te duñu dagg
Defar audio ci jamono dëgg ak modelaasioŋu kàddu yuy jëfandikoo forme onde yu ñor yi ci anam wu jaar yoon
Deployment ci aparey bi wala ci catu barab bi stade recurrent bu ndaw bu am dayo fixe di denc mémoire ak cache buy gëna yokk
Risk yi ak balustrade yi
Lépp lu jaarul yoon mën na dugg ci rapoor yi, jàppale ci liggéey bi, wala ci njariñu gëstu bi.
Sensibilite bu gaaw mën na jur njariñ yu wuute ci laajte yu noonu mel.
Done yu am solo mën nañu feeñ sudee seytu jëfandikoo gi néew doole.
Roadmap ngir samp gi
Mandargal formaa génne gi, melokaan bi, ak standard kalite yi laata ngay dugal ko.
Tontu yu am solo ak balluwaay yu wóor saa yu dëggu bi di am solo.
Fexeel am barabu xool nit ñi ngir am njariñ yu am solo.
Toppal anami gacce yi ak di faral di tàggataat ay laaj wala def-liggéey.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mamba and Selective State Spaces quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Modèle spatial d'État et Mamba
Laaj yi ñuy faral di laaj
What is Mamba and Selective State Spaces?
Mamba xeetu toppalante la buñ tabax ci kaw xeetu espaas yu réew mi (SSMs) yuy doxal mbind ci diir bu ligneer, di joxe beneen pexe bu gaaw ci bàyyi xel ci Transformer bi. Li gëna am solo mooy fexe ba model bi tànn limu wara fàttaliku ak limu wara fàtte, lépp di aju ci limu dugal ci boppam.
Lan mooy njariñu Mamba ci wàllu ordinatër ci Transformer buñ miin?
Mamba dafay moytu méngale token yépp, moo tax njëg bi dafay màgg lineairement ak guddaayi sekans bi moo gën quadratically melni bàyyi xel.
Lan la baat tànneef di tekki ci Mamba?
Tanneef dafay tekki ni paramet yu melni B, C, ak delta nekk nañu ay fonction ci dugal bi fi nekk, di joxe mémoire buy xam ëmbiit li.
Naka la ko modelu espace etaa di wane jaar-jaaru benn toppalante?
SSMs ay xeetu xeetu recurrent lañu yuy boole lu weesu ci benn tolluwaayu dayo bu takku, nuru RNN.
Lu tax Mamba mënu jëfandikoo gaawaayu convolution bu gaaw bi SSM yi njëkka jëfandikoo?
Melokaanu convolusioŋ dafay laaj ay paramet yu takku (yu dul soppiku ci waxtu); parametre yi aju ci dugal dañuy dog loolu, moo tax Mamba dafay jëfandikoo scanner parallèle.
Ban pexe la Mamba di jëfandikoo ngir mëna gaaw doonte daa ñàkk gaawaayu convolution bi?
Mamba dafay jëfandikoo ab scanner buñu tann bu xam hardware bi, muy tëye etaa yu digg yi ci SRAM bu gaaw ngir defaraat produit bi.