Oghere Mamba na Nhọrọ steeti
Mamba bụ usoro usoro arụnyere na ụdị oghere steeti (SSM) nke na-ahazi ederede n'oge ahịrị, na-enye ngwa ngwa ọzọ na nlebara anya akụkụ anọ nke Transformer.
Nchịkọta
Its key trick is making the model selectively decide what to remember and forget based on the input itself.
Ime miri emi
Mamba, nke Albert Gu na Tri Dao webatara na ngwụcha 2023, ka ewuru ya na ụdị oghere steeti ahaziri ahazi. SSM kpochapụwo na-akpakọ akụkọ ihe mere eme nke usoro n'ime ọnọdụ zoro ezo nke nwere oke ma na-emelite ya site na nzọụkwụ, dị ka netwọk ọkaibe na-emegharị ugboro ugboro. Ọganihu bụ selectivity: Mamba na-eme ka SSM paramita ( ole ka ọ ga-edebe, ole ka ị ga-esi banye) dabere na akara ngosi dị ugbu a, ya mere ihe nlereanya ahụ nwere ike ilekwasị anya na okwu ndị dị mkpa ma na-eleghara ihe mkpuchi anya. Nke a na-ahapụ otu steeti nwere oke ka ọ rụọ ọrụ dị ka ebe nchekwa maara ọdịnaya. N'ihi na ọ na-ezere ịtụnyere akara ọ bụla na akara ngosi ọ bụla ọzọ, Mamba na-agbakọ n'ahịrị n'usoro n'ogologo wee na-adị ngwa ngwa na ntinye ogologo dị ka genomes, ọdịyo, ma ọ bụ ederede ogologo akwụkwọ.
Nghọta nka nka
Ụdị oghere steeti na-esetịpụ usoro ntinye na mmepụta site na usoro ahịrị na-aga n'ihu nke matrices A, B, C kọwapụtara site na nha delta. Ndị SSM ndị gara aga debere ndị a, na-enye ohere nlegharị anya ngwa ngwa. Mamba na-eme B, C, na delta ọrụ nke ntinye, nke na-agbaji ụzọ mkpirisi convolution, yabụ na ọ na-eji nyocha ngwaike maara nke ọma na GPU SRAM ngwa ngwa iji nwetaghachi ọsọ mgbe ị na-enweta ebe nchekwa dabere na ntinye.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke Mamba na oghere steeti ahọpụtara
Mamba na onye nọchiri ya Mamba-2 na-abanye n'ime ihe owuwu ngwakọ nke na-etinye nlebara anya ole na ole na ọtụtụ ọkwa SSM, na-ewere ike nke abụọ ahụ. Na-atụ anya SSM na ndị enyemaka ogologo oge, ụdị ngwaọrụ ebe ebe nchekwa na-egbochi, yana ngalaba na-abụghị ederede dị ka DNA na ọdịyo. Nnyocha na-enyocha ma SSM dị ọcha nwere ike dakọtara Transformers na ọrụ chọrọ ncheta nke ọma, yana ma hà ruru nha ụdị kachasị ukwuu.
Mmejuputa n'ezie n'ụwa
Ịmepụta usoro DNA dị ogologo nke ukwuu ebe ndị ntụgharị nde token dị oke ọnụ
Na-akwado ndị enyemaka asụsụ ogologo okwu nke na-achịkọta akwụkwọ niile na-enweghị mkpọpu
Ọgbọ ọdịyo na nhazi okwu n'ezie na-ahazi ụdị ebili mmiri nke ọma
Na-ebuga ngwaọrụ ma ọ bụ n'ọnụ ebe obere ọnọdụ na-emegharị ugboro ugboro na-echekwa ebe nchekwa yana cache nlebara anya na-eto eto.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mamba and Selective State Spaces quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ụdị Oghere Ọchịchị steeti na Mamba
Ajụjụ a na-ajụkarị
What is Mamba and Selective State Spaces?
Mamba bụ usoro usoro arụnyere na ụdị oghere steeti (SSM) nke na-ahazi ederede n'oge ahịrị, na-enye ngwa ngwa ọzọ na nlebara anya akụkụ anọ nke Transformer. Isi aghụghọ ya bụ ime ka ihe nlereanya họrọ họrọ ihe ị ga-echeta ma chefuo dabere na ntinye n'onwe ya.
Gịnị bụ isi uru mgbakọ na mwepụ nke Mamba karịa ọkọlọtọ Transformer?
Mamba na-ezere ntụnyere akara ụzọ abụọ niile, yabụ ọnụ ahịa ya na-eto n'usoro n'usoro ogologo karịa nlebara anya anọ.
Kedu ihe okwu 'nhọrọ' na-ezo aka na Mamba?
Nhọrọ nhọrọ pụtara parampat dị ka B, C, na delta na-aghọ ọrụ nke ntinye ugbu a, na-enye ebe nchekwa mara ọdịnaya.
Kedu ka ụdị oghere steeti si anọchi anya akụkọ ihe mere eme nke usoro?
SSM bụ ụdị ụdị nke na-atụgharị oge gara aga n'ime steeti nwere oke, dịka RNN.
Kedu ihe kpatara na Mamba enweghị ike iji ụzọ mkpirisi convolution ngwa ngwa nke ndị SSM mbụ jiri mee ihe?
Nlegharị anya nke mgbagwoju anya na-achọ paramita ndị a kapịrị ọnụ (oge na-adịghị agbanwe agbanwe); ntinye dabere na parampat na-agbaji nke ahụ, yabụ Mamba na-eji nyocha yiri ya kama.
Kedu usoro Mamba na-eji nọrọ ngwa ngwa n'agbanyeghị ụzọ mkpirisi convolution kwụsịrị?
Mamba na-eji nyocha nhọrọ nke maara ngwaike nke na-edobe steeti etiti na SRAM ngwa ngwa iji nwetaghachi mmepụta.