InoteveraGaidhi rinotevera
State Space Models uye Mamba
Basics
Mutauro AI GUIDE
Mamba uye Selective State Spaces
Mamba imodeli inoteedzana yakavakirwa pamamiriro emuchadenga (SSMs) ayo anogadzirisa zvinyorwa munguva yemutsara, ichipa imwe inokurumidza imwe nzira kune iyo Transformer's quadratic attention.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
Hunyengeri hwayo hwakakosha kuita kuti modhi isarudze zvekurangarira uye kukanganwa zvichibva pane inopinza pachayo.
Kudzika Kwakadzika
Mamba, yakaunzwa naAlbert Gu naTri Dao mukupera kwa2023, yakavakirwa pane yakarongeka mamiriro enzvimbo. A classic SSM inomanikidza iyo nhoroondo yese yekutevedzana kuita yakagadziriswa-saizi yakavanzika mamiriro uye inovandudza iyo nhanho nhanho, seyakaomesesa inodzokororwa network. Kubudirira ndiko kusarudza: Mamba inoita iyo SSM parameters (yakawanda sei yekuchengeta, yakawanda sei yekupinda) zvinoenderana nechiratidzo chazvino, saka modhi inogona kutarisa pamashoko akakodzera uye kufuratira filler. Izvi zvinobvumira imwe yakagadziriswa-saizi mamiriro kuita senge-inoziva ndangariro. Nekuti inodzivirira kuenzanisa chiratidzo chese kune chimwe chiratidzo chese, Mamba inoyera zvakatevedzana nehurefu hwekutevedzana uye inogara ichikurumidza pane yakarebesa yekupinda senge genomes, odhiyo, kana bhuku-kureba mavara.
Kuitwa muNyika yeChokwadi
Kuenzanisira kureba kweDNA kutevedzana uko miriyoni-token Transformers inodhura zvakanyanya
Kugonesa vabatsiri vemitauro mirefu vanopfupisa mabhuku ese pasina kudimburira
Chaiyo-nguva yekugadzira odhiyo uye yekutaura modhi iyo inogadzirisa yakasvibira waveforms zvinobudirira
Pa-mudziyo kana kumucheto deployments uko diki yakagadziriswa-saizi inodzokororwa inochengetedza ndangariro inopesana neinokura yekutarisisa cache.
Ramangwana reMamba uye Selective State Spaces
Mamba nemutsivi wayo Mamba-2 vari kusunda muzvivakwa zvakasanganiswa izvo zvinosanganisa mashoma ekutarisisa akaturikidzana ane akawanda SSM akaturikidzana, achitora masimba ezvose. Tarisira maSSM mune akareba-context vabatsiri, pane-mudziyo modhi apo ndangariro inomanikidzwa, uye asiri-mavara madomasi seDNA uye odhiyo. Tsvagiridzo iri kuongorora kana maSSM akachena anogona kuenderana neTransformers pamabasa anoda kunyatsoyeukwa, uye kuti anoyera kusvika kune yakakura modhi saizi.
Kuzvishandisa
Nei zvichikosha
- Kumhanya uye chiyero. Mutauro workflows inogona kufamba nekukurumidza pasina kupira kuenderana.
- Svika uye svika. Inopamhidzira kupinda mumitauro yese nemataera ekutaurirana.
- Sarudzo dzakajeka. Zvikwata zvinogona kupedza nguva yakawanda pakutonga uku otomatiki ichibata kudzokorora.
Chenjera
- Chokwadi chehuroyi chinogona kupinda chinyararire mishumo, kuyerera kwetsigiro, kana tsvakiridzo.
- Kunzwa nekukasira kunogona kugadzira mhedzisiro isingaenderane pane zvikumbiro zvakafanana.
- Sensitive text data inogona kuburitswa kana zvidhiraivho zvisina kusimba.
Matanho anotevera
- Tsanangura chimiro chekubuda, toni, uye mhando zviyero usati waburitsa.
- Mhinduro dzepasi neakavimbika masosi pese pazvine basa.
- Chengetedza ongororo yekuongorora yemunhu kune yakakwira-stake zvinobuda.
- Tevera maitiro ekutadza uye dzidzisazve kukurudzira kana mafambiro ebasa nguva nenguva.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Mamba and Selective State Spaces quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
Chii chinonzi Mamba uye Selective State Spaces?
Mamba imodeli inoteedzana yakavakirwa pamamiriro emuchadenga (SSMs) ayo anogadzirisa zvinyorwa munguva yemutsara, ichipa imwe inokurumidza imwe nzira kune iyo Transformer's quadratic attention. Hunyengeri hwayo hwakakosha kuita kuti modhi isarudze zvekurangarira uye kukanganwa zvichibva pane inoisa pachayo.
Chii chakakosha computational mukana weMamba pamusoro peyakajairwa Transformer?
Mamba inodzivirira ese-maviri chiratidzo kuenzanisa, saka mutengo wayo unokura zvakatevedzana nekutevedzana kureba kwete quadratically sekutarisa.
Ko izwi rekuti 'selective' rinorevei kuMamba?
Kusarudzika kunoreva paramita senge B, C, uye delta inova mabasa eiyo yazvino yekuisa, ichipa zvemukati-inoziva ndangariro.
Ko modhi yemuchadenga inomiririra sei nhoroondo yekutevedzana?
MaSSM ndiwo anodzokororwa-maitiro emhando anopeta zvakapfuura kuita yakagadziriswa-saizi mamiriro, akafanana neRNN.
Sei Mamba isingashandise nzira yekudimbudzira yekukurumidza yakamboshandiswa neSSM?
Iyo convolution maonero inoda yakagadziriswa (nguva-invariant) parameters; kuisa-inotsamira paramita inotyora izvo, saka Mamba inoshandisa parallel scan pachinzvimbo.
Ndeipi nzira inoshandiswa neMamba kuramba yakatsanya kunyangwe yarasa nzira yekudimbudzira?
Mamba inoshandisa hardware-aware selective scan inochengeta nyika dzepakati muSRAM inokurumidza kudzoreredza kubuda.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi