Block-Sparse iyo Feejignaan yar oo Dhalad ah
Dareenka yar ee xannibaadda iyo dareenka dhaladka ah waxay u oggolaadaan transformers-ku inay ka soo qaybgalaan oo keliya qaybaha ugu habboon ee taxanaha dheer halkii ay ka ahaan lahaayeen calaamad kasta, iyagoo hoos u dhigaya qiimaha quadratic ee dareenka caadiga ah.
Dulmar
This is what makes efficient long-context models practical on real hardware.
quusid qoto dheer
Heerarka is-fiirinta ayaa isbarbardhigga calaamad kasta iyo calaamad kasta oo kale, markaa kharashku wuxuu u koraa afar-geesood dhererka isku xigxiga, wuxuu noqonayaa mamnuuc dukumentiyo aad u dheer. Fiiro gaar ah ayaa calaamad kasta ku xaddidaya qayb hoose oo kuwa kale ah. Dariiqooyinka xannibaadda-yar-yar waxay u kala qaybiyaan taxanaha blocks waxayna xisaabiyaan dareenka kaliya ee lamaanayaasha block ee la doortay, kuwaas oo khariidado si wax ku ool ah ugu dhejinaya koojiyeyaasha GPU tensor. Native Sparse Attention (NSA), oo ka socota DeepSeek, way sii socotaa: waa dhamaad-ilaa-dhammaad iyo qalab-ku-habboon, oo isku-dhafan saddex laamood, isku-duubni calaamo-miro-dhala, xulashada hab-wanaagsan ee baloogyada ugu muhiimsan, iyo daaqad toosan oo loogu talagalay macnaha guud. Sababtoo ah qaabka jilicsanaanta waxaa la bartaa inta lagu guda jiro tababarka ka hor intii lagu xiri lahaa ka dib, NSA waxay ilaalisaa saxnaanta iyada oo gudbinaysa xawli badan oo isdabajoog ah.
Aragtida Farsamada
NSA waxay ku socodsiisaa furayaasha iyo qiyamka iyada oo loo marayo saddex waddo oo isbarbar socda, ka dibna waxay ku daraan albaabo la bartay. Cadaadisku wuxuu isu geeyaa baloogyo calaamado ah matalaad kooban; doorashadu waxay xannibaysaa oo waxay ilaalisaa oo keliya kuwa ugu sarreeya si ay fiiro buuxda u yeelato; Daaqad simbiriirixan leh ayaa daboosha calaamadaha u dhow. Hawlgallada heerka xannibaadda waxay la jaan qaadayaan gelitaanka xusuusta GPU-da iyo soo-saarka tensor-core, markaa kaydinta aragtida FLOP waxay u tarjuntaa xawaaraha saacadaha derbiga dhabta ah inta lagu jiro labadaba tababarka iyo ka-fiirsashada, gaar ahaan tallaabada dejinta xusuusta-xiran.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Block-Sparse iyo Feejignaanta Naafada Dhaladka ah
Tababbarka la qaadan karo, ka warhaynta qalabku waxa ay noqonaysaa dariiqa loo maro macnaha guud ee malaayiinta iyada oo aan kharash la qarxin. Filo dareenka aan badnayn in la iskula qaabeeyey kernels iyo dardargeliyayaasha, lagu dhex milmay fiiro gaar ah iyo fikrado goob dawladeed, oo lagu qaato soohdintooda muddada dheer iyo moodooyinka sababaynta. Marka ay nidaamyadu noqdaan kuwo la baran karo oo firfircoon, moodooyinka ayaa u qoondayn doona miisaaniyad fiiro gaar ah su'aal kasta, iyo bartilmaameedyadu waxay si sii kordheysa u cabbiri doonaan dejinta codeeynta ee taxanaha dheer, ma aha oo kaliya tayada cayriin.
Dhaqangelinta Adduunka-dhabta ah
Ku socodsiinta moodal dhan codebase ama qandaraas sharci oo dheer halkaas oo dareenka buuxa uu daali doono xusuusta GPU.
DeepSeek's NSA waxay dardargelinaysaa tababbarka horudhaca ah iyo ka-fiirsashada muddada-dheer labadaba iyadoo la barbardhigayo ama garaacaya saxnaanta feejignaanta.
Soo koobida dukumeenti dhererka buugaag adoo ka qayb galaya soo koobida xannibaadda iyo tuducyada deegaanka khuseeya.
Dedejinta caawiyayaasha wada sheekaysiga muddada-dheer kuwaas oo tillaabada codeeyntu ay ku xidhan tahay xusuusta iyada oo calaamad kasta lagu xaddidayo baloogyada darajada sare leh.
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Block-Sparse and Native Sparse Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Qaababka Fiiro Gaarka Leh
Su'aalaha soo noqnoqda
What is Block-Sparse and Native Sparse Attention?
Dareenka yar ee xannibaadda iyo dareenka dhaladka ah waxay u oggolaadaan transformers-ku inay ka soo qaybgalaan oo keliya qaybaha ugu habboon ee taxanaha dheer halkii ay ka ahaan lahaayeen calaamad kasta, iyagoo hoos u dhigaya qiimaha quadratic ee dareenka caadiga ah. Tani waa waxa ka dhigaya moodooyinka wax ku oolka ah ee muddada-dheer ee waxtarka leh ee qalabka dhabta ah.
Waa maxay sababta is-fiirinta caadiga ah ay qaali u tahay taxanaha dheer?
Calaamad kastaa waxay la socotaa calaamad kasta oo kale, markaa ku xisaabi oo cabbir xusuusta oo leh labajibbaaran dhererka isku xigxiga.
Waa maxay fikradda asaasiga ah ee feejignaanta xannibaadda?
Taxanuhu wuxuu u kala qaybsan yahay baloog, dareenkana waxaa lagu xisaabiyaa oo keliya lammaanaha la doortay, kaas oo sidoo kale si fiican u sawiraya xudunta tensor GPU.
Maxaa ka dhigaya Feejignaanta Dhaladka ee Sparse (NSA) kaga duwan tahay habab badan oo hore oo aan fiicneyn?
NSA waxa ay barataa qaabka ay u yar tahay inta lagu jiro tababarka ka hor waxaana loogu talagalay in lagu waafajiyo qalabka, si ay u ilaaliso saxnaanta inta ay si degdeg ah u socoto.
Waa kuwee saddexda laamood ee NSA ay isugu geyso furaheeda iyo qiimaheeda?
NSA waxay isku daraysaa isku xidhka calamada, xulashada xannibaadda wanaagsan, iyo daaqad simbiriirixan oo maxalli ah iyadoo la isticmaalayo albaabo la bartay.
Maxay NSA u isticmaashaa hawlgallada heerka-hortagga ah halkii ay ka ahaan lahayd caydh-calaamad aan sabab lahayn?
Nidaamyada gelitaanka heerka xannibaadda waxay ku habboon yihiin qalabka GPU-da, markaa kaydinta FLOP ee aragtida waxay noqotaa xawaare-saacad-darbi oo dhab ah.