Nleba anya nleba anya nke mgbochi na obere ala
Nlebara anya na-adịghị mma na nke nwa amaala na-ahapụ ndị na-agbanwe agbanwe na-aga naanị n'usoro dị ogologo karịa akara ọ bụla, na-ebelata ọnụ ahịa quadratic nke nlebara anya ọkọlọtọ.
Nchịkọta
This is what makes efficient long-context models practical on real hardware.
Ime miri emi
Ọkọlọtọ nlebara anya onwe onye na-atụnyere akara ọ bụla na akara ngosi ọ bụla ọzọ, yabụ ọnụ ahịa na-eto n'ụzọ anọ na ogologo usoro, na-aghọ ihe mgbochi maka akwụkwọ dị ogologo. Nlebara anya dị obere na-amachibido akara ngosi ọ bụla na mpaghara ndị ọzọ. Ụzọ mgbochi-sparse na-ekewa usoro n'ime ngọngọ wee gbakọọ nlebara anya naanị maka ụzọ abụọ ngọngọ ahọpụtara, nke maapụ nke ọma na GPU tensor cores. Native Sparse Attention (NSA), nke sitere na DeepSeek, na-aga n'ihu: ọ nwere ike ịzụ azụ na njedebe na ngwaike, na-ejikọta alaka atọ, mkpakọ token na-enweghị isi, nhọrọ dị mma nke ihe mgbochi kachasị mkpa, na windo na-amị amị maka ọnọdụ mpaghara. N'ihi na a na-amụta ụkpụrụ sparsity n'oge ọzụzụ ọzụzụ kama ịgbachitere ya ma emechaa, NSA na-echekwa izi ezi mgbe ọ na-ebuga nnukwu ọsọ ọsọ na ogologo usoro.
Nghọta nka nka
NSA na-ahazi igodo na ụkpụrụ site na ụzọ atọ yiri ya, wee jikọta ha na ọnụ ụzọ mmụta. Mkpakọ na-achịkọta blọk nke token na nchikota nchịkọta; nhọrọ na-egbochi ọnụ ọgụgụ ma na-edobe naanị ndị kachasị elu maka nlebara anya zuru oke; windo na-amị amị na-ekpuchi akara ndị dị nso. Arụmọrụ mgbochi-larịị dabara na ohere nchekwa GPU na ntinye tensor-core, yabụ nchekwa nchekwa FLOP usoro iwu tụgharịrị n'ime ezigbo elekere mgbidi n'oge ọzụzụ yana ntinye, ọkachasị maka nzọụkwụ ngbanwe ebe nchekwa.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke Block-Sparse na Nleba anya nlebara anya nke obodo
Enwere ike inye ọzụzụ, ngwaike maara sparity na-aghọ ụzọ na-aga gburugburu nde-token na-enweghị ọnụ ahịa mgbawa. Na-atụ anya nlebara anya na-adịghị ahụkebe ka ejiri kernels na osooso mee ya, jikọta ya na nlebara anya n'ahịrị na echiche oghere steeti, ma nabata ya n'usoro ogologo oge na ụdị echiche. Dị ka ụkpụrụ na-aghọ ihe mmụta na ike, ụdị ga-ekenye nlebara anya mmefu ego n'otu n'otu, na benchmarks ga-esiwanye na-atụ decoding mmepụta na ogologo usoro, ọ bụghị naanị raw mma.
Mmejuputa n'ezie n'ụwa
Na-eme ihe nlereanya n'ofe koodu codebase ma ọ bụ ogologo nkwekọrịta iwu ebe nlebara anya zuru oke ga-agwụ ebe nchekwa GPU.
DeepSeek's NSA na-eme ngwa ngwa ma ọzụzụ izizi yana ntinye ogologo okwu mgbe ọ na-adakọ ma ọ bụ na-eti nlebara anya zuru oke.
Na-achịkọta akwụkwọ ogologo akwụkwọ site na ịga na nchịkọta ngọngọ abịakọrọ gbakwunyere akụkụ dị mkpa na mpaghara.
Na-eme ka ndị enyemaka mkparịta ụka dị ogologo oge dị ngwa nke usoro ngbanwe koodu na-adabere na ebe nchekwa site na ịmachi akara nke ọ bụla na ngọngọ ndị nwere ọkwa dị elu.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Block-Sparse and Native Sparse Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ụdị nlebara anya dị obere
Ajụjụ a na-ajụkarị
What is Block-Sparse and Native Sparse Attention?
Nlebara anya na-adịghị mma na nke nwa amaala na-ahapụ ndị na-agbanwe agbanwe na-aga naanị n'usoro dị ogologo karịa akara ọ bụla, na-ebelata ọnụ ahịa quadratic nke nlebara anya ọkọlọtọ. Nke a bụ ihe na-eme ka ụdịdị ogologo oge dị mma dị irè na ngwaike n'ezie.
Kedu ihe kpatara nlebara anya onwe onye ọkọlọtọ ji dị oke ọnụ maka usoro dị ogologo?
Token ọ bụla na-aga na akara ngosi ọ bụla ọzọ, yabụ gbakọọ na ọnụ ọgụgụ ebe nchekwa na square nke ogologo usoro.
Gịnị bụ isi echiche nke ngọngọ-sparse nlebara anya?
A na-ekewa usoro ahụ n'ime ngọngọ ma gbakọọ nlebara anya naanị maka ụzọ abụọ ngọngọ ahọpụtara, nke na-esetịpụkwa nke ọma na GPU tensor cores.
Kedu ihe na-eme Native Sparse Attention (NSA) dị iche na ọtụtụ ụzọ ndị mbụ na-adịghị ahụkebe?
NSA na-amụta ụkpụrụ sparsity ya n'oge ọzụzụ ọzụzụ ma emebere ya ka ọ kwekọọ na ngwaike, yabụ ọ na-echekwa izi ezi mgbe ọ na-agba ọsọ ọsọ.
Kedu alaka atọ NSA na-ejikọta maka igodo na ụkpụrụ ya?
NSA na-ejikọta mkpakọ token na-enweghị isi, nhọrọ ngọngọ nke ọma, yana windo mwụda mpaghara na-eji ọnụ ụzọ amụtara.
Kedu ihe kpatara NSA ji eji arụ ọrụ n'ọkwa mgbochi karịa nhịahụ ọkwa ọkwa aka ike?
Usoro nnweta ọkwa mgbochi dabara ngwaike GPU, yabụ nchekwa nchekwa FLOP a na-eche echiche na-aghọ ngwa ngwa elekere mgbidi n'ezie.