Ntị Flash
Ntị Flash bụ ụzọ dị nkọ iji gbakọọ ntinye uche n'ime Transformers na-enweghị mgbe ọ na-ede nnukwu nlebara anya matriks iji belata ebe nchekwa.
Nchịkọta
It makes long-context models far faster and more memory-efficient without changing their math.
Ime miri emi
Standard attention compares every token to every other token, producing an N-by-N score matrix that grows quadratically with sequence length. N'amaghị ama, edere matriks ahụ ma gụọ ya site na ebe nchekwa bandwidth dị elu nke GPU (HBM), na nkwụsị ahụ - ọ bụghị ịba ụba - bụ ezigbo ihe mgbochi. Flash Attention, introduced by Tri Dao and colleagues in 2022, reorganizes the computation so the matrix is never fully stored. Ọ na-ahazi ajụjụ, igodo, na ụkpụrụ n'ime obere taịlị dabara na SRAM ngwa ngwa, na-agbakọ arụpụtaghị akụkụ, wee dụkọta ha ọnụ site na iji aghụghọ na-agba ọsọ-softmax n'ịntanetị. The output is mathematically identical to ordinary attention but uses linear memory and runs several times faster, especially on long sequences.
Nghọta nka nka
Isi aghụghọ bụ tiling gbakwunyere softmax n'ịntanetị. Softmax na-achọkarị ahịrị akara niile iji gbakọọ ọnụọgụ ya, mana Flash Attention na-edobe oke na-agba ọsọ yana nchikota nchikota ka ọ na-ebufe taịlị ọ bụla, na-emegharị mpụta akụkụ akụkụ mbụ ka nsonaazụ ikpeazụ wee bụrụ kpọmkwem. N'ihi na akara etiti na-anọ na SRAM (iwu nke ịdị ukwuu karịa HBM), algọridim bụ IO-maara: ọ na-ebelata ọgụgụ ebe nchekwa na-agụ ma na-ede kama ịrụ ọrụ mgbakọ na mwepụ.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke Ntị Flash
Ntị Flash abụrụla ngọngọ ụlọ nke ndabara, yana FlashAttention-2 na FlashAttention-3 na-amịpụta mmepụta karịa site na GPU ọhụrụ dị ka H100 site na imelite nkewa ọrụ na iji ụzọ FP8 dị ala. Na-atụ anya na-aga n'ihu na-emewe ya na ngwaike, ntinye siri ike n'ime ọzụzụ na usoro nrịbama, yana ụdị dị iche iche echedoro maka obere, windo na-amị amị, na nlebara anya ogologo oge. As context windows stretch toward millions of tokens, IO-aware kernels like this remain essential to keeping memory and speed practical.
Mmejuputa n'ezie n'ụwa
Ọzụzụ nnukwu ụdị asụsụ dị ka Llama na sistemụ klaasị GPT nwere windo ọnọdụ dị ogologo na ọnụ ala ebe nchekwa dị ala.
Ijere ndị enyemaka nkata ozi ọsọ ọsọ site na-eme ka ọkwa njupụta dị ngwa ebe a na-ebu ụzọ gụọ ogologo oge.
Na-enyere ngwaọrụ nyocha akwụkwọ na-etinye akwụkwọ niile ma ọ bụ codebases site n'ime ka nlebara anya ogologo oge dị na otu GPU.
Ọhụụ na-enye ike na ihe ntụgharị ọdịyo ebe ntinye mkpebi dị elu na-emepụta usoro akara ogologo ogologo.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Flash Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nlebaanya nlebara anya na ịkwachaa isi
Ajụjụ a na-ajụkarị
What is Flash Attention?
Ntị Flash bụ ụzọ dị nkọ iji gbakọọ ntinye uche n'ime Transformers na-enweghị mgbe ọ na-ede nnukwu nlebara anya matriks iji belata ebe nchekwa. It makes long-context models far faster and more memory-efficient without changing their math.
Kedu ihe bụ isi ihe mgbochi Flash Attention lekwasịrị anya?
Ntị Flash bụ IO-maara: ọ na-ebelata data agbachiri n'etiti ngwa ngwa SRAM na mgbawa na ebe nchekwa bandwit dị nwayọ, nke bụ ezigbo mkpọ kama ịgbakọ n'onwe ya.
Kedu ka nlebara anya Flash si ezere ịchekwa matrix N-by-N zuru ezu?
Ọ na-agbakọ kọmpụta ahụ ka ngọngọ ọ bụla dabara na SRAM ngwa ngwa, na-agbakọ ma na-achịkọta mpụta akụkụ na-enweghị ihe ọ bụla na-emepụta matriks niile na HBM.
Kedu usoro na-ahapụ nleba anya Flash gbakọọ softmax n'ụzọ ziri ezi na-ahụghị ahịrị niile otu mgbe?
Ngwa softmax dị n'ịntanetị na-edobe oke na-agba ọsọ ma na-agba ọsọ mgbe ọ na-enuba taịlị, na-emegharị mpụta akụkụ mbụ nke mbụ ka nhazi ikpeazụ kwesịrị ekwesị.
Kedu ihe kpatara nlebara anya Flash na-enye aka na usoro ogologo?
Matriks nlebara anya na-enweghị uche dị ka N-squared na ebe nchekwa, yabụ na-ezere nchekwa ya zuru oke na-eweta nnukwu ego nchekwa ozugbo mgbe usoro dị ogologo.
Kedu ihe atụmatụ 'IO-aware' nke Flash Attention na-ebute ụzọ mbelata?
IO-aware pụtara na emebere algọridim gburugburu ọnụ ahịa mbugharị data na ndị isi ebe nchekwa, na-ebelata okporo ụzọ HBM kama ịrụ ọrụ mgbakọ na mwepụ.