Hankalin walƙiya
Hankalin Flash hanya ce mai wayo don ƙididdige matakin kulawa a cikin Transformers ba tare da taɓa rubuta babban matrix ɗin hankali don rage ƙwaƙwalwar ajiya ba.
Dubawa
It makes long-context models far faster and more memory-efficient without changing their math.
Zurfafa nutsewa
Daidaitaccen hankali yana kwatanta kowane alamar da kowane alamar, yana samar da matrix N-by-N wanda ke girma da ƙarfi tare da tsayin jeri. A hankali, an rubuta wannan matrix ɗin zuwa kuma an sake karantawa daga ƙwaƙwalwar ajiyar bandwidth mai girma na GPU (HBM), kuma wannan rufewa - ba haɓakawa ba - shine ainihin ƙugiya. Hankalin Flash, wanda Tri Dao da abokan aiki suka gabatar a cikin 2022, yana sake tsara lissafin don haka matrix ɗin ba a taɓa adanawa sosai ba. Yana aiwatar da tambayoyi, maɓallai, da ƙima a cikin ƙananan fale-falen fale-falen da suka dace da sauri akan SRAM akan guntu, yana ƙididdige sakamako na ɓangarori, kuma yana ɗinke su tare ta amfani da dabarar kan layi mai gudana-softmax. Fitowar ta lissafi iri ɗaya ce ga hankali na yau da kullun amma yana amfani da ƙwaƙwalwar layin layi kuma yana aiki da sauri sau da yawa, musamman akan dogayen jeri.
Fahimtar Fasaha
Babban dabarar shine tiling tare da software na kan layi. Softmax yawanci yana buƙatar duka jeri na maki don ƙididdige adadin adadin sa, amma Flash Attention yana kiyaye iyakar gudu da jimlar gudu yayin da yake gudana kowane tayal, yana sake fasalin abubuwan da aka fitar a baya don haka sakamakon ƙarshe daidai ne. Saboda matsakaicin maki yana tsayawa a cikin SRAM (umarni na girma da sauri fiye da HBM), algorithm ɗin yana sane da IO: yana rage yawan karantawa da rubutu maimakon ayyukan ƙididdiga.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Flash Hankali
Hankalin Flash ya zama tsoho tubalan gini, tare da FlashAttention-2 da FlashAttention-3 suna matsi ƙarin kayan aiki daga sabbin GPUs kamar H100 ta hanyar haɓaka rarrabuwar aiki da amfani da ƙananan madaidaicin hanyoyin FP8. Yi tsammanin ci gaba da ƙira tare da kayan masarufi, haɗaɗɗiyar haɗa kai cikin horo da tsarin ƙima, da bambance-bambancen da aka kunna don ƙarancin taga, zamewa-taga, da kulawa mai tsayin yanayi. Kamar yadda mahallin windows ke shimfiɗa zuwa miliyoyin alamu, IO-sane kernels kamar wannan suna da mahimmanci don kiyaye ƙwaƙwalwar ajiya da saurin aiki.
Aiwatar da Gaskiyar Duniya
Horar da manyan nau'ikan harshe kamar Llama da tsarin ajin GPT tare da tagogi masu tsayi mai tsayi a ƙananan farashin ƙwaƙwalwar ajiya.
Bauta wa mataimakan hira cikin sauri ta hanyar hanzarta matakin cikawa inda aka fara karanta dogon lokaci.
Ƙaddamar da kayan aikin nazarin daftarin aiki waɗanda ke shigar da littattafai gabaɗaya ko ma'ajin ƙididdiga ta hanyar ba da kulawar dogon layi mai yiwuwa akan GPU ɗaya.
Ƙarfafa hangen nesa da masu Canza sauti inda manyan bayanai masu ƙarfi ke haifar da jerin alamun dogon lokaci.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Flash Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Hankali Maida Hankali da Yanke Kai
Tambayoyin da ake yawan yi
What is Flash Attention?
Hankalin Flash hanya ce mai wayo don ƙididdige matakin kulawa a cikin Transformers ba tare da taɓa rubuta babban matrix ɗin hankali don rage ƙwaƙwalwar ajiya ba. Yana sa samfuran yanayi mai tsayi da sauri da sauri da ingantaccen ƙwaƙwalwar ajiya ba tare da canza lissafin su ba.
Menene babban ginshiƙin da Flash Attention ke hari?
Hankalin Flash shine IO-sane: yana rage bayanan da aka rufe tsakanin sauri SRAM akan guntu da jinkirin ƙwaƙwalwar bandwidth mai girma, wanda shine ainihin ƙugiya maimakon lissafin kanta.
Ta yaya Hankalin Flash ke guje wa adana cikakken matrix N-by-N?
Yana fale-falen lissafin don haka kowane toshe ya dace da sauri SRAM, ƙididdigewa da tara abubuwan da aka fitar ba tare da taɓa yin amfani da dukkan matrix a cikin HBM ba.
Wace dabara ce ke ba da damar Flash Attention yayi lissafin softmax daidai ba tare da ganin layin gaba ɗaya ba?
Software na kan layi na softmax yana riƙe matsakaicin matsakaicin aiki da ƙididdige ƙididdigewa yayin watsa fale-falen fale-falen fale-falen buraka, yana sake fasalin abubuwan da suka gabata a baya don daidaitawa na ƙarshe daidai ne.
Me yasa Hankalin Flash ya fi taimakawa tare da dogon jeri?
Matrix na hankali na hankali yana daidaita ma'auni kamar N-squared a cikin ƙwaƙwalwar ajiya, don haka guje wa cikakken ajiyarsa yana haifar da babban tanadi daidai lokacin da jerin suna da tsawo.
Menene ƙirar 'IO-sane' na Hankalin Flash ke ba da fifikon rage girman?
IO-sane yana nufin algorithm an tsara shi a kusa da farashin motsin bayanai a cikin tsarin ƙwaƙwalwar ajiya, rage zirga-zirgar HBM maimakon ayyukan lissafi.