FlashAttention
FlashAttention shine algorithm mai inganci mai ƙwaƙwalwar ajiya wanda ke ƙididdige ainihin hankali ɗaya kamar daidaitattun masu canji amma ba tare da taɓa rubuta babban matrix ɗin hankali don rage ƙwaƙwalwar GPU ba.
Dubawa
It made long-context training and inference dramatically faster and cheaper.
Zurfafa nutsewa
Daidaitaccen hankali yana ƙididdige maki ga kowane nau'i na alamomi, yana samar da matrix N-by-N. Don jerin alamar 4,000 wanda ke da maki miliyan 16, kuma dole ne a rubuta matrix ɗin zuwa kuma a karanta baya daga ƙwaƙwalwar babban bandwidth na GPU (HBM). Wannan zirga-zirgar ƙwaƙwalwar ajiya, ba lissafi ba, shine ainihin ƙulli. FlashAttention, wanda Tri Dao da abokan aiki suka gabatar a cikin 2022, yana sake fasalin lissafin don haka matrix ɗin ba zai taɓa zama cikakke ba. Yana aiwatar da jeri a cikin fale-falen fale-falen da suka dace a cikin ƙaramin GPU's, ultra-sauri akan guntu SRAM, lissafin softmax yana ƙaruwa yayin da yake tafiya. Sakamakon lissafin yayi kama da daidaitaccen kulawa amma yana amfani da ƙarancin ƙwaƙwalwar ajiya kuma yana tafiyar da sauri sau da yawa, yana ba da damar windows mahallin mai tsayi.
Fahimtar Fasaha
Dabarar ita ce 'online softmax' hade da tiling. FlashAttention yana ɗaukar ƙananan tubalan tambayoyi, maɓallai, da ƙima cikin SRAM, yana ƙididdige abubuwan da ake samu na hankali, kuma yana sake ƙididdige ƙimar gudu yayin da sabbin tubalan suka isa don haka daidaitawar softmax ya tsaya daidai ba tare da ganin duk maki lokaci ɗaya ba. Domin ba ya taɓa adana cikakken matrix N-by-N a cikin HBM, ƙwaƙwalwar ajiya tana daidaita layi maimakon quadratically, kuma an haɗa kernel cikin aikin GPU guda ɗaya don rage jinkirin karantawa da rubutu.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar FlashAttention
FlashAttention ya zama tsoho tubalan gini. FlashAttention-2 ya inganta rarrabuwar aikin GPU, kuma FlashAttention-3 yana amfani da sabbin fasalolin kayan masarufi na Hopper kamar asynchrony da ƙarancin madaidaicin FP8. Yi tsammanin ci gaba da ƙira tare da kwakwalwan kwamfuta, haɗin kai mai zurfi cikin sabobin ƙididdigewa don dogayen takardu, da bambance-bambancen da aka saurara don ƙarancin kulawar taga ko zamewa. Kamar yadda mahallin windows ke matsawa zuwa miliyoyin alamu, IO-sane kernels kamar wannan suna da mahimmanci don kiyaye horo da ba da kuɗin sarrafawa.
Aiwatar da Gaskiyar Duniya
Horar da manyan nau'ikan harshe kamar Llama da tsarin tsarin GPT cikin sauri kuma a ƙananan farashin GPU
Hidimar mataimakan taɗi na lokaci mai tsawo waɗanda ke cinye duka littattafai ko ma'auni ba tare da ƙarewar ƙwaƙwalwar ajiya ba
Ƙaddamar da bututun taƙaita daftarin aiki waɗanda ke sarrafa dubun-dubatar alamu a lokaci ɗaya
Ƙarfafa hangen nesa da masu canji na multimodal inda dogayen jeri na facin hoto ke sa hankali ya yi tsada
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the FlashAttention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Rukunin Matsayin Rotary
Tambayoyin da ake yawan yi
What is FlashAttention?
FlashAttention shine algorithm mai inganci mai ƙwaƙwalwar ajiya wanda ke ƙididdige ainihin hankali ɗaya kamar daidaitattun masu canji amma ba tare da taɓa rubuta babban matrix ɗin hankali don rage ƙwaƙwalwar GPU ba. Ya sanya horon yanayi mai tsawo da ƙima cikin sauri da rahusa.
Menene babban ƙulli FlashAttention hari a daidaitaccen kulawa?
Daidaitaccen hankali yana ɗaure ƙwaƙwalwar ajiya: rufe babbar matrix N-by-N zuwa kuma daga ƙwaƙwalwar bandwidth mai girma ta mamaye lokaci, ba lissafin kanta ba.
Ta yaya fitowar FlashAttention ya kwatanta da daidaitaccen kulawa?
FlashAttention yana lissafta ainihin ƙimar kulawa iri ɗaya; kawai yana sake tsara lissafin don gujewa yin cikakken matrix.
Wane ƙwaƙwalwar GPU ne FlashAttention ke amfani da shi ta hanyar sarrafa bayanai a cikin tayal?
FlashAttention yana ɗora ƙananan fale-falen fale-falen a cikin GPU's mai sauri akan guntu SRAM, yana kiyaye aiki mai nauyi kusa da raka'a.
Wace dabara ce ke ba FlashAttention damar lissafta softmax ba tare da ganin duk maki lokaci ɗaya ba?
Software na kan layi na softmax yana kula da iyakar gudu da jimla, yana sake ƙididdige sakamako na ɗan lokaci yayin da sabbin fale-falen fale-falen ya zo don daidaitawa na ƙarshe daidai.
Menene FlashAttention-3 musamman yayi amfani da shi?
FlashAttention-3 an tsara shi tare da Hopper GPUs na zamani, ta amfani da aiwatar da asynchronous da FP8 mara ƙanƙanta don ƙarin saurin gudu.