Hankalin Hanyoyi
Hankali ya bar samfurin ya yanke shawarar waɗanne kalmomi a cikin jimla mafi mahimmanci yayin fassara kowace kalma.
Dubawa
It is the core idea that made the transformer — and therefore modern AI like ChatGPT — possible.
Zurfafa nutsewa
Hankali yana amsa tambaya mai sauƙi ga kowace kalma: wadanne kalmomi zan duba don fahimtar wannan? Takardar 2017 'Attention Is All You Bue'' na Vaswani da abokan aiki a Google sun gabatar da na'urar taransifoma, wanda ke amfani da hankali a matsayin babban injinsa kuma yana sauke tsofaffin ƙira mai maimaitawa. Ana juya kowace alama zuwa nau'i uku: tambaya (menene nake nema?), maɓalli (menene zan bayar?), da ƙima (bayanin da nake ɗauka). Ana kwatanta tambayar alamar da kowane maɓalli na alamar don samar da ma'aunin hankali, wanda sannan ya haɗa dabi'u tare. Hankalin kai yana yin wannan a cikin jeri ɗaya don kowace kalma za ta iya shiga cikin kowace kalma kai tsaye. Hankalin kai-da-kai yana gudanar da irin waɗannan kwatancen da yawa a layi daya, kowanne yana mai da hankali kan alamu daban-daban.
Fahimtar Fasaha
Math ɗin yana ƙaddamar da hankali samfurin ɗigo: softmax(QK^T / √d_k) V. Samfurin dige-dige na tambayoyin da maɓalli ya ƙididdige yadda kowane nau'i biyu ke da alaƙa; Rarraba ta tushen murabba'in maɓalli na maɓalli (√d_k) yana kiyaye waɗannan maki daga girma da yawa; softmax yana juya su zuwa ma'aunin nauyi waɗanda jimlar zuwa ɗaya; da ninkawa ta V yana samar da ma'auni masu nauyi. Saboda kowane alamar yana kwatanta da kowane, farashi yana girma tare da murabba'in tsayin jeri - O (n²) - wanda shine dalilin da yasa dogon bayanai ke da tsada kuma me yasa ingantawa kamar FlashAttention ya wanzu.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Hanyoyi na Hankali
Hankali yana nan don tsayawa, amma farashin sa na quadratic yana haifar da bincike mai zurfi. FlashAttention ya sanya daidaitaccen kulawa cikin sauri da ingantaccen ƙwaƙwalwar ajiya ta hanyar sake yin oda. Sabbin kwatance sun haɗa da hankali da madaidaiciyar hankali, tattarawa da kulawar tambaya da yawa don rage ƙwaƙwalwar ajiya yayin tsarawa, da ƙirar ƙira waɗanda ke haɗa hankali tare da ƙirar sararin samaniya kamar Mamba don shigarwar dogon lokaci. Yi tsammanin tsarin nan gaba don kiyaye sassaucin hankali yayin lanƙwasa layin farashi ta yadda sarrafa tsawon littafin ko abubuwan bayanai masu yawa ya zama na yau da kullun kuma mai araha.
Aiwatar da Gaskiyar Duniya
Fassarar na'ura, inda samfurin ke halartar mahimman kalmomin tushe lokacin samar da kowace kalma da aka fassara.
Takaitawa, inda hankali ke taimaka wa ƙirar ta mayar da hankali kan mahimman jimloli a cikin dogon labarin.
Mataimakan lamba waɗanda ke zuwa baya zuwa ma'anar ma'anar ma'anar farko lokacin tsinkayar layi na gaba.
Amsar tambaya akan takarda, inda hankali ya danganta kalmomin tambaya zuwa sashin da ke ɗauke da amsar.
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Attention Mechanisms quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Hankalin Tagar Zamiya
Tambayoyin da ake yawan yi
What is Attention Mechanisms?
Hankali ya bar samfurin ya yanke shawarar waɗanne kalmomi a cikin jimla mafi mahimmanci yayin fassara kowace kalma. Babban ra'ayi ne ya sanya transformer - don haka AI na zamani kamar ChatGPT - mai yiwuwa.
A cikin jumla ɗaya, menene tsarin kulawa yake yi?
Hankali yana ƙididdige ma'auni waɗanda ke yanke shawarar nawa kowace alama za ta zana kan sauran alamun yayin samar da wakilcin sa.
Wace takarda ce ta 2017 ta gabatar da tsarin gine-ginen transfoma?
'Attention Shine Duk Bukatar Ku' (Vaswani et al., 2017) ya ba da shawarar injin mai canzawa, wanda ya dogara da hankali maimakon maimaitawa.
What are the three vectors computed for each token in attention?
Kowace alama tana samar da tambaya, maɓalli, da ƙima; Tambayoyi sun dace da maɓallan don auna ƙimar.
A cikin dabarar softmax (QK^T / √d_k) V, me yasa aka raba ta √d_k?
Ƙaddamar da tushen murabba'in maɓalli na maɓalli yana hana manyan samfuran ɗigo waɗanda za su tura softmax zuwa ƙananan gradients, kiyaye horarwa.
Me yasa daidaitaccen kulawar kai ya zama tsada don dogon bayanai?
Kowane alama yana halartar kowace alama, don haka adadin kwatancen ma'auni kamar n², yana yin dogon jeri mai tsada cikin lokaci da ƙwaƙwalwa.