Daidaita Layer
Daidaita Layer yana daidaita horo ta hanyar sake fasalin kunnawa a cikin kowane misali don haka ba su da ma'ana da bambancin raka'a.
Dubawa
It is a quiet but essential ingredient that makes deep transformers trainable.
Zurfafa nutsewa
Ba, Kiros, da Hinton ne suka gabatar da su a cikin 2016, daidaitawar Layer (LayerNorm) yana magance matsalar cewa kunnawa a cikin cibiyar sadarwa mai zurfi na iya yin tafiya zuwa ma'auni daban-daban yayin da sigina ke wucewa ta cikin yadudduka da yawa, raguwa ko lalata koyo. Ba kamar daidaitawar tsari ba, wanda ke daidaita kowane fasali a cikin misalan a cikin ƙaramin tsari, LayerNorm yana daidaita fasalin fasalin misali guda ɗaya. Wannan ya sa ya zama mai zaman kansa ba tare da girman batch kuma ana iya amfani da shi daidai a horo da ƙididdigewa, kuma yana aiki ta dabi'a tare da jeri mai tsayi, wanda shine dalilin da ya sa ya zama ma'auni na masu canji masu ƙarfi da ke ba da ikon ƙirar harshe na zamani. Bayan daidaitawa, yana amfani da sikelin koyo (gamma) da kuma matsawa (beta) don haka hanyar sadarwar zata iya dawo da kowane wakilcin da take buƙata.
Fahimtar Fasaha
Don siffar vector x, LayerNorm yana ƙididdige ma'ana da bambance-bambancen akan waɗannan abubuwan vector, sannan ya fitar da gamma * (x - ma'ana) / sqrt (bambancin + epsilon) + beta. Saboda ƙididdiga sun fito daga samfurin guda ɗaya, hali iri ɗaya ne ko tsari yana da misalai 1 ko 1000. Bambanci mafi sauƙi, RMSNorm, tsallake yana nufin ragi da rarraba kawai ta tushen-ma'ana-square, adana lissafi; ana amfani dashi a cikin samfura kamar Llama. Sanyawa kuma yana da mahimmanci: 'pre-al'ada' (na daidaitawa a gaban kowane sublayer) yana sa masu canji mai zurfi da sauƙin horarwa fiye da 'post-norm'.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Daidaita Layer
Ana daidaita tsarin daidaitawa don dacewa a ma'auni. RMSNorm ya maye gurbin LayerNorm a cikin sababbin manyan harsunan harshe saboda yana da arha kuma yana aiki daidai, kuma pre-al'ada jeri yanzu shine tsoho don tarin zurfafa. Masu bincike suna ci gaba da binciken gine-gine marasa daidaituwa waɗanda ke amfani da farawa da hankali ko dabaru a maimakon, da nufin yanke sama yayin da suke kiyaye kwanciyar hankalin horon da daidaitawa ke samarwa.
Aiwatar da Gaskiyar Duniya
Tsayar da kowane toshe tafsiri a cikin nau'ikan yare kamar GPT da BERT.
Bayar da RMSNorm azaman zaɓin daidaitawa mafi sauƙi a cikin ƙirar dangin Llama.
Daidaita bayanan jeri mai canzawa a cikin magana da ƙirar fassarar inda girman tsari ya bambanta.
Bada ingantaccen horo tare da girman tsari ɗaya, kamar a cikin wasu saitunan koyo na ƙarfafawa.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Layer Normalization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
RMSNorm da Pre-Layer Normalization
Tambayoyin da ake yawan yi
What is Layer Normalization?
Daidaita Layer yana daidaita horo ta hanyar sake fasalin kunnawa a cikin kowane misali don haka ba su da ma'ana da bambancin raka'a. Abu ne mai natsuwa amma mai mahimmanci wanda ke sa zurfin tasfofi su iya horarwa.
A cikin mene ne daidaitawar Layer ke ƙididdige ma'anarsa da bambancinsa?
LayerNorm yana daidaita girman fasalin samfurin mutum ɗaya, yana mai da shi mai zaman kansa daga sauran misalan a cikin tsari.
Me yasa aka fi son daidaita Layer fiye da daidaita tsari a cikin tasfofi?
Saboda kididdigar sa sun fito daga misali guda ɗaya, LayerNorm yana aiki akai-akai ba tare da la'akari da girman tsari ba kuma ya dace da jerin rubutu masu tsayi.
Menene sigogin da ake iya koyo gamma da beta suka bar LayerNorm suyi?
Bayan daidaitawa zuwa ma'anar sifili da bambance-bambancen raka'a, ma'aunin gamma da beta suna canza sakamakon don haka ba a tilasta ƙirar cikin ƙayyadadden rarrabawa ba.
Ta yaya RMSNorm ya bambanta da daidaitaccen LayerNorm?
RMSNorm yana barin ma'auni mai ma'ana da ma'auni ta tushen-ma'ana-square na kunnawa, wanda ya fi rahusa kuma ana amfani dashi a cikin ƙira kamar Llama.
Wace matsala ta daidaita Layer yana taimakawa da farko a cikin hanyoyin sadarwa masu zurfi?
Yayin da sigina ke ratsa cikin yadudduka da yawa ma'auninsu na iya busawa ko raguwa; daidaitawa yana kiyaye kunnawa a cikin tsayayyen kewayo don haka gradients suna da kyau.