Caadiyan lakabka
Caadiyan lakabka waxay dejisaa tababarka iyadoo dib u habeyn ku sameyneysa dhaqdhaqaaqa ku jira tusaale kasta si ay u yeeshaan kala duwanaansho eber iyo unug.
Dulmar
It is a quiet but essential ingredient that makes deep transformers trainable.
quusid qoto dheer
Waxaa soo bandhigay Ba, Kiros, iyo Hinton 2016, lakabka caadiga ah (LayerNorm) wuxuu wax ka qabtaa dhibaatada in dhaqdhaqaaqa gudaha shabakad qoto dheer ay u gudbi karto miisaan kala duwan marka calaamadaha ay maraan lakabyo badan, gaabinaya ama xasilloonida waxbarashada. Si ka duwan sidii caadiga ahayd ee dufcaddii, taas oo caadi ka dhigaysa sifa kasta oo ka mid ah tusaalooyinka dufcad-yar, LayerNorm waxay caadi ka dhigtaa dhammaan sifooyinka hal tusaale. Tani waxay ka dhigaysaa mid ka madax bannaan cabbirka dufcadda oo si siman looga isticmaali karo tababarka iyo soo-jeedinta, waxayna si dabiici ah ugu shaqeysaa taxanaha dhererka doorsoomayaasha ah, waana sababta ay u noqotay halbeegga transformers-ka awood u leh moodooyinka luqadda casriga ah. Caadiyeynta ka dib, waxay khusaysaa cabirka la baran karo (gamma) iyo shift (beta) si ay shabakadu u soo ceshato matalaad kasta oo ay u baahan tahay.
Aragtida Farsamada
Sifada x, LayerNorm waxa ay xisaabisaa celceliska iyo kala duwanaanshaha canaasiirtaas vector, ka dib waxa ay soo saartaa gamma * (x - mean) / sqrt( duwanaansho + epsilon) + beta. Sababtoo ah tirakoobyadu waxay ka yimaadeen hal muunad, habdhaqanku waa isku mid haddii dufcaddu leedahay 1 ama 1000 tusaale. Kala duwanaansho fudud, RMSNorm, boodadu waxay ka dhigan tahay kala-goyn oo u qaybiya oo keliya xidid-celcelis-square, kaydinta xisaabinta; waxaa loo isticmaalaa moodooyinka sida Llama. Meelaynta sidoo kale waa arrin: 'hore-caadiga' (caadiyan ka hor horraysiiye kasta) ayaa ka dhigaysa transformers qoto dheer in la tababaro si ka badan 'post-norm'.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Caadiga Lakabka
Caadiyan ayaa loo habeeyey si waxtarka loo cabbiro. RMSNorm waxa ay si weyn u bedeshay LayerNorm ee moodooyinka cusub ee luqadaha waaweyn sababtoo ah way ka jaban tahay waxayna u shaqaysaa si la mid ah, iyo meelaynta caadiga ah ee hore ayaa hadda ah meesha ugu hooseysa ee xirmooyinka aadka u qoto dheer. Cilmi-baadhayaashu waxay sii wadaan sahaminta qaab-dhismeedyada xorta ah ee caadiga ah kuwaas oo isticmaala bilawga taxaddarka leh ama khiyaamaynta, taas beddelkeeda, iyaga oo ujeedadoodu tahay in ay gooyaan korka iyagoo ilaalinaya xasilloonida tababarka ee caadiga ah waxay bixiyaan.
Dhaqangelinta Adduunka-dhabta ah
Dejinta xannibaad kasta oo beddelka ah ee noocyada luqadaha sida GPT iyo BERT.
Awood u siinta RMSNorm sidii doorashada caadiga ah ee fudud gudaha moodooyinka qoyska Llama.
Caadiyeeynta xogta isku xigxiga dhererka doorsooma ee moodooyinka hadalka iyo tarjumaadda halkaasoo cabbirrada dufcaddu ay ku kala duwan yihiin.
Oggolaanshaha tabobar la isku halayn karo oo leh cabbir hal dufcood ah, sida qaar ka mid ah dejinta waxbarashada xoojinta.
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Layer Normalization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
RMSNorm iyo Caadiyeynta Lakabka Kahor
Su'aalaha soo noqnoqda
What is Layer Normalization?
Caadiyan lakabka waxay dejisaa tababarka iyadoo dib u habeyn ku sameyneysa dhaqdhaqaaqa ku jira tusaale kasta si ay u yeeshaan kala duwanaansho eber iyo unug. Waa shay degan laakiin lama huraan ah oo ka dhigaysa transformers qoto dheer kuwa la tababari karo.
Isku soo wada duuboo waa maxay caadiynta lakabku waxay xisaabisaa macnaha iyo kala duwanaanshaheeda?
LayerNorm waxay caadi ka dhigtaa cabbirka sifada hal muunad gaar ah, taasoo ka dhigaysa mid ka madax bannaan tusaalayaasha kale ee dufcadda.
Waa maxay sababta caadiga ah ee lakabka looga door biday caadi ka dhigista Dufcadaha ee Transformers?
Sababtoo ah tira-koobkeedu wuxuu ka yimid hal tusaale, LayerNorm wuxuu u dhaqmaa si joogto ah iyadoon loo eegin cabbirka dufcadda oo ku habboon taxanaha qoraalka dhererka doorsooma.
Maxay yihiin cabbirrada la baran karo gamma iyo beta u oggolaadaan LayerNorm?
Ka dib markii la caadaystay kala duwanaanshaha eber iyo kala duwanaanshaha unugga, miisaanka gamma iyo beta waxay beddelaan natiijada si moodeelka aan loogu qasbin qaybin go'an.
Sidee RMSNorm uga duwan yahay heerka LayerNorm?
RMSNorm waxay meesha ka saaraysaa tillaabada dhexdhexaadka ah iyo miisaanka xididka-celceliska-square ee firfircoonida, kaas oo ka jaban oo loo isticmaalo moodooyinka sida Llama.
Dhibaato noocee ah ayaa caadi ka dhigista lakabka ugu horrayn caawiyaa xallinta shabakadaha qoto dheer?
Marka ay calaamaduhu maraan lakabyo badan miisaankoodu wuu qarxi karaa ama wuu yaraan karaa; caadiyeyntu waxay ku haysaa hawl-qabadyadu meel deggan sidaa darteed gradients-ku si fiican ayey u dhaqmaan.