Lookahead iyo Lion Optimizers
Lookahead iyo Lion waa laba maroojin casri ah oo ku saabsan hagaajinta shabakada neural-ka.
Dulmar
Lookahead waxay ku duubtaa wanaajinta saldhig kasta oo leh 'gaabis' iyo 'dhakhso leh' miisaan si loo horumariyo xasilloonida, halka Libaaxa (EvoLved Sign Momentum) uu daahfuray barnaamijka AI oo cusboonaysiiyay miisaanka iyada oo la adeegsanayo kaliya calaamadda erey dardargelin ah - taasoo ka dhigaysa xusuusta-iftiin oo inta badan ka dhaqso badan Adam.
quusid qoto dheer
Lookahead, oo ay soo jeediyeen Zhang, Hinton iyo asxaabtiisa sanadka 2019, waxa uu wadaa wax qabad heer 'dhakhso leh' (sida Adam ama SGD) ee talaabooyinka k, ka dib waxa uu nugladaa qayb gooni ah oo miisaan ' gaabis ah qayb ka mid ah dariiqa uu ku dhamaanayo miisaanka degdega ah. Tani waxay yaraynaysaa rogrogmada waxayna yaraynaysaa dareenka hyperparameters. Libaax, oo uu daabacay Google sanadka 2023, waxa uu ka soo baxay raadinta barnaamijka sumadda ee algorithmizers. Waxay la socotaa xawliga laakiin waxay ku dabaqdaa shaqada calaamada cusboonaysiinta, sidaa darteed halbeeg kasta wuxuu ku dhaqaaqaa cabbir go'an oo u socda calaamadda jaan-goynta urursan. Libaaxa waxa uu kaydiyaa kaydka degdega ah (dalka Aadan badhkii, kaas oo laba ilaashanaya), waxa uu isticmaalaa qudhunka miisaanka weyn iyo heerka waxbarasho oo yar, waxana uu ku beegay ama ku garaacay Aadam aragga waaweyn iyo moodooyinka luqadda isaga oo si degdeg ah u tababaranaya.
Aragtida Farsamada
Cusboonaysiinta madax-fiirinta: ka dib k tillaabooyinka degdega ah ee soo saaraya miisaannada θ_dhakhso ah, miisaannada tartiib tartiib ah waxay u dhaqaaqaan sida φ ← φ + α(θ_fast - φ), ka bacdi horumariyaha degdega ah wuxuu dib ugu noqdaa φ. Cusboonaysiinta libaaxa: m ← β1·m + (1-β1) · g isdhexgalka, laakiin talaabada miisaanka waa θ ← θ - η · (calaamad (β2·m + (1-β2) · g) + λθ). Hawlgalka calaamaduhu waxa uu ka dhigayaa iskudubarid kasta lebbiskiisa cusboonaysiinta cabbirka, kaas oo u dhaqma sidii caadiga ahayd oo kale oo sharraxaysa sababta Libaaxu ugu baahan yahay heerka waxbarasho ka yar kan Aadan.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Lookhead iyo Libaax Optimizers
Libaaxa waxaa lagu qaatay dhowr tabobar oo baaxad leh maxaa yeelay waxa ay dhimaysaa xusuusta wax qabad waxayna dedejin kartaa isku-xidhnaanta, daah-furkeeduna waxa uu soo bandhigaa raadinta otomaatigga ah ee 'AI-designing-AI' algorithm oo ah isha dhabta ah ee faa'iidooyinka la taaban karo. Filo wanaajiyaal badan oo raadinta laga soo qaatay, qorshayaal isku-dhafan oo isku daraya qaabka-Lookahead-qaabka miisaanka tartiibka ah iyo cusbooneysiin ku saleysan calaamadda, iyo xiisaha sii kordhaya ee wax qabadyada xusuusta waxtarka leh maadaama cabbirrada moodelku ay sii adkeynayaan miisaaniyadda xusuusta GPU.
Dhaqangelinta Adduunka-dhabta ah
Ku-duridda Adam ee loo yaqaan 'Lookhead' si loo dejiyo tababbarka taransfoorayaasha oo loo yareeyo dadaalka hagaajinta hyperparameter.
Isticmaalka Lion si loo tababaro moodooyinka aragga waaweyn (tusaale, ViT) oo leh xusuusta wax-qabad ka hooseeya kan Adam.
Kahortababaridda moodooyinka luqadda ee Libaaxa si loo gaaro saxsanaan la mid ah qiimaha xisaabinta oo la dhimay.
Isku darka Lookahead iyo SGD ee wakiilada xoojinta-barashada si loo fududeeyo cusboonaysiinta siyaasadda buuqa.
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Lookahead and Lion Optimizers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
U-dejinta Gobolka Optimizer ee CPU iyo NVMe
Su'aalaha soo noqnoqda
Waa maxay Lookahead iyo Lion Optimizers?
Lookahead iyo Lion waa laba maroojin casri ah oo ku saabsan hagaajinta shabakada neural-ka. Lookahead waxay ku duubtaa wanaajiye kasta oo saldhig ah oo leh 'gaabis' iyo 'dhakhso leh' miisaan si loo horumariyo xasilloonida, halka Libaaxa (EvoLved Sign Momentum) uu helay barnaamijka AI raadinta wuxuuna cusbooneysiiyaa miisaanka iyadoo la adeegsanayo kaliya calaamadda erey firfircoon - taasoo ka dhigaysa xusuusta-iftiin oo badanaa ka dhaqso badan Adam.
Waa maxay tillaabada xisaabinta ee cusboonaynta miisaanka Libaaxa?
Libaaxu waxa uu ku dabaqaa shaqada calaamada erey-xakameyn is-dhexgal ah, sidaa awgeed halbeeg kasta waxa uu ku dhaqaaqaa tallaabo labis ah jihada calaamadda.
Sidee buu Lookhead u qaabeeyaa hagaajintiisa?
Lookahead waxa ay waddaa hagaajin gudaha degdega ah ee talaabooyinka k, dabadeed waxa uu dhaqaajiyaa miisaan tartiib tartiib ah oo ka mid ah dariiqa loo maro miisaannada degdega ah iyo dib u dejinta.
Sidee baa markii hore loo helay hagaajinta libaaxa?
Libaaxa (EvoLved Sign Momentum) waxa uu ka soo baxay nidaamka raadinta barnaamij kaas oo horumariyay lana qiimeeyay barnaamijyada tayaynta musharaxiinta.
Marka la barbardhigo Aadan, waa maxay faa'iidada xusuusta muhiimka ah ee libaaxa?
Aadan wuxuu raadraacaa daqiiqadaha koowaad iyo labaad labadaba; Libaaxu waxa uu hayaa hal mar oo kaliya, isaga oo qiyaas ahaan kala badh u dhigaya xusuusta gobolka wax hagaajiya.
Muxuu libaaxu caadi ahaan ugu baahan yahay waxbarasho ka yar kan Aadan?
Mar haddii shaqada calaamaddu ay hagaajiso baaxadda cusboonaysiinta iskudubarid kasta, tallaabada wax ku oolka ahi waa weyn tahay, markaa heerka waxbarashada oo ka yar (iyo miisaan weyn oo qudhun ah) ayaa la isticmaalaa.