Dubawa
Lookahead da Lion su ne juyi na zamani guda biyu akan inganta hanyar sadarwa. Lookhead yana kunshe duk wani mai inganta tushe tare da 'hankali' da 'sauri' ma'aunin nauyi don ƙarin kwanciyar hankali, yayin da Lion (EvoLved Sign Momentum) ya gano ta hanyar binciken shirin AI kuma yana sabunta ma'auni ta amfani da alamar lokaci kawai - yana mai da shi ƙwaƙwalwar ajiya-haske kuma sau da yawa sauri fiye da Adam.
Lookahead da Lion Optimizers wani shingen ginin fasaha ne wanda ke shafar ingancin samfuri, farashin kayayyakin more rayuwa, latency, da aminci a sikeli.
Zurfafa nutsewa
Lookahead, wanda Zhang, Hinton da abokan aiki suka gabatar a shekarar 2019, yana gudanar da ma'auni na 'sauri' mai ingantawa (kamar Adam ko SGD) don matakan k, sannan ya nudge wani nau'in ma'aunin nauyi na 'hankali' kadan daga hanyar zuwa inda ma'aunin sauri ya ƙare. Wannan yana damun oscillations kuma yana rage hankali ga hyperparameters. Zaki, wanda Google ya buga a cikin 2023, ya fito daga bincike na alama akan abubuwan ingantawa. Yana bin ƙwaƙƙwara amma yana aiwatar da aikin alamar zuwa sabuntawa, don haka kowane ma'auni yana motsawa ta ƙayyadadden girman mataki zuwa ga tarin alamar gradient. Zaki yana adana madaidaicin buffer kawai (rabin jihar Adamu, wanda ke riƙe biyu), yana amfani da lalata nauyi mafi girma da ƙarancin koyo, kuma ya daidaita ko ya doke Adamu akan manyan hangen nesa da ƙirar harshe yayin horo cikin sauri da rahusa.
Fahimtar Fasaha
Duba kai: bayan k matakai masu sauri suna samar da ma'auni θ_sauri, jinkirin ma'aunin nauyi yana tafiya kamar φ ← φ + α(θ_fast - φ), sannan mai saurin ingantawa yana sake saitawa zuwa φ. Sabuntawar zaki: m ← β1·m + (1-β1) · g don interpolation, amma matakin nauyi shine θ ← θ - η · (alama (β2 · m + (1-β2) · g) + λθ). Aikin alamar yana yin yumfom ɗin ɗaukaka sabuntawar kowane haɗin gwiwa, wanda ke aiki kamar daidaitacce kuma yana bayyana dalilin da yasa zaki yana buƙatar ƙaramin koyo fiye da Adamu.
Mastering Lookhead and Lion Optimizers
Lookahead da Lion su ne juyi na zamani guda biyu akan inganta hanyar sadarwa. Lookhead yana kunshe duk wani mai inganta tushe tare da 'hankali' da 'sauri' ma'aunin nauyi don ƙarin kwanciyar hankali, yayin da Lion (EvoLved Sign Momentum) ya gano ta hanyar binciken shirin AI kuma yana sabunta ma'auni ta amfani da alamar lokaci kawai - yana mai da shi ƙwaƙwalwar ajiya-haske kuma sau da yawa sauri fiye da Adam. Lookahead da Lion Optimizers wani shingen ginin fasaha ne wanda ke shafar ingancin samfuri, farashin kayayyakin more rayuwa, latency, da aminci a sikeli. Don gina zurfin fahimta, kula da Lookahead da Lion Optimizers a matsayin samfurin aiki, ba fasali ɗaya ba: ayyana sakamakon da ake so, fayyace zato, da raba abin da tsarin zai iya yi da dogaro daga abin da har yanzu yana buƙatar yanke hukunci na ƙwararru.
A aikace, ƙungiyoyi masu ƙarfi da ke amfani da Lookahead da Lion Optimizers suna haɓaka gine-gine, bayanai, da zaɓin abubuwan more rayuwa akan dogaro da farashi. Suna rubuta ƙayyadaddun ƙa'idodin nasara, gwaji akan bayanan gaskiya da gudanawar aiki, da jujjuyawar bisa ga tsarin gazawar da aka lura maimakon cin nasara na lokaci ɗaya. Wannan shine inda fahimtar ka'idar ta juya zuwa iyawa mai dorewa a cikin samfura, manufofi, da ayyuka.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A lokaci guda, Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin. Hanyar da ta fi dacewa ita ce haɗa saurin gwaji tare da horon gudanarwa: gudanar da matukin jirgi, kama shaida, buga rajistan ayyukan yanke shawara, da ci gaba da sabunta abubuwan tsaro kamar yadda halayen ƙira, tsammanin mai amfani, da buƙatun tsari ke tasowa.
Dabarun Tasiri
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Aiwatar da Gaskiyar Duniya
Rufe Adam tare da Lookhead don daidaita horar da masu canza wuta da rage kokarin daidaita hyperparameter.
Amfani da Lion don horar da manyan nau'ikan hangen nesa (misali, ViT) tare da ƙananan ƙwaƙwalwar ingantawa fiye da Adam.
Horar da ƙirar harshe tare da Lion don cimma daidaito mai kama da ƙarancin ƙididdige farashi.
Haɗa Lookahead tare da SGD a cikin wakilai masu ƙarfafawa-koyo don sassauƙar sabunta manufofin hayaniya.
Hanyoyin Aiwatarwa
Lookhead and Lion Optimizers a aikace
Rufe Adam tare da Lookhead don daidaita horar da masu canza wuta da rage kokarin daidaita hyperparameter.
Rufe Adam tare da Lookahead don daidaita horar da masu canji da rage ƙoƙarin daidaita hyperparameter Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ƙofofin inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da bin diddigin nasarorin samarwa da ƙimar kuskure akan lokaci.
Lookhead and Lion Optimizers a aikace
Amfani da Lion don horar da manyan nau'ikan hangen nesa (misali, ViT) tare da ƙananan ƙwaƙwalwar ingantawa fiye da Adam.
Yin amfani da Lion don horar da manyan nau'ikan hangen nesa (misali, ViT) tare da ƙananan ƙwaƙwalwar haɓakawa fiye da Ƙungiyoyin Adam yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ƙofofin inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da bin diddigin abubuwan samarwa da ƙimar kuskure akan lokaci.
Lookhead and Lion Optimizers a aikace
Horar da ƙirar harshe tare da Lion don cimma daidaito mai kama da ƙarancin ƙididdige farashi.
Ƙaddamar da ƙirar harshe tare da Lion don cimma daidaitattun daidaito a rage ƙimar ƙididdigewa Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da bin diddigin nasarorin samarwa da ƙimar kuskure a kan lokaci.
Lookhead and Lion Optimizers a aikace
Haɗa Lookahead tare da SGD a cikin wakilai masu ƙarfafawa-koyo don sassauƙar sabunta manufofin hayaniya.
Haɗa Lookahead tare da SGD a cikin wakilai na ƙarfafawa-koyo don sassaukar sabunta manufofin manufofin Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ƙofofin inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da bin diddigin nasarorin samarwa da ƙimar kuskure akan lokaci.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.