Yawan Koyon Zagaye
Matsakaicin koyo na cyclical akai-akai suna sake zagayowar ƙimar koyo sama da ƙasa tsakanin ƙasa da babba maimakon lalata shi kawai.
Dubawa
This counterintuitive bouncing can speed up convergence and helps the optimizer escape sharp local minima and saddle points.
Zurfafa nutsewa
Leslie Smith ta gabatar a cikin 2015, ƙimar koyo na cyclical (CLR) ya ƙalubalanci tunanin cewa ƙimar ya kamata ya ragu kawai. Madadin haka, yana oscillates tsakanin ƙarami da matsakaicin iyaka akan ƙayyadadden adadin maimaitawa ('zagayowar' zagayowar'), sau da yawa tare da siffar triangular. Hankali: haɓaka ƙimar lokaci-lokaci yana ba da fashewar kuzari wanda zai ba da damar ƙirar ta yi tsalle daga matalauci, ƙarami mai kaifi da madaidaicin wuraren sirdi, yayin da ƙananan matakan ke barin shi daidaita. Smith kuma ya gabatar da 'gwajin kewayon LR' - ɗan gajeren gudu wanda ke share ƙimar sama yayin kallon asarar - don nemo iyakoki mai kyau ta atomatik. Triangular, triangular-tare da rugujewa, da kuma sanannen manufofin sake zagayowar daya duk sun ginu akan wannan ra'ayin.
Fahimtar Fasaha
Manufofin triangular a layi ɗaya suna ƙara ƙimar daga tushe zuwa max fiye da rabin zagayowar, sannan a layi-layi ta rage shi baya fiye da sauran rabin. Tsawon zagayowar yawanci ana saita shi zuwa ƙimar ƴan lokuta kaɗan na maimaitawa. Manufar sake zagayowar guda ɗaya tana amfani da zagayowar tsayi guda ɗaya: ƙimar ya tashi sannan ya faɗi ƙasa da farkon farawa, yayin da kuzari ke motsawa da ƙetare - babba lokacin da ƙimar ya yi ƙasa kuma akasin haka - wanda ke aiki azaman mai daidaitawa kuma yana ba da damar 'super-convergence' akan wasu ayyuka.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Ƙididdigar Koyon Juyi
Jadawalin cyclical da manufofin sake zagayowar sau ɗaya sun kasance sananne don horarwa cikin sauri akan hangen nesa da ayyuka na tebur, kuma gwajin kewayon LR daidaitaccen dabarar kunnawa ne. Don manyan nau'ikan harshe, santsin dumama-da-cosine jadawalai suna da rinjaye, amma fahimi - cewa dabarun haɓaka yana taimakawa tserewa mummunan yankuna na yanayin asara - yana ba da sanarwar sake kunnawa mai dumi (SGDR) da hanyoyin haɗakarwa waɗanda ke ɗaukar hoto a kowane ƙaramin zagayowar. Yi tsammanin ci gaba da rikice-rikice tsakanin ra'ayoyin kewayawa da daidaitawa, masu tsara jadawalin daidaitawa.
Aiwatar da Gaskiyar Duniya
fast.ai ya yada manufar sake zagayowar sau ɗaya a matsayin tsoho don horar da masu rarraba hoto da sauri zuwa babban daidaito cikin ƴan zamani.
Gwajin kewayon LR yana share ƙimar sama sama da ƴan batches ɗari don ɗaukar min da max iyakoki kafin gudu na gaske.
Haɗin hoto yana adana wurin bincike na ƙira a ƙarshen kowane zagayowar, yana samar da tarin kyauta daga gudu ɗaya na horo.
Saukowar Stochastic Gradient tare da Sake farawa Dumi (SGDR) lokaci-lokaci yana sake saita ƙimar zuwa babban ƙima don tserewa ƙarami mai kaifi.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Cyclical Learning Rates quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Jadawalin Yawan Koyo
Tambayoyin da ake yawan yi
What is Cyclical Learning Rates?
Matsakaicin koyo na cyclical akai-akai suna sake zagayowar ƙimar koyo sama da ƙasa tsakanin ƙasa da babba maimakon lalata shi kawai. Wannan bouncing na rashin fahimta na iya hanzarta haɗuwa kuma yana taimaka wa mai haɓakawa tserewa ƙaƙƙarfan ƙaramar gida da wuraren sirdi.
Wane babban zato ne ke ƙalubalantar ƙimar koyo na cyclical?
CLR da gangan yana ɗaga ƙima akai-akai, yana ƙin ra'ayin gargajiya cewa dole ne kawai ya lalace kawai.
Me yasa ƙara ƙimar koyo lokaci-lokaci zai taimaka?
Fashe mafi girma na iya tura mai ingantawa daga matalauta, kaifi ƙarami ko kashe wuraren sirdi don samun ingantattun yankuna.
Menene 'gwajin kewayon LR' ke yi?
Yana gudana a taƙaice tare da a hankali haɓaka ƙimar; Madaidaicin hasara yana bayyana mafi ƙarancin hankali da matsakaicin don amfani da hawan keke.
A cikin manufar sake zagayowar sau ɗaya, ta yaya ƙwaƙƙwaran ƙaƙƙarfan ya shafi ƙimar koyo?
Manufofin sake zagayowar sau ɗaya yana rage ƙarfi yayin da ƙimar ke ƙaruwa kuma yana ɗaga shi yayin da adadin ya ragu, wanda ke ƙara tasirin daidaitawa.
Menene 'haɗuwar hoto' a cikin mahallin jadawali na zagaye?
Domin kowane zagayowar ya daidaita zuwa mafi ƙarancin mabambanta, adana waɗannan wuraren binciken yana haifar da nau'ikan nau'ikan nau'ikan nau'ikan gudu ɗaya waɗanda za'a iya haɗa su.