Ịhazi ọnụ ọgụgụ mmụta
Usoro ọnụego mmụta na-agbanwe nha nzọụkwụ n'oge ọzụzụ kama ijide ya.
Nchịkọta
Getting it right is often the single biggest lever for whether a model converges quickly and reaches high accuracy.
Ime miri emi
Ọnụego mmụta na-achịkwa otú nnukwu nzọụkwụ onye na-ebuli elu si ewe mmelite ọ bụla. Oke elu na ọzụzụ diverges; dị oke ala ma ọ na-arara ma ọ bụ rapara. Ịhazi oge na-agbanwe uru a ka oge na-aga. Ntụziaka ọgbara ọhụrụ a na-emekarị bụ ikpo ọkụ na-esochi ire ere: malite nso efu wee gbagote n'elu narị ole na ole ma ọ bụ puku nzọụkwụ mbụ (ya mere n'isi ụtụtụ, gradients na-eme mkpọtụ anaghị afụ ụfụ na-adịghị akwụ ụgwọ), wee jiri nwayọọ nwayọọ belata. Ụdị ire ere na-ewu ewu na-agụnye ire ere nzọụkwụ (dabere site n'otu ihe na oge a na-edozi), ire ere na-egbuke egbuke, na annealing cosine, nke na-esochi usoro ọkara-cosine nke ọma ruo na nso efu. Nhazi oge cosine nwere ikpo ọkụ linear bụ ọkọlọtọ maka ịzụ ụdị asụsụ buru ibu, ebe atumatu cyclical na otu okirikiri nwere ike mee ka ọzụzụ dị nta dị ngwa.
Nghọta nka nka
Ihe na-ekpo ọkụ n'ihi na ndị na-eme mgbanwe dị ka Adam nwere atụmatụ oge nke abụọ na-enweghị ntụkwasị obi na nzọụkwụ mbụ; Obere mmụta ọnụego na-ezere imebi ịdị arọ tupu ọnụ ọgụgụ ndị ahụ edozi. Cosine annealing sets lr = lr_min + 0.5 * (lr_max - lr_min) * (1 + cos(pi * t / T)), na-enye ọganihu ngwa ngwa n'isi na obere, na-edozi usoro dị mma nso na njedebe. Ụfọdụ nhazi oge na-agbakwunye mmalite ọkụ, na-awụghachi ọnụego azụ iji gbanarị minima dị nkọ.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke nhazi ọnụego mmụta
Ka ọzụzụ na-aga n'ihu na-abawanye ọnụ, a na-ahazi usoro nhazi ya na ndị na-eme ka ọ dị mma na nha batch, ndị na-eme nchọpụta na-amụkwa iwu scaling iji buru amụma ọnụ ọgụgụ kasị mma tupu ọzụzụ. Ndị na-emezigharị oge na-enweghị usoro nke na-ewepụ mkpa ọ dị ịhọrọ ụzọ ire ere tupu oge eruo na-enweta traction, na mgbanwe, nhazi usoro nzaghachi nke na-anabata usoro ọnwụ ndụ nwere ike ibelata ule-na-njehie nke ka na-achịkwa ọzụzụ buru ibu.
Mmejuputa n'ezie n'ụwa
A na-eji ikpo ọkụ Linear gbakwunyere ire ere cosine mgbe a na-azụ ụdị asụsụ mgbanwe.
Nrụrụ nzọụkwụ nke na-agbada ọnụego mmụta 10x na oge 30, 60, na 90 mgbe ị na-azụ ndị nhazi onyonyo na ImageNet.
Amụma otu okirikiri na fast.ai iji zụọ ihe nlereanya ka ọ dị mma n'ime oge ole na ole.
Annealing Cosine na-ekpo ọkụ malitegharịa iji gbanarị obere mfu dị nkọ ma melite mkpokọta.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Learning Rate Scheduling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ọnụego mmụta cyclical
Ajụjụ a na-ajụkarị
What is Learning Rate Scheduling?
Usoro ọnụego mmụta na-agbanwe nha nzọụkwụ n'oge ọzụzụ kama ijide ya. Inweta ya nke ọma na-abụkarị otu nnukwu lever maka ma ihe nlereanya na-agbakọta ngwa ngwa wee rute oke ziri ezi.
Gịnị bụ ebumnobi nke usoro 'ekpo ọkụ' na nhazi ọnụego mmụta?
Igwe ọkụ na-amalite nso efu ma na-abawanye ọnụego ka mmelite mmalite na-adịghị akwụsị akwụsị ghara imebi ihe nlereanya ahụ tupu ọnụ ọgụgụ njikarịcha edozi.
Kedu usoro nhazi nke na-esote usoro ọkara-cosine gbadaa na opekempe?
Annealing cosine na-emebi ọnụ ọgụgụ mmụta n'ogo ọkara cosine, na-enye nnukwu nzọụkwụ n'isi na obere nzọụkwụ nso na njedebe.
Kedu ihe ga - eme ma ọ bụrụ na edobere ọnụego mmụta dị oke oke?
Ọnụego dị elu gabigara ókè na-akpata ịgbaghara, ya mere mfu ahụ nwere ike ibili gburugburu ma ọ bụ fesa kama idozi.
Kedu ihe nrụrụ ụkwụ na-eme na ọnụego mmụta?
Nrụrụ nzọụkwụ na-amụba ọnụego site n'otu ihe (dịka ọmụmaatụ, 0.1) n'ihe dị ịrịba ama a họọrọ, na-emepụta ụkpụrụ steepụ.
Kedu ihe kpatara eji agbakwunye 'mmalite ọkụ' mgbe ụfọdụ na nhazi oge?
Malitegharịa ekwentị na-ekpo ọkụ na-awụli elu n'ịmụ ihe n'oge oge, na-enyere onye na-ebupụta ihe pụọ na obere obere warara wee chọpụta azịza ka mma.