Jadwalka diirimaadka iyo Cosine Annealing
Diirimaadku wuxuu si tartiib ah kor ugu qaadayaa heerka waxbarashada isagoo u dhow eber ka hor tababarka, ka dibna jilitaanka cosine-ka ayaa si habsami leh u burburisa iyada oo raacaysa qalooca cosine.
Dulmar
Together they stabilize early training and squeeze out better final accuracy, which is why nearly every modern transformer is trained this way.
quusid qoto dheer
Marka tababarku bilaabmo, miisaannada moodeelku waa random iyo jaangooyooyinku waxay noqon karaan kuwo aad u weyn, markaa si toos ah ugu boodboodka heerka waxbarasho ee weyn waxay badanaa keentaa luminta ama kala duwanaansho - gaar ahaan kuwa wax hagaajiya sida Adam, kuwaas oo qiyaasahooda kala duwanaanshuhu aan la isku halayn karin tallaabooyinka ugu horreeya. Warmup ayaa tan hagaajiya iyadoo si toos ah u kordhinaysa heerka dhawr boqol ilaa dhawr kun oo tallaabo. Marka moodelku uu ku socdo cag xasiloon, xinjirowga cosine ayaa la wareegaya, iyada oo hoos u dhigaysa heerka 0.5 * (1 + cos (pi * t / T)) ee ugu sarreeya. Qaabka cosine-ku wuxuu xajiyaa heerka sare ee horumarka degdega ah, ka dibna wuu fududeeyaa si tartiib tartiib ah si ay u wanaajiyaha u dejiyo ugu yaraan wanaagsan halkii uu ka soo boodboodi lahaa hareeraheeda.
Aragtida Farsamada
Annealing cosine waxay qiyaastaa heerka waxbarashada 0.5 * (1 + cos(pi * t / T)), halkaas oo t ay tahay tillaabada hadda iyo T waa wadarta guud. Tani waxay wakhti dheer ku qaadataa meel u dhow heerka ugu sarreeya, waxay u qudhuntaa sida ugu dhaqsaha badan ee dhexda, ka dibna waxay ku sii simaan meel eber u dhow dhammaadka - si ka duwan suuska toosan ee toosan. Diirimaadku caadi ahaan waa toosan oo gaaban. Qalooca la isku daray wuxuu u eg yahay buuro siman: kor, plateau-ish, ka dibna dhalaalid jilicsan oo ku dhow eber.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Jadwalada Diirinta iyo Cosine Annealing
Warmup-plus-cosine ayaa weli ah habka caadiga ah ee moodooyinka luqadaha waaweyn, laakiin kala duwanaanshiyaha ayaa sii faafaya. Warmup-stable-decay (WSD) waxa ay haysaa heer joogto ah ka dibna si fiican ayey u qudhuntaa dhammaadka, taas oo sahlaysa in la dheereeyo orodyada iyada oo aan dib loo dhigin dherer go'an. Cilmi-baadhayaashu waxay sidoo kale baranayaan sababta diirimaadka u shaqeeyo - iyada oo lagu xirayo qaylada tartiib-tartiib ah iyo qalooca muuqaalka-luminta - iyo qalabyada si isa soo taraya u hagaajinaya dhererka kuleylka iyo heerka ugu sarreeya, taas oo yaraynaysa tijaabada-iyo-qallada gacanta ee maanta xukuma.
Dhaqangelinta Adduunka-dhabta ah
Moodooyinka luuqada GPT-style iyo qaabka BERT waxay adeegsadaan kululaynta toosan ee u horaysa ~ 1-2% ee tillaabooyinka ay raacdo suuska cosine ilaa eber u dhow.
Transformers Vision (ViT) waxay ku tabobarayaan soo saarista cosine iyo kulayl gaaban si looga fogaado kala duwanaanshaha hore ee ImageNet.
Isku-habaynta Wajiga Transformers waxay bixisaa 'hel_cosine_schedule_with_warmup' sidii jadwal hal-line ah oo loogu talagalay shaqooyinka hagaajinta wanaagsan.
Faafinta Xasilloonida ah iyo moodooyinka kale ee faafinta-ku-dheellitirka kulaylka si looga hortago qaraxyo is-jiid-jiid ah marka la qabsanayo miisaan la sii tababbaray.
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Warmup and Cosine Annealing Schedules quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Yaraynta Fiican-ka warqabka
Su'aalaha soo noqnoqda
What is Warmup and Cosine Annealing Schedules?
Diirimaadku wuxuu si tartiib ah kor ugu qaadayaa heerka waxbarashada isagoo u dhow eber ka hor tababarka, ka dibna jilitaanka cosine-ka ayaa si habsami leh u burburisa iyada oo raacaysa qalooca cosine. Si wada jir ah ayay u dejiyaan tababbarka hore oo ay ka saaraan saxnaanta kama dambaysta ah, taas oo ah sababta ku dhawaad qalab kasta oo casri ah sidan loogu tababaro.
Waa maxay ujeedada ugu weyn ee wejiga kululaynta ee bilowga tababarka?
Bilaabidda heerka barasho weyn ee miisaanka aan tooska ahayn waxay sababi kartaa lumay ama kala duwanaansho, markaa kulaylku wuxuu kor u qaadayaa heerka si tartiib ah si loo ilaaliyo tallaabooyinka hore.
Annealing cosine waxa ay kharribasaa heerka waxbarashada ee raacaya qaabkee?
Sicirka waxaa lagu qiyaasay 0.5 * (1 + cos(pi * t / T)), taasoo soo saarta buur siman oo joogsata goor hore oo ku soo dhowaanaysa eber dhamaadka.
Kee baa ka faa'iidaysanaya kululaynta sababtoo ah qiyaasaha kala duwanaanshaheeda ayaa ah kuwa aan la isku halayn karin goor hore?
Qiyaasta kala duwanaanta orod ee Aadan waxay ku salaysan tahay muunado aad u yar tallaabooyinka hore, taasoo ka dhigaysa cabbirka tallaabada wax ku oolka ah mid khaldan - kululaynta ayaa tan yaraynaysaa.
Qaabka cosine-ka, muxuu T ka dhigan yahay?
T waa cirifka ka sarreeya kaas oo qiimihiisu hoos u dhaco marka loo eego heerka ugu sarreeya una dhow eber; t waa talaabada hadda jirta
Waa maxay mid ka mid ah faa'iidooyinka kala duwanaanshaha-kuleelka-stable- suus (WSD) ee ka sarreeya geedka dhererka go'an?
WSD waxay haysaa heer joogto ah ka dibna si xoog leh ayay u qudhuntaa dhammaadka, si aad u sii wadi karto marxaladda xasilloon oo aad go'aansato barta joogsiga dambe.