Adam na Adaptive Optimizers
Adam ni akazi keza cyane inyuma yimiyoboro myinshi igezweho, ihita ihuza igipimo cyo kwiga kuri buri kintu cyose.
Incamake
It matters because it makes training deep models faster and far less finicky than plain gradient descent.
Kwibira cyane
Adam (Adaptive Moment Estimation), yatangijwe na Kingma na Ba muri 2014, ihuza ibitekerezo bibiri. Ubwa mbere, umuvuduko: ituma impuzandengo yangirika yikigereranyo cya gradients zashize (umwanya wambere) bityo ivugurura ryubaka umuvuduko mubyerekezo bihamye. Icya kabiri, kuri buri kintu gipima: ikurikirana impuzandengo ya gradiyo ya kare (umwanya wa kabiri) kandi igabanya buri ntambwe kumuzi ya kare ya gaciro, bityo ibipimo hamwe nini nini, urusaku rufata intambwe ntoya kandi gake ivugururwa ifata intambwe nini. Guhuza n'imihindagurikire y'ikirere bivuze ko ushobora gukoresha igipimo kimwe cyo kwiga kumurongo wose. Impinduka, AdamW, ikuramo uburemere bwangirika kuva buhoro buhoro kandi byabaye nkibisanzwe byo guhugura impinduka nini nicyitegererezo cyururimi.
Ubushishozi
Adam agumana impuzandengo ebyiri zo kwiruka kuri buri kintu: m (gradients) na v (kwaduka kwaduka), ivugururwa nigipimo cyangirika beta1 (mubisanzwe 0.9) na beta2 (mubisanzwe 0,999). Kuberako byombi bitangirira kuri zeru, babogamye-bakosowe no kugabana na (1 - beta ^ t). Ivugurura ni theta = theta - lr * m_hat / (sqrt (v_hat) + epsilon), aho epsilon (hafi 1e-8) irinda kugabana na zeru. Niyo mpamvu Adamu akeneye kwiga bike-igipimo cyo kugereranya ugereranije na SGD isanzwe.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza ha Adamu na Optimizers
Adam na AdamW bakomeje kwiganza, ariko ubushakashatsi burimo gukora neza kuri moderi ya miriyari-miriyoni, aho kubika indangagaciro ebyiri ziyongera kuburemere bihenze. Ibikoresho byo kwibuka-urumuri nka Adafactor, 8-bit ya Adam, hamwe na optimizers nshya nka Ntare (ikoresha imbaraga zishingiye gusa ku kimenyetso) na Sophia bagamije guhuza ubuziranenge bwa Adamu nibuke buke cyangwa guhuza byihuse. Tegereza uburyo bwo guhuza n'imihindagurikire y'ikirere bwateguwe neza kugira ngo bukwirakwizwe, amahugurwa make-yo gukomeza kugira ngo akomeze gutera imbere.
Gushyira mu bikorwa Isi
Guhugura indimi nini nka GPT na Llama, zikoresha AdamW nka optimizer isanzwe.
Kuringaniza neza ibishushanyo mbonera byateganijwe (urugero, ResNet) kuri dataset yihariye hamwe nigipimo cyambere cyo kwiga cya Adamu.
Kumenyereza ikwirakwizwa rya moderi inyuma ya generator zishusho nka Diffusion Stable.
Gukoresha 8-bit ya Adam mumasomero nka bitsandbytes kugirango uhuze optimizer leta mububiko buke bwa GPU.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Adam and Adaptive Optimizers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
ZeRO na Optimizers
Ibibazo bikunze kubazwa
What is Adam and Adaptive Optimizers?
Adam ni akazi keza cyane inyuma yimiyoboro myinshi igezweho, ihita ihuza igipimo cyo kwiga kuri buri kintu cyose. Nibyingenzi kuko ituma imyitozo yimbitse yihuta kandi ntigabanuke cyane kuruta kumanuka gahoro gahoro.
Nibihe bintu bibiri Adamu akurikirana kuri buri kintu?
Adamu agumana impuzandengo yangirika ya gradients (umwanya wambere) hamwe na gradiyo ya kare (umwanya wa kabiri) kuri buri kintu cyose.
Kuki Adamu akoresha kubogama kubigereranyo byigihe?
Byombi m na v byatangijwe kuri zeru, bityo ibigereranyo hakiri kare bibogamye hasi; kugabana na (1 - beta ^ t) ikosora ibi.
Ni irihe tandukaniro nyamukuru riri hagati ya Adam na AdamW?
AdamW ikoresha kubora kuburemere aho kubivanga mumagambo yo guhuza n'imiterere, bigahindura rusange muri transformateur.
Ni uruhe ruhare ijambo rito rya epsilon rigira mu mategeko yo kuvugurura Adamu?
Epsilon (hafi 1e-8) yongewe kumurongo kugirango ibipimo bifite impuzandengo ya zeru hafi ya zeru-ntagereranywa.
Kuki Adamu akunze kuvugwa ko 'ahuza n'imiterere'?
Mugabanye kwaduka kwaduka ya buri kintu cya kare cyagereranijwe, Adamu apima ingano yintambwe kuri parameter aho gukoresha agaciro kamwe kwisi.