Adam na ndị na-eme mgbanwe
Adam bụ onye na-arụ ọrụ nke ọma n'azụ ọtụtụ netwọkụ akwara ọgbara ọhụrụ, na-emegharị ọnụego mmụta dị iche maka oke ọ bụla.
Nchịkọta
It matters because it makes training deep models faster and far less finicky than plain gradient descent.
Ime miri emi
Adam (Atụmatụ Oge Ndagharị), nke Kingma na Ba webatara na 2014, jikọtara echiche abụọ. Nke mbụ, ume: ọ na-edobe nkezi na-emebi emebi nke gradients gara aga (oge mbụ) ka mmelite na-ewulite ọsọ n'ụzọ na-agbanwe agbanwe. Nke abụọ, per-parameter scaling: ọ na-esochi nkezi nke gradients squared (oge nke abụọ) wee kewaa nzọụkwụ ọ bụla site na mgbọrọgwụ square nke uru ahụ, ya mere paramita nwere nnukwu gradients na-eme mkpọtụ na-ewere nzọụkwụ dị nta na ndị na-adịghị emelite na-ewere nzọụkwụ ka ukwuu. Ngbanwe a pụtara na ị nwere ike na-ejikarị otu ọnụego mmụta n'ofe netwọkụ niile. Ụdị dị iche iche, AdamW, na-ewepụ ire ere site na mmelite gradient wee bụrụ ihe ndabere maka ịzụ nnukwu mgbanwe na ụdị asụsụ.
Nghọta nka nka
Adam na-ejigide ọnụọgụ abụọ na-agba ọsọ kwa oke: m (gradients) na v (gradients squared), emelitere site na ọnụego ire ere beta1 (nke na-abụkarị 0.9) na beta2 (nke na-abụkarị 0.999). N'ihi na ha abụọ na-amalite na efu, a na-agbaziri ha site n'ikewa site na (1 - beta ^ t). Mmelite ahụ bụ theta = theta - lr * m_hat / (sqrt (v_hat) + epsilon), ebe epsilon (gburugburu 1e-8) na-egbochi nkewa site na efu. Nke a bụ ya mere Adam ji chọọ obere nlegharị anya mmụta-ọnụego ya ma e jiri ya tụnyere SGD dị larịị.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke Adam na ndị na-eme mgbanwe
Adam na AdamW ka na-achị, mana nyocha na-akwalite arụmọrụ maka ụdị puku ijeri puku ijeri, ebe ịchekwa ụkpụrụ abụọ ọzọ kwa ịdị arọ na-efu ọnụ. Ọdịiche dị iche iche nke ìhè ebe nchekwa dị ka Adafactor, 8-bit Adam, na ndị nrụpụta ọhụrụ dị ka ọdụm (nke na-eji naanị ọkụ dabere na akara) yana Sophia chọrọ iji dakọọ ogo Adam na obere ebe nchekwa ma ọ bụ nchikota ngwa ngwa. Na-atụ anya ka ndị na-eme mgbanwe na-ege ntị kpọmkwem maka nkesa, ọzụzụ dị obere ka ọ na-aga n'ihu.
Mmejuputa n'ezie n'ụwa
Ọzụzụ ụdị asụsụ buru ibu dị ka GPT na Llama, nke na-eji AdamW dị ka ihe kacha mma.
Idozi nke ọma nhazi ọkwa ihe onyonyo a zụrụ azụ (dịka, ResNet) na dataset omenala yana naanị ọnụego mmụta Adam ndabara.
Ịzụ ụdị mgbasa ozi n'azụ ndị na-emepụta ihe oyiyi dị ka Stable Diffusion.
Na-agba ọsọ 8-bit Adam n'ọbá akwụkwọ dị ka bitsandbytes ka ọ dabara steeti optimizer na ebe nchekwa GPU nwere oke.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Adam and Adaptive Optimizers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
ZeRO na Sharded Optimizers
Ajụjụ a na-ajụkarị
What is Adam and Adaptive Optimizers?
Adam bụ onye na-arụ ọrụ nke ọma n'azụ ọtụtụ netwọkụ akwara ọgbara ọhụrụ, na-emegharị ọnụego mmụta dị iche maka oke ọ bụla. Ọ dị mkpa n'ihi na ọ na-eme ka ọzụzụ ụdị dị omimi dị ngwa ngwa ma dịkwa obere nke ọma karịa mgbada gradient.
Kedu ọnụọgụ abụọ ka Adam na-agbaso maka oke ọ bụla?
Adam na-edobe nkezi nke gradients (oge mbụ) na nke gradients square (oge nke abụọ) maka oke ọ bụla.
N’ihi gịnị ka Adam ji tinye mgbazi n’eleghị anya n’atụmatụ oge ya?
Ma m na v na-amalite ka ọ bụrụ efu, ya mere atụmatụ n'oge na-adịghị ala ala; nkewa site na (1 - beta^t) na-edozi nke a.
Kedu ihe dị iche n'etiti Adam na AdamW?
AdamW na-etinye ire ire arọ ozugbo n'arọ kama ịgwakọta ya n'ime okwu gradient na-agbanwe agbanwe, nke na-eme ka izugbe na mgbanwe mgbanwe dị mma.
Kedu ọrụ obere okwu epsilon na-arụ n'ọchịchị mmelite nke Adam?
Epsilon (ihe dị ka 1e-8) na-agbakwunye na denominator ka parampat nwere ihe dị nso-efu squared-gradient nkezi adịghị ebute mgbawa.
Gịnị mere a na-akpọkarị Adam dị ka 'onye na-eme mgbanwe'?
Site n'ikewa site na mgbọrọgwụ square nke nkezi squared-gradient nke ọ bụla, Adam na-atụba nha nzọụkwụ n'otu oke kama iji otu uru zuru ụwa ọnụ.