Nesterov Accelerated Gradient
Nesterov Accelerated Gradient (NAG) bụ ụdị ọkụ ọkụ ka mma nke na-elepụ anya n'ihu tupu ịgbakọ gradient, na-enye ya nlele na-agbazi n'ihu.
Nchịkọta
It often converges faster and more stably than classical momentum.
Ime miri emi
Oge oge gboo na-agbakọ gradient n'ọnọdụ dị ugbu a, wee gbakwụnye ọsọ agbakọbara. Nghọta Nesterov, site na ọrụ Yurii Nesterov rụrụ na 1983 na njikarịcha convex accelerated, bụ ibu ụzọ were nzọụkwụ dị ike gaa leba anya n'ihu wee nyochaa gradient ebe ahụ. Nke a na-eme ka onye na-ahụ maka njikarịcha tụọ anya ebe ọkụ na-ebu ya ma tinye mgbazi tupu ọ gbasa ya, dị ka onye na-agba ọsọ na-ahụ ụzọ n'ihu ma na-agbanwe n'oge karịa ka emechara. Maka nsogbu convex dị nro Usoro Nesterov na-enweta ọnụ ọgụgụ nchịkọta kachasị mma nke usoro 1/k^2 n'ime ọnụ ọgụgụ nke usoro, ọganihu nwere ike ime karịa 1/k gradient dị larịị. N'ime mmụta miri emi, a na-enye ya dị ka nhọrọ dị mfe n'ọtụtụ usoro ma na-amịpụta ngwa ngwa ngwa ngwa, ọzụzụ oscillatory dị obere karịa oge ọkọlọtọ n'otu ọnụọgụ.
Nghọta nka nka
Isi ihe dị iche bụ ebe a na-enyocha gradient. Ọkọlọtọ ọkụ na-eji gradient na paramita dị ugbu a; Nesterov na-enyocha ya na ọnọdụ ihu-n'ihu params mwepu mmụta ọnụego oge beta ọsọ. Nke a na-atụ anya gradient n'ụzọ dị irè na-agbakwunye mgbazi kwekọrọ na mgbanwe na gradient, na-akụda overshoot n'akụkụ obere mkpụkọ agbagọ. N'omume, usoro nhazi na-emejuputa mmelite a haziri n'ụzọ algebra ka ọnụ ahịa agbakwunyere n'oge nkịtị bụ nke a na-agaghị efu.
Mmetụta atụmatụ
Mkpebi doro anya
Ọ na-enyere gị aka ikewapụta nkwupụta ọrụ aka doro anya na asụsụ ahịa.
Ọnụ ego na mmefu ego
Ị nwere ike ịjụ ajụjụ mmejuputa iwu ka mma tupu itinye ego ma ọ bụ oge.
Team na usoro ọrụ
Ndị otu nwere nghọta na-eme ka ngwaahịa, amụma na mkpebi mmụta ka mma.
Ọdịnihu nke Nesterov Accelerated Gradient
Nesterov momentum bụ ọkọlọtọ arụnyere n'ime ihe nrụpụta n'ofe PyTorch, TensorFlow, na ndị ọzọ, yana ụdị Nesterov nke Adam (Nadam) na-agwakọta anya n'ihu na ngbanwe mgbanwe. Usoro osooso ya na-aga n'ihu na-akpali nyocha n'ime ụzọ ndị na-agba ọsọ, atụmatụ ịmalitegharị, yana nyocha ihe kpatara osooso na-enyere aka na netwọk miri emi na-abụghị convex. Na-atụ anya n'ihu ụdị Nesterov ka ọ ga-abụ ihe ndabara na-adị jụụ maka ndị na-eme ihe na-achụ ọsọ ọsọ na-agakọ ọnụ.
Mmejuputa n'ezie n'ụwa
Na-eme ka nesterov = Ezi ọkọlọtọ na PyTorch ma ọ bụ TensorFlow SGD maka ọzụzụ ọsọ ọsọ, dị nro.
Na-eme ka nkwekọ na-eme ngwa ngwa na nsogbu convex dị nro dị ka nnukwu mgbagha mgbagha.
Mbelata overshoot na oscillation mgbe ọzụzụ netwọk miri emi nso minima nkọ.
Ịkwado Nadam optimizer, nke na-agbakwunye Nesterov anya n'ihu Adam.
Ihe ize ndụ & okporo ụzọ nche
Otu dị iche iche nwere ike iji otu okwu ahụ mee ihe n'ụzọ dị iche, yabụ kọwapụta oge n'oge.
Ihe nrịbama nwere ike ịdị ike ebe arụmọrụ ụwa na-adaghị adaba.
Ileghara ogo data na atụmatụ nyocha anya na-emepụtakarị nsonaazụ na-adịghị mma.
Map mmejuputa
Malite na nkọwa asụsụ dị larịị nke nsonaazụ ịchọrọ.
Họrọ otu metrik ịga nke ọma na otu ọnọdụ ọdịda tupu nnwale.
Gbaa obere onye na-anya ụgbọ elu nwere data nnọchite anya, ọ bụghị ihe ngosi ngosi na-egbu maramara.
Akwụkwọ ebe Nesterov Accelerated Gradient na-enyere aka yana ebe ụzọ ndị dị mfe dị mma.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Nesterov Accelerated Gradient quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ọdịda gradient
Ajụjụ a na-ajụkarị
What is Nesterov Accelerated Gradient?
Nesterov Accelerated Gradient (NAG) bụ ụdị ọkụ ọkụ ka mma nke na-elepụ anya n'ihu tupu ịgbakọ gradient, na-enye ya nlele na-agbazi n'ihu. Ọ na-agbakọtakarị ngwa ngwa yana kwụsie ike karịa oge oge gboo.
Gịnị bụ nkọwa nkọwa nke Nesterov Accelerated Gradient ma e jiri ya tụnyere oge gboo?
Nesterov na-ebu ụzọ tinye nzọụkwụ ahụ iji ruo ọnọdụ anya n'ihu, wee gbakọọ gradient ebe ahụ, na-enye mgbazi na-atụ anya.
Kedu ọnụego nnabata nke usoro Nesterov na-enweta na nsogbu convex dị nro?
Usoro Nesterov na-eme ngwa ngwa na-enweta ọnụego 1/k^2 kacha mma maka ebumnuche convex dị nro, ngwa ngwa karịa 1/k gradient.
Onye bu ụzọ webatara usoro gradient a ngwa ngwa?
Yurii Nesterov bipụtara usoro gradient accelerated na 1983 maka njikarịcha convex.
Kedu ihe kpatara gradient anya n'ihu na-enye aka n'akụkụ obere mkpụkọ agbagọ?
Site n'ịtụle gradient ebe ume na-aga, Nesterov nwere ike mezie ihe nfefe na-abịa tupu oge ochie.
Na omume, kedu ka ọnụ ahịa mgbako nke Nesterov siri jiri tụnyere oge oge gboo?
Frameworks na-emejuputa ụdị ahaziri ahazi ka Nesterov na-agbakwunye ego na-enweghị atụ karịa oge nkịtị.