Icyiciro cya kabiri Optimisiyoneri hamwe nuburyo bwa Newton
Icyiciro cya kabiri cyo gutezimbere gikoresha amakuru yagoramye (matrike ya Hessian ya kabiri ikomoka) kugirango itere intambwe nziza igana byibuze, ntabwo ihanamye.
Incamake
It can converge in dramatically fewer iterations than plain gradient descent, but the cost of computing curvature makes it tricky to scale.
Kwibira cyane
Gradient descente izi gusa ahahanamye kurubu, bityo itoranya intambwe ihamye cyangwa intoki zingana nintambwe kandi yizeye ibyiza. Uburyo bwa Newton burakomeza: burareba kandi uburyo umusozi uhinduka (kugabanuka), wafashwe na Hessian, matrix yibikomoka ku gice cya kabiri cyose. Ivugurura rigwiza Hessian ihindagurika na gradient, ihita ikuraho buri cyerekezo ikamanuka hafi yuburinganire bwa hafi ya quadratic. Kubakure ya quadratique neza, uburyo bwa Newton bugera hepfo murwego rumwe. Gufata ni ubugome: icyitegererezo gifite ibipimo bya N gifite N-by-N Hessian, kubika no guhinduranya bisaba hafi ububiko bwa N-kare na compte ya N-cubed. Kuri miliyari-ibipimo byimikorere bidashoboka, niyo mpamvu abimenyereza bakoresha ibigereranyo bihendutse.
Ubushishozi
Ivugurura ryibanze rya Newton ni x_new = x - H_ibihe byikurikiranya, aho H ni Hessian. Uburyo bwa Quasi-Newton nka BFGS na L-BFGS birinda kubara H muburyo bwubaka igereranya ryimikorere yinyuma iva kubitandukanya bikurikirana. L-BFGS ibika gusa icyiciro cya nyuma cya gradient na intambwe aho kuba matrix yuzuye, guca kwibuka kuva N-kwaduka kugeza kuri bike bya N mugihe ukomeza kwihuta kwinshi.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Kazoza ka kabiri-Itondekanya Optimisiyoneri na Newton Methods
Kumiyoboro nini nini, uburyo bwa kabiri-butondekanya uburyo bukomeza kuba budasanzwe, ariko ibigereranyo bigenda byiyongera. Optimizers nka K-FAC na Shampoo igereranije igoramye ukoresheje blok-diagonal cyangwa Kronecker-yubatswe, hamwe nuburyo bushya nka Sophia na Muon bakoresha igereranya rihendutse kugirango bagabanye umuvuduko munini wururimi. Tegereza imbaraga zogukomeza gufata ibimenyetso byingirakamaro mugihe cyegereye-cyambere cyateganijwe, kugabanya itandukaniro riri hagati ya Adam nintambwe nyayo ya Newton.
Gushyira mu bikorwa Isi
L-BFGS ikwiranye no gusubira inyuma hamwe nizindi moderi za convex muri scikit-wige, aho ikunze gukubita hasi ya gradient kumanuka kuri dataseti nto kandi ziciriritse
Guhindura bundle mukubaka 3D hamwe na SLAM, aho Gauss-Newton na Levenberg-Marquardt batunganya kamera yerekana imyanya.
Guhugura utuntu duto twa fiziki-yamenyeshejwe imiyoboro yimitsi aho L-BFGS igera kubintu byukuri Adamu arwanira kugera
Shampoo na K-FAC byihutisha amahugurwa manini yo kwiga yimbitse ugereranije imiterere ya Hessian
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Second-Order Optimization and Newton Methods quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Gukwirakwiza Politiki Yerekeye Amatsinda
Ibibazo bikunze kubazwa
What is Second-Order Optimization and Newton Methods?
Icyiciro cya kabiri cyo gutezimbere gikoresha amakuru yagoramye (matrike ya Hessian ya kabiri ikomoka) kugirango itere intambwe nziza igana byibuze, ntabwo ihanamye. Irashobora guhurira muburyo butangaje cyane kuruta kugabanuka kumanuka, ariko ikiguzi cyo kubara kugabanuka bituma bigora gupima.
Ni ayahe makuru uburyo Newton akoresha ubwo buryo bwa gradient descente budakoreshwa?
Uburyo bwa Newton bwongerera icyiciro hamwe na curvature kuva Hessian, ikareka ikuraho icyerekezo kandi igereranya byibuze kwadratike yaho.
Kugirango intego ya quadratic itunganijwe neza, ni inzira zingahe uburyo bwa Newton bukeneye kugera byibuze?
Kuri kwadratike nyayo, moderi ya quadratic yaho ihwanye numurimo wukuri, nuko intambwe imwe ya Newton isimbuka igana byibuze.
Ni ukubera iki uburyo bwuzuye bwa Newton budashoboka kuri miliyari-ibipimo by'imitsi iva mu mitsi?
Hamwe na N ibipimo Hessian ifite N-kwaduka ibyinjira no kuyihindura umunzani nka N-cubed, bikaba bidashoboka kuri miliyari y'ibipimo.
Ni ubuhe buryo bwa quasi-Newton nka BFGS bukora kugirango wirinde ikiguzi cya Hessian?
BFGS ivugurura byimazeyo igereranya rya Hessian ihindagurika ukoresheje impinduka muri gradient hagati yintambwe, wirinda kubara bitaziguye.
Nigute L-BFGS igabanya kwibuka ugereranije na BFGS?
L