Haɓaka oda na biyu da hanyoyin Newton
Haɓaka oda na biyu yana amfani da bayanan curvature (matrix Hessian na abubuwan haɓakawa na biyu) don ɗaukar matakai mafi wayo zuwa mafi ƙaranci, ba kawai gangara ba.
Dubawa
It can converge in dramatically fewer iterations than plain gradient descent, but the cost of computing curvature makes it tricky to scale.
Zurfafa nutsewa
Zuriyar gradient kawai ya san gangara a halin yanzu, don haka yana ɗaukar ƙayyadaddun girman matakin da aka daidaita ko kuma yana fatan mafi kyau. Hanyar Newton ta ci gaba: tana kuma kallon yadda gangaren ke canzawa (curvature), wanda Hessian ya kama, matrix na duk abubuwan da suka samo asali na biyu. Sabuntawa yana ninka juzu'in Hessian ta hanyar gradient, wanda ke sake daidaita kowane shugabanci ta atomatik kuma ya faɗi kusa da mafi ƙanƙantar ƙa'ida ta gida. Don kwano mai kwarjini mai kyau, hanyar Newton ta kai ƙasa a mataki ɗaya. Kama yana da muni: samfurin tare da sigogin N yana da N-by-N Hessian, don haka adanawa da jujjuya shi yana kashe kusan ƙwaƙwalwar N-squared da lissafin N-cubed. Don cibiyoyin sadarwar biliyan-biliyan da ba zai yiwu ba, wanda shine dalilin da ya sa masu aiki ke amfani da ƙima mai rahusa.
Fahimtar Fasaha
Babban sabuntawar Newton shine x_new = x - H_inverse sau da gradient, inda H shine Hessian. Hanyoyin Quasi-Newton kamar BFGS da L-BFGS suna guje wa lissafin H kai tsaye ta hanyar gina kusantar juzu'in sa daga bambance-bambancen gradient masu zuwa. L-BFGS yana adana ƴan gradient na ƙarshe da matakan mataki maimakon cikakken matrix, yankan ƙwaƙwalwar ajiya daga N-squared zuwa ƙarami mai yawa na N yayin kiyaye yawancin saurin haɗuwa.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Haɓaka oda na biyu da hanyoyin Newton
Ga manyan hanyoyin sadarwa na jijiyoyi, cikakkun hanyoyin yin oda na biyu sun kasance ba su da amfani, amma kusani suna samun ƙasa. Masu ingantawa kamar K-FAC da Shampoo kimanin curvature ta amfani da tsarin toshe-diagonal ko Kronecker-factored, da sabbin hanyoyin kamar Sophia da Muon suna amfani da ƙididdiga masu arha don haɓaka babban ƙirar ƙirar harshe. Yi tsammanin ci gaba da ƙoƙari don ɗaukar siginar lanƙwasa mai amfani a farashi na kusan-farko, yana rage tazara tsakanin matakan Adam da na gaskiya na Newton.
Aiwatar da Gaskiyar Duniya
L-BFGS dacewa koma bayan dabaru da sauran nau'ikan ƙira a cikin scikit-koyi, inda sau da yawa yana bugun zuriyar ƙarami a kan ƙarami zuwa matsakaicin bayanan bayanan.
Daidaita damfara a cikin sake ginawa na 3D da SLAM, inda Gauss-Newton da Levenberg-Marquardt ke daidaita kamara da matsayi.
Horar da ƙananan hanyoyin sadarwar jijiya na ilimin kimiyyar lissafi inda L-BFGS ya cimma daidaito wanda Adam ke ƙoƙarin isa.
Shampoo da K-FAC suna haɓaka babban horo mai zurfi na koyo ta hanyar kusantar tsarin Hessian
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Second-Order Optimization and Newton Methods quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Inganta Manufofin Dangi na Ƙungiya
Tambayoyin da ake yawan yi
What is Second-Order Optimization and Newton Methods?
Haɓaka oda na biyu yana amfani da bayanan curvature (matrix Hessian na abubuwan haɓakawa na biyu) don ɗaukar matakai mafi wayo zuwa mafi ƙaranci, ba kawai gangara ba. Yana iya haɗuwa a cikin ƴan ƙaranci fiye da saukowa a fili, amma farashin ƙididdiga na ƙididdigewa yana sa ya zama mai wahala don ƙima.
Wane bayani ne hanyar Newton ke amfani da shi wanda a fili ba ya sauka?
Hanyar Newton tana ƙara gradient tare da curvature daga Hessian, yana barin ta sake daidaita kwatance da kusan mafi ƙanƙanta na gida.
Don cikakkiyar manufa ta huɗu, matakai nawa ne hanyar Newton ke buƙatar isa mafi ƙanƙanta?
A kan madaidaicin ma'auni, ƙirar ƙwanƙwasa ta gida tana daidai da aikin gaskiya, don haka mataki ɗaya na Newton yana tsalle kai tsaye zuwa ƙarami.
Me yasa cikakkiyar hanyar Newton ba ta da amfani ga cibiyoyin sadarwa na siga na biliyan?
Tare da sigogin N Hessian yana da shigarwar N-squared kuma yana jujjuya shi ma'auni kamar N-cubed, wanda ba shi yiwuwa a biliyoyin sigogi.
Menene hanyoyin quasi-Newton kamar BFGS suke yi don guje wa farashin Hessian?
BFGS akai-akai tana sabunta kiyasin Hessian mai juyowa ta amfani da canje-canje a cikin gradient tsakanin matakai, guje wa ƙididdigewa kai tsaye.
Ta yaya L-BFGS ke rage ƙwaƙwalwar ajiya idan aka kwatanta da BFGS?
The 'L' yana tsaye ne don ƙayyadaddun ƙwaƙwalwar ajiya: L-BFGS yana adana ɗimbin ƙididdiga na kwanan nan, yana rage ajiya daga N-squared zuwa kusan ƙaramin adadin N.