InoteveraGaidhi rinotevera
Boka Relative Policy Optimization
Tekinoroji
Nhungamiro yehunyanzvi
Yechipiri-Odha Optimization uye Newton Nzira
Chechipiri-odha optimization inoshandisa curvature ruzivo (iyo yeHessian matrix yechipiri inotorwa) kutora nhanho dzakangwara kuenda kune hushoma, kwete kutsetseka chete.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
It can converge in dramatically fewer iterations than plain gradient descent, but the cost of computing curvature makes it tricky to scale.
Kudzika Kwakadzika
Gradient descent inongoziva kutsetseka kwauri ikozvino, saka inotora yakamisikidzwa kana ruoko-yakarongedzwa nhanho saizi uye inotarisira yezvakanakisa. Nzira yaNewton inoenderera mberi: inotarisawo kuti kutsetseka kuri kuchinja sei ( curvature ), yakatorwa neHessian, matrix ezvimwe zvikamu zvechipiri. Iyo yekuvandudza inowanza iyo yakatenderedza Hessian neiyo gradient, iyo inoereketa yega yega nzira uye inogara padhuze nehushoma hwepanzvimbo quadratic approximation. Kuti uwane mbiya yequadratic yakakwana, nzira yaNewton inosvika pasi nenhanho imwe chete. Kubata kwacho kune hutsinye: modhi ine N paramita ine N-by-N Hessian, saka kuchengetedza uye kuchinjisa kunodhura zvinosvika N-squared memory uye N-cubed compute. Kune mabhiriyoni-parameter network izvo hazvigoneke, ndosaka varapi vachishandisa zvakachipa approximations.
Kuitwa muNyika yeChokwadi
L-BFGS inokodzera kudzoreredza uye mamwe maconvex modhi mu scikit-dzidza, uko inowanzo rova plain gradient kudzika pamadiki kusvika epakati dataset.
Bundle gadziriso mu 3D kuvakazve uye SLAM, uko Gauss-Newton uye Levenberg-Marquardt inonatsa kamera inomira uye inonongedza nzvimbo.
Kudzidzisa diki fizikisi-inoziva neural network uko L-BFGS inowana chaiyo iyo Adhamu anotamburira kusvika.
Shampoo uye K-FAC inomhanyisa kudzidziswa kwakadzama kwakadzama nekuenzanisa chimiro cheHessian.
Ramangwana reKunatsiridza Kwechipiri-Odha uye Newton Nzira
Kune hofori neural network, yakazara yechipiri-yekurongeka nzira dzinogara dzisingashande, asi fungidziro dziri kuwedzera. Optimizers seK-FAC uye Shampoo akaenzana curvature achishandisa block-diagonal kana Kronecker-factored chimiro, uye nzira nyowani dzakaita seSophia neMuon dzinoshandisa zvakachipa curvature fungidziro kuti ikurumidze kudzidzira mutauro wakakura modhi. Tarisira kuenderera mberi kwekuedza kutora chinobatsira curvature chiratidzo pamutengo wepedyo-yekutanga-odha, kuderedza mukaha pakati paAdama neNewton nhanho dzechokwadi.
Kuzvishandisa
Nei zvichikosha
- Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
- Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
- Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Chenjera
- Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
- Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
- Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Matanho anotevera
- Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
- Benchmark pasi pechokwadi mutoro uye data mamiriro.
- Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
- Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Second-Order Optimization and Newton Methods quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
What is Second-Order Optimization and Newton Methods?
Chechipiri-odha optimization inoshandisa curvature ruzivo (iyo yeHessian matrix yechipiri inotorwa) kutora nhanho dzakangwara kuenda kune hushoma, kwete kutsetseka chete. Inogona kuungana mune yakaderera iterations pane yakajeka gradient kudzika, asi mutengo wekombuta curvature unoita hunyengeri kuyera.
Ndeupi ruzivo runoshandiswa nenzira yaNewton iyo isina kudzika?
Nzira yaNewton inowedzera gradient nekukombama kubva kuHessian, ichiirega ichidzoreredza mafambiro uye kufungidzira hushoma hwenzvimbo yequadratic.
Pachinangwa che quadratic chakakwana, matanho mangani anodiwa neNewton kuti asvike pashoma?
Pane chaiyo quadratic, iyo yemunharaunda quadratic modhi yakaenzana nebasa rechokwadi, saka imwe Newton nhanho inosvetuka yakananga kune shoma.
Nei nzira yaNewton izere isingashande kune mabhiriyoni-parameter neural network?
Iine N paramita iyo Hessian ine N-squared mapindiro uye inverting iyo zviyero seN-cubed, izvo zvisingagoneke pamabhiriyoni emaparamita.
Chii chinonzi quasi-Newton nzira dzakaita seBFGS kudzivirira mutengo weHessian?
BFGS inodzokorodza inodzokorodza fungidziro yeinopesana Hessian ichishandisa shanduko mugradient pakati pematanho, kudzivirira yakananga computation.
L-BFGS inoderedza sei ndangariro ichienzaniswa neBFGS?
Iyo 'L' inomiririra inogumira-yeuko: L-BFGS inongochengeta mashoma mashoma mavheji, ichideredza chengetedzo kubva kuN-yakaenzana kusvika kudiki diki reN.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi