Ubuyobozi bwa tekiniki

Icyiciro cya kabiri Optimisiyoneri hamwe nuburyo bwa Newton

Icyiciro cya kabiri cyo gutezimbere gikoresha amakuru yagoramye (matrike ya Hessian ya kabiri ikomoka) kugirango itere intambwe nziza igana byibuze, ntabwo ihanamye.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kazoza ka kabiri-Itondekanya Optimisiyoneri na Newton Methods
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Ishobora guhurira mu mibare mike cyane y'ubusubiramo ugereranyije n'uburyo busanzwe bwa gradient descent, ariko ikiguzi cyo kubara curvature gituma bigora kuyagura.

Kwibira cyane

Gradient descente izi gusa ahahanamye kurubu, bityo itoranya intambwe ihamye cyangwa intoki zingana nintambwe kandi yizeye ibyiza. Uburyo bwa Newton burakomeza: burareba kandi uburyo umusozi uhinduka (kugabanuka), wafashwe na Hessian, matrix yibikomoka ku gice cya kabiri cyose. Ivugurura rigwiza Hessian ihindagurika na gradient, ihita ikuraho buri cyerekezo ikamanuka hafi yuburinganire bwa hafi ya quadratic. Kubakure ya quadratique neza, uburyo bwa Newton bugera hepfo murwego rumwe. Gufata ni ubugome: icyitegererezo gifite ibipimo bya N gifite N-by-N Hessian, kubika no guhinduranya bisaba hafi ububiko bwa N-kare na compte ya N-cubed. Kuri miliyari-ibipimo byimikorere bidashoboka, niyo mpamvu abimenyereza bakoresha ibigereranyo bihendutse.

Gushyira mu bikorwa Isi

L-BFGS ikwiranye no gusubira inyuma hamwe nizindi moderi za convex muri scikit-wige, aho ikunze gukubita hasi ya gradient kumanuka kuri dataseti nto kandi ziciriritse

Guhindura bundle mukubaka 3D hamwe na SLAM, aho Gauss-Newton na Levenberg-Marquardt batunganya kamera yerekana imyanya.

Guhugura utuntu duto twa fiziki-yamenyeshejwe imiyoboro yimitsi aho L-BFGS igera kubintu byukuri Adamu arwanira kugera

Shampoo na K-FAC byihutisha amahugurwa manini yo kwiga yimbitse ugereranije imiterere ya Hessian

Kazoza ka kabiri-Itondekanya Optimisiyoneri na Newton Methods

Kumiyoboro nini nini, uburyo bwa kabiri-butondekanya uburyo bukomeza kuba budasanzwe, ariko ibigereranyo bigenda byiyongera. Optimizers nka K-FAC na Shampoo igereranije igoramye ukoresheje blok-diagonal cyangwa Kronecker-yubatswe, hamwe nuburyo bushya nka Sophia na Muon bakoresha igereranya rihendutse kugirango bagabanye umuvuduko munini wururimi. Tegereza imbaraga zogukomeza gufata ibimenyetso byingirakamaro mugihe cyegereye-cyambere cyateganijwe, kugabanya itandukaniro riri hagati ya Adam nintambwe nyayo ya Newton.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Kora ikizamini cyerekeye Second-Order Optimization na Newton Methods

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Second-Order Optimization na Newton Methods ni iki?

Icyiciro cya kabiri cyo gutezimbere gikoresha amakuru yagoramye (matrike ya Hessian ya kabiri ikomoka) kugirango itere intambwe nziza igana byibuze, ntabwo ihanamye. Irashobora guhurira muburyo butangaje cyane kuruta kugabanuka kumanuka, ariko ikiguzi cyo kubara kugabanuka bituma bigora gupima.

Ni ayahe makuru uburyo Newton akoresha ubwo buryo bwa gradient descente budakoreshwa?

Uburyo bwa Newton bwongerera icyiciro hamwe na curvature kuva Hessian, ikareka ikuraho icyerekezo kandi igereranya byibuze kwadratike yaho.

Kugirango intego ya quadratic itunganijwe neza, ni inzira zingahe uburyo bwa Newton bukeneye kugera byibuze?

Kuri kwadratike nyayo, moderi ya quadratic yaho ihwanye numurimo wukuri, nuko intambwe imwe ya Newton isimbuka igana byibuze.

Ni ukubera iki uburyo bwuzuye bwa Newton budashoboka kuri miliyari-ibipimo by'imitsi iva mu mitsi?

Hamwe na N ibipimo Hessian ifite N-kwaduka ibyinjira no kuyihindura umunzani nka N-cubed, bikaba bidashoboka kuri miliyari y'ibipimo.

Ni ubuhe buryo bwa quasi-Newton nka BFGS bukora kugirango wirinde ikiguzi cya Hessian?

BFGS ivugurura byimazeyo igereranya rya Hessian ihindagurika ukoresheje impinduka muri gradient hagati yintambwe, wirinda kubara bitaziguye.

Nigute L-BFGS igabanya kwibuka ugereranije na BFGS?

L