Uburyo bw'abakinnyi-banegura
Uburyo bw'abakinnyi-banegura bahuza abiga babiri: 'umukinnyi' uhitamo ibikorwa na 'kunegura' ucira urubanza uko ibyo bikorwa byari byiza.
Incamake
This pairing makes reinforcement learning more stable and sample-efficient than using either approach alone.
Kwibira cyane
Kwiga gushimangira bifite uburyo bubiri bwagutse: uburyo bushingiye kuri politiki bwiga neza icyo gukora, nuburyo bushingiye ku gaciro bwiga uko leta nziza ari nziza. Abakinnyi-banegura. Umukinnyi ni politiki isohora ibikorwa bishoboka; kunegura nigikorwa cyagaciro kigereranya kugaruka. Nyuma ya buri ntambwe, uwinegura abara ikosa-itandukaniro ryigihe gito ryerekana niba ibisubizo byari byiza cyangwa bibi kuruta uko byari byitezwe. Umukinnyi akoresha iri kosa kugirango asunike politiki yayo kubikorwa byatsinze ibiteganijwe kandi kure yibyo bidakora neza. Kuberako abanegura batanga ibipimo fatizo bito, igereranya ryabakinnyi ryikigereranyo ntirisakuza cyane kuruta muburyo bwa politiki-yerekana uburyo nka REINFORCE, mugihe ugikora umwanya uhoraho wibikorwa uburyo bwagaciro gusa nka Q-Kwiga bubona nabi.
Ubushishozi
Umukinnyi avugurura ibipimo byayo bya politiki mu cyerekezo cya politiki igenda igabanywa, yapimwe ninyungu A (s, a) = Q (s, a) - V (s), abanegura bavuga (akenshi binyuze mu ikosa rya TD r + gamma * V (s ') - V (s)). Inyungu ipima uburyo igikorwa cyiza kuruta ikigereranyo cya leta, bityo ibyiza byiza bishimangira ibikorwa nibibi bikabihagarika. Umunegura yatojwe ukwe kugirango agabanye amakosa ya TD.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza h'abakinnyi-banegura uburyo
Umukinnyi-Kunegura ni inkingi ya RL igezweho. Algorithms nka A3C, A2C, PPO, SAC, na DDPG byose byubakiyeho, hiyongeraho amayeri nkintego zaciwe kugirango zigezweho zihamye, ibihembo bya entropiya kubushakashatsi, hamwe nabakinnyi babangikanye nibisohoka. Witege ko hazakomeza kwiyongera muri robo, imashini nini nini, hamwe na RL uhereye kubitekerezo byabantu kugirango bahuze imiterere yindimi, aho gutuza no gukora neza byingenzi.
Gushyira mu bikorwa Isi
Guhugura amaboko ya robo hamwe nabashinzwe kugenzura ibikorwa hamwe na torque zihoraho (urugero, ukoresheje PPO cyangwa SAC)
Guhuza imvugo nini yindimi ukoresheje RLHF, aho PPO (uburyo bwabakinnyi-banegura) itunganya ibisubizo kubihembo byigihembo
Kumenya ingamba zingamba zimikino nka StarCraft II na Dota 2
Data-center gukonjesha no kugenzura ingufu-biga biga guhinduka bikomeje
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Actor-Critic Methods quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Icyiciro cya kabiri Optimisiyoneri hamwe nuburyo bwa Newton
Ibibazo bikunze kubazwa
What is Actor-Critic Methods?
Uburyo bw'abakinnyi-banegura bahuza abiga babiri: 'umukinnyi' uhitamo ibikorwa na 'kunegura' ucira urubanza uko ibyo bikorwa byari byiza. Uku guhuza bituma kwiga gushimangira biga neza kandi byintangarugero kuruta gukoresha inzira yonyine.
Muburyo bw'abakinnyi-banegura, ni uruhe ruhare 'kunegura'?
Umunegura yiga imikorere yagaciro kandi atanga ibimenyetso byo gusuzuma (nkikosa rya TD) ubwira umukinnyi niba ibikorwa byayo byari byiza cyangwa bibi kuruta uko byari byitezwe.
Niki 'akarusho' A (s, a) = Q (s, a) - V (s) bipima iki?
Ibyiza bitandukanya uburyo ibikorwa byihariye biruta ibisubizo bisanzwe biva muri leta, bitanga ikimenyetso gisukuye kuruta kugaruka.
Ni ukubera iki kongeraho kunegura bigabanya itandukaniro ugereranije nuburyo bwiza bwa politiki-buhoro buhoro nka REINFORCE?
Gukuramo igereranyo cyagaciro kunegura nkibanze fatizo bigabanya itandukaniro ryikigereranyo cya gradient utongeyeho kubogama, kwiga byihuse.
Ni ubuhe bushobozi uburyo Abakinnyi-banegura bafite ubwo buryo busanzwe bushingiye ku gaciro nka Q-Kwiga bitwaye nabi?
Kuberako umukinnyi asobanura neza politiki, irashobora gusohora ibikorwa bikomeza (urugero, torque ihuriweho) idakeneye max hejuru yibikorwa byinshi bitagira akagero.
Ni ikihe kimenyetso umukinnyi akoresha muguhindura politiki yacyo?
Umukinnyi ahindura politiki yayo mu cyerekezo cyatanzwe ninyungu zo kunegura / Ikimenyetso cya TD-ikosa, gishimangira ibikorwa byiza kuruta ibyateganijwe.