Dubawa
Hanyoyin wasan kwaikwayo-Critic sun haɗu da xalibai biyu: 'dan wasan kwaikwayo' wanda ke zabar ayyuka da kuma 'mai zargi' wanda ke yin la'akari da yadda waɗannan ayyukan suka yi kyau. Wannan haɗe-haɗe yana sa koyo ƙarfafawa ya fi karɓuwa da samfuri mai inganci fiye da amfani da kowane hanya kaɗai.
Hanyoyin wasan kwaikwayo-Critic tubalan fasaha ne na ginin gine-gine wanda ke shafar ingancin samfurin, farashin kayan aiki, latency, da aminci a ma'auni.
Zurfafa nutsewa
Ƙarfafa ilmantarwa yana da nau'i mai faɗi guda biyu: hanyoyin tushen siyasa waɗanda ke koyon abin da za a yi kai tsaye, da hanyoyin da suka dogara da ƙima waɗanda ke koyan yadda kyawawan jihohi suke. Actor-Critic fuses su. Mai wasan kwaikwayo wata manufa ce da ke fitar da yuwuwar aiki; mai sukar aiki ne mai ƙima wanda ya kiyasta dawowar da ake sa ran. Bayan kowane mataki, mai suka yana ƙididdige kuskuren bambancin ɗan lokaci da ke nuna ko sakamakon ya fi kyau ko muni fiye da yadda ake tsammani. Mai wasan kwaikwayo yana amfani da wannan kuskuren don tura manufofinsa zuwa ayyukan da suka doke tsammanin kuma daga waɗanda ba su yi aiki ba. Saboda mai sukar yana ba da ginshiƙan ƙarancin bambance-bambance, ƙididdigar gradient na ɗan wasan kwaikwayo ba su da hayaniya fiye da a cikin tsantsar manufofin-gradient hanyoyin kamar REINFORCE, yayin da har yanzu ana ci gaba da aiwatar da wuraren ayyuka masu ƙima kawai kamar Q-Learning.
Fahimtar Fasaha
Mai wasan kwaikwayo yana sabunta sigoginsa na manufofinsa a cikin tsarin gradient na manufofin, wanda aka ƙaddara ta fa'idar A(s,a) = Q(s,a) - V(s), wanda masu sukar suka ƙididdigewa (sau da yawa ta hanyar kuskuren TD r + gamma*V(s') - V(s)). Amfani yana auna yadda mafi kyawun aiki ya fi matsakaicin jihar, don haka fa'idodi masu kyau suna ƙarfafa ayyuka kuma waɗanda ba su da kyau suna danne su. An horar da mai suka daban don rage kuskuren TD.
Ƙwararren Ƙwararru-Hanyoyin Sukar
Hanyoyin wasan kwaikwayo-Critic sun haɗu da xalibai biyu: 'dan wasan kwaikwayo' wanda ke zabar ayyuka da kuma 'mai zargi' wanda ke yin la'akari da yadda waɗannan ayyukan suka yi kyau. Wannan haɗe-haɗe yana sa koyo ƙarfafawa ya fi karɓuwa da samfuri mai inganci fiye da amfani da kowane hanya kaɗai. Hanyoyin wasan kwaikwayo-Critic tubalan fasaha ne na ginin gine-gine wanda ke shafar ingancin samfurin, farashin kayan aiki, latency, da aminci a ma'auni. Don gina zurfin fahimta, bi Hanyoyi-Critic-Actor a matsayin samfurin aiki, ba fasali ɗaya ba: ayyana sakamakon da ake so, fayyace zato, da raba abin da tsarin zai iya yi da dogaro daga abin da har yanzu yana buƙatar yanke hukunci na ƙwararru.
A aikace, ƙungiyoyi masu ƙarfi da ke amfani da Hanyoyin Actor-Critic suna haɓaka gine-gine, bayanai, da zaɓin abubuwan more rayuwa tare da dogaro da farashi. Suna rubuta ƙayyadaddun ƙa'idodin nasara, gwaji akan bayanan gaskiya da gudanawar aiki, da jujjuyawar bisa ga tsarin gazawar da aka lura maimakon cin nasara na lokaci ɗaya. Wannan shine inda fahimtar ka'idar ta juya zuwa iyawa mai dorewa a cikin samfura, manufofi, da ayyuka.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A lokaci guda, Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin. Hanyar da ta fi dacewa ita ce haɗa saurin gwaji tare da horon gudanarwa: gudanar da matukin jirgi, kama shaida, buga rajistan ayyukan yanke shawara, da ci gaba da sabunta abubuwan tsaro kamar yadda halayen ƙira, tsammanin mai amfani, da buƙatun tsari ke tasowa.
Dabarun Tasiri
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Aiwatar da Gaskiyar Duniya
Horar da makaman robobi da masu sarrafa locomotion tare da ci gaba da jujjuyawar haɗin gwiwa (misali, ta amfani da PPO ko SAC)
Daidaita manyan nau'ikan harshe ta hanyar RLHF, inda PPO (hanyar mai sukar mai wasan kwaikwayo) ke inganta martani akan samfurin lada
Jagorar hadaddun dabarun wasanni kamar StarCraft II da Dota 2
Mai sanyaya cibiyar bayanai da masu kula da makamashi waɗanda ke koyon gyare-gyare mai sauƙi
Hanyoyin Aiwatarwa
Hanyoyin wasan kwaikwayo-Critic a aikace
Horar da makamai masu linzami da masu kula da locomotion tare da ci gaba da jujjuyawar haɗin gwiwa (misali, ta amfani da PPO ko SAC).
Horar da makamai masu linzami da masu sarrafa motsi tare da ci gaba da jujjuyawar haɗin gwiwa (misali, ta yin amfani da PPO ko SAC) Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da kuma bin diddigin abubuwan da ake samu da ƙimar kuɗi a kan lokaci.
Hanyoyin wasan kwaikwayo-Critic a aikace
Daidaita manyan samfuran harshe ta hanyar RLHF, inda PPO (hanyar mai sukar mai wasan kwaikwayo) ke inganta martani akan samfurin lada.
Daidaita manyan nau'ikan harshe ta hanyar RLHF, inda PPO (hanyar mai sukar mai wasan kwaikwayo) ke inganta martani game da samfurin lada Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don shari'o'in gefe, da bin diddigin nasarorin samarwa da ƙimar kuskure akan lokaci.
Hanyoyin wasan kwaikwayo-Critic a aikace
Jagorar hadaddun dabarun wasanni kamar StarCraft II da Dota 2.
Ƙwaƙwalwar dabarun dabarun wasanni kamar StarCraft II da Dota 2 Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don shari'o'in gefe, da bin duk nasarorin samarwa da farashi na kuskure akan lokaci.
Hanyoyin wasan kwaikwayo-Critic a aikace
Mai sanyaya cibiyar bayanai da masu kula da makamashi waɗanda ke koyon gyare-gyare mai sauƙi.
Cibiyoyin sanyaya bayanai da masu kula da sarrafa makamashi waɗanda ke koyon gyare-gyare mai sauƙi da sauƙi Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da bin diddigin nasarorin samarwa da ƙimar kuskure a kan lokaci.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.