Gushimangira Kwiga Biturutse kubitekerezo byabantu
RLHF ni tekinike ihindura icyitegererezo cyururimi ruto nkumufasha ufasha, ufite ikinyabupfura ubitoza kubyo abantu bakunda.
Incamake
It matters because it aligns model behavior with what people actually want, not just what is statistically likely.
Kwibira cyane
Ururimi rwateganijwe mbere yerekana inyandiko zemewe, ariko birashoboka ntabwo arimwe bifasha, inyangamugayo, cyangwa umutekano. RLHF ikosora ibi mubyiciro. Ubwa mbere, kugenzurwa neza-gutunganya byigisha icyitegererezo gukurikiza amabwiriza ukoresheje ibisubizo byanditse byabantu. Ibikurikira, abantu bagereranya ibice bibiri byicyitegererezo kubisubizo bimwe hanyuma bagahitamo icyiza; kugereranya gutoza icyitegererezo cyigihembo gitanga igisubizo icyo aricyo cyose. Hanyuma, imvugo yindimi itezimbere hamwe no gushimangira kwiga gutanga ibisubizo igipimo cyicyitegererezo cyigihembo. Igihano kirinda gutembera kure yicyitegererezo cyambere kugirango gikomeze kuvuga neza kandi ntigikoresha ibibazo byuburyo bwigihembo. RLHF yari ingenzi mu gutuma __AIU_PROTECTED_7 __- abafasha b'uburyo bukoreshwa.
Ubushishozi
Icyitegererezo cyo guhemba gikunze gutozwa kubiri hamwe no gutakaza uburyo bwa Bradley-Terry, biga gutanga igisubizo gikundwa numuntu amanota menshi ya scalar. Politiki noneho ivugururwa hamwe na PPO (Proximal Policy Optimisation), itanga ibihembo byinshi mugihe igihano cya KL-cyo gutandukanya icyitegererezo cyerekanwe kubuza kurenza urugero no 'guhemba ibihembo'. Kuberako PPO idahwitse, uburyo bushya nka DPO (Optimisation Directeur Directeur) gusimbuka icyitegererezo cyigihembo cyihariye hamwe no gushimangira imbaraga, guhindura politiki biturutse kubantu bombi.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza h'imbaraga zo Kwiga Biturutse kubitekerezo byabantu
RLHF irimo kugenda neza kandi igice cyikora. DPO hamwe nuburyo bujyanye nuburyo bukurikira busimbuza umuyoboro uremereye wa PPO kumakipe menshi, kandi RLAIF ikoresha ibitekerezo byatanzwe na AI (nko muri AI Itegekonshinga) kugirango igabanye ibiciro. Ubushakashatsi burimo guhangana nigihembo cyibihembo, kubogama kwa annotator, hamwe ningorabahizi zo gusuzuma ibisubizo birebire cyangwa byinzobere, hamwe nubuhanga nko kugenzura inzira no kujya impaka. Witege guhuza guhuza ibitekerezo byabantu na AI, ibimenyetso byinshi byigihembo birenze igikumwe kimwe, hamwe no kugenzura neza uwatanze ibyo akunda nindangagaciro bashiraho.
Gushyira mu bikorwa Isi
Guhuza umufasha wibiganiro kugirango yange ibyifuzo byangiza kandi bitange ibisubizo byingirakamaro, byubatswe neza aho kuba inyandiko yemewe.
Gutondekanya ibice bibiri byincamake kubantu bakunda gutoza icyitegererezo cyandika incamake abantu basanga ari ingirakamaro.
Kugabanya umusaruro wuburozi cyangwa kubogama mugushimira ibisubizo abatanga amanota yabantu bubaha kandi bafite umutekano.
Gukoresha DPO kuri dataset yikundwa na vanze ibisubizo kugirango uhuze moderi ifunguye isoko idakoresheje PPO yuzuye.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Reinforcement Learning From Human Feedback quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kwiga gushimangira
Ibibazo bikunze kubazwa
What is Reinforcement Learning From Human Feedback?
RLHF ni tekinike ihindura icyitegererezo cyururimi ruto nkumufasha ufasha, ufite ikinyabupfura ubitoza kubyo abantu bakunda. Nibyingenzi kuko bihuza imyitwarire yicyitegererezo nibyo abantu bashaka mubyukuri, ntabwo aribyo bishoboka gusa mubibare.
Niyihe ntego nyamukuru ya RLHF kurugero rwururimi?
RLHF iyobora icyitegererezo kubisubizo abantu bakunda, bikarushaho gufasha, kuba inyangamugayo, n'umutekano aho kuba byemewe gusa.
Ni ubuhe buryo bwo guhemba muri RLHF bwiga gukora?
Icyitegererezo cyigihembo cyahuguwe kubigereranyo byabantu kugereranya amanota kugirango batange ibisubizo, bitanga ikimenyetso politiki nziza.
Ni ukubera iki igihano cya KL-gutandukanya icyitegererezo cyambere cyakoreshejwe mugihe cya RL?
Igihano cya KL gikomeza politiki yagezweho hafi yicyitegererezo, irinda ibihembo byigihembo no gutakaza kuvuga neza.
Nigute amakuru yatoranijwe yakusanyirijwe muburyo bwo guhemba?
Annotator batoranya igisubizo cyatoranijwe mugereranya, itoza icyitegererezo cyo guhemba binyuze mu gihombo cya Bradley-Terry.
Ni izihe nyungu DPO (Optimisation itaziguye) itanga hejuru ya RLHF ya kera?
DPO ikura intego iturutse kubyo ukunda, irinda icyitegererezo cyigihembo cyihariye hamwe nintambwe yo kwiga ishimangira.