Kwerekana Ingororano
Icyitegererezo cyigihembo numuyoboro udasanzwe watojwe guhanura uburyo igisubizo cya AI ari cyiza, gikora nk'imikorere yihariye yo guca imanza zabantu.
Incamake
It is the scoring engine that makes reinforcement learning from human feedback possible at scale.
Kwibira cyane
Kugereranya ibihembo bikemura ikibazo gifatika: abantu ntibashobora kugereranya buri kimwe cya miriyoni zumusaruro icyitegererezo gitanga mugihe cyamahugurwa. Ahubwo, labelers igereranya agace gato k'ibisubizo, mubisanzwe gutoranya mubisubizo bibiri kubisubizo bimwe nibyiza. Icyitegererezo cyo guhemba noneho cyatojwe kubigereranya kugirango bisohore amanota imwe ya scalar kubantu bose bahita basubiza. Intego isanzwe yo guhugura ni moderi ya Bradley-Terry, ihindura ibyifuzo byombi muburyo igisubizo gishobora kurenza ikindi. Iyo umaze guhugurwa, iyi moderi yigihembo irashobora gusuzuma bihendutse ibisubizo bishya bitagira imipaka, bitanga ikimenyetso ko algorithms nka PPO ikoresha mugutezimbere ururimi. Ingero zigihembo nazo zongera gukoreshwa mugihe cyo gufata umwanzuro mwiza-wa-N, aho abakandida benshi batangirwa kandi amanota menshi agasubizwa.
Ubushishozi
Icyitegererezo cyigihembo nicyitegererezo cyururimi rwibanze hamwe na token-guhanura umutwe wasimbujwe umurongo umwe ugaragara usohora scalar imwe. Amahugurwa yerekana byinshi-bishoboka ko igisubizo cyatoranijwe gifite amanota arenze ayo yanze: igihombo = -log (sigmoid (r_hitamo - r_yanze)). Gusa itandukaniro ritandukanye rifite akamaro, igipimo cyuzuye rero uko bishakiye. Ubwiza bufatika kuri label ihamye no gukwirakwiza uburyo bwo gusubiza.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Igihe kizaza cyo kwerekana ibihembo
Ubushakashatsi burimo guhangana nicyitegererezo cyigihembo 'intege nke zikomeye: zirashobora' kwibasirwa '(moderi ikoresha ibisambo nko gutonesha uburebure), kandi bigenda biva mubisaranganya uko politiki igenda itera imbere. Icyerekezo cyiza kirimo uburyo bwo gutanga ibihembo byerekana amanota buri ntambwe yo gutekereza, guhuza hamwe no kugereranya gushidikanya kurwanya hacking, ibirango byatoranijwe na AI (RLAIF), hamwe nigihembo cyibyara inyungu zitanga kunegura no gushyira mu gaciro aho kuba umubare wambaye ubusa.
Gushyira mu bikorwa Isi
Guha ingufu RLHF kubafasha nka ChatGPT na Claude utanga amanota yabakandida mugihe cy'amahugurwa ya PPO
Ibyiza-bya-N icyitegererezo, aho icyitegererezo gitanga ibisubizo byinshi kandi ibihembo byigihembo bihitamo ibyiza kubakoresha
Imibare na code 'verifiers' cyangwa uburyo bwo gutanga ibihembo byerekana amanota hagati yo gutekereza kugirango utezimbere ibibazo
Gutondekanya no gushungura amakuru yimyitozo ngororamubiri, kugumya gusa ibisekuru byinshi-amanota menshi kugirango arusheho guhuza neza
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Reward Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kwerekana ibihembo bya Bradley-Terry
Ibibazo bikunze kubazwa
What is Reward Modeling?
Icyitegererezo cyigihembo numuyoboro udasanzwe watojwe guhanura uburyo igisubizo cya AI ari cyiza, gikora nk'imikorere yihariye yo guca imanza zabantu. Moteri yo gutanga amanota niyo ituma imbaraga zishimangira kubitekerezo byabantu bishoboka mubipimo.
Nibihe bisohoka byibanze byigihembo cyicyitegererezo cyatanzwe-igisubizo?
Icyitegererezo cyigihembo cyerekana ikarita nigisubizo kuri numero imwe ya scalar yerekana ubuziranenge bwahanuwe cyangwa ibyo abantu bakunda.
Ni ubuhe bwoko bw'amakuru akoreshwa cyane mugutoza ibihembo?
Abakozi mubisanzwe bagereranya ibisubizo bibiri kubibazo bimwe hanyuma bagahitamo icyiza; moderi ya Bradley-Terry ihindura ibi mubihembo bya scalar.
Niki igihombo gisanzwe-icyitegererezo cyatakaye?
Igihombo cya Bradley-Terry, -log (sigmoid (r_chosen - r_rejected)), gusa yita kubijyanye no gutumiza ugereranije, bityo igipimo cyuzuye ntigisanzwe.
'Kwiba ibihembo' ni iki?
Ibihembo byigihembo bibaho mugihe icyitegererezo kibonye shortcuts (nkuburebure burenze urugero cyangwa kureshya) ko ibihembo byigihembo bidatunganye cyane ariko abantu ntibabikora.
Nigute icyitegererezo cyigihembo cyubatswe muburyo bw'ururimi?
Icyitegererezo cyigihembo cyongeye gukoresha umugongo wa LM ariko uhinduranya urwego rwa nyuma kuri kimwe gisohora ibihembo bya scalar.