Kwerekana ibihembo bya Bradley-Terry
Moderi ya Bradley-Terry nuburyo bumaze ibinyejana bwibarurishamibare bwo guhindura ibigereranyo byombi (A gukubita B) mumibare yumubare.
Incamake
In modern AI it powers reward models that learn human preferences from 'which answer is better?' labels, the backbone of RLHF.
Kwibira cyane
Bradley-Terry, yatangijwe mu 1952, yibwira ko buri kintu gifite amanota yihishe, kandi birashoboka ko ikintu A gikubita ikintu B nigikorwa cyo gutandukanya amanota yabo. Muguhuza AI, iyi karita neza neza kumakuru yatoranijwe: labelers yabantu ibona ibisubizo bibiri byicyitegererezo hanyuma igahitamo icyiza, aho gutanga amanota-bigoye. Icyitegererezo cyigihembo, mubisanzwe icyitegererezo cyururimi hamwe na scalar isohoka umutwe, yaratojwe kugirango igisubizo abantu bahisemo kibone ibihembo byinshi bya scalar. Igihombo nikibi kibi-gishobora kuba Bradley-Terry bishoboka: kugwiza log-sigmoid ya (igihembo cyatoranijwe gukuramo ibihembo byo kwangwa). Icyitegererezo cyibihembo byavuyemo noneho gitanga ibisubizo uko bishakiye, gitanga ikimenyetso cyerekana imbaraga zo kwiga algorithms nka PPO guhitamo kurwanya kugirango moderi zifashe kandi zihuze.
Ubushishozi
Igihombo cyamahugurwa yo kugereranya ni ukuyemo gusa log-sigmoid ya (r_guhitamo - r_yanze), bityo icyitegererezo cyiga gusa itandukaniro ugereranije. Ibi bivuze ko ibihembo byamenyekanye gusa kugeza byongeweho bihoraho; igipimo ntarengwa ni uko. Kuberako kugereranya byoroshye kandi bihamye kubantu kuruta amanota 1 kugeza kuri 10, amakuru ya Bradley-Terry ntabwo ari urusaku. Ibyifuzo bya Optimisation itaziguye nyuma yerekanaga ko ushobora gusimbuka ibihembo bitandukanye hanyuma ugahindura intego ya Bradley-Terry kuri politiki.
Ingaruka z'Ingamba
Ibyemezo bisobanutse
Iragufasha gutandukanya ibyifuzo bya tekiniki bisobanutse nururimi rwo kwamamaza.
Igiciro na bije
Urashobora kubaza ibibazo byiza byo gushyira mubikorwa mbere yo gukoresha amafaranga cyangwa igihe.
Itsinda hamwe nakazi
Amakipe asangiye ibitekerezo akora ibicuruzwa byiza, politiki, nibyemezo byo kwiga.
Kazoza ka Bradley-Terry Igihembo Cyitegererezo
Bradley-Terry afata urutonde rumwe ruhoraho hamwe nibyifuzo byinzibacyuho, bisenyuka mugihe abantu batavuga rumwe cyangwa ibyifuzo byabo. Ubushakashatsi bugenda bugana ku cyitegererezo gifata ibyifuzo bikunzwe, ibihembo byinshi (ubufasha, umutekano, ubunyangamugayo bwatsindiye ukwe), hamwe nuburyo nka Nash kwigira kubitekerezo byabantu bigabanya igitekerezo kimwe. DPO nibihinduka byayo bigenda byiyongera intego ya Bradley-Terry mumahugurwa ya politiki. Tegereza gahunda nziza yo kugereranya, harimo urutonde rwibintu birenga bibiri hamwe nibyifuzo biremereye, kugirango ugabanye ibihembo.
Gushyira mu bikorwa Isi
Guhugura icyitegererezo cyigihembo muri RLHF itondekanya ibisubizo bibiri bya chatbot kandi igaburira ibimenyetso byiza-bibi kuri PPO neza.
Ibyifuzo Byibanze Optimisiyoneri-gutunganya neza icyitegererezo muburyo bwatoranijwe-bwanze ibisubizo byombi ukoresheje igihombo cya Bradley-Terry igihombo-sigmoid.
Urutonde rwa chess cyangwa rwohereza abakinnyi binyuze kuri Elo, mubare ni mubyara wa hafi wa moderi ya Bradley-Terry kubisubizo byimikino.
Kubaka urutonde rwibyifuzo biva mubakoresha 'bahisemo A kurenza B' kanda amakuru aho kuba inyenyeri yuzuye.
Ingaruka & Kurinda
Amakipe atandukanye arashobora gukoresha ijambo rimwe muburyo butandukanye, sobanura intera hakiri kare.
Ibipimo birashobora kugaragara bikomeye mugihe imikorere-yisi-itaringaniye.
Kwirengagiza ubuziranenge bwamakuru na gahunda yo gusuzuma akenshi bitanga ibisubizo byoroshye.
Igishushanyo mbonera
Tangira nururimi rusobanutse rwibisubizo ukeneye.
Toranya intsinzi imwe hamwe nuburyo bumwe bwo gutsindwa mbere yo kwipimisha.
Koresha umuderevu muto hamwe namakuru ahagarariye, ntabwo ari demo yashizweho.
Inyandiko aho Bradley-Terry Igihembo Cyerekana Model ifasha kandi nuburyo bworoshye bworoshye.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Bradley-Terry Reward Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kwerekana Ingororano
Ibibazo bikunze kubazwa
What is Bradley-Terry Reward Modeling?
Moderi ya Bradley-Terry nuburyo bumaze ibinyejana bwibarurishamibare bwo guhindura ibigereranyo byombi (A gukubita B) mumibare yumubare. Muri AI igezweho iha ibihembo icyitegererezo cyiga ibyo abantu bakunda 'ni ikihe gisubizo cyiza?' ibirango, umugongo wa RLHF.
Niki moderi ya Bradley-Terry ihinduka mumibare?
Bradley-Terry afata igereranya 'A Beats B' kandi agashyiramo amanota yihishe kuri buri kintu, niyo mpamvu bihuye neza nibyifuzo byabantu.
Muri Bradley-Terry, birashoboka ko A gukubita B ari imikorere yiki?
Icyitegererezo gishyiraho P (A ikubita B) ingana na logistique (sigmoid) yo gutandukanya amanota ya A na B yihishe amanota.
Ni ukubera iki ibihembo bya Bradley-Terry bigaragazwa gusa kugeza igihe byongeweho?
Igihombo cyamahugurwa gikoresha gusa itandukaniro riri hagati yigihembo cyatoranijwe no kwangwa, bityo guhindura amanota yose kumurongo umwe uhoraho bituma igihombo kidahinduka; igipimo ntarengwa ni uko.
Ni ikihe gihombo gisanzwe cyo kugereranya icyifuzo cyo kwerekana ibihembo?
Bradley-Terry itari nziza-birashoboka kugabanuka kuri log-sigmoid yo gutandukanya-gukuramo ibihembo-gutandukanya ibihembo, bigatuma ibihembo byatoranijwe bihinduka hejuru.
Nubuhe buryo bwasibye icyitegererezo cyigihembo mugutezimbere intego ya Bradley-Terry kuri politiki?
DPO ivugurura intego ya Bradley-Terry kugirango ishobore gukoreshwa muburyo bwa politiki, ikuraho amahugurwa no kubaza icyitegererezo cyigihembo.