AMABWIRIZA Yibanze

Kwerekana ibihembo bya Bradley-Terry

Moderi ya Bradley-Terry nuburyo bumaze ibinyejana bwibarurishamibare bwo guhindura ibigereranyo byombi (A gukubita B) mumibare yumubare.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kazoza ka Bradley-Terry Igihembo Cyitegererezo
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

In modern AI it powers reward models that learn human preferences from 'which answer is better?' labels, the backbone of RLHF.

Kwibira cyane

Bradley-Terry, yatangijwe mu 1952, yibwira ko buri kintu gifite amanota yihishe, kandi birashoboka ko ikintu A gikubita ikintu B nigikorwa cyo gutandukanya amanota yabo. Muguhuza AI, iyi karita neza neza kumakuru yatoranijwe: labelers yabantu ibona ibisubizo bibiri byicyitegererezo hanyuma igahitamo icyiza, aho gutanga amanota-bigoye. Icyitegererezo cyigihembo, mubisanzwe icyitegererezo cyururimi hamwe na scalar isohoka umutwe, yaratojwe kugirango igisubizo abantu bahisemo kibone ibihembo byinshi bya scalar. Igihombo nikibi kibi-gishobora kuba Bradley-Terry bishoboka: kugwiza log-sigmoid ya (igihembo cyatoranijwe gukuramo ibihembo byo kwangwa). Icyitegererezo cyibihembo byavuyemo noneho gitanga ibisubizo uko bishakiye, gitanga ikimenyetso cyerekana imbaraga zo kwiga algorithms nka PPO guhitamo kurwanya kugirango moderi zifashe kandi zihuze.

Gushyira mu bikorwa Isi

Guhugura icyitegererezo cyigihembo muri RLHF itondekanya ibisubizo bibiri bya chatbot kandi igaburira ibimenyetso byiza-bibi kuri PPO neza.

Ibyifuzo Byibanze Optimisiyoneri-gutunganya neza icyitegererezo muburyo bwatoranijwe-bwanze ibisubizo byombi ukoresheje igihombo cya Bradley-Terry igihombo-sigmoid.

Urutonde rwa chess cyangwa rwohereza abakinnyi binyuze kuri Elo, mubare ni mubyara wa hafi wa moderi ya Bradley-Terry kubisubizo byimikino.

Kubaka urutonde rwibyifuzo biva mubakoresha 'bahisemo A kurenza B' kanda amakuru aho kuba inyenyeri yuzuye.

Kazoza ka Bradley-Terry Igihembo Cyitegererezo

Bradley-Terry afata urutonde rumwe ruhoraho hamwe nibyifuzo byinzibacyuho, bisenyuka mugihe abantu batavuga rumwe cyangwa ibyifuzo byabo. Ubushakashatsi bugenda bugana ku cyitegererezo gifata ibyifuzo bikunzwe, ibihembo byinshi (ubufasha, umutekano, ubunyangamugayo bwatsindiye ukwe), hamwe nuburyo nka Nash kwigira kubitekerezo byabantu bigabanya igitekerezo kimwe. DPO nibihinduka byayo bigenda byiyongera intego ya Bradley-Terry mumahugurwa ya politiki. Tegereza gahunda nziza yo kugereranya, harimo urutonde rwibintu birenga bibiri hamwe nibyifuzo biremereye, kugirango ugabanye ibihembo.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Ibyemezo bisobanutse. Iragufasha gutandukanya ibyifuzo bya tekiniki bisobanutse nururimi rwo kwamamaza.
  • Igiciro na bije. Urashobora kubaza ibibazo byiza byo gushyira mubikorwa mbere yo gukoresha amafaranga cyangwa igihe.
  • Itsinda hamwe nakazi. Amakipe asangiye ibitekerezo akora ibicuruzwa byiza, politiki, nibyemezo byo kwiga.

Witondere

  • Amakipe atandukanye arashobora gukoresha ijambo rimwe muburyo butandukanye, sobanura intera hakiri kare.
  • Ibipimo birashobora kugaragara bikomeye mugihe imikorere-yisi-itaringaniye.
  • Kwirengagiza ubuziranenge bwamakuru na gahunda yo gusuzuma akenshi bitanga ibisubizo byoroshye.

Intambwe ikurikira

  1. Tangira nururimi rusobanutse rwibisubizo ukeneye.
  2. Toranya intsinzi imwe hamwe nuburyo bumwe bwo gutsindwa mbere yo kwipimisha.
  3. Koresha umuderevu muto hamwe namakuru ahagarariye, ntabwo ari demo yashizweho.
  4. Inyandiko aho Bradley-Terry Igihembo Cyerekana Model ifasha kandi nuburyo bworoshye bworoshye.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the Bradley-Terry Reward Modeling quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

What is Bradley-Terry Reward Modeling?

Moderi ya Bradley-Terry nuburyo bumaze ibinyejana bwibarurishamibare bwo guhindura ibigereranyo byombi (A gukubita B) mumibare yumubare. Muri AI igezweho iha ibihembo icyitegererezo cyiga ibyo abantu bakunda 'ni ikihe gisubizo cyiza?' ibirango, umugongo wa RLHF.

Niki moderi ya Bradley-Terry ihinduka mumibare?

Bradley-Terry afata igereranya 'A Beats B' kandi agashyiramo amanota yihishe kuri buri kintu, niyo mpamvu bihuye neza nibyifuzo byabantu.

Muri Bradley-Terry, birashoboka ko A gukubita B ari imikorere yiki?

Icyitegererezo gishyiraho P (A ikubita B) ingana na logistique (sigmoid) yo gutandukanya amanota ya A na B yihishe amanota.

Ni ukubera iki ibihembo bya Bradley-Terry bigaragazwa gusa kugeza igihe byongeweho?

Igihombo cyamahugurwa gikoresha gusa itandukaniro riri hagati yigihembo cyatoranijwe no kwangwa, bityo guhindura amanota yose kumurongo umwe uhoraho bituma igihombo kidahinduka; igipimo ntarengwa ni uko.

Ni ikihe gihombo gisanzwe cyo kugereranya icyifuzo cyo kwerekana ibihembo?

Bradley-Terry itari nziza-birashoboka kugabanuka kuri log-sigmoid yo gutandukanya-gukuramo ibihembo-gutandukanya ibihembo, bigatuma ibihembo byatoranijwe bihinduka hejuru.

Nubuhe buryo bwasibye icyitegererezo cyigihembo mugutezimbere intego ya Bradley-Terry kuri politiki?

DPO ivugurura intego ya Bradley-Terry kugirango ishobore gukoreshwa muburyo bwa politiki, ikuraho amahugurwa no kubaza icyitegererezo cyigihembo.