Samfuran Kyautar Bradley-Terry
Samfurin Bradley-Terry hanya ce ta ƙididdiga ta ƙarni don juyar da kwatance biyu (A beats B) zuwa maki na lamba.
Dubawa
In modern AI it powers reward models that learn human preferences from 'which answer is better?' labels, the backbone of RLHF.
Zurfafa nutsewa
Bradley-Terry, wanda aka gabatar a cikin 1952, yana ɗauka cewa kowane abu yana da maƙiyi mai ɓoye, kuma yuwuwar abu A ya doke abu B shine aikin dabaru na bambancin maki. A cikin daidaitawar AI, wannan taswirori a hankali kan bayanan fifiko: masu lakabin ɗan adam suna ganin martanin samfuri guda biyu kuma sun zaɓi mafi kyau, maimakon ba da ƙima mai ƙarfi-to-daidaitacce. Samfurin lada, yawanci ƙirar harshe tare da fitaccen kan fitarwa, ana horar da shi ta yadda martanin da mutane suka fi so ya sami sakamako mai girma. Asarar ita ce mummunan ra'ayi na yiwuwar Bradley-Terry: haɓaka log-sigmoid na (ladan da aka zaɓa na ragi da aka ƙi). Samfurin lada da aka samu sannan ya ba da sakamako na sabani, yana ba da siginar da ke ƙarfafa algorithms na koyo kamar PPO don ingantawa da yin samfura mafi taimako da daidaitawa.
Fahimtar Fasaha
Asarar horarwa don kwatancen shine kawai cire log-sigmoid na (r_chosen - r_rejected), don haka samfurin kawai yana koyon bambance-bambancen dangi. Wannan yana nufin ana iya gane lada har zuwa ƙarawa na dindindin; cikakken ma'auni na sabani ne. Saboda kwatancen sun fi sauƙi kuma sun fi dacewa ga mutane fiye da maki 1-zuwa-10, bayanan Bradley-Terry ba su da hayaniya. Inganta fifikon fifiko kai tsaye daga baya ya nuna zaku iya tsallake ƙirar lada daban kuma ku inganta manufar Bradley-Terry kai tsaye akan manufofin.
Dabarun Tasiri
Shawarwari masu haske
Yana taimaka muku keɓance bayyanannen da'awar fasaha daga harshen talla.
Kudin da kasafin kuɗi
Kuna iya yin mafi kyawun tambayoyin aiwatarwa kafin kashe kuɗi ko lokaci.
Ƙungiya da aikin aiki
Ƙungiyoyin da ke da fahimtar juna suna yin mafi kyawun samfura, manufofi, da yanke shawara na koyo.
Makomar Samfuran Kyautar Bradley-Terry
Bradley-Terry yana ɗaukar madaidaicin matsayi guda ɗaya da zaɓin canzawa, wanda ke rushewa lokacin da mutane ba su yarda ba ko zagayowar zaɓin. Bincike yana motsawa zuwa ƙirar ƙira waɗanda ke ɗaukar rabe-raben fifiko, lada masu yawa (taimako, aminci, gaskiya da aka zana daban), da kuma hanyoyin kamar Nash koyo daga ra'ayoyin ɗan adam waɗanda ke sauke zato-maki ɗaya. DPO da bambance-bambancensa suna ƙara ninka manufar Bradley-Terry kai tsaye zuwa horar da manufofi. Yi tsammanin ingantattun tsare-tsare na kwatance, gami da martaba na abubuwa sama da biyu da abubuwan da za a iya ɗauka, don rage hacking ɗin lada.
Aiwatar da Gaskiyar Duniya
Horar da samfurin lada a cikin RLHF wanda ke ba da martani biyu na chatbot kuma yana ciyar da siginar mafi muni ga daidaitawar PPO.
Inganta fifikon fifikon kai tsaye daidaitaccen samfuri kai tsaye akan nau'ikan amsa da aka zaɓa-da-an ƙi ta amfani da asarar Bradley-Terry log-sigmoid.
Matsayin dara ko fitar da ƴan wasa ta hanyar Elo, wanda a ilimin lissafi ɗan uwan na kusa ne na ƙirar Bradley-Terry akan sakamakon wasan.
Gina matsayi na shawarwarin abun ciki daga 'masu amfani sun fi son A kan B' bayanan danna maimakon cikakken kimar taurari.
Hatsari & Tsare-tsare
Ƙungiyoyi daban-daban na iya amfani da kalmar iri ɗaya daban, don haka ayyana iyaka da wuri.
Alamomi na iya yin kama da ƙarfi yayin da aikin zahirin duniya bai yi daidai ba.
Yin watsi da ingancin bayanai da tsare-tsaren kimantawa galibi yana haifar da sakamako mara ƙarfi.
Taswirar Hanya
Fara da ma'anar harshe a sarari na sakamakon da kuke buƙata.
Zaɓi ma'aunin nasara ɗaya da yanayin gazawa ɗaya kafin gwaji.
Gudun ƙaramin matukin jirgi tare da bayanan wakilci, ba saitin demo da aka goge ba.
Daftarin aiki inda Bradley-Terry Reward Modelling yana taimakawa kuma inda hanyoyin mafi sauƙi suka fi kyau.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Bradley-Terry Reward Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Samfuran Kyauta
Tambayoyin da ake yawan yi
What is Bradley-Terry Reward Modeling?
Samfurin Bradley-Terry hanya ce ta ƙididdiga ta ƙarni don juyar da kwatance biyu (A beats B) zuwa maki na lamba. A cikin AI na zamani yana ba da ikon samfuran lada waɗanda ke koyon abubuwan da ɗan adam daga 'wace amsa ce ta fi kyau?' labels, kashin baya na RLHF.
Menene samfurin Bradley-Terry ke juyawa zuwa makin lambobi?
Bradley-Terry yana ɗaukar kwatancen 'A ta doke B' biyu kuma yana ba da ƙima mai ƙarfi ga kowane abu, wanda shine dalilin da ya sa ya dace da lakabin fifikon ɗan adam da kyau.
A cikin Bradley-Terry, yuwuwar A ta doke B aiki ne na menene?
Samfurin ya tsara P(A ya doke B) daidai da dabaru (sigmoid) na bambanci tsakanin maƙiyin ƙarfin A da B.
Me yasa ba za a iya gane ladan Bradley-Terry ba har zuwa kari na dindindin?
Asarar horarwa tana amfani ne kawai da bambanci tsakanin zaɓaɓɓun lada da aka ƙi, don haka canza duk maki ta hanyar akai-akai yana barin asarar baya canzawa; cikakken ma'auni na sabani ne.
Menene ma'aunin hasara don kwatancen fifiko guda ɗaya a cikin ƙirar lada?
Yiwuwar rajista mara kyau na Bradley-Terry yana rage zuwa rage log-sigmoid na zaɓin-rasa-ƙimar lada, yana tura ladan amsa da aka zaɓa mafi girma.
Wace hanya ce ta tsallake samfurin lada daban ta hanyar inganta manufar Bradley-Terry kai tsaye akan manufofin?
DPO ya sake fasalin manufar fifikon Bradley-Terry don a iya amfani da shi kai tsaye zuwa tsarin manufofin, cire buƙatar horarwa da neman samfurin lada mai zaman kansa.