Qaabaynta Abaalmarinta Bradley-Terry
Qaabka Bradley-Terry waa hab xisaabeed qarni jir ah oo loogu beddelayo isbarbardhigga laba-geesoodka ah (A garaaca B) dhibcaha tirada.
Dulmar
In modern AI it powers reward models that learn human preferences from 'which answer is better?' labels, the backbone of RLHF.
quusid qoto dheer
Bradley-Terry, oo la soo bandhigay 1952, wuxuu u maleynayaa in shay kasta uu leeyahay dhibcaha xoogga qarsoon, iyo suurtogalnimada in shayga A uu garaaco shayga B waa shaqada saadka ee farqiga dhibcaha. Isku toosinta AI, khariidadahani waxay si habsami leh ugu dul socdaan xogta doorbidida: calaamadeeyayaasha bini'aadamka waxay arkaan laba jawaabood oo moodeel ah oo waxay doortaan mid ka wanaagsan, halkii ay ka bixin lahaayeen qiimaynta saxda ah ee adag. Qaabka abaalgudka, sida caadiga ah nooca luqadda ee leh madax wax soo saar leh, ayaa loo tababaray si jawaabta ay dadku door bidaan ay u helaan abaal marin sare. Khasaaruhu waa suurtogalnimada gal-gacan ee Bradley-Terry: sare u qaad log-sigmoid ee (abaalmarinta la doortay ee laga jaray abaalmarinta diidmada). Qaabka abaalgudka ee natiijada ka soo baxay ayaa markaa keenaya wax soo saar aan sabab lahayn, isaga oo siinaya calaamada xoojinaysa algorithmsyada barashada sida PPO si ay uga hortagto in moodooyinka laga dhigo kuwo waxtar leh oo toosan.
Aragtida Farsamada
Khasaaraha tababarka ee isbarbardhigga si fudud ayaa looga jaray log-sigmoid ee (r_chosen - r_rejected), marka moodeelka kaliya ayaa waligiis barta kala duwanaanshiyaha. Tani waxay ka dhigan tahay in abaal-marinnada la aqoonsan karo oo keliya ilaa joogto ah wax lagu daro; Miisaanka saxda ah waa gardarro. Sababtoo ah isbarbardhigga ayaa u fudud oo si joogto ah u ah bini'aadamka marka loo eego buundooyinka 1-ilaa-10, xogta Bradley-Terry waa mid buuq yar. Xulashada Tooska ah markii dambe waxay muujisay inaad ka boodi karto qaabka abaal-marinta oo aad si toos ah u wanaajiso ujeeddada Bradley-Terry ee siyaasadda.
Saamaynta Istiraatijiyadeed
Go'aamo cad
Waxay kaa caawinaysaa inaad kala saartid sheegashooyinka farsamada cad iyo luqadda suuq-geynta.
Qiimaha iyo miisaaniyada
Waxaad waydiin kartaa su'aalo fulineed oo wanaagsan ka hor inta aadan lacag ama waqti bixin.
Kooxda iyo socodka shaqada
Kooxaha fahamka la wadaago waxay sameeyaan wax soo saar, siyaasad, iyo go'aano waxbarasho oo wanaagsan.
Mustaqbalka Abaalmarinta Bradley-Terry Modelinka
Bradley-Terry waxa uu qaadanayaa hal darajo oo joogto ah iyo dookhyada ku meel gaadhka ah, kaas oo burbura marka aadamigu khilaafo ama meerto doorashadu. Cilmi-baaristu waxay u socotaa moodooyinka qabsada qaybinta doorbidka, abaal-marinno dhinacyo badan leh (caawin, badbaado, daacadnimo si gaar ah loo dhaliyay), iyo hababka sida Nash oo ka baranaya jawaab-celinta aadanaha taasoo hoos u dhigaysa malo-awaalka hal-dhibcood. DPO iyo noocyadeeda ayaa si isa soo taraysa ugu laalaabaya ujeeddada Bradley-Terry si toos ah tababarka siyaasadda. Filo qorshayaasha isbarbardhigga ee hodanka ah, oo ay ku jiraan darajooyinka in ka badan laba shay iyo dookhyada kalsoonida leh, si loo yareeyo jabsiga abaalmarinta.
Dhaqangelinta Adduunka-dhabta ah
Tababarka qaabka abaalmarinta ee RLHF kaas oo ku daraya laba jawaabood oo chatbot ah oo quudiya calaamada ka sii liidata ee hagaajinta PPO.
Xulashada tooska ah ee hagaajinta qaab toos ah oo toos ah oo ku saabsan lammaanaha jawaabaha la doortay iyo kuwa la diiday iyadoo la adeegsanayo luminta Bradley-Terry log-sigmoid.
Qiimaynta chess-ka ama waxay ku soo dirtaa ciyaartoyda iyada oo loo sii marayo Elo, kaas oo xisaab ahaan ilma adeer dhow la ah qaabka Bradley-Terry ee natiijooyinka ciyaarta.
Dhisidda qiimeynta talinta nuxurka ee 'isticmaalayaasha waxay door bidaan A in ka badan B' xogta riix halkii ay ka ahaan lahaayeen qiimeynta xiddigaha dhammaystiran.
Khatarta & Dariiqyada Ilaalada
Kooxo kala duwan ayaa laga yaabaa inay isla erey u isticmaalaan si kala duwan, marka hore u qeex baaxadda.
Tilmaamaha ayaa u ekaan kara kuwo xooggan halka waxqabadka dhabta ah ee dunidu aanu sinnayn.
In la iska indho tiro tayada xogta iyo qorshayaasha qiimayntu waxay inta badan abuurtaa natiijooyin jilicsan.
Qorshe Hawleedka Dhaqangelinta
Ka bilow qeexidda luqadda cad ee natiijada aad u baahan tahay.
Dooro hal cabbir guusha iyo hal xaalad guuldarro ka hor tijaabada.
Ku orod duuliye yar oo wata xogta matale, ee ma aha bandhig muuqaal ah.
Dukumeenti halka Bradley-Terry Abaalmarinta Modeling caawinayo iyo meelaha hababka fudud ay ka fiican yihiin.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Bradley-Terry Reward Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Qaabaynta Abaalmarinta
Su'aalaha soo noqnoqda
What is Bradley-Terry Reward Modeling?
Qaabka Bradley-Terry waa hab xisaabeed qarni jir ah oo loogu beddelayo isbarbardhigga laba-geesoodka ah (A garaaca B) dhibcaha tirada. Casrigan AI waxa ay awood u siinaysaa moodooyinka abaal-marineed ee ka barta dookhyada aadanaha 'jawaabtee ayaa fiican?' calaamadaha, laf dhabarta RLHF.
Muxuu qaabka Bradley-Terry u beddelaa buundooyinka tirada?
Bradley-Terry waxay qaadataa laba-geesood isbarbardhigga 'A garaaca B' oo wuxuu ku cabiraa dhibcaha xoogga qarsoon ee shay kasta, waana sababta ay ugu habboon tahay calaamadaynta doorbidida aadanaha.
Bradley-Terry gudaheeda, itimaalka in A garaaco B waa maxay shaqada?
Qaabku wuxuu dhigayaa P (A garaaca B) oo le'eg saadka (sigmoid) ee farqiga u dhexeeya dhibcaha xoogga qarsoon ee A iyo B.
Waa maxay sababta abaal-marinnada Bradley-Terry loo aqoonsan karo oo keliya ilaa joogto ah wax lagu daro?
Khasaaraha tabobarku wuxuu isticmaalaa kaliya farqiga u dhexeeya abaalmarinta la doortay iyo kuwa la diiday, markaa beddelidda dhammaan buundooyinka isku mid ah waxay ka tagtaa khasaaraha isma beddelin; Miisaanka saxda ah waa gardarro.
Waa maxay khasaaraha caadiga ah ee hal doorbidida marka la barbardhigo qaabaynta abaal-marinta?
Suurtogalnimada Bradley-Terry diidmada waxay hoos u dhigtaa in laga jaro log-sigmoid ee kala duwanaanshaha abaal-marin ee la doortay-laga jaray, taasoo riixaysa abaalmarinta jawaabta la doortay.
Qaabkee ayaa ka boodaya moodal abaal-marineed oo gaar ah iyadoo wanaajinaysa ujeeddada Bradley-Terry si toos ah siyaasadda?
DPO waxay dib u habayn ku samaysaa ujeedadii doorbidida Bradley-Terry si si toos ah loogu dabaqo qaabka siyaasadda, iyada oo meesha ka saaraysa baahida loo qabo in la tababaro oo la waydiiyo qaabka abaal-marineed ee goonida ah.