Samfuran Kyauta
Samfurin lada wata hanyar sadarwa ce ta jijiyoyi da aka horar da ita don hasashen yadda kyakkyawar amsawar AI take, tana aiki azaman tsayawa ta atomatik don yanke hukuncin ɗan adam.
Dubawa
Injin mai ƙira ne ke ba da ƙarfafa koyo daga ra'ayoyin ɗan adam mai yiwuwa a sikeli.
Zurfafa nutsewa
Tsarin lada yana magance matsala mai amfani: mutane ba za su iya ƙididdige kowane ɗayan miliyoyin abubuwan da samfurin ke samarwa yayin horo ba. Madadin haka, masu lakabin suna kwatanta ƙaramin saƙon martani, yawanci ɗaukar wanne daga cikin amsoshi biyu zuwa ga faɗakarwa ɗaya ya fi kyau. Sannan ana horar da samfurin lada akan waɗannan kwatancen don fitar da ma'auni guda ɗaya don kowane nau'i-nau'i-mai amsawa. Maƙasudin horarwa shine ƙirar Bradley-Terry, wanda ke juya zaɓin biyu zuwa yuwuwar amsa ɗaya ta zarce wani. Da zarar an horar da shi, wannan samfurin lada zai iya kimanta sabbin abubuwan da ba su da iyaka a arha, yana ba da siginar da algorithms kamar PPO ke amfani da shi don haɓaka ƙirar harshe. Hakanan ana sake amfani da samfuran lada a lokacin ƙididdigewa don mafi kyawun samfuri na N, inda aka samar da ƴan takara da yawa kuma aka dawo da wanda ya fi nasara.
Fahimtar Fasaha
Samfurin lada yawanci shine ƙirar harshe mai tushe tare da kan alamar hasashen sa wanda aka maye gurbinsa da layin layi ɗaya wanda ke fitar da sikeli ɗaya. Horowa yana haɓaka yuwuwar log ɗin cewa zaɓin amsa ya yi sama da wanda aka ƙi: asara = -log(sigmoid(r_chosen - r_rejected)). Bambancin dangi kawai yana da mahimmanci, don haka cikakkiyar ma'auni na sabani. Ingantacciyar ma'auni akan daidaiton lakabin da faffadan ɗaukar hoto na salon amsawa.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Samfuran Lada
Bincike yana magance manyan gazawar samfuran lada: ana iya 'ɓata su' (samfuran suna amfani da ƙima kamar tsayin daka), kuma suna fita daga rarraba yayin da manufofin ke inganta. Sharuɗɗan alƙawarin sun haɗa da tsarin lada na tsari waɗanda ke ƙididdige kowane matakin tunani, tarawa da ƙididdiga marasa tabbas don ƙin shiga ba tare da izini ba, alamomin fifikon AI da aka ƙirƙira (RLAIF), da samfuran lada masu ƙima waɗanda ke samar da zargi da dalilai maimakon ƙima.
Aiwatar da Gaskiyar Duniya
Ƙarfafa RLHF don mataimaka kamar ChatGPT da Claude ta hanyar ba da amsan ɗan takara yayin horon PPO
Mafi kyawun samfurin N, inda samfurin ke samar da amsoshi da yawa kuma samfurin lada ya zaɓi mafi kyau ga mai amfani
Math da codeing 'masu tabbatarwa' ko aiwatar da tsarin lada waɗanda ke nuna matsakaicin matakan tunani don haɓaka warware matsala
Matsayi da tace bayanan horo na roba, adana tsararraki masu ƙima kawai don ƙarin daidaitawa
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Reward Modeling quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Samfuran Kyautar Bradley-Terry
Tambayoyin da ake yawan yi
Menene Samfuran Kyauta?
Samfurin lada wata hanyar sadarwa ce ta jijiyoyi da aka horar da ita don hasashen yadda kyakkyawar amsawar AI take, tana aiki azaman tsayawa ta atomatik don yanke hukuncin ɗan adam. Injin ƙira ne ke ba da ƙarfafa koyo daga amsawar ɗan adam mai yiwuwa a sikeli.
Menene farkon fitowar samfurin lada don nau'ikan amsa da sauri?
Samfurin lada yana tsara taswirar gaggawa da amsa ga lamba ɗaya mai wakiltar inganci ko fifikon ɗan adam.
Wane irin bayanan ɗan adam ne aka fi amfani da shi don horar da ƙirar lada?
Masu lakabin yawanci suna kwatanta martani biyu zuwa ga sauri ɗaya kuma su zaɓi mafi kyau; Samfurin Bradley-Terry ya canza waɗannan zuwa sakamako mai ƙima.
Menene daidaitaccen hasara-samfurin lada ke inganta?
Asarar Bradley-Terry, -log (sigmoid (r_chosen - r_rejected)), kawai yana kula da odar dangi, don haka cikakkiyar ma'auni mai sabani ne.
Menene 'Hacking na lada'?
Hacking ɗin lada yana faruwa ne lokacin da ƙirar ta sami gajerun hanyoyi (kamar tsayin daka mai yawa ko baƙar magana) wanda ƙirar lada mara kyau tana ƙima sosai amma mutane ba za su yi ba.
Ta yaya samfurin lada ya samo asali daga tsarin harshe?
Samfurin lada yawanci yana sake amfani da kashin baya na LM amma yana musanya Layer na ƙarshe don wanda ke fitar da lada guda ɗaya.