Ƙimar Samfura Mai Kyau-Tuning
Rejection Sampling Fine-Tuning (RFT) yana haifar da amsoshi da yawa na ɗan takara, yana kiyaye mafi kyawun maki kawai, kuma yana sake horar da ƙima akan waɗanda suka yi nasara.
Dubawa
It matters because it offers much of RLHF's benefit using straightforward supervised learning instead of complex reinforcement learning.
Zurfafa nutsewa
Ƙimar Samfuran Ƙarfafa-Tuning, wani lokacin ana kiransa mafi kyawun-na-tun-tuning mai kyau, shine maɓalli mai mahimmanci a cikin yadda aka daidaita samfura kamar Meta's Llama 2 da Llama 3. Girke-girke mai sauƙi ne: ga kowane faɗakarwa, samfurin amsa da yawa (ce 4 zuwa 64) daga samfurin na yanzu, saka kowanne da samfurin lada ko mai duba atomatik, sannan jefar ('ƙi') duka sai dai manyan abubuwan da aka samu. Samfurori masu inganci masu tsira sun zama sabbin bayanai masu kyau da ake kula da su, kuma an horar da samfurin akan su tare da asarar gaba-gaba na yau da kullun. Maimaita wannan madauki akai-akai yana karkatar da samfurin zuwa samar da ingantattun amsoshi da kansa. Saboda samfurin yana koya daga abubuwan da aka tace, RFT yana guje wa rashin zaman lafiya da daidaita ciwon kai na manufofin-gradient RL yayin da har yanzu yana ba da siginar lada.
Fahimtar Fasaha
RFT yana amfani da gaskiyar cewa yin samfuri sau da yawa da kuma kiyaye matsakaicin martanin sakamako mai ƙima wanda aka ɗauka daga ƙayyadaddun rarraba mai inganci. Horarwa akan waɗancan masu nasara ta hanyar daidaitaccen giciye-entropy yadda ya kamata yana kawar da mafi kyawun halayen N baya cikin samfuran samfuri guda ɗaya. Don wuraren da za a iya tabbatarwa kamar lissafi ko lamba, 'ladan' na iya kasancewa kawai ko amsar ƙarshe ko gwajin naúrar ta wuce, cire buƙatar samfurin lada koyo gaba ɗaya.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Ƙimar Samfura Mai Kyau
RFT shine tsakiya ga horo na zamani, yawanci ana amfani dashi kafin ko tare da hanyoyin RL kamar PPO da DPO. Rokonsa yana girma tare da rahusa rahusa da masu tabbatarwa ta atomatik mai ƙarfi: yayin da ƙirar ke samun ƙwaƙƙwara a ƙirƙira kai da bincika kai, ƙirƙira samfurin ƙira yana goyan bayan bayanan roba da madaukai na haɓaka kai. Yi tsammanin haɗin kai tare da ƙirar tunani waɗanda ke samar da tabbataccen sarƙoƙi na tunani, da ci gaba da nazarin yadda za a guje wa satar lada da rugujewar bambance-bambance yayin horo akai-akai kan abin da samfurin ya samu.
Aiwatar da Gaskiyar Duniya
Daidaita nau'ikan nau'ikan Llama ta hanyar samar da amsoshi da yawa a cikin gaggawa, kiyaye mafi girman ƙididdiga-samfurin, sannan SFT akan waɗancan.
Inganta mai warware lissafi ta hanyar samar da mafita da yawa da kuma riƙe waɗanda suka kai daidai, amsar da za a iya bincikawa kawai
Ƙirƙirar lambar inda ake ajiye ƴan takara kawai idan sun ci jarrabawar raka'a, sannan a yi amfani da su azaman bayanan horo
Gina bayanan koyarwar roba ta hanyar tace samfurin nasa mafi kyawun martanin da ya samar da kansa don zagayen horo na gaba.
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Rejection Sampling Fine-Tuning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
QLoRA da 4-Bit Fine-Tuning
Tambayoyin da ake yawan yi
What is Rejection Sampling Fine-Tuning?
Rejection Sampling Fine-Tuning (RFT) yana haifar da amsoshi da yawa na ɗan takara, yana kiyaye mafi kyawun maki kawai, kuma yana sake horar da ƙima akan waɗanda suka yi nasara. Yana da mahimmanci saboda yana ba da yawancin fa'idodin RLHF ta amfani da koyo mai kulawa kai tsaye maimakon haɗaɗɗiyar ƙarfafa koyo.
Menene ainihin ra'ayin kin Samfur Fine-Tuning?
Samfuran RFT suna ba da amsa da yawa, tacewa ga waɗanda suka fi zira kwallaye, kuma suna daidaita ƙirar akan waɗanda suka yi nasara.
A cikin wuraren da za a iya tabbatarwa kamar lissafi ko lamba, menene zai iya zama lada?
Don ayyukan da za a iya bincikawa, RFT na iya amfani da daidaitattun daidaito ko gwaje-gwajen naúrar, guje wa samfurin lada da aka koya.
Wanne iyali ne suka shahara suka yi amfani da samfurin kin amincewa yayin jeri?
Meta ya bayyana ta yin amfani da ƙidayar ƙima a matsayin wani ɓangare na girke-girke na bayan horo don ƙirar Llama 2 da Llama 3.
Me yasa ƙungiyoyi zasu fi son RFT akan tsarin tsarin RL kamar PPO?
RFT ya sake horarwa tare da daidaitaccen asarar alama ta gaba akan samfuran da aka tace, tare da kawar da wahalar daidaitawa da rashin kwanciyar hankali na hanyoyin dabarun-girmamawa.
Horo akan mafi girman samfuran lada yadda ya kamata yayi menene?
Ta hanyar koyo daga manyan bayanan da aka tace, samfurin yana shigar da ɗabi'a mai inganci a cikin tsara guda.