Nlele Nlele Ọjụjụ Dị Mma-Tuning
Ọjụjụ Sampling Fine-Tuning (RFT) na-ewepụta ọtụtụ azịza ndị ndoro-ndoro ochichi, na-edobe naanị ndị kacha nwee akara, ma na-azụghachi ihe nlereanya na ndị mmeri ahụ.
Nchịkọta
It matters because it offers much of RLHF's benefit using straightforward supervised learning instead of complex reinforcement learning.
Ime miri emi
Nlele nlegharị anya dị mma, nke a na-akpọ mgbe ụfọdụ kacha mma-nke-N mma-tuning, bụ isi ihe na otu ụdị dịka Meta's Llama 2 na Llama 3 si kwekọọ. Ntụziaka dị mfe: maka ngwa ngwa ọ bụla, lelee nzaghachi dị iche iche (kwuo 4 ruo 64) site na ụdị dị ugbu a, tụọ nke ọ bụla n'ụdị ụgwọ ọrụ ma ọ bụ onye na-enyocha akpaaka, wee tụfuo ('jụ') niile ma ọ bụghị ọkwa kachasị elu. Ihe nlele dị elu nke dị ndụ na-aghọ dataset nlegharị anya nke ọma na-elekọta, a na-azụkwa ihe nlereanya na ha na ọnwụ na-esote. Ikwughachi loop a na-eweghachite usoro ahụ n'ịwepụta azịza ndị ka mma n'onwe ya. N'ihi na ihe nlereanya ahụ na-amụta site na nzacha nke ya, RFT na-ezere enweghị ntụkwasị obi na imezi isi ọwụwa nke amụma-gradient RL ka ọ ka na-eji akara ngosi ụgwọ ọrụ.
Nghọta nka nka
RFT na-erigbu eziokwu ahụ bụ na ịlele ọtụtụ oge yana idobe nzaghachi ụgwọ ọrụ kacha nso na-ewere site na nkesa dị nkọ, dị elu. Ọzụzụ na ndị mmeri ahụ site na ọkọlọtọ cross-entropy na-eme ka omume ahụ kachasị mma laghachi azụ na nsonye otu ihe nlereanya. Maka ngalaba enwere ike ịgbagha dị ka mgbakọ na mwepụ ma ọ bụ koodu, 'ụgwọ ọrụ' nwere ike ịbụ ma azịza ikpeazụ ma ọ bụ ule nkeji gafere, wepụ mkpa maka ụdị ụgwọ ọrụ mmụta kpamkpam.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke Nlele Nlebaanya Ọdịnihu
RFT bụ etiti na ọzụzụ ọzụzụ ọgbara ọhụrụ, nke a na-ejikarị tupu ma ọ bụ n'akụkụ ụzọ RL dị ka PPO na DPO. Mkpesa ya na-eto site na ntinye ọnụ ala yana ndị na-enyocha akpaaka siri ike: ka ụdị na-akawanye mma na ịmepụta onwe na nyocha onwe ya, nlele ajụjụ ọnụ na-akwado synthetic-data na nwelite loops. Na-atụ anya njikọta siri ike na ụdị ntụgharị uche nke na-emepụta ụdọ echiche a ga-ekwe omume, na ọmụmụ ihe na-aga n'ihu maka otu esi zere hacking ụgwọ ọrụ na ndakpọ dị iche iche mgbe ọzụzụ ugboro ugboro na mmepụta nke ihe nlereanya.
Mmejuputa n'ezie n'ụwa
Idozi ụdị ụdị Llama site n'ịlele azịza ọtụtụ ajụjụ n'otu oge, na-edobe akara ngosi kacha elu, wee SFT na ndị ahụ.
Imelite onye na-edozi mgbakọ na mwepụ site n'ịwepụta ọtụtụ ngwọta na idowe naanị ndị ruru azịza ziri ezi, enwere ike ịlele
Ọgbọ koodu ebe a na-edobe ndị mmeri naanị ma ọ bụrụ na ha gafere ule otu, wee jiri ya dị ka data ọzụzụ
Iwulite akwụkwọ ntuziaka sịntetik site na nzacha azịza nke kacha mma nke ihe nlereanya nwere maka agba ọzụzụ na-esote.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Rejection Sampling Fine-Tuning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
QLoRA na 4-Bit Fine-Tuning
Ajụjụ a na-ajụkarị
What is Rejection Sampling Fine-Tuning?
Ọjụjụ Sampling Fine-Tuning (RFT) na-ewepụta ọtụtụ azịza ndị ndoro-ndoro ochichi, na-edobe naanị ndị kacha nwee akara, ma na-azụghachi ihe nlereanya na ndị mmeri ahụ. Ọ dị mkpa n'ihi na ọ na-enye ọtụtụ uru RLHF site na iji mmụta ziri ezi na-elekọta kama mmụta nkwado siri ike.
Gịnị bụ isi echiche nke Ọjụjụ Sampling Fine-Tuning?
RFT na-enyocha ọtụtụ nzaghachi, na-enyocha ndị kacha nwee akara, ma na-emegharị ihe nlereanya na ndị mmeri ahụ.
N'ime ngalaba nwere ike ịnwapụta dị ka mgbakọ na mwepụ ma ọ bụ koodu, gịnị nwere ike ịbụ ụgwọ ọrụ?
Maka ọrụ enwere ike ịlele, RFT nwere ike iji izi ezi ma ọ bụ ule nkeji gafere, na-ezere ụdị ụgwọ ọrụ mụtara.
Kedu ezinaụlọ nlereanya nke ama ama jiri nlele jụrụ ajụ n'oge nhazi?
Meta kọwara iji nlere njụta dị ka akụkụ nke usoro ọzụzụ ọzụzụ maka ụdị Llama 2 na Llama 3.
Kedu ihe kpatara ndị otu nwere ike ji ahọrọ RFT karịa iwu-gradient RL dị ka PPO?
RFT na-ejighachi ọkọlọtọ ọkọlọtọ na-esote n'ụdị nzacha, na-ewepụ ihe isi ike n'iji ya gee ntị na enweghị ntụkwasị obi nke ụzọ amụma-gradient.
Ọzụzụ na kacha-ụgwọ ọrụ samples n'ụzọ dị irè na-eme gịnị?
Site n'ịmụta site na nzaghachi ndị a gbazere n'elu, ihe nlereanya a na-eme ka omume dị elu dị elu n'otu ọgbọ.