Sharciyada Isku-dheellitirka Chinchilla
Shuruucda miisaanaynta Chinchilla, ee ka soo jeeda DeepMind ee 2022, waxay tuseen in badi moodooyinka luqadaha waaweyn si xun loo tabobaray: miisaaniyad xisaabeed go'an, waa inaad cabbirtaa cabbirka qaabka iyo xogta tababarka qiyaas siman.
Dulmar
It matters because it redefined what 'optimal' model size means and reshaped how labs spend compute.
quusid qoto dheer
Chinchilla ka hor, isbeddelku wuxuu ahaa in la dhiso moodooyinka weligood ka sii weyn (sida 175B-parameter GPT-3) iyadoo la tababarayo qadar yar oo xog ah. DeepMind waxay tababartay in ka badan 400 nooc oo cabbirro badan iyo miisaaniyado xog ah, ka dibna qaloocyada ku habboon saadaalinta luminta iyadoo ay ka mid yihiin cabbirro iyo calaamado hoos yimaada xisaabinta go'an (FLOP). Helitaanka: halbeegyada iyo calaamadaha tababbarku waa in ay isbarbardhigaan, qiyaastii 1-ilaa-1 saamiga, taas oo tusinaysa ilaa 20 calaamadood oo xogta tababarka halbeeggiiba. Si loo caddeeyo, waxay tababareen Chinchilla, oo ah nooc 70B-parameter ah oo ku saabsan 1.4 trillion tokens, kaas oo ka sarreeyay 280B-parameter Gopher inkasta oo la isticmaalo isla xisaabinta, sababtoo ah waxaa lagu tababaray xog aad u badan.
Aragtida Farsamada
Sharciyadu waxay ka imanayaan ku habboonaanta shaqada luminta parametric L(N, D) halkaasoo N ay tahay halbeegyo iyo D ay tahay calaamado, oo ay ku jiraan khasaare-la'aan, cabbir-qaab, iyo ereyada cabbirka xogta. Yaraynta khasaaraha iyada oo loo eegayo xaddidaadda xisaabinta (koombuyuutarku waxay qiyaas ahaan u dhigantaa N times D) waxay soo saartaa natiijada in N iyo D labaduba u koraan sidii awood xisaabeed oo leh jibbaaro la mid ah, markaa saamiga ugu fiican ee xisaabinta ayaa ku dhow 20 token halkiibeeg.
Saamaynta Istiraatijiyadeed
Xawaaraha iyo miisaanka
Socodka shaqada luqaddu si dhakhso leh ayay u socon kartaa iyada oo aan la hurayn joogteynta.
Helitaanka iyo gaarsiinta
Waxay balaadhisaa gelitaanka luqadaha iyo qaababka isgaarsiinta.
Go'aamo cad
Kooxuhu waxay waqti badan ku qaadan karaan xukunka halka otomaatiggu uu qabanayo ku celcelinta.
Mustaqbalka Shuruucda Baadhista Chinchilla
Chinchilla waxay ka beddeshay goobta inay raacdo tirinta cabbirka una wareejisay moodooyinka quudinta xog tayo sare leh oo aad uga badan, moodooyinka casriga ahi waxay inta badan si fiican u tababbaraan inay dhaafaan barta 'compute-optima' si ay uga dhigaan raqiis. Maaddaama qoraalka mareegaha tayada sare leh uu noqdo mid gabaabsi ah, dareenka ayaa u soo jeestay hagaajinta xogta, xogta synthetic, xilliyo badan, iyo xogta qaab-dhismeedka kala duwan si loo ilaaliyo miisaanka. Casharka udub-dhexaadka ahi waa uu jiraa: xogta iyo cabbiraadaha waa in ay ahaadaan kuwo dheellitiran, iyo cabbirka cayriin oo keliya hadda ma aha yoolka.
Dhaqangelinta Adduunka-dhabta ah
DeepMind's 70B-parameter Chinchilla waxay ku garaacday 280B Gopher ee bartilmaameedka iyadoo la adeegsanayo xisaabin siman, iyadoo la tababarayo xog aad u badan.
Kooxaha hagaaya in ay miisaaniyadaan ku dhawaad 20 calaamadood oo tababar halkiibeeg marka la qorshaynayo qaab-xog-xog
Cadaynta moodooyinka yar yar, xogta hodanka ku ah sida LLAMA kuwaas oo ka raqiisan in lagu shaqeeyo wakhtiga fikradda
Qiimaynta in qaabka la qorsheeyay uu 'si hoose loo tabobaran' oo uu ka faa'iidaysan doono xog dheeraad ah marka loo eego cabbirrada dheeraadka ah
Khatarta & Dariiqyada Ilaalada
Xaqiiqooyinka dhalanteed waxay si deggan u geli karaan warbixinnada, taageerada socodka, ama natiijooyinka cilmi-baarista.
Dareenka degdega ahi wuxuu abuuri karaa natiijooyin aan iswaafaqayn codsiyada la midka ah.
Xogta qoraalka xasaasiga ah ayaa laga yaabaa in la kashifo haddii kontaroolada gelitaanka ay daciif yihiin.
Qorshe Hawleedka Dhaqangelinta
Qeex qaabka wax soo saarka, codka, iyo heerarka tayada ka hor inta aan la baahin.
Jawaabaha salka ku haya ilo lagu kalsoon yahay mar kasta oo saxnidu ay muhiim tahay.
Hayso isbaarada dib u eegista bini aadamka ee wax soo saarka sare.
Lasoco qaababka guuldarada oo dib u leyli dardargelinta ama socodka shaqada si joogto ah.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Chinchilla Scaling Laws quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Shuruucda Cabbiraadda Shabakadda Neural
Su'aalaha soo noqnoqda
What is Chinchilla Scaling Laws?
Shuruucda miisaanaynta Chinchilla, ee ka soo jeeda DeepMind ee 2022, waxay tuseen in badi moodooyinka luqadaha waaweyn si xun loo tabobaray: miisaaniyad xisaabeed go'an, waa inaad cabbirtaa cabbirka qaabka iyo xogta tababarka qiyaas siman. Waa arrin sababtoo ah waxay dib u qeexday waxa cabbirka moodeelka 'fiican' macnaheedu yahay oo dib u qaabeeyey sida shaybaadhku u xisaabiyo.
Maxay ahayd natiijada udub-dhexaadka ah ee sharciyada miisaannada Chinchilla?
Chinchilla waxay muujisay in miisaaniyad xisaabeed go'an, cabbirada iyo calaamadaha tababarka ay tahay inay wada koraan, qiyaastii 1-ilaa-1.
Qiyaastii immisa calaamadood oo tababbarro ah halkiibeeg kasta ayay Chinchilla soo jeedisay inay tahay xisaabin-u-fiican?
Saamiga xisaabinta-u-fiican wuxuu ka shaqeeyaa qiyaastii 20 calaamadood oo tababbarro ah oo cabbir kasta ah.
Qaabkee ayuu Chinchilla ka fiicnaa inkastoo uu aad uga yaraa?
70B-parameter Chinchilla ayaa garaacday DeepMind's 280B-parameter Gopher iyadoo la adeegsanayo xisaab isku mid ah, sababtoo ah waxay u tababartay xog aad u badan.
Maxay Chinchilla ka dhigan tahay moodooyinka sida GPT-3 wakhtigaas?
Noocyo badan oo waaweyn oo waagaas ah ayaa si hoose loo tababaray, taasoo la micno ah inay si ka wanaagsan wax ugu qaban lahaayeen xog badan oo tirinta cabbirkooda.
Qiyaas ahaan sidee loo xisaabiyay qiyaas ahaan falanqaynta Chinchilla?
Xisaabinta tababbarka (FLOPs) waxay qiyaastii u dhigantaa tirada cabbirrada lagu dhufto tirada calaamadaha tababarka.