Chinchilla Compute-Tababarka ugu Fiican
Chinchilla waa DeepMind 2022 oo la ogaaday in moodooyinka waaweyn ee luqadaha intooda badan si xun loo tababaray: miisaaniyad xisaabeed go'an waa inaad cabbirtaa cabbirada iyo xogta si isku mid ah, maaha kaliya inaad dhisto moodal ka weyn.
Dulmar
It reshaped how the industry balances model size against training data.
quusid qoto dheer
Waraaqda DeepMind's Chinchilla waxay dib u eegtay miisaanaynta waxayna tababartay in ka badan 400 oo nooc si loo helo dheelitirka ugu fiican. Xeerka ciwaanka ee suulka: cabbirka moodeelka iyo calaamadaha tababbarka waa in ay koraan meel quful ah, qiyaastii 20 calaamadood oo tababbarro ah halbeeggiiba. Si loo caddeeyo, waxay tababareen Chinchilla, oo ah nooc 70-bilyan-parameter ah oo ku saabsan 1.4 trillion token, iyaga oo isticmaalaya xisaab la mid ah kii 280-bilyan-parameter Gopher oo lagu tababaray calaamado aad uga yar. Chinchilla, in kasta oo ay afar jeer ka yar tahay, waxay ka sarraysay Gopher, GPT-3, iyo kooxaha kale ee ku dhawaad beeg kasta. Casharku waxa uu rogay gabagabada OpenAI hore ee ahaa in cabbirka laga door biday xogta, oo tusaysa noocyo badan oo calanka ah oo ka tagaya waxqabadka miiska iyaga oo aad u weyn oo aad xog u gaajooday.
Aragtida Farsamada
Chinchilla waxay ku habboon tahay luminta sida L (N, D) = E + A·N ^ (-α) + B·D^ (-β), oo leh α iyo β labadaba u dhow 0.34, taasoo la micno ah cabbirrada iyo xogtu waxay gacan ka geystaan ku dhawaad simmetrically. Ku wanaajinta tan iyada oo la raacayo xaddidaad xisaabeed go'an (kumbuyuutar ≈ 6 · N·D ee transformers) waxay soo saartaa natiijada miisaan siman. Qaab ka yar, xogta qani ku ah sidoo kale waa ka raqiisan yahay in lagu socodsiiyo fikradda, sidaa darteed faa'iidooyinkeeda xeryahooda ee geynta, ma aha kaliya tababarka.
Saamaynta Istiraatijiyadeed
Go'aamo cad
Waxay kaa caawinaysaa inaad kala saartid sheegashooyinka farsamada cad iyo luqadda suuq-geynta.
Qiimaha iyo miisaaniyada
Waxaad waydiin kartaa su'aalo fulineed oo wanaagsan ka hor inta aadan lacag ama waqti bixin.
Kooxda iyo socodka shaqada
Kooxaha fahamka la wadaago waxay sameeyaan wax soo saar, siyaasad, iyo go'aano waxbarasho oo wanaagsan.
Mustaqbalka Chinchilla Compute-Tababarka ugu Fiican
Moodooyinka casriga ah sida Llama 3 ayaa si ula kac ah u riixay meel fog Chinchilla's 20-tokens-per-parameter ratio, tababarka moodooyinka yaryar ee trillions of tokens si ay u qiimeeyaan raqiis, aqbala xisaabinta tababarka hoose. Marka xogta wanaagsani sii yaraanayso, xiisaha ayaa kor u kacaya waayihii soo noqnoqda, xogta synthetic, iyo shaandhaynta tayada. Chinchilla ayaa weli ah barta tixraaca, laakiin sida ugu wanaagsan ee sii kordheysa waxay kuxirantahay kharashka nolosha, ma aha kaliya miisaaniyada tababarka hal mar ah.
Dhaqangelinta Adduunka-dhabta ah
Doorashada in lagu tababaro qaab 7-bilyan-beeg-beeg ah oo ku saabsan 2 trillion token halkii ay ka ahaan lahayd qaab 30-bilyan oo xog aad u yar isla miisaaniyada.
Qiyaasta in moodal 10-bilyan oo cabbir ah uu doonayo ku dhawaad 200 bilyan oo calaamadood si uu ugu dhufto barta macaan ee ugu fiican.
Cadaynta nooc ka yar oo la diray si loo dhimo kharashaadka soo jeedinta weydiinta iyadoo la waafajinayo tayada xafiiltanka weyn.
Hanti-dhawrka qaabka jira iyo gunaanadka ayaa ahaa mid si hoose loo tababaray, ka dibna qorsheynta tababar dheer oo dheeraad ah halkii la kordhin lahaa qiyaasta.
Khatarta & Dariiqyada Ilaalada
Kooxo kala duwan ayaa laga yaabaa inay isla erey u isticmaalaan si kala duwan, marka hore u qeex baaxadda.
Tilmaamaha ayaa u ekaan kara kuwo xooggan halka waxqabadka dhabta ah ee dunidu aanu sinnayn.
In la iska indho tiro tayada xogta iyo qorshayaasha qiimayntu waxay inta badan abuurtaa natiijooyin jilicsan.
Qorshe Hawleedka Dhaqangelinta
Ka bilow qeexidda luqadda cad ee natiijada aad u baahan tahay.
Dooro hal cabbir guusha iyo hal xaalad guuldarro ka hor tijaabada.
Ku orod duuliye yar oo wata xogta matale, ee ma aha bandhig muuqaal ah.
Dukumeenti meesha Chinchilla Compute-Tabbarka ugu Fiican ay ka caawiso iyo meelaha hababka fudud ay ka fiican yihiin.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Chinchilla Compute-Optimal Training quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Tababarka-Waqtiga Imtixaanka
Su'aalaha soo noqnoqda
What is Chinchilla Compute-Optimal Training?
Chinchilla waa DeepMind 2022 oo la ogaaday in moodooyinka waaweyn ee luqadaha intooda badan si xun loo tababaray: miisaaniyad xisaabeed go'an waa inaad cabbirtaa cabbirada iyo xogta si isku mid ah, maaha kaliya inaad dhisto moodal ka weyn. Waxay dib u qaabaysay sida warshaduhu isugu dheelitiraan cabbirka moodeelka iyo xogta tababarka.
Waa maxay qaanuunka caanka ah ee Chinchilla ee tababarka ugu fiican ee xisaabinta?
DeepMind waxay heshay halbeegyo iyo calaamaduhu waa inay si wada jir ah u miisaamaan qiyaas ahaan 20 calaamadood halkiibeegga miisaaniyad xisaabeedka la bixiyay.
Sidee baa 70B-parameter Chinchilla la barbardhigay 280B-parameter Gopher?
Waxaa lagu tababaray calaamado aad u badan oo isla xisaabin la mid ah, Chinchilla waxay garaacday Gopher-ka aadka u weyn marka loo eego bartilmaameedyada ugu badan.
Dhibaato noocee ah ayaa warqadda Chinchilla ka muujisay moodooyinka waaweyn ee hore?
Moodooyinka sida GPT-3 iyo Gopher waxay ahaayeen kuwo aad u weyn marka loo eego xogta tababarkooda, taasoo ka dhigeysa waxqabad la'aan.
Qiyaastii imisa calaamadood ayuu xeerka Chinchilla u soo jeedinayaa moodel 10-bilyan oo cabbir ah?
Marka la eego 20 calaamadood halkiibeeg kasta, 10 bilyan oo cabbir ayaa tilmaamaya ilaa 200 bilyan oo calaamado tababar ah.
Waxtarka tababarka ka sokow, muxuu model ka yar, xogta qani ugu yahay soo jiidashada hawlgelinta?
Halbeegyada yaryar waxay ka dhigan yihiin xisaabinta su'aal kasta oo hoose, sidaas darteed xarunta kaydka mar kasta oo qaabka la isticmaalo.