Imithetho ye-Chinchilla Scaling
Imithetho yokukala ye-Chinchilla, evela ku-DeepMind ngo-2022, yabonisa ukuthi amamodeli amaningi ezilimi amakhulu ayengaqeqeshwanga kahle kakhulu: kubhajethi yekhompiyutha engaguquki, kufanele ukale usayizi wamamodeli kanye nedatha yokuqeqeshwa cishe ngesilinganiso esilinganayo.
Uhlolojikelele
It matters because it redefined what 'optimal' model size means and reshaped how labs spend compute.
I-Deep Dive
Ngaphambi kwe-Chinchilla, inkambiso bekuwukwakha amamodeli amakhudlwana njalo (njengepharamitha engu-175B GPT-3) kuyilapho uqeqeshwa ngamanani amancane kakhulu edatha. I-DeepMind iqeqeshe amamodeli angaphezu kuka-400 kumasayizi amaningi nesabelomali sedatha, bese ilinganisa amajika abikezela ukulahleka njengomsebenzi wamapharamitha namathokheni ngaphansi kwebhajethi yekhompuyutha egxilile (i-FLOP). Ukuthola kwabo: amapharamitha namathokheni okuqeqesha kufanele alinganise ndawonye, cishe isilinganiso esingu-1 kuya ku-1, okusho cishe amathokheni angama-20 wedatha yokuqeqeshwa ipharamitha ngayinye. Ukufakazela lokho, baqeqesha Chinchilla, imodeli 70B-parameter ku-1.4 amathokheni ayizigidi eziyizinkulungwane ezingu, okuyinto yaphumelela kakhulu 280B-parameter Gopher naphezu kokusebenzisa ikhompuyutha efanayo, ngoba waqeqeshelwa idatha kude kakhulu.
I-Technical Insight
Imithetho ivela ekufakeni umsebenzi wokulahlekelwa kwepharamethikhi L(N, D) lapho okuthi N kuyimingcele futhi D kungamathokheni, okuhlanganisa ukulahlekelwa okungenakunqandeka, usayizi wemodeli, namagama osayizi wedatha. Ukunciphisa ukulahlekelwa kuncike ekuvinjweni kwekhompuyutha (ukubala kucishe kulingane nezikhathi ezingu-N D) kunikeza umphumela wokuthi i-N no-D efanelekile kokubili kukhule njengamandla wokubala ngama-exponents afanayo, ngakho-ke isilinganiso sekhompiyutha esilungile sihlala eduze kwamathokheni angu-20 ipharamitha ngayinye.
I-Strategic Impact
Isivinini nesikali
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Finyelela futhi ufinyelele
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Izinqumo ezicacile
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Ikusasa Lemithetho Yokukala I-Chinchilla
I-Chinchilla ishintshe inkundla isuka ekujaheni ipharamitha yayisa kumamodeli wokuphakela idatha yekhwalithi ephezulu kakhulu, futhi amamodeli esimanje avame ukuqeqesha adlule iphuzu 'le-compute-optimal' ukuze enze ukucabangela kushibhe. Njengoba umbhalo wewebhu wekhwalithi ephezulu uya untuleka, ukunakwa kuphendukela ekwakhiweni kwedatha, idatha yokwenziwa, izinkathi eziningi, kanye nedatha ye-multimodal ukuze kuqhubeke ukukala. Isifundo esiwumongo siyaqhubeka: idatha namapharamitha kufanele kulinganiswe, futhi usayizi ongahluziwe wodwa awusewona umgomo.
Ukuqaliswa Komhlaba Wangempela
I-DeepMind's 70B-parameter Chinchilla ihlula i-280B Gopher kumabhentshimakhi isebenzisa ikhompuyutha elinganayo, ngokuqeqeshwa ngedatha eyengeziwe.
Amaqembu aqondisayo ukuthi enze ibhajethi cishe amathokheni okuqeqesha angama-20 ngepharamitha ngayinye lapho ehlela imodeli esuka ekuqaleni
Ukuqinisekisa amamodeli amancane, anothile ngedatha njenge-LLaMA ashibhile ukusebenzisa ngesikhathi sokunquma
Ukulinganisa ukuthi imodeli ehleliwe 'ayiqeqeshelwanga ngokwanele' futhi ingazuza kakhulu kudatha eyengeziwe kunamapharamitha engeziwe
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Chinchilla Scaling Laws quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Imithetho yokukala yeNeural Networks
Imibuzo evame ukubuzwa
What is Chinchilla Scaling Laws?
Imithetho yokukala ye-Chinchilla, evela ku-DeepMind ngo-2022, yabonisa ukuthi amamodeli amaningi ezilimi amakhulu ayengaqeqeshwanga kahle kakhulu: kubhajethi yekhompiyutha engaguquki, kufanele ukale usayizi wamamodeli kanye nedatha yokuqeqeshwa cishe ngesilinganiso esilinganayo. Ibalulekile ngoba ichaze kabusha ukuthi kusho ukuthini usayizi wemodeli 'okulungile' futhi yabumba kabusha indlela amalebhu achitha ngayo ngokubala.
Yini eyatholwa emaphakathi yemithetho yokukala yeChinchilla?
I-Chinchilla ibonise ukuthi ngesabelomali esinqunyiwe sekhompiyutha, imingcele namathokheni okuqeqesha kufanele akhule ndawonye, cishe i-1-to-1.
Cishe mangaki amathokheni okuqeqesha ngepharamitha uChinchilla aphakamisa ukuthi afaneleke?
Isilinganiso se-compute-optimal ratio sisebenza cishe kumathokheni okuqeqesha angama-20 kuyo yonke ipharamitha yemodeli.
Iyiphi imodeli uChinchilla ayenza kahle kakhulu yize yayimncane kakhulu?
Ipharamitha engu-70B i-Chinchilla yehlula i-DeepMind's i-Gopher engupharamitha engu-280B isebenzisa ikhompuyutha efanayo, ngoba iqeqeshelwe idatha eyengeziwe kakhulu.
IChinchilla yayisho ukuthini ngamamodeli afana ne-GPT-3 ngaleso sikhathi?
Amamodeli amaningi amakhulu aleso sikhathi ayengaqeqeshwanga kahle, okusho ukuthi ngabe enze kangcono ngedatha eyengeziwe yesibalo samapharamitha awo.
Kulinganiselwa ukuthi ukubala kwalinganiselwa kanjani ekuhlaziyeni kwe-Chinchilla?
I-Training compute (i-FLOPs) icishe ilingane nenani lamapharamitha aphindwe ngenani lamathokheni okuqeqesha.