Dokokin Sikelin Chinchilla
Dokokin sikelin Chinchilla, daga DeepMind a cikin 2022, sun nuna cewa galibin manyan nau'ikan harshe ba su da kyau sosai: don ƙayyadadden lissafin kasafin kuɗi, yakamata ku daidaita girman samfurin da bayanan horo daidai gwargwado.
Dubawa
It matters because it redefined what 'optimal' model size means and reshaped how labs spend compute.
Zurfafa nutsewa
Kafin Chinchilla, yanayin shine gina ƙira mafi girma (kamar 175B-parameter GPT-3) yayin da ake horarwa akan ƙaramin adadin bayanai. DeepMind ya horar da sama da nau'ikan 400 a cikin masu girma dabam da kuma kasafin kuɗi na bayanai, sannan ya dace da masu lankwasa tsinkayar asara azaman aikin sigogi da alamu ƙarƙashin ƙayyadaddun lissafin ƙididdiga (FLOP). Neman su: sigogi da alamun horo yakamata su daidaita tare, kusan kashi 1-zuwa-1, yana nuna kusan alamomi 20 na bayanan horo kowace siga. Don tabbatar da hakan, sun horar da Chinchilla, samfurin siga na 70B akan alamomin tiriliyan 1.4, wanda ya zarce Gopher mai girman 280B-parameter mafi girma duk da yin amfani da lissafi iri ɗaya, saboda an horar da shi akan ƙarin bayanai.
Fahimtar Fasaha
Dokokin sun fito ne daga dacewa da aikin hasarar ma'auni L(N, D) inda N shine sigogi kuma D alama ce, gami da rashin lalacewa, girman ƙira, da sharuddan girman bayanai. Rage asarar da ke ƙarƙashin ƙayyadaddun ƙididdigewa (ƙididdigar ta kusan daidai da lokutan N sau D) yana haifar da sakamakon cewa mafi kyawun N da D duka suna girma azaman ƙarfin ƙididdigewa tare da ma'auni iri ɗaya, don haka ƙididdige mafi kyawun rabo yana zama kusa da alamu 20 a kowace siga.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Dokokin Scaling Chinchilla
Chinchilla ya canza filin daga bin ƙididdigar ma'auni zuwa tsarin ciyar da ƙididdiga masu inganci sosai, kuma samfuran zamani galibi suna horarwa da wuce ma'anar 'mafi kyawun lissafi' don yin rahusa. Yayin da ingantattun rubutun gidan yanar gizo ke zama karanci, hankali yana juyowa ga sarrafa bayanai, bayanan roba, zamanin da yawa, da bayanan multimodal don ci gaba da ƙima. Babban darasi yana dawwama: bayanai da sigogi dole ne su daidaita, kuma danyen girman kawai ba shine burin ba.
Aiwatar da Gaskiyar Duniya
DeepMind's 70B-parameter Chinchilla yana bugun 280B Gopher akan ma'auni ta amfani da ƙididdiga daidai, ta hanyar horarwa akan ƙarin bayanai.
Ƙungiyoyi masu ba da jagoranci don yin kasafin kuɗi kusan alamun horo 20 a kowace siga lokacin da suke tsara ƙirar ƙira.
Tabbatar da ƙarami, samfura masu wadatar bayanai kamar LLAMA waɗanda ke da arha don aiki a lokacin ƙididdigewa
Ƙididdiga ko ƙirar da aka tsara ba a 'ƙasa horo' kuma zai amfana da ƙarin bayanai fiye da ƙarin sigogi
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Chinchilla Scaling Laws quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Dokokin Sikeli don hanyoyin sadarwar Jijiya
Tambayoyin da ake yawan yi
What is Chinchilla Scaling Laws?
Dokokin sikelin Chinchilla, daga DeepMind a cikin 2022, sun nuna cewa galibin manyan nau'ikan harshe ba su da kyau sosai: don ƙayyadadden lissafin kasafin kuɗi, yakamata ku daidaita girman samfurin da bayanan horo daidai gwargwado. Yana da mahimmanci saboda ya sake fayyace ma'anar girman samfurin 'mafi kyau' kuma ya sake fasalin yadda labs ke kashe lissafin.
Menene tsakiyar binciken da dokokin sikelin Chinchilla?
Chinchilla ya nuna cewa don ƙayyadaddun lissafin lissafin kasafin kuɗi, sigogi da alamun horo yakamata su girma tare, kusan 1-to-1.
Kusan alamun horo nawa ne a kowace siga ta Chinchilla ta ba da shawarar mafi inganci?
Ƙididdigar mafi kyawun ƙididdiga tana aiki zuwa kusan alamun horo 20 don kowane sigar ƙira.
Wanne samfurin Chinchilla ya yi fice duk da kasancewarsa karami?
Ma'aunin 70B-Chinchilla ya doke Gopher na DeepMind na kansa 280B-parameter ta amfani da lissafi iri ɗaya, saboda ya horar da ƙarin bayanai.
Menene Chinchilla ke nufi game da samfura kamar GPT-3 a lokacin?
Yawancin manyan samfura na wancan lokacin ba a horar da su ba, ma'ana da sun yi aiki mafi kyau tare da ƙarin bayanai don ƙidayar sigar su.
Kusan ta yaya aka ƙididdige ƙididdiga a cikin binciken Chinchilla?
Ƙididdigar horo (FLOPs) yana kusan daidai da adadin sigogi da aka ninka ta adadin alamun horo.