Chinchilla Compute-Mafi kyawun horo
Chinchilla shine 2022 DeepMind gano cewa galibin manyan nau'ikan harshe ba a horar da su sosai: don ƙayyadadden lissafin kasafin kuɗi ya kamata ku daidaita sigogi da bayanai daidai gwargwado, ba kawai gina babban samfuri ba.
Dubawa
It reshaped how the industry balances model size against training data.
Zurfafa nutsewa
Takardar Chinchilla ta DeepMind ta sake duba sikeli kuma ta horar da samfura sama da 400 don nemo ma'auni-mafi kyawu. Ka'idar babban yatsan yatsa: Girman samfuri da alamun horo yakamata suyi girma a cikin kulle-kulle, kusan alamun horo 20 a kowace siga. Don tabbatar da hakan, sun horar da Chinchilla, samfurin siga-biliyan 70 akan alamomin tiriliyan 1.4, ta yin amfani da lissafi iri ɗaya da na Gopher-biliyan 280 da aka horar akan alamomi kaɗan. Chinchilla, duk da kasancewarsa sau huɗu ƙarami, ta fi Gopher, GPT-3, da sauran ƙattai akan kusan kowane ma'auni. Darasi ya juyar da ƙarshe na OpenAI na farko cewa girman fifiko fiye da bayanai, yana nuna yawancin ƙirar ƙira suna barin aiki akan tebur ta zama babba da yunwar bayanai.
Fahimtar Fasaha
Chinchilla dace asara kamar yadda L (N, D) = E + A·N ^ (-α) + B·D ^ (-β), tare da α da β duka kusa da 0.34, ma'ana sigogi da bayanai suna ba da gudummawa kusan daidai. Haɓaka wannan a ƙarƙashin ƙayyadadden ƙayyadaddun ƙididdigewa (ƙididdigar ≈ 6 · N·D don tasfoma) yana haifar da daidaitaccen sakamako. Karami, samfurin mai wadatar bayanai shima yana da arha don gudanar da tunani, don haka fa'idarsa a cikin turawa, ba kawai horo ba.
Dabarun Tasiri
Shawarwari masu haske
Yana taimaka muku keɓance bayyanannen da'awar fasaha daga harshen talla.
Kudin da kasafin kuɗi
Kuna iya yin mafi kyawun tambayoyin aiwatarwa kafin kashe kuɗi ko lokaci.
Ƙungiya da aikin aiki
Ƙungiyoyin da ke da fahimtar juna suna yin mafi kyawun samfura, manufofi, da yanke shawara na koyo.
Makomar Chinchilla Compute-Mafi kyawun horo
Nau'in zamani kamar Llama 3 da gangan sun ƙetare nisa na Chinchilla's 20-tokens-per-parameter ratio, horar da ƙananan ƙira akan tiriliyoyin alamu don yin rahusa, karɓar ƙididdige horo na ƙasa. Yayin da bayanai masu kyau ke yin karanci, sha'awa na karuwa a cikin lokuta masu maimaitawa, bayanan roba, da tacewa mai inganci. Chinchilla ya kasance maƙasudin magana, amma mafi kyawun ƙara ya dogara da ƙimar ƙimar rayuwa, ba kawai kasafin horo na lokaci ɗaya ba.
Aiwatar da Gaskiyar Duniya
Zaɓin horar da samfurin siga na biliyan 7 akan alamomin tiriliyan 2 maimakon ƙirar biliyan 30 akan ƙarancin bayanai don kasafin kuɗi ɗaya.
Ƙididdiga cewa samfurin siga-biliyan 10 yana son alamun kusan biliyan 200 don buga mafi kyawun wuri mai daɗi.
Tabbatar da ƙaramin samfurin da aka tura don rage ƙimar ƙimar ƙimar kowane tambaya yayin da ya dace da ingancin abokin hamayya.
Binciken samfurin da ake da shi tare da kammala shi ba a horar da shi ba, sannan kuma tsara tsarin horo mai tsawo maimakon haɓakar siga.
Hatsari & Tsare-tsare
Ƙungiyoyi daban-daban na iya amfani da kalmar iri ɗaya daban, don haka ayyana iyaka da wuri.
Alamomi na iya yin kama da ƙarfi yayin da aikin zahirin duniya bai yi daidai ba.
Yin watsi da ingancin bayanai da tsare-tsaren kimantawa galibi yana haifar da sakamako mara ƙarfi.
Taswirar Hanya
Fara da ma'anar harshe a sarari na sakamakon da kuke buƙata.
Zaɓi ma'aunin nasara ɗaya da yanayin gazawa ɗaya kafin gwaji.
Gudun ƙaramin matukin jirgi tare da bayanan wakilci, ba saitin demo da aka goge ba.
Takardun inda Chinchilla Compute-Mafi kyawun horo ke taimakawa kuma inda hanyoyin mafi sauƙi suka fi kyau.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Chinchilla Compute-Optimal Training quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Horon-Lokacin Gwaji
Tambayoyin da ake yawan yi
What is Chinchilla Compute-Optimal Training?
Chinchilla shine 2022 DeepMind gano cewa galibin manyan nau'ikan harshe ba a horar da su sosai: don ƙayyadadden lissafin kasafin kuɗi ya kamata ku daidaita sigogi da bayanai daidai gwargwado, ba kawai gina babban samfuri ba. Ya sake fasalin yadda masana'antar ke daidaita girman samfurin akan bayanan horo.
Menene sanannen ƙa'idar babban yatsa na Chinchilla don ƙididdige mafi kyawun horo?
DeepMind ya samo sigogi da alamun ya kamata su daidaita tare a kusan alamu 20 a kowace siga don ƙididdige kasafin kuɗi.
Ta yaya ma'aunin 70B-Chinchilla ya kwatanta da Gopher-parameter 280B?
An horar da su akan ƙarin alamu masu ƙididdigewa iri ɗaya, Chinchilla ta doke Gopher mafi girma a mafi yawan ma'auni.
Wace babbar matsala takardar Chinchilla ta bayyana game da manyan samfuran da suka gabata?
Samfura kamar GPT-3 da Gopher sun kasance babba dangane da bayanan horon su, suna barin aikin ba a gane su ba.
Kusan alamu nawa ne dokar Chinchilla ke ba da shawarar samfurin siga na biliyan 10?
A alamu 20 a kowace siga, sigogi biliyan 10 suna nuna alamun horo kusan biliyan 200.
Bayan ingantacciyar horarwa, me yasa ƙarami, samfuri mai wadatar bayanai ke da kyau wajen turawa?
Ƙananan sigogi suna nufin ƙananan ƙididdige kowane tambaya, don haka ma'auni na tanadi duk lokacin da aka yi amfani da samfurin.