Dokokin Sikeli don hanyoyin sadarwar Jijiya
Dokokin ƙididdige ƙididdiga masu ƙima waɗanda ke nuna cewa asarar hanyar sadarwar jijiyoyi tana faɗuwa da ƙima yayin da kuke girma girman samfuri, girman saitin bayanai, da ƙididdigewa.
Dubawa
They matter because they let researchers forecast performance before spending millions on training a giant model.
Zurfafa nutsewa
Dokokin ƙididdigewa, wanda OpenAI's 2020 takarda ta Kaplan da abokan aiki, sun gano cewa asarar gwajin tana raguwa a matsayin ka'idar wutar lantarki mai santsi cikin adadi uku: ƙididdigar ma'auni (N), alamun horo (D), da jimlar lissafi (C). An ƙera shi akan gatura na log-log, hasara tare da kowane ma'ana yana samar da layi mai kusan madaidaiciya wanda ya mamaye yawancin umarni na girma. Dangantakar suna ɗaukar sigar hasara ≈ a + b·X^(-c), inda X shine ma'aunin sikeli. Mahimmanci, aikin asali ya ba da shawarar girman samfurin ya fi bayanai, yana haifar da tsere zuwa ga ƙira mafi girma kamar sigogin GPT-3 na biliyan 175. Dokokin daidaitawa sun juya zurfin koyo daga zato zuwa tsarin aikin injiniya wanda za'a iya tsinkaya, barin ƙungiyoyi suyi hasashen babban sakamako daga ƙananan gwaje-gwaje masu arha.
Fahimtar Fasaha
Tsarin dokar-ikon yana nufin kowane ƙayyadadden haɓaka mai yawa a cikin ƙididdigewa yana haifar da kusan raguwar ƙari na asara. Ana auna hasara a cikin nats ko bita kowace alamar giciye-entropy. Saboda ma'anar c yana ƙarami (sau da yawa a kusa da 0.05-0.1), samun riba na gaske amma yana raguwa: ƙididdigewa sau biyu yana taimakawa ƙasa da na farko. Mahimmanci, waɗannan dokokin suna bayyana hasara mara lalacewa-da-raguwa, inda akai-akai ke ɗaukar ainihin bayanan bayanan da babu wani samfurin da zai iya doke shi.
Dabarun Tasiri
Shawarwari masu haske
Yana taimaka muku keɓance bayyanannen da'awar fasaha daga harshen talla.
Kudin da kasafin kuɗi
Kuna iya yin mafi kyawun tambayoyin aiwatarwa kafin kashe kuɗi ko lokaci.
Ƙungiya da aikin aiki
Ƙungiyoyin da ke da fahimtar juna suna yin mafi kyawun samfura, manufofi, da yanke shawara na koyo.
Makomar Dokokin Scaling don Cibiyoyin Neural
Masu bincike suna tsawaita dokokin ƙididdiga fiye da ƙaddamar da asara zuwa ƙasa daidaitaccen aiki, ƙirar multimodal, da ƙididdige lokacin ƙididdigewa, inda ƙirar tunani ke ciyar da ƙarin tunani kowace tambaya. Yayin da rubutu mai inganci ya yi karanci, hankali yana jujjuya zuwa ingancin bayanai, bayanan roba, da ka'idojin maimaita bayanai. Wasu suna jayayya da ɗanyen sikelin yana bugun iyakoki masu amfani na kuɗi, kuzari, da rubutu da ake samu, yana tura filin zuwa ingantaccen algorithmic da sabbin gine-gine maimakon kawai haɓaka girma.
Aiwatar da Gaskiyar Duniya
Hasashen hasarar ƙarshe na ƙirar siga-biliyan 70 da aka tsara daga jerin ƙananan gwajin miliyon 100 kafin aiwatar da kasafin kudin GPU.
Yanke shawarar tiriliyoyin nawa don tarawa don haka ƙayyadadden lissafin kasafin kuɗi ba a ɓata akan ƙirar da ba ta da horo.
Kwatanta gine-ginen gine-gine guda biyu da rahusa ta hanyar dacewa da madaidaicin ma'aunin su a ƙaramin sikelin maimakon horar da duka biyun da girmansu.
Saita sahihancin sahihancin tsammanin masu saka hannun jari ko baiwa masu bita kyauta ta hanyar fitar da yanayin hasara zuwa matakin ƙididdige manufa.
Hatsari & Tsare-tsare
Ƙungiyoyi daban-daban na iya amfani da kalmar iri ɗaya daban, don haka ayyana iyaka da wuri.
Alamomi na iya yin kama da ƙarfi yayin da aikin zahirin duniya bai yi daidai ba.
Yin watsi da ingancin bayanai da tsare-tsaren kimantawa galibi yana haifar da sakamako mara ƙarfi.
Taswirar Hanya
Fara da ma'anar harshe a sarari na sakamakon da kuke buƙata.
Zaɓi ma'aunin nasara ɗaya da yanayin gazawa ɗaya kafin gwaji.
Gudun ƙaramin matukin jirgi tare da bayanan wakilci, ba saitin demo da aka goge ba.
Takaddun inda Dokokin Sikeli don Cibiyoyin Sadarwar Jijiya ke taimakawa kuma inda hanyoyin mafi sauƙi suka fi kyau.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Scaling Laws for Neural Networks quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Hanyoyin Sadarwar Jijiya Na Juyin Halitta
Tambayoyin da ake yawan yi
What is Scaling Laws for Neural Networks?
Dokokin ƙididdige ƙididdiga masu ƙima waɗanda ke nuna cewa asarar hanyar sadarwar jijiyoyi tana faɗuwa da ƙima yayin da kuke girma girman samfuri, girman saitin bayanai, da ƙididdigewa. Suna da mahimmanci saboda sun bar masu bincike suyi hasashen aiki kafin kashe miliyoyin akan horar da babban samfuri.
A kan gatura na log-log, ta yaya asarar gwajin yawanci ke nunawa yayin da ƙididdige ƙididdigewa ke ƙaruwa ƙarƙashin dokokin ƙima?
Asara tare da ƙididdigewa, girman ƙira, ko bayanai suna samar da layi na kusa-daidai akan makircin log-log, sa hannun alakar ikon-dokar.
Wadanne adadi uku ne ainihin dokokin sikelin ke da alaƙa da asara?
Kaplan et al. yayi karatun hasara azaman dokar wuta a cikin siga (N), alamun horo (D), da jimlar lissafi (C).
Me yasa dokokin sikeli suke da kima ga labs na AI?
Ta hanyar daidaita masu lankwasa a ƙaramin sikeli, ƙungiyoyi suna hasashen aikin ƙaƙƙarfan ƙira kafin kashe makudan kuɗi.
Menene madaidaicin lokaci (marasa raguwa) a cikin dabarar asarar doka mai ƙima ke wakilta?
Asara tana da yanki mai raguwa wanda ke raguwa tare da ma'auni tare da bene mara lahani da aka saita ta ainihin bazuwar bayanan.
Me yasa aka kwatanta nasarorin da aka samu da 'raguwa'?
Saboda ƙaƙƙarfan ƙaƙƙarfan ikon ƙarami ne, ƙididdige ƙididdigewa daidai gwargwado yana ƙara yawan samun ci gaba a hankali ƙarami cikakkiyar ragi.