Uhlolojikelele
Imithetho yokukala ye-Chinchilla, evela ku-DeepMind ngo-2022, yabonisa ukuthi amamodeli amaningi ezilimi amakhulu ayengaqeqeshwanga kahle kakhulu: kubhajethi yekhompiyutha engaguquki, kufanele ukale usayizi wamamodeli kanye nedatha yokuqeqeshwa cishe ngesilinganiso esilinganayo. Ibalulekile ngoba ichaze kabusha ukuthi kusho ukuthini usayizi wemodeli 'okulungile' futhi yabumba kabusha indlela amalebhu achitha ngayo ngokubala.
I-Chinchilla Scaling Laws iyingxenye yesitaki solimi-AI esisetshenziselwa ukufunda, ukukhiqiza, ukuhlukanisa, nokuguqula umbhalo nenkulumo ngezinga.
I-Deep Dive
Ngaphambi kwe-Chinchilla, inkambiso bekuwukwakha amamodeli amakhudlwana njalo (njengepharamitha engu-175B GPT-3) kuyilapho uqeqeshwa ngamanani amancane kakhulu edatha. I-DeepMind iqeqeshe amamodeli angaphezu kuka-400 kumasayizi amaningi nesabelomali sedatha, bese ilinganisa amajika abikezela ukulahleka njengomsebenzi wamapharamitha namathokheni ngaphansi kwebhajethi yekhompuyutha egxilile (i-FLOP). Ukuthola kwabo: amapharamitha namathokheni okuqeqesha kufanele alinganise ndawonye, cishe isilinganiso esingu-1 kuya ku-1, okusho cishe amathokheni angama-20 wedatha yokuqeqeshwa ipharamitha ngayinye. Ukufakazela lokho, baqeqesha Chinchilla, imodeli 70B-parameter ku-1.4 amathokheni ayizigidi eziyizinkulungwane ezingu, okuyinto yaphumelela kakhulu 280B-parameter Gopher naphezu kokusebenzisa ikhompuyutha efanayo, ngoba waqeqeshelwa idatha kude kakhulu.
I-Technical Insight
Imithetho ivela ekufakeni umsebenzi wokulahlekelwa kwepharamethikhi L(N, D) lapho okuthi N kuyimingcele futhi D kungamathokheni, okuhlanganisa ukulahlekelwa okungenakunqandeka, usayizi wemodeli, namagama osayizi wedatha. Ukunciphisa ukulahlekelwa kuncike ekuvinjweni kwekhompuyutha (ukubala kucishe kulingane nezikhathi ezingu-N D) kunikeza umphumela wokuthi i-N no-D efanelekile kokubili kukhule njengamandla wokubala ngama-exponents afanayo, ngakho-ke isilinganiso sekhompiyutha esilungile sihlala eduze kwamathokheni angu-20 ipharamitha ngayinye.
Ukufundisa Imithetho Yokukala kweChinchilla
Imithetho yokukala ye-Chinchilla, evela ku-DeepMind ngo-2022, yabonisa ukuthi amamodeli amaningi ezilimi amakhulu ayengaqeqeshwanga kahle kakhulu: kubhajethi yekhompiyutha engaguquki, kufanele ukale usayizi wamamodeli kanye nedatha yokuqeqeshwa cishe ngesilinganiso esilinganayo. Ibalulekile ngoba ichaze kabusha ukuthi kusho ukuthini usayizi wemodeli 'okulungile' futhi yabumba kabusha indlela amalebhu achitha ngayo ngokubala. I-Chinchilla Scaling Laws iyingxenye yesitaki solimi-AI esisetshenziselwa ukufunda, ukukhiqiza, ukuhlukanisa, nokuguqula umbhalo nenkulumo ngezinga. Ukuze wakhe ukuqonda okujulile, phatha i-Chinchilla Scaling Laws njengemodeli yokusebenza, hhayi isici esisodwa: chaza imiphumela efiselekayo, ucacise ukucabanga, futhi uhlukanise lokho uhlelo olungakwenza ngokwethembeka kulokho okusadinga ukwahlulela kochwepheshe.
Empeleni, amaqembu aqinile asebenzisa i-Chinchilla Scaling Laws aklama imiyalelo, ukubuyisa, nokubuyekeza amalophu njengohlelo olulodwa lokuxhumana oludidiyelwe. Babhala imibandela yempumelelo ecacile, ukuhlola okuqhathaniswa nedatha engokoqobo nokugeleza komsebenzi, futhi baphindaphinde ngokusekelwe kumaphethini okuhluleka aqashiwe esikhundleni sokuwina kwebhentshimakhi yesikhathi esisodwa. Yilapho ukuqonda kwethiyori kuguquka kube amandla ahlala njalo kuwo wonke umkhiqizo, inqubomgomo, kanye nokusebenza.
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana. Ngesikhathi esifanayo, amaqiniso Akhohliwe angafaka imibiko buthule, ukugeleza kosekelo, noma imiphumela yocwaningo. Indlela eqine kakhulu iwukuhlanganisa isivinini sokuhlola nesiyalo sokuphatha: qhuba abashayeli bezindiza, bamba ubufakazi, ushicilele amalogi ezinqumo, futhi ubuyekeze izivikelo ngokuqhubekayo njengoba imodeli yokuziphatha, okulindelwe ngabasebenzisi, kanye nezimfuneko zokulawula zishintsha.
I-Strategic Impact
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda. Ekusetshenzisweni kwekhwalithi ephezulu, lokhu kuhunyushwa emithethweni yokusebenza elinganisekayo, imingcele yobunikazi, nemikhuba yokubuyekeza ephindelelayo ukuze amaqembu akwazi ukukala ukuzethemba esikhundleni sokukala ukungaqondakali.
Ukuqaliswa Komhlaba Wangempela
I-DeepMind's 70B-parameter Chinchilla ihlula i-280B Gopher kumabhentshimakhi isebenzisa ikhompuyutha elinganayo, ngokuqeqeshwa ngedatha eyengeziwe.
Amaqembu aqondisayo ukuthi enze ibhajethi cishe amathokheni okuqeqesha angama-20 ngepharamitha ngayinye lapho ehlela imodeli esuka ekuqaleni
Ukuqinisekisa amamodeli amancane, anothile ngedatha njenge-LLaMA ashibhile ukusebenzisa ngesikhathi sokunquma
Ukulinganisa ukuthi imodeli ehleliwe 'ayiqeqeshelwanga ngokwanele' futhi ingazuza kakhulu kudatha eyengeziwe kunamapharamitha engeziwe
Amaphethini Okusebenzisa
Imithetho ye-Chinchilla Scaling in practice
I-DeepMind's 70B-parameter Chinchilla ihlula i-280B Gopher kumabhentshimakhi isebenzisa ikhompuyutha elinganayo, ngokuqeqeshwa ngedatha eyengeziwe kakhulu.
I-DeepMind's 70B-parameter Chinchilla ishaya i-280B Gopher kuma-benchmarks isebenzisa ikhompuyutha elinganayo, ngokuqeqeshwa ngedatha eyengeziwe Amaqembu ngokuvamile athola imiphumela engcono uma echaza izinga eliphezulu ngaphambili, agcine indlela yokukhuphuka komuntu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Imithetho ye-Chinchilla Scaling in practice
Amaqembu aqondisayo ukuthi enze ibhajethi cishe amathokheni okuqeqesha angama-20 ngepharamitha ngayinye lapho ehlela imodeli esuka ekuqaleni.
Amaqembu aqondisayo ukuthi enze isabelomali esingaba amathokheni okuqeqesha angu-20 ipharamitha ngayinye lapho ehlela imodeli esuka ekuqaleni Amathimba ngokuvamile athola imiphumela engcono lapho echaza izinga eliphezulu ngaphambili, egcina indlela yokukhuphuka kwabantu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Imithetho ye-Chinchilla Scaling in practice
Ukuqinisekisa amamodeli amancane, anothile ngedatha njenge-LLaMA ashibhile ukusebenzisa ngesikhathi sokunquma.
Ukuqinisekisa amamodeli amancane, anothile ngedatha njenge-LLaMA ashibhile ukusebenzisa ngesikhathi sokubikezela Amathimba ngokuvamile athola imiphumela engcono uma echaza izinga eliphezulu ngaphambili, egcina indlela yokukhuphuka yabantu yamakesi asemaphethelweni, futhi alandelele kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Imithetho ye-Chinchilla Scaling in practice
Ukulinganisa ukuthi imodeli ehleliwe 'ayiqeqeshelwanga ngokwanele' futhi ingazuza kakhulu kudatha eyengeziwe kunamapharamitha engeziwe.
Ukulinganisa ukuthi imodeli ehleliwe 'ayiqeqeshelwe kahle yini' futhi ingazuza kakhulu kudatha eyengeziwe kunamapharamitha engeziwe Amaqembu ngokuvamile athola imiphumela engcono uma echaza izinga eliphezulu ngaphambili, egcina indlela yokukhuphuka komuntu yamakesi asemaphethelweni, futhi elandelela kokubili izinzuzo zokukhiqiza nezindleko zamaphutha ngokuhamba kwesikhathi.
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo. Phatha isinyathelo ngasinye njengesango lobufakazi: uma imibandela ingafinyelelwa, misa ukukhishwa, vala igebe, bese unweba ukusetshenziswa.