Overview
EleutherAI iboka rekutsvaga risingabatsiri rakapayona rakavhurika-source hombe mhando dzemitauro apo muganho AI wakavharirwa kuseri kwemadziro emakambani. Zvakaratidza kuti nharaunda inozvipira inogona kuvaka uye kuburitsa zvakasununguka mamodheru anokwikwidzana akavharwa masisitimu, kuumbazve uyo anotora chikamu mukutsvaga kweAI.
EleutherAI inonzwisiswa zvakanyanya mumamiriro ehurongwa, kuwana modhi, sarudzo dzepuratifomu, uye ecosystem kudyidzana.
Deep Dive
EleutherAI yakatanga muna Chikunguru 2020 senharaunda yeDiscord yakarongwa naConnor Leahy, Sid Black, naLeo Gao, pakutanga vachivavarira kudzokorora OpenAI's GPT-3. Kudzidzisa mamodheru akadai, vakatanga kuvaka nekuburitsa The Pile, 825GB curated text dataset iyo yakava yakajairwa yakavhurika yekudzidzisa corpus. Vakabva vaburitsa GPT-Neo, GPT-J-6B, uye 20-bhiriyoni-parameter GPT-NeoX-20B, pakati pemienzaniso mikuru yaiwanika pachena yemitauro yenguva yavo. Zvishandiso zvavo, zvinosanganisira raibhurari yekudzidzisa yeGPT-NeoX uye LM Evaluation Harness yakashandisa indasitiri-yakakura kuita mabhenji, yakava zvivakwa zvakavakwa nevamwe. Muna 2023 EleutherAI yakagadziridzwa senge isiri purofiti yekutsvagisa, ichiwedzera mukududzira, kurongeka, uye sainzi yekuti mamodheru anodzidza sei.
Technical Insight
Mamodheru eEleutherAI anoshandisa dhizaini yekushandura, asi GPT-J neGPT-NeoX yakaunza sarudzo dzinoshanda dzeinjiniya senge Rotary Positional Embeddings (RoPE) yekuisa encoding tokeni zvinzvimbo uye parallel kutarisisa-plus-feedforward layers kukurumidza kudzidziswa. Zvakanyanya kukosha, vakadzidzira maTPU nemaGPU akapihwa kuburikidza nekudyidzana kwakafanana neGoogle's TPU Research Cloud neCoreWeave, zvichiratidza kuti compute yakagoverwa, inotsigirwa nevatsigiri inogona kutsiva datacenter yekambani kana yapetwa nekodhi yakavhurika.
Kudzidzira EleutherAI
EleutherAI iboka rekutsvaga risingabatsiri rakapayona rakavhurika-source hombe mhando dzemitauro apo muganho AI wakavharirwa kuseri kwemadziro emakambani. Zvakaratidza kuti nharaunda inozvipira inogona kuvaka uye kuburitsa zvakasununguka mamodheru anokwikwidzana akavharwa masisitimu, kuumbazve uyo anotora chikamu mukutsvaga kweAI. EleutherAI inonzwisiswa zvakanyanya mumamiriro ehurongwa, kuwana modhi, sarudzo dzepuratifomu, uye ecosystem kudyidzana. Kuvaka kunzwisisa kwakadzama, tora EleutherAI semuenzaniso wekushanda, kwete chinhu chimwe chete: tsanangura zvinodikanwa, tsanangura fungidziro, uye patsanura izvo zvinogonekwa nehurongwa hwakavimbika kubva kune zvichiri kuda kutonga kwenyanzvi.
Mukuita, zvikwata zvakasimba zvinoshandisa EleutherAI zvinoongorora zano revatengesi, kuvimbika kwemepu yemugwagwa, uye njodzi yekuvhara isati yaita. Ivo vanonyora zvakajeka maitiro ebudiriro, bvunzo vachipokana ne data rechokwadi uye mafambiro ebasa, uye iterate zvichibva pane zvakacherechedzwa maitiro ekutadza kwete kuhwina-nguva imwe chete yebhenji. Apa ndipo apo kunzwisisa kwe theoretical kunoshanduka kuve kugona kwakasimba pane chigadzirwa, mutemo, uye mashandiro.
Mamepu emigwagwa emutengesi anopesvedzera izvo izvo timu yako inogona kugadzira inotevera. Panguva imwecheteyo, zviziviso zveLaunch zvinogona kupfuura kugadzikana mune chaiyo yekugadzira workflows. Nzira yakatsiga ndeyekubatanidza kukurumidza kuyedza nekutonga: mhanyisa vatyairi vendege, tora humbowo, buritsa matanda esarudzo, uye urambe uchivandudza chengetedzo semaitiro emuenzaniso, zvinotarisirwa nemushandisi, uye zvinodikanwa zvekutonga.
Strategic Impact
Mamepu emigwagwa emutengesi anopesvedzera izvo izvo timu yako inogona kugadzira inotevera.
Mamepu emigwagwa emutengesi anopesvedzera izvo izvo timu yako inogona kugadzira inotevera. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Mamiriro ezvekutengeserana uye sarudzo dzekuendesa dzinokanganisa mutengo wenguva refu uye njodzi.
Mamiriro ezvekutengeserana uye sarudzo dzekuendesa dzinokanganisa mutengo wenguva refu uye njodzi. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Kambani inokurudzira inogadzirisa kusarudzika kwechigadzirwa, mamiriro ekuchengetedza, uye kuvhurika.
Kambani inokurudzira inogadzirisa kusarudzika kwechigadzirwa, mamiriro ekuchengetedza, uye kuvhurika. Mukutumirwa kwemhando yepamusoro, izvi zvinoshandurirwa kuita mitemo inoyerwa yekushanda, miganhu yevaridzi, uye tsika dzekudzokorora dzinodzokororwa kuitira kuti zvikwata zvikwire kuvimba pane kukwidza kusajeka.
Real-World Implementation
Iyo Pile dataset inoshandiswa nevatsvaguri pasi rese kudzidzisa nekudzidza mamodheru emitauro yakavhurika.
GPT-J-6B uye GPT-NeoX-20B inoiswa nevanotanga uye vadzidzi sedzimwe nzira dzemahara kune dzekutengesa API modhi.
Iyo LM Evaluation Harness ndiyo yakajairwa chishandiso chinoshandiswa nemalabhu akawanda kuenzanisa kuita kwemodhi mumazana emabasa.
Kuzvimiririra kuchengetedza uye kududzira vaongorori vanoshandisa EleutherAI's yakavhurika uremu kudzidza modhi yemukati iyo yakavhara APIs kuvanda.
Maitiro Ekuita
EleutherAI mukuita
Iyo Pile dataset inoshandiswa nevatsvaguri pasi rese kudzidzisa nekudzidza mamodheru emitauro yakavhurika.
Iyo Pile dataset inoshandiswa nevatsvakurudzi pasi rese kudzidzisa nekudzidza mamodheru emitauro yakavhurika nenzira inobuditsa Matimu anowanzo kuwana mibairo iri nani kana achinge atsanangura hunhu hwepamberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
EleutherAI mukuita
GPT-J-6B uye GPT-NeoX-20B inoiswa nevanotanga uye vadzidzi sedzimwe nzira dzemahara kune dzekutengesa API modhi.
GPT-J-6B uye GPT-NeoX-20B inoiswa nevekutanga uye vadzidzi sedzimwe nzira dzemahara kune dzekutengesa API modhi Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando yepamusoro kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
EleutherAI mukuita
Iyo LM Evaluation Harness ndiyo yakajairwa chishandiso chinoshandiswa nemalabhu akawanda kuenzanisa kuita kwemodhi mumazana emabasa.
Iyo LM Evaluation Harness ndiyo yakajairika chishandiso chinoshandiswa nemalabhu kuenzanisa kuita kwemuenzaniso mumazana emabasa Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura hunhu hwepamberi, chengetedza nzira yekukwira kwevanhu yemakesi emupendero, uye kuteedzera zvese zvakawanikwa zvechigadzirwa nemitengo yekukanganisa nekufamba kwenguva.
EleutherAI mukuita
Kuzvimiririra kuchengetedza uye kududzira vaongorori vanoshandisa EleutherAI's yakavhurika uremu kudzidza modhi yemukati iyo yakavhara APIs kuvanda.
Kuzvimiririra kuchengetedza uye kududzira vaongorori vanoshandisa EleutherAI's yakavhurika uremu kudzidza modhi yemukati iyo yakavharwa APIs kuvanza Matimu anowanzo kuwana mhedzisiro iri nani kana achinge atsanangura emhando zvikumbaridzo kumberi, chengetedza nzira yekukwira kwevanhu yemakesi ekumucheto, uye kuteedzera zvese zvakawanikwa zvechigadzirwa uye mutengo wekukanganisa nekufamba kwenguva.
Njodzi & Guardrails
Zviziviso zvekutanga zvinogona kupfuura kugadzikana mune chaiyo yekugadzira workflows.
Mitengo yeAPI kana shanduko yepolicy inogona kukanganisa fungidziro husiku.
Kutsamira kune mumwe-mutengesi kunowedzera kukiya-mukati uye mari yekufambisa.
Implementation Roadmap
Ongorora vanopa uchishandisa ako ega mabasa uye dataset.
Ongorora vanopa uchishandisa ako ega mabasa uye dataset. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Wongorora zvakavanzika, chengetedzo, uye mazwi emutemo usati wabatanidzwa.
Wongorora zvakavanzika, chengetedzo, uye mazwi emutemo usati wabatanidzwa. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Chengetedza chirongwa chekudzokera kumashure kune mamodheru kana vatengesi.
Chengetedza chirongwa chekudzokera kumashure kune mamodheru kana vatengesi. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.
Tarisa zvinyorwa zvekuburitsa kuitira kuti shanduko yemigwagwa isashamise zvikwata.
Tarisa zvinyorwa zvekuburitsa kuitira kuti shanduko yemigwagwa isashamise zvikwata. Bata nhanho yega yega segedhi rehumbowo: kana maitiro asina kusangana, imbomira kuburitsa, vhara gaka, uye wobva wawedzera kushandiswa.