EleutherAI
EleutherAI iboka rekutsvaga risingabatsiri rakapayona rakavhurika-source hombe mhando dzemitauro apo muganho AI wakavharirwa kuseri kwemadziro emakambani.
Pfupiso
It proved that a volunteer community could build and freely release models rivaling closed systems, reshaping who gets to participate in AI research.
Kudzika Kwakadzika
EleutherAI yakatanga muna Chikunguru 2020 senharaunda yeDiscord yakarongwa naConnor Leahy, Sid Black, naLeo Gao, pakutanga vachivavarira kudzokorora OpenAI's GPT-3. Kudzidzisa mamodheru akadai, vakatanga kuvaka nekuburitsa The Pile, 825GB curated text dataset iyo yakava yakajairwa yakavhurika yekudzidzisa corpus. Vakabva vaburitsa GPT-Neo, GPT-J-6B, uye 20-bhiriyoni-parameter GPT-NeoX-20B, pakati pemienzaniso mikuru yaiwanika pachena yemitauro yenguva yavo. Zvishandiso zvavo, zvinosanganisira raibhurari yekudzidzisa yeGPT-NeoX uye LM Evaluation Harness yakashandisa indasitiri-yakakura kuita mabhenji, yakava zvivakwa zvakavakwa nevamwe. Muna 2023 EleutherAI yakagadziridzwa senge isiri purofiti yekutsvagisa, ichiwedzera mukududzira, kurongeka, uye sainzi yekuti mamodheru anodzidza sei.
Technical Insight
Mamodheru eEleutherAI anoshandisa dhizaini yekushandura, asi GPT-J neGPT-NeoX yakaunza sarudzo dzinoshanda dzeinjiniya senge Rotary Positional Embeddings (RoPE) yekuisa encoding tokeni zvinzvimbo uye parallel kutarisisa-plus-feedforward layers kukurumidza kudzidziswa. Zvakanyanya kukosha, vakadzidzira maTPU nemaGPU akapihwa kuburikidza nekudyidzana kwakafanana neGoogle's TPU Research Cloud neCoreWeave, zvichiratidza kuti compute yakagoverwa, inotsigirwa nevatsigiri inogona kutsiva datacenter yekambani kana yapetwa nekodhi yakavhurika.
Strategic Impact
Vendor strategy
Mamepu emigwagwa emutengesi anopesvedzera izvo izvo timu yako inogona kugadzira inotevera.
Mutengo uye bhajeti
Mamiriro ezvekutengeserana uye sarudzo dzekuendesa dzinokanganisa mutengo wenguva refu uye njodzi.
Ngozi uye kuchengeteka
Kambani inokurudzira inogadzirisa kusarudzika kwechigadzirwa, mamiriro ekuchengetedza, uye kuvhurika.
Ramangwana reEleutherAI
EleutherAI iri kusuduruka kubva kumujaho wakachena wekuenzanisa uchienda pakududzira, kudzidziswa-data pachena, uye kuongororwa kwakasimba, nzvimbo dzinonyanya kudiwa sainzi yakavhurika. Tarisira kuenderera mberi basa rekunzwisisa kuti ndeapi mamodheru anomiririra mukati, kuburitsa akanyorwa zvakanaka dataset, uye kutsigira yakazvimirira kuchengetedza kuchengetedza. Sezvo marebhu emuganho achikura zvakanyanya kuvanzika, basa reEleutherAI seyeruzhinji-inofarira counterweight, kudzidzisa chizvarwa chinotevera chevatsvaguri, zvingangove zvine basa kupfuura kuverengerwa kweparamendi yemhando imwe chete yainotakura.
Real-World Implementation
Iyo Pile dataset inoshandiswa nevatsvaguri pasi rese kudzidzisa nekudzidza mamodheru emitauro yakavhurika.
GPT-J-6B uye GPT-NeoX-20B inoiswa nevanotanga uye vadzidzi sedzimwe nzira dzemahara kune dzekutengesa API modhi.
Iyo LM Evaluation Harness ndiyo yakajairwa chishandiso chinoshandiswa nemalabhu akawanda kuenzanisa kuita kwemodhi mumazana emabasa.
Kuzvimiririra kuchengetedza uye kududzira vaongorori vanoshandisa EleutherAI's yakavhurika uremu kudzidza modhi yemukati iyo yakavhara APIs kuvanda.
Njodzi & Guardrails
Zviziviso zvekutanga zvinogona kupfuura kugadzikana mune chaiyo yekugadzira workflows.
Mitengo yeAPI kana shanduko yepolicy inogona kukanganisa fungidziro husiku.
Kutsamira kune mumwe-mutengesi kunowedzera kukiya-mukati uye mari yekufambisa.
Implementation Roadmap
Ongorora vanopa uchishandisa ako ega mabasa uye dataset.
Wongorora zvakavanzika, chengetedzo, uye mazwi emutemo usati wabatanidzwa.
Chengetedza chirongwa chekudzokera kumashure kune mamodheru kana vatengesi.
Tarisa zvinyorwa zvekuburitsa kuitira kuti shanduko yemigwagwa isashamise zvikwata.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the EleutherAI quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Weights & Biases
Mibvunzo inowanzo bvunzwa
What is EleutherAI?
EleutherAI iboka rekutsvaga risingabatsiri rakapayona rakavhurika-source hombe mhando dzemitauro apo muganho AI wakavharirwa kuseri kwemadziro emakambani. Zvakaratidza kuti nharaunda inozvipira inogona kuvaka uye kuburitsa zvakasununguka mamodheru anokwikwidzana akavharwa masisitimu, kuumbazve uyo anotora chikamu mukutsvaga kweAI.
EleutherAI yakatanga sei muna 2020?
EleutherAI yakatanga semubatanidzwa weDiscord isina kurongwa wevazvipiri vakaronga kudzokorora iyo yakavharwa GPT-3.
Chii chinonzi 'The Pile'?
Iyo Murwi ihombe, yakasiyana-siyana, yakaburitswa pachena zvinyorwa corpus EleutherAI yakavakirwa kudzidzisa uye kudzidza mhando dzemitauro.
Ndeupi weiyi muenzaniso wemutauro weEleutherAI?
GPT-NeoX-20B ndeimwe yeEleutherAI's flagship yakavhurika modhi; vamwe vanobva kuGoogle kana Meta.
Ko LM Evaluation Harness inoshandiswa nei?
Iyo harness chishandiso chakavhurika chakagamuchirwa chakamisikidzwa chemhando dzemitauro pamazana emabhenji.
EleutherAI akawana sei komputa yekudzidzisa mamodheru makuru kutanga?
Kushaya datacenter yemakambani, EleutherAI yaivimba nevatsigiri-vakapihwa TPU neGPU zviwanikwa kudzidzisa mhando dzayo.