EleuterAI
EleutherAI kuréel la guy def ay gëstu yu amul benn njariñ, moo njëkka amal xeetu làkk yu yaatu yu ubbeeku yi, bi ñu tëjoo frontier IA ci ginnàaw miiru liggéeyukaay yi.
Résumé
It proved that a volunteer community could build and freely release models rivaling closed systems, reshaping who gets to participate in AI research.
Plongeur bu xóot
EleutherAI mingi tàmbali ci weeru sulet 2020 ni askanu Discord bu Connor Leahy, Sid Black, ak Leo Gao amalee, seen mébet mooy nuru GPT-3 bu AIU. Ngir tàggat model yu mel noonu, ñu njëkka tabax ba noppi génne The Pile, muy 825GB dataset buñ tànn mu nekk corpus tàggat buñ ubbi. Ginaaw loolu ñu génne GPT-Neo, GPT-J-6B, ak GPT-NeoX-20B bu am 20 milyaar ciy parametre, muy benn ci xeeti làkk yi gëna mag te ñépp mëna ko jëfandikoo ci seen jamono. Seeni jumtukaay, lu ci melni GPT-NeoX bibliothèque de formation ak LM Evaluation Harness bi ñuy jëfandikoo ci liggéeyukaay bi yépp ngir def ay référence, nekk nañu jumtukaay yu ñeneen ñi tabax ci kaw. Ci atum 2023 EleutherAI dafa formalise nekk daara gëstu buy def te Yàlla tax, muy yaatal ci tekki, méngale, ak science ci ni model yi di jàngee.
Gis-gis xarala
Royuwaayi EleutherAI dañuy jëfandikoo dekodeer transformatër, waaye GPT-J ak GPT-NeoX dugal nañu tànneefi ingenieur yu am solo yu melni Rotary Positional Embeddings (RoPE) ngir enkode position token yi ak feedforward yuñ boole ngir gaawal tàggat yaram. Ci anam wu gëna am solo, dañu tàggat ci TPUs ak GPUs yuñ joxe ci lëkkaloo yu melni Google's TPU Research Cloud ak CoreWeave, di wane ni ordinatër buñ séddale, bi ñuy fay parrain mën na wecci santaru done bu liggéeyukaay bi suñu ko boole ak kode bu ubbeeku.
njeextalu pexe
Pexem jaaykat
Kartu yoonu jaaykat yi ñooy wane man-man yi sa ekip mëna tabax ci kanam.
Njëgg ak budget
Anamu jënd ak jaay ak tànneefi dugal dañu am njeexital ci njëg ak risk ci diir bu xawa yàgg.
Risk ak kaaraange
Li liggéeyukaay bi di ñaax mooy tëral ni produit bi di doxee, kaaraange gi ak ubbeeku gi.
Ëlëgu EleutherAI
EleutherAI mingi joge ci kursu model-scaling bu seer dem ci tekki, tàggat-done leer, ak jàngat bu dëgër, wàll yi science ubbeeku gëna soxla. Xaarandi liggéey bu wéy ngir xam li model yi di representé ci biir, génne ay done yuñ bind bu baax, ak jàppale gëstub kaaraange bu moom boppam. Bi laboratuwaaru frontiere yi di gëna am sekkere, cër bi EleutherAI di def ci wàllu njariñu mbooloo, di tàggat gëstukat yu bees yi, mën na nekk lu ëppu limu parametre yi ci benn model bumuy yónnee.
Doxal ci àdduna dëgg
Gëstukat yi ci àdduna bi yépp dañuy jëfandikoo Pile dataset ngir tàggat ak jàngat xeeti làkk yu ubbeeku ci anam wu ñu mëna joxewaat.
GPT-J-6B ak GPT-NeoX-20B, ñiy sooga tàmbali liggéey ak jàngat yi ñoo leen di jëfandikoo ngir wuutu xeetu API yi ñuy jaay.
LM Evaluation Harness mooy jumtukaay bi lab yu bari di jëfandikoo ngir méngale model bi ci téemeeri liggéey.
Gëstukat yi moom seen bopp ci wàllu kaaraange ak mëna tekki dañuy jëfandikoo poid yu ubbeeku yu EleutherAI ngir jàngat xeetu biir yi API yu tëju yi di nëbb.
Risk yi ak balustrade yi
Koom-koomu ubbite mën na raw stabilite ci def liggéeyu defar dëgg.
Njëg yi ci API wala coppite ci sàrt yi mën nañu dindi xalaat yi ci guddi gi.
Dependence ci benn jaaykat dafay yokk njëgu tëjug ak migraasioŋ.
Roadmap ngir samp gi
Saytu sa fournisseur yi nga jëfandikoo sa liggéey ak say done.
Xoolaat mbir yu nëbbu, kaaraange ak sàrti yoon balaa ngay boole.
Fexe am palaŋu fallback ci model yi wala jaaykat yi.
Xool notu génne yi suko defee coppite yi ci kàrtu yoon du jaaxal ekip yi.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the EleutherAI quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Poids & Bias
Laaj yi ñuy faral di laaj
What is EleutherAI?
EleutherAI kuréel la guy def ay gëstu yu amul benn njariñ, moo njëkka amal xeetu làkk yu yaatu yu ubbeeku yi, bi ñu tëjoo frontier IA ci ginnàaw miiru liggéeyukaay yi. Firndeelna ni askanu deff te Yalla tax munna tabax ak genne ci lu yomb ay model yuy joŋante ak sistem yu tëju yi, di soppali ki wara bokk ci gestu juntuwaay yu bees yi.
Naka la EleutherAI tàmbalee ci 2020?
EleutherAI mingi tàmbali nekk kuréelu Discord bu amul benn formalite, ñu defaree ko ngir nuru GPT-3 biñ tëjoon ci jamono jooju.
Luy 'Pile' bi?
Pile bi dafa yaatu, bari ay mbir, ñu génne ko ci EleutherAI, ñu tabax ko ngir tàggat ak jàngat xeeti làkk yi.
Ci yii, yan ci ñoom mooy xeetu làkku EleutherAI?
GPT-NeoX-20B benn la ci xeetu ubbeeku yi gëna mag ci EleutherAI; ñeneen ñi ñu ngi bawoo ci Google wala Meta.
Lan lañuy jëfandikoo ngir jàngat LM?
Harnais bi jumtukaay la bu ñu bari jëfandikoo ngir jàngat modeli làkk yi ci téemeeri benchmark.
naka la EleutherAI amee ordinatër ngir tàggat ay model yu mag ci teel?
Ndax EleutherAI amul benn datacentre, mu ngi wéeru ci TPU ak GPU yu parrain yi joxe ngir tàggat ay modelam.