Qaababka isku dhafan iyo kuwa aan badnayn
Mixtral waa Mistral AI moodelka khubarada isku dhafka ah ee furan kaas oo bixiya tayada moodeelka xawliga yar-yar.
Dulmar
Sparse models like it activate only a fraction of their parameters per token, cutting compute without sacrificing capability.
quusid qoto dheer
Mixtral 8x7B, oo ay sii daysay Mistral AI dabayaaqadii 2023, ayaa faafisay habka khubarada isku dhafka ah (MoE) ee moodooyinka furan. Waxay ka kooban tahay sideed shabakadood oo kala duwan oo 'khabiir' ah oo quudin ah lakabkiiba, oo leh qiyaastii 47 bilyan oo cabbir, laakiin router-ka fudud ayaa calaamad kasta u doorta laba khabiir. Natiijo ahaan, qiyaas ahaan 13 bilyan oo xuduudo ah ayaa shaqeynaya calaamad kasta, markaa fikraddu waxay u socotaa sida ugu dhakhsaha badan ee 13B moodeel cufan iyadoo la gaarayo tayada la barbardhigi karo kuwa aad uga weyn. Mixtral waxay ku beegnayd ama ku garaacday GPT-3.5 iyo Llama 2 70B jaangooyo badan iyadoo aad u dhaqso badan ugana raqiisan tahay in loo adeego. Mistral ayaa markii dambe sii daayay Mixtral 8x22B. Qaabka si furan ayaa shati loogu haystaa Apache 2.0, isaga oo hurinaya korsashada degdega ah iyo hagaajinta bulshada isha furan.
Aragtida Farsamada
Lakabka MoE ee aadka u yar, baloogga hore ee quudinta cufan waxaa beddelaya shabakadaha khabiirada N iyo shabakad yar oo gating ah (router). Calaamad kasta, router-ku wuxuu xisaabiyaa buundooyinka wuxuuna soo xushaa khubarada sare-k (top-2 gudaha Mixtral), isaga oo calaamadeynaya kaliya kuwa. Wax soo saarkoodu waa la miisaamay oo la soo koobay. Sababtoo ah khubarada intooda badani waxay joogaan shaqo la'aan halkii calaamad, moodeelku waxa uu hayaa xaddi badan oo xusuusta ah haddana ma sameeyo xisaabin aad u yar. Ganacsiga-off: khubarada oo dhan waa in lagu shubaa VRAM in kasta oo qaarkood ay ordaan.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka moodooyinka isku dhafan iyo kuwa aan badnayn
Sparse MoE hadda waxay udub dhexaad u tahay xudduudaha AI. Filo sii dayn badan oo MoE ah, la wareegid si fiican loo maro khubaro yar yar oo badan, iyo naqshado khabiiro la wadaago ama isku-dhafan kuwaas oo si dheeraad ah u wanaajiya waxtarka. Sida moodellada ay u cabbirayaan ilaa trillions oo cabbirro wadarta, tabar-darrada ayaa ah kabaalka ugu weyn ee lagu ilaalinayo qaddarinta la awoodi karo. Cilmi-baadhistu waxay la tacaalaysaa dhibcooyinka daciifka ah ee MoE, dheellitirka culayska ee khubarada oo dhan, xusuusta dusheeda, iyo xasilloonida tababarka, halka qalabka iyo kaydinta adeegyadu ay si isa soo taraysa u wanaajinayaan hab-socodka khabiirada.
Dhaqangelinta Adduunka-dhabta ah
U adeegida chatbot tayo sare leh oo qiimaheedu yahay iyo xawaaraha nooc cufan oo aad uga yar
Is-martigelinta qaabka Apache-2.0 shatiyaysan ee alaabada ganacsiga iyada oo aan la isticmaalin khidmad
Hagaajinta dabeecadaha shakhsi ahaaneed ee Mixtral si loo codeeyo, soo koobid, ama hawlaha luqadaha badan
Ku socodsiinta qiimaynta degdega ah ee hal server-GPU-badan ah halkaas oo qaabka cufan ee 70B uu aad u gaabis noqon doono
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixtral and Sparse Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Qaabka iyo Tubooyinka Isbarbar-dhigga
Su'aalaha soo noqnoqda
What is Mixtral and Sparse Models?
Mixtral waa Mistral AI moodelka khubarada isku dhafka ah ee furan kaas oo bixiya tayada moodeelka xawliga yar-yar. Moodooyinka aadka u yar ee la midka ah waxa ay dhaqaajiyaan qayb yar oo ka mid ah halbeegyadooda calaamad kasta, iyaga oo jaraya xisaabinta iyada oo aan la hurayn awoodda.
Mixtral 8x7B, imisa khubaro ah ayaa farsameeya calaamad kasta?
Mixtral waxay isticmaashaa habka-2-sare: router-ku wuxuu doortaa laba ka mid ah siddeedda khabiir si uu u farsameeyo calaamad kasta.
Qaybtee ayaa go'aamisa khabiirada calaamada loo diro?
Shabakad yar oo gating ah, oo loo yaqaan router, ayaa dhibcaha khubarada oo doorta kuwa gacanta ku haya calaamad kasta.
In kasta oo Mixtral 8x7B uu leeyahay qiyaastii 47B wadarta, qiyaas ahaan imisa ayaa firfircooni halkii calaamad?
Sababtoo ah laba khabiir oo kaliya ayaa ordaya halkii calaamad, qiyaastii 13 bilyan oo cabbir ayaa firfircoon, taasoo siinaysa xawaaraha nooc aad u yar.
Waa maxay ganacsiga furaha ah ee moodooyinka MoE-ga yar sida Mixtral?
MoE waxay kaydisaa xisaabinta calaamad kasta laakiin waxay weli u baahan tahay dhammaan khubarada in lagu hayo VRAM, kordhinta baahiyaha xusuusta.
Ruqsaddee ayay Mistral AI ku sii daysay Mixtral, taasoo sii shiday korsashada furan?
Mixtral waxaa lagu siidaayay shatiga la ogolyahay ee Apache 2.0, taasoo u ogolaanaysa isticmaalka ganacsi ee bilaashka ah iyo hagaajinta ganaaxa.