Amamodeli we-Mixtral nama-Sparse
I-Mistral AI iyimodeli evulekile yenhlanganisela yochwepheshe ye-Mistral eletha ikhwalithi yamamodeli amakhulu ngesivinini samamodeli amancane.
Uhlolojikelele
Sparse models like it activate only a fraction of their parameters per token, cutting compute without sacrificing capability.
I-Deep Dive
I-Mistral 8x7B, ekhishwe yi-Mistral AI ngasekupheleni kuka-2023, yenze yaduma indlela yochwepheshe abayingxubevange (i-MoE) kumamodeli avulekile. Iqukethe amanethiwekhi okuphakelayo 'ochwepheshe' ahlukene ayisishiyagalombili isendlalelo ngasinye, anamapharamitha aphelele angaba yizigidi eziyizinkulungwane ezingama-47, kodwa irutha engasindi ikhetha ochwepheshe ababili kuphela ngethokheni ngayinye. Ngenxa yalokhu, cishe amapharamitha ayizigidi eziyizinkulungwane eziyi-13 asebenzayo ithokheni ngayinye, ngakho ukuqagela kusebenza ngokushesha njengemodeli eminyene engu-13B kuyilapho kufinyelelwa ikhwalithi eqhathaniswa namakhudlwana. I-Mixtral ifaniswe noma ishaye i-GPT-3.5 ne-Llama 2 70B kumabhentshimakhi amaningi kuyilapho ishesha futhi ishibhile ukuyinikeza. UMistral kamuva wakhipha iMixtral 8x22B. Imodeli inikezwe ilayisense ngokusobala ngaphansi kwe-Apache 2.0, ebhebhezela ukutholwa ngokushesha kanye nokulungisa kahle emphakathini wemithombo evulekile.
I-Technical Insight
Kusendlalelo esincane se-MoE, ibhulokhi yokudlulisela phambili eminyene ithathelwa indawo amanethiwekhi ochwepheshe we-N kanye nenethiwekhi encane yokugenca (umzila). Kuthokheni ngayinye, irutha ibala amaphuzu bese ikhetha ochwepheshe be-top-k (top-2 ku-Mixtral), ihambisa ithokheni kulabo kuphela. Imiphumela yabo iyakalwa futhi ifinyezwe. Ngenxa yokuthi ochwepheshe abaningi bahlala bengenzi lutho ngethokheni ngayinye, imodeli ibamba amapharamitha amaningi enkumbulweni kodwa yenza ukubala okuncane kakhulu. Ukuhwebelana: bonke ochwepheshe kumele balayishwe ku-VRAM yize abanye besebenza.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Ikusasa lamamodeli we-Mixtral kanye ne-Sparse
I-Sparse MoE manje isimaphakathi nomngcele we-AI. Lindela ukukhishwa okuvulekile okwengeziwe kwe-MoE, umzila ocolisekile nochwepheshe abaningi abancane, nemiklamo eyabiwe noma eyingxube yochwepheshe ethuthukisa ukusebenza kahle ngokuqhubekayo. Njengoba amamodeli efinyelela kuzigidigidi zamapharamitha esewonke, i-sparsity iyisivikelo esiyinhloko sokugcina umbono ufinyeleleka. Ucwaningo lubhekana nezindawo ezibuthakathaka ze-MoE, ukulinganisa komthwalo kubo bonke ochwepheshe, inkumbulo engaphezulu, nokuzinza kokuqeqeshwa, kuyilapho izingxenyekazi zekhompiyutha nezitaki eziphakelayo ziya zithuthukiswa ngokukhethekile ukuze kusetshenziswe umzila ongochwepheshe.
Ukuqaliswa Komhlaba Wangempela
Inikeza i-chatbot yekhwalithi ephezulu ngentengo nesivinini semodeli emincane kakhulu eminyene
Ukuzisingatha ngokwakho imodeli enelayisense ye-Apache-2.0 yemikhiqizo yokuthengisa ngaphandle kwezimali zokusebenzisa
Ukushuna kahle ukuziphatha komuntu ngamunye ku-Mixtral ukuze kufakwe amakhodi, ukufingqa, noma imisebenzi yezilimi eziningi
Isebenzisa inkomba esheshayo kuseva eyodwa ye-multi-GPU lapho imodeli eminyene engu-70B izohamba kancane kakhulu
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixtral and Sparse Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukufana kweModel kanye nePipeline
Imibuzo evame ukubuzwa
What is Mixtral and Sparse Models?
I-Mistral AI iyimodeli evulekile yenhlanganisela yochwepheshe ye-Mistral eletha ikhwalithi yamamodeli amakhulu ngesivinini samamodeli amancane. Amamodeli amancane afana nayo enza kusebenze ingxenye encane kuphela yamapharamitha awo ngethokheni, anqamule ikhompuyutha ngaphandle kokudela amandla.
Ku-Mixtral 8x7B, bangaki ochwepheshe abacubungula ithokheni ngayinye?
I-Mixtral isebenzisa umzila ophezulu-2: irutha ikhetha ochwepheshe ababili kwabayisishiyagalombili ukuze bacubungule ithokheni ngayinye.
Iyiphi ingxenye enquma ukuthi yibaphi ochwepheshe ithokheni ethunyelwa kubo?
Inethiwekhi encane yokugenca, ebizwa ngokuthi i-router, ihlola ochwepheshe bese ikhetha ukuthi yiziphi eziphatha ithokheni ngayinye.
Nakuba i-Mixtral 8x7B inamapharamitha angu-47B esewonke, cishe mangaki asebenzayo ithokheni ngayinye?
Ngenxa yokuthi ochwepheshe ababili kuphela abagijima ithokheni ngayinye, cishe amapharamitha ayizigidi eziyizinkulungwane ezingu-13 ayasebenza, okwenza kube nesivinini semodeli encane kakhulu.
Ikuphi ukuhwebelana okubalulekile kwamamodeli amancane we-MoE njenge-Mixtral?
I-MoE yonga ikhompuyutha ngethokheni ngayinye kodwa isadinga ukuthi bonke ochwepheshe babanjwe ku-VRAM, okwandisa izidingo zenkumbulo.
I-Mistral AI yakhipha ngaphansi kwayiphi ilayisensi i-Mistral, ebhebhezela ukutholwa okuvulekile?
I-Mixtral ikhishwe ngaphansi kwelayisense evumelekile ye-Apache 2.0, evumela ukusetshenziswa kwamahhala kwezentengiso nokulungisa kahle.