Mixtral da Sparse Model
Mixtral shine samfurin ƙwararrun ƙwararrun buɗewar Mistral AI wanda ke ba da ingantacciyar ƙira a ƙaramin ƙirar ƙira.
Dubawa
Sparse models like it activate only a fraction of their parameters per token, cutting compute without sacrificing capability.
Zurfafa nutsewa
Mixtral 8x7B, wanda Mistral AI ya sake shi a ƙarshen 2023, ya ba da fifiko ga ƙwararrun masana (MoE) a cikin buɗaɗɗen samfura. Yana ƙunshe da cibiyoyin sadarwa daban-daban na 'gwani' na gaba a kowane Layer, tare da jimlar jimlar biliyan 47, amma na'ura mai ba da hanya tsakanin hanyoyin sadarwa mara nauyi yana zaɓar ƙwararru biyu kawai ga kowane alamar. Sakamakon haka, kusan sigogi biliyan 13 ne kawai ke aiki a kowace alama, don haka ƙaddamarwa tana tafiya da sauri kamar ƙirar ƙira mai girma 13B yayin da ta kai inganci mai kama da waɗanda suka fi girma. Mixtral yayi daidai ko doke GPT-3.5 da Llama 2 70B akan ma'auni da yawa yayin da yake sauri da arha don hidima. Daga baya Mistral ya fito da Mixtral 8x22B. Samfurin yana da lasisi a bayyane a ƙarƙashin Apache 2.0, yana haɓaka karɓuwa cikin sauri da kuma daidaitawa a cikin al'ummar buɗe ido.
Fahimtar Fasaha
A cikin tarkace MoE Layer, babban katange-gaba da ciyarwa ana maye gurbinsu da cibiyoyin ƙwararrun N da ƙaramar hanyar sadarwar gating (na'ura mai ba da hanya tsakanin hanyoyin sadarwa). Ga kowane alamar, na'ura mai ba da hanya tsakanin hanyoyin sadarwa yana lissafta maki kuma ya zaɓi manyan-k ƙwararrun (top-2 a Mixtral), yana sarrafa alamar kawai ta waɗancan. Abubuwan da aka fitar suna da nauyi kuma an tara su. Saboda yawancin ƙwararru suna zama marasa aiki a kowace alama, ƙirar tana riƙe da sigogi da yawa a ƙwaƙwalwar ajiya amma ba ta yin ƙididdigewa sosai. Cinikin ciniki: duk ƙwararrun dole ne a ɗora su cikin VRAM duk da cewa wasu kawai ke gudana.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Samfuran Mixtral da Sparse
Sparse MoE yanzu shine tsakiya zuwa iyakar AI. Yi tsammanin ƙarin buɗaɗɗen sakin MoE, ingantacciyar hanya tare da ƙwararrun ƙwararrun masana, da ƙwararrun ƙwararrun ƙwararrun ƙira waɗanda ke haɓaka ingantaccen aiki. Kamar yadda ƙila ke yin girma zuwa ga tiriliyan na jimlar sigogi, ɓarna shine babban lever don kiyaye ƙima mai araha. Bincike yana magance raunin raunin MoE, daidaita nauyi a cikin ƙwararru, ƙwaƙwalwar ajiya, da kwanciyar hankali na horo, yayin da kayan aiki da kayan aiki da kayan aiki suna ƙara haɓaka musamman don ƙwararrun ƙwararru.
Aiwatar da Gaskiyar Duniya
Hidima mai inganci na chatbot akan farashi da saurin ƙaramin ƙima mai yawa
Mai ɗaukar nauyin samfurin Apache-2.0 mai lasisi don samfuran kasuwanci ba tare da kuɗin amfani ba
Ingantaccen ɗabi'un mutum ɗaya akan Mixtral don ƙididdigewa, taƙaitawa, ko ayyukan harsuna da yawa
Ƙaddamar da sauri a kan sabar GPU mai yawa guda ɗaya inda samfurin 70B mai yawa zai yi jinkiri
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mixtral and Sparse Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Model da Bututu Daidaita
Tambayoyin da ake yawan yi
What is Mixtral and Sparse Models?
Mixtral shine samfurin ƙwararrun ƙwararrun buɗewar Mistral AI wanda ke ba da ingantacciyar ƙira a ƙaramin ƙirar ƙira. Samfuran da ba su da yawa kamar sa suna kunna ɗan juzu'i na sigogin su kowace alama, yanke lissafin ba tare da sadaukar da iyawa ba.
A cikin Mixtral 8x7B, masana nawa ne ke aiwatar da alamar kowane mutum?
Mixtral yana amfani da hanyar sama-2: na'ura mai ba da hanya tsakanin hanyoyin sadarwa ta zaɓi biyu daga cikin ƙwararrun ƙwararrun takwas don aiwatar da kowace alama.
Wane bangare ne ke yanke shawarar wane ƙwararrun aka aika da alama?
Wata karamar hanyar sadarwa ta gating, da ake kira router, tana tantance kwararrun kuma ta zabi wadanda suke rike da kowace alama.
Kodayake Mixtral 8x7B yana da kusan jumlolin 47B, kusan nawa ne ke aiki kowace alama?
Domin ƙwararrun ƙwararru biyu ne kawai ke gudanar da kowace alama, kusan sigogin biliyan 13 suna aiki, suna ba shi saurin ƙaramin ƙima.
Menene maɓalli mai mahimmanci na samfuran MoE marasa ƙarfi kamar Mixtral?
MoE yana adana lissafin kowace alama amma har yanzu yana buƙatar duk ƙwararrun a riƙe su a cikin VRAM, haɓaka buƙatun ƙwaƙwalwar ajiya.
A karkashin wane lasisi Mistral AI ya saki Mixtral, yana haifar da tallafi na bude ido?
An saki Mixtral a ƙarƙashin lasisin Apache 2.0 mai izini, yana ba da damar amfani da kasuwanci kyauta da daidaitawa.