Jamba Hybrid Transformer-Mamba Model
Jamba babban samfurin yare ne daga AI21 Labs wanda ke ba da ra'ayi na Transformer tare da shimfidar sararin samaniya na Mamba (da cakuda-ƙwararrun masana) don samun ingantaccen yanayi na dogon lokaci ba tare da barin ingancin Transformer ba.
Dubawa
Yana da mahimmanci saboda yana nuna gine-ginen gine-gine na iya doke tsarkakakken Transformers akan ƙwaƙwalwar ajiya da wucewa a dogon jerin tsawo.
Zurfafa nutsewa
Masu Canza-zafi masu tsafta suna biyan farashi mai ƙididdigewa cikin kulawa yayin da mahallin ke girma, da maɓalli na ma'aunin ma'auni tare da tsayin jeri. Samfuran sararin samaniya masu tsafta kamar Mamba ma'auni a layi da kuma kiyaye ƙayyadaddun yanayi mai maimaitawa, amma a tarihi suna jan hankalin wasu ayyuka. Jamba yana haɗa duka biyun: yana tattara tubalan inda yawancin yadudduka suke Mamba (mai rahusa, madaidaiciya, mai girma don jerin dogon lokaci) da ƙaramin lamba suna da daidaitattun kulawa (ƙarfi a daidai tunowa da tunani cikin mahalli). Hakanan yana ƙara yadudduka cakuda-na-ƙwararru (MoE) don haɓaka iya aiki yayin kiyaye sigogi masu aiki da sauƙi. Jamba ta farko da aka saki tare da taga mahallin alamar 256K kuma zai iya dacewa da mahallin mahallin akan GPU guda ɗaya fiye da masu canzawa masu kama da juna, godiya ga ƙaramin ma'ajin KV ɗin sa.
Fahimtar Fasaha
Mamba zaɓaɓɓen samfurin sararin samaniya ne: maimakon halartar kowane alamar da ta gabata, tana kiyaye yanayin da aka sabunta akai-akai akai-akai, tare da ƙofofin da ke dogaro da shigar da ke yanke shawarar abin da za a kiyaye ko mantawa. Jamba ta raba ƴan yadudduka masu cikakken kulawa a tsakanin yawancin yadudduka na Mamba don haka ƙirar tana riƙe da ainihin kulawa ta dogon zango yayin da yawancin ƙididdigewa da ƙwaƙwalwar ajiya ke kasancewa a layi ɗaya, kuma hanyar MoE tana kunna juzu'in ƙwararru ne kawai.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Jamba Hybrid Transformer-Mamba Model
Hankali mai haɗe-haɗe da ƙirar sararin samaniya suna fitowa a matsayin jagorar girke-girke don ingantattun samfuran yanayi mai tsayi, kuma Jamba ya taimaka yaɗa tsarin. Yi tsammanin ƙarin buɗaɗɗen ƙira da ƙira don ɗaukar rikodi masu gauraya, daidaita ma'aunin hankali-zuwa-SSM, da haɗa su da dabaru na MoE da KV-cache. Yayin da buƙatun mahallin ke girma zuwa miliyoyin alamu, fa'idar ƙwaƙwalwar-daidaitacce na shimfidar sararin samaniya yana sa matasan ƙawayen ƙayatarwa musamman ga kayan aiki da kayan aiki masu tsada.
Aiwatar da Gaskiyar Duniya
Sarrafa abubuwan shigarwar alamar 256K kamar dogayen bayanan shari'a ko manyan ma'ajiyar lambobi akan GPU guda ɗaya waɗanda ba za su iya dacewa da cache na Transformer's KV ba.
Hidimar tattaunawa mai zurfi mai zurfi inda ƙayyadaddun yanayin Mamba ke kiyaye ƙwaƙwalwar ajiya yayin da tattaunawa ke girma.
Binciken daftarin aiki da haɓaka-dawowa-ƙararru sama da manyan sansanonin ilimi cushe kai tsaye cikin mahallin
Gudun buɗaɗɗen nauyi mai tsayi LLM (An fito da Jamba tare da buɗaɗɗen ma'auni) don bincike kan ƙirar gine-gine
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Jamba Hybrid Transformer-Mamba Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Samfuran Sararin Samaniya na Jiha da Mamba
Tambayoyin da ake yawan yi
Menene Jamba Hybrid Transformer-Mamba Models?
Jamba babban samfurin yare ne daga AI21 Labs wanda ke ba da ra'ayi na Transformer tare da shimfidar sararin samaniya na Mamba (da cakuda-ƙwararrun masana) don samun ingantaccen yanayi na dogon lokaci ba tare da barin ingancin Transformer ba. Yana da mahimmanci saboda yana nuna ƙirar gine-ginen na iya doke tsaftataccen Transformers akan ƙwaƙwalwar ajiya da fitarwa a tsayin jeri.
Wadanne nau'ikan nau'ikan tushe guda biyu ne Jamba ke shiga?
Siffar ma'anar Jamba ita ce ta tara masarrafan sararin jihar Mamba tare da ƙarami na yaduddukan kulawar Transformer.
Wace ƙarin fasaha Jamba ke amfani da ita don ƙara ƙarfin aiki yayin da ke rage madaidaicin aiki?
Jamba yana ƙara yadudduka na MoE don haka ƙwararrun ƙwararrun ƙwararru ne kawai ke aiki kowace alama, suna haɓaka ƙarfin duka ba tare da ƙididdige ƙididdigewa ba.
Me yasa Mamba ma'auni ya fi kulawa don dogon jerin?
Mamba yana riƙe da matsa lamba, daidaitacce jihar da aka sabunta ta hanyar layi, yana guje wa farashin kulawa da ƙima mai ƙima mai girma.
Wane taga mahallin na farko samfurin Jamba ya goyi bayan?
An ƙaddamar da Jamba tare da taga mahallin alama mai nauyin 256K, wanda aka kunna shi ta hanyar ƙaramin sawun cache na KV.
Me yasa Jamba ke rike da hankali maimakon tafiya da tsarki Mamba?
ƴan yadudduka masu cikakken hankali suna riƙe da ainihin dubawa da iyawar yanayi inda tsantsar ƙirar sararin samaniya za ta iya ja da baya.