Impuguke zibangikanye na MoE Gukorera
Impuguke zibangikanye zigabanya Imvange-y-Impuguke ya moderi nyinshi zo kugaburira-imbere 'impuguke' muri GPU zitandukanye kuburyo buri gikoresho gifata igice cyibipimo gusa.
Incamake
It is the key to serving trillion-parameter MoE models cheaply, since only a few experts run per token.
Kwibira cyane
Uruvange-rwinzobere (MoE) urwego rusimbuza umuyoboro munini wo kugaburira imbere hamwe nindi ntoya (abahanga) wongeyeho na router itora top-k (akenshi 1 cyangwa 2) kuri buri kimenyetso. Impuguke zibangikanye (EP) ishyira abahanga batandukanye kuri GPU zitandukanye. Mugusuzuma, router ihitamo impuguke buri kimenyetso gikeneye, hanyuma intambwe-kuri-yose yo gutumanaho intambwe ihindura ibimenyetso kuri GPUs ifite abahanga bahisemo, ikayobora FFN, hanyuma igahindura ibisubizo inyuma. Ibi bireka icyitegererezo gifite ibipimo binini byuzuye (gake) mugihe ukora agace gato gusa kuri token (FLOPs). Moderi nka Mixtral 8x7B, DeepSeek-V3, na GPT-OSS ikoresha ibi. Ibice bikomeye ni imitwaro iringaniza abahanga kandi byombi bihenze byose-kuri-hops kuri buri cyiciro.
Ubushishozi
Umukanishi wibanze ni bibiri byose-kuri-byose hamwe kuri MoE layer: kohereza (ohereza ibimenyetso kubuhanga bwabo) hanyuma uhuze (gukusanya ibisubizo inyuma). Kuberako inzira igenda iterwa namakuru, umubare wibimenyetso bikubita buri mpuguke uratandukanye, bigatera ubusumbane bwumutwaro na 'stragglers.' Sisitemu yo gutanga serivisi yongerera ubushobozi ibintu, impuguke zimpuguke, hamwe nikimenyetso cyo guta cyangwa padi kugirango GEMMs (matrix kugwiza) imwe, kandi akenshi ihuza itumanaho-kuri-byose hamwe na comptabilite yo kubara kugirango ihishe ubukererwe.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza h'impuguke zibangikanye na MoE Gukorera
Witegereze gufatanya gushushanya inzira hamwe nibikoresho: byahujwe byoherejwe-kubara-guhuza intete, guhuriza hamwe GEMMs ihuza abahanga benshi, na NVLink / InfiniBand-izi-byose-kuri-byose. Tekinike nka DeepSeek ifasha-igihombo-kuringaniza iringaniza hamwe na node igarukira bigabanya urujya n'uruza. Serivise itandukanijwe izatanga 'impuguke' GPU itandukanye no kwitabwaho na GPU, kandi impuguke nini (amagana) hamwe na top-k nziza izatera MoE kugana ku buryo bukabije mugihe igiciro kuri buri kimenyetso.
Gushyira mu bikorwa Isi
Gukorera Mixtral 8x7B muri 2-4 GPUs ushyira 2-4 muri 8 zinzobere zayo kuri buri gikoresho
DeepSeek-V3 ukoresheje inzira igarukira kuri node kugirango ugabanye umubare wimpuguke za token impuguke zingana, guca inter-node byose-kuri-byose
Ukoresheje vLLM cyangwa SGLang impuguke-parallel uburyo bwo kwakira moderi ya 200B + gake kuri node imwe ya 8-GPU
Guhuza impuguke zibangikanye na tensor parallelism kurwego rwo kwitondera muburyo bwa Hybrid EP + TP
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Expert Parallelism for MoE Serving quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Gutandukanya Prefill na Decode Gukora
Ibibazo bikunze kubazwa
What is Expert Parallelism for MoE Serving?
Impuguke zibangikanye zigabanya Imvange-y-Impuguke ya moderi nyinshi zo kugaburira-imbere 'impuguke' muri GPU zitandukanye kuburyo buri gikoresho gifata igice cyibipimo gusa. Nurufunguzo rwo gutanga trillion-parameter ya moderi ya MoE ihendutse, kubera ko abahanga bake aribo bakora kuri buri kimenyetso.
Niki impuguke ibangikanye ikwirakwiza muri GPUs?
Impuguke zibangikanye zishyira impuguke zitandukanye (imiyoboro ya FFN yumurongo wa MoE) kuri GPU zitandukanye, buri gikoresho rero gifite igice cyinzobere gusa.
Ni ubuhe buryo bw'itumanaho bwibanze kuri MoE impuguke?
Buri cyiciro cya MoE gikenera byose-kuri-byose byohereza ibimenyetso kubuhanga bwabo nibindi byose-kuri-byose kugirango bahuze ibisubizo inyuma.
Ni ukubera iki MoE ikomeza kubara kuri buri kimenyetso nubwo ibipimo byinshi byuzuye?
Router ikora gusa impuguke zo hejuru-k (akenshi 1-2) kuri buri kimenyetso, FLOPs rero ikomeza kuba nto nubwo icyitegererezo gifite abahanga benshi muri rusange.
Ni ikihe kibazo kivuka kubera ko inzira zishingiye ku makuru?
Kubera ko amahitamo ya router aterwa ninjiza, abahanga bamwe bakira ibimenyetso byinshi kurenza ibindi, bigatera uburemere bwimitwaro hamwe nabagenzi.
Nubuhe buryo busanzwe bwo kugumana matrise yinzobere igwiza kimwe mubunini?
Gukora ibice bishyiraho ubushobozi bwinzobere (kubara umubare wikimenyetso) hamwe na padi cyangwa ibitonyanga birenze ibyo rero GEMM ya buri mpuguke ifite imiterere iteganijwe.