HejuruUbuyobozi bukurikira
Gutandukanya Prefill na Decode Gukora
Tekiniki
Ubuyobozi bwa tekiniki
Expert parallelism distributes a Mixture-of-Experts layer’s feed-forward experts across devices.
Umva iki gitaboAI-byavuzwe · ~2 min
It can reduce the expert weights held per device, while routing and communication costs affect whether serving is efficient.
Uruvange-rwinzobere (MoE) urwego rusimbuza umuyoboro munini wo kugaburira imbere hamwe nindi ntoya (abahanga) wongeyeho na router itora top-k (akenshi 1 cyangwa 2) kuri buri kimenyetso. Impuguke zibangikanye (EP) ishyira abahanga batandukanye kuri GPU zitandukanye. Mugusuzuma, router ihitamo impuguke buri kimenyetso gikeneye, hanyuma intambwe-kuri-yose yo gutumanaho intambwe ihindura ibimenyetso kuri GPUs ifite abahanga bahisemo, ikayobora FFN, hanyuma igahindura ibisubizo inyuma. Ibi bireka icyitegererezo gifite ibipimo binini byuzuye (gake) mugihe ukora agace gato gusa kuri token (FLOPs). Moderi nka Mixtral 8x7B, DeepSeek-V3, na GPT-OSS ikoresha ibi. Ibice bikomeye ni imitwaro iringaniza abahanga kandi byombi bihenze byose-kuri-hops kuri buri cyiciro.
Gukorera Mixtral 8x7B muri 2-4 GPUs ushyira 2-4 muri 8 zinzobere zayo kuri buri gikoresho
DeepSeek-V3 ukoresheje inzira igarukira kuri node kugirango ugabanye umubare wimpuguke za token impuguke zingana, guca inter-node byose-kuri-byose
Ukoresheje vLLM cyangwa SGLang impuguke-parallel uburyo bwo kwakira moderi ya 200B + gake kuri node imwe ya 8-GPU
Guhuza impuguke zibangikanye na tensor parallelism kurwego rwo kwitondera muburyo bwa Hybrid EP + TP
Witegereze gufatanya gushushanya inzira hamwe nibikoresho: byahujwe byoherejwe-kubara-guhuza intete, guhuriza hamwe GEMMs ihuza abahanga benshi, na NVLink / InfiniBand-izi-byose-kuri-byose. Tekinike nka DeepSeek ifasha-igihombo-kuringaniza iringaniza hamwe na node igarukira bigabanya urujya n'uruza. Serivise itandukanijwe izatanga 'impuguke' GPU itandukanye no kwitabwaho na GPU, kandi impuguke nini (amagana) hamwe na top-k nziza izatera MoE kugana ku buryo bukabije mugihe igiciro kuri buri kimenyetso.
Ibaruwa y'amakuru y'ubuntu
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Expert parallelism distributes a Mixture-of-Experts layer’s feed-forward experts across devices. It can reduce the expert weights held per device, while routing and communication costs affect whether serving is efficient.
Impuguke zibangikanye zishyira impuguke zitandukanye (imiyoboro ya FFN yumurongo wa MoE) kuri GPU zitandukanye, buri gikoresho rero gifite igice cyinzobere gusa.
Buri cyiciro cya MoE gikenera byose-kuri-byose byohereza ibimenyetso kubuhanga bwabo nibindi byose-kuri-byose kugirango bahuze ibisubizo inyuma.
Router ikora gusa impuguke zo hejuru-k (akenshi 1-2) kuri buri kimenyetso, FLOPs rero ikomeza kuba nto nubwo icyitegererezo gifite abahanga benshi muri rusange.
Kubera ko amahitamo ya router aterwa ninjiza, abahanga bamwe bakira ibimenyetso byinshi kurenza ibindi, bigatera uburemere bwimitwaro hamwe nabagenzi.
Capacity-limited designs can pad or drop work, but dropless approaches also exist. This is an implementation choice rather than a universal serving requirement.
Komeza wige
Abandi bayobozi batoranijwe kuriyi ngingo
HejuruUbuyobozi bukurikira
Gutandukanya Prefill na Decode Gukora
Tekiniki