Gutandukanya Prefill na Decode Gukora
Ubwubatsi bukora bugabanya imvugo nini yerekana urugero mubice bibiri bitandukanye - prefill na decode - ikanabikoresha kubidendezi bitandukanye bya GPU.
Incamake
It matters because these two phases have opposite hardware appetites, and forcing them onto the same machines wastes capacity and hurts latency.
Kwibira cyane
Iyo LLM isubije, ikora mubyiciro bibiri. Prefill isoma icyarimwe icyarimwe hanyuma yubake urufunguzo-agaciro (KV) cache; iyi nini nini, ibangikanye, kubara-guturika kuzuza ibice byimibare ya GPU. Decode noneho itanga ibimenyetso kimwekimwe, buri ntambwe isoma cache yose ya KV-yibuka-umurongo-wuzuye, uhuza-kubara. Kwiruka hamwe, prefill ndende ihagarika decode ya buriwese (umutwe-wumurongo uhagarika), kandi gukubita byombi bitera kwivanga. Gutandukana bishyira prefill kuri pisine imwe ya GPU hanyuma ukarangiza kurindi, kwimura cache cache hagati yabo hejuru yimikoranire yihuse nka NVLink cyangwa InfiniBand. Buri kidendezi cyarateguwe kandi gipimwa mu bwigenge, kunoza ibicuruzwa byiza, koroshya umurizo, no kureka ababikora bagakubita igihe-kuri-mbere-ikimenyetso-nigihe-cyo-gusohora-icyerekezo icyarimwe.
Ubushishozi
Ibyiciro byombi biratandukanye muburyo bwabo. Kuzuza ibintu byose byihutirwa muburyo bubangikanye, nuko FLOPs igipimo cyayo hamwe nuburebure bwihuse kandi ikarenza tensor cores. Decode ni autoregressive: buri kimenyetso gishya gikenera inzira imwe yimbere yongeye gusoma kashe yuzuye ya KV kuva HBM, bityo ibyinjira byinjizwa numuyoboro mugari, ntabwo ubara. Gutandukana kubikoresha mugupima, gutobora, ndetse no guhitamo kubangikanya kuri buri kidendezi, hanyuma kohereza cache cache kuva kubakozi bambere kugeza kubakozi.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Kazoza Kutuzuye Kuzuza no Gukora Dode
Witege ko gutandukana bizaba nkibisanzwe mubikorwa. Sisitemu nka DistServe, Splitwise, na Mooncake yarayamamaye, kandi vLLM na NVIDIA Dynamo ubu ubwato butandukanye. Ubushakashatsi burimo gusunika KV-cache kwimura optimizasiyo, guhuza cache no kongera gukoresha ibyifuzo, imbaraga zo kongera kuringaniza ibipimo bya prefill / decode munsi yimodoka ihindagurika, hamwe no guhuza cyane hamwe na prefix cishing hamwe na prefill chunked prefill. Mugihe imiterere ya Windows ikura miriyoni yikimenyetso, gutandukanya ibi byiciro biba ngombwa cyane kubiciro bikoresha neza, bitinze cyane.
Gushyira mu bikorwa Isi
Ikiganiro cyunganira kuganira inzira ndende yerekana compte-iremereye ya prefill cluster, hanyuma imigezi isubiza uhereye kumyibutsa-optimiz ya decode cluster kugirango ukomeze kwandika neza.
NVIDIA Dynamo na vLLM bareke abakoresha bashire prefill zitandukanye hamwe na decode yitsinda ryabakozi kugirango guturika kwigihe kirekire ntibishobora guhagarika ibisekuru bikomeza.
Mooncake (ikoreshwa na Kimi ya Moonshot AI) itandukanya prefill na decode kandi ikongeramo pisine yagabanijwe ya KV-cache kugirango igabanye kwishura inshuro nyinshi mubipimo.
Serivisi yo kurangiza-kode yegurira pisine ntoya ya progaramu ya progaramu ngufi na pisine nini ya decode, kubera ko ikiguzi kinini kiva mugutanga ibimenyetso byinshi bisohoka.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Disaggregated Prefill and Decode Serving quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
KServe na Model Gukorera Kubernetes
Ibibazo bikunze kubazwa
What is Disaggregated Prefill and Decode Serving?
Ubwubatsi bukora bugabanya imvugo nini yerekana urugero mubice bibiri bitandukanye - prefill na decode - ikanabikoresha kubidendezi bitandukanye bya GPU. Ntacyo bitwaye kuko ibyo byiciro byombi bifite ubushake buke bwibikoresho, kandi kubahatira kumashini imwe isesagura ubushobozi kandi bikababaza ubukererwe.
Niyihe mpamvu yibanze yibikoresho byo gutandukanya prefill na decode kubidendezi bitandukanye bya GPU?
Prefill itunganya ibintu byose muburyo bubangikanye kandi yuzuza compte, mugihe decode isoma cache cache buri ntambwe kandi igarukira kububiko bwibuke-butandukanye nibyifuzo byerekana ibidengeri bitandukanye, byigenga.
Ni ubuhe buryo bw'imibare bugomba kwimurwa kuva kubakozi bambere kugeza kubakozi?
Prefill yubaka cache cache kubibazo; decode ikeneye iyo cache kugirango ikomeze kubyara, bityo cache yoherejwe hejuru yihuta ihuza pisine.
Ni ikihe kibazo gutandukana bitagabanije muburyo busangiwe-GPU?
Kuri GPU isangiwe birebire biturika birashobora guhagarika intambwe ya decode ikomeza; kubitandukanya birinda ibyo kwivanga no guhagarika ubukererwe bwumurizo.
Ni ukubera iki prefill ishobora gutegurwa bikabije ariko ikagabanya inyungu ziva muburyo butandukanye?
Kuzuza ibintu byose byihuta hamwe, bityo ibyiciro binini bigaburira tensor cores neza; decode itanga ikimenyetso kimwe icyarimwe kandi igashyirwaho na memoire, bityo igapima ukundi.
Ni ubuhe buryo buhuza busanzwe bukoreshwa mu kwimura KV cache hagati y'ibidendezi bitandukanijwe?
Umuyoboro mwinshi, umuyoboro muto-utinda nka NVLink (intra-node) na InfiniBand (inter-node) birakenewe kugirango ihererekanyabubasha rya KV ntirihinduka icyuho gishya.