Disaggregated Prefill uye Decode Serving
Chivakwa chinoshandira chinotsemura hombe modhi yemutauro kuita zvikamu zviviri zvakaparadzana-prefill uye decode-uye inovamhanyisa pamadziva akasiyana eGPU.
Pfupiso
It matters because these two phases have opposite hardware appetites, and forcing them onto the same machines wastes capacity and hurts latency.
Kudzika Kwakadzika
Kana LLM yapindura, inoshanda mumatanho maviri. Prefill inoverenga kukurumidza kwese kamwechete uye inovaka kiyi-kukosha (KV) cache; uku kukuru, kwakafanana, kuputika-kusungwa kwekombuta kunozadza zvikamu zvemasvomhu zveGPU. Decode yobva yagadzira tokeni imwe panguva, nhanho imwe neimwe ichiverenga yese KV cache-yekuyeuka-bandwidth-yakasungwa, zvishoma-compute trickle. Mhanyai pamwe chete, refu prefill inomisa decode yemunhu wese (musoro-we-mutsara kuvharira), uye batching izvo zviviri zvinogadzira kukanganisa. Disaggregation inoisa prefill pane imwe GPU dziva uye decode pane imwe, kuendesa iyo KV cache pakati payo pamusoro inokurumidza kubatanidza seNVLink kana InfiniBand. Dziva rega rega rakarongedzerwa uye kuyerwa rakazvimiririra, kuvandudza goodput, kutsetseka muswe latency, uye kuita kuti vashandisi varove zvakasimba nguva-kusvika-yekutanga-chiratidzo uye nguva-yega-yekubuda-chiratidzo chetariro panguva imwe chete.
Technical Insight
Zvikamu zviviri izvi zvinosiyana muhombodo yavo. Prefill inogadzirisa zvese zvinokatyama tokens zvakafanana, saka FLOPs yayo inoyera nekukasira kureba uye inowedzera kunze tensor cores. Decode is autoregressive: chiratidzo chega chega chinoda pass yekumberi iyo inoverengazve yakazara KV cache kubva kuHBM, saka throughput inovharwa nendangariro bandwidth, kwete compute. Disaggregation inoshandisa izvi nekukura, batching, uye kunyange kusarudza akasiyana parallelism padziva rega rega, wozotumira iyo KV cache kubva kuvashandi veprefill kuti vatore vashandi.
Strategic Impact
Mutengo uye bhajeti
Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
Sarudzo dzakajeka
Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
Kudzora kwemhando yepamusoro
Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Ramangwana reDisaggregated Prefill uye Decode Serving
Tarisira kuti disaggregation ive yekusagadzika mumatura ekugadzira. Masisitimu akaita seDistServe, Splitwise, uye Mooncake akasimudzira, uye vLLM neNVIDIA Dynamo ikozvino inotumira nzira dzakapatsanurwa. Tsvagiridzo iri kusundira KV-cache kutamisa optimizations, cache kubatanidza uye kushandisa zvakare pane zvese zvikumbiro, zvine simba kuenzanirana kweprefill/decode reshiyo pasi pekuchinja kwetraffic, uye kusanganisa kwakasimba ne prefix caching uye chunked prefill. Sezvo mahwindo emamiriro anokura kuita mamirioni ezviratidzo, kupatsanura zvikamu izvi kunowedzera kukosha kune inodhura-inoshanda, yakaderera-latency kushumira.
Real-World Implementation
Mubatsiri wechat anoendesa gwaro refu richienda kune compute-inorema prefill cluster, ipapo nzizi mhinduro kubva mu memory-optimized decode cluster kuti kutaipa kurambe kwakapfava.
NVIDIA Dynamo uye vLLM regai vashandisi vatumire zvakaparadzana prefill uye decode mapoka evashandi kuitira kuti kuputika kwekureba kurege kuomesa zvizvarwa zvirikuenderera mberi.
Mooncake (inoshandiswa neMoonshot AI's Kimi) inopatsanura prefill uye decode uye inowedzera yakagoverwa yeKV-cache dziva kucheka kukurumidza kudzokororwa pachiyero.
Iyo kodhi-yekupedzisa sevhisi inotsaurira diki prefill dziva rekukurumidza kukurumidza uye hombe decode dziva, sezvo mari zhinji inobva mukutepfenyura akawanda anobuda tokeni.
Njodzi & Guardrails
Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Implementation Roadmap
Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
Benchmark pasi pechokwadi mutoro uye data mamiriro.
Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Disaggregated Prefill and Decode Serving quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
KServe uye Model Kushanda paKubernetes
Mibvunzo inowanzo bvunzwa
What is Disaggregated Prefill and Decode Serving?
Chivakwa chinoshandira chinotsemura hombe modhi yemutauro kuita zvikamu zviviri zvakaparadzana-prefill uye decode-uye inovamhanyisa pamadziva akasiyana eGPU. Izvo zvine basa nekuti zvikamu zviviri izvi zvine zvakapesana nezvishuwo zvehardware, uye kuvamanikidza kuti vapinde mumichina imwechete kutambisa huwandu uye kukuvadza latency.
Ndechipi chikonzero chehardware chikonzero chekuparadzanisa prefill uye decode pamadziva eGPU akasiyana?
Prefill inogadzirisa iyo yese kukurumidza mukufanana uye inozadza compute, nepo decode ichiverenga iyo KV cache nhanho imwe neimwe uye inoganhurwa nendangariro bandwidth-inopikisa zvishuwo zvinoruramisa akaparadzana, akazvimirira madziva akarongeka.
Ndechipi chimiro chedata chinofanira kutamiswa kubva kuvashandi vanofanozadza kuti vatore vashandi?
Prefill inovaka iyo KV cache yekukurumidza; decode inoda iyo cache kuti ienderere mberi nekugadzira, saka cache inotumirwa pamusoro pekukurumidza kubatanidzwa kune decode dziva.
Nderipi dambudziko iro disaggregation inoderedza zvakanyanya mukugovaniswa-GPU setup?
PamaGPU akagovaniswa kureba kwekuzadza kwekutanga kunogona kuvharira kuenderera mberi decode matanho; kuvaparadzanisa kunodzivirira kupindira ikoko uye kunodzikamisa muswe latency.
Nei prefill ichigona kuunganidzwa zvine hukasha asi decode mabhenefiti kubva kune akasiyana tuning?
Prefill maitiro ese ekukurumidza tokens pamwechete, saka mabheji akakura anodyisa ma tensor cores zvakanaka; decode inogadzira chiratidzo chimwe panguva uye inovharwa nendangariro, saka inoyera zvakasiyana.
Ndedzipi dzekubatanidza dzinowanzo shandiswa kufambisa iyo KV cache pakati pemadziva akapatsanurwa?
Yakakwira-bandwidth, yakaderera-latency zvinongedzo seNVLink (intra-node) uye InfiniBand (inter-node) inodiwa kuitira kuti KV-cache kutamisa kusave iyo nyowani bhodhoro.