Prefill ak decode buñ xaaj
Architecture buy liggéey buy xaaj modelu làkk bu yaatu ci ñaari fase yu wuute - prefill ak decode - ba noppi doxal leen ci pool yu wuute ci GPUs.
Résumé
It matters because these two phases have opposite hardware appetites, and forcing them onto the same machines wastes capacity and hurts latency.
Plongeur bu xóot
Su LLM tontu, dafay liggéey ci ñaari pàcc. Prefill dafay jàng laaj bi yépp benn yoon ba noppi tabax cache key-valeur (KV); lii ab burst bu rëy la, paralel, buñu tënk ci xayma buy feesal GPU bi ci math. Decode dafay defar ay token benn-benn, jéego bu nekk di jàng cache KV bi yépp - ab bandwidth bu memory, trickle bu woyof. Daw ñoom ñaar, ab prefill bu gudd dafay taxawal decode ku nekk (bopp-of-line blocking), ak batching ñaar yi defar ay jafe-jafe. Desagrégation dafay def prefill ci benn pool GPU ak decode ci beneen, toxal cache KV ci seen biir ci lëkkaloo yu gaaw yu melni NVLink wala InfiniBand. Pool bu nekk dañu koy ajuste ak scale moom boppam, gëna baaxal produit bi, gëna yombal latency geen gi, ba noppi may operatër yi ñu mëna dóor time-to-first-token ak time-per-output-token ci benn yoon.
Gis-gis xarala
Ñaari fase yi wuute nañu ci seeni bottleneck. Prefill dafay doxal bépp token bu gaaw ci paralel, moo tax FLOPs yi dañuy yokk guddaay bu gaaw bi ba noppi di gëna bari tensor core yi. Decode autoregressif la: token bu bees bu nekk dafa soxla benn jéego bu jëm kanam buy jàngaat cache KV bi yépp ci HBM, kon génne gi dafay jaar ci yaatuwaayu mémoire bi, du calcul. Desaggregation dafay jëfandikoo lii ci dimensionnement, batching, ba ci tànn parallelism bu wuute ngir pool bu nekk, ba noppi yónnee cache KV ci liggéeykat yiy prefill ngir decode liggéeykat yi.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu prefill ak decode buñ xaaj
Xaarandil ni disagregation nekk na lu jaadu ci pilu liggéey yi. Sistem yu melni DistServe, Xaajale, ak Mooncake ñoo ko siiwal, vLLM ak NVIDIA Dynamo leegi dañuy yónnee ay anam yu wuute. Gëstu dafay puus KV-cache toxal gëna xéewale, cache pooling ak jëfandikoowaat ci laaj yi, ekilibre dinaamik bu prefill / decode ratios ci suufu dem bi ak dikk bi, ak boole bu gëna dëgër ak cache prefix ak prefill chunked. Lu palanteer yi di màgg ba nekk ay milioŋ ciy jeton, tàqale fase yooyu dafay gëna am solo ngir am xaalis bu bari te baax.
Doxal ci àdduna dëgg
Assistant chat dafay yónnee dokimaa yu gudd yi ci cluster prefill bu bari ordinatër, ba noppi mu yónnee tontu yi ci cluster decode buñ defaree mémoire ngir mëna wéy di bind.
NVIDIA Dynamo ak vLLM may nañu operatër yi ñu tàqale prefill ak decode grupu liggéeykat yi suko defee ñu bari ay laaj yu gudd duñu gelé jikkoom yiy wéy.
Mooncake (Kimi bu Moonshot AI moo ko jëfandikoo) dafay xaaj prefill ak decode ba noppi yokk ci pool KV-cache buñ séddale ngir dagg xayma yu bari yi ci eskaal bi.
Ab sarwis buy yeggali kode dafay jagleel benn piscine bu ndaw ngir ay laaj yu gàtt ak benn piscine decode bu mag, ndax njëg yi gëna bari ñu ngi bawoo ci streaming ay token yu bari.
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Disaggregated Prefill and Decode Serving quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
KServe ak xeetu liggéey ci Kubernetes
Laaj yi ñuy faral di laaj
What is Disaggregated Prefill and Decode Serving?
Architecture buy liggéey buy xaaj modelu làkk bu yaatu ci ñaari fase yu wuute - prefill ak decode - ba noppi doxal leen ci pool yu wuute ci GPUs. Dafa am solo ndax ñaari fase yooyu dañu am bëgg-bëggu hardware bu wuute, te forse leen ñu nekk ci benn masin dafay yàq kàttan ba noppi gaañ latency.
Lan mooy sabab bi tax ñuy tàqale prefill ak decode ci pool GPU yu wuute?
Prefill dafay doxal prompt bi yépp ci paralel ba noppi saturate calcul, ci noonu la decode di jàng cache KV jéego bu nekk te dafay yam ci bandwidth memory-appetites yu wuute yuy justifier pool yu wuute, yuñ defar seen bopp.
Ban jumtukaayu done lañu wara toxal ci liggéeykat yiy prefill dem ci liggéeykat yiy decode?
Prefill dafay tabax cache KV ngir laaj bi; decode mingi soxla cache bi ngir mëna wéy di defar, moo tax cache bi dafay jaar ci interconnexion bu gaaw ci pool decode bi.
Ban jafe-jafe la disaggregation di wàññi ci tabb GPU buñ bokk?
Ci GPU yiñ bokk, ab prefill bu gudd mën na gàllankoor jéego decode yiy wéy; tàqale leen dafay tere jafe-jafe yi ba noppi dakkal latency geen gi.
Lan moo waral prefill mën nañu ko batch bu baax waaye decode njariñu tuning bu wuute?
Prefill dafay boole jeton prompt yépp, suko defee lots yu gëna mag di dundal core tensor yi bu baax; decode dafay defar benn token benn yoon, te memory bi dafay tënk ko, moo tax dafay wuute ci eskaal yi.
Ban lëkkaloo lañuy gëna jëfandikoo ngir toxal cache KV ci diggante piscine yuñ xaajale?
Bandwidth bu rëy, lëkkalekaay yu néew latency yu melni NVLink (ci biir node) ak InfiniBand (ci biir node) lañu soxla suko defee toxal KV-cache du nekk bottleneck bu bees.