Buuxinta la kala saaray oo Decode u adeegida
Qaab dhismeed adeege ah oo u kala qaybiya tusaalaynta tusaalaha luqadda ee laba weji oo kala duwan — horu-buuxinta iyo qeexidda — oo ku socodsiisa barkadaha kala duwan ee GPU-yada.
Dulmar
It matters because these two phases have opposite hardware appetites, and forcing them onto the same machines wastes capacity and hurts latency.
quusid qoto dheer
Marka LLM ka jawaabo, waxay u shaqeysaa laba marxaladood. Prefill wuxuu akhriyaa dhammaan isla markiiba wuxuuna dhisayaa kaydka qiimaha-furaha (KV); kani waa qarax weyn, barbar socda, kombuyuutar ku xidhan kaas oo buuxinaya cutubyada xisaabta GPU-da. Go'aaminta markaa waxay soo saartaa calaamado hal mar, tillaabo kastaa waxay akhrinaysaa dhammaan khasnadaha KV-xasuusta-bandwidth-ku-xidhan, si fudud-u xisaabin. Si wada jir ah u orda, horudhac dheer oo horudhac ah ayaa qof walba joojinaya koodka (madaxa-line-ka-hortagga), iyo isku-duubnida labada waxay abuurtaa faragelin. Kala qaybsanaantu waxay hordhigtaa hal barkad GPU ah oo waxay dejisaa mid kale, iyada oo u wareejinaysa kaydka KV dhexdooda isku xidhka degdega ah sida NVLink ama InfiniBand. Barkad kastaa si madaxbanaan ayaa loo hagaajiyay oo loo miisaamay, iyada oo hagaajinaysa wax-soo-saarka, hagaajinta daahnimada dabada, iyo u oggolaanaysa hawl-wadeenada inay garaacaan bartilmaameedyada calamadaha waqti-ilaa-koowaad iyo waqti-wax-soo-saar-calaamad isku mar.
Aragtida Farsamada
Labada weji waxay ku kala duwan yihiin dhalooyinkooda. Hore u-buuxinta waxay u shaqaysaa dhammaan calaamadaha degdegga ah ee isbarbar socda, markaa FLOPs waxay cabbirtaa dhererka degdegga ah waxayna kor u qaadaysaa koodhka tensor-ka. Decode waa autoregressive: calaamad kasta oo cusub waxay u baahan tahay hal baas oo hore ah oo dib u akhriya kaydka KV ee buuxa ee HBM, markaa wax-soo-saarka waxaa lagu xiraa xajmiga xajmiga xusuusta, ee ma xisaabiyo. Kala qaybsanaantu waxay tan kaga faa'iidaysanaysaa iyada oo cabbiraysa, qaybinaysa, iyo xataa dooranaysa isbarbaryaac kala duwan ee barkad kasta, ka dibna ka soo raridda kaydka KV ee shaqaalaha horudhaca ah si loo kala saaro shaqaalaha.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Buuxinta La Kala Saaray oo Go'aamin Adeegga
Filo in kala qaybsanaantu ay noqoto mid aan ku habboonayn xirmooyinka wax-soo-saarka. Nidaamyada ay ka midka yihiin DistServe, Splitwise, iyo Mooncake ayaa faafiyay, iyo vLLM iyo NVIDIA Dynamo hadda waxay raraan qaababka la kala saarayo. Cilmi-baadhistu waxay riixaysaa hagaajinta wareejinta khasnado ee KV, kaydinta kaydinta iyo dib-u-isticmaalka dhammaan codsiyada, dib-u-dheellitirka firfircoonida leh ee prefill/decode ratio ee hoos yimaada taraafikada beddelka, iyo is-dhexgalka adag ee kaydinta horgalayaasha iyo hor-buuxinta. Marka ay daaqadaha macnaha guud u koraan malaayiin calaamado ah, kala soocida wejiyadan ayaa sii kordheysa lama huraan u ah adeegida kharash-ku-ool ah, oo hooseeya.
Dhaqangelinta Adduunka-dhabta ah
Kaaliyaha sheekadu waxa uu dariiqa dukumeenti dheer ku dhiiri galinayaa in uu xisaabiyo koox horudhac ah oo culus,kadibna durdurrada ayaa ka jawaabaya kooxda go'aaminta xusuusta ee la hagaajiyay si ay u ilaaliso qorista daahida.
NVIDIA Dynamo iyo vLLM waxay u oggolaadaan hawl-wadeenadu inay geeyaan horudhacyo gaar ah oo ay dejiyaan kooxaha shaqaalaha si dillaaca degdegga ah uusan u qaboojin jiilalka socda.
Mooncake (oo ay adeegsato Moonshot AI's Kimi) waxay kala saartaa buuxinta oo kala saarta oo waxay ku daraysaa barkad kaydka KV-ga oo la qaybiyay si loo dhimo xisaabinta degdega ah ee qiyaasta.
Adeega dhamaystirka koodka ayaa u hibeeya barkad yar oo horudhac ah oo loogu talagalay dardar gelinta gaaban iyo barkad decode weyn, maadaama kharashka badankiisu ka yimaado qulqulka calaamado badan oo wax soo saar ah.
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Disaggregated Prefill and Decode Serving quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
KServe iyo Model u adeega Kubernetes
Su'aalaha soo noqnoqda
What is Disaggregated Prefill and Decode Serving?
Qaab dhismeed adeege ah oo u kala qaybiya tusaalaynta tusaalaha luqadda ee laba weji oo kala duwan — horu-buuxinta iyo qeexidda — oo ku socodsiisa barkadaha kala duwan ee GPU-yada. Waa arrin sababtoo ah labadan weji waxay leeyihiin rabitaanka cuntada qalabka oo iska soo horjeeda, oo ku qasbida isla mishiinada waxay luminaysaa awoodda waxayna dhaawacaysaa daahitaanka.
Waa maxay sababta qalabka aasaasiga ah ee kala soocida hor-buuxinta iyo kala-saarista barkadaha GPU ee kala duwan?
Buuxinta horudhacdu waxay u shaqaysaa isla markiiba si isbar-bar socda oo waxay xisaabisaa, halka kala-saarku akhriyo khasnadda KV tallaabo kasta oo ay xaddidan tahay bandwidth-xusuusta-rabitaanka cuntada ka soo horjeeda oo caddaynaya barkadaha si madaxbannaan loo hagaajiyay.
Qaabkee xogta ay tahay in laga wareejiyo shaqaalaha hordhaca ah si loo kala saaro shaqaalaha?
Prefill waxay u dhistaa kaydka KV ee degdegga ah; Decode waxay u baahan tahay kaydkaas si uu u sii wado soo saarista, markaa khasnadda waxa lagu soo raray si degdeg ah oo isku xidha barkada decode.
Dhibkee ayaa kala qaybsanaantu si gaar ah u yaraynaysaa habaynta GPU-ga la wadaago?
GPU-yada la wadaago qarxinta horudhaca dheer waxay xannibi kartaa tillaabooyinka goynta socda; kala soocidooda waxay ka hortagtaa faragelintaas waxayna dejisaa daahitaanka dabada.
Waa maxay sababta hor-u-buuxinta loogu habayn karaa si adag balse loo dejiyo faa'iidooyinka habaynta kala duwan?
Buuxinta horudhaca dhammaan calaamadaha isla markiiba, markaa dufcadaha waaweyn waxay si fiican u quudiyaan xudunta tensor-ka; Decode waxay soo saartaa hal calaamad markiiba waxaana lagu xidhaa xusuusta, markaa si kala duwan ayuu u miisaamaa.
Waa kuwee kuwa isku xidha ee sida caadiga ah loo isticmaalo in lagu raro kaydka KV inta u dhaxaysa barkadaha la kala saaray?
Isku-xidhka-bandwidth-sare, xidhiidhada daahsoon ee hooseeya sida NVLink (intra-node) iyo InfiniBand (inter-node) ayaa loo baahan yahay si wareejinta KV-cache aysan u noqon caqabadda cusub.