Ukugcwalisa Ngaphambilini Okuhlukanisiwe kanye Nokusebenza Kwekhodi
I-architecture esebenzayo ehlukanisa ukuchazwa kwemodeli yolimi ibe izigaba ezimbili ezihlukene—ukugcwalisa kuqala nokunquma—futhi iwaqhube kumachibi ahlukene e-GPU.
Uhlolojikelele
It matters because these two phases have opposite hardware appetites, and forcing them onto the same machines wastes capacity and hurts latency.
I-Deep Dive
Uma i-LLM iphendula, isebenza ngezigaba ezimbili. Ukugcwalisa kuqala kufunda wonke ukwaziswa ngesikhathi esisodwa futhi kwakha inqolobane yenani lokhiye (KV); lokhu ukuqhuma okukhulu, okuhambisanayo, okuboshelwe kukhompuyutha okugcwele amayunithi ezibalo e-GPU. I-Decode bese ikhiqiza amathokheni elilodwa ngesikhathi, isinyathelo ngasinye sifunda yonke inqolobane ye-KV—indlela ebopha umkhawulokudonsa, i-compute elula. Hambani ndawonye, ukugcwalisa okude kudala amakhodi awo wonke umuntu (ukuvinjwa kwekhanda lomugqa), futhi ukuhlanganisa kokubili kudala ukuphazamiseka. Ukwahlukanisa kubeka ukugcwalisa kuqala kuchibi elilodwa le-GPU bese kuqoshwa kwelinye, kudlulisa inqolobane ye-KV phakathi kwayo ngoxhumano olusheshayo njenge-NVLink noma i-InfiniBand. I-pool ngayinye ishunwa futhi ikalwe ngokuzimela, ithuthukisa i-goodput, i-smoothing tail latency, futhi ivumela opharetha ukuthi bashaye okuhloswe ngaso ithokheni yesikhathi ukuya kokuqala kanye nethokheni yesikhathi ngasinye ngesikhathi esisodwa.
I-Technical Insight
Zombili izigaba zihlukene ngebhodlela lazo. Ukugcwalisa kuqala kucubungula wonke amathokheni okwaziswa ngokuhambisana, ukuze i-FLOPs yayo ikale ngobude obusheshayo futhi ikhulisa ama-tensor cores. I-Decode i-autoregressive: ithokheni ngayinye entsha idinga iphasi elilodwa eliya phambili elifunda kabusha inqolobane ye-KV egcwele evela ku-HBM, ngakho ukuphuma kufakwe kusango lomkhawulokudonsa wememori, hhayi ukubala. Ukwahlukanisa kusebenzisa lokhu ngosayizi, ukuhlanganisa, ngisho nokukhetha ukufana okuhlukile kwechibi ngalinye, bese kuthunyelwa inqolobane ye-KV kusuka kubasebenzi abagcwaliswa ngaphambilini ukuze kuqondwe abasebenzi.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Ikusasa Lokugcwalisa Ngaphambilini Okuhlukanisiwe kanye Nokunikezwa Kwekhodi Yekhodi
Lindela ukuhlukaniswa ukuze kube okuzenzakalelayo kuzitaki zokukhiqiza. Amasistimu afana ne-DistServe, i-Splitwise, ne-Mooncake ayenze yaduma, futhi i-vLLM ne-NVIDIA Dynamo manje ithumela izindlela ezihlukanisiwe. Ucwaningo luphusha ukulungiselelwa kokudluliswa kwenqolobane ye-KV, ukuhlanganisa inqolobane nokuphinda kusetshenziswe kuzo zonke izicelo, ukulinganisa kabusha okunamandla kokugcwalisa kusengaphambili/ikhodi yokukala ngaphansi kwethrafikhi eguquguqukayo, kanye nokuhlanganiswa okuqinile nokugcinwa kwesikhashana kwesiqalo kanye nokugcwalisa kuqala okuhlanganisiwe. Njengoba amafasitela omongo ekhula abe izigidi zamathokheni, ukuhlukanisa lezi zigaba kuba semqoka kakhulu ekukhonzeni okubiza izindleko, ukubambezeleka okuphansi.
Ukuqaliswa Komhlaba Wangempela
Umsizi wengxoxo uhambisa idokhumenti ende eya kuqoqo lokugcwalisa esindayo, bese izimpendulo ezisakazwayo zisuka kuqoqo lekhodi elungiselelwe inkumbulo ukuze kugcinwe ukuthayipha ukubambezeleka kushelela.
I-NVIDIA Dynamo kanye ne-vLLM ivumela opharetha ukuthi basebenzise amaqembu ezisebenzi ahlukene okugcwalisa kusengaphambili futhi akhiphe ikhodi ukuze ukuqhuma kokwaziswa okude kungavimbi izizukulwane eziqhubekayo.
I-Mooncake (esetshenziswa i-Moonshot AI's Kimi) ihlukanisa ukugcwalisa kuqala bese ikhipha ikhodi futhi yengeza inqolobane ye-KV-cache esabalalisiwe ukuze kunqandwe ukuphinda kusetshenziswe ngokushesha okungafuneki esikalini.
Isevisi yokuqedela ikhodi inikezela ngechibi elincane lokugcwalisa kusengaphambili ukuze uthole imiyalo emifushane kanye nephuli enkulu yokukhipha ikhodi, njengoba izindleko eziningi zivela ekusakazeni amathokheni amaningi okukhiphayo.
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Disaggregated Prefill and Decode Serving quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
I-KServe kanye Nokusebenzela Imodeli ku-Kubernetes
Imibuzo evame ukubuzwa
What is Disaggregated Prefill and Decode Serving?
I-architecture esebenzayo ehlukanisa ukuchazwa kwemodeli yolimi ibe izigaba ezimbili ezihlukene—ukugcwalisa kuqala nokunquma—futhi iwaqhube kumachibi ahlukene e-GPU. Kubalulekile ngoba lezi zigaba ezimbili zinezifiso ezihlukile zehadiwe, futhi ukuziphoqa ukuthi zingene emshinini ofanayo kumosha umthamo futhi kulimaza ukubambezeleka.
Siyini isizathu sezingxenyekazi zekhompuyutha esiyinhloko sokuhlukanisa ukugcwalisa kuqala nokukhipha ikhodi kumachibi e-GPU ahlukene?
Ukugcwalisa kuqala kucubungula yonke imiyalo ngokuhambisana futhi kugcwale ikhompuyutha, kuyilapho i-decode ifunda inqolobane ye-KV isinyathelo ngasinye futhi ikhawulwa umkhawulokudonsa wememori—izifiso eziphambene ezithethelela amachibi ahlukene, ashunwe ngokuzimela.
Iluphi uhlaka lwedatha okufanele ludluliswe lusuka kubasebenzi abagcwalisa kuqala ukuze luqoshwe abasebenzi?
Ukugcwalisa kuqala kwakha inqolobane ye-KV yokwaziswa; i-decode idinga leyo cache ukuze iqhubeke nokukhiqiza, ngakho inqolobane ithunyelwa ngoxhumano olusheshayo endaweni yokukhipha ikhodi.
Iyiphi inkinga ukuhlukaniswa okuyehlisa ngokukhethekile ekusethweni kwe-GPU eyabiwe?
Kuma-GPU abiwe ukuqhuma kwesikhathi eside kokugcwalisa kungavimba izinyathelo eziqhubekayo zokunquma; ukuwahlukanisa kuvimbela lokho kugxambukela futhi kuzinzise ukubambezeleka komsila.
Kungani ukugcwalisa kuqala kungase kuhlanganiswe ngamandla kodwa kuqondwe izinzuzo ezivela ekushuneni okuhlukile?
Ukugcwalisa kuqala kucubungula wonke amathokheni asheshayo ndawonye, ukuze amaqoqo amakhulu ondle ama-tensor cores kahle; i-decode ikhiqiza ithokheni eyodwa ngesikhathi futhi ivalwa yinkumbulo, ngakho ikala ngokuhlukile.
Yiziphi izixhumo ezivame ukusetshenziselwa ukuhambisa inqolobane ye-KV phakathi kwamachibi ahlukanisiwe?
Izixhumanisi zomkhawulokudonsa ophezulu, ezingabambeki kancane njenge-NVLink (intra-node) kanye ne-InfiniBand (inter-node) ziyadingeka ukuze ukudluliswa kwenqolobane ye-KV kungabi ibhodlela elisha.