Cikawar da aka raba da yanke Hidima
Gine-ginen hidimar da ke raba babban ƙirar ƙirar harshe zuwa matakai daban-daban guda biyu - prefill da yanke hukunci - kuma yana gudanar da su akan wuraren tafki na GPUs daban-daban.
Dubawa
It matters because these two phases have opposite hardware appetites, and forcing them onto the same machines wastes capacity and hurts latency.
Zurfafa nutsewa
Lokacin da LLM ya amsa, yana aiki a matakai biyu. Prefill yana karanta duk faɗakarwa lokaci ɗaya kuma yana gina mahimmin ƙimar maɓalli (KV); wannan babba ne, mai layi daya, fashe mai ɗaure lissafi wanda ya cika raka'o'in lissafi na GPU. Ƙididdigar sai ya haifar da alamomi ɗaya bayan ɗaya, kowane mataki yana karanta dukkan ma'ajin KV-mai ɗaure ƙwaƙwalwar ajiya-bandwidth, mai sauƙin ƙididdigewa. Gudu tare, dogon prefill yana dakatar da yanke shawarar kowa (takewa kan layi), kuma batching biyun yana haifar da tsangwama. Rarrabawa yana sanya prefill akan tafkin GPU guda ɗaya kuma yana yanke lamba akan wani, yana canja wurin cache na KV a tsakanin su ta hanyar haɗin kai mai sauri kamar NVLink ko InfiniBand. Kowane tafkin ana daidaita shi kuma yana daidaita shi da kansa, yana haɓaka kayan aiki mai kyau, rage jinkirin wutsiya, da barin masu aiki su buga maƙasudin lokaci-zuwa-farko-alama da lokaci-kowa-fito-alama a lokaci guda.
Fahimtar Fasaha
Hanyoyi biyu sun bambanta a cikin kwalbar su. Ci gaba da aiwatar da duk alamun faɗakarwa a cikin layi daya, don haka ma'aunin FLOP ɗin sa tare da tsayin gaggawa kuma yana ƙara girman nau'in tensor. Ƙididdigar ƙididdiga ta atomatik: kowane sabon alamar yana buƙatar wucewa ɗaya na gaba wanda zai sake karanta cikakken cache na KV daga HBM, don haka abin da ake amfani da shi yana cike da bandwidth na ƙwaƙwalwar ajiya, ba ƙididdigewa ba. Rarraba yana amfani da wannan ta hanyar ƙima, batching, har ma da zabar daidaici daban-daban ga kowane tafkin, sannan jigilar ma'ajin KV daga ma'aikatan da aka rigaya don yanke ma'aikata.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Ƙarfafa Filla-filla da Yanke Hidima
Yi tsammanin rarrabuwa ya zama tsoho a cikin tarin samarwa. Tsarukan kamar DistServe, Splitwise, da Mooncake sun shahara da shi, kuma vLLM da NVIDIA Dynamo yanzu suna jigilar hanyoyin da aka rarraba. Bincike yana tura haɓakawar canja wurin cache-KV, cache cache da sake amfani da shi a cikin buƙatun, sake daidaita ma'auni na prefill/decode a ƙarƙashin zirga-zirgar zirga-zirgar zirga-zirgar zirga-zirgar zirga-zirgar zirga-zirgar zirga-zirgar zirga-zirgar zirga-zirgar zirga-zirga, da ƙarin haɗin kai tare da caching prefix da chunked prefill. Kamar yadda mahallin windows ke girma zuwa miliyoyin alamu, rarrabuwar waɗannan matakan yana ƙara zama mahimmanci ga farashi mai tsada, ƙarancin latency.
Aiwatar da Gaskiyar Duniya
Mataimakin taɗi yana bibiyar dogayen daftarin aiki zuwa gungu mai nauyi mai nauyi, sannan rafukan rafuka suna ba da amsa daga gungu na ƙayyadadden ƙwaƙwalwar ajiya don ci gaba da buga latency a santsi.
NVIDIA Dynamo da vLLM sun ƙyale masu aiki su tura prefill daban daban da ɓata ƙungiyoyin ma'aikata don fashe na dogon lokaci ba zai daskare tsararraki masu gudana ba.
Mooncake (wanda Moonshot AI's Kimi ke amfani da shi) yana rarraba prefill da yanke lamba kuma yana ƙara wurin shakatawa na KV-cache da aka rarraba don yanke sake lissafin gaggawa a ma'auni.
Sabis na kammala lambar yana keɓance ƙaramin tafkin prefill don gajeriyar faɗakarwa da babban wurin waƙa, tunda yawancin farashi yana zuwa daga yawo alamun fitarwa da yawa.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Disaggregated Prefill and Decode Serving quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
KServe da Model Hidima akan Kubernetes
Tambayoyin da ake yawan yi
What is Disaggregated Prefill and Decode Serving?
Gine-ginen hidimar da ke raba babban ƙirar ƙirar harshe zuwa matakai daban-daban guda biyu - prefill da yanke hukunci - kuma yana gudanar da su akan wuraren tafki na GPUs daban-daban. Yana da mahimmanci saboda waɗannan matakai guda biyu suna da sabanin sha'awar kayan masarufi, kuma tilasta su kan injuna iri ɗaya yana ɓata ƙarfin aiki kuma yana cutar da latti.
Menene ainihin dalilin hardware na raba prefill da yanke lamba akan wuraren waha na GPU daban-daban?
Prefill yana aiwatar da gabaɗayan faɗakarwa a cikin layi ɗaya kuma yana ƙididdige ƙididdigewa, yayin yanke hukunci yana karanta ma'ajin KV kowane mataki kuma yana iyakance ta hanyar bandwidth ƙwaƙwalwar ajiya - madaidaicin sha'awar da ke ba da izinin keɓance, wuraren waha mai zaman kansa.
Wane tsari na bayanai dole ne a canza shi daga ma'aikatan da aka cika su don yanke ma'aikata?
Prefill yana gina ma'ajin KV don faɗakarwa; Yanke buƙatun yana buƙatar cache ɗin don ci gaba da samarwa, don haka ana jigilar cache ɗin ta hanyar haɗin yanar gizo mai sauri zuwa wurin yanke kogon.
Wace matsala ce ke raguwa musamman a saitin GPU-na raba?
A kan GPUs da aka raba dogon fashe fashe na iya toshe matakai masu gudana; raba su yana hana wannan tsangwama kuma yana daidaita lattin wutsiya.
Me yasa za'a iya yin amfani da prefill da ƙarfi amma yanke fa'idodin daga kunnawa daban-daban?
Gabatar da aiwatar da dukkan alamu na gaggawa tare, don haka manyan batches suna ciyar da muryoyin tensor da kyau; yanke hukunci yana haifar da alama ɗaya a lokaci guda kuma ana kulle shi ta hanyar ƙwaƙwalwar ajiya, don haka yana sikeli daban.
Wadanne hanyoyin haɗin kai ne yawanci ake amfani da su don matsar da ma'ajin KV tsakanin wuraren waha da aka raba?
Babban-bandwidth, ƙananan latency hanyoyin haɗin gwiwa kamar NVLink (intra-node) da InfiniBand (inter-node) ana buƙatar don haka canja wurin cache-KV ba ya zama sabon ƙulli.