Dubawa
Gine-ginen hidimar da ke raba babban ƙirar ƙirar harshe zuwa matakai daban-daban guda biyu - prefill da yanke hukunci - kuma yana gudanar da su akan wuraren tafki na GPUs daban-daban. Yana da mahimmanci saboda waɗannan matakai guda biyu suna da sabanin sha'awar kayan masarufi, kuma tilasta su kan injuna iri ɗaya yana ɓata ƙarfin aiki kuma yana cutar da latti.
Fil ɗin da aka Rarraba da Ƙaddamarwa Hidima wani shingen ginin fasaha ne wanda ke shafar ingancin ƙira, farashin kayayyakin more rayuwa, jinkiri, da aminci a sikeli.
Zurfafa nutsewa
Lokacin da LLM ya amsa, yana aiki a matakai biyu. Prefill yana karanta duk faɗakarwa lokaci ɗaya kuma yana gina mahimmin ƙimar maɓalli (KV); wannan babba ne, mai layi daya, fashe mai ɗaure lissafi wanda ya cika raka'o'in lissafi na GPU. Ƙididdigar sai ya haifar da alamomi ɗaya bayan ɗaya, kowane mataki yana karanta dukkan ma'ajin KV-mai ɗaure ƙwaƙwalwar ajiya-bandwidth, mai sauƙin ƙididdigewa. Gudu tare, dogon prefill yana dakatar da yanke shawarar kowa (takewa kan layi), kuma batching biyun yana haifar da tsangwama. Rarrabawa yana sanya prefill akan tafkin GPU guda ɗaya kuma yana yanke lamba akan wani, yana canja wurin cache na KV a tsakanin su ta hanyar haɗin kai mai sauri kamar NVLink ko InfiniBand. Kowane tafkin ana daidaita shi kuma yana daidaita shi da kansa, yana haɓaka kayan aiki mai kyau, rage jinkirin wutsiya, da barin masu aiki su buga maƙasudin lokaci-zuwa-farko-alama da lokaci-kowa-fito-alama a lokaci guda.
Fahimtar Fasaha
Hanyoyi biyu sun bambanta a cikin kwalbar su. Ci gaba da aiwatar da duk alamun faɗakarwa a cikin layi daya, don haka ma'aunin FLOP ɗin sa tare da tsayin gaggawa kuma yana ƙara girman nau'in tensor. Ƙididdigar ƙididdiga ta atomatik: kowane sabon alamar yana buƙatar wucewa ɗaya na gaba wanda zai sake karanta cikakken cache na KV daga HBM, don haka abin da ake amfani da shi yana cike da bandwidth na ƙwaƙwalwar ajiya, ba ƙididdigewa ba. Rarraba yana amfani da wannan ta hanyar ƙima, batching, har ma da zabar daidaici daban-daban ga kowane tafkin, sannan jigilar ma'ajin KV daga ma'aikatan da aka rigaya don yanke ma'aikata.
Jagorar Filayen da aka Rarraba da Yanke Hidima
Gine-ginen hidimar da ke raba babban ƙirar ƙirar harshe zuwa matakai daban-daban guda biyu - prefill da yanke hukunci - kuma yana gudanar da su akan wuraren tafki na GPUs daban-daban. Yana da mahimmanci saboda waɗannan matakai guda biyu suna da sabanin sha'awar kayan masarufi, kuma tilasta su kan injuna iri ɗaya yana ɓata ƙarfin aiki kuma yana cutar da latti. Fil ɗin da aka Rarraba da Ƙaddamarwa Hidima wani shingen ginin fasaha ne wanda ke shafar ingancin ƙira, farashin kayayyakin more rayuwa, jinkiri, da aminci a sikeli. Don haɓaka fahimta mai zurfi, bi da Prefill da aka Rarraba da Yanke Hidima azaman ƙirar aiki, ba fasali ɗaya ba: ayyana sakamakon da ake so, fayyace zato, da raba abin da tsarin zai iya yi da dogaro daga abin da har yanzu ke buƙatar yanke hukunci na ƙwararru.
A aikace, ƙungiyoyi masu ƙarfi masu amfani da Rarraba Prefill da Yanke Hidima suna haɓaka gine-gine, bayanai, da zaɓin abubuwan more rayuwa da aminci da farashi. Suna rubuta ƙayyadaddun ƙa'idodin nasara, gwaji akan bayanan gaskiya da gudanawar aiki, da jujjuyawar bisa ga tsarin gazawar da aka lura maimakon cin nasara na lokaci ɗaya. Wannan shine inda fahimtar ka'idar ta juya zuwa iyawa mai dorewa a cikin samfura, manufofi, da ayyuka.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A lokaci guda, Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin. Hanyar da ta fi dacewa ita ce haɗa saurin gwaji tare da horon gudanarwa: gudanar da matukin jirgi, kama shaida, buga rajistan ayyukan yanke shawara, da ci gaba da sabunta abubuwan tsaro kamar yadda halayen ƙira, tsammanin mai amfani, da buƙatun tsari ke tasowa.
Dabarun Tasiri
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa. A cikin ƙawance masu inganci, ana fassara wannan zuwa ƙa'idodin aiki waɗanda za a iya aunawa, iyakokin ikon mallaka, da kuma bita-da-kullin bita don ƙungiyoyi su iya haɓaka kwarin gwiwa a maimakon ɓata shakku.
Aiwatar da Gaskiyar Duniya
Mataimakin taɗi yana bibiyar dogayen daftarin aiki zuwa gungu mai nauyi mai nauyi, sannan rafukan rafuka suna ba da amsa daga gungu na ƙayyadadden ƙwaƙwalwar ajiya don ci gaba da buga latency a santsi.
NVIDIA Dynamo da vLLM sun ƙyale masu aiki su tura prefill daban daban da ɓata ƙungiyoyin ma'aikata don fashe na dogon lokaci ba zai daskare tsararraki masu gudana ba.
Mooncake (wanda Moonshot AI's Kimi ke amfani da shi) yana rarraba prefill da yanke lamba kuma yana ƙara wurin shakatawa na KV-cache da aka rarraba don yanke sake lissafin gaggawa a ma'auni.
Sabis na kammala lambar yana keɓance ƙaramin tafkin prefill don gajeriyar faɗakarwa da babban wurin waƙa, tunda yawancin farashi yana zuwa daga yawo alamun fitarwa da yawa.
Hanyoyin Aiwatarwa
Fil ɗin da aka Rarraba da Yanke Hidima a aikace
Mataimakin taɗi yana bibiyar dogayen daftarin aiki zuwa gungu mai nauyi mai nauyi, sannan rafukan rafuka suna ba da amsa daga gungu na ƙayyadadden ƙwaƙwalwar ajiya don ci gaba da buga latency a santsi.
Mataimakin taɗi yana bibiyar dogayen daftarin aiki zuwa gungu mai cike da ƙima, sannan rafukan suna ba da amsa daga gungu na ƙayyadaddun ƙididdigewa don ci gaba da buga latency ƙwalƙwalwa yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓaka ɗan adam don ƙararraki, da bin diddigin nasarorin samarwa da tsadar kurakurai akan lokaci.
Fil ɗin da aka Rarraba da Yanke Hidima a aikace
NVIDIA Dynamo da vLLM sun ƙyale masu aiki su tura prefill daban daban da ɓata ƙungiyoyin ma'aikata don fashe na dogon lokaci ba zai daskare tsararraki masu gudana ba.
NVIDIA Dynamo da vLLM sun bar masu aiki su tura prefill daban-daban da kuma rarraba ƙungiyoyin ma'aikata don haka fashe na dogon lokaci ba zai daskare tsararraki masu gudana Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓaka ɗan adam don ƙararraki, da bin diddigin nasarorin samarwa da ƙimar kuskure akan lokaci.
Fil ɗin da aka Rarraba da Yanke Hidima a aikace
Mooncake (wanda Moonshot AI's Kimi ke amfani da shi) yana rarraba prefill da yanke lamba kuma yana ƙara wurin shakatawa na KV-cache da aka rarraba don yanke sake lissafin gaggawa a ma'auni.
Mooncake (wanda Moonshot AI's Kimi ke amfani da shi) yana rarraba prefill da ƙaddamarwa kuma yana ƙara waƙar KV-cache da aka rarraba don yanke ƙididdige ƙididdigewa da sauri a ma'auni Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓaka ɗan adam don ƙararraki, da bin diddigin abubuwan samarwa da tsadar kurakurai akan lokaci.
Fil ɗin da aka Rarraba da Yanke Hidima a aikace
Sabis na kammala lambar yana keɓance ƙaramin tafkin prefill don gajeriyar faɗakarwa da babban wurin waƙa, tunda yawancin farashi yana zuwa daga yawo alamun fitarwa da yawa.
Sabis na kammala lambar yana keɓe ƙaramin wurin tafki don gajeriyar faɗakarwa da babban wurin waƙa, tunda yawancin farashi yana zuwa daga yawo da yawa alamun fitarwa Ƙungiyoyi yawanci suna samun sakamako mafi kyau lokacin da suka ayyana ma'auni masu inganci a gaba, kiyaye hanyar haɓakar ɗan adam don ƙararraki, da kuma bin diddigin abubuwan da aka samu na samarwa da ƙimar kuskure akan lokaci.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli. Ɗauki kowane mataki azaman ƙofar shaida: idan ba a cika sharuɗɗa ba, dakatar da fitar, rufe tazarar, sannan kawai faɗaɗa amfani.