Incamake
Ubwihisho bwa KV bubika urufunguzo kandi ruha agaciro transformateur yamaze kubara kuburyo idashobora kongera gukora kuri buri kimenyetso gishya - ariko irashobora ballon kuri gigabytes. KV cache optimizasiyo iragabanuka kandi ikayobora ubwo kwibuka kuburyo moderi ikora ibintu birebire kubakoresha benshi icyarimwe.
KV Cache Optimisiyoneri ni inyubako ya tekiniki yubaka igira ingaruka kumiterere yicyitegererezo, igiciro cyibikorwa remezo, ubukererwe, no kwizerwa mubipimo.
Kwibira cyane
Muri transformateur, buri kimenyetso gishya cyitabira ibimenyetso byose byabanjirije ukoresheje urufunguzo rwo kwitondera (K) nagaciro (V). Gusubiramo K na V kumurongo wose kuri buri ntambwe yaba quadratic kandi isesagura, moderi rero irabashiraho: KV cache. Ikibi ni ubunini. Ubwihisho bukura neza hamwe nuburebure bwikurikiranya, ingano yicyiciro, ibice, hamwe numutwe, kubwibyo birebire-bisabwa birashobora gukoresha ububiko bwa GPU burenze uburemere bwikitegererezo ubwabo. Optimisation ikemura ibi uhereye kumpande nyinshi: ububiko bwa page (vLLM's PagedAttention) ibika cache mubice bidahuza kugirango ikureho ibice kandi itume gusangira; kubara bibika K na V muri 8-bit cyangwa 4-bit; nimpinduka zubwubatsi nka Grouped-Query Attention (GQA) hamwe na Multi-Query Attention (MQA) reka imitwe myinshi yibibazo isangire bike urufunguzo / agaciro imitwe, igabanya cache ingano yinkomoko.
Ubushishozi
Urupapuro rwitondewe ruguza pratique-yibuka ya paji ivuye muri sisitemu y'imikorere: cache ibaho mubunini-bunini bwashushanyije hifashishijwe imbonerahamwe yo kureba, bityo rero gusaba gukoresha gusa ibibikenewe bakeneye hamwe nibisobanuro bisa (nka sisitemu isangiwe na sisitemu) birashobora kwerekanwa kumurongo umwe. Multi-head Latent Attention (Umudepite), ikoreshwa muri moderi ya DeepSeek, ikanda K na V mubice bito bisangiwe byihishe, bigabanya cyane kwibuka mugihe bikomeza neza.
Kumenya KV Cache Optimisation
Ubwihisho bwa KV bubika urufunguzo kandi ruha agaciro transformateur yamaze kubara kuburyo idashobora kongera gukora kuri buri kimenyetso gishya - ariko irashobora ballon kuri gigabytes. KV cache optimizasiyo iragabanuka kandi ikayobora ubwo kwibuka kuburyo moderi ikora ibintu birebire kubakoresha benshi icyarimwe. KV Cache Optimisiyoneri ni inyubako ya tekiniki yubaka igira ingaruka kumiterere yicyitegererezo, igiciro cyibikorwa remezo, ubukererwe, no kwizerwa mubipimo. Kugirango wubake byimbitse, fata KV Cache Optimisation nkicyitegererezo gikora, ntabwo ari ikintu kimwe: gusobanura ibyagezweho, gusobanura ibitekerezo, no gutandukanya ibyo sisitemu ishobora gukora byizewe nibisaba guca imanza zinzobere.
Mubimenyerezo, amatsinda akomeye akoresha KV Cache Optimisiyoneri yorohereza imyubakire, amakuru, nibikorwa remezo guhitamo kwizerwa nigiciro. Bandika ibipimo ngenderwaho byerekana intsinzi, bagerageza kurwanya amakuru afatika hamwe nakazi keza, kandi bagasubiramo bashingiye kubikorwa byagaragaye ko batsinzwe aho gutsinda inshuro imwe. Aha niho imyumvire yubumenyi ihinduka mubushobozi burambye kubicuruzwa, politiki, nibikorwa.
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka. Mugihe kimwe, Guhindura igipimo kimwe gishobora guhisha intege nke za sisitemu. Uburyo bukomeye cyane ni uguhuza umuvuduko wikigereranyo hamwe na disipuline yimiyoborere: kuyobora abaderevu, gufata ibimenyetso, gutangaza ibyemezo byicyemezo, no gukomeza kuvugurura uburyo bwo kwirinda nkimyitwarire yicyitegererezo, ibyo abakoresha bategereje, nibisabwa n'amategeko bigenda bihinduka.
Ingaruka z'Ingamba
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa. Mubikorwa byujuje ubuziranenge, ibi bihindurwa mumategeko agenga imikorere, imipaka nyirubwite, hamwe n'imihango yo gusubiramo kenshi kugirango amakipe ashobore kwigirira ikizere aho gupima ibidasobanutse.
Gushyira mu bikorwa Isi
Urupapuro rwa vLLMIbyitonderwa bikora ibiganiro byinshi hamwe mugupakira KV bidafite ibice byo kwibuka
Itsinda-Ikibazo Cyitonderwa muri moderi ya Llama igabanya ubunini bwa KV cache kuburyo burebure buringaniye mububiko bwa GPU
Kugabanya cache ya KV kuri 8-bit (KV8) kugirango igabanye hafi cache yibuka mugihe kirekire-incamake.
Prefix cishing yongeye gukoresha KV bloks ya sisitemu isangiwe mubihumbi n'ibisabwa API
Uburyo bwo Gushyira mu bikorwa
KV Cache Optimisation mubikorwa
VLLM's PagedIbyifuzo bitanga ibiganiro byinshi icyarimwe mugupakira KV bice bitagabanije kwibuka.
VLLM's PagedIcyifuzo gikora ibiganiro byinshi mugihe cyo gupakira uduce twa KV nta gutandukanya kwibuka Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
KV Cache Optimisation mubikorwa
Itsinda-Ikibazo Cyitonderwa muri moderi ya Llama igabanya ubunini bwa KV cache kuburyo burebure buringaniye mububiko bwa GPU.
Itsinda-Kubaza Icyitonderwa muri moderi ya Llama igabanya ubunini bwa KV cache kuburyo ibintu birebire bikwiranye na GPU yibuka Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byiza byimbere imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe.
KV Cache Optimisation mubikorwa
Kugabanya ubwihisho bwa KV kuri 8-bit (KV8) kugirango ugabanye hafi cache yibuka mugihe kirekire-incamake.
Kugabanya ubwihisho bwa KV kugeza kuri 8-bit (KV8) kugirango ugabanye hafi cache yibuka mugihe cyinyandiko ndende yerekana incamake Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo byimbere, kandi agakurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
KV Cache Optimisation mubikorwa
Prefix cishing yongeye gukoresha KV bloks ya sisitemu isangiwe mubihumbi n'ibisabwa API.
Prefix cishing yongeye gukoresha KV yibice bya sisitemu isangiwe mugihe cyibihumbi n'ibihumbi API isaba Amakipe ubusanzwe abona ibisubizo byiza iyo asobanuye ibipimo byujuje ubuziranenge imbere, agakomeza inzira yo kuzamura abantu kubibazo, kandi akurikirana inyungu zibyara umusaruro hamwe nibiciro byamakosa mugihe runaka.
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima. Fata buri ntambwe nk irembo ryibimenyetso: niba ibipimo bitujujwe, hagarika kuzenguruka, funga icyuho, hanyuma noneho wagure imikoreshereze.