Inqolobane ye-KV
Inqolobane ye-KV igcina ama-vectors angukhiye kanye nenani i-transformer eseyibalile kumathokheni wangaphambilini, ngakho-ke akudingeki ukuthi iwabuyisele kuwo wonke amagama amasha ewakhayo.
Uhlolojikelele
It is the single biggest reason text generation is fast — and the main thing eating your GPU memory during long conversations.
I-Deep Dive
Ama-Transformer akhiqiza umbhalo ithokheni eyodwa ngesikhathi, futhi isendlalelo sokunaka sethokheni entsha ngayinye sidinga ukuqhathanisa nethokheni ngayinye yangaphambili. Indlela yokunaka iphendula ithokheni ngayinye ibe umbuzo, ukhiye, kanye nevekhtha yenani. Ngaphandle kokugcinwa kunqolobane, ukukhiqiza inombolo yethokheni engu-1,000 kungasho ukubala kabusha okhiye namanani awo wonke amathokheni angaphambili angama-999 ngaso sonke isikhathi - umsebenzi we-quadratic, omoshayo. Inqolobane ye-KV ilondoloza lawo mavektha angukhiye kanye nenani ngemva kokuba esenziwe ikhompuyutha okokuqala futhi aphinde asetshenziswe, ngakho isinyathelo esisha ngasinye sihlanganisa kuphela ama-vectors kuthokheni eyodwa entsha futhi ihambela inqolobane egciniwe. Lokhu kunciphisa izindleko zethokheni ngayinye ukusuka ekulinganiseni ngobude bokulandelana kuye kokucishe kube njalo. Ukuhwebelana kuyinkumbulo: inqolobane ikhula ngokulandelana ngobude bomongo, inani lezendlalelo, nezinhloko zokunaka, ngokuvamile kuba umthengi wenkumbulo ovelele ekunikezeni okuqukethwe okude.
I-Technical Insight
Ngesikhathi 'sesigaba sokugcwalisa' imodeli icubungula konke ukwaziswa futhi igcwalise inqolobane; ngesikhathi 'sokukhipha ikhodi' ihlanganisa i-K/V yethokheni ngayinye ngesinyathelo ngasinye futhi iphinda ibambe. Isilinganiso senqolobane yesayizi njengo-2 (K no-V) × izendlalelo × amakhanda × ikhanda_dim × ukulandelana_ubude × inqwaba, ngokunemba okukhethiwe. Ukuze kuncishiswe lokhu, amamodeli esimanje asebenzisa ukunakwa okuhlanganisiwe noma imibuzo eminingi ukuze abelane ngokhiye/amanani kuwo wonke amakhanda, futhi amasistimu wokuphakela njenge-vLLM asebenzisa i-PagedAttention ukwaba inqolobane kumabhulokhi angahlangani, ukuhlukanisa ukuhlukaniswa kanye nemfucuza.
I-Strategic Impact
Isivinini nesikali
Ukugeleza komsebenzi wolimi kungahamba ngokushesha ngaphandle kokudela ukuvumelana.
Finyelela futhi ufinyelele
Yandisa ukufinyelela kuzo zonke izilimi nezitayela zokuxhumana.
Izinqumo ezicacile
Amaqembu angachitha isikhathi esiningi ekwahluleleni kuyilapho i-automation isingatha impinda.
Ikusasa Lenqolobane ye-KV
Njengoba umongo amafasitela afinyelela kumakhulu ezinkulungwane zamathokheni, inqolobane ye-KV iba ibhodlela eliphakathi, ngakho-ke ukuqanjwa okusha kushubile: ukugcinwa kwenqolobane kumabhithi ayi-8 noma angu-4, izinqubomgomo zokuxoshwa ezilahla amathokheni abaluleke kakhulu, ukwabelana kwesiqalo sesicelo esiphambene, kanye nokulayishwa ku-CPU noma idiski. Ukushintsha kwezakhiwo njengokunakwa okucashile kwamakhanda amaningi kucindezela inqolobane ngokwayo. Lindela idizayini eqhubekayo ehambisanayo yokuhlukahluka kokunakwa nezinhlelo zenkumbulo ezihloselwe ukusebenzela izimo ezinde kakhulu ngeshibhile nangokusebenza okuphezulu.
Ukuqaliswa Komhlaba Wangempela
Ukusheshisa izimpendulo ze-chatbot ngokusebenzisa kabusha okhiye/amanani agcinwe kunqolobane asuka kumlando wengxoxo esikhundleni sokuyicubungula ijika ngalinye.
Isiqalo sokulondoloza isikhashana esabelana ngenqolobane yokwaziswa kwesistimu ende kubasebenzisi abaningi, ukunciphisa izindleko nokubambezeleka.
I-PagedAttention ye-vLLM ephatha inqolobane ye-KV emabhuloki ukuze kusetshenziswe izicelo eziningi ngesikhathi esisodwa ku-GPU eyodwa ngempumelelo.
Ukulinganisa inqolobane ye-KV ukuze kwehliswe ukunemba ukuze kulingane okuqukethwe okude kumemori ye-GPU elinganiselwe.
Izingozi & Guardrails
Amaqiniso akhonjiwe angafaka ngokuthula imibiko, ukugeleza kosekelo, noma imiphumela yocwaningo.
Ukuzwela okusheshayo kungadala imiphumela engahambisani kuzo zonke izicelo ezifanayo.
Idatha yombhalo ebucayi ingase idalulwe uma izilawuli zokufinyelela zibuthakathaka.
Ukuqalisa Umhlahlandlela
Chaza ifomethi yokuphumayo, ithoni, namazinga wekhwalithi ngaphambi kokukhishwa.
Izimpendulo eziyisisekelo ngemithombo ethembekile noma nini lapho ukunemba kubalulekile.
Gcina indawo yokuhlola isibuyekezo somuntu ukuze uthole imiphumela ephezulu.
Landela amaphethini okuhluleka futhi uqeqeshe kabusha imiyalo noma ukuhamba komsebenzi njalo.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the KV Cache quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukuthuthukiswa Kwenqolobane ye-KV
Imibuzo evame ukubuzwa
What is KV Cache?
Inqolobane ye-KV igcina ama-vectors angukhiye kanye nenani i-transformer eseyibalile kumathokheni wangaphambilini, ngakho-ke akudingeki ukuthi iwabuyisele kuwo wonke amagama amasha ewakhayo. Kuyisizathu esisodwa esikhulu esenza umbhalo usheshe - futhi into esemqoka edla inkumbulo yakho ye-GPU phakathi nezingxoxo ezinde.
Igcina ini inqolobane ye-KV?
Inqolobane ye-KV iphethe izihluthulelo kanye nenani lama-vector asuka kumathokheni angaphambili ukuze ukunakwa kukwazi ukuphinda kusetshenziswe esikhundleni sokubala kabusha.
Iyiphi inkinga inqolobane ye-KV eyixazulula ngokuyinhloko?
Ngaphandle kwe-caching, ithokheni entsha ngayinye izodinga ukubalwa kabusha kwe-K/V kuwo wonke amathokheni angaphambilini, okumoshayo; inqolobane yenza izindleko zethokheni ngayinye zilingane.
Iluphi uhlangothi oluphansi lwenqolobane ye-KV?
Inqolobane ikhula ngobude bokulandelana, izendlalelo, namakhanda, ngokuvamile okuba umthengi ovelele wememori ye-GPU ekusetshenzisweni kokuqukethwe okude.
Iyiphi inqubo eyehlisa usayizi wenqolobane ye-KV ngokwabelana ngokhiye namanani kuwo wonke amakhanda okunaka?
Imibuzo ehlanganisiwe kanye nokunakwa kwemibuzo eminingi kuvumela izinhloko zemibuzo eminingi ukwabelana ngamasethi ambalwa okhiye/inani, kunciphe kakhulu inqolobane.
Yiziphi izigaba ezimbili ze-transformer inference ehlobene nenqolobane ye-KV?
Ukugcwalisa kusengaphambili kugcwalisa inqolobane nge-K/V yokwaziswa; i-decode yengeza i-K/V yethokheni entsha isinyathelo ngasinye futhi iphinde ibambe inqolobane.