KV Cache Ingantawa
Cache na KV yana adana maɓallai da ƙimar mai taswira ya riga ya ƙididdige shi don haka baya sake yin aiki ga kowane sabon alama - amma yana iya yin balloon zuwa gigabytes.
Dubawa
Inganta cache na KV yana raguwa kuma yana sarrafa wannan ƙwaƙwalwar ajiya don haka samfuran suna hidimar mahallin da ya fi tsayi ga ƙarin masu amfani a lokaci guda.
Zurfafa nutsewa
A cikin na'ura mai canzawa, kowane sabon alama yana halartar duk alamun da suka gabata ta hanyar maɓallan hankali (K) da ƙima (V). Sake lissafin K da V gabaɗayan jeri a kowane mataki zai zama mai ƙididdigewa da almubazzaranci, don haka samfura suna adana su: cache KV. The downside size ne. Cache yana girma a layi tare da tsayin jeri, girman tsari, yadudduka, da kawunansu, don haka buƙatun yanayi mai tsawo na iya cinye ƙarin ƙwaƙwalwar GPU fiye da ma'aunin ƙirar da kansu. Ingantawa yana magance wannan daga kusurwoyi da yawa: ƙwaƙwalwar ajiyar shafi (vLLM's PagedAttention) yana adana cache a cikin ɓangarorin da ba su da alaƙa don kawar da rarrabuwa da ba da damar rabawa; ƙididdigewa yana adana K da V a cikin 8-bit ko 4-bit; da canje-canje na gine-gine kamar Rukuni-Tambayoyi Hankali (GQA) da Multi-Query Attention (MQA) sun bar shugabannin tambaya da yawa su raba ƴan maɓalli/masu ƙima, yanke girman cache a tushen.
Fahimtar Fasaha
PagedAttention yana ɗaukar bayanan ƙwaƙwalwar ajiya daga tsarin aiki: cache yana rayuwa a cikin ƙayyadaddun tubalan da aka tsara ta hanyar tebur, don haka buƙatun suna amfani da tubalan da suke buƙata kawai da kuma prefixes iri ɗaya (kamar tsarin tsarin da aka raba) na iya nuna tubalan iri ɗaya. Multi-head Latent Attention (MLA), wanda aka yi amfani da shi a cikin ƙirar DeepSeek, yana matsawa K da V cikin ƙaramin sikelin ɓoyayyiyar ɓoyayyiyar ɓoyayyen ɓoyayyen ɓoyayyen ɓoyayyen ɓoyayyen ɓoyayyen ɓoyayyen ɓoyayyen ɓoyayyiya, yana yanke ƙwaƙwalwar ajiya da ƙarfi yayin kiyaye daidaito.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar KV Cache ingantawa
Kamar yadda mahallin windows ke shimfiɗa zuwa ɗaruruwan dubbai ko miliyoyin alamu, ma'ajin KV ya zama babban farashin hidima. Yi tsammanin matsananciyar cache da korar (saukar da ƙananan alamun kulawa), raba prefix na buƙatu azaman tsoho, saukar da cache mai sanyi zuwa CPU ko NVMe, da gine-gine kamar MLA da GQA sun zama daidaitattun. Gudanar da cache zai ƙara kama da cikakken tsarin ƙwaƙwalwar ajiya tare da tiers da prefetching mai wayo.
Aiwatar da Gaskiyar Duniya
vLLM's Paged Hankali yana ba da yawancin zaman taɗi na lokaci ɗaya ta hanyar tattara abubuwan KV ba tare da rarrabuwar ƙwaƙwalwa ba.
Hankalin Rukuni-Tambaya a cikin ƙirar Llama yana rage girman cache na KV don haka mafi tsayin mahallin ya dace da ƙwaƙwalwar GPU
Ƙididdiga cache na KV zuwa 8-bit (KV8) don kusan rabin ƙwaƙwalwar ajiyar cache yayin taƙaitaccen takaddun bayanai.
Prefix caching wanda ke sake amfani da tubalan KV na tsarin raba gardama a cikin dubunnan buƙatun API
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the KV Cache Optimization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
KV cache
Tambayoyin da ake yawan yi
Menene KV Cache Optimization?
Cache na KV yana adana maɓallai da ƙimar mai taswira ya riga ya ƙididdige shi don haka baya sake yin aiki ga kowane sabon alama - amma yana iya yin balloon zuwa gigabytes. KV cache ingantawa yana raguwa kuma yana sarrafa wannan ƙwaƙwalwar don haka samfura suna ba da damar yanayi mai tsawo ga ƙarin masu amfani a lokaci ɗaya.
Menene KV cache ke adanawa kuma me yasa?
Maɓallai da ƙima daga alamun farko suna guje wa sake yin lissafin hankali akan kowane sabon alama, adana lokaci.
Me yasa cache na KV zai iya zama matsalar ƙwaƙwalwa?
Girman ma'auni tare da tsayin mahallin da daidaituwa, don haka dogayen buƙatun na iya amfani da ɗimbin adadin ƙwaƙwalwar GPU.
Wane tsarin tsarin aiki PagedAttention ya aro?
PagedAttention yana adana cache a cikin ƙayyadaddun ƙayyadaddun ƙayyadaddun ƙayyadaddun ƙayyadaddun ƙayyadaddun ƙayyadaddun taswira ta hanyar tebur, kamar fage na OS.
Ta yaya Tambayoyin Rukuni da Hankalin Tambayoyi da yawa ke rage girman cache na KV?
Raba kawunan maɓalli/daraja a kan shugabannin tambaya da yawa yana nufin ƙarancin K da V vectors don adanawa.
Menene fa'ida ɗaya na raba prefix a cikin cache KV?
Tsarin tsarin da aka raba yana samar da shigarwar KV iri ɗaya, don haka buƙatun da yawa na iya nuna tubalan ɗaya maimakon kwafi su.