KV Cache
Kaydka KV waxa uu kaydiyaa furaha iyo qiimaha vectors uu taransfooriyuhu mar hore u xisaabiyay calaamadihii hore, markaa maaha in dib loogu xisaabiyo erey kasta oo cusub oo uu soo saaro.
Dulmar
It is the single biggest reason text generation is fast — and the main thing eating your GPU memory during long conversations.
quusid qoto dheer
Transformers ayaa soo saara qoraal hal marba calaamad, iyo lakab kasta oo cusub ee dareenka calaamaduhu waxa uu u baahan yahay in la barbardhigo calaamad kasta oo hore. Farsamaynta feejignaanta ayaa calaamad kasta u rogta su'aal, fure, iyo ra'yiga qiimaha leh. Haddii aan la kaydin, soo saarista calaamadda lambarka 1,000 waxay la macno tahay dib u xisaabinta furayaasha iyo qiyamka dhammaan 999 calaamadihii hore ee tallaabo kasta - afar-geesood, shaqo khasaara. Khasnadda KV-gu waxa ay badbaadisaa furayaashaas iyo kuwa qiimaha leh ka dib marka ugu horraysa la xisaabiyo oo dib loo isticmaalo, markaa tillaabo kasta oo cusub waxay kaliya u xisaabisaa vectors calaamadda ugu cusub oo waxay ka qaybgashaa kaydinta kaydsan. Tani waxay hoos u dhigeysaa qiimaha calaamad kasta laga bilaabo isbarbardhigga dhererka isku xiga ilaa qiyaas joogto ah. Ka-ganacsigu waa xusuusta: kaydku wuxuu u koraa si toos ah dhererka macnaha guud, tirada lakabyada, iyo madax dareenka, oo inta badan noqda macaamiisha xusuusta ugu badan ee adeega muddada-dheer.
Aragtida Farsamada
Inta lagu jiro marxaladda 'hore-buuxinta' qaabku wuxuu socodsiiyaa dhammaan degdegga wuxuuna buuxiyaa kaydka; Inta lagu jiro 'decode' waxay ku dhejinaysaa hal calaamad K/V tallaabo kasta oo dib ayay u akhridaa. Cabbirrada kaydinta sida 2 (K iyo V) × lakabyo × madax × madax_dim × isku xigxiga_dhererka × dufcaddii, ee saxda ah ee la doortay. Si tan loo dajiyo, moodooyinka casriga ahi waxay adeegsadaan feejignaan kooxaysan ama waydiimo badan si ay u wadaagaan furayaasha/qiyamka madaxa, iyo u adeegida nidaamyada sida vLLM waxay isticmaalaan PagedAttention si ay ugu qoondeeyaan kaydka baloogyada aan isku xidhnayn, jarida tafaraaruqa iyo qashinka.
Saamaynta Istiraatijiyadeed
Xawaaraha iyo miisaanka
Socodka shaqada luqaddu si dhakhso leh ayay u socon kartaa iyada oo aan la hurayn joogteynta.
Helitaanka iyo gaarsiinta
Waxay balaadhisaa gelitaanka luqadaha iyo qaababka isgaarsiinta.
Go'aamo cad
Kooxuhu waxay waqti badan ku qaadan karaan xukunka halka otomaatiggu uu qabanayo ku celcelinta.
Mustaqbalka KV Cache
Sida daaqadaha macnaha guud ay u fidiyaan boqollaal kun oo calaamado ah, khasnadda KV waxay noqotaa dhexda dhexe, sidaa darteed hal-abuurnimadu waa adag tahay: cabbirka kaydinta 8 ama 4 bits, siyaasadaha ka saarida ee hoos u dhigaya calaamado muhiim ah, wadaagis horgale codsi ah, iyo rarista CPU ama disk. Isbedelka naqshadaha sida feejignaanta daahsoon ee madax-badan ayaa isku cadaadiya kaydka laftiisa. Filo naqshadaynta sii socota ee kala duwanaanshaha dareenka iyo nidaamyada xusuusta ee loogu talagalay in loogu adeego xaaladaha aadka u dheer si raqiis ah iyo waxsoosaar sare.
Dhaqangelinta Adduunka-dhabta ah
Dedejinta chatbot-ka jawaabaha iyadoo dib loo isticmaalayo furayaasha/qiyamka la kaydiyay ee taariikhda wada sheekeysiga halkii aad dib u habayn lahayd mar kasta.
Horgale kaydinta wadaaga khasnadda nidaam dheer oo degdeg ah isticmaalayaasha badan, dhimista qiimaha iyo daahitaanka.
vLLM's Paged Feejignaanta maaraynta kaydka KV ee baloogyada si loogu adeego codsiyo badan oo isku mid ah hal GPU si hufan.
Tirinta khasnadda KV si loo yareeyo saxnaanta si loogu waafajiyo xaaladaha dhaadheer ee xusuusta GPU xaddidan.
Khatarta & Dariiqyada Ilaalada
Xaqiiqooyinka dhalanteed waxay si deggan u geli karaan warbixinnada, taageerada socodka, ama natiijooyinka cilmi-baarista.
Dareenka degdega ahi wuxuu abuuri karaa natiijooyin aan iswaafaqayn codsiyada la midka ah.
Xogta qoraalka xasaasiga ah ayaa laga yaabaa in la kashifo haddii kontaroolada gelitaanka ay daciif yihiin.
Qorshe Hawleedka Dhaqangelinta
Qeex qaabka wax soo saarka, codka, iyo heerarka tayada ka hor inta aan la baahin.
Jawaabaha salka ku haya ilo lagu kalsoon yahay mar kasta oo saxnidu ay muhiim tahay.
Hayso isbaarada dib u eegista bini aadamka ee wax soo saarka sare.
Lasoco qaababka guuldarada oo dib u leyli dardargelinta ama socodka shaqada si joogto ah.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the KV Cache quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Hagaajinta KV Cache
Su'aalaha soo noqnoqda
What is KV Cache?
Kaydka KV waxa uu kaydiyaa furaha iyo qiimaha vectors uu taransfooriyuhu mar hore u xisaabiyay calaamadihii hore, markaa maaha in dib loogu xisaabiyo erey kasta oo cusub oo uu soo saaro. Waa sababta kaliya ee ugu weyn ee jiilka qoraalku u degdego - iyo waxa ugu weyn ee cunista xusuusta GPU-gaaga inta lagu jiro wadahadalka dheer.
Muxuu kaydiyaa kaydka KV?
Kaydka KV wuxuu hayaa furaha iyo qiyamka vectors ee calaamadihii hore si dareenka dib loogu isticmaalo halkii dib loo xisaabin lahaa.
Dhibaato noocee ah ayuu ugu horrayn xalliyaa kaydka KV?
Haddii aan la kaydin, calaamad kasta oo cusub waxay u baahan doontaa dib u xisaabinta K/V calaamad kasta oo hore, taas oo ah khasaare; khasnadda ayaa ka dhigaysa halkii-token qiime qiyaas ahaan joogto ah.
Waa maxay cilladda ugu weyn ee kaydka KV?
Cache-ku wuxuu ku koraa dhererka isku xigxiga, lakabyada, iyo madax, inta badana noqda macmiilka ugu sarreeya xusuusta GPU-da adeegga muddada-dheer.
Farsamokee ayaa yaraynaysa cabbirka kaydka KV iyadoo la wadaagayo furayaasha iyo qiyamka madaxa dareenka?
Su'aalo-kooxeed iyo feejignaan su'aalo badan ayaa u oggolaanaya madax badan oo weydiimo ah inay wadaagaan furayaal/qiimo tiro yar, iyagoo si weyn u yareynaya kaydka.
Waa maxay labada weji ee soo jeedinta transformer marka loo eego kaydka KV?
Buuxinta hordhaca ah waxay ka buuxinaysaa kaydka K/V ee degdega ah; Go'aaminta waxay ku dhejisaa hal calaamad oo cusub oo K/V tallaabo kasta oo waxay dib ugu celisaa kaydka.