What does KV Cache mean? Definition | AI Understanding

Definition

Stored key and value tensors from previous tokens that let transformers generate new tokens without recomputing past attention.

Related terms

Continual Learning

Training approaches that let a model keep learning from new data without forgetting prior knowledge.

A decoding strategy that samples only from the k most likely next tokens.

Unsupervised Learning

Learning patterns from unlabeled data without explicit target outputs.

Sliding Window Attention

An attention pattern where each token attends only to a fixed-size window of nearby tokens to reduce compute.

A language-agnostic tokenizer that learns subword units directly from raw text without pre-splitting on whitespace.

Presence Penalty

A decoding setting that reduces the probability of tokens that have appeared at all, encouraging new topics.

Learn more in our free guides

KV Cache KV Cache Optimization

See also

Speculative Decoding MCP (Model Context Protocol)Zero Data Retention Agentic Loop Usage-Based Billing ReAct Trust Calibration Tree of Thoughts

Browse the full AI Glossary Explore all free AI guides