在本页4 分钟阅读
概述
The main ones are dropping old turns, summarizing earlier parts, compacting the whole history into a condensed brief and clearing old tool outputs. It matters because a language model has no memory between calls beyond what the application resends. Once the history outgrows the window, something must be cut, and what gets cut decides whether the assistant stays coherent.
深入探讨
Most chat APIs are stateless. Every request resends the conversation so far, and the model reads it from scratch. Tokens, cost and latency therefore grow as the session gets longer, until the history hits the context limit and the request fails or gets truncated. Quality can drop even before that point, because relevant details are buried among old material. The simplest strategy is truncation, usually as a sliding window: keep the system prompt and the most recent turns, and drop the rest. It is cheap and predictable, but it forgets early facts, such as constraints the user stated at the start. A common refinement is to pin important messages so they are never dropped. Rolling summarization replaces older turns with a model-written summary that is updated as the conversation grows. It keeps the gist in far fewer tokens. The cost is that summaries lose detail and can introduce errors, and those errors compound when a summary is itself summarized again. Compaction is a more deliberate version used in agent tools. When token usage crosses a threshold, the system asks the model to write a structured handoff covering the goal, progress, decisions, open problems and key file names. It then starts a fresh context with that handoff plus the most recent turns. Claude Code, for example, has a compact command and can also compact automatically near the limit. Tool-result clearing targets the biggest source of bloat in agent sessions: large outputs from searches, file reads and commands that were useful once. Swapping them for short placeholders frees space and still records that the call happened. External memory stores facts in a file or database and retrieves them when relevant, so they survive any trimming. A common misconception is that the model remembers earlier parts of a chat on its own. Any continuity comes from what the application chooses to resend or retrieve.
战略影响
成本与预算
多年来，架构决策决定着性能和运营成本。
更清晰的判决
技术教育帮助团队选择正确的堆栈，而不仅仅是最新的堆栈。
质量控制
更好的工程选择可以减少生产中的可靠性事故。
The Future of Context Compaction and Conversation History Management
Providers have started building history management into their platforms. Features include server-side conversation state, automatic compaction and context editing that clears old tool results. This cuts boilerplate, but developers still have to decide what must never be forgotten. Research continues on models that use long contexts more reliably and on memory systems that store and retrieve facts across sessions. Compaction will probably become a default behavior in agent frameworks, but the tradeoff between detail and space will not go away. No method keeps everything for free, so testing what gets lost will remain part of the job.
现实世界的实施
A chatbot keeps the system prompt plus the last 20 messages and drops anything older. This is cheap, but it forgets the name and budget the user gave in the first message.
Every 10 turns, a smaller model summarizes the older part of the conversation into one paragraph that stays at the top of the context. This is a rolling summary.
A coding agent close to its limit writes a handoff: the goal, files changed, open bugs and decisions made. It then starts a fresh context containing that handoff and the most recent files.
An agent replaces search results older than a few turns with a short placeholder, such as '[web search output cleared: 12 results]'. Raw results are rarely needed again once the agent has acted on them.
风险与防护栏
优化一项基准测试可以隐藏更广泛的系统弱点。
基础设施和维护成本常常被低估。
随着系统变得更加复杂，安全性和可观察性差距可能会扩大。
实施路线图
在实施之前定义延迟、质量和成本目标。
在实际负载和数据条件下进行基准测试。
仪器监控错误、漂移和用户影响。
在扩展之前准备回滚和事件响应路径。
不断探索
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Context Compaction and Conversation History Management quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
常见问题
What is Context Compaction and Conversation History Management?
Conversation history management is the set of techniques that keep a long chat or agent session under a model's context limit. The main ones are dropping old turns, summarizing earlier parts, compacting the whole history into a condensed brief and clearing old tool outputs. It matters because a language model has no memory between calls beyond what the application resends. Once the history outgrows the window, something must be cut, and what gets cut decides whether the assistant stays coherent.
Why do long chat sessions become steadily more expensive?
Stateless APIs process the whole history on each call. As the history grows, so do the tokens, cost and latency of every request.
What is the main weakness of sliding-window truncation?
Keeping only recent turns is cheap and predictable, but anything that falls out of the window is gone unless it was pinned.
Which risk is specific to rolling summarization?
Each round of summarization is lossy. A mistake in one summary carries into the next, so errors can build up over a long session.
What does compaction typically produce in agent tools?
Compaction condenses the session into a deliberate brief and starts a new context with it and the most recent turns. The agent can continue without the full history.
Why is clearing old tool results so effective in agent sessions?
Tool outputs can be very large and are often useful only once. Swapping them for placeholders frees a lot of space and keeps a record that the call happened.
继续学习
相关指南
为此主题精选的更多指南