Give long-running agents a context budget

A long-running agent can accumulate instructions, documents, screenshots, tool results, and failed attempts. Even when the model accepts the full history, the application still needs a policy for what belongs in working context. The objective is not to use the largest possible window. It is to provide the information needed for the next correct decision.

Anthropic’s context-window documentation explains that inputs and generated output share the available window, and that cached input still occupies context. Its prompt-caching guide describes a separate optimization for reusing stable prompt prefixes. Those distinctions are useful regardless of which application you are building.

Give each piece of information a job

Divide the context into four practical groups: durable instructions, current task state, relevant evidence, and incidental history. Durable instructions define boundaries and output requirements. Current state records what has been completed and what remains. Evidence supports the next decision. Incidental history includes obsolete searches and attempts that no longer change the plan.

This classification is an editorial design aid, not a provider API requirement. Its value is that each group can have a different retention rule. A permission boundary should not disappear just because an old terminal result becomes irrelevant.

Preserve decisions, not every transcript detail

Consider an assistant preparing a comparison of three software services. It may inspect many pages, but the final decision state needs the user’s criteria, verified facts, source links, unresolved questions, and rejected options that still matter. Repeating every unsuccessful search result consumes attention without necessarily improving the comparison.

When compressing history, preserve the distinction between a confirmed fact, a tentative inference, and a next step. A useful state note might say: “The retention setting is verified for plan A; plan B remains unconfirmed.” A vague note saying “privacy checked” can erase the exact uncertainty the next step needs to resolve.

Keep source material recoverable

A summary should not be the only surviving evidence for an important statement. Store references to the original document or task artifact, subject to the application’s access and retention rules. Let the agent retrieve the relevant passage again when a decision depends on details omitted from the summary.

For changing web content, record when the source was checked. For internal files, record a version or revision where possible. This makes a later contradiction easier to investigate and avoids treating an old summary as a permanently current source.

Budget for the next turn

Monitor growth before the application reaches a hard limit. Reserve room for the expected tool response and the answer the user needs. A document-analysis task and a short classification request can require different reserves; one global threshold may be unnecessarily restrictive for one and insufficient for the other.

Use provider usage reporting and token-counting facilities where available. Treat estimates as planning inputs rather than exact billing statements. Record large contributors separately so that an oversized tool schema is not mistaken for an unavoidable property of the user’s document.

Optimize the right bottleneck

If repeated stable instructions dominate processing cost, investigate caching. If tool outputs dominate the window, consider narrower retrieval or structured extraction. If a task repeatedly revisits the same decisions, improve its state representation. If network payload is the problem, evaluate transport features separately.

Do not make all of these changes at once. Select a representative task, change one mechanism, and compare the accepted result, elapsed time, and usage. A lower token count is not a success if the agent forgets a constraint and requires the user to repair the work.

Test what happens after compression

Include tasks that resume after a summary or context transition. Ask whether the agent still respects a prohibition, remembers an unresolved issue, and can recover the evidence for a claim. Also test a corrected instruction: the latest correction must survive while the superseded instruction must not silently regain authority.

A context budget is a quality tool as much as a cost tool. The most useful working memory is focused, recoverable, and explicit about uncertainty. Build that policy around the decisions your agent makes, then use provider features to implement and measure it.