Skip to content
Recent Articles
2/5 - The Working Set, Not the Window: Context Engineering for Agents
8 min · readRECENT ARTICLES

2/5 - The Working Set, Not the Window: Context Engineering for Agents

#Context Engineering #RAG #Agent Memory #Prompt Caching
Abhishek Kumar
Abhishek Kumar

The context window is a capacity limit, not a strategy. An agent can technically accept a large number of tokens and still perform worse because the useful fact is buried among stale tool output, duplicated documents, old plans, and irrelevant conversation. Context engineering is the work of deciding what the model should see on this turn, in what order, at what fidelity, and for how long.

That makes context a working-set problem. An operating system does not put every byte on a CPU register because registers are fast. It selects a small working set and moves data through a hierarchy. Agent context benefits from the same discipline: gather broadly enough to avoid missing a critical fact, then rank, compress, and place information so the next decision is grounded.

The mental model: context is assembled per turn

A useful context packet contains more than the user prompt. It can include retrieved documents, durable preferences, recent observations, tool results, an active plan, policy, and the output contract. These sources have different authority and freshness. A customer record should not be treated like an old summary; a tool error should not be mistaken for a successful observation; and untrusted text should never gain instruction priority merely because it was retrieved.

Context engineering builds a working setContext engineering treats model input as a finite, attributed working set rather than an ever-growing transcript.
Context engineering builds a working set
Gather candidate information, rank it by relevance and authority, then pack a bounded context for one model turn.
1 · Candidate sourcesQuery, retrieved chunks, durable memory, recent turns, tool outputs, policy, and task state.
2 · Selection and rankingScore relevance, authority, freshness, diversity, and expected value; remove duplicates and stale branches.
3 · Packing and placementReserve space for instructions and output; put critical facts where the model can reliably use them.
4 · Evidence and expiryAttach provenance, timestamps, and source IDs; expire or revalidate context as the run changes.
A larger window increases capacity, but a better working set improves signal-to-noise, cost, latency, and recoverability.
Context engineering treats model input as a finite, attributed working set rather than an ever-growing transcript.

Description: The pipeline has two separate responsibilities. Selection decides what belongs. Packing decides how much of it can fit and how it should be represented. A summary can be useful for navigation but unsafe as the only source for a precise decision; retain a pointer to the original evidence when the task demands verification.

Retrieval is not context engineering by itself

Retrieval finds candidates. Context engineering decides whether candidates should enter the request. A vector search result may be semantically similar and still be outdated, low-authority, duplicated, or from the wrong tenant. Use metadata filters before semantic ranking when access scope matters. Rerank where the value justifies the latency, and keep the selected source IDs in the trace so an operator can reconstruct what the model saw.

The same distinction applies to memory. A durable memory store should not be a dumping ground for every model output. Write memories with an owner, a scope, a timestamp, a confidence or evidence pointer, and an update rule. A preference that was true for one project may be wrong for another. Memory should reduce repeated work without silently changing the user’s authority or policy.

The context lifecycle

Context changes after every tool call. A large result may need to be summarized, a failed call may need to be marked as failed rather than blended into the plan, and a new artifact may supersede an old one. The lifecycle is therefore gather, curate, pack, execute, observe, and update. Each turn should be explainable as a projection from durable state plus fresh evidence.

The context lifecycleA context pipeline makes freshness, scope, and token budget explicit at each turn.
The context lifecycle
The working set is refreshed after evidence changes; it is not a transcript that grows forever.
1 · GatherCollect query and candidatesApply scope and tenant rulesRead current task state
2 · CurateDeduplicate sourcesRank relevance and authorityDrop stale branches
3 · PackReserve instruction spaceBudget input and outputPlace evidence with labels
4 · RunCall the model or toolKeep source identityEnforce the deadline
5 · UpdateRecord new observationExpire invalid contextPrepare the next turn
The next turn starts with a new working set, not an unexamined copy of the last one.
A useful invariant
Every included item should answer one of three questions: what is the task, what evidence supports it, or what state must persist to make the next step safe?
A context pipeline makes freshness, scope, and token budget explicit at each turn.

Compression with a receipt

Summaries are lossy. That is not an argument against them; it is a reason to label them honestly. Store the summary alongside source references and a coverage note. If the next decision depends on an exact number, quote, identifier, or policy clause, retrieve the source again instead of trusting a compressed sentence. Compression should remove repetition and low-value detail, not erase the ability to inspect the basis of a decision.

A practical context budget has at least four reservations: fixed instructions, current task and recent state, retrieved evidence, and output headroom. If a request consumes all available input tokens, the model has no space to respond. If output is capped too aggressively, the application may receive a truncated JSON object or an incomplete tool call. Track input, output, cache-read, and cache-write tokens separately because they have different cost and latency behavior.

Failure modes and context rot

Context rot is the gradual loss of usefulness as a long-running run accumulates more material. The first symptom may be that the agent repeats a question. Later it may cite a superseded plan, ignore an important constraint, or spend tokens re-reading huge tool results. Context rot is not always caused by a small window. It can happen in a large window when selection and expiry are weak.

Context failure and recoveryContext failures are information-selection failures as often as they are token-limit failures.
Context failure and recovery
When the working set loses signal, fix ranking, provenance, and expiry before simply raising the token limit.
The working set contains more history than useful evidenceStale, duplicated, or weakly attributed material competes with the current task
CorrectnessOld fact overrides new factThe answer cannot cite its basis
PerformanceInput tokens dominateRetrieval and serialization slow the turn
OperationsMemory grows without boundsOperators cannot reproduce a decision
Containment and recoveryRebuild from durable state, retrieve authoritative evidence, compact with source IDs, and expire the old working set before retrying.
A context reset is safe only when durable state and source references preserve what the next turn needs.
Context failures are information-selection failures as often as they are token-limit failures.

Ordering is a policy decision

Put the current task and output contract where they remain visible, then give the model evidence with clear source labels. Do not assume one universal ordering works for every model or task; test it. For some workflows, recent tool results deserve priority. For others, a short task state and a small set of authoritative documents are more useful than a full transcript. The evaluation set should include cases where the most relevant evidence is old, conflicting, or near the token boundary.

Treat retrieved content as data. A document can contain a prompt injection, an instruction that conflicts with policy, or a false claim. The context builder should mark trust boundaries, and the harness should enforce permissions independently. Context engineering improves grounding; it does not convert external text into authority.

A production recipe

Define a context item schema: source ID, tenant or scope, authority class, created and updated time, content, token estimate, and retention policy. Build a deterministic assembly function that receives the task state and returns the packet plus an explanation of what was omitted. Cache stable prefixes where the model runtime supports it, but invalidate on policy or instruction changes. Keep a small canary set that checks retrieval, packing, stale-fact handling, source attribution, and truncation.

For a long-running coding task, a good working set might include the goal, current plan, files changed since the last checkpoint, the latest test output, relevant repository instructions, and pointers to earlier decisions. It does not need every terminal line from the last six hours. Persist the full evidence outside the model input; pass the smallest representation that preserves the next decision.

Primary references

The takeaway

Context engineering is the discipline of selecting the right evidence for the next decision. It balances relevance, authority, freshness, token budget, latency, and auditability. A model cannot use information it never sees, but showing it everything is not a reliable solution. Build a working set, preserve receipts, and make expiry a feature rather than an emergency cleanup job.

1/5 - The Message That Survives Reality: Prompt Engineering for Working Systems3/5 - The Agent's Operating System: Harness Engineering Without the Hype