The prompt is not the whole system

A language model does not arrive knowing which customer is involved, what the user is allowed to see, whether an invoice has been paid, what happened in the previous support case or which business rule applies today. It sees the information the application supplies at inference time. If that information is incomplete, stale, excessive or poorly structured, even a capable model can produce an unhelpful answer.

Context engineering is the work of deciding what information the system needs, where it comes from, how it is represented, when it should be retrieved and how its quality is evaluated. It includes far more than searching documents. Business context often lives in application state, relationships, events, permissions and tool results. The objective is not to fill the context window. It is to assemble the smallest useful view of the current job.

Begin with structured application state

Most business systems already contain valuable structure. A customer has an identifier. An invoice has an amount, due date and status. A booking belongs to a location and time. A support case has an owner and current state. These fields should not be flattened into prose and rediscovered by a model. They should remain typed, validated and queryable.

Structured state gives the agent reliable anchors. It also lets application code enforce invariants before and after inference. The model may propose that a case needs escalation, but the application decides whether the case is already closed, whether the current user can view it and which transitions are valid. Context engineering starts by respecting the product domain rather than treating the whole business as documents.

Assemble context for a task

The right context depends on the job. A payment follow-up needs account and communication history. A support response needs the current issue, product state and approved knowledge. A sales summary needs the opportunity, participants, previous commitments and next scheduled event. Sending every available record to every task increases cost and can make the result worse by giving irrelevant information equal weight.

A context service can accept a typed task and identity, load the core record, follow allowed relationships, retrieve relevant knowledge and produce a documented context package. Each item should carry provenance and time information where possible. The agent should be able to distinguish a current account status from a two-year-old note and an approved policy from an informal observation.

task + identity → core state → related entities → relevant retrieval → policy → tool observations → bounded context package

Retrieval is one component

Semantic retrieval is useful when wording varies and relevant knowledge cannot be found with an exact key. It can surface a policy paragraph, previous case or product note based on meaning. But similarity is not the same as authority or relevance. A nearby document may be outdated, apply to another customer or describe an exception rather than the normal rule.

Retrieval therefore needs filters, ranking and evidence. Metadata can restrict results by customer, product, date, status, jurisdiction or approval state. Hybrid search can combine semantic similarity with keywords and structured fields. Reranking can improve order, while citations allow the application and reviewer to see what supported the answer. Retrieval evaluation should test whether the necessary source appears and whether irrelevant sources are excluded.

Memory should have a reason to exist

Memory is context retained across tasks or conversations. It can include explicit business state, meaningful events, user preferences or summaries of observations. The system should not store every interaction indefinitely and call that intelligence. More memory can increase privacy exposure, retrieval noise and the chance that an old assumption is treated as a current fact.

A memory policy should explain what is recorded, why it may change future work, who can correct or delete it and how long it remains useful. Conversation memory may last only for a session. A commitment to call a customer next Tuesday belongs in structured state. An important event may become episodic memory. A repeated operational pattern may be summarised as an observation after enough evidence exists. These forms have different retention and trust levels.

  • Conversation memory for the current exchange
  • Structured business state for durable facts and commitments
  • Episodic memory for important events
  • Observational memory for selected recurring patterns
  • Semantic recall for approved knowledge and similar examples

Relationships create business meaning

A customer is connected to a company, invoices, bookings, emails, support cases, tasks and previous actions. Those relationships often matter more than the text inside a single record. Following them allows the system to build a coherent task view: this invoice belongs to this account, the last email discussed this dispute, and the open task is owned by this colleague.

People sometimes use the phrase “context graph” for this connected model. The implementation may use a graph database, relational joins, document references, search indexes or a combination. The important design is the entities, relationships, time and permissions—not a particular database logo. A graph database should be selected when its traversal and modelling benefits justify the operational cost.

Tools add live observations

Context is not always assembled before the agent begins. A tool call may retrieve live payment state, calendar availability or a current CRM record. The result becomes part of the decision context. Tools need typed inputs, bounded outputs, timeouts and clear error states so the agent can distinguish “no matching invoice” from “the payment service did not respond.”

Read tools and write tools should be treated differently. Reading an approved record may be low risk. Sending an email, creating a refund or changing a contract state can have external consequences. The context package should include the current identity and permissions, while policy decides whether a proposed tool call may run, needs approval or must be rejected.

Context has a budget

Every additional piece of context adds tokens, latency and processing cost. Large contexts can also reduce attention to the most relevant evidence. A production system needs selection and compression strategies rather than assuming the model window is free space. The budget should be allocated by task value and evidence importance.

Core structured state may be rendered in a compact format. Documents can be chunked around meaningful sections and retrieved with provenance. Long histories can be summarised, but the original events should remain available for audit where required. Tool outputs can be normalised into the fields the task uses. Measurements should show whether compression changes accuracy or removes important exceptions.

Permissions are part of context

The same question can have different valid answers depending on who asks it. A finance manager may view account balances that a support agent cannot. A regional operator may see one group of locations. A website visitor should not gain access to internal project material because the retrieval system found it semantically relevant.

Authorization should happen before information enters the model context, not only before the final answer is displayed. Tool access should propagate identity and scope. Logs and traces need their own privacy controls because they may contain the assembled context. This is why context engineering belongs inside application and security architecture, not as a prompt-writing activity performed after the product is built.

Evaluate the context separately from the answer

When an agent gives a wrong answer, the model may not be the cause. The correct record may have been missing, an outdated policy may have ranked first, a relationship may have been broken or a tool may have returned an ambiguous error. Evaluation should make those stages visible.

Create representative tasks with expected sources and facts. Measure whether core state was correct, whether retrieval included supporting evidence, whether forbidden information was excluded and whether the assembled package fit the task. Then evaluate the reasoning and final action. Separating the stages turns vague dissatisfaction into an engineering problem that can be diagnosed and improved.

  • State correctness and freshness
  • Retrieval recall and precision
  • Permission and data-boundary adherence
  • Citation and provenance coverage
  • Context size, latency and cost
  • Downstream decision and task quality

A practical first implementation

Choose one task and document how a capable person gathers information for it. Mark each source as structured state, related entity, knowledge, memory or live tool result. Identify which items are required, which are optional and which the current user may not access. Build a context package that can be inspected before connecting a model.

Test that package against real examples. Only then add reasoning, structured output and an approval or action path. This sequence may feel slower than writing a large prompt, but it exposes the work that determines production quality. A strong model cannot compensate consistently for missing business state, unclear permissions or the wrong evidence.

Context engineering is product engineering

The best context system is not a hidden technical layer. It affects what users can inspect, how they correct mistakes, why the software suggests an action and what appears in an audit. Product interfaces should show meaningful sources and current state without exposing raw model internals. Operators need tools to repair relationships, update approved knowledge and understand why context was selected.

When software can reason and act, the information supplied at decision time becomes part of the product’s authority. Designing that information carefully is how an agent moves from a convincing conversation to a useful business system. Context engineering is the bridge between a general model and the specific, accountable work the organisation needs done.