Context engineering

Decide what goes in the prompt and in what order — the work that has quietly replaced most of what used to be called prompt engineering.

16of 24
≈ 13Min read
5Parts

The prompt is assembled, not written

In a real application almost nothing in the final prompt was typed by a person. It is built on every request out of a system prompt, some tool definitions, a conversation history, several retrieved chunks and the question — and the interesting decisions are about selection and arrangement, not wording.

That is what context engineering is: the code that decides which few thousand tokens the model gets to see.

Position matters, measurably

The "lost in the middle" result: a fact placed at the start or the end of a long context is found reliably, and the same fact in the middle is not. It has held across model generations and it is the single most actionable finding about long prompts.

So: the question last, the most relevant chunk first or last, and the least relevant in the middle — where, if it has to be forgotten, it does the least harm.

text
accuracy on "find the fact in this context"

  ▲
  │ ███                                     ███
  │ ███ ███                         ███ ███ ███
  │ ███ ███ ███             ███ ███ ███ ███ ███
  └──────────────────────────────────────────────▶
    start          middle of the context        end

Less is more, and that is measurable too

Adding plausible-but-irrelevant passages to a context lowers accuracy. Not "wastes tokens" — lowers accuracy, because the model has to decide which of several similar-looking passages to believe and it sometimes chooses wrongly.

Which means the top-k in your retriever is a quality parameter, not a generosity dial. Five good chunks beat twenty mixed ones, and the only way to find your number is to measure it — that is the next lesson.

Compression, when you genuinely need it

Structured state is under-used. An agent does not need a transcript of what it has done; it needs a few fields saying what is known. Replacing three thousand tokens of history with a two-hundred-token object usually improves behaviour as well as cost, because the important facts stop competing with the chatter.

TechniqueUse when
Extractive selectionKeep only the sentences that match the question. Cheap, lossless for what it keeps
Summarise old turnsA long conversation. One model call, and the summary can drift
Structured stateReplace prose history with a small object: what is decided, what is pending. Far cheaper and more reliable
Drop the middlePin the first and last turns. Free, and usually enough

Assemble it in one place

One function, one budget, and a record of what it had to leave out. That last part matters: a system that silently drops the retrieved context and answers anyway is indistinguishable from one that is hallucinating, and you will not be able to tell them apart from the logs.

Log the assembled prompt, or at least its parts and sizes, for every request. When someone asks "why did it say that", the prompt is the answer, and reconstructing it afterwards from four services is not realistic.

js
function buildContext({ system, tools, history, retrieved, question }, budget) {
  const parts = [
    { name: "system",    text: system,               priority: 1 },
    { name: "retrieved", text: format(retrieved),    priority: 2 },
    { name: "history",   text: format(history),      priority: 3 },
    { name: "question",  text: question,             priority: 1 },
  ];

  /* drop by priority until it fits, and record what was dropped */
  const { kept, dropped } = fit(parts, budget);

  return { prompt: kept.map((p) => p.text).join("\n\n"), dropped };
}

Practice

Next →Evaluation