Context Engineering

Context engineering is the practice of deciding what information enters a model's context window, in what form, and in what order, for a given step of a task. It is distinct from prompt engineering, which shapes the instruction itself: prompt engineering asks how to phrase the request, context engineering asks which of the ten thousand available tokens deserve the two thousand slots sitting next to it.

The term was popularized in mid-2025, with Andrej Karpathy and Shopify's Tobi Lütke among those arguing that "prompt engineering" undersold what production LLM systems actually do. The distinction acquires teeth as soon as an application is more than a chat box. An agent's context on any given turn is assembled from a system prompt, tool definitions, conversation history, tool results, and retrieved documents, and each of those is a budget decision rather than a given. Longer is not automatically better: Chroma's July 2025 research on context rot documented performance degrading as input length grows, on tasks well inside the advertised window, so filling a million-token context because the model allows it is a reliability choice with a cost. Context engineering covers what to include, what to summarize, what to evict, and how to order what remains.

Most of that context is retrieved, so the quality of what comes back from the web sets the ceiling on everything else. In a typical agent the largest single contributor of tokens is the output of a search call or a page fetch. If extraction returns a page with navigation, cookie banner, and related-articles rail intact, a third of the budget went to furniture. If it returns Markdown with headings preserved, the same information costs roughly half as many tokens and stays structured enough to trim by section. If the search API returns ten shallow snippets when the answer needed one full page, no amount of prompt tuning recovers it. Firecrawl and Jina AI's Reader exist to make that first conversion cheap and consistent, Tavily returns pre-scored passages sized for agent context, and Exa can return full page contents alongside results, which moves the trimming decision to you rather than making it for you.

For a team building an AI product, context engineering is a budgeting exercise you can measure. Instrument token counts by source: how many go to tool definitions, how many to conversation history, how many to retrieved content. Teams that do this usually find the tool schemas or an unpruned history eating a surprising share, and the retrieved documents that were supposed to be the point getting the leftovers. The standard remedies are compaction (summarize old turns rather than carrying them verbatim), just-in-time retrieval (fetch a document when the agent needs it instead of front-loading everything it might need), and keeping structured notes outside the window so state does not have to live in it. All three are cheaper than moving to a larger model, and all three depend on the quality of what comes back from the web in the first place.