Context Engineering Explained: Beyond Prompt Engineering

TL;DR — Key Takeaways

• Context engineering is the discipline of managing the entire information environment an AI model operates in — not just the prompt, but memory, retrieved documents, tool definitions, and conversation state.
• The term crystallized in June 2025 when Shopify CEO Tobi Lütke and AI researcher Andrej Karpathy both endorsed it publicly within six days of each other.
• Gartner declared “context engineering is in, prompt engineering is out” in July 2025, predicting it will be embedded in 80% of AI development tools by 2028 and will raise agentic AI accuracy by at least 30%.
• Chroma Research tested 18 frontier models (July 2025) and found every single one degraded as context length grew — some dropping from 95% to 60% accuracy well before the window limit. This is called context rot.
• LangChain’s four-pillar framework — Write, Select, Compress, Isolate — is the most widely used practical taxonomy for applying context engineering today.
• Prompt engineering is not dead; it is now one layer inside a larger system. The CPU analogy holds: the model is the CPU, the context window is RAM, and context engineering decides what gets loaded.

Introduction

In June 2025, Tobi Lütke, the CEO of Shopify, posted something on X that quietly made a lot of AI engineers feel seen. He said he preferred the term “context engineering” over “prompt engineering” — and described it as “the art of providing all the context for the task to be plausibly solvable by the LLM.” Six days later, Andrej Karpathy, the former OpenAI researcher and Tesla AI Director who has a knack for naming things precisely, amplified the term with a sharper definition: context engineering is “the delicate art and science of filling the context window with just the right information for the next step” (Karpathy, X, June 25, 2025). That post got 14,000 likes and 9,100 bookmarks.

Here’s what’s interesting — neither of them invented something new. They named something practitioners had been wrestling with for years without a clean label.

By July 2025, Gartner had formalized it into analyst briefings. An academic survey by Lingrui Mei et al. (arXiv:2507.13334v1, July 2025) analyzed over 1,300 papers and declared context engineering a distinct discipline. If you’re building anything beyond a one-turn chatbot in 2026, this is the skill the field converged on.

What Is Context Engineering, and Why Does It Matter Now?

Context engineering is the practice of designing the full information pipeline that feeds an AI model — not just the prompt, but every token that enters the context window: system instructions, retrieved documents, conversation history, tool definitions, application state, and the outputs of prior steps.

The 40–60 word extraction block: Context engineering is the discipline of deciding which tokens earn a place in an AI model’s context window at each step. It encompasses system prompts, retrieved documents, tool definitions, conversation history, and agent state — not just the user instruction. According to Andrej Karpathy (2025), it is “the delicate art and science of filling the context window with just the right information for the next step.”

The timing of the concept’s rise is not coincidental. The shift happened when developers started building AI agents — systems that run in loops, call tools, gather state, and make decisions at step 47 carrying the residue of steps 1 through 46. For those systems, the prompt is maybe 10% of the problem. Everything else is context. Walden Yan at Cognition (the team building the Devin autonomous coding agent) was writing about context engineering earlier in 2025, before Lütke and Karpathy’s posts gave it mass adoption. The practice existed. The name caught up.

One important disambiguation: the context window is the technical capacity — how many tokens a model can process at once. Context engineering is the discipline that decides what to put in that window and what to leave out. Getting one confused with the other leads directly to the trap of assuming a bigger window solves the problem. It doesn’t. More on that when we get to context rot.

Why Did Prompt Engineering Stop Being Enough?

Prompt engineering works well for one thing: getting a clean output from a single, well-scoped input. Refining your phrasing, adding few-shot examples, adjusting output format — that still matters and always will. The problem is that production AI systems don’t operate in single turns.

Consider what an AI coding agent is actually doing during a 30-step task. It reads a file, runs a tool, gets back a result, decides what to do next, writes to memory, calls a search function, parses its output, and repeats. By step 15, the accumulated noise — failed tool calls, intermediate reasoning traces, outdated file excerpts — competes with the actually relevant information. The prompt hasn’t changed. The context has gotten poisoned.

Research identified the failure modes precisely. LangChain documented four ways agent context breaks down (LangChain blog, 2025): context poisoning (irrelevant or wrong details slip in), context distraction (critical information gets buried under noise), context confusion (too much unrelated data makes the agent lose focus), and context clash (contradictory inputs cause inconsistent behavior). These are not prompt problems. You can’t fix context clash by rewording your instruction.

There’s also the version-drift issue. Prompt engineering research has documented that semantically equivalent prompts produce measurably different outputs across model updates. Adding a single word like “please” can shift output from concise bullets to verbose paragraphs (various benchmarks, 2024–2025). A prompt tuned for Claude 3.7 may produce different behavior on Claude 4 Sonnet. When your entire reliability strategy depends on prompt wording, every model update becomes a fragility event.

None of this means prompts stopped mattering. The wording of your instruction still determines whether the model understands what you want. But wording is the least of your problems once the surrounding context is a disaster.

Prompt Engineering vs. Context Engineering: A Side-by-Side View

The table makes the relationship clearer than most explanations do. Prompt engineering sits inside context engineering — it handles the instruction-wording layer. Context engineering handles everything around it. Declaring one “dead” because the other exists is the wrong framing, but more on that in a moment.

The Four Pillars: Write, Select, Compress, Isolate

LangChain’s engineering team formalized the most durable practitioner taxonomy for context engineering (blog.langchain.com, 2025). Four strategies, applicable to any agent framework — LangGraph, OpenAI Agents SDK, Anthropic’s Claude agent tooling, or a raw API loop.

Write: Save Context Outside the Window

Agents — like humans — need to take notes. The Write strategy means storing context externally so the model doesn’t have to re-derive it on every turn. Scratchpads, memory files, progress logs, running task plans. The LangChain GitHub repo (github.com/langchain-ai/context_engineering, indexed July 14, 2025) shows this as notebook 1_write_context.ipynb — the first pillar because everything else depends on having a place to offload state.

Practically: if your agent just finished a web search and found three relevant facts, write those facts to a structured memory file. Don’t assume the conversation history will remain clean enough to surface them at step 25.

Select: Pull In the Right Information at the Right Moment

The Select strategy is active retrieval — reading files, querying a vector store, calling a search tool — timed precisely to when that information is needed. Not loaded upfront “just in case.” Retrieval-augmented generation (RAG) is the most common implementation: chunk documents, embed them, retrieve the k-nearest chunks at inference time. But Select goes beyond RAG. It also covers which tool definitions to include in a given step, which parts of conversation history to surface, and which sections of a codebase to read. For a practical walkthrough of setting this up without cloud dependencies, the retrieval-augmented generation for local AI guide covers chunking strategy, embedding choices, and vector store options that work on local hardware.

The key discipline here: every piece of context you don’t select is a win. An empty slot in your context window can’t poison, distract, or confuse.

Compress: Reduce What’s Already in the Window

As a multi-turn agent accumulates conversation history, tool outputs, and retrieved content, the window fills. Compress means summarizing, trimming, or distilling what’s already in context into a shorter representation without losing critical meaning. LangGraph provides message trimming utilities; a simpler approach is asking the model itself to summarize its last N turns before continuing.

A practical rule of thumb from the LangGraph documentation: start monitoring token counts from turn 5 and apply compression from turn 8–10, keeping the last 2–3 turns verbatim and summarizing the rest.

Isolate: Separate Sub-tasks Across Independent Context Windows

Multi-agent architectures implement Isolate by splitting work across sub-agents, each with its own clean context window. A primary orchestrator delegates a bounded task — “search the web for X and return a 200-word summary” — to a sub-agent. The sub-agent runs, does its thing, and returns only its compact output. The orchestrator never sees the sub-agent’s full working context. No contamination, no confusion.

Isolate is the hardest of the four to implement but the most powerful for long-horizon tasks. It’s why Devin and similar autonomous coding agents are architecturally multi-agent even when they appear to be a single assistant.

What Is Context Rot, and Why Should Builders Care?

Here’s the assumption that broke a lot of 2024 agent projects. Model vendors spent that year racing to expand context windows — Claude 3 to 200K tokens, GPT-4 Turbo to 128K. The reasonable inference was: bigger window, more context, better results. And then Chroma Research ran the actual numbers.

In July 2025, Kelly Hong, Anton Troynikov, and Jeff Huber at Chroma published “Context Rot: How Increasing Input Tokens Impacts LLM Performance” (trychroma.com/research/context-rot). They tested 18 frontier models — including GPT-4.1, the Claude 4 family, Gemini 2.5 Pro, and Qwen3 — on retrieval and replication tasks at increasing context lengths. Every single one degraded as input grew. Not most. All 18. Some models held at 95% accuracy, then dropped to 60% once input crossed a threshold — no gradual slide, just a cliff. A model with a 200,000-token window showed serious accuracy loss at 50,000 tokens of input (Chroma Research, July 2025).

The mechanism has a name borrowed from an earlier 2023 study: the “lost-in-the-middle” effect (Liu et al., 2023). Transformer attention naturally weights tokens near the beginning and end of context more heavily than those buried in the middle. When you load 80,000 tokens of retrieved documents into a 128K window, the facts sitting at position 40,000 get attended to far less reliably than the facts at position 500 or position 127,500.

There’s a non-obvious finding from the Chroma research worth highlighting: accuracy degrades faster when the surrounding context is semantically similar to the answer — well-formatted, relevant-looking noise is more dangerous than random noise. Put differently: a neatly organized but irrelevant document that looks like your target answer actively misleads the model more than a messy, clearly-off-topic chunk does. The implication for builders is uncomfortable. Your carefully formatted, professionally written internal knowledge base might be making your retrieval worse if the selection step isn’t precise.

Vendors do not build monitoring products for problems that aren’t costing teams real money. In July 2026, Amazon shipped CloudWatch Coding Agent Insights — a dedicated monitoring tool showing how Claude Code, Codex, and GitHub Copilot agents perform across an organization. Context rot went from academic curiosity to a line item engineering leaders now track.

The Contrarian Take: Prompt Engineering Isn’t Actually Dead

Most guides frame this as a succession story. Prompt engineering had its era; context engineering replaced it. That narrative is clean, quotable, and wrong.

Prompt engineering is not dead. Declaring it dead mistakes a subdiscipline for the whole field. The wording of your system prompt, the structure of your few-shot examples, the way you ask the model to format its reasoning — all of that still directly determines output quality. The instruction layer is one component of the context engineering stack, not an appendage you can drop.

A better analogy: CSS didn’t die when design systems and component libraries emerged. CSS is still there, still doing what it always did, still causing developers grief when specificity goes wrong. But CSS stopped being the whole job. Frontend engineering now encompasses bundlers, component architecture, accessibility, state management — and CSS. Prompt engineering is the CSS of context engineering. Still essential. No longer the ceiling.

The practitioners who get into trouble are the ones who treated prompt engineering as architecture. They tuned their system prompt to 3,000 words, called the job done, and wondered why their agent fell apart at step 10. Context engineering is the recognition that the prompt is one layer, and the surrounding system is the actual job.

Phil Schmid put it directly: “the secret to building truly effective AI agents has less to do with code complexity and everything to do with the quality of context you provide” (philschmid.de, 2025). He didn’t say prompts don’t matter. He said context quality is the controlling variable.

What Context Engineering Looks Like in a Real Project

Concrete example — a coding agent tasked with adding a user authentication feature to an existing codebase.

Naive approach (prompt engineering only): System prompt says “You are an expert software engineer. Add JWT authentication to this app.” Paste the full codebase into context. Wait for output. At a 200K-token window, the model tries to reason over 50,000 lines of code it mostly doesn’t need, hits context rot at the midpoint, and produces authentication code that references functions that don’t exist in the actual files.

Context engineering approach — token budget breakdown:

The total budget is, say, 32,000 tokens for a local model. Here’s how a competent context engineer allocates it:

• System prompt (Write): 600 tokens — concise role description, task constraints, output format. Not 3,000 tokens of elaborate personality.
• Selected files (Select): 8,000 tokens — only the auth-relevant files: the existing user model, the routes file, the middleware folder. Not the entire codebase.
• Tool definitions (Select): 1,200 tokens — only the tools the agent will need for this task (read_file, write_file, run_tests). Not all 20 available tools.
• Working memory / scratchpad (Write): 2,000 tokens — a structured running plan the agent updates each turn with decisions made, files modified, tests passed.
• Conversation history (Compress): 4,000 tokens — last 3 turns verbatim; prior turns summarized to a 200-word digest.
• Retrieved code context (Select): 6,000 tokens — relevant code snippets from vector search on the codebase.
• Available for reasoning and output: ~10,200 tokens — breathing room.

Same task. Same model. Radically different reliability. The model didn’t get smarter — the information environment it was operating in got better-designed.

Does Context Engineering Matter for Local AI Builds?

It matters more, not less. Here’s why.

When you’re running Qwen3:14B on Ollama on a local machine with a 32,000-token context window, you start constrained before the first tool call. Enterprise cloud deployments work with Claude 4 Sonnet’s 200K window or Gemini 2.5 Pro’s 1-million-token capacity. You don’t have that buffer. Every token allocation decision is a tradeoff that hits your agent directly.

A 32K window sounds like a lot until you do the math. A system prompt with careful instructions: ~800 tokens. A RAG retrieval of four document chunks: ~3,600 tokens. Tool definitions for eight tools: ~2,400 tokens. Five turns of conversation history: ~3,000 tokens. You’re at 9,800 tokens before the model has produced a single output token. With a local 7B model running at 4-bit quantization, you may have a smaller effective context than the spec suggests — quantization degrades long-context coherence faster than full-precision inference.

The practical playbook for local AI context engineering: compress aggressively (summarize conversation history after turn 4), select surgically (retrieve three chunks maximum, not ten), and isolate by default (use sub-agent patterns even for simple two-step tasks). The Write pillar is especially powerful here — a structured scratchpad file that the model reads and updates at each step is cheap in tokens and dramatically reduces the agent’s need to re-derive prior decisions.

This is where context engineering stops being a conceptual framework and becomes an actual survival requirement. The frontier-model builders at Google and Anthropic have a 1M-token runway to work with. You don’t. Context engineering is how you close that gap.

What Comes Next

Context engineering is not a prediction anymore. By July 2026, it’s the default vocabulary for anyone building AI agents in production — visible in ICML 2026’s workshop programme (at least 60 of 247 accepted proposals engaged with agentic AI), in Amazon shipping dedicated monitoring tooling for agent context performance, and in Gartner projecting 80% tool adoption by 2028.

The actual work ahead is not theoretical. It’s deciding, for your specific system, how to allocate a finite token budget — which history to compress, which documents to retrieve, which tools to expose, and which sub-tasks to isolate. If you’re building on frontier models with million-token windows, the margin for error is high. If you’re running Qwen3 or Llama 3.3 locally, context engineering is not optional overhead — it’s the engineering.

One honest recommendation: start with the Compress and Select pillars. They deliver the most immediate reliability improvement with the least architectural change. Add Write as soon as your agent needs to persist decisions across turns. And when your tasks get complex enough that one agent’s mess is contaminating another’s work, reach for Isolate.

The four-word version: manage the window deliberately.

Sources

• Andrej Karpathy, X post, June 25, 2025 — https://x.com/karpathy/status/1937902205765607626
• LangChain — “Context Engineering for Agents” — https://www.langchain.com/blog/context-engineering-for-agents
• Chroma Research — “Context Rot: How Increasing Input Tokens Impacts LLM Performance” (Hong, Troynikov, Huber) — https://www.trychroma.com/research/context-rot
• Phil Schmid — “The New Skill in AI is Not Prompting, It’s Context Engineering” — https://www.philschmid.de/context-engineering
• Lingrui Mei et al. — “A Survey of Context Engineering for Large Language Models” — https://arxiv.org/html/2507.13334v1
• Sourcegraph — “Context Engineering: A Practical Guide for AI Agents” — https://sourcegraph.com/blog/context-engineering
• LangChain GitHub — context_engineering repository — https://github.com/langchain-ai/context_engineering
• Atlan — “What Is Context Engineering? Complete 2026 Guide” — https://atlan.com/know/what-is-context-engineering/ (for Gartner stat citation)
• CruxDigits — “Context Engineering: The 2026 Playbook for AI Agents” — https://cruxdigits.nl/blog/context-engineering-ai-agents-2026/ (for ICML 2026 and CloudWatch data)

FAQ