The Limits of Prompting and Context as a New Discipline in 2026

Prompting got you this far, but it will not take you further. You can write the most elegant instruction in the world and still watch helplessly as a model that no longer answers a single message, but runs as an agent across dozens of steps, calls tools, and accumulates history, begins to forget, contradict itself, and make mistakes — not because the prompt is bad, but because the prompt has stopped being the problem. This is where context engineering comes in: the discipline of curating and managing the entire set of tokens that enters an inference window, not just the prompt you write yourself.

From a good question to a well-managed state

In 2023–2024, "being good at AI" essentially meant formulating clever prompts. You knew a few techniques — few-shot, chain-of-thought, instruction delimiting — and you got results well above average. That skill set remains valid and is taught in detail in the it-02 Prompt Engineering course. But it describes a single act: composing one input message.

The problem is that real applications in 2026 no longer send a single message. An assistant resolving a support ticket, an agent navigating a codebase, or a system planning a trip executes a loop: it thinks, calls a tool, receives a result, thinks again, calls another tool, and so on. At every iteration, the context window fills with new material — API responses, search results, intermediate messages, reasoning traces. Your initial prompt becomes an ever smaller fraction of a whole you no longer control directly.

This is where the conceptual break happens. Prompting optimizes one message. Context engineering optimizes everything the model sees at the moment of each inference. The two are not in competition; the second contains the first. The system prompt remains an essential component — but it is one component within a token budget that also includes conversational history, tool definitions, retrieved documents, persistent memory, and execution results.

What "context" means in this discipline

When we say context, we mean the complete set of tokens present in the inference window at a given moment. According to the official Anthropic post "Effective context engineering for AI agents" (anthropic.com/engineering), this set typically includes:

• The system prompt — the persistent instructions, the role, the rules of behavior.
• Tool definitions — the schemas of the functions the agent can call.
• Conversational history — the previous messages from the user and the model.
• Tool call results — outputs that can be large and noisy.
• Retrieved information — document fragments brought in through retrieval.
• Persistent memory — facts and preferences kept across sessions.

Context engineering is the art and engineering of deciding, for each of these components, what goes in, how much goes in, in what form, and at what moment — so that the model receives exactly the signal it needs, without the noise that degrades its performance.

Context is a finite resource, not a bottomless basket

The natural temptation of any engineer who discovers large context windows is to "throw everything in." If the model accepts a million tokens, why not give it all the documentation, the entire history, every file? The Anthropic post confronts this intuition head-on: context is a finite resource with diminishing marginal returns. Every added token has a cost — not just financial, but attentional.

The reason lies in the mechanics of transformer models. Attention — the mechanism through which the model weighs the relevance of each token against the others — operates on pairs of tokens. As the number of tokens grows, the model's "attention budget" stretches ever thinner across ever more material. The model was trained predominantly on sequences shorter than the window's theoretical maximum, so its experience with dependencies spanning very long distances is comparatively limited.

The practical consequence goes by the name context rot in the post: as the number of tokens in a window grows, the model's ability to accurately retrieve a specific piece of information decreases. It is not a defect; it is a property. What matters is that you treat it qualitatively and prudently — not as a hard universal figure. The exact degree of degradation depends on the model, the version, the position of the information, and the nature of the task, and it should not be presented as a fixed percentage valid everywhere.

The guiding principle, formulated by Anthropic, is easy to remember: seek the smallest possible set of high-signal tokens that solves your task. Not the maximum amount of information, but signal density.

A useful analogy: attention as limited working memory

Think of the context window as the working memory of a human expert reading a case file. A brilliant lawyer who receives five essential pages will build a coherent argument. The same lawyer, buried under five hundred pages of which 480 are irrelevant, will miss details, confuse facts, and grow tired. Not because they became less competent, but because the signal was diluted in noise. Models suffer from an analogous dynamic, and context engineering is the discipline that keeps your case file clean.

How "pure" prompting fails in the world of agents

Let's walk through three concrete failure patterns in which a perfect, isolated prompt does not save you.

1. History accumulation. By the twentieth iteration of an agentic loop, the window contains nineteen tool results, many of them stale or redundant. The system prompt is impeccable, but the model is drowning in old results. The solution is not a better prompt, but a history management strategy — a topic this course covers in the modules on compaction.

2. Noisy tools. A single call to an API can return a ten-thousand-token JSON of which nine thousand tokens are useless metadata. No prompt compensates for this pollution; you need a discipline of curating tool outputs before they enter the context.

3. Missing memory. The user told you three sessions ago that they prefer concise, to-the-point answers. The system prompt does not "know" this, because the prompt is static. You need a layer of persistent memory that injects the right fact at the right moment — exactly the central subject of this course.

Where this course sits on your skills map

It is crucial to position this discipline correctly so you do not relearn what you already know.

• Relative to it-02 Prompt Engineering: there you learn to write excellent prompts — structure, examples, format control. Here we assume you already know that and we deal with the layer above: how you manage all of the context and the persistent memory that make an agent reliable over the long run. We will not re-teach prompt-writing techniques; we consider them a prerequisite.
• Relative to it-05 AI Agents: there you build agents from scratch — architecture, control loops, tool use, planning. Here we do not rebuild the agent; we add the layer of context and memory that makes it trustworthy when it runs for hours on end or comes back days later. We will reference agent architecture, but we will not repeat it.

In short: it-02 gives you the message, it-05 gives you the machinery, and this course gives you the discipline of feeding that machinery exactly the information it needs, at the right moment, without suffocating it.

What you will be able to do by the end of the course

This module is a map. Over the span of the course you will gain the ability to:

1. Reason about the context window as a budget and allocate it deliberately across components.
2. Recognize and counteract context rot through curation and structuring of information.
3. Design memory layers (working, semantic, episodic, procedural) suited to the task.
4. Decide when information should be loaded up front and when it should be retrieved on demand.
5. Explicitly version models and their windows, because these dimensions change fast.

The large windows of 2026 — Claude Opus 4.8 and 4.7 with one million tokens, GPT-5.5 with one million, Gemini 3.1 Pro — make context engineering more important, not less. Large space does not exempt you from discipline; it only raises the stakes of bad choices. The next lesson dissects exactly this window as a finite resource and introduces the vocabulary of budgeting.

Four signals that you have crossed from the world of prompting into the world of context engineering

Many engineers cross the border without realizing it and then wonder why "the same prompt that worked perfectly" suddenly started producing inconsistent results. Here are four concrete signals that your problem is no longer a prompting problem, but a context engineering one. Use them as a diagnostic checklist.

Signal 1 — Behavior degrades as the conversation advances. The first few exchanges are impeccable, but after ten or fifteen rounds the model begins to forget instructions from the beginning, to contradict itself, or to reintroduce errors that were already fixed. This is not a bad prompt; it is unmanaged history that has pushed critical information into the risk zone of the window. No rewrite of the system prompt fixes a history that has ballooned out of control.

Signal 2 — A single tool poisons your entire context. You add a tool that queries an external system and, suddenly, answer quality drops everywhere, including on tasks unrelated to that tool. The cause is almost certainly a voluminous, noisy tool output that has flooded the window. The solution is not to write better instructions, but to clean the tool's result before letting it into the context.

Signal 3 — You need the agent to "remember" between sessions. The moment the user expects the agent to recall a preference expressed yesterday or a decision made last week, you have completely left the domain of prompting. The prompt is, by definition, static and amnesic. Persistent memory is a separate, first-class layer that you design deliberately.

Signal 4 — The model has all the information, yet it still gets it wrong. You put into the context precisely the document that contains the correct answer and, even so, the model answers incorrectly or evasively. This is the classic signature of signal dilution: the information is present, but it is not accessible, because it gets lost in a mass of competing material. The next lesson explains why.

If you recognize any of these four signals in a project of your own, you are already, without having asked for it, in the middle of a context engineering problem. The rest of the course gives you the instruments to solve it systematically, instead of treating it through trial and error at the prompt level.

A brief history of the transition: 2023–2026

It is worth understanding why this discipline appears now and not earlier, because the history explains the structure of the skill.

In 2023, models had small windows and were used predominantly conversationally: one question, one answer. Prompting was the whole story, and few-shot was the leading technique. In 2024, windows grew and the first retrieval-augmented generation patterns appeared, but context largely remained something you composed manually before the call.

The qualitative leap comes with the maturing of agents: systems that call tools in a loop, run dozens or hundreds of steps, and produce a large part of their own context by themselves. In this regime, the human can no longer compose each window manually, because the window rewrites itself at every step. Thus arose the need for strategies — rules and mechanisms that automatically decide the composition of the context. The Anthropic post "Effective context engineering for AI agents" names and systematizes exactly this set of strategies, elevating an ad-hoc practice to the rank of a discipline.

In other words, context engineering did not "replace" prompting; it became indispensable only when agents made manual context management impossible. But understanding why the discipline appeared is not the same as knowing how to allocate a token budget, how to counteract context rot, or how to design the memory that keeps an agent lucid for hours on end. From here on, each lesson puts a concrete instrument in your hands for solving systematically what, until now, you have only learned to recognize.

Summary

• Prompting optimizes one message; context engineering optimizes the entire inference window of a multi-step agent.
• Context is a finite resource; more does not mean better, because of the phenomenon of context rot.
• The Anthropic principle: seek the smallest set of high-signal tokens that solves the task.
• The discipline sits on top of it-02 (writing the prompt) and complements it-05 (building the agent) with the layer of context and memory.