A field guide · October 2026

The Context Window

What a model can see, and how to choose it
by Mat Siems
Part I

The Ground Floor

What the context window is, and is not.

Chapter 1 · Part I

The Desk, Not the Mind

Welcome. This is a field guide to the context window: the stretch of text a language model can see at the moment it answers you. It is a hundred short chapters, each meant to teach one thing you can use this week, whether you are writing a chatbot's system prompt, wiring up retrieval, or watching a coding agent burn through a session. The subject sounds technical. It is, mostly, a subject about choosing.

Start with the picture that will save you the most grief. A model is not a mind that happens to be talking to you. It is closer to a very fast, very well-read clerk sitting at a desk. On the desk is everything you put there: your instructions, the conversation so far, the documents you pasted, the results of any tools it called. The clerk reads the desk and writes a reply. That is all. There is no drawer underneath with your last conversation in it, no filing cabinet of your preferences, no quiet sense of who you are. If it is not on the desk, it does not exist.

This sounds obvious until you notice how often people behave as if it were false. They ask a model to use the style we agreed on in a fresh chat. They assume it knows the document they discussed with a colleague. They are surprised that the agent which fixed a bug yesterday has no idea today what it fixed. None of this is the model being forgetful. Forgetful implies it once remembered. It never held anything except what was in front of it.

The model does not know what you know. It knows what you showed it.

The useful consequence is that almost every disappointing answer can be traced to the desk. Either something necessary was missing, or something unnecessary was crowding it, or the right thing was there but buried where it would not be noticed. Those three failures, absence, clutter and burial, are the subject of most of this book. They are also, pleasingly, things you control. You cannot change how the model was trained. You can change what you hand it.

So here is the first exercise, and it costs nothing. Take the last answer from a model that annoyed you. Before blaming the model, write down exactly what was on its desk at that moment: the system prompt, every message, every pasted file. Then ask whether a capable human, given only that, would have done better. Often the honest answer is no. The model was not stupid. It was uninformed, which is a problem with a far better prognosis. Set the desk, and the clerk will mostly do the rest.

What the model can see If it is not on the desk, it does not exist. ON THE DESK Instructions system prompt Conversation so far every turn, re-sent Pasted documents files, passages Tool results commands, searches The model reads, then writes Reply only from the desk NOT ON THE DESK last chat · your preferences · yesterday's fix THREE WAYS A DESK FAILS Absence needed, but missing Clutter unneeded, crowding Burial present, unnoticed
Fig 1 · The Desk, Not the Mind. The model answers only from what is on its desk; absence, clutter and burial spoil it.
Chapter 2 · Part I

Stateless by Design

A language model, at the level where the arithmetic happens, is stateless. Each call takes a block of text in and produces a block of text out, and when it is finished nothing survives. No impression lingers, no lesson is quietly learned, no grudge is held. The next call starts exactly as clean as the first. This is not an oversight waiting for a patch. It is the design, and it is a sensible one.

Think of a calculator. You would not want it to remember your last sum and lean on it slightly when doing the next. You want the same input to produce the same kind of answer, every time, for everyone. Statelessness makes models predictable to operate, easy to scale and safe to share across millions of people who would rather their conversations did not leak into each other's. The price is that anything resembling memory has to be built around the model, not inside it.

And it is built, constantly. The chat interface that seems to remember your earlier messages is simply sending them all again, every turn. The assistant that recalls your name has stored a note somewhere and pasted it back in. The coding agent that picks up where it left off is reading a progress file it wrote last time. Every one of these is apparatus: a mechanism that smuggles yesterday onto today's desk. When it works, it feels like memory. When it fails, you see the machinery.

Everything a model seems to remember, someone arranged for it to be told again.

Knowing this changes how you debug. When an assistant forgets an instruction halfway through a long chat, the question is not why did it forget but was the instruction still being sent, and could it still be found. When a memory feature recalls something stale, the question is not why is it confused but which store did that come from, and who last updated it. You stop treating the model as a person with lapses and start treating the system as plumbing with leaks. Plumbing is easier to fix.

It also tells you where to put effort. If you want consistent behaviour across sessions, do not hope the model will absorb it. Write it into something that is reliably re-sent: a system prompt, an instructions file, a stored profile. If you want it to know what happened yesterday, write yesterday down in a form today can read. The model will not do this for you unless you build something that does.

There is a strange freedom in this. Every session is a fresh start, which means every session can be a better-prepared one. The model brings no baggage. The only baggage in the room is what you packed.

Memory is apparatus, not model User you App the apparatus Model stateless Store notes, profile CALL 1 message 1 system + msg 1 reply 1 nothing survives save a note CALL 2 message 2 load the note resend it all + note reply 2 Everything it seems to remember, someone arranged for it to be told again.
Fig 2 · Stateless by Design. Two calls: the app re-sends history and stored notes because the model keeps nothing.
Chapter 3 · Part I

Tokens, the Unit of Everything

Models do not read words. They read tokens: fragments of text, often a whole short word, sometimes part of a longer one, sometimes a single punctuation mark or a space glued to the front of a word. A tokeniser chops your text into these pieces before the model sees anything, and every limit, every bill and every measure of how much fits is counted in them. If you work with context, tokens are your unit, the way grams are a baker's.

The rough rule for English prose is that a token is about three quarters of a word, so a thousand words comes to something over a thousand tokens. That rule bends quickly. Code tokenises differently from prose, because of all the brackets and indentation. Unusual names, long numbers and identifiers in snake case break into many small pieces. Languages other than English often cost more tokens for the same meaning. Tables pasted as text, with their pipes and padding, can be startlingly expensive for what they say. JSON with deep nesting spends a surprising amount on quotation marks.

You do not need to memorise any of that. You need the habit of measuring rather than guessing. Most model providers offer a way to count tokens before you send, and most agent tools will show you how full the window is. Use them. People who guess almost always underestimate, because they think in pages and the model thinks in fragments. A document that feels short can be large; a log file that feels like noise can be enormous.

Count in the unit the machine counts in, or be surprised by its arithmetic.

There is a second reason to care. Tokens are not only a limit, they are a cost in time. The model processes your input and generates its output token by token, so more context means slower first replies and more output means longer waits. A prompt that is twice as long is not merely twice as expensive; it is twice as much for the model to read before it can begin, and twice as much material competing for its attention.

So this week, do one small audit. Take a prompt or system message you use often and count it. Then count the pieces: the instructions, the examples, the pasted reference material, the boilerplate nobody remembers adding. You will almost certainly find one section that costs far more than it earns. That section is your first edit. The goal is not stinginess for its own sake. It is to spend the budget on purpose, which is the only way anyone has ever kept a budget.

Tokens, not words Your text words, code, tables Tokeniser chops into pieces The ·refund _policy _v 2 ·applies . about 3/4 of a word per token, in English prose SAME MEANING, BIGGER BILL (ILLUSTRATIVE) English prose the baseline Code brackets, indents snake_case names many small pieces Other languages often more Tables as text pipes and padding Deeply nested JSON quotes, nesting Count in the unit the machine counts in, or be surprised by its arithmetic.
Fig 3 · Tokens, the Unit of Everything. A tokeniser splits text into pieces; code, tables and JSON cost more for one meaning.
Chapter 4 · Part I

The Window Has Edges

Every model has a maximum context: the most tokens it can take in, plus what it writes back, in a single call. Over the last few years that ceiling has risen from a few pages to something like a shelf of novels, and the largest windows now hold more text than most people will ever paste in. It is tempting to read this as the end of the problem. If everything fits, why choose?

Because the edge moved; it did not vanish. There is still a hard limit, and agents find it faster than you would expect. A coding session that reads a few dozen files, runs the tests a dozen times and keeps every result in its history can fill a very large window in an afternoon. A support bot that keeps whole conversation histories plus retrieved policy documents will reach the wall on its longest, angriest customers, which are exactly the ones where you most want it to behave. When the limit is hit, something has to give: the call fails, or older material is dropped, or the system compacts it into a summary. None of those happen at a convenient moment.

There is also a softer edge, which matters more. Long before the hard limit, quality begins to slip. A model asked to use one fact buried in a vast context does worse than the same model given that fact in a short one. It is slower and costs more as well. So the practical window, the size at which a model reliably does its best work for your task, is smaller than the advertised one, sometimes much smaller. You will not find that number on a spec sheet. You find it by testing.

The advertised window is a ceiling. The useful window is a floor you have to discover.

The working habit is to treat the maximum as an emergency reserve, not a target. Design your system so that a normal request sits comfortably inside a fraction of the window, leaving room for long conversations, large tool results and the reply itself. When you see sessions regularly approaching the edge, treat that as a design smell rather than a capacity problem. Something is being kept that need not be.

And keep an eye on the gauge. Most agent tools show how full the window is, as a percentage or a bar. Glance at it the way a driver glances at fuel. Not anxiously, not constantly, but often enough that the warning light is never the first you hear of it. A bigger tank is lovely. It is not a reason to stop looking at the gauge.

Two edges of the window advertised maximum: the ceiling useful window: found by testing a normal request system history tool results reply headroom soft edge: quality slips hard edge WHEN THE HARD EDGE IS HIT, SOMETHING GIVES The call fails error, mid-task Old turns dropped often silently Compacted into a summary Treat the maximum as a reserve, not a target. Watch the gauge.
Fig 4 · The Window Has Edges. The useful window sits well inside the advertised maximum; the hard edge breaks things.
Chapter 5 · Part I

A Conversation Is a Document

A chat feels like a conversation: you say something, it says something, you take turns. Underneath, it is a document that grows by one entry every turn and is read in full, from the top, every time the model replies. The model is not continuing a thought it had a moment ago. It is re-reading the whole transcript as if for the first time and writing the next paragraph.

This has consequences that surprise people. The first is cost. Turn fifty does not cost the same as turn one; it costs the system prompt plus forty-nine turns of history plus the new message. A long chat gets steadily more expensive and steadily slower, and nothing about the interface tells you so. The second is drift. Everything you said earlier is still on the page, including the instruction you abandoned, the draft you rejected, and the tangent about lunch. The model has no way to know which of these you consider closed unless the document says so.

The third consequence is the useful one. Because the transcript is a document, you can edit it. Many tools let you change an earlier message and regenerate from there, which removes the wrong turn from history entirely rather than piling a correction on top. You can start a new chat with a clean summary rather than dragging forty turns of exploration behind you. In an application you build, you decide exactly what history to send: all of it, the last few turns, a running summary, or a summary plus the recent turns verbatim. Each of those is a different document, and the model will behave differently with each.

The model does not remember the conversation. It reads the minutes.

So write good minutes. When a conversation has wandered and you have finally worked out what you want, say so plainly: ignore the earlier drafts; here is the brief. Better still, open a fresh session and paste only that brief. When you correct the model, correct it in a way that would make sense to someone reading the transcript cold, because that is literally who will read it. And when you build a chat product, treat history as something you assemble each turn, not something that simply accumulates.

There is a small comic dignity in this. The model is the most diligent reader of meeting notes ever built. It reads every line, every time, without complaint. It is just not very good at telling which bits of the meeting mattered. That part, as ever, is the chair's job.

The transcript, re-read every turn call 1 system user 1 reads 2 call 2 system user 1 reply 1 user 2 reads 4 call 3 system user 1 reply 1 user 2 reply 2 user 3 reads 6 cost, delay and drift grow with each turn WHAT YOUR APP SENDS EACH TURN All history complete, costly, drifts Last few turns cheap, may lose the brief Running summary compact, loses detail Summary + recent turns a sensible default The model does not remember the conversation. It reads the minutes. to fix a wrong turn: edit and regenerate, or start fresh with a clean brief
Fig 5 · A Conversation Is a Document. Each call re-reads the whole transcript, so apps choose which history to send.
Chapter 6 · Part I

Input and Output Share the Room

The context window is not only for what you send. The model's reply has to fit inside it too. If a model has a given maximum, and your prompt uses most of it, there is only a sliver left for the answer, and the answer will be cut off, sometimes mid-sentence, sometimes mid-function. The window is one room, and both the question and the response have to stand in it.

Most APIs make this explicit with a separate limit on output length, a maximum number of tokens the model may generate. Set it too low and you get truncated answers that look complete until you reach the end and find no ending. Set it high and you leave room, but the room is taken from the same total. In agent loops this matters even more, because the model's output becomes the next turn's input. A verbose agent fills its own window with its own commentary, and then has less space to read the files it actually needs.

There is a further subtlety with models that think before answering. Many current models can spend tokens on a reasoning phase before writing the visible reply, and that thinking is also generated text with a budget. Give a hard problem too little thinking room and the model rushes; give an easy one too much and you pay for deliberation nobody needed. Depending on the system, earlier thinking may or may not be kept in later turns. Either way, it is part of the arithmetic, and you should know which way your tools handle it.

Leave room for the answer. It is, after all, the reason you asked.

The practical habits are simple. Decide roughly how long a good answer should be before you ask, and set the output limit with headroom above that. Ask for the length you want in the prompt, because a model told answer in three sentences will generally comply and save you both time and window. When you need long output, a full document or a large file, consider producing it in sections across several calls rather than one heroic generation that might hit the ceiling. And in agent work, ask for terse progress notes rather than running commentary; the agent does not need to narrate its feelings about each file.

Watch for one symptom in particular: an answer that stops abruptly, or a structured output that is missing its closing bracket. Nine times out of ten that is not the model losing its nerve. It is the model running out of room, which is a configuration problem wearing a quality problem's coat.

One room, two tenants PROMPT FILLS THE ROOM system history documents + prompt reply answer stops mid-sentence; the JSON never gets its closing bracket LEAVE ROOM FOR THE ANSWER system history prompt thinking complete reply spare output limit set with headroom above a good answer's length IN AGENT LOOPS, OUTPUT BECOMES INPUT Verbose output narrating each file Next turn's input window fills itself Less room for files so ask for terse notes
Fig 6 · Input and Output Share the Room. Prompt, thinking and reply share one window; leave headroom or the answer is cut off.
Chapter 7 · Part I

Garbage In, Still

The oldest rule in computing is that a program given rubbish will return rubbish, efficiently. Language models were supposed to be different. They are forgiving of typos, generous with vague requests, able to make something plausible out of almost anything. And they are, which is precisely why the old rule now hides better. The rubbish still goes in. It simply comes out looking presentable.

Consider what counts as garbage in a context window. Outdated documents, still confidently worded. Two versions of the same policy, one superseded. A pasted email thread where the decision is in message eleven and messages one to ten argue the opposite. Retrieved passages that share keywords with the question but answer a different one. Tool output full of warnings that do not matter, with the one error that does matter in the middle. None of this looks like garbage. It looks like information. That is what makes it dangerous.

A model handed this material will not usually refuse or complain. It will do its best, which means blending what it was given into a fluent answer. If the old policy and the new policy are both present, you may get an elegant compromise that matches neither. If the decision is buried under the argument, you may get the argument. The output inherits the quality of its inputs, then adds polish, so the flaws arrive well dressed and are easy to wave through.

A model does not clean your inputs. It launders them.

The defence is unglamorous: curate before you send. Remove superseded documents rather than hoping the model will spot the date. When you paste a thread, add one line saying what was finally decided. When retrieval returns ten passages, check whether all ten belong there. Prefer one authoritative source to five overlapping ones. When something must be included but is of doubtful quality, label it as such: this is a draft from last year and may be wrong. A label is cheap and the model will use it.

Then test the claim. Take a task that produces middling answers and try it twice: once with everything you would normally include, once with only what a careful expert would hand over. In my experience the second version is frequently better and almost never worse, and it is always cheaper. The old rule was never repealed. It just learned to speak nicely.

Garbage in, polish out AS SENT Superseded policy Old and new versions Decision buried in msg 11 Near-miss passages Model blends it all Polished answer flaws, well dressed CURATED FIRST Current policy only Thread + what was decided Checked passages Doubtful item, labelled Model uses what's there Grounded answer cheaper, often better A model does not clean your inputs. It launders them.
Fig 7 · Garbage In, Still. Unchecked inputs come out as a polished answer; curated inputs give a grounded one.
Chapter 8 · Part I

The Prompt Was Never Enough

For a while the craft was called prompt engineering, and it concentrated on phrasing. Which words made the model behave. Whether to say please. Whether to tell it that it was an expert. Some of those tricks worked for a season and some still help at the margins, but the name always described the smallest part of the job. The prompt is one paragraph on the desk. The desk is the whole job.

The shift in language over the last couple of years, from prompt engineering to context engineering, is not a rebranding exercise. It reflects what practitioners discovered once models were used for real work. The difference between a mediocre assistant and an excellent one is rarely the wording of the request. It is whether the right documents were retrieved, whether the conversation history was trimmed sensibly, whether the tool outputs were readable, whether the standing instructions were current, and whether any of it contradicted the rest. A beautifully phrased question asked over a messy desk still gets a messy answer.

Context engineering, then, is the practice of deciding what the model sees on each call. It includes the prompt, but also everything assembled around it: system instructions, memory, retrieved material, tool definitions, tool results, examples and history. Each of these has its own failure modes and its own techniques, which is why this book has parts for most of them. The skills overlap with old crafts. Some of it is editing. Some of it is information architecture. Some of it is plain systems design, with tokens instead of bytes.

Prompting is what you say. Context is everything the model hears.

None of this means wording is irrelevant. A clear request beats a muddled one, and later chapters cover how to write instructions a model can follow. But wording is the last ten per cent. If you find yourself rephrasing the same question for the fifth time, hoping for a better answer, stop and look at the rest of the desk instead. The problem is usually not in the sentence you keep changing. It is in the material you never looked at.

The practical test is to ask, of any disappointing interaction, what did the model actually have? Not what you meant, not what you assumed it knew, but the literal assembled context. Most tools will show you, if you ask. The first time you read one carefully, you will likely find the answer to your problem written there, in the form of something missing.

The prompt is one item on the desk CONTEXT: EVERYTHING ASSEMBLED FOR THIS CALL System rules standing orders Memory stored notes Retrieved docs passages Tool definitions what it can call Tool results what came back Examples what good looks like History turns so far The prompt what you say WHERE THE QUALITY COMES FROM selection, trimming, structure, currency words Prompting is what you say. Context is everything the model hears.
Fig 8 · The Prompt Was Never Enough. The prompt is one of eight things assembled into context; wording is the least lever.
Chapter 9 · Part I

Context Is a Verb

It is natural to talk about context as a thing: the context, as if it were a fixed bundle you attach to a request. In practice it behaves more like an activity. Each call, something has to decide what goes in and what stays out. That decision is made either by you, deliberately, or by defaults nobody chose. Either way it is being made, every time.

Look at what happens in a typical agent session. On the first turn the context is a system prompt, some instructions and your request. By turn ten it includes files the agent read, commands it ran and their output. By turn thirty it may have compacted the early history into a summary, dropped some tool results, and pulled in new documents. Nobody sat down and designed the turn-thirty context. It assembled itself, through a series of small decisions made by code, by the agent and by you. The quality of the session depends heavily on those decisions, which is why it pays to make some of them on purpose.

Thinking of context as a verb changes the questions you ask. Instead of what is the context for this assistant, you ask what should this assistant see right now, given this request. Instead of loading a fixed bundle of documents at the start, you fetch the relevant ones when they become relevant. Instead of keeping all history forever, you decide at each point what deserves to remain. This is the design stance behind most of the techniques in this book: retrieval, compaction, subagents, just-in-time loading. Each is a way of contextualising freshly rather than accumulating blindly.

Context is not what you have. It is what you choose, again, each turn.

For people building systems, the action item is to locate the code where context is assembled and make it explicit. Somewhere there is a function, a template or a framework default that decides what each call contains. Find it. Log what it produces. Read a few examples. Many teams discover that their context assembly has never been reviewed by anyone, because it was written in a hurry once and then left to run. It deserves the same care as any other critical code path.

For people simply using assistants, the action is smaller and just as useful. Before a long session, ask yourself what the model needs for the next step, not the whole project. Give it that. When the step changes, change what it sees. It is a little more effort per turn and a great deal less confusion per hour. Context is something you do. Do it on purpose.

Context assembles itself, unless you choose system prompt instructions your request system prompt instructions request files read command output command output system prompt instructions summary of turns new documents tool results latest request system prompt brief for this step the two files needed request turn 1 chosen by: defaults turn 10 chosen by: code + agent turn 30 chosen by: code + agent next turn chosen by: you, on purpose Context is not what you have. It is what you choose, again, each turn.
Fig 9 · Context Is a Verb. Context drifts from turn to turn by default; the next turn can be assembled on purpose.
Chapter 10 · Part I

The First Discipline

If this book had to be reduced to one skill, it would be subtraction. Nearly every beginner's instinct with a context window is additive. The answer was weak, so add more instructions. The model missed something, so add more documents. It forgot a rule, so repeat the rule in capitals. Each addition feels responsible. Taken together, they bury the request under a pile of well-meant material, and the model's attention, which is not unlimited, gets spread thinner with each layer.

Subtraction asks a harder question: what can come out? Which instructions are now irrelevant, duplicated or contradicted elsewhere? Which retrieved documents are near misses? Which parts of the history are finished business? Which tool outputs were useful once and are now just noise? Removing these is not laziness. It is the act of making the remaining material legible, the way a good editor makes an essay stronger by cutting its weakest paragraph.

It is also counter-intuitive enough that teams resist it. A long system prompt feels thorough; a short one feels negligent. A retrieval system that returns twenty passages feels safer than one that returns four. But models, like people, perform better when the signal is clear. More context helps when the extra material is relevant and well organised. When it is merely available, it mostly costs money, time and accuracy.

The best context is not the most complete. It is the most chosen.

Make subtraction a routine rather than a mood. When you revise a system prompt, try deleting a section and running your tests; if nothing gets worse, leave it out. When you set up retrieval, start with fewer results and add more only when evaluation shows a gain. In long agent sessions, clear or compact when the history stops being useful, not when the window is full. For instructions files, schedule an occasional pruning, because they grow by accretion and nobody ever removes the line about the database you migrated off two years ago.

This is the ground floor on which the rest of the book stands. Later parts will explain how the model reads, how to write standing instructions, how to manage memory and retrieval, how tools and agents fill the window, and what goes wrong when nobody is choosing. Underneath all of it is the same discipline. Put in what the task needs. Leave out what it does not. Then look again, because what the task needs has probably changed. The window is a gift. Clutter is how people refuse it.

Subtraction, the first discipline Everything you could add minus irrelevant instructions minus near-miss documents minus finished history minus stale tool output What the task needs MAKE IT ROUTINE Delete, then test no worse? leave it out Fewer results first add only on evidence Compact early when history stops helping Prune on a schedule files grow by accretion The best context is not the most complete. It is the most chosen.
Fig 10 · The First Discipline. Subtraction removes stale, duplicated and near-miss material until only the need remains.
Free preview · Part I

Keep reading

You have read the first 10 of 100 chapters of The Context Window. The full book has every chapter and plate, as PDF and EPUB in English, French, Polish and Spanish.

The full book is coming soon to the shop.

Already bought it? Read the full edition online

The Context Window · First Edition, October 2026
100 chapters · 10 parts · one hundred diagrams
by Mat Siems · MS Books, No. 14 · 2026