In this article＋
01 / ARTIFICIAL INTELLIGENCE
It is a workbench, not long-term memory
When the workbench fills, a product must remove old content, summarise it, move it into external memory or refuse more input. Conversation alone does not permanently update model parameters.
The strangest long-thread moment is when the model quotes something you said three hours ago but forgets the decision you reversed twenty minutes ago. Visible is not the same as prioritised.
In a long thread, early assumptions, later corrections and one-off exceptions get tangled. The model may still 'see' a sentence yet let newer, louder material become the spine. Humans do this in meetings too: full minutes, fuzzy decisions.
02 / ARTIFICIAL INTELLIGENCE
Two kinds of forgetting look identical
Two kinds of forgetting look the same. Information may be pushed out, or remain inside while the model fails to use it.
Lost in the Middle experiments found stronger performance when evidence appeared near the beginning or end than in the middle. Fitting information and retrieving it reliably are separate capabilities.
It is especially irritating in a site, code or contract edit: you think it is following along, while it has quietly grabbed an old version.
A resilient long task periodically rewrites a summary, labels confirmed versus open items, separates source documents from working notes and retrieves only what is needed instead of dragging the entire history forever.
03 / ARTIFICIAL INTELLIGENCE
A large window creates false confidence
Putting a complete contract, every email and dozens of messages into one prompt feels comprehensive. Irrelevant passages compete, duplicate versions conflict, and old instructions interfere with the current objective.
A larger window changes the problem from does it fit to will the right relationship be found.
After the second repeated correction, I usually start a clean thread. It feels like giving up. It is really saving future me time.
There is a limit. Beyond it, content may be dropped, compressed or replaced by a rough summary. Even inside the limit, position, formatting and repetition change what gets used.
04 / ARTIFICIAL INTELLIGENCE
Summaries save space and slowly deform
Products often compress older conversation into summaries. A summary preserves what appeared important and drops tone, exceptions and detail.
Repeated summarisation accumulates distortion like copies of copies. A later change of position may mean a decisive limitation was classified as background during compression.
05 / ARTIFICIAL INTELLIGENCE
Real product memory usually lives outside the model
Persistent product memory usually lives outside the model. Preferences, task state and files are stored and retrieved into later context.
Good designs preserve source, date, scope and deletion. Bad designs promote a temporary model guess into a permanent user fact.
External memory is valuable because it can be inspected and corrected.
06 / ARTIFICIAL INTELLIGENCE
How to make long conversations more reliable
Keep the current goal, constraints and requested output near the question. Retrieve relevant chunks instead of dumping whole files.
Confirm state summaries and store important decisions structurally. Ask not only how large the window is, but how information enters, is selected, compressed, versioned and updated.
The context budget includes things you cannot see
The context budget includes things the user cannot see: policies, role messages, tool schemas, tool returns, retrieved chunks and output allowance. A product advertising a long window may leave less room for the document itself.
Large tool descriptions consume capacity merely to describe abilities that might be used.
A model may remember a sentence but forget its status
A model may repeat a number while forgetting it was provisional, or remember a rule while losing its exception. Long-context difficulty includes relationships: which version replaced which, who asserted a claim and when it became valid.
Structured state preserves those relationships more reliably than raw capacity alone.
Capacity is not comprehension
Window size is a hardware-and-model capability; usable context is a product outcome. The latter depends on retrieval, ordering, conflict handling, summaries and evaluation.
This distinction explains why two products using the same model can behave differently on the same long file. One merely fits the file; the other prepares a small, well-labelled set of facts the model can actually use.
More space helps, but information architecture determines whether that space becomes memory or noise.
07 / ARTIFICIAL INTELLIGENCE
The bottleneck is often not capacity. It is attention and order.
In a long thread, early assumptions, later corrections and one-off exceptions get tangled. The model may still 'see' a sentence yet let newer, louder material become the spine. Humans do this in meetings too: full minutes, fuzzy decisions.
A resilient long task periodically rewrites a summary, labels confirmed versus open items, separates source documents from working notes and retrieves only what is needed instead of dragging the entire history forever.
08 / ARTIFICIAL INTELLIGENCE
A large context is not infinite memory
There is a limit. Beyond it, content may be dropped, compressed or replaced by a rough summary. Even inside the limit, position, formatting and repetition change what gets used.
The practical test is your own material: place a critical clause in different positions and see whether the system actually retrieves it, rather than admiring a headline context number.
09 / ARTIFICIAL INTELLIGENCE
Questions people actually ask
Should chunks always be smaller?
No. Tiny chunks can split a definition from its exception. Chunk by meaning and keep headings, page numbers and other cues.
Can a summary replace the source?
It is useful for navigation, not as an automatic substitute for a contract, policy or number. Verify important claims against the source.
When should I start a fresh thread?
When the goal or assumptions changed, or you are correcting the same misunderstanding repeatedly. A clean workspace is often faster.
Sources
Sources
Sources support the mechanisms and limitations discussed here. Models, products and prices change; check the official page and date when a detail matters.