Daniel's been thinking about ceilings. Specifically, the million-token context window that's been sitting over agentic AI like a low roof, quietly dictating every architectural decision we make. His question is: what if we wake up tomorrow and it's ten million? Some breakthrough lands, the constraint jumps an order of magnitude overnight. With the tools we have right now, what changes? His suspicion — and he's invited us to demolish it — is that a lot of what we call sophisticated engineering is really just compensation for a limit that's about to move. Context optimization, compaction, retrieval strategies, splitting work across subagents so each one gets a manageable slice. A lot of it might look distinctly retro the morning after the ceiling lifts.
And he wants to know what survives. If the reason you split work across agents was never really the window — if it was parallelism, or isolating failures, or keeping one agent's mistakes from contaminating another's reasoning, or just the fact that attention over ten million tokens may not behave anything like attention over one — then the architecture doesn't collapse when the constraint lifts.
Right. And he wants the research aired properly — the real evidence that bigger windows don't automatically buy you better use of what's in them. So we're going to take apart the assumption that bigger is simply better, and figure out what's load-bearing and what's scaffolding.
The thing is, I think Daniel's mostly right about the scaffolding part. But the research objection is stronger than he might expect, and it changes where the line falls. Let's set the stage first.
Where are we right now?
We've been sitting at roughly a million tokens for a while. Gemini has hit two million in some configurations, Claude and GPT have been in the high six figures to low millions. No one's broken through to anything qualitatively different in... long enough that the whole field has organized itself around that ceiling. And that's the key word — organized. The million-token limit isn't just a number. It's shaped the entire practice of building AI systems.
It's the gravity well. Everything orbits it.
Everything. Retrieval-augmented generation exists because you can't fit your entire codebase or document corpus into context, so you pull in relevant chunks on demand. Multi-agent orchestration patterns — the whole subagent architecture — that's partly about dividing cognitive labor, sure, but it's also about dividing the context budget. Each agent gets its own window, its own slice of the problem. Context compaction strategies, summarization pipelines, the entire field of prompt compression...
All of it is saying the same thing: we can't fit everything in, so what do we throw away?
And what do we keep, and in what order, and how do we make sure the important stuff doesn't end up in the middle where the model forgets it. The whole discipline has become an exercise in information triage.
Which is exactly Daniel's point. If the ceiling lifts, triage becomes less urgent. You can just... keep more.
That's the intuition. And I think for a certain class of problems, it's correct. But to understand where it breaks, we need to look under the hood at how attention actually works.
Take us there.
The core mechanism is self-attention. Every token in the context window attends to every other token. The computational cost scales quadratically — double the tokens, quadruple the computation. That's been the hard physical constraint on context window growth. But the behavioral issue is different. It's about what attention does, not just how much it costs.
And what it does is... degrade.
In specific, predictable ways. There was a study — and this is the one Daniel's gesturing toward — where researchers took a long context and replaced the irrelevant tokens with blanks. Literally blanked them out. The reasoning still collapsed.
Wait. The tokens that didn't matter — they removed them entirely — and performance still dropped?
Still dropped. The model wasn't confused by the content of the irrelevant tokens. It was confused by the presence of tokens. The mere fact of a long context, even one padded with blanks, degraded its ability to reason about the relevant information.
So it's not a signal-to-noise problem. It's a... distance problem.
That's the leading interpretation. The model's attention is spread across the entire window, and the longer the window, the more diluted the attention on any given piece of information. It's not that the model can't find the relevant stuff. It's that it can't focus on it with the same intensity.
Which means a ten-million-token window isn't just a bigger bucket. It's a bucket where the thing you're looking for is even harder to see.
And there's a related phenomenon called "lost in the middle." Models are reliably better at recalling information from the beginning and the end of a context window than from the middle. You can see this in benchmark after benchmark. Put a critical fact at position five hundred thousand in a million-token context, and the model might miss it entirely. Put it in the first thousand tokens or the last thousand tokens, and recall is much higher.
Does that pattern hold as windows get bigger?
It does. And in some studies, it actually gets worse. The middle gets wider, and the model's attention gets more U-shaped. Beginning and end are fine. The vast middle is... a fog.
So if I have a ten-million-token window, my "fog zone" is now enormous.
Potentially most of the window. And that's the strongest objection to Daniel's premise. He's imagining a world where the ceiling lifts and we can just dump everything in — no more triage, no more retrieval strategies, no more splitting work across agents. But if attention doesn't scale with context, then dumping everything in is exactly what you don't want to do. You've just moved your valuable information into a much larger haystack.
The needle didn't get bigger. The haystack did.
And the model's ability to find the needle didn't improve. It might have gotten worse.
So Daniel's suspicion that context optimization becomes retro — that might be exactly backwards. If windows grow but attention doesn't, optimization becomes more important, not less.
I think that's right. But let me push back on myself for a second, because there's a counterargument. The blank-token study and the lost-in-the-middle research were done on models with certain attention architectures. If the breakthrough that gives us ten million tokens also involves architectural improvements to attention itself — which it probably would, because quadratic scaling alone makes ten million tokens computationally brutal with current approaches — then the behavioral problems might improve too.
So the ceiling lift and the attention fix might arrive in the same package.
They might. You'd need something like a more efficient attention mechanism — sparse attention, linear attention, some kind of hierarchical scheme. And those architectural changes could, in principle, also address the focus problem. But we can't assume it. The research shows that bigger windows, in isolation, don't solve attention degradation. If the breakthrough is just "we made the window bigger," the behavioral problems remain.
And the history here isn't encouraging. We've scaled context windows by orders of magnitude already — from a few thousand tokens to a million — and the lost-in-the-middle problem didn't go away. It persisted.
It persisted. So the safe bet is that a ten-million-token window comes with the same attention physics, just stretched further. Which means triage isn't going anywhere.
Alright. So let's get to the core question. Daniel wants to know: which of today's practices are load-bearing, and which are scaffolding around a temporary limit? How do we tell the difference?
I think the diagnostic is to look at the failure mode. Why did you introduce this practice? If the answer is "because we ran out of room," it's probably scaffolding. If the answer is "because the model's reasoning degraded even though it had room," it's probably load-bearing.
Give me an example of each.
Context compaction — summarizing earlier parts of a conversation to free up tokens for new material. That's mostly scaffolding. You're doing it because you hit the ceiling. If the ceiling moves to ten million, you can afford to be less aggressive about summarization. You might keep more raw context, summarize less frequently. The practice doesn't vanish, but its importance drops.
And load-bearing?
Splitting work across subagents for failure isolation. Imagine you have an agent that's writing code and another agent that's reviewing it. You don't want the reviewer's reasoning to be contaminated by the writer's mistakes. You want it to approach the code fresh. That's not about context limits — that's about cognitive hygiene. Even with an infinite window, you'd still want separate agents for that.
Because the problem isn't "we can't fit both perspectives in context." The problem is "we don't want one perspective to infect the other."
Contamination prevention is load-bearing. It survives any ceiling lift.
What about parallelism?
Same thing. If you're splitting a task across five agents because each one needs to work on a different subproblem and you want them running concurrently, that's not about context. That's about wall-clock time. Even if one agent could hold the entire problem in context, five agents working in parallel finish faster. Parallelism survives.
So the rule of thumb might be: if the practice is about managing the model's attention, it's load-bearing. If it's about managing the token count, it's scaffolding.
That's... actually a really clean way to put it. And it explains why retrieval-augmented generation is in a weird middle ground. Part of RAG is about token count — you can't fit the whole corpus, so you retrieve. That's scaffolding. But part of RAG is about attention management — even if you could fit the whole corpus, the model might not attend to the right parts. Retrieval acts as a focusing mechanism. That part is load-bearing.
So RAG doesn't go away. It evolves.
It evolves. In a ten-million-token world, you might do less retrieval and more in-context reference. But you'd still want some mechanism to surface the relevant information and put it where the model can see it — preferably near the beginning or the end.
The U-shaped attention curve becomes a design principle.
It already is, for people who are paying attention. No pun intended. You put your system prompt at the beginning, your most recent instructions at the end, and you try not to bury anything critical in the middle. With ten million tokens, that discipline becomes even more important.
Because the middle is now vast.
The middle is a continent. You could lose entire libraries in there.
Let's talk about what Daniel called context rot. The thing where an agent forgets or misremembers something from three hours ago, even though it's still in the window.
This is one of the most practically important phenomena in agent engineering right now. You're deep in a coding session with an agent. Three hours in, it asks you about a design decision you already discussed at length in hour one. The information is technically in context — it hasn't been truncated — but the agent can't reliably access it.
And a bigger window doesn't fix that.
It might make it worse. If context rot is caused by attention dilution over long sequences, adding more tokens just dilutes further. The agent has more context to get lost in.
So the practices that address context rot — checkpointing, periodic summarization, explicit memory retrieval — those are load-bearing.
They're load-bearing. They're not compensating for a token limit. They're compensating for an attention limit. And until attention mechanisms fundamentally change, those practices remain essential.
Which brings us to the investment question. Daniel's wondering whether heavy investment in context optimization right now is a forward-thinking bet or a sunk cost.
I think it depends on what kind of optimization. If you're investing in token-level compression — getting more information into fewer tokens — that's probably a bet with a limited shelf life. When windows expand, the pressure to compress eases. Not to zero, but significantly.
And if you're investing in attention-aware structuring?
That's a bet on a durable problem. Understanding where in the context window the model is actually paying attention, and designing your information architecture around that — that's going to matter for a long time. Maybe forever.
What about the multi-agent orchestration frameworks? The whole ecosystem of tools for spinning up subagents, managing their context windows, passing messages between them?
Some of that is scaffolding. The parts that exist purely to keep individual agent contexts under the token limit — those get less critical. But the parts that handle agent handoffs, that manage shared state, that resolve conflicts between agents — those are addressing coordination problems, not context problems. They survive.
So the orchestration layer doesn't collapse. It thins out in some places and thickens in others.
I think it thickens in interesting ways. If individual agents can hold much more context, you might have fewer agents overall — one agent can handle a broader scope. But those agents are more capable, which means the coordination between them becomes higher-stakes. The handoff points matter more.
Fewer agents, but each one is more... sovereign.
And the sovereignty creates new failure pattern. If an agent with a ten-million-token context makes a mistake, the blast radius is larger. It's been operating on a much wider set of assumptions. Debugging that is harder.
Which makes failure isolation even more important. You want containment.
You want containment. And that's another load-bearing principle. Even with infinite context, you don't want one agent's reasoning errors to cascade unchecked through the system. You want firebreaks.
So Daniel's suspicion that a lot of current practice is scaffolding — I think we're mostly agreeing with that framing. But the interesting part is what counts as scaffolding and what doesn't.
And the research says the line isn't where most people would draw it. The intuitive line is: "if we had more tokens, we wouldn't need to do X." But the real line is: "if attention worked differently, we wouldn't need to do X." And those are not the same thing.
The token count is a red herring. Attention is the real constraint.
Always has been. The million-token ceiling is just the most visible symptom.
So let's get concrete. Daniel asked how you'd tell the difference before the answer arrives. What's the diagnostic?
I'd propose a simple test. Take your system and ask: if I doubled the context window tomorrow, would this practice become unnecessary? If the answer is yes, it's probably scaffolding. If the answer is "no, because the model would still struggle to attend to the right information," it's load-bearing.
And if you don't know?
Then you run the experiment. You simulate a larger window by taking a problem that currently requires aggressive context management and giving an agent much more room — use a model with the largest available window and don't optimize. See where it breaks. If it breaks because it can't find information, that's attention. If it breaks because it literally ran out of tokens, that's the ceiling.
Most people are going to discover it's attention.
Most people are going to discover it's attention. And that's the strongest objection to Daniel's premise. The ceiling lift might not change as much as he expects, because the ceiling was never the real problem.
Though I want to push on that a little. There are genuinely token-constrained workflows. If you're doing long-horizon autonomous tasks — an agent that operates for hours or days, accumulating context the whole time — the million-token ceiling is a real, binding constraint. You literally run out of room.
And for those workflows, a ten-million-token window is transformative. You go from "we must aggressively summarize and prune" to "we can be much more relaxed about what we keep." That's a genuine qualitative shift.
So the ceiling lift does matter. It just doesn't make the attention problem go away.
It doesn't. It changes which problems you're dealing with. At a million tokens, you're fighting both the token limit and the attention limit. At ten million, the token limit recedes, and the attention limit becomes the dominant constraint. You trade one bottleneck for another.
The new bottleneck might be harder to engineer around. Token limits are brute and obvious — you can count tokens, you know when you're close, the failure pattern is clear. Attention degradation is fuzzier. Harder to measure, harder to predict.
Much harder. You don't get a nice error message that says "attention diluted beyond usable threshold." You get subtle failures. The agent seems fine, but it's quietly ignoring something important from six hours ago.
That's the nightmare scenario. Everything looks normal until it isn't.
That's why the diagnostic framework matters. If you don't know whether your system's failure pattern is token pressure or attention pressure, you're engineering in the dark.
Before we wrap, Hilbert's been itching to say something. And it involves index cards.
Hilbert: Nineteen ninety-seven. I was working for a search startup called QueryPoint. Went under in '99. My job title was "data janitor."
Data janitor.
Hilbert: We had a sixty-four kilobyte RAM limit on the index server. That was our context window. Sixty-four K. I spent my days deciding what went in and what stayed out. Had a card catalog — physical index cards — with annotations about which terms were worth the RAM.
You were doing context optimization. By hand.
Hilbert: With paper. The instinct to triage — it's not a workaround. It's just what you do when you have more information than capacity. The capacity always runs out somewhere. If it's not tokens, it's attention. If it's not attention, it's time. If it's not time, it's money.
You're saying the optimization impulse survives any particular constraint.
Hilbert: I'm saying I still have the index cards. Three shoeboxes in the garage. Been thinking about digitizing them. But now I'm wondering if it's worth it.
Because the context window is bigger now.
Hilbert: That's what I'm told.
Hilbert, your index cards are... what, twenty-nine years old?
Hilbert: Twenty-nine years. The paper's holding up fine. The ink's a little faded.
What was QueryPoint searching?
Hilbert: Academic paper abstracts. Pre-print servers, mostly. Physics. We had about forty thousand abstracts indexed by the time the money ran out.
You personally decided which forty thousand.
Hilbert: Me and a guy named Rick. Rick had opinions about particle physics. We'd argue about which terms deserved the RAM. Spent more time arguing than indexing, probably.
Do you still talk to Rick?
Hilbert: He owes me four hundred dollars. So no.
The point about triage being a permanent instinct — I think that's actually the right framing. The specific techniques change, but the need to prioritize information doesn't go away.
Hilbert: The cards worked fine. You could flip through them faster than any computer I've used since. Instant random access. No attention degradation.
You're saying your index cards had better retrieval performance than a transformer.
Hilbert: I'm saying I knew where everything was. Still do. Box two, third divider, "quark-gluon plasma."
Hilbert, you've just described a retrieval system that outperforms our most advanced AI architectures. With shoeboxes.
Hilbert: Shoeboxes don't have a lost-in-the-middle problem. The middle is just more cards.
I have so many follow-up questions and I'm not sure I want the answers to any of them.
Hilbert: The cards are organized by how often I needed them. High-frequency terms up front. Low-frequency in the back. It's the same thing you're talking about — put the important stuff where you can find it.
That's literally an attention-aware structuring strategy. You were doing in 1997 what we're now building into agent architectures.
Hilbert: Wasn't complicated. You just pay attention to what you actually use.
Hilbert, are you going to digitize the cards or not?
Hilbert: Probably not. The whole point was they worked without electricity. If I put them in a computer, I'm just trading one set of problems for another.
That might be the most coherent thing anyone's said this entire episode.
Hilbert: I'm going to go home now. Rick still owes me that money.
The ceiling is going to move. The question is whether our architectures are ready to move with it, or if they're just waiting for it to crack. And I think the answer we've landed on is: some of them are, some of them aren't, and the difference comes down to whether you've been solving for tokens or solving for attention.
The next few years are going to be revealing. If someone cracks the attention scaling problem — truly cracks it, not just makes the window bigger — then a lot of today's practices really do become retro. But if attention doesn't scale, we're going to look back at the million-token era as a kind of golden age where the constraints were at least legible.
The hard constraint was also the clear constraint. You knew when you were out of tokens. The attention problem is murkier, and murkier problems produce murkier failures.
That's the open question. When the ceiling lifts, do we get a renaissance of simpler, single-agent systems that can just hold everything in context? Or do we discover that multi-agent orchestration was never really about the ceiling — that it was solving deeper problems of focus, isolation, and coordination that survive any window size?
I suspect the answer is both. Some workflows collapse back into simpler architectures. Others get more complex, because the agents are more capable and the stakes of coordination are higher.
We'll find out. Probably sooner than we expect. The breakthrough could land any day.
Thanks to Hilbert Flumingtop for producing, and for reminding us that the best retrieval system might be three shoeboxes in a garage.
This has been My Weird Prompts. We're at my weird prompts dot com. Email the show at show at my weird prompts dot com.
We'll be back soon. Try not to lose anything in the middle.
