Daniel's been spending a lot of time in Claude Code sessions that run for hours, and he's watching the context window fill up. Every tool call, every file read, every error message consumes tokens, and eventually performance degrades. He's looking at two levers you can pull to manage this. The first is a trailing context cutoff — a hard trim where everything below a certain token position just doesn't get sent in the next API call. The second is Claude's auto-compact threshold, which defaults to two hundred thousand tokens and triggers a built-in summarization step. Daniel's question is whether tinkering with that default is wise — he compares it to asking an arsonist not to touch the products in a firecracker warehouse, which, fair. He wants the pros and cons of setting it more aggressively so compaction happens sooner, versus leaving it right up until the window is nearly consumed. And he wants to know how both compare to the trailing cutoff approach.
Because the thing is — Claude's documentation does caution against messing with this setting, but it's not a "don't touch this or everything explodes" warning. It's more of a "we picked this number for a reason, and if you change it, understand what you're trading off." So let's start by making sure we're all on the same page about what these two levers actually are, because they're philosophically very different.
One's a guillotine and one's an editor.
That's... actually exactly right. The trailing context cutoff is the guillotine. You set a token position — say, everything below one hundred thousand tokens — and anything older than that simply isn't included in the next API call. No summarization, no intelligent preservation, no recovery. It's common in tools like Open WebUI, but you almost never see it in commercial products because it's so blunt. The auto-compact mechanism is the editor. When the conversation reaches the threshold, Claude spawns a separate model call that reads the entire conversation history and produces a compressed summary. That summary then replaces the original conversation text in the context window. The original is stored alongside it, so you can inspect it or reverse the compaction if needed.
So the guillotine is permanent data loss, and the editor is lossy but reversible.
Right. And the threshold setting — that two hundred thousand token default — controls when the editor gets called in. Set it lower, say one hundred fifty thousand, and compaction fires earlier and more often. Set it higher, say two hundred fifty thousand, and compaction fires later, if at all, but you're running closer to a full context window the whole time.
Which is where the "lost in the middle" problem lives.
And that's the thing — the window doesn't have to be completely full for performance to degrade. Transformer-based models have this documented phenomenon where information in the middle of a long context is less reliably accessed. So even at one hundred eighty thousand tokens, with a two hundred thousand default threshold, compaction hasn't fired yet, but the model may already be struggling to recall details from turns fifty through one hundred fifty. You're in this weird limbo where the window isn't full enough to trigger compaction, but it's full enough to degrade performance.
And Daniel's asking whether you should just lower the threshold and get compaction over with earlier, before that limbo even starts.
Let's pop the hood on the mechanism itself first, because understanding what actually happens during compaction is where most of the tradeoffs become visible. When the conversation hits the threshold, Claude Code makes a separate model call — it's not the same call that's handling your coding session. That separate call reads the entire conversation history and produces a summary. The architecture here is three-tier. You've got the raw conversation history — every turn, every tool call, every file read, every error. Then you've got the compacted summary that replaces it. And then you've got the working context, which is untouched — your active code, pending edits, the current instruction buffer. Compaction only touches the history portion.
So the thing you're currently working on doesn't get summarized out from under you.
Right. The working context is preserved verbatim. What gets compacted is everything that led up to this point. And here's where the threshold setting gets interesting. If you set it to one hundred fifty thousand tokens, compaction fires earlier. That means the summarization model has less material to compress — maybe one hundred forty thousand tokens of actual conversation rather than one hundred ninety thousand. Less material to compress generally means higher-quality summaries with fewer omissions. But it also means compaction fires more often over a long session, and each compaction is a lossy step.
And lossy steps compound.
That's the key risk. Say you're eight hours into a debugging session. At one hundred fifty thousand threshold, you might compact three times. Compaction one happens at turn eighty — it summarizes your early setup work, and in doing so, it drops the exact syntax of an API call you made, keeping only "set up the database connection." Compaction two happens at turn one hundred sixty — it summarizes the first debugging pass, and now the summary of compaction one is part of what it's summarizing. If compaction one already lost the API syntax, compaction two can't recover it. Compaction three at turn two hundred forty is now building on two layers of lossy compression.
It's a telephone game where each round loses a little more signal.
And the thing that gets lost might not matter — until it does. That API syntax from turn eighty might be exactly what you need at turn three hundred when you're trying to understand why a particular fix was applied four hours ago. The compaction summary might say "there was an error with the database connection," but it won't have the specific SQLSTATE code or the stack trace.
So more frequent compaction keeps the window cleaner but risks compounding information loss across multiple passes. What about the other direction — leaving the threshold at two hundred thousand or even pushing it higher?
The pitfall there is that the window can get nearly full before compaction fires. At two hundred fifty thousand threshold, you might hit two hundred thirty thousand tokens before the mechanism kicks in. At that point, the model is already deep in "lost in the middle" territory — information from the middle of the session is being recalled less reliably, even though it's technically still in the context. And when compaction finally does fire, the summarization model now has to compress a massive amount of text. Two hundred thirty thousand tokens is a lot of material to summarize accurately. The risk of important details being lost in that single compression step is higher.
And if that single compression step produces a bad summary, all subsequent reasoning is built on a flawed foundation.
There's a failure cascade here that's worth tracing. Imagine a session at two hundred fifty thousand threshold. Compaction fires once, at turn three hundred. The summarization model has to compress two hundred thirty-seven thousand tokens of history. It drops the entire debugging chain from turns fifty through one hundred twenty — all the false starts, the dead ends, the error messages that led to the correct fix. The summary says "the database connection issue was resolved by updating the configuration." But it doesn't say why that configuration change was chosen, what alternatives were tried and failed, or what specific error prompted the change. Three hours later, a similar error appears. The model can't explain why the original fix was applied because that reasoning chain is gone. It wasn't cut off — it was summarized away.
So the aggressive approach risks death by a thousand cuts, and the conservative approach risks one big amputation.
That's the tension. And it's why the default of two hundred thousand exists — it's a compromise. Not so low that you're compacting constantly and compounding errors, not so high that you're running the window to the edge and risking a bad single compression.
But Daniel's asking whether you should pull the lever anyway, and in which direction.
Let me walk through a concrete comparison. Two sessions, same eight-hour coding task. Session A has the threshold at one hundred fifty thousand. It compacts three times. The first compaction at turn eighty loses the exact syntax of an early API call, but preserves the fact that the call was made and what it accomplished. The second compaction at turn one hundred sixty preserves the corrected version of that API call, because by then the correction had been made and was part of the active working context that got summarized. The third compaction at turn two hundred forty has a coherent narrative of the session — some details are fuzzy, but the broad structure of what happened and why is intact. Session B has the threshold at two hundred fifty thousand. It compacts once, at turn three hundred, compressing two hundred thirty-seven thousand tokens. The single summary drops the entire debugging chain from turns fifty through one hundred twenty. The model can't explain why any of the early fixes were applied.
So Session A lost some trees but kept the forest. Session B lost whole sections of the forest and doesn't know they're missing.
And here's the thing about the trailing cutoff approach that makes this comparison even sharper. The guillotine doesn't summarize. It just drops everything below the line. If you set a trailing cutoff at one hundred thousand tokens, everything from turns one through whatever turn got you to one hundred thousand tokens is simply gone. Not summarized, not compressed — deleted from the next API call. There's no recovery path because the original text was never stored alongside anything. The auto-compact mechanism, for all its flaws, at least stores the original conversation. You can inspect it. You can say "wait, what was that error message from four hours ago?" and go look. With trailing cutoff, you can't.
So the editor leaves you a paper trail and the guillotine doesn't.
But — and this is where Daniel's question gets genuinely interesting — there are cases where the guillotine might actually be preferable. If the compaction model is weaker than the main model — say it's a smaller, faster variant — the summaries it produces might introduce inaccuracies. A bad summary that says "the user installed the wrong package version" when the actual error was a permissions issue is worse than no summary at all. At least with a clean cutoff, the model knows it doesn't know what happened before the cutoff point. It's not operating on false information.
A bad summary is a confident lie. A cutoff is honest ignorance.
And the confidence is the dangerous part. The model reads the summary and treats it as ground truth. If the summary is wrong, every subsequent reasoning step builds on that wrongness. The model doesn't know the summary might be inaccurate — it just sees text in its context window and reasons from it.
So the quality of the compaction model is a hidden variable in all of this.
It's the hidden variable. If the compaction model is as capable as the main model, the summaries are probably quite good, and the tradeoff favors more frequent compaction — keep the window clean, accept some minor information loss in exchange for avoiding the "lost in the middle" degradation. But if the compaction model is weaker, you're introducing a telephone game where each compaction degrades the signal in ways you can't predict. And you don't necessarily know which model is doing the compaction.
Which brings us back to the firecracker warehouse. Claude's documentation cautions against tinkering because they know the compaction model's capabilities and they've tuned the default threshold to match. If you lower the threshold, you're asking that model to do more work, more often. If the model is good enough, that might be fine. If it's not, you're making things worse while thinking you're making them better.
There's another dimension here that I think gets overlooked. The threshold isn't just about token counts — it's about the dependency distance in your specific session. How far apart, in turns, are two pieces of related information? In a debugging session, the dependency distance might be short. Error on turn fifty, fix attempted on turn fifty-one, fix verified on turn fifty-two. The whole chain fits in a few turns. Compaction can summarize that cleanly. But in an architecture session, the dependency distance might be enormous. A design decision on turn twenty doesn't get implemented until turn one hundred fifty, and doesn't get refactored until turn three hundred. If compaction fires at turn one hundred, before the implementation happens, it might summarize the design decision as "considered using a factory pattern" and drop the specific rationale. When the implementation happens fifty turns later, the rationale is gone.
The model implements the factory pattern without knowing why.
Or implements something else entirely because the summary didn't capture that the factory pattern was chosen for a specific reason that's still relevant. The threshold that works for a debugging session — compact early, keep the window clean, the dependencies are short — might be disastrous for an architecture session where you need to preserve rationale across hundreds of turns.
There's no single right answer. The optimal threshold depends on what kind of work you're doing.
You can't know in advance what the dependency distances will be. You might start a session thinking you're doing a quick bug fix, and three hours later you're refactoring the entire authentication system. The threshold you set at the beginning was optimized for short dependencies, and now you're in a long-dependency session with a threshold that's compacting too aggressively.
Which is maybe the strongest argument for leaving the default alone. Not because it's optimal for any specific session, but because it's a reasonable compromise for the session you don't know you're going to have yet.
I think that's right, but with a caveat. If you know what kind of session you're starting — if you're sitting down to do a multi-hour architecture refactor — you might be better served by raising the threshold. Accept that the window will get full, but preserve those long dependency chains. The single late compaction will be riskier, but the alternative is multiple early compactions that each snip one link in the chain.
If you're doing rapid-fire debugging where the dependency chains are short, lowering the threshold might make sense. Keep the window clean, accept that you'll lose some detail from early turns, but those early turns are less likely to matter later.
The arsonist analogy is actually perfect here, but not in the way Daniel framed it. It's not that tinkering with the threshold is inherently dangerous. It's that the danger depends on what you're standing next to when you strike the match. A debugging session is a puddle of water. An architecture session is the firecracker warehouse.
The real skill isn't avoiding the firecrackers. It's knowing which ones are worth lighting.
That depends entirely on what you're trying to build.
Hilbert: Nineteen ninety-nine. I was designing retention policies for a document management system at a firm that handled class-action litigation. Email archives. The question was always the same: how long do you keep what, and when do you summarize versus truncate?
That's... the exact same problem.
Hilbert: The lawyers who set their retention to thirty days were furious when the smoking-gun email from day thirty-one was gone. The ones who set it to ninety days ended up with such bloated archives that the search system crashed every Tuesday. There's no perfect number. Claude's two hundred thousand default is the system administrators saying "we picked something that works for most people." But every session has its own optimal threshold based on what kind of information you're generating and how far apart the dependencies are.
The dependency distance.
Hilbert: That's what we called it, yeah. Not in those words. We called it the "how far back does this matter" problem. In the legal archives, we discovered that the optimal retention period depended on how far apart in time two related documents typically appeared. For patent litigation, the dependency distance was enormous — a memo from nineteen ninety-five could be relevant to a filing from two thousand two. For slip-and-fall cases, it was maybe six weeks. Same principle applies here. The dependency distance between a variable declaration and its first use might be fifty turns. Between a library installation and its first import, two hundred turns.
The threshold should ideally be tuned to the session's dependency distance, which is impossible to know in advance.
Hilbert: Impossible, but not unpredictable. Debugging sessions have short dependency distances — error, fix, verify, all within a few turns. Architecture sessions have long ones — design decision, implementation, refactor, spread across hundreds of turns. You can't know the exact number, but you can know the shape of the session you're walking into.
Set the threshold accordingly before you start.
Hilbert: If you're paying attention to what kind of work you're about to do, yeah. Most people aren't. They just start typing.
That's the thing, though — sessions change shape. You start debugging and end up refactoring. The threshold you set for short dependencies is now wrong.
Hilbert: Same thing happened with the legal archives. A firm would set a thirty-day policy for routine correspondence, then get hit with a discovery order that reached back three years. The policy was fine for what they thought they were doing. It was wrong for what they actually ended up doing. We never solved that either. The best we could do was make it easy to change the policy mid-stream, and make sure people knew when the policy had changed and what they'd lost.
Which is exactly what the auto-compact mechanism does — it stores the original conversation so you can inspect what was lost.
Hilbert: The trailing cutoff approach doesn't give you that. It's the thirty-day retention policy with no backup. Once day thirty-one hits, the email is gone and nobody can tell you what was in it. At least with compaction, you've got the summary and the original in storage. You can go look.
Even if the summary is imperfect, you've got a paper trail.
Hilbert: Paper trails matter. I spent five years explaining that to lawyers who thought disk space was more expensive than losing a case. Disk space got cheaper. Losing a case didn't.
The dependency distance idea is actually a really useful lens for thinking about this, because it reframes the whole question. It's not "what threshold is optimal" in the abstract — it's "what threshold matches the dependency distance of the work I'm about to do."
If you don't know the dependency distance, the default is a reasonable hedge. Not perfect, but reasonable. The thing I keep coming back to is that the compaction model's quality is the hidden variable that makes all of this hard to reason about. If the compaction model is excellent, more frequent compaction is probably fine — the summaries will be accurate enough that the information loss is minimal, and you get the benefit of a cleaner context window. If the compaction model is mediocre, every compaction is a small degradation, and multiple compactions compound into something worse.
You don't necessarily know which model is doing the compaction.
You don't. Which is why the documentation's caution makes sense. They're not saying "never touch this." They're saying "we tuned this for the model we're using, and if you change it, the tuning no longer applies."
The misconception I think most people walk around with is that setting the threshold lower always preserves more information because it keeps the context window from filling up. The intuition is "compact early, compact often, keep things clean." But more frequent compactions mean more lossy compression steps, and those errors compound. You can end up with a cleaner window and less actual knowledge about what happened in your session.
The flip side misconception is that the trailing cutoff is strictly worse because it loses information. In some cases, a clean cutoff is better than a confident but wrong summary. At least with the cutoff, the model knows it doesn't know what came before. With a bad summary, it thinks it knows and it's wrong.
The default of two hundred thousand isn't optimal for everything. It's a starting point. The question Daniel's really asking is how to think about moving away from it, and the answer depends on what kind of session you're in, what the dependency distances look like, and how much you trust the compaction model to get the summaries right.
Whether you're willing to inspect the stored originals if something goes wrong. The paper trail is there. Most people never look at it.
Which raises an open question I've been turning over. Is there a way to make the threshold dynamic — to have Claude Code learn the optimal setting for a given project by analyzing dependency distances in real time? If the system could detect that you're in a long-dependency session, it could raise the threshold automatically. If it detects short dependencies, it could lower it. You wouldn't have to guess at the start.
That's a interesting feature idea. The challenge is that detecting dependency distance requires understanding the semantic relationship between turns that might be hundreds of steps apart. That's a hard problem. But not an impossible one — especially if the system is already summarizing those relationships during compaction. It has the data.
The arsonist in the firecracker warehouse is a good line. But maybe the real skill isn't avoiding the firecrackers. It's knowing which ones are worth lighting. And that depends entirely on what you're trying to build, and how far apart the pieces are.
Thanks to Hilbert Flumingtop for producing, and for the legal archives perspective that I'm going to be thinking about for the rest of the day.
This has been My Weird Prompts. If you enjoyed this episode, tell someone who's lost a four-hour debugging session to a bad context window. We're at my weird prompts dot com.
We'll be back soon.
