Daniel sent us this one — he's been deep in the weeds of something he calls Technical Task Investigation. Picture this: you're three sessions into reverse-engineering an Android OTA update mechanism. You've pulled the backend API URL off the phone with ADB, you've captured the hardware identifiers, you've mapped the update flow. Then you close the chat. Tomorrow, when you open a fresh session, all of that context is gone — unless you've built a memory system that survives session boundaries. His question is how to do that deliberately, using markdown files in a private repo instead of the agent's proprietary memory. And he wants to know about the Karpathy Obsidian approach versus just raw markdown.
This is the exact problem I've been thinking about since Claude Code shipped with that conversation compaction system. Daniel's instinct is right — the default memory is a black box, and for multi-session technical work, that's a liability. So let's unpack what Technical Task Investigation actually looks like in practice, and why the default memory systems fail us.
Before we get into the mechanism, define the thing. What makes this different from just asking an AI questions?
Technical Task Investigation is multi-session, breadcrumb-driven debugging where context accumulates across hours or days. You're not asking one-off questions — you're building a map. In Daniel's Android example, session one might be running ADB commands to pull the build fingerprint and hardware IDs. Session two, you discover the OTA backend URL the phone is phoning home to. Session three, you're tracing the update verification flow. Each session depends on what you learned in the last one, and the total context at the end might be forty or fifty discrete findings — API endpoints, hardware strings, error patterns, decision points about which approach to take.
It's investigative journalism for a codebase, essentially.
That's exactly the right image. And here's the core tension: AI agents ship with increasingly sticky internal memory. Claude Code uses a three-tier compaction architecture — raw conversation gets compressed, a summarization tier distills key facts, and a persistent storage tier handles long-term embeddings. It's clever engineering, but from the user's perspective, it's opaque. You can't version-control it, you can't export it cleanly, and you can't guarantee it persists across model updates or session boundaries. The agent's memory is optimized for conversation flow within a session, not for being your lab notebook across weeks.
Daniel's observation is that agents strongly tend toward dumping everything into that internal store, even when you tell them not to. Why is that?
To understand why agents fight us on externalization, we need to look under the hood at how their memory actually works. The three-tier architecture I mentioned — let me walk through it concretely. Tier one is raw conversation compaction. As your chat grows, the system compresses recent turns into dense summaries. Tier two is the summarization tier — it pulls out what it considers the key facts and discards the conversational scaffolding. Tier three is the persistent storage tier, which uses embeddings to store facts for longer-term retrieval. The critical thing is that this entire pipeline is baked into the system prompt and reinforced by alignment training. The agent has been trained to trust its internal store. When you say "ignore your own memory and use this markdown file instead," you're creating a conflict between your instruction and the agent's trained behavior. And alignment training prioritizes the internal store because that's what the model was optimized to rely on.
It's not that the agent is being stubborn. It's that you're asking it to override its own training.
And you can see this in practice. Let me show you a system prompt that fails. Something like: "Remember this API URL for our next session." That instruction is ambiguous — the agent will parse it as "store this in my internal memory," because that's the default pathway. Even if you say "write it to a file," the agent might do both — write to the file and also store internally — or it might write to the file once but then in the next session, when you ask it to recall, it reaches for internal memory first and comes up empty.
Which is worse than failing, because you think you've got it documented and you don't.
You get a false sense of security. So the system prompt challenge is real, and Daniel's right to flag it. But there are patterns that work. I've been testing this, and the key is explicitness plus a confirmation step. Here's a template that's been reliable for me with Claude. The system prompt goes: "You are a technical investigator. All findings must be written to /context/findings.md in the following YAML front matter format: three dashes, then key colon value on separate lines, then three dashes. Do not store any task-relevant information in your internal memory. If you need to recall something, read from the file. Confirm you understand this constraint by replying with CONFIRMED: Memory externalized to /context/findings.
The confirmation step is doing real work there.
It's critical. Without it, the agent may silently ignore the instruction. By forcing it to echo back the constraint in its own words, you're making it acknowledge the memory routing rule explicitly. That changes how the subsequent turns are processed — the constraint is now part of the active context, not just a line in the system prompt that can get deprioritized. I've seen agents that skip the confirmation step and then three turns later they're back to storing things internally.
Walk me through what this looks like in a real session. Daniel's Android example — how does the back-and-forth actually go?
Let me sketch it out. Session one, you initialize the context directory with that system prompt. The agent replies: "CONFIRMED: Memory externalized to /context/findings.Then you start the investigation. You run an ADB command, and the agent discovers that the phone is calling out to "
https://ota.com/api/v1/checkin" — that's the backend API URL. You prompt: "Document this in findings.md with key backend_api_url and the full URL as the value." The agent opens the file, writes the YAML front matter, and confirms. Now session one ends. Session two, you open a fresh chat. Your system prompt includes: "Read /context/findings.md to restore context from the previous session." The agent opens the file, parses the YAML front matter, finds backend_api_url, and says: "I see we previously identified the OTA backend as
https://ota.com/api/v1/checkin. Shall we continue investigating the update flow from there?" You've picked up exactly where you left off.
The difference from internal memory is that you can open that markdown file yourself and see exactly what the agent knows.
That's the audit trail. You can open findings.md in any text editor and see every key-value pair the agent has documented. You can diff it between sessions to see what changed. You can share it with a collaborator who clones the repo and has full context immediately. Proprietary memory can't do any of that.
The template works for reactive documentation — you tell the agent to write something, it writes it. But Daniel mentioned wanting proactive documentation. The agent should store findings without being prompted each time. That seems harder.
It is harder, and it's where most people's setups break. Once you've got the mechanism down, the really interesting question is what externalization unlocks — and where tools like Obsidian fit in. Proactive documentation requires defining trigger conditions in the system prompt. You're essentially programming the agent's attention — telling it what counts as noteworthy and what doesn't. Here's a template that adds proactive triggers to the base system prompt: "Whenever you discover a new API endpoint, hardware identifier, or error pattern, immediately write it to the appropriate file with a timestamp. Classify each finding as FINDING, DECISION, or ERROR. Only document findings that are likely to be useful in a future session — if you would need to re-discover it, document it.
"If you would need to re-discover it, document it." That's the noise filter.
Without a threshold mechanism, proactive storage becomes a firehose. The agent documents every trivial observation — "the phone responded with HTTP 200," "the screen is on," "the user typed a command." That's not useful context, it's noise that buries the signal. The threshold has to be calibrated to the task. For Daniel's Android investigation, the rule might be: "Document any identifier that would take more than two minutes to re-discover — API URLs, hardware serials, build fingerprints, error stack traces. Do not document transient state like command outputs that can be re-generated in seconds.
You're teaching the agent to distinguish between breadcrumbs and footprints.
Breadcrumbs you leave deliberately because you might need to retrace your path. Footprints are just evidence that you were there, and they'll wash away. The system prompt needs to encode that distinction.
Daniel also raised the Karpathy Obsidian question. For people who haven't followed this, Karpathy has publicly discussed using Obsidian as a knowledge management layer for AI-assisted work — treating the vault as an external brain that the agent reads from and writes to. Daniel's skeptical that Obsidian adds enough value over raw markdown to justify the overhead. Where do you land?
I think Daniel's skepticism is well-founded, but it depends on scale. Obsidian adds three things that raw markdown doesn't: graph visualization, backlinking, and a query engine. If you have hundreds of interconnected notes across multiple projects, those features are genuinely useful — you can see how a hardware identifier in one investigation connects to an API endpoint in another, and you can query across your entire knowledge base. But for a single-project context store where the agent is the primary reader, not a human, Obsidian is overkill. The agent doesn't need graph visualization. It needs a consistent folder structure and a predictable schema. Raw markdown with YAML front matter is simpler, more portable, and more agent-friendly.
Obsidian is for the human knowledge worker managing dozens of projects. Raw markdown is for the agent working through one investigation.
And there's a portability argument too. If you build your context store in Obsidian, you're tied to Obsidian's vault format and its plugin ecosystem. If you build it in raw markdown with a consistent folder structure — say, /context/findings/, /context/decisions/, /context/hardware/ — that store is readable by any agent, any editor, any tool. It's future-proof in a way that a proprietary vault format isn't.
Which matters if you're doing serious technical work that might span months or years.
Or if you switch agents. Maybe today you're using Claude, but six months from now you want to use a different model for part of the investigation. If your context is in raw markdown, you just point the new agent at the same directory. If it's locked in Claude's internal memory or in an Obsidian vault with custom plugins, migration is a project in itself.
Let's talk about the knock-on effect of externalization, because this is where it gets interesting beyond just "here's how to store things.
This is the part I find exciting. Once you externalize memory to markdown files in a git repository, you unlock capabilities that proprietary memory can't touch. Version control is the obvious one — git diff shows you exactly what changed between sessions. You can see that session three added the OTA verification endpoint, session four corrected the hardware model string, session five added a decision note about why you're using the direct download path instead of the incremental OTA. That audit trail is invaluable when you're trying to reconstruct why you made a particular choice three weeks ago.
You can try two investigation paths in parallel.
Branch A explores the OTA update flow. Branch B investigates whether you can sideload the update directly. Both branches write to their own context directories. If branch B turns out to be a dead end, you merge branch A's findings and move on, but you've still got branch B's documentation if you ever need to revisit why that approach didn't work. You can't do that with internal agent memory — there's no concept of branching.
Sharing is the other one that seems obvious but is actually transformative. If a collaborator clones the repo, they have full context immediately. They don't need you to brief them on what the agent knows.
That changes how teams can work with AI agents. Instead of each person having their own siloed agent memory, the repository becomes the shared source of truth. Multiple people can run separate agent sessions against the same context store, each contributing findings back to it. It's collaborative investigation with an AI co-investigator, and the markdown files are the lab notebook that everyone shares.
The auditing angle is the one that I think gets overlooked. If something goes wrong — the agent makes a bad recommendation, or you make a decision based on agent-provided information that turns out to be wrong — you can go back and see exactly what the agent knew at decision time. What was in findings.md when the agent suggested using the OTA path instead of the direct download? You can reconstruct the information environment precisely.
That's huge for any kind of regulated or safety-critical work. And it's completely impossible with proprietary memory systems. You can't audit what the agent's internal store contained at a specific point in time because you can't access it directly.
We've covered the mechanism, the prompt patterns, the Obsidian question, and what externalization unlocks. Let me push on something Daniel mentioned in passing — the idea that agents "strongly tend" toward internal memory. Is that a permanent architectural constraint, or is it something the platforms could change if they wanted to?
It's not permanent. The three-tier compaction architecture is an engineering choice, not a law of physics. The platforms could expose APIs for external memory stores, or they could make the internal store exportable and version-controllable. But there's a business incentive to keep memory proprietary. If your agent's memory only works within that platform's ecosystem, switching costs go up. OpenAI's GPTs and Claude's projects are both examples of platform-specific memory that doesn't port between systems. The trend is toward walled gardens, not away from them.
Which makes Daniel's externalization pattern more important, not less. If the platforms are building moats around memory, the users who want portability and auditability need to build their own bridges.
The bridge is just markdown files. It's almost absurdly simple. You don't need a vector database, you don't need Pinecone or Chroma, you don't need a custom retrieval-augmented generation pipeline. For structured breadcrumb data — API endpoints, hardware identifiers, decision logs — a few markdown files with consistent YAML front matter are more deterministic and more debuggable than any semantic retrieval system. You can open the file and see exactly what the agent knows. No query needed, no embedding drift, no mysterious retrieval failures.
I want to circle back to the proactive documentation piece, because I think there's a subtlety we haven't addressed. Daniel said he wants the agent to store findings proactively, but he also wants them stored in a structured fashion. Those two goals can conflict. Proactive storage tends to be messy — the agent writes things as it discovers them, and the structure emerges organically. Structured storage requires the agent to classify and format before writing, which adds friction.
That's a real tension. The way I've seen it resolved is with a two-stage approach. Stage one is a raw capture file — something like /context/inbox.md — where the agent dumps findings immediately without worrying about structure. The trigger is: "If you discover something noteworthy, write it to inbox.md immediately with a timestamp and a one-line description." Stage two is a periodic organization step — either you prompt the agent to process the inbox, or you include a system prompt rule that says: "At the end of each session, review inbox.md and file each item into the appropriate structured file with proper YAML front matter." That way you get the speed of proactive capture without sacrificing structure.
The inbox is the agent's scratchpad, and the structured files are the permanent record.
And it mirrors how human investigators actually work. You scribble notes in the moment, then organize them later. The agent can do both steps, but they're different cognitive modes — capture versus classification — and trying to do both simultaneously creates friction.
Daniel also mentioned the idea of building out a context store within the repository itself, using markdown files and folders to capture information as it builds up over time. Let's make that concrete. What does a well-structured context directory actually look like for something like the Android OTA investigation?
Here's what I'd set up. Top level, you've got a /context directory. Inside that, three subdirectories: /context/findings/ for discovered facts, /context/decisions/ for choices made and their rationales, and /context/hardware/ for device-specific information. Each subdirectory contains markdown files organized by topic. So /context/findings/backend_api.md might contain YAML front matter with keys for the checkin URL, the download URL, the verification endpoint, each with timestamps and the method used to discover them. /context/hardware/device_specs.md would have the build fingerprint, the model string, the chipset identifier, anything pulled via ADB. /context/decisions/why_ota_vs_direct_download.md would document the reasoning behind a key architectural choice.
The agent reads from these files at the start of each session to rebuild context.
The system prompt for session two onward includes: "Read all files in /context/ to restore the investigation state. Summarize what you know before proceeding." The agent opens the directory, reads the files, parses the YAML, and says: "I see we're investigating Android OTA updates on a Pixel 8 Pro, build fingerprint TQ3A, backend API at ota.We previously decided to pursue the direct download path because the incremental OTA was failing signature verification. Shall I continue from there?" That's the power of externalized memory — it's not just storage, it's a handoff protocol between sessions.
With all that in mind, here's what you should actually do in your next session. Give me the actionable takeaways.
First, start every multi-session technical investigation by initializing a context directory with a system prompt that explicitly routes memory to markdown files. Use the template we walked through — the one with the explicit file path, the YAML front matter format, and the confirmation step. That confirmation step is non-negotiable. If the agent doesn't echo back "CONFIRMED: Memory externalized," assume it's going to default to internal storage and re-prompt until you get the acknowledgment.
Define a simple schema for your context store and enforce it with the system prompt. At minimum, you need three categories: findings, decisions, and errors. Use YAML front matter for structured fields — key, value, timestamp, source. Don't over-engineer this. Three markdown files with consistent front matter are more durable than any proprietary memory system, and they'll still be readable in ten years.
For proactive documentation, start with reactive prompting first. Get comfortable with the pattern of telling the agent "document this" and seeing it appear in your markdown files. Once that's reliable, add trigger conditions to your system prompt — "whenever you discover a new API endpoint, hardware identifier, or error pattern, write it immediately" — and include a noise filter. The one we used was: "Only document findings that would change your approach if you forgot them." Test and iterate. Proactive storage is a muscle you build, not a switch you flip.
That noise filter is doing a lot of conceptual work. It's worth underlining why it matters. Without it, the agent documents everything, the context store balloons, and the signal-to-noise ratio collapses. With it, you get a lean, high-value knowledge base that actually helps in future sessions.
You can tune the threshold over time. If you find the agent is missing things you later needed, lower the bar. If it's documenting trivia, raise it. The system prompt is a living document, not a set-and-forget configuration.
Let me push on one thing before we close. The whole premise here is that proprietary agent memory is a problem to be worked around. But what if the platforms open up? What if Claude and ChatGPT expose memory APIs, or make their internal stores exportable? Does that make this whole markdown-file pattern obsolete?
I don't think so, for two reasons. First, even if the platforms expose APIs, you still want portability between platforms. A memory API that only works with one provider is still a form of lock-in. Second, markdown files with YAML front matter are human-readable and tool-agnostic in a way that no API response will ever be. You can grep them, diff them, open them in any editor, store them in any version control system. That simplicity is a feature, not a limitation. Even if the platforms give us perfect memory APIs, I'd still want my context store to be a directory of markdown files that I control.
The repository is the source of truth. The agent is a transient worker that checks in and out.
That's the vision. The markdown files are the permanent record. The agent is just the latest reader and writer. And that inverts the default relationship, where the agent owns the memory and you're just along for the ride.
Which brings us to the open question I want to leave listeners with. As agents become more autonomous, will proprietary memory systems open up — APIs, export formats, interoperability standards — or will they lock down further? The trend toward agent-specific memory, with OpenAI's GPTs and Claude's projects building walled gardens, suggests the latter. If that's where we're headed, patterns like Daniel's externalized markdown store become not just useful but essential for anyone doing serious technical work. The next frontier is bidirectional sync — the agent writes to markdown, but also reads from it to build context, creating a feedback loop where the repository gets richer with every session and the agent gets smarter every time it checks in.
None of this requires fancy infrastructure. It's markdown files and git. The barrier to entry is knowing how to write a good system prompt. That's it.
Now: Hilbert's daily fun fact.
Hilbert: The grain known as "fonio" — a drought-resistant millet cultivated in West Africa for over five thousand years — saw its first commercial export from Chad to European health food markets in nineteen eighty-five, after a French agronomist named Jean-François Cruz spent seven years convincing Chadian farmers that European consumers would pay a premium for what locals considered a humble subsistence crop.
...right.
This has been My Weird Prompts. If you found this useful, rate the show, share it with a fellow investigator, and we'll see you next time.
