Alright, we've got a good one today. Most AI agents you meet are basically amnesiacs—they have a brilliant conversation with you and then completely forget you exist the next time you chat. Today's prompt from Daniel wants us to dig into a project that's trying to fix that exact problem: Letta.
Herman Poppleberry: This is a fantastic topic. As agents move from being cool demos to actually running production workloads, that persistent memory issue isn't just an annoyance, it's the main thing holding them back from being truly useful. Letta, from the team behind MemGPT, is one of the most interesting approaches I've seen because it treats memory not as a feature, but as the foundation.
So it's an agentic backend, but with a memory-first architecture. What immediately sets it apart from something like LangChain or CrewAI?
The core difference is philosophical. Most frameworks treat the agent's memory as an appendage—you've got your vector database for retrieval, maybe some way to save state between steps. Letta flips it. The agent's persistent, structured memory is the central system. The reasoning, the tool use, the planning—they all orbit around this constantly updating memory store. It’s built from the ground up to be stateful.
That sounds like a heavier lift. Is the trade-off worth the complexity?
For the right use case, absolutely. Think about a traditional RAG system. You shout a query into a library and hope the right book falls off the shelf. The agent doesn't really own that information; it's just borrowing it for a moment. Letta's model gives the agent its own memory palace. It can write things down, organize them, revisit and revise its own understanding over time. This isn't just retrieval, it's cultivation.
Okay, so the agent isn't just a stateless function caller anymore. It's building a model of the world, or at least its specific domain, that persists. How does that actually work under the hood? What's a memory block?
Right, the memory block is the key innovation. It's a structured, queryable data store that lives outside the LLM's context window. The agent interacts with it through function calls—it can write new memories, search existing ones, update them, or create relationships between them. These blocks are persistent across sessions, tasks, even days or weeks. And they're not just blobs of text; they have structure, metadata, and can be semantically searched.
Can you give us a concrete example of what a memory block actually contains? Like, if I asked an agent to remember my coffee order, what would that block look like beyond just the text "large oat milk latte"?
Great question. It would be a structured JSON object. It might have fields like entity: "user_preference", key: "coffee_order", and value: "large oat milk latte". But crucially, it could also have context: "ordered every Tuesday morning" and source: "user stated on 2024-03-12". It could link to other blocks—like a block about your general dietary preferences or your past feedback on coffee strength. The structure allows the agent to query and reason about these facts relationally, not just by keyword matching.
So if I'm building a customer support agent with Letta, it's not just looking up past tickets in a database. It's maintaining an active, evolving understanding of each user?
Exactly that. It could remember that a user named Sarah prefers detailed explanations, had an issue with billing last month that was resolved, and is generally polite but gets frustrated after long hold times. That's not just a retrieved fact from a log; it's a synthesized persona model the agent builds and refines over every interaction. The next time Sarah calls, the agent starts from that understanding, not from zero.
That's powerful. It also feels a bit... intimate. The agent is forming a persistent, detailed model of a person.
It is, which raises immediate questions about privacy and data governance, which Letta has to handle. But technically, it enables a level of personalization that's otherwise impossible. The agent learns and adapts to the individual, not just the conversation. Imagine a learning tutor AI that remembers not just what a student got wrong, but how they approached the problem three weeks ago, and can see if their problem-solving strategy has evolved. That depth of continuity is new.
You mentioned a persona system. Is that separate from the memory blocks?
It's built on top of them. The persona is essentially a specialized memory block—or a collection of them—that represents the agent's self-model and its understanding of its role. This can evolve. An agent tasked with research might start with a persona of "curious assistant," but over time, as it succeeds and fails, it might update that to "methodical analyst who double-checks sources." This isn't just a static system prompt; it's a living part of the agent's memory.
So it's not just learning about the task, it's learning about itself performing the task. That's meta.
Precisely. And that self-model can influence future actions. If its memory tells it "the last three times I made a factual claim without citing a source, the user asked for verification," it might update its persona to include "prefers citations" and automatically start providing them. It's a form of low-level, continuous self-improvement based on lived experience.
That's the self-improvement angle. By having a persistent record of what worked and what didn't, it can tune its own approach. Let's get practical. What does this look like in a real workflow? Walk me through a research agent built with Letta.
Sure. You deploy it on a long-running project, say, tracking developments in solid-state batteries. Day one, it's given some initial sources and a goal. It reads papers, summarizes them, and stores those summaries in memory blocks, tagging them with concepts like "anode materials" or "manufacturing challenges." It also logs its own actions: "Tried to compare X and Y papers, found conflicting data on cycle life."
So it's building a knowledge base, but also a meta-memory of its own research process.
Right. A week later, you ask it about a new breakthrough. Instead of just doing a fresh web search, it first queries its own memory: "What do I already know about sulfide electrolytes?" It pulls up its previous summaries, notes the gaps it identified, and then directs its new search to fill those gaps specifically. Over time, it becomes less of a search assistant and more of a domain expert with its own curated knowledge graph.
That's a compelling shift from reactive tool use to proactive knowledge management. But what's the cost? This sounds more computationally expensive than a simple, stateless agent that just calls a search API.
It is. There's overhead. You're running more LLM calls to manage the memory—deciding what to store, how to structure it, when to retrieve. It's not the right tool for high-throughput, stateless tasks. If you need to process ten thousand support tickets an hour with a simple classification, you don't want this overhead. But if you need one agent to manage a complex project over six months, learning as it goes, that's where Letta shines. The trade-off is latency for longitudinal intelligence.
So the sweet spot is long-running, stateful, and preferably complex workflows. Personal AI assistants, project co-pilots, research companions, maybe even creative writing partners that develop a sense of your style.
That's the niche. It's for when you want the agent to develop expertise, not just execute a function. The January update made this even more interesting by adding multi-agent memory sharing.
Wait, they can share memories?
Yes. Different agents, say a research agent and a writing agent, can now access and contribute to shared memory blocks. The researcher fills a block with facts and citations; the writer accesses that same block to draft a report, adding notes about tone and structure. They're collaborating through a shared, evolving memory space. It's a step toward truly collaborative AI teams.
That's a fascinating second-order effect. It moves the coordination problem from message passing between agents to managing a shared truth. But how do they avoid conflict? What if the research agent writes "the sky is green" and the writing agent needs to use that?
That's where the structure and metadata come in. Memories aren't just taken as gospel; they have provenance. The writing agent could see that the "sky is green" memory came from a source labeled "low confidence" or from a specific, perhaps dubious, research paper. It could then decide to flag it for review or seek corroborating memories. The shared memory becomes a living document with a history, not a simple bulletin board.
How does this compare to something like LangGraph, which is all about orchestrating complex, cyclic workflows?
Great question. LangGraph is brilliant at defining the process—the steps, the cycles, the decision points in a workflow. It's about control flow. Letta is focused on the state—the knowledge and context that persists through that flow and beyond it. You could almost see them as complementary. LangGraph manages how the agent moves; Letta manages what it knows and remembers along the way. In practice, you might use Letta for the persistent memory layer within a LangGraph-defined agent. The orchestration framework handles the sequence, and Letta provides the long-term memory for each node in that graph.
So it's not necessarily an either-or. You might use Letta's memory system inside an agent orchestrated by another framework. That brings us to a practical takeaway for developers listening. If I'm building an agent today and I'm hitting memory limits, what should I do? Rewrite everything in Letta?
Probably not a full rewrite. But evaluating Letta's memory block concept is worthwhile. Even if you stay on your current framework, understanding this memory-first approach might inspire how you structure your own data stores. For a new project that's clearly in that long-running, stateful sweet spot, starting with Letta could save you from bolting on a memory system later. They have a playground you can try, and their API is designed to be integrated.
And it's open source, which is huge for understanding the mechanics. Speaking of practical use, you mentioned Letta Code. Can you expand on that? How does a memory-first coding assistant differ from my current Copilot?
Glad you asked. A standard Copilot is essentially stateless per session. It might have some broad fine-tuning on public code, but it doesn't remember your patterns from yesterday. Letta Code sits in your IDE and builds a memory of your project. It learns that you always write validation functions a certain way, that you prefer a particular testing framework pattern, that module X is fragile and your comments often say "handle with care." Over weeks, it becomes hyper-personalized. It's not just completing lines; it's completing lines in the style of your codebase, and it can reference architectural decisions you made months ago that are now in its memory palace.
That sounds like it could dramatically reduce the "context burden" of pasting in files or explaining your project structure every time.
The agent carries that context forward. It's like having a new junior dev who actually reads the entire codebase history and remembers every discussion, versus one you have to re-brief every morning.
It's interesting that this comes from Mem.ai. They've been thinking about personal memory management for years, just for humans. Now they're applying that philosophy to AI.
It makes perfect sense. Their whole thesis is that useful intelligence, human or artificial, is built on memory. The CEO, Charles Packer, has said that all the powerful agent characteristics—personalization, self-improvement, reasoning—are fundamentally memory management problems. When you frame it that way, building memory-first isn't an optimization; it's the core requirement.
That leads to a bigger question. Do you think this memory-native architecture will become the standard, or will it remain a specialized tool for certain kinds of agents?
I think we'll see a bifurcation. For simple, transactional agents—order a pizza, check the weather—stateless is fine, maybe preferable. But for any agent that's meant to be a persistent collaborator, a coach, a teammate, memory will become non-negotiable. The trust factor alone demands it. If your project management agent forgets what it decided yesterday, you'll never rely on it. As we ask agents to do more complex, multi-session tasks, Letta's approach, or something like it, will stop being a niche and start being the expected foundation.
It also forces us to think about what we want these agents to remember. Curating memory becomes a new design challenge. You don't want your agent remembering and reinforcing every mistake, or developing a weird bias based on early data.
A hundred percent. Memory management is now an AI safety and alignment issue. Letta gives you transparency—you can view and edit the memory palace—which is a good start. But it's a new layer of complexity. The agent isn't just executing code; it's forming a worldview. That's a profound shift. You need mechanisms for memory decay, for prioritizing important memories, for conflict resolution when memories contradict. It’s a whole new field of agent design.
Fun fact that ties into this: The concept of a "memory palace" we've been using isn't just an analogy. It's a real mnemonic technique, the method of loci, used since ancient Greece. Orators would mentally place information in specific rooms of a building to remember long speeches. Letta is essentially automating that for AI—building a digital, queryable memory palace. It's an ancient human trick, scaled by software.
I love that. And it highlights that we're not inventing a new concept of memory, we're just finally giving AI access to a very old and powerful human tool.
It makes the agent less of a tool and more of... a entity with a history. Which is exactly what makes it useful, and also a bit spooky.
That's the trade-off at the frontier. The more human-like in capability, the more human-like in complexity. Letta is tackling one of the biggest parts of that complexity head-on. By the way, today's script is being powered by DeepSeek v3.2.
Neat. So, final takeaway for someone listening who's maybe tinkering with agents? Where should they start with these ideas?
I'd say first, identify if your agent problem is stateful or stateless. If it's truly a single-session thing, you probably don't need this. But if you find yourself constantly pasting context back in, or wishing the agent remembered the last ten interactions, go look at Letta's documentation. Even just reading about their memory block system will change how you think about structuring agent data. And if you're into code, spin up Letta Code and see how a memory-first coding assistant feels different.
What about the learning curve? Is it a steep climb from, say, a simple OpenAI Assistants API implementation?
It's steeper, yes. You're managing a more complex system. But their docs are good, and the payoff for the right project is immense. Start by defining just one or two types of memory blocks you want to persist—like "user_preference" or "project_decision"—and build from there. Don't try to model the entire world on day one.
It feels like we're moving from the era of the AI conversation to the era of the AI relationship. And relationships, for better or worse, are built on memory.
This isn't just a technical upgrade; it's a fundamental change in the interaction model. We're moving from transactional queries to continuous collaboration. The agent that remembers you is the agent you come back to.
Well, thanks as always to our producer, Hilbert Flumingtop. And big thanks to Modal for providing the GPU credits that power this show.
If you're enjoying the show, a quick review on your podcast app helps us reach new listeners.
This has been My Weird Prompts. We'll catch you next time.
