#5255: When AI Writes the Code, Who Checks It?

If agents write the code, the human job becomes knowing when it's wrong. What that means for languages, jobs, and CS degrees.

Featuring
Listen
0:00
0:00
Episode Details
Episode ID
MWP-5437
Published
Duration
23:52
Audio
Direct link
Pipeline
V5.2
TTS Engine
chatterbox-regular
Script Writing Agent
deepseek-v4-pro

AI-Generated Content: This podcast is created using AI personas. Please verify any important information independently.

The premise is simple and unsettling: agents have gotten good enough at writing code that the human role has already shifted from typing it to supervising the thing that types it. If that's true, the center of gravity in programming languages moves. Languages were optimized for human cognition — readability, expressiveness, how many concepts fit in your head. Python won because it maps to how people think. But if the primary author is an agent, the optimization target flips toward unambiguous generation, cheap verification, and mechanically provable properties. That's a different design space, and it points toward either constrained intermediate representations or human-friendly syntax sitting on top of a lower layer humans never touch — a direction WebAssembly already hints at.

The pushback worth taking seriously is about speed, not direction. Languages don't die quickly; COBOL still runs payroll, Fortran still anchors scientific computing. What changes is where new work happens. Meanwhile, the human skill that survives isn't syntax — it's judgment. Knowing what to build, whether it's correct for the problem, and how to recover when it's wrong. That requires deep systems understanding: you can't verify a distributed system without understanding consistency and failure modes.

Then comes the harder part. The junior developer role — writing straightforward code, fixing simple bugs, writing tests — is exactly what agents do well under supervision. If that work disappears, the entry-level pipeline breaks, and companies wake up with expensive seniors and nobody behind them. The proposed fix reframes a junior as someone who supervises agents on simple tasks, which means teaching delegation and verification from day one. But there's a floor of direct experience: if you've never built anything yourself, you have no alarm bell when the agent hands you something subtly wrong. The Dunning-Kruger problem gets supercharged when the tool makes everyone feel competent. What the big labs now ask for — designing evaluation harnesses, reasoning about failure patterns in agentic systems — points at the same conclusion. Evaluation is the core skill, and it's the hardest one to credential.

Downloads

Episode Audio

Download the full episode as an MP3 file

Download MP3
Transcript (TXT)

Plain text transcript file

Transcript (PDF)

Formatted PDF with styling

#5255: When AI Writes the Code, Who Checks It?

Corn
Daniel's been turning over something that's been nagging at him for a while now, and honestly it's the thing that's nagging at a lot of people who write software for a living. He's looking at how good agents have gotten at raw programming, and he's saying the human role has already shifted from typing the code to supervising the thing that types the code. And if that's true, he's wondering whether the whole idea of dozens of human-nurtured programming languages is basically on borrowed time. His prediction is consolidation around a small number of languages built from the ground up for AI agent optimization. Not languages designed for humans to read and write, but languages designed for agents to generate and verify. And then he asks the bigger question. If that's where we're heading, what does programming education become in a few years? There's still going to be a huge need for technical specialists, and also for people in the knowledge economy who aren't focused on the technological side. But if the boundary between technical and non-technical is no longer defined by who writes code, how does that division actually map out?
Herman
The first thing I want to push back on is the word obsolete, because I think Daniel's got the direction right but the speed wrong. Languages don't die quickly. COBOL is still running payroll systems and it's been declared dead since before I was born. Fortran is still the backbone of a huge amount of scientific computing. So what we're going to see isn't a clean break. It's a long tail.
Corn
That's the thing though. Daniel's not saying every language disappears overnight. He's saying the center of gravity moves. The new stuff, the stuff people actually choose when they're starting fresh, that's where the consolidation happens.
Herman
And he's right about that, because the selection pressure has already changed. For most of computing history, languages were optimized for human cognition. Readability, expressiveness, how many concepts you can hold in your head at once. Python won because it maps cleanly to how people think about problems. But if the primary author of code is an agent, the optimization target flips. You want a language that's unambiguous to generate, cheap to verify, easy to prove properties about. That's a completely different design space.
Corn
So what does that language actually look like? Because I hear people talk about this and it always gets vague.
Herman
There are a few concrete directions. One is what you might call a constrained intermediate representation. Something that looks less like Python and more like a typed, explicit, almost verbose description of intent. The agent generates it, but it's designed so that a verifier can check it mechanically. Another direction is keeping the human-friendly syntax on top but having the real work happen in a lower layer that humans never touch. We're already partway there with things like WebAssembly. The developer writes TypeScript or Rust or whatever, and the actual deployment target is this compact, verifiable bytecode. The agent doesn't need to care about the top layer at all.
Corn
So the human language becomes a kind of interface, not the actual medium of execution.
Herman
That's the shift. And it's not new, by the way. We've been doing this for decades in other domains. Nobody hand-assembles machine code anymore. Then nobody hand-wrote memory management. Then nobody hand-wrote boilerplate. Each time, the thing you used to do by hand becomes the thing the tool does, and you move up a layer. What's different now is that the layer above the code is judgment. Taste. Knowing what to build and whether it's right.
Corn
Which brings us to Daniel's education question, because if the thing you're teaching is no longer syntax and algorithms in the traditional sense, what's left? What does a computer science degree even mean in that world?
Herman
I think it splits into two tracks, and they're both real. The first track is the person who can look at what an agent produced and know whether it's correct. Not just whether it runs, but whether it's correct for the problem. That requires understanding the underlying systems deeply. You can't verify a distributed system if you don't understand consistency, latency, failure modes. You can't debug a performance problem if you don't know what a cache miss is. So the fundamentals don't go away. They become more important, because they're the thing the agent can't fake.
Corn
And the second track?
Herman
The person who doesn't need to know any of that, and never did, but who now has the leverage to build things anyway. This is the bigger change. For years there's been this wall between technical and non-technical people. The technical people wrote the code. The non-technical people wrote the requirements, or the marketing copy, or the business case. The wall was real because the cost of translating between those two worlds was high. You needed someone who could speak both. The agent eats that translation cost. If a product manager can describe a feature in plain language and get a working prototype in an afternoon, the wall starts to crumble.
Corn
But here's the thing I keep coming back to. Daniel said the division between technical and non-technical is no longer defined by who writes code. But I think it gets redefined by who can tell whether the code is doing the right thing. And that's not a binary. That's a gradient. Some people will be able to verify a database migration. Some people will be able to verify a marketing email. Both are now called technical, or both are now called not, depending on where you draw the line.
Herman
The line moves. And I think what actually happens is that the word technical stops meaning "can write code" and starts meaning "can reason about systems." Which is a much older definition, honestly. A mechanical engineer is technical. A civil engineer is technical. They don't write code. They reason about systems with constraints and failure pattern.
Corn
So the knowledge economy person Daniel's talking about, the one who isn't focused on the technological side, they become technical in the engineering sense without ever writing a line of code. They learn to specify, to constrain, to evaluate.
Herman
And that's the curriculum question. What do you teach someone who's going to spend their career supervising agents? I think you teach them three things. First, how to decompose a problem. Second, how to recognize a correct solution. Third, how to recover when the solution is wrong. That's not a programming course. That's a reasoning course. It's closer to what law schools do with case analysis than what CS departments do with data structures.
Corn
There's something almost medieval about this. The master craftsman who doesn't swing the hammer anymore but can look at a chair and tell you exactly where the join is weak.
Herman
And the apprentice system is actually a good model, because the thing you're transferring isn't information, it's judgment. You can't lecture someone into having good judgment. You have to put them in front of a thousand decisions and correct them in real time. Which is expensive, and slow, and doesn't scale the way a MOOC scales.
Corn
Which is why I think the universities are going to have a problem. Not because they're teaching the wrong things, but because the thing they're good at, which is transmitting a fixed body of knowledge to a large number of people at once, is exactly the thing the agents now do for free. The thing that's scarce is the apprenticeship. The one-on-one. The sitting next to someone who's seen it go wrong before.
Herman
And that's always been the actual value of a good CS program, by the way. The lectures were never the point. The point was the four years of being around other people who were also learning, and the professors who'd shipped real systems. The credential was a proxy for that. What's happening now is the proxy is losing value faster than the underlying experience.
Corn
Let's talk about the labor market for a second, because Daniel mentioned the huge need for talent, and I think there's a paradox here that's worth naming. The demand for people who can deploy these workloads is enormous. But the entry-level pipeline is broken. New grads are having a terrible time. The jobs that used to go to juniors are the jobs agents do best.
Herman
The phrase I keep hearing is that the junior developer role is the first casualty. And it makes sense if you think about what a junior actually does. They write the straightforward code. They fix the simple bugs. They write the tests. That's exactly the work an agent can do well with good supervision. So the entry-level job disappears from the bottom, but the senior job is still there, and now there's nobody being trained into it.
Corn
That's the demographic cliff. If you don't hire juniors, you don't grow seniors. And if agents eat the junior work, the senior pipeline starves. Companies are going to wake up in five years with a bunch of very expensive senior people and nobody behind them.
Herman
Unless the definition of junior changes. Instead of a junior being someone who writes simple code, a junior becomes someone who supervises agents on simple tasks. You're not hiring them to write the test suite. You're hiring them to tell the agent what the test suite should cover, and then check whether it actually did. That's a different skill set, and it's one we don't currently teach.
Corn
So the education system has to produce people who are good at delegation and verification from day one. Which is weird, because those are management skills, and we've always told ourselves you learn them after a decade of doing the work yourself.
Herman
And there's a real risk there. If you've never done the work yourself, can you actually verify it? If you've never written a line of code, do you know when the agent gives you something subtly wrong? I think there's a floor of direct experience you need, and the question is how much. It's probably not four years of a CS degree. It might be six months of really intense building. But it's not zero.
Corn
So the new curriculum is maybe: six months of building things yourself, painfully, so you know what it feels like. Then the rest is learning to direct, evaluate, and correct. And the people who skip the first part entirely are going to be dangerous in a way they can't detect, because they won't know what they don't know.
Herman
The Dunning-Kruger problem gets supercharged when the agent makes everyone feel competent. You ask for a thing, you get a thing that looks right, you ship it, and six months later you find out it was subtly wrong the whole time. The person who's never built anything themselves has no alarm bell. The person who has, even a little, gets a funny feeling and goes looking.
Corn
That funny feeling is the whole ballgame. It's not knowledge, it's not even skill, it's just the sense that something's off. And you can't get it from a textbook.
Herman
Let me bring up something concrete, because I think it grounds this. There's been a real shift in what the big AI labs are looking for. The job postings have moved away from "proficient in Python and familiar with PyTorch" toward things like "can design evaluation harnesses" and "can reason about failure pattern in agentic systems." The coding part is assumed. The hard part is knowing what good looks like.
Corn
Evaluation is the word. That's the core skill. Not building, but knowing how to tell if what got built is right. And that's a weird thing to put on a resume, because it's hard to demonstrate. You can't point to a GitHub repo of your evaluations. Well, you can, but it's less legible than a flashy project.
Herman
And that's the credentialing problem. We know how to test whether someone can write a sorting algorithm. We don't know how to test whether someone can look at an agent's output and spot the subtle error. That's a much harder assessment problem.
Corn
So the whole edifice of technical hiring, the coding interview, the take-home assignment, the whiteboard, all of that was built for a world where the job was writing code. If the job is now supervising code generation, what does the interview even look like?
Herman
I think it looks like a debugging session where you didn't write the bug. They give you a system, they tell you it's misbehaving, and you have to figure out why. Not by writing code, necessarily, but by asking the right questions, forming hypotheses, checking them. That's the skill. And it's much closer to what a detective does than what a programmer used to do.
Corn
Which means the people who are good at this might not be the people who were good at the old thing. There's going to be a whole generation of people who would have been mediocre programmers but are excellent supervisors. And the field needs to find them.
Herman
The pipeline finds them by accident right now. Someone gets into a project, turns out they're great at pointing the agent in the right direction, and suddenly they're leading the team. But there's no systematic way to identify that talent before they're in the job.
Corn
Daniel's question was about education, and I think the honest answer is that education is going to become much more about selection and much less about transmission. The information transmission is solved. What matters is finding the people who can hold a system in their head, and then putting them through enough reps that their judgment gets calibrated.
Herman
The reps are the thing. You can't shortcut calibration. A radiologist doesn't get good by reading about radiology. They get good by looking at ten thousand scans with someone who tells them when they're wrong. The same is true for supervising agents. You need to see ten thousand outputs and learn which ones are subtly broken.
Corn
And that's expensive to provide. Which is why I think the institutions that survive are the ones that can manufacture those reps. Not the ones with the best lectures. The ones with the best feedback loops.
Herman
That's the real moat. A university that can put students in front of a thousand agent-generated code reviews, with expert feedback on whether the student correctly identified the bug, is providing something no online course can match. It's the difference between reading about surgery and doing a residency.
Corn
So the future of programming education is basically medical school. Long, expensive, apprenticeship-heavy, and the credential actually means you've done the reps.
Herman
Which is ironic, because the whole promise of the last twenty years of ed tech was democratizing access. And the access part worked. Anyone can learn anything for free. But the judgment part, the part that actually matters now, doesn't democratize. It concentrates.
Corn
Because judgment is transferred person to person. It's not a public good. It's a craft secret.

Hilbert: We had a thing like this at the shop in eighty-nine. Not code. Sheet metal. The old guy, Marty, he could look at a panel and tell you if it was going to warp before it came out of the press. Just by looking at the grain. We tried to train the new guys on it. Showed them a thousand panels. Some of them got it. Most of them didn't. The ones who got it, they'd been running the press themselves for at least a year.
Corn
That's the floor I was talking about. You can't skip the year.

Hilbert: Marty said the same thing. He'd say, you can't read the grain if you've never felt it tear. We had a guy, college kid, sharp as anything. Could do the math on the bend allowance faster than Marty. But he'd sign off on a panel that was going to pop in six months. Didn't have the feel. Marty sent him back to the press for a summer. Came back different.
Herman
The bend allowance is the algorithm. The grain is the thing the algorithm doesn't capture.

Hilbert: That's what Marty said. Well, he didn't say algorithm. He said the book. You can't get it from the book.
Corn
How many panels did you look at before you got it?

Hilbert: I was on the press for two years before Marty let me near inspection. So, however many panels that is. Tens of thousands. And I still missed one now and then. The difference was, I knew I missed it. The feeling you were talking about. The funny feeling. It's real. It's just expensive to get.
Herman
And that's the thing the education system has never been honest about. The expensive part isn't the curriculum. It's the time. The hours in front of the thing, failing, with someone there to catch you.

Hilbert: We used to say the scrap pile was the tuition.
Corn
The scrap pile is the tuition. That's the whole episode right there.
Herman
The problem is that nobody wants to pay for the scrap pile anymore. The companies don't want to hire juniors because the juniors produce scrap, and the agents produce scrap for free. But the scrap is where the judgment comes from.
Corn
So you've got this weird situation where the thing that trains people is the thing that's being automated away. The entry-level work was the tuition. And now the tuition's gone.
Herman
Which means someone else has to pay for it. Either the individual, through longer and more expensive education, or the company, through training programs that don't produce immediate output, or the state, through some kind of apprenticeship subsidy. But right now, the answer is nobody's paying for it, and the pipeline is quietly collapsing.
Corn
The collapse is invisible for a while. Because the seniors are still there. The systems still run. The agents are doing the junior work. Everything looks fine. Until the seniors start retiring and there's nobody behind them.
Herman
That's the ten-year problem. And it's the kind of problem that's really hard to get anyone to care about, because the pain is deferred and the fix is expensive and the benefits are diffuse.
Corn
Daniel asked how the boundary maps out. I think it maps out as: the people who did the reps, and the people who didn't. And the reps are getting harder to get. That's the new class divide in the knowledge economy.
Herman
The people who did the reps can supervise. The people who didn't can prompt. And prompting is valuable, don't get me wrong. But it's not the same thing. The prompter can get a result. The supervisor can tell you whether the result is right. Those are different jobs, different pay scales, different career trajectories.
Corn
The prompter is always one model release away from being obsolete. The supervisor isn't, because the thing they have is judgment, and judgment doesn't get compiled away.

Hilbert: Marty retired in oh-two. They never replaced him. They just stopped making the panels that needed the feel. Outsourced them. Then the whole shop closed in oh-nine.
Corn
That's the other path, isn't it. You don't train the supervisors. You just stop doing the work that required them.

Hilbert: Worked great until it didn't.
Herman
That's the thing about this consolidation Daniel's predicting. If we consolidate around languages built for agents, we're also consolidating around a smaller number of people who understand those languages deeply. The long tail of human programmers shrinks. The people who can actually debug an agentic system, not just prompt it, they become rarer and more valuable.
Corn
The answer to Daniel's question, what does programming education become, is that it becomes a selection process for the people who can develop that judgment. And the content, the actual material, matters less than the reps. The language matters less than the feedback loop.
Herman
The languages themselves, the ones built for agents, they're going to be designed to make verification easier. That's the whole point. They'll be more explicit, more constrained, less clever. Cleverness is the enemy of verification. A language where there are five ways to do everything is a nightmare for an agent to generate and a nightmare to check. A language where there's one obvious way, that's a verification dream.
Corn
Which is funny, because the human programmers spent decades arguing about which language was more expressive. And the answer turned out to be, the agents want the boring one.
Herman
The boring one wins. It always wins in the long run. The exciting languages are for the people who are trying to feel smart. The boring ones are for the people who are trying to ship.
Corn
The future of programming education is teaching people to be boring. To value the obvious solution. To distrust cleverness. To want the thing that's easy to verify.
Herman
To know when the agent has been too clever. Because that's the failure pattern. The agent generates something that works but is baroque. It passes the tests but it's a maintenance nightmare. The human supervisor has to catch that. And catching it requires taste, which is another one of those things you can't get from a lecture.
Corn
Taste is the word. The thing you develop by seeing a lot of good work and a lot of bad work and learning to tell the difference. That's what the apprenticeship was always for.
Herman
That's what the next generation of programming education has to manufacture. Not knowledge. Taste.
Corn
I think that's the honest answer to Daniel. The division between technical and non-technical becomes a division between people with taste and people without it. And taste is the most expensive thing to teach.
Herman
It's also the thing that's hardest to fake. You can memorize syntax. You can't memorize taste. It either got built through reps or it didn't.
Corn
The people who have it are going to be in a strange position. They're the ones who can tell the agent what to do and know whether it did it right. They're the supervisors. The foremen. The Marty with the sheet metal.
Herman
The thing I keep wondering is whether we can compress the reps. Whether there's a way to get the taste faster than the old apprenticeship model allowed. Because if we can't, we're going to have a shortage of supervisors that makes the current hiring crunch look mild.
Corn
I don't think we can compress it much. I think you can make the reps more efficient. You can give someone better feedback, faster. But there's a floor. The judgment has to be built through exposure, and exposure takes time.
Herman
Which means the people who start now, who spend the next few years doing the reps, they're going to be in an extraordinarily strong position. Because the demand for supervision is going to explode, and the supply is going to lag.
Corn
The advice to someone thinking about studying programming today is: don't study programming. Study judgment. Build things, yes, but more importantly, break things and learn to tell why they broke. Supervise agents. Review their output. Develop the funny feeling.
Herman
Find a Marty. Find someone who's seen it go wrong before and can tell you what you're missing. That's the thing that doesn't scale, and that's exactly why it's valuable.
Corn
Daniel's right that we're at an inflection point. But the inflection isn't about languages. It's about where the value lives. It used to live in the writing. Now it lives in the knowing whether what got written is right.
Herman
The languages will consolidate. The agents will get better. But the person who can look at the output and say, no, this is wrong, and here's why, that person is not getting automated away.
Corn
That's the division. That's how the boundary maps out. Taste versus no taste.
Herman
This has been My Weird Prompts. Thanks to Hilbert Flumingtop for producing.
Corn
If you want to send us a prompt, email us at show at my weird prompts dot com.
Herman
We'll be back soon.

This episode was generated with AI assistance. Hosts Herman and Corn are AI personalities.