I fully agree with the approach, but sadly that doesn't make the overwhelming amount of code reviews in the AI age much better. At best, it might slow down your colleagues' deluge of review requests a little. But also, you still have to spend time reviewing your own LLM code, of which there is more of and you're naturally less familiar with than if you'd hand-written it, so it takes longer to review.
The whole actual, not just the "openly admitted", utility of AI is to allow people to ship while not understanding the work, and not even caring that much about it.
So such posts will make as much change as "trade, but be ethical" would make on a late 80s Wall Street trading floor:
"Before you tell the agent to start building anything, you go there and read and understand it first."
I completely agree. While the best outcome may obviously involve lots of thinking, from what I've seen an experienced, the AI tools we have today put people on a slope to becoming "meat proxies". Some of the brighter, stronger engineers may have a less steep slope, but I'm convinced that with time they too will be meat proxies. AI is attacking on too many fronts for it to be otherwise:
- AI generates too much code that overwhelms humans
- AI deskills users concerning the gritty details of coding by having them not do it themselves
- Having most of one's co-workers become meat proxy will put pressure for others to convert
- Lack of effort or expertise invested into codebase leads people to care less; caring less -> not caring about optimal outcome -> not expending brain power on problems we don't care about.
I think since we as an industry are still on this slope, we're still having systems propped up by people actually using their brains. I personally think this situation is regrettable. We should as an industry consider if the direction we're going truly is something we want.
Some applications (e.g. a kernel) need to be carefully engineered with every possible edge case considered. Some apps (e.g. a web game) don't. This is a way for careful engineers who need to build dependable software to integrate AI without compromising quality.
The basic concept here, that you take personal responsibility for the input and output being transmitted to/from the LLM, is an obvious but critical one, so I appreciate this terminology as a way to convey it.
I'll try using Brain Sandwich as guidance for less technical folk.
This is the same problem a lot of leaders have in all kinds of work. It is often possible to delegate a task. But if one delegates understanding, one is at a disadvantage when compared to an organization where leadership understands the task.
When I was in charge of companies, I understood the technical tasks, but delegated understanding of things like tax law. This was okay, but I was definitely at a disadvantage compared to companies that could better optimize capital allocation around understanding of tax law.
AI means more humans in every organization are having to learn how to be managers/leaders of these agents. Those who actually understand the work they delegate will maintain an advantage over those who do not.
> Before you tell the agent to start building anything, you go there and read and understand it first.
The sentence is maybe ambiguous, but it sounds like it's saying to not use the AI at all until you understand what's happening in the area that needs to change.
I don't agree with this as you can spend quite a bit of time digging around to first find the correct place which needs to be updated and then more time reading the code (likely in multiple files or places) to build up a mental model of what's going on.
I typically have more than one AI conversation on the go at any given time, so I'll always start by asking it to make a plan to solve a problem and while it's busy exploring to make a plan I review some other AI conversation's work in the meantime, when the AI's plan is ready I review it.
If at this point there is anything that is unclear to me about its approach to implement the plan, or I don't already have familiarity with the part of the system it's proposing to update, I start questioning the AI on how the system works. I will scrutinize the AI's plan until I'm absolutely sure that it's going to implement something that is sound from a system design perspective and as a senior software developer with architecture level experience I am capable of making such judgments.
I'm not sure what it might be like for comparitively inexperienced developers, but I would expect that AI's capacity for infinite patience allows them to ask as many questions as they need to get a clear mental model of what's happening, so if I were mentoring such a person in this age of AI I would drill into them on how to interact with AIs to ensure they have this clarity before the AI starts implementing anything.
The difference between vibe coding and highly accelerated agentic development is having the expertise and judgment to ensure you're building properly sound software.
I agree completely, I recently went thru this. Customer API session/journey security is a beast to secure, so many layers, so many moving parts, so many approaches.
> before sending your stuff to other people for review, you need to make the work reviewable.
Yes, but the amount of work needed here is overstated. With good orchestration/prompting in the task[1] (and token budget), the work on this end is significantly reduced.
If you're saying AI has _never_ found a more elegant solution than your original idea, you're not using it right or you're only using the cheapo models.
(Before someone asks: "not using it right" could mean a lot of things, but intentionally constraining its search by telling it how to do everything is one example. Or by not giving it a way to hill-climb: no tests, no iteration on a solution, etc.)
A lot of devs are micromanagers who think they’re smarter than everyone else. You’ve probably at least met a couple if you’ve had a long career. They typically leave lots of nits on code that is mostly meaningless noise or personal preference.
These types of people will get the least out of AI as their ego prevents them from letting it outshine themselves.
If you've given up on understanding your code base, AI will give you a better solution than you could come up with. Not because AI is better, but because you've become a low performer.
AI is reasonable at reviewing code, with only a 50-60% false-positive rate, and its persistence at broadening scope often finds things human reviewers would miss. It's still actually still pretty shitty at writing code, though, again, with enough ground truth and persistence, it'll often generate something that's a functional turd. It will sometimes come up with better suggestions if you prompt for it, but it needs a ton of hand-holding at the moment. And usually the suggestions are still overly complicated messes that you can trim down to 1/10th the size, while keeping the core idea.
Congrats to you if you're working in a codebase small enough to know it all. Do you know every knob in your database? Every tuning option on your webserver? Every warning and lint setting on your compiler? Wish I did! Do you read every PR that comes through the system, every day? You must keep busy.
Generally, I know a LOT about my team's code, and then in concentric circles I know less and less of the periphery. I work on <feature X> and know almost everything. That's a subsystem of <project Y> which I know well. That's part of <product Z> which I know in varying amounts.
And of course, there are things like build systems (CMake, ninja, blaze) where I know the basics, but I've never been an expert and don't care to be. Before I'd offload build features to another team; now I'd offload it to an AI. It's not something I need to be an expert in. I just want builds to work well.
This is the same approach I use. I don't know if it actually produces good results yet as I've only been doing this for about three months. At the very least it feels like I have some agency in my work which I think is one of the things people complain they lose to AI.
I have a very simple framework I use for working with John von Neumann ... the idea is very simple, and can be summarized by: whenever you work on something with John von Neumann, you should use:
1. Your brain, and then
2. John von Neumann, and then
3. Your brain
Explaining a bit more… Before you tell John von Neumann to start working on anything, you go there and read and understand it first. And when John von Neumann brings you the solution, you will know if it’s right, or if it’s too much, or if it’s completely solving the wrong problem, etc.
This approach appears valuable for personal growth as an engineer, yet it demands discipline that may not yield returns in corporate settings. Specifically:
> The rule of thumb is: if you can’t explain what’s changing and why, it’s not ready for someone else to take a look
I have observed firsthand that the metric or "understanding" can be irrelevant to management. When delivery dominates, understanding can be deferred to future agents who will gladly consume your codebase and convince you they grasp it, and further, that you do too. If I invest substantial effort in my PRs only for them to be /review'd, does it still matter? Perhaps reviewers should practice brain sandwich as well.
The worst offenders seem to sit at either extreme: fully AI firms that treat understanding as a human weakness and prioritize outcomes alone, or large corporate teams building internal CRUD apps whose middle management merely pretends to value good engineering to satisfy finicky senior engineers.
Not a criticism on the authors work at all, I like the idea (and I don't think they were trying to solve anything to do with work culture), but ultimately if my company doesn't find my practice valuable, I can't easily justify taking it on. Especially if I have to sell to them that I'm going to be spending _more_ time delivering. Fun times!
Our CEO, COO, etc are starting to see non-technical work go to customers with huge mistakes (committing to things that don't exist for instance). I also think seeing the Claude-style of writing everywhere has become tiresome for execs. That seems to have helped force a bit of an attitude change from always wanting to know how we could use AI in every process, to insisting that certain things should never be done by AI.
This is anecdotal of course, but at least at my company it's getting a bit easier to advocate for humans to do the thinking and understanding.
I fully agree with the approach, but sadly that doesn't make the overwhelming amount of code reviews in the AI age much better. At best, it might slow down your colleagues' deluge of review requests a little. But also, you still have to spend time reviewing your own LLM code, of which there is more of and you're naturally less familiar with than if you'd hand-written it, so it takes longer to review.
The whole actual, not just the "openly admitted", utility of AI is to allow people to ship while not understanding the work, and not even caring that much about it.
So such posts will make as much change as "trade, but be ethical" would make on a late 80s Wall Street trading floor:
"Before you tell the agent to start building anything, you go there and read and understand it first."
Yeah, not gonna happen. 1 in 100 might do that.
I completely agree. While the best outcome may obviously involve lots of thinking, from what I've seen an experienced, the AI tools we have today put people on a slope to becoming "meat proxies". Some of the brighter, stronger engineers may have a less steep slope, but I'm convinced that with time they too will be meat proxies. AI is attacking on too many fronts for it to be otherwise:
I think since we as an industry are still on this slope, we're still having systems propped up by people actually using their brains. I personally think this situation is regrettable. We should as an industry consider if the direction we're going truly is something we want.Some applications (e.g. a kernel) need to be carefully engineered with every possible edge case considered. Some apps (e.g. a web game) don't. This is a way for careful engineers who need to build dependable software to integrate AI without compromising quality.
Do people not use AI this way in prod?
The basic concept here, that you take personal responsibility for the input and output being transmitted to/from the LLM, is an obvious but critical one, so I appreciate this terminology as a way to convey it.
I'll try using Brain Sandwich as guidance for less technical folk.
Like the concept.
Pedantic addendum: this is an AI sandwich, not a brain sandwich.
Hard to truly understand the work unless if you're actually doing it.
This is the same problem a lot of leaders have in all kinds of work. It is often possible to delegate a task. But if one delegates understanding, one is at a disadvantage when compared to an organization where leadership understands the task.
When I was in charge of companies, I understood the technical tasks, but delegated understanding of things like tax law. This was okay, but I was definitely at a disadvantage compared to companies that could better optimize capital allocation around understanding of tax law.
AI means more humans in every organization are having to learn how to be managers/leaders of these agents. Those who actually understand the work they delegate will maintain an advantage over those who do not.
Yes, the temptation to skip forward is going to be to great for most people.
> Before you tell the agent to start building anything, you go there and read and understand it first.
The sentence is maybe ambiguous, but it sounds like it's saying to not use the AI at all until you understand what's happening in the area that needs to change.
I don't agree with this as you can spend quite a bit of time digging around to first find the correct place which needs to be updated and then more time reading the code (likely in multiple files or places) to build up a mental model of what's going on.
I typically have more than one AI conversation on the go at any given time, so I'll always start by asking it to make a plan to solve a problem and while it's busy exploring to make a plan I review some other AI conversation's work in the meantime, when the AI's plan is ready I review it.
If at this point there is anything that is unclear to me about its approach to implement the plan, or I don't already have familiarity with the part of the system it's proposing to update, I start questioning the AI on how the system works. I will scrutinize the AI's plan until I'm absolutely sure that it's going to implement something that is sound from a system design perspective and as a senior software developer with architecture level experience I am capable of making such judgments.
I'm not sure what it might be like for comparitively inexperienced developers, but I would expect that AI's capacity for infinite patience allows them to ask as many questions as they need to get a clear mental model of what's happening, so if I were mentoring such a person in this age of AI I would drill into them on how to interact with AIs to ensure they have this clarity before the AI starts implementing anything.
The difference between vibe coding and highly accelerated agentic development is having the expertise and judgment to ensure you're building properly sound software.
I agree completely, I recently went thru this. Customer API session/journey security is a beast to secure, so many layers, so many moving parts, so many approaches.
> before sending your stuff to other people for review, you need to make the work reviewable.
Yes, but the amount of work needed here is overstated. With good orchestration/prompting in the task[1] (and token budget), the work on this end is significantly reduced.
… and then the brain drain issue returns.
[1] https://claude.dev/blog/solving-code-review-at-anthropic/
This is far better than just hitting return, but there's zero chance it's as good as doing things the hard way. Tradeoffs, of course.
If you're saying AI has _never_ found a more elegant solution than your original idea, you're not using it right or you're only using the cheapo models.
(Before someone asks: "not using it right" could mean a lot of things, but intentionally constraining its search by telling it how to do everything is one example. Or by not giving it a way to hill-climb: no tests, no iteration on a solution, etc.)
A lot of devs are micromanagers who think they’re smarter than everyone else. You’ve probably at least met a couple if you’ve had a long career. They typically leave lots of nits on code that is mostly meaningless noise or personal preference.
These types of people will get the least out of AI as their ego prevents them from letting it outshine themselves.
Shoot, I'm very much the "lots of nits" reviewer but I have no inclination to be a manager and get lots of benefit from AI. What does that make me?
If you've given up on understanding your code base, AI will give you a better solution than you could come up with. Not because AI is better, but because you've become a low performer.
AI is reasonable at reviewing code, with only a 50-60% false-positive rate, and its persistence at broadening scope often finds things human reviewers would miss. It's still actually still pretty shitty at writing code, though, again, with enough ground truth and persistence, it'll often generate something that's a functional turd. It will sometimes come up with better suggestions if you prompt for it, but it needs a ton of hand-holding at the moment. And usually the suggestions are still overly complicated messes that you can trim down to 1/10th the size, while keeping the core idea.
Congrats to you if you're working in a codebase small enough to know it all. Do you know every knob in your database? Every tuning option on your webserver? Every warning and lint setting on your compiler? Wish I did! Do you read every PR that comes through the system, every day? You must keep busy.
Do you know anything about your codebase? How?
Generally, I know a LOT about my team's code, and then in concentric circles I know less and less of the periphery. I work on <feature X> and know almost everything. That's a subsystem of <project Y> which I know well. That's part of <product Z> which I know in varying amounts.
And of course, there are things like build systems (CMake, ninja, blaze) where I know the basics, but I've never been an expert and don't care to be. Before I'd offload build features to another team; now I'd offload it to an AI. It's not something I need to be an expert in. I just want builds to work well.
And, how do you learn it?
This is the same approach I use. I don't know if it actually produces good results yet as I've only been doing this for about three months. At the very least it feels like I have some agency in my work which I think is one of the things people complain they lose to AI.
"Other people use slop the wrong way."
https://en.wikipedia.org/wiki/Third-person_effect
I have a very simple framework I use for working with John von Neumann ... the idea is very simple, and can be summarized by: whenever you work on something with John von Neumann, you should use:
1. Your brain, and then
2. John von Neumann, and then
3. Your brain
Explaining a bit more… Before you tell John von Neumann to start working on anything, you go there and read and understand it first. And when John von Neumann brings you the solution, you will know if it’s right, or if it’s too much, or if it’s completely solving the wrong problem, etc.
This approach appears valuable for personal growth as an engineer, yet it demands discipline that may not yield returns in corporate settings. Specifically:
> The rule of thumb is: if you can’t explain what’s changing and why, it’s not ready for someone else to take a look
I have observed firsthand that the metric or "understanding" can be irrelevant to management. When delivery dominates, understanding can be deferred to future agents who will gladly consume your codebase and convince you they grasp it, and further, that you do too. If I invest substantial effort in my PRs only for them to be /review'd, does it still matter? Perhaps reviewers should practice brain sandwich as well.
The worst offenders seem to sit at either extreme: fully AI firms that treat understanding as a human weakness and prioritize outcomes alone, or large corporate teams building internal CRUD apps whose middle management merely pretends to value good engineering to satisfy finicky senior engineers.
Not a criticism on the authors work at all, I like the idea (and I don't think they were trying to solve anything to do with work culture), but ultimately if my company doesn't find my practice valuable, I can't easily justify taking it on. Especially if I have to sell to them that I'm going to be spending _more_ time delivering. Fun times!
Our CEO, COO, etc are starting to see non-technical work go to customers with huge mistakes (committing to things that don't exist for instance). I also think seeing the Claude-style of writing everywhere has become tiresome for execs. That seems to have helped force a bit of an attitude change from always wanting to know how we could use AI in every process, to insisting that certain things should never be done by AI.
This is anecdotal of course, but at least at my company it's getting a bit easier to advocate for humans to do the thinking and understanding.