I'm a principal engineer. Code reviews still matter for production software.
Maybe your opinion is that I'm not vibe coding hard enough, and if I did AI better we wouldn't need code reviews. I disagree - both engineers and LLMs are still adding unneeded complexity or choose the wrong abstractions in ways that will cause problems down the road, don't think about all the corner cases, and pick the wrong handling for error conditions. They add in things that seem like a good idea to them and don't stick to the spec.
If my company stops doing code reviews for the software that matters, I'll quit.
Engineers not in the startup crunch should aim for using LLMs to code better, not faster. Faster just creates technical debt and ups your long term costs.
The fundamental issue is that traditional code review doesn’t scale.
I now use a “Controls” linter which applies deterministic methods to constrain AI output, requires explicit declaration when key resources are used (file system access, network egress, outside libraries, or whatever I feel is important), and ties directly to other external requirements for business level controls that may be audited. (And more, but this gets the idea across)
Individual control violations can block PR mergers, block release, or just flag for review.
I genuinely don’t care that much about AI code quality these days (though I exclusively use SOTA models like Astra, which could be related). I care about code structure and architectural quality. I’ve found those can be deterministically enforced and made visible. If structure and architecture are high quality, I have no need to nitpick code quality.
If I were working with humans, this system would be cruel. It is far harsher than any manual code review I’ve ever given to humans. I let humans get away with small issues all the time. Dealing with the consequences of small mistakes is how people learn. For anything that really matters, I am ruthless with AI developers.
The system itself needs to be audited, so code review is still necessary. It’s more spot checking than full review though. You need to be a decent traditional software developer and architect to build new controls and to run control audits.
I think it also depends on what kind of software you are writing.
If you're in finance, aviation, or some other industry where the cost of errors is very high, I do kinda want a human to look at every line of code even today.
Agree here. At my current place we have LLM apply deterministic rules where given thing cannot be checked statically. For example - all unit tests shouldn't access database. This can be checked statically. All comments should document why, rather than what - this cannot.
Humans tend to bring up things that matter to them + correctness, when they have time to think about it.
There are definitely areas where I’m reviewing individual lines. Protocols are high on the list. I always want to know that high-level services aren’t leaking their internal details to the outside world and that whatever entity owns a protocol is controlling what is being communicated.
AI is bad at certain kinds of abstraction (for now). For example:
Connect service A to service B. Service C exists in some deployments and will do the real work in those cases, but A must never use B as a passthrough to C or use C’s language. That would cause future planned changes in the app to require a massive refactor.
Then, later, in review: Why is A’s client protocol to B referencing functionality that only exists in C? B’s entire purpose in life is to prevent this!
I haven’t tried to solve this via automation yet. I think it will involve limiting what AI has access to.
How could I learn more about this linter approach (especially around architecture quality)? Any other terms to describe it or resources that explain it?
Declare and deterministically detect the entities you want to control. Figure out how to list them. Then, write rules that enforce structure and behavior of or between those entities.
It’s just lots of declaration, detection, and fancy tests. But you need to apply a lot of attention to detail and make sure you have confidence that a given rule is enforced, and that a given entity can be detected. There’s more to it than that, but if you start playing around I’m sure you’ll figure the rest out.
I will warn that the goal isn’t to get AI to write code like a human would. It’s to enable controlled, high-velocity AI-driven SDLC that scales to large, complex applications.
Code reviews matter in the sense that you need to check the code is reasonably sane occasionally, but that's not the same as doing PRs. Leaving checks to the end has always been a dubious strategy - it's a lot easier to catch problems if you're designing the code up front and then reviewing that things are on track during the dev process. Finding that there's a problem at the end is the worst possible time to change something.
AI has solidified notion that for me. When your team are cranking out PRs at a rate of several a day (not building faster; making small, atomic change requests of independent bits) you'll never have the bandwidth to review everything by hand. You need time at the start to understand the change, lots comms during the process of making the change, and lots of automation around deterministic tooling (linting, formatting, unit tests, integration tests, e2e tests, fuzzing, property testing, smoke tests, etc) to make sure your AI change hasn't broken anything. If you have that in place the need to review at the end of a change is greatly reduced. Down to zero in my opinion but your milage may vary.
> both engineers and LLMs are still adding unneeded complexity or choose the wrong abstractions in ways that will cause problems down the road
Completely agree. But why do you think the reviewer would choose a better abstraction?
Yes, an experienced dev has a high chance of doing so. But in most teams there are not enough experienced devs.
And even I, an experienced dev, choose wrong abstractions all the time.
But fixing even large oopsies is just a prompt away. So imo it does matter less and less.
What was and is true: iteration speed just matters more. I would bet a lot of money that a team that goes from written code to merge in hours has a better quality codebase than one that takes weeks.
"Fixing large oopsies is just a prompt away" really depends on what you are doing.
I write software responsible for updating customer controlled devices. In theory, these devices may be in use for decades. I don't get to dictate when customers update their software. I have to use whatever software is on device as part of the update process, and the software update process really should work from every single version that's ever been deployed.
So whatever decisions I make / the team makes, generally we're stuck dealing with in some form for the device lifetime.
there is also sentiment I see that engineers should move from the code review to the testing of the end result. which is nonsensical because with the code review you can catch problems which are borderline impossible to catch via testing. but
the code review requires a tamed agent producing comprehensible code.
We decided that we know the system well enough that we can self approve PRs
I would say people might have slightly different opinions I. The company, but in general, if it is not critical and it can go to preprod to check, we can merge.
I particularly do an automated code review and use some setup to check idea, code and documentation. But if it is critical or too important to go wrong I seek for code reviews.
The cost of writing code has gone down but not the cost for engineering.
I see my friends struggling with vibe coded projects because they didnt plan how to grow the baseline.
The line got blurred as it became cheaper to do-redo but you can check any vibe-coder code cost through time and it does increase at least linearly.
I will say that now it is easier to plan specific changes that can put in and out in case of problems whether before we would need to do several rollout steps so multiple people could work on a task.
Now several people work on several things and the struggles are others.
Code quality has been a pain for everyone.
I think for an LLM doing things one way or the other is a matter of what fitted the chances by the prompt and not good practices or choices. By this I mean, if there are two ways of writing the same code and one is clearly harder to read or performs worse, it can be more a matter of being lucky/unlucky to get it right throughout the whole code.
There are weeks I get 30 PRs done, there are weeks I get 3 PRs, like it was before.
But reading LLM code is painful. It is obtuse and without lots of guidance it invent rules to problems that do not exist.
Of course I see that from my bubble, but I guess as experienced engineers are seeing the problems piling up and it the models are getting more expensive and not getting better as faster as before.
These three things can be where we will delimit the usage of LLM for coding.
I particularly like that I don't need to do some of the annoying tasks that took me only checking 48302 files to be sure I got all the same string correct because some past bad decisions. But when I am working in important stuff, the cracks in llm makes me much slower.
A personal challenge has been to keep up knowing the codebase well enough while people (including me) are dumping stuff in there.
> still adding unneeded complexity or choose the wrong abstractions in ways that will cause problems down the road, don't think about all the corner cases, and pick the wrong handling for error conditions.
These are all one of those things which have never been easily quantifiable and therefore not easily sold to the stakeholders (your boss, boss of your boss, etc.). The AI will only emphasize this condition.
Why do so many people think that code reviews are primarily about mistrust or spotting mistakes?
IMO they’re primarily about ensuring shared understanding and agreement, not just about how things are changing now, but how they’ll change in the future..
>Why do so many people think that code reviews are primarily about mistrust or spotting mistakes?
because people generally think things they are told, basic memetics.
There are a set of things code review is for, this set of things is for written information transfer and rather too large for speech, so when choosing to speak about what things code review is for people choose some, shared understanding is slightly more abstract than catching and reducing bugs so catching and reducing bugs gets repeated slightly more, and the next generation of people hearing what code review is for repeat it.
As a general rule I have seldom seen bugs get caught in review, I have caught a few bugs in review, but far fewer than the amount of code I have reviewed. I have very few times had bugs caught in review of my code, this is not because I don't make bugs, they just get caught out in testing.
Normally my experience is much of the communication I receive about code I've written is generally bike-shedding, probably 70-80%, above that it starts to become things we can absolutely agree is useful in some way (even if not at the bug catching stage), bug catching is maybe 0.01% of the comments.
2. make me understand what the agent team did, while it spinned for hours
I think no 1 will be solved by N-version coding, keeping a population of replicas which provide differential testing on fuzzed inputs or prod data. This also helps no 2 - where execution diverges it can be a bug or a missing spec. Either fix code or fix spec. The reason I think N-version approach will win is that it is easier to code again the same app than to imagine code execution and its edge cases. Let real execution flesh out the divergences. It is easier than formal verification but still complex because it requires a careful setup where randomness, APIs, databases and other external state can be controlled for comparison.
OP said they use this question "What’s the cost that made this a bad idea, and is that cost still there?" to triage. In our case it is getting cheaper to spend tokens, N-version stops being too expensive. Having a population of replicas instead of a singleton would be the safe approach. It can even be faster, you can build the replicas in parallel, while test creation would take extra time on top of implementation. It's basically unbounded testing from input side. Makes performance optimization of one replica safer too.
Having worked in SWE for two decades, I trust AI far more than humans. The top 10% of human programmers did great work when given the right environment, not under pressure by leadership to deliver too quickly, not forced to use or work with partner orgs that sucked, etc. But the majority of people I've worked with, while lovely people, produced very human code. This is why AI is able to find so many exploits.
Yes, people are struggling with status and purpose, but the reality is Opus 5.5 onwards has completely changed reality. We have to harness it.
N-version isn't a new idea. You quickly run into a problem with evaluation. Fuzzing is slow, non-deterministic, expensive. Building and verifying a test suite similarly so. Building a program that will evaluate whether an arbitrary program conforms to an arbitrary spec is an unsolved problem (consider how you would verify such a program in the first place).
People keep thinking of the coding step as a discrete, and thus also slow, expensive, non-deterministic step, but engineers did a lot to make it less so, whether it was techniques, tools, process, culture, whatever. Like, imagine an incredibly good agent working on a 2kloc PR for ~120 hours, periodically checking in with 4-5 other also incredibly good agents with near perfect knowledge of the system they're working on, constantly reevaluating the approach, periodically considering whether the existing codebase and architecture can facilitate the types of functionality you're adding, spending considerable effort to make it as easy as possible for other agents to add features, improving itself through these efforts, building ideas for new tools to help it, etc etc. It wasn't just code generation; so many useful things were happening in parallel.
You can think of it like dating in a huge city. In smaller cities incidental contact is frequent and encouraged. Dating is a bunch of meet cutes, dating in the workplace, ad hoc get togethers, etc. In huge cities incidental contact is overwhelming, oppressive, and discouraged, so if you want to meet people you have to expend considerable energy deliberately doing something you used to get relatively for free: talking to people you may want to socialize with/date/whatever. It's of course possible and people do it, but it's much harder, and we're only starting to become aware of the second order consequences of it (loneliness epidemic, growing political separation between men and women, falling fertility rates, etc). It's to the point where getting into a relationship as an adult is a part-time job, and fewer and fewer people can bring themselves to put forth the effort.
Swapping in LLMs for humans for the code generation process has the same dynamics. You can have more meetings about architecture, you can have it slop out more tools, you can have it analyze the last N PRs and have it slop out something that would've made those PRs smaller/simpler/whatever, you can have it rearchitect whatever you want (kinda) but you have to constantly push yourself to do it. You're no longer internally motivated to do it because you're no longer personally affected, fewer and fewer engineers will be able to put in the effort, and things will start to fall apart.
Maybe this won't be a problem. Maybe the code will be good enough and we'll find we don't need these higher level operations, i.e. we won't need engineers to spend weeks doing tedious tech debt sweeps every quarter in order to keep the lights on. But my guess is our ambitions will grow until we do.
Do you need to look into the code for that though? Can't you talk with your agent about the changes introduced and the direction the PR is taking your code?
P.S. Have a feeling this will get down-voted but posting since I am interested in counter-arguments for this approach.
Yes you do. Definitely not in the excruciating detail we used to. But an agent on itself typically catches only (relatively) superficial issues. It hasn't ever occurred to me to get a higher-level disagreement on the changes of the PR by an agent.
E.g. why re-implement the oauth workflow instead of using the established package of the ecosystem? Why monkeypatch an existing python method for all code instead of implementing and using where needed a context manager?
The agent can give you an assessment on such alternative approaches if you ask it, but I haven't seen it taking the initiative to radically steer a PR.
I don't think your observation is limited to programmer culture.
AFAICT, it applies to everything. Few people are exercising any thinking skills because they put in a half-formed broken thought in fractured english into the prompt, and then use the result.
Emails, general comms, problem-solving... everything is now at the point where the lazy fucks are no longer detectable in the workplace.
They also pose the question: 'Are we building the right thing?'. Which is something that is really hard to resolve when it is just you and your LLM echo chamber. You don't know what you don't know.
I probably disagree, and this sounds like coping rationalization.
Reviews ensure that the writer's hypothesis has an antithesis to create a better synthesis. It follows fairly naturally from the academic background of engineers'. Structural engineers don't have drawing reviews to ensure shared understanding and agreement; they really just want to avoid being liable for killing people. That's the structure that software engineering evolved from.
> But a lot of the reason rewrites were so dangerous is that they took forever, and the old system kept changing underneath yo
I think a bigger reason is that people don't understand why the current system is the way it is. That risk is increasing with lower j understanding of the codebase. (doesn't apply as much to building the same thing in a different language)
I'm going to talk about the item on this list that I know best. The "building instead of buying" is the one I’d question.
The cost of writing the tool is mostly gone, on that I agree. The cost of owning it hasn’t moved that much I think. If I look after the software stack of a couple of small companies I know, the custom tools are those that end up hurting the most. Nobody remembers it was custom made, until the API call breaks, or the person who built it leaves. I think the cost of maintenance is particularly high.
An afternoon build doesn’t fix long terme maintenance, nor does it provide functional knowledge. I think generalists stand to gain a lot from AI, but there will still be a huge role for experts to guide and maintain software in the future.
Yeah, that resonates with me. Like a lot of people, I started building several small tools earlier this year that I didn't have time for before, both for convenience and to alleviate pain points that I had. Turns out that this also means that I now have additional maintenance tasks that crop up at unexpected moments and I'm not sure if I'm getting a net benefit from it aside from the few that I use regularly.
There are maintenance risks for bought software too. A saas can change the interface in an undesirable way or the way a feature works or even introduce a new feature you don't want. Custom software can be more stable in that way.
I fully agree that there are maintenance risks when you buy software, but imho, those risks are much much more easier to mitigate: buy only "mature" software, make sure that maintenance is governed by contract, know your path to reversibility, etc.
Ultimately, in my opinion, this “make or buy” issue goes back to the age-old debate about how companies choose to adapt to the processes already built into the software they purchase. The companies that will “make” are probably the ones that would have developed custom solutions at some point anyway.
Sure - but it's going to be 70% vibe coding (because you can understand something about a not-too-exotic language) instead of not making changes at all. And that's the thing about the article, what the guy is effectively saying is "just vibe it".
To your note about nobody doing code reviews. I've found that to be fine in fairly straightforward applications and/or small teams where individual SWEs can own verticals.
As soon as you have larger teams with complex architectures trying to make decisions and push larger initiatives forwards, decisions made in one part of the stack really matter in other parts as they get built on and have momentum that can be hard to reverse, even if code is cheap.
Not to shill my work too much (https://www.archme.io) but I think for the reason above the importance of code review has moved from code to architecture, and code in general is the wrong abstraction to be working with as a human engineer in 2026+.
However, as other folks have pointed out, it is not possible resolve all things initially. Especially with customers who have feedback. Building and iterating is critical.
IMO It’s important to have a way to collaborate and move forwards together as a team while building. Keep everyone working off the same abstraction so mental models don’t drift and people start building the wrong things
There’s a middle ground between not writing any sort of design documentation at all and insisting that every little detail is nailed down in a Word document and carefully laid out on a Gantt chart before a single line of of code can be written.
Sometimes jumping straight in and seeing what happens is the right path, other times the changes you’re making will impact a bunch of systems and teams so coming to some agreement on what’s being done is valuable.
I’m glad someone called that out. No one person being able to get a change from their laptop to production without review is a baseline requirement of a lot of certifications, because while I trust the other people on my team the security team at a client’s client have no reason to.
Self review is absolutely a thing you can get away with in the early days of a startup, and at some point will probably be necessary because you’ve only got one developer, but if you have enterprise clients just turning off review requirements is a fast track to losing those clients.
Yeah. The problem with AI brained people is they have zero concept of information security. Most of their ideas would be shot down immediately, but they're shadow-ITing them into existence because "WeRe UsINg FrOnTiER Ai"
Something's wrong with that code review strategy. There should be knowledge share.
And yeah perverse incentive is a thing when you grad your own homework.
And these two things combine. Who decides what "done" means? If everyone faces consequences for getting that wrong the next day it's probably fine, but they probably don't.
I like the broader point about questioning the economics behind established practices. But building software faster and owning software for years are still very different things
I found this insight very useful: "When my gut tells me something’s a bad idea, I’m trying to ask the follow-up question: what’s the cost that made this a bad idea, and is that cost still there?"
I am reflecting on earlier "tooling transitions" and how they affected my methods.
Switching from a typewriter to a text editor made drafts and rewrites much, much cheaper. Shared edit documents sped things up further.
Text comparison tools (and related tools for source code) made design reviews much faster.
Moving from handcrafted financial scenarios to computer spreadsheets made "what if" and scenario generation and comparison two orders of magnitude cheaper.
Many aspects of code review have already been automated; LLMs just extend our capabilities further in this regard.
On one hand, I like the idea of being 100% responsible for reviewing my own code because it allows me to get changes in quicker.
The way we've been working is that we own a feature end-to-end rather than just a slice of it. So I understand my work, and other people understand theirs. In that context, not having code reviews makes some sense.
This was a change forced by management, I am but a peon
But part of me already misses what code reviews used to bring: learning opportunities. I can no longer rely on someone more senior than me to give me feedback, challenge my decisions, or help develop a shared understanding of the codebase.
> I can say to Claude "hey, bring me up to speed on this thing they've been building - how does it work, where is it at, and what still needs to be done?"
I think there's an opportunity to solve this differently. If you could see your teammates' conversations with their agents, you could actually follow the decisions they made and understand why. Although I appreciate that's a bit "big brother".
> What I'm recommending instead is to still build a very strong foundation in engineering fundamentals, because that's where I think we as humans offer the most value right now.
Whilst I partly agree with this, how do you actually do it in practice?
How can you possibly know what good looks like without writing good code yourself and experiencing the consequences of bad code?
I've been thinking of this as a privileged point of view, particularly from engineers who've spent more of their careers working pre-AI than post-AI.
You've had the benefit of years of experience developing the judgment needed to distinguish good engineering from bad engineering. You can now use AI effectively precisely because you've built that foundation.
But how are newer engineers supposed to develop that same judgment, especially if their workplace operates like yours?
But it's valuable to experiment with these instincts.
The common theme across those different examples seems to be: "Before, we had to be careful to allow each team member to build experience and expertise with our codebase and avoid dynamics that could form knowledge silos. Now this stuff doesn't matter anymore because everyone can just ask Claude to explain everything to them."
I'm not so sure about that, honestly. In the short term, it works, yes. But in the long term, it might be as if you have some very good books, but zero hands-on experience with a thing. We all know that book knowledge is different from learned experience.
> The thing that’s really changed here is that the cost of context switching has dropped dramatically, because we’re operating a level higher than we used to.
I completely disagree with this.
I would argue that the cost of context switching has dramatically increased, since within the agentic pattern, an engineer is now forced to work at a higher pace across multiple topics; even if operating at a higher level, the amount of information required to be processed by a human brain is generally overwhelming.
I think it depends on how you approach vibe coding.
If you want to actually understand the code that goes into production or the engineering behind it, then yes, cost for context switching has increased massively.
If instead you just hand off engineering and understanding to the machine entirely, and reduce yourself to copy&pasting github issues into Claude Code and clicking the green button on its PRs, then there is pretty much no cost to context switching (because your brain is devoid of context).
Without commenting on the specific cases, this is very true in general. AI is rewriting the costs of certain activities and so we need to recalibrate the cost benefit analyses that were infrained into people. Now is the time to ask why and why not. Because it might turn out that the reason we just didnt do something might not exist anymore
Disagree that you no longer need PRs. You no longer need them to check the code, but you still need them as a mechanism for consensus on achieving a shared goal. If my teammates are just going crazy on the code with Claude, it may as well be a solo project.
And that way, bit by bit, we become dependent on a few global corporations that own the brain that we used yo use. Brain as a service, dammit. Don't get banned on a whim or you'll lose your ability to think.
You could say the same thing about computers which are required for programmers to do their job. The entire profession hinges on a few global corporations deciding not to stop making computers.
Well, one difference is that you don't rent your computers over the iterne... oh wait! (That is what "There is no cloud, only other people's computers" is about.)
> second set of eyes ... spreads knowledge around the team, and keeps any one person from quietly drifting the codebase off in their own direction...
Supposedly, this is what the "deep well" and "15 years of experience" dictates - knowledge must be spread, drift should be detected/prevented early.
> And it’s been working great! ... we’re only one month into the change
But this is just the most obvious nonsense? You can't assess that kind of change in a month?Especially since the big issues can be "pretty rare".
I guess, drift could be swept under the Claude is better at inconsistencies, but then there is no knowledge spreading. What's up with that? It's wisely not even mentioned...
> Deep expertise in a specific technology is a lot less valuable when you have a collaborator that has read all of the source code for that framework and can answer any question about it in a few seconds. This is another case (like what I mentioned above) where knowledge is less valuable, but where judgement is still important
Yet you can't even judge the obvious - that assumption is a fantasy, the models can't answer any question about it. Also, less obvious, the lack of knowledge constrains the quality of questions you can ask, which negatively impacts the quality of the solutions you implement based on those frameworks
> That human understands the problem better than anyone else on the team, because they’re the one who’s been working on it.
It used to be like that, now that human copy user story to Claude, and then "review" this thousands LOCs.
> It’s pretty rare (at least on the teams I’ve worked on) for big things like architectural flaws or unintended product impact to be found at this stage.
I found AI like to stick to this big things, regardless how good or bad they are and focus on minor things (at second thought - just like humans)
How do you handle teammates who haven’t yet mastered getting their agents to produce a level of quality and elegance? That’s where I see code review still being useful…
> Deep expertise in a specific technology is a lot less valuable when you have a collaborator that has read all of the source code for that framework and can answer any question about it in a few seconds. This is another case (like what I mentioned above) where knowledge is less valuable, but where judgement is still important.
Agree with the first part, but I would say experience has more value now because that is what provides the intuition needed to effectively use AI agents without needing to review every little thing. The problems they create are now at an architectural or design level, and you need a corresponding level of expertise to catch those issues and direct them properly.
The rewrite being cheap is about to become really disruptive.
I started rewriting the client's product and they don't even know I've started. The neat thing about software is that you can have many concurrent versions and you can keep any number of those private for however long you please. It's not something you need to make a big deal about anymore. You just put your head down for a few weeks and then it's surprise demo time. Don't even give the bastards a chance to see it coming. They'll almost certainly try to stop you given how the last attempts probably went.
We've done similar thing as the OP, I think it will be 2 months now. Our processes were pretty focused on spotting defects beforehands already: we have standardised tasks defined with ACs following given/when/then format that we can write behaviour tests for, we have automated linting, static analysis, automated tests. Then there's also round of tests: both QA (LLM/human driven) and UAT (human). Since I joined we have added architecture tests to that mix, including static as well as LLM guided that look at how code is built and organised + writing documentation for what is being built / product rules (LLM driven). Looking at the stats - the amount of merged PRs has trippled, the amount of issues is decreasing, PRs time-to-merge went down to roughly a day (I don't attribute this particular metric to usage of LLMs in review process).
What I've noticed = good sides: LLMs are better than humans in spotting edge cases and reviewing code, given they have established conventions and standards (I saw PRS that I'd approve, but LLM was pedantic in reviewing a comment or class name); they are good in finding connections, ie I've had many cases where an agent built something where there was no guidance in the ACs and the reason was that it found related tickets in Jira mentioning given thing and filled in the functionality. LLMs are "cheap" to run = one can run review process for multiple PRs at the same time and forget while they run; they code defensively requiring developer to intervene - for example: I'm working on functionality where user is assigned to review a thing. Only one user can be assigned to review given thing at a time and there are checks for this. In last 3 PRs LLM is telling me - add locks because if there are multiple people reviewing given thing at the same time then the data will be lost. Yes, if we ever change business rules in this place this can happen. Finally LLMs are semi-consistent; sure sometimes they'll do stuff for shits and giggles following some convention I've not seen anywhere in the codebase, I ask what happened, and it tells me it picked it up from some tangentially related legacy place. Most often they follow already established conventions / adhere to rules in place.
Now the bad points: many times QA catches things that would emerge during planning or if developer would question what they're building or look at their work thinking critically (I find it that people are loosing the ability to question things) - LLMs dont help with preventing this from happening. People are loosing track of domain and product rules - in the places I've worked I was never just a developer, I was always a developer/something (forever to my dismay), so during planning or review I could say, from product perspective: this doesn't make sense, we don't need to build this. This no longer happens - there's no time where I can remember the rules that govern the product; PM often becomes the bottleneck of what the rules should even be. I'm also finding that people do not develop good taste - this piece of code that LLM used? Sure, looks good, go ahead, ship it. Path of least resistance is becoming a norm. It's also easier to create bigger PRs.
Anyhow - I'm old and tired, so sometimes I'm finding the LLM pros are greater than cons
I'm a principal engineer. Code reviews still matter for production software.
Maybe your opinion is that I'm not vibe coding hard enough, and if I did AI better we wouldn't need code reviews. I disagree - both engineers and LLMs are still adding unneeded complexity or choose the wrong abstractions in ways that will cause problems down the road, don't think about all the corner cases, and pick the wrong handling for error conditions. They add in things that seem like a good idea to them and don't stick to the spec.
If my company stops doing code reviews for the software that matters, I'll quit.
Engineers not in the startup crunch should aim for using LLMs to code better, not faster. Faster just creates technical debt and ups your long term costs.
I am also a principal engineer.
The fundamental issue is that traditional code review doesn’t scale.
I now use a “Controls” linter which applies deterministic methods to constrain AI output, requires explicit declaration when key resources are used (file system access, network egress, outside libraries, or whatever I feel is important), and ties directly to other external requirements for business level controls that may be audited. (And more, but this gets the idea across)
Individual control violations can block PR mergers, block release, or just flag for review.
I genuinely don’t care that much about AI code quality these days (though I exclusively use SOTA models like Astra, which could be related). I care about code structure and architectural quality. I’ve found those can be deterministically enforced and made visible. If structure and architecture are high quality, I have no need to nitpick code quality.
If I were working with humans, this system would be cruel. It is far harsher than any manual code review I’ve ever given to humans. I let humans get away with small issues all the time. Dealing with the consequences of small mistakes is how people learn. For anything that really matters, I am ruthless with AI developers.
The system itself needs to be audited, so code review is still necessary. It’s more spot checking than full review though. You need to be a decent traditional software developer and architect to build new controls and to run control audits.
I think it also depends on what kind of software you are writing.
If you're in finance, aviation, or some other industry where the cost of errors is very high, I do kinda want a human to look at every line of code even today.
Not only to look but also write
Generating? Fine. But type it out again when placing it into the codebase to make sure.
But I'm not worried, some planes still update with a floppy. I'm sure that code is handwritten.
Agree here. At my current place we have LLM apply deterministic rules where given thing cannot be checked statically. For example - all unit tests shouldn't access database. This can be checked statically. All comments should document why, rather than what - this cannot.
Humans tend to bring up things that matter to them + correctness, when they have time to think about it.
You can test comments with a decision model like Jev.
The context is: > At my current place we have LLM apply deterministic rules where given thing cannot be checked statically.
So in this example quality of comments is checked my LLM and not statically
I think we're in violent agreement that the sort of code review we used to do is no longer necessary, but that some sort of review still is.
However, "doesn't scale" doesn't sound like a valid reason to skip code review to me. More code does not directly translate to more shipped features.
I don’t completely disagree.
There are definitely areas where I’m reviewing individual lines. Protocols are high on the list. I always want to know that high-level services aren’t leaking their internal details to the outside world and that whatever entity owns a protocol is controlling what is being communicated.
AI is bad at certain kinds of abstraction (for now). For example:
Connect service A to service B. Service C exists in some deployments and will do the real work in those cases, but A must never use B as a passthrough to C or use C’s language. That would cause future planned changes in the app to require a massive refactor.
Then, later, in review: Why is A’s client protocol to B referencing functionality that only exists in C? B’s entire purpose in life is to prevent this!
I haven’t tried to solve this via automation yet. I think it will involve limiting what AI has access to.
How could I learn more about this linter approach (especially around architecture quality)? Any other terms to describe it or resources that explain it?
Sounds very interesting!
Declare and deterministically detect the entities you want to control. Figure out how to list them. Then, write rules that enforce structure and behavior of or between those entities.
It’s just lots of declaration, detection, and fancy tests. But you need to apply a lot of attention to detail and make sure you have confidence that a given rule is enforced, and that a given entity can be detected. There’s more to it than that, but if you start playing around I’m sure you’ll figure the rest out.
I will warn that the goal isn’t to get AI to write code like a human would. It’s to enable controlled, high-velocity AI-driven SDLC that scales to large, complex applications.
I’d be really interested in how you do longing for architectural issues. I can’t see how that would work, especially deterministically?
Code reviews matter in the sense that you need to check the code is reasonably sane occasionally, but that's not the same as doing PRs. Leaving checks to the end has always been a dubious strategy - it's a lot easier to catch problems if you're designing the code up front and then reviewing that things are on track during the dev process. Finding that there's a problem at the end is the worst possible time to change something.
AI has solidified notion that for me. When your team are cranking out PRs at a rate of several a day (not building faster; making small, atomic change requests of independent bits) you'll never have the bandwidth to review everything by hand. You need time at the start to understand the change, lots comms during the process of making the change, and lots of automation around deterministic tooling (linting, formatting, unit tests, integration tests, e2e tests, fuzzing, property testing, smoke tests, etc) to make sure your AI change hasn't broken anything. If you have that in place the need to review at the end of a change is greatly reduced. Down to zero in my opinion but your milage may vary.
I'm not reviewing entirely by hand - I'm using an LLM too - but I know enough to guide the LLM towards areas that look like problem areas.
> both engineers and LLMs are still adding unneeded complexity or choose the wrong abstractions in ways that will cause problems down the road
Completely agree. But why do you think the reviewer would choose a better abstraction?
Yes, an experienced dev has a high chance of doing so. But in most teams there are not enough experienced devs.
And even I, an experienced dev, choose wrong abstractions all the time.
But fixing even large oopsies is just a prompt away. So imo it does matter less and less.
What was and is true: iteration speed just matters more. I would bet a lot of money that a team that goes from written code to merge in hours has a better quality codebase than one that takes weeks.
"Fixing large oopsies is just a prompt away" really depends on what you are doing.
I write software responsible for updating customer controlled devices. In theory, these devices may be in use for decades. I don't get to dictate when customers update their software. I have to use whatever software is on device as part of the update process, and the software update process really should work from every single version that's ever been deployed.
So whatever decisions I make / the team makes, generally we're stuck dealing with in some form for the device lifetime.
there is also sentiment I see that engineers should move from the code review to the testing of the end result. which is nonsensical because with the code review you can catch problems which are borderline impossible to catch via testing. but
the code review requires a tamed agent producing comprehensible code.
Totally agree.
We decided that we know the system well enough that we can self approve PRs
I would say people might have slightly different opinions I. The company, but in general, if it is not critical and it can go to preprod to check, we can merge.
I particularly do an automated code review and use some setup to check idea, code and documentation. But if it is critical or too important to go wrong I seek for code reviews.
The cost of writing code has gone down but not the cost for engineering.
I see my friends struggling with vibe coded projects because they didnt plan how to grow the baseline. The line got blurred as it became cheaper to do-redo but you can check any vibe-coder code cost through time and it does increase at least linearly.
I will say that now it is easier to plan specific changes that can put in and out in case of problems whether before we would need to do several rollout steps so multiple people could work on a task.
Now several people work on several things and the struggles are others.
Code quality has been a pain for everyone. I think for an LLM doing things one way or the other is a matter of what fitted the chances by the prompt and not good practices or choices. By this I mean, if there are two ways of writing the same code and one is clearly harder to read or performs worse, it can be more a matter of being lucky/unlucky to get it right throughout the whole code.
There are weeks I get 30 PRs done, there are weeks I get 3 PRs, like it was before. But reading LLM code is painful. It is obtuse and without lots of guidance it invent rules to problems that do not exist.
Of course I see that from my bubble, but I guess as experienced engineers are seeing the problems piling up and it the models are getting more expensive and not getting better as faster as before. These three things can be where we will delimit the usage of LLM for coding.
I particularly like that I don't need to do some of the annoying tasks that took me only checking 48302 files to be sure I got all the same string correct because some past bad decisions. But when I am working in important stuff, the cracks in llm makes me much slower.
A personal challenge has been to keep up knowing the codebase well enough while people (including me) are dumping stuff in there.
> still adding unneeded complexity or choose the wrong abstractions in ways that will cause problems down the road, don't think about all the corner cases, and pick the wrong handling for error conditions.
These are all one of those things which have never been easily quantifiable and therefore not easily sold to the stakeholders (your boss, boss of your boss, etc.). The AI will only emphasize this condition.
I agree that we should be using these tools to work out more optimal solutions.
We need to go slow before we can go fast.
Yeah, there was a lot of "things are working out so far" in that post. Perhaps they'll be proved right, but it set my spidey senses tingling.
[flagged]
Why do so many people think that code reviews are primarily about mistrust or spotting mistakes?
IMO they’re primarily about ensuring shared understanding and agreement, not just about how things are changing now, but how they’ll change in the future..
>Why do so many people think that code reviews are primarily about mistrust or spotting mistakes?
because people generally think things they are told, basic memetics.
There are a set of things code review is for, this set of things is for written information transfer and rather too large for speech, so when choosing to speak about what things code review is for people choose some, shared understanding is slightly more abstract than catching and reducing bugs so catching and reducing bugs gets repeated slightly more, and the next generation of people hearing what code review is for repeat it.
As a general rule I have seldom seen bugs get caught in review, I have caught a few bugs in review, but far fewer than the amount of code I have reviewed. I have very few times had bugs caught in review of my code, this is not because I don't make bugs, they just get caught out in testing.
Normally my experience is much of the communication I receive about code I've written is generally bike-shedding, probably 70-80%, above that it starts to become things we can absolutely agree is useful in some way (even if not at the bug catching stage), bug catching is maybe 0.01% of the comments.
I think both are real problems -
1. why should we trust the AI work? and
2. make me understand what the agent team did, while it spinned for hours
I think no 1 will be solved by N-version coding, keeping a population of replicas which provide differential testing on fuzzed inputs or prod data. This also helps no 2 - where execution diverges it can be a bug or a missing spec. Either fix code or fix spec. The reason I think N-version approach will win is that it is easier to code again the same app than to imagine code execution and its edge cases. Let real execution flesh out the divergences. It is easier than formal verification but still complex because it requires a careful setup where randomness, APIs, databases and other external state can be controlled for comparison.
OP said they use this question "What’s the cost that made this a bad idea, and is that cost still there?" to triage. In our case it is getting cheaper to spend tokens, N-version stops being too expensive. Having a population of replicas instead of a singleton would be the safe approach. It can even be faster, you can build the replicas in parallel, while test creation would take extra time on top of implementation. It's basically unbounded testing from input side. Makes performance optimization of one replica safer too.
Having worked in SWE for two decades, I trust AI far more than humans. The top 10% of human programmers did great work when given the right environment, not under pressure by leadership to deliver too quickly, not forced to use or work with partner orgs that sucked, etc. But the majority of people I've worked with, while lovely people, produced very human code. This is why AI is able to find so many exploits.
Yes, people are struggling with status and purpose, but the reality is Opus 5.5 onwards has completely changed reality. We have to harness it.
> This is why AI is able to find so many exploits.
If this is your metric, why aren't you starting with something like a CI step where an LLM looks for vulns?
N-version isn't a new idea. You quickly run into a problem with evaluation. Fuzzing is slow, non-deterministic, expensive. Building and verifying a test suite similarly so. Building a program that will evaluate whether an arbitrary program conforms to an arbitrary spec is an unsolved problem (consider how you would verify such a program in the first place).
People keep thinking of the coding step as a discrete, and thus also slow, expensive, non-deterministic step, but engineers did a lot to make it less so, whether it was techniques, tools, process, culture, whatever. Like, imagine an incredibly good agent working on a 2kloc PR for ~120 hours, periodically checking in with 4-5 other also incredibly good agents with near perfect knowledge of the system they're working on, constantly reevaluating the approach, periodically considering whether the existing codebase and architecture can facilitate the types of functionality you're adding, spending considerable effort to make it as easy as possible for other agents to add features, improving itself through these efforts, building ideas for new tools to help it, etc etc. It wasn't just code generation; so many useful things were happening in parallel.
You can think of it like dating in a huge city. In smaller cities incidental contact is frequent and encouraged. Dating is a bunch of meet cutes, dating in the workplace, ad hoc get togethers, etc. In huge cities incidental contact is overwhelming, oppressive, and discouraged, so if you want to meet people you have to expend considerable energy deliberately doing something you used to get relatively for free: talking to people you may want to socialize with/date/whatever. It's of course possible and people do it, but it's much harder, and we're only starting to become aware of the second order consequences of it (loneliness epidemic, growing political separation between men and women, falling fertility rates, etc). It's to the point where getting into a relationship as an adult is a part-time job, and fewer and fewer people can bring themselves to put forth the effort.
Swapping in LLMs for humans for the code generation process has the same dynamics. You can have more meetings about architecture, you can have it slop out more tools, you can have it analyze the last N PRs and have it slop out something that would've made those PRs smaller/simpler/whatever, you can have it rearchitect whatever you want (kinda) but you have to constantly push yourself to do it. You're no longer internally motivated to do it because you're no longer personally affected, fewer and fewer engineers will be able to put in the effort, and things will start to fall apart.
Maybe this won't be a problem. Maybe the code will be good enough and we'll find we don't need these higher level operations, i.e. we won't need engineers to spend weeks doing tedious tech debt sweeps every quarter in order to keep the lights on. But my guess is our ambitions will grow until we do.
Do you need to look into the code for that though? Can't you talk with your agent about the changes introduced and the direction the PR is taking your code?
P.S. Have a feeling this will get down-voted but posting since I am interested in counter-arguments for this approach.
Yes you do. Definitely not in the excruciating detail we used to. But an agent on itself typically catches only (relatively) superficial issues. It hasn't ever occurred to me to get a higher-level disagreement on the changes of the PR by an agent.
E.g. why re-implement the oauth workflow instead of using the established package of the ecosystem? Why monkeypatch an existing python method for all code instead of implementing and using where needed a context manager?
The agent can give you an assessment on such alternative approaches if you ask it, but I haven't seen it taking the initiative to radically steer a PR.
No because your agent will hallucinate. Code is truth.
Of all the many things I hate about this era of tech, the elevation of bad programmer culture to the stratosphere is definitely top 5.
I don't think your observation is limited to programmer culture.
AFAICT, it applies to everything. Few people are exercising any thinking skills because they put in a half-formed broken thought in fractured english into the prompt, and then use the result.
Emails, general comms, problem-solving... everything is now at the point where the lazy fucks are no longer detectable in the workplace.
They also pose the question: 'Are we building the right thing?'. Which is something that is really hard to resolve when it is just you and your LLM echo chamber. You don't know what you don't know.
I probably disagree, and this sounds like coping rationalization.
Reviews ensure that the writer's hypothesis has an antithesis to create a better synthesis. It follows fairly naturally from the academic background of engineers'. Structural engineers don't have drawing reviews to ensure shared understanding and agreement; they really just want to avoid being liable for killing people. That's the structure that software engineering evolved from.
Do you mean this?
> second set of eyes ... spreads knowledge around the team, and keeps any one person from quietly drifting the codebase off in their own direction
[dead]
We don't need shared understanding and agreement in the AI era. The AI will do that for us.
Until the cowboy at the prompt adds something you don’t want and your product is on the hook for it.
> But a lot of the reason rewrites were so dangerous is that they took forever, and the old system kept changing underneath yo
I think a bigger reason is that people don't understand why the current system is the way it is. That risk is increasing with lower j understanding of the codebase. (doesn't apply as much to building the same thing in a different language)
I'm going to talk about the item on this list that I know best. The "building instead of buying" is the one I’d question.
The cost of writing the tool is mostly gone, on that I agree. The cost of owning it hasn’t moved that much I think. If I look after the software stack of a couple of small companies I know, the custom tools are those that end up hurting the most. Nobody remembers it was custom made, until the API call breaks, or the person who built it leaves. I think the cost of maintenance is particularly high.
An afternoon build doesn’t fix long terme maintenance, nor does it provide functional knowledge. I think generalists stand to gain a lot from AI, but there will still be a huge role for experts to guide and maintain software in the future.
Yeah, that resonates with me. Like a lot of people, I started building several small tools earlier this year that I didn't have time for before, both for convenience and to alleviate pain points that I had. Turns out that this also means that I now have additional maintenance tasks that crop up at unexpected moments and I'm not sure if I'm getting a net benefit from it aside from the few that I use regularly.
There are maintenance risks for bought software too. A saas can change the interface in an undesirable way or the way a feature works or even introduce a new feature you don't want. Custom software can be more stable in that way.
I fully agree that there are maintenance risks when you buy software, but imho, those risks are much much more easier to mitigate: buy only "mature" software, make sure that maintenance is governed by contract, know your path to reversibility, etc.
Ultimately, in my opinion, this “make or buy” issue goes back to the age-old debate about how companies choose to adapt to the processes already built into the software they purchase. The companies that will “make” are probably the ones that would have developed custom solutions at some point anyway.
>Introducing a language or tool nobody on the team knows [is not a bad idea anymore]
OK, now I'm sure that the guy is full of shit.
Still bad, but definitely not as bad as before.
Sure - but it's going to be 70% vibe coding (because you can understand something about a not-too-exotic language) instead of not making changes at all. And that's the thing about the article, what the guy is effectively saying is "just vibe it".
To your note about nobody doing code reviews. I've found that to be fine in fairly straightforward applications and/or small teams where individual SWEs can own verticals.
As soon as you have larger teams with complex architectures trying to make decisions and push larger initiatives forwards, decisions made in one part of the stack really matter in other parts as they get built on and have momentum that can be hard to reverse, even if code is cheap.
Not to shill my work too much (https://www.archme.io) but I think for the reason above the importance of code review has moved from code to architecture, and code in general is the wrong abstraction to be working with as a human engineer in 2026+.
This is why you should start with design spec before starting a line of code.
If the spec cannot resolve ambiguity you refine the spec not the code.
Initial design and specs are very important.
However, as other folks have pointed out, it is not possible resolve all things initially. Especially with customers who have feedback. Building and iterating is critical.
IMO It’s important to have a way to collaborate and move forwards together as a team while building. Keep everyone working off the same abstraction so mental models don’t drift and people start building the wrong things
Most spec dev/designs doc are written by bots because this is what management forces us to do... ( by doing same in all other areas of company).
and yes you're describing waterfall as sibling comment said...
Development is about building shared understandingbwhat we want including code as physical manifestation. Not only code matters.
An unambiguous spec is indistinguishable from code.
This works for nothing but the trivial or almost trivial software.
a.k.a. the waterfall model
that straw man that someone came up with to illustrate bad software processes and then people started actually using it
There’s a middle ground between not writing any sort of design documentation at all and insisting that every little detail is nailed down in a Word document and carefully laid out on a Gantt chart before a single line of of code can be written.
Sometimes jumping straight in and seeing what happens is the right path, other times the changes you’re making will impact a bunch of systems and teams so coming to some agreement on what’s being done is valuable.
If your threat landscape includes "malicious insider" to any significant degree, this is not a path you can walk.
I’m glad someone called that out. No one person being able to get a change from their laptop to production without review is a baseline requirement of a lot of certifications, because while I trust the other people on my team the security team at a client’s client have no reason to.
Self review is absolutely a thing you can get away with in the early days of a startup, and at some point will probably be necessary because you’ve only got one developer, but if you have enterprise clients just turning off review requirements is a fast track to losing those clients.
Yeah. The problem with AI brained people is they have zero concept of information security. Most of their ideas would be shot down immediately, but they're shadow-ITing them into existence because "WeRe UsINg FrOnTiER Ai"
Something's wrong with that code review strategy. There should be knowledge share.
And yeah perverse incentive is a thing when you grad your own homework.
And these two things combine. Who decides what "done" means? If everyone faces consequences for getting that wrong the next day it's probably fine, but they probably don't.
[flagged]
I like the broader point about questioning the economics behind established practices. But building software faster and owning software for years are still very different things
I found this insight very useful: "When my gut tells me something’s a bad idea, I’m trying to ask the follow-up question: what’s the cost that made this a bad idea, and is that cost still there?"
I am reflecting on earlier "tooling transitions" and how they affected my methods.
Switching from a typewriter to a text editor made drafts and rewrites much, much cheaper. Shared edit documents sped things up further.
Text comparison tools (and related tools for source code) made design reviews much faster.
Moving from handcrafted financial scenarios to computer spreadsheets made "what if" and scenario generation and comparison two orders of magnitude cheaper.
Many aspects of code review have already been automated; LLMs just extend our capabilities further in this regard.
On one hand, I like the idea of being 100% responsible for reviewing my own code because it allows me to get changes in quicker.
The way we've been working is that we own a feature end-to-end rather than just a slice of it. So I understand my work, and other people understand theirs. In that context, not having code reviews makes some sense.
This was a change forced by management, I am but a peon
But part of me already misses what code reviews used to bring: learning opportunities. I can no longer rely on someone more senior than me to give me feedback, challenge my decisions, or help develop a shared understanding of the codebase.
> I can say to Claude "hey, bring me up to speed on this thing they've been building - how does it work, where is it at, and what still needs to be done?"
I think there's an opportunity to solve this differently. If you could see your teammates' conversations with their agents, you could actually follow the decisions they made and understand why. Although I appreciate that's a bit "big brother".
> What I'm recommending instead is to still build a very strong foundation in engineering fundamentals, because that's where I think we as humans offer the most value right now.
Whilst I partly agree with this, how do you actually do it in practice?
How can you possibly know what good looks like without writing good code yourself and experiencing the consequences of bad code?
I've been thinking of this as a privileged point of view, particularly from engineers who've spent more of their careers working pre-AI than post-AI.
You've had the benefit of years of experience developing the judgment needed to distinguish good engineering from bad engineering. You can now use AI effectively precisely because you've built that foundation.
But how are newer engineers supposed to develop that same judgment, especially if their workplace operates like yours?
But it's valuable to experiment with these instincts.
Isn't viewing others chats sessions and decisions just a poor man's ADR with way more (too much) noise?
Yeah it is really, never thought about it like that!
Even a slopped up ADR would probably be more useful than direct access to chats.
The common theme across those different examples seems to be: "Before, we had to be careful to allow each team member to build experience and expertise with our codebase and avoid dynamics that could form knowledge silos. Now this stuff doesn't matter anymore because everyone can just ask Claude to explain everything to them."
I'm not so sure about that, honestly. In the short term, it works, yes. But in the long term, it might be as if you have some very good books, but zero hands-on experience with a thing. We all know that book knowledge is different from learned experience.
> The thing that’s really changed here is that the cost of context switching has dropped dramatically, because we’re operating a level higher than we used to.
I completely disagree with this.
I would argue that the cost of context switching has dramatically increased, since within the agentic pattern, an engineer is now forced to work at a higher pace across multiple topics; even if operating at a higher level, the amount of information required to be processed by a human brain is generally overwhelming.
[flagged]
I think it depends on how you approach vibe coding.
If you want to actually understand the code that goes into production or the engineering behind it, then yes, cost for context switching has increased massively.
If instead you just hand off engineering and understanding to the machine entirely, and reduce yourself to copy&pasting github issues into Claude Code and clicking the green button on its PRs, then there is pretty much no cost to context switching (because your brain is devoid of context).
Without commenting on the specific cases, this is very true in general. AI is rewriting the costs of certain activities and so we need to recalibrate the cost benefit analyses that were infrained into people. Now is the time to ask why and why not. Because it might turn out that the reason we just didnt do something might not exist anymore
Disagree that you no longer need PRs. You no longer need them to check the code, but you still need them as a mechanism for consensus on achieving a shared goal. If my teammates are just going crazy on the code with Claude, it may as well be a solo project.
And that way, bit by bit, we become dependent on a few global corporations that own the brain that we used yo use. Brain as a service, dammit. Don't get banned on a whim or you'll lose your ability to think.
You could say the same thing about computers which are required for programmers to do their job. The entire profession hinges on a few global corporations deciding not to stop making computers.
Well, one difference is that you don't rent your computers over the iterne... oh wait! (That is what "There is no cloud, only other people's computers" is about.)
> second set of eyes ... spreads knowledge around the team, and keeps any one person from quietly drifting the codebase off in their own direction...
Supposedly, this is what the "deep well" and "15 years of experience" dictates - knowledge must be spread, drift should be detected/prevented early.
> And it’s been working great! ... we’re only one month into the change
But this is just the most obvious nonsense? You can't assess that kind of change in a month?Especially since the big issues can be "pretty rare".
I guess, drift could be swept under the Claude is better at inconsistencies, but then there is no knowledge spreading. What's up with that? It's wisely not even mentioned...
> Deep expertise in a specific technology is a lot less valuable when you have a collaborator that has read all of the source code for that framework and can answer any question about it in a few seconds. This is another case (like what I mentioned above) where knowledge is less valuable, but where judgement is still important
Yet you can't even judge the obvious - that assumption is a fantasy, the models can't answer any question about it. Also, less obvious, the lack of knowledge constrains the quality of questions you can ask, which negatively impacts the quality of the solutions you implement based on those frameworks
> Bad ideas that aren't bad anymore
...
> About a month ago we changed that [PR process] completely.
...
> And it’s been working great!
How can you tell it's not a bad idea in just a month? How can you tell that it's been working great in just a single month?
Seems that all you are measuring is short-term gains, not long-term liabilities.
You can see the same "gains" in any endeavour if you're willing to eat the seed-corn.
Exactly. That’s how I can tell this person has absolutely no engineering sense at all.
> That human understands the problem better than anyone else on the team, because they’re the one who’s been working on it.
It used to be like that, now that human copy user story to Claude, and then "review" this thousands LOCs.
> It’s pretty rare (at least on the teams I’ve worked on) for big things like architectural flaws or unintended product impact to be found at this stage.
I found AI like to stick to this big things, regardless how good or bad they are and focus on minor things (at second thought - just like humans)
How do you handle teammates who haven’t yet mastered getting their agents to produce a level of quality and elegance? That’s where I see code review still being useful…
> Deep expertise in a specific technology is a lot less valuable when you have a collaborator that has read all of the source code for that framework and can answer any question about it in a few seconds. This is another case (like what I mentioned above) where knowledge is less valuable, but where judgement is still important.
How can one judge on something they don't know?
They were only bad ideas for puny human brains. But for super intelligences...
Experience has less value now.
Agree with the first part, but I would say experience has more value now because that is what provides the intuition needed to effectively use AI agents without needing to review every little thing. The problems they create are now at an architectural or design level, and you need a corresponding level of expertise to catch those issues and direct them properly.
The rewrite being cheap is about to become really disruptive.
I started rewriting the client's product and they don't even know I've started. The neat thing about software is that you can have many concurrent versions and you can keep any number of those private for however long you please. It's not something you need to make a big deal about anymore. You just put your head down for a few weeks and then it's surprise demo time. Don't even give the bastards a chance to see it coming. They'll almost certainly try to stop you given how the last attempts probably went.
Most of these were never bad ideas. Big tech has been poorly incentivised for a decade or more. We killed code review before we got copilot.
This feels like this was written by an LLM, so of course it's trying to defend bad ideas.
We've done similar thing as the OP, I think it will be 2 months now. Our processes were pretty focused on spotting defects beforehands already: we have standardised tasks defined with ACs following given/when/then format that we can write behaviour tests for, we have automated linting, static analysis, automated tests. Then there's also round of tests: both QA (LLM/human driven) and UAT (human). Since I joined we have added architecture tests to that mix, including static as well as LLM guided that look at how code is built and organised + writing documentation for what is being built / product rules (LLM driven). Looking at the stats - the amount of merged PRs has trippled, the amount of issues is decreasing, PRs time-to-merge went down to roughly a day (I don't attribute this particular metric to usage of LLMs in review process).
What I've noticed = good sides: LLMs are better than humans in spotting edge cases and reviewing code, given they have established conventions and standards (I saw PRS that I'd approve, but LLM was pedantic in reviewing a comment or class name); they are good in finding connections, ie I've had many cases where an agent built something where there was no guidance in the ACs and the reason was that it found related tickets in Jira mentioning given thing and filled in the functionality. LLMs are "cheap" to run = one can run review process for multiple PRs at the same time and forget while they run; they code defensively requiring developer to intervene - for example: I'm working on functionality where user is assigned to review a thing. Only one user can be assigned to review given thing at a time and there are checks for this. In last 3 PRs LLM is telling me - add locks because if there are multiple people reviewing given thing at the same time then the data will be lost. Yes, if we ever change business rules in this place this can happen. Finally LLMs are semi-consistent; sure sometimes they'll do stuff for shits and giggles following some convention I've not seen anywhere in the codebase, I ask what happened, and it tells me it picked it up from some tangentially related legacy place. Most often they follow already established conventions / adhere to rules in place.
Now the bad points: many times QA catches things that would emerge during planning or if developer would question what they're building or look at their work thinking critically (I find it that people are loosing the ability to question things) - LLMs dont help with preventing this from happening. People are loosing track of domain and product rules - in the places I've worked I was never just a developer, I was always a developer/something (forever to my dismay), so during planning or review I could say, from product perspective: this doesn't make sense, we don't need to build this. This no longer happens - there's no time where I can remember the rules that govern the product; PM often becomes the bottleneck of what the rules should even be. I'm also finding that people do not develop good taste - this piece of code that LLM used? Sure, looks good, go ahead, ship it. Path of least resistance is becoming a norm. It's also easier to create bigger PRs.
Anyhow - I'm old and tired, so sometimes I'm finding the LLM pros are greater than cons
[flagged]
[dead]
[dead]