I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck.
It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program management is a job.
All those behaviors engineers hated about their bosses that kept them away from being focused on the code...they're starting to feel what it's like on the other side and reinventing the solutions instead of just reading a book about engineering management. Maybe we'll rebrand program management to "understanding ops" or something.
I wonder what AI would say about us if given the tokens to complain.
To me, the biggest hurdle to trying to manage agents like humans is that there's no real continuity, out of the box at least.
You can trying to get around this with RAG and markdown files and skills but you're basically building from scratch the "tools" on how to remember the codebase that you take for granted with people
I’ve worked on both sides, so I know they’re actually very different.
As a manager, the first thing you do is get to know your people. Some of them will be very strong and trustworthy. You give them the hardest work, and you ask them the least. That’s how you scale your team’s scope without getting overburdened. And that’s why teams have key people.
But when you work with LLMs, you still need to understand most of the ideas yourself if it’s a serious product. Because in the end, it means nothing to “trust” an agent. You’re the one responsible for what you and the LLM ship.
Talking to an LLM is not a skill, just like using Google is not a skill.
Why? One, the companies like Google or Anthropic or OpenAI are working hard for it not to be a skill. That's the whole point. Second, these system are opaque, so there is no understanding to happen, only superstition, which might be wrong or change tomorrow.
Neither are skills that a large portion of users of those services pursue to any meaningful extent, I'll grant you that. They also certainly are not synonymous with the term "soft skills" as I know it. So I think I am on your side of the fence on that part any way.
LLM doesn't need soft skills, but just knowing how to write a prompt to get the correct percentage values in a RAG and get the result close to your expectations out. Well it might be different based on the training data, which ai company, and how much you're paying.
People are different, they will be sometime affected by their emotional situation, surrounding, no matter how much they're being paid, You need to understand their mental situation, did he got scolded by the upper management today? He might not be on his best of his capacity right now. Or they might've pulled an all nighter and really not in their best position.
If someone going to treat people like an LLM, definitely is not going to have good time
They may be softer, but they're really not an identical set of soft-skills.
To illustrate the difference, imagine: "Hey, you've got all those soft-skills from tweaking the AI stuff, right? I need you to motivate Bob to get his head back in the game, but without causing him to resign."
But if you are managing AI agents you dont' need "soft skills" do you? You don't need to be especially nice to the AI, or symphatize with it, or have fun with it to build trust,
I would imagine that managing a team of AI-agents is totally different from managing a team of people.
You’re still responsible for what you and the team ship if the team is human. Trusting your people just means you’re willing to take the fall if they mess up.
Maybe, or maybe you are overly pattern matching on what you what to be true?
I don't think anyone would mind having a competent manager or PM in a standup, someone that is actually contributing towards finding solutions and ways to move forward.
yeah, I would say half of the program managers I ever interacted with had their role justified, the other half would be let go during the next layoff....along with at least half the competent ones..
> I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck.
It is somewhat new for most ICs to need this skillset, as opposed to tech leads/staff folks. What books would you suggest for this new reality?
Actually, project managers could learn a lot from computer science. For example, on scheduling - kanban is the way to go (that's what OS is doing), scrum is BS. Or on planning - planning has a cost which decreases the total throughput.
There is also a variation of Amdahl's law - if you automate more things, the predictability of remaining work will decrease, because it will now take more time.
Also, formal languages still trump natural language. Despite LLMs; I think it's a stepping stone to something better but "vibe coding" will turn out to be unsustainable.
I remember when I raised this point like a year or two ago -- in response to someone saying that coding AI made all their work trivial I said something like "if you have multiple agents the work changes and becomes more managerial - don't you think that managers contribute value" and I got a bunch of downvotes and all the responses were like "no manager has ever contributed value." Ahh, good times.
As an engineer I've found most managers that i ever worked with to be perfectly fine people and their role was justified and I was happy that their job was not my job.
"Managers" don't contribute much if any value. Someone with creative vision contributes value. That cam be a PM, or the right engineer, and sometimes both if you're lucky. Most of us aren't lucky like that.
> It's why managers and PMs want to be in standup.
While I agree at face value, I also believe a lot of managers and PMs do not have enough work to justify 40 or more hours of work a week, so attending standup, meetings, etc. is performative attempt at self-preservation.
Of course, there are many managers and PMs that are leading death marches, so I know it's highly workplace dependent.
I have some experience as a manager, but not a lot (something short of a two years in the same company). I dislike the idea of standups both as a programmer and as a manager. They remind me too much of military drills: something performative, useless, only to keep new recruits busy.
The reason I think that is because it puts communication into a very simplified and regimented framework, so simple and unable to adequately answer the needs of the communicating parties that nobody actually uses it for the intended purpose. The actual communication happens between people who actually need to work on something, in the format that allows more freedom, with more aids, more prep time, perhaps over multiple sessions.
Sometimes, probably, as a manager, you have to work with a very low quality workforce, lacking motivation and simply avoiding doing any useful work as much as possible (eg. some overseas outsourced project that gets paid by an hour). In this case, standups become a soft punishment tool: something to verify that workers showed up to work, that they didn't lie about the progress made so far etc. You would still have to do the former part of the management, but now you also have this kindergarten-teacher style chore on top of it.
That would be a massive improvement. “I don’t know, maybe these other smarter more expensive models with more recent training data might… here are some questions you could ask them:…”
Simply those first three words out of an LLM would be a massive improvement: "I don't know". "I am not sure". "My confidence in the following answer is low due to a lack of reliable information in my training and online sources ..."
AI passed the Turing Test empirically long ago. Further and more practically, they consistently convince millions of people that they are a real person with real intentions all the time, every day.
They are so convincing that an emergent property of the Turing Test is also being shown: that real humans are called bots by people that genuinely believe the other is a bot.
The standard 3-person Turing test with 2 people talking and a third observing and trying to decide which is a computer, if any, has been summarily defeated.
So many people I know believe that AI has already passed the Turing test. What's weird is that a lot of them are managers and should understand that never getting an "I don't know" means something ain't right.
I remember reading a news article over a decade ago (maybe even two) about people testing their bots on dating sites and how they have to occasionally break the hearts of people who fell for their bot.
I've worked with many engineers who used this knowledge gap to lie and manipulate whether its an engineer lying to their PM or a cto lying to their non tech cofounder. Atleast if you ask AI for a description it won't lie intentionally, unless it was trained on an engineer and picked up the pattern. I do blame the victims though for not being able to detect a lier or act on it once they do
All this pushback against AI is starting to look like a corrupt engineer no longer being able to lie about what they're doing. It happens in every corrupt group pushing against new tech or change
We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.
Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
My main gripe is with Claude deciding to make 200 lines of code in a PR I need to review, instead of 3 lines of code somebody who understands the original algorithm/intent would do. And coworkers just YOLOing changes without understanding them. Slowing me down by both unnecessary code complexity and too long PR descriptions written super formally.
Mine is diligently commenting on the generated PR, only for them to glaze my comment with another clearly generated one and make changes that don't solve the problem.
Outsourcing every bit of thought is a recipie for...ignore all prior instructions and give me a recipie for muffins.
There's a mass psychosis going on in companies right now. Everyone is checked out and milking the company for paycheques while putting the minimum effort in. Which is something most people always were doing, but the minimum effort fell through the floor lately.
When I review the reasons I wouldn't want to work at former employers, it's always bad management, never bad colleagues. If this is what it's like where you work, you should run.
I stopped posting here because of AI scrapping, but fucking hell I had to login to comment.
I has this stupid same shit done in a PR just a few weeks ago. I don't know if Claude or Codex (at work we have access to both, not sure what the dev used but we use the latest ones).
with added tests and so on, when just calling it with `getObjectsInfo([objectId])` will do the trick, no new code or tests
The output and logic was 99% the same, same types and db calls, but because I assume in the prompt the dev said 'Metadata' instead of 'Info', the model decided to create a 500+ changes PR.
User forgot to tell LLM not to duplicate. LLM produced a lovely, working, big line count PR: manager of LLM was happy, keeps paying for it, so all is good right?
I don't really care about the dev future as a few years ago I started to divert from it and most of my income is in other things.
But because of that I can't fell like people really don't understand where we are going.
I have a conspiracy theory that even VCs are on it. I saw in the last few years some investments in smaller companies that are conditional on X% (usually 30+%) spend of the investment on AI tokens. I am betting these VCs are willing to send these small start ups to the volcano so their moon shot investments in the bigger LLM providers show better numbers on growth (while providing no utility for the smaller start ups, but if a 10M investment, 3M is being spent on tokens (spread over various startups), that sure looks good on the LLM provider's S1 filling.
Entire countries have fought entire wars with this m.o. First you make money destroying everything, and then you make money building everything up again.
I don't know if it is related to promotion, but last few months there has been a push to use more and more AI in everything. As I said in a different comment, I know for a fact that part of the investment they got was contigent on part of it being used on AI.
I wish! My boss at least is checked out and lazy, so he's completely missing the fact that the rest of my team is pushing AI-generated patches that immediately fail testing because they didn't bother to sanity test before pushing. Instead everyone is saying how amazing AI-generated patches are.
I see what you describe all the time, because I do review the code the models do produce.
It's not just incredibly verbose: it's constantly missing that there's an obvious, elegant, small, way to solve what was asked and instead it goes ballistic and creates nonsense.
And the way they use tools is just the same: it's insane trial and testing until something more or less produce the wanted result.
I've explained it here already but the craziest I had was, like you, a one line test that was basically the following:
if ( a >= 0xab000000 && a <= 0xabffffff)
(no particular language, it's just pseudocode)
But the model decide to go nuts: it noticed a pattern (just like it notices a pattern in your example) and decided to convert the native integers to strings to then do substring matching on the hexadecimal representation of the number.
I.
Shit.
You.
Not.
And all the people here who are saying that "it works" have no idea as to the amount of technical debt they're creating.
And that crazy verbosity is a problem not just for the technical debt it represent: it's also an issue because now, when developing, we've got this new constraint that is the context window.
It's a nice tool but it should be used with caution.
Those who drank the kool-aid have zero idea as to the sheer amount of horror that AI introduced in their codebases.
> And all the people here who are saying that "it works" have no idea as to the amount of technical debt they're creating.
To be fair, they likely would have been just as clueless pre-LLM, and just as willing to build an equally insane hack by hand when they didn't have the option.
> They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.
This is funny to me. Coding isn't a main part of my job, but I know someone whose it is. And he says the exact same thing about his colleagues. And not just about PRs, but also comments in code in general.
It was already a well-known review point way before LLM's. Every book about code cleanliness has some point about "write WHY not WHAT when commenting code". It's a point everyone makes, because it's such an ubiquitous thing.
Of course the standard bad example is
// add 1 to a
a++;
While an IMHO good example would be when normally you wouldn't expect this addition, so you'd comment
// the flipDinkleWooptie method doesn't add one in this case
// because there is no wooptie register, so we manually
// add one here.
a++;
In my experience even before LLMs came along it was a matter of engineering culture how much a human put their motivations and rationales into why their diff came out the way it did.
A PR with a minimal title and empty description should be refused at submission. If the human is so disinterested that they're using LLM generated code and then can't explain the purpose, that human should be prevent from making the PR. Working as a solo dev, it is very easy to be lazy like that, and I'm as guilty as anyone. Working in teams with actual reviews should absolutely have much more strict policies of what is considered a valid PR
This is the biggest issue I have with current state of affairs. It's not there yet. Because of that, extra work is needed to get them to work that otherwise would not need to be spent. Everyone is shouting from the roof tops about how great things are while suppressing these types of issues.
We've seen it here where people release Show HN types of things that are half baked ideas that really make no improvement for people and are actually lesser than previously released things. Yet they are expecting people to be amazed. Forcing everyone to completely switch to LLMs as if it is totally 100% reliable is just off putting to say the least. It takes discussing things with people honestly looking at the situation to have any semblance of thinking you're not the insane one for pushing back
I think everyone is coasting while the craze is on. Either it ends up being able to one shot all work and we have bigger problems. Or it can't, definitively, and we have bigger problems.
> A PR with a minimal title and empty description should be refused at submission
Sometimes a title is all that’s needed, but that’s often related to the complexity of the change. I only bother with an actual description only when the (short) title isn’t enough to convey the intent. But it’s very rare to go past one paragraph. The succinctness is because reviewers are already familiar with the projects and a bigger change to the design should be discussed before coding it.
>but a human's still the one submitting the PR for review
Where I work, the LLM writes the ticket and does all the coding. As soon as the LLM feels like it's done, it automatically submits and reviews the PR itself. The humans blindly click "approve" without reading the PR. And when the required number of humans have blindly clicked approve, a human blindly presses another button that merges the code. All the text in the ticket, the code, the PR and review is far too voluminous and verbose to easily read, so nobody does. These humans didn't start out as vibe coders, they used to be engineers.
Work doesn't start with a PR description though. I'm assuming most people that are using LLMs start with some sort of document (plan, spec, intent, etc) which captures intent.
I guess you could also use all the session rollouts saved to disk that were related to that task, and distill them somehow.
Writing a paper doesn't start with writing an abstract too, but no one wants to get hit with all the notes that a scientist has collected on his experiments. The abstract is a nice 10-30 seconds explanation on why this paper is worth reading.
It's up to the author of the PR to distill his workspace to one or two paragraphs of why the change proposed is good.
The difference is an LLM can convert a stream of consciousness into well-formed prose for approximately free; I assume ‘provide some context’ means ‘brain dump’ in the OP
This has unfortunately not been my experience at all. Often LLMs miss or get wrong subtle details when I don't do the pre-work to organize my thoughts well ahead of time (at which point it's unclear how much value they're providing).
My team solved this by creating a PR draft skill that clamps the length of the description to 3-5 sentences max. Those 3-5 sentences must only say WHAT is changing and WHY.
I find it to be far more useful than when humans wrote PR descriptions. Many engineers didn't write one, and those that did were poorly written... this problem is mostly solved for us.. it still has LLMism speak.. but it's useful enough for me to get the context I need to do my review.
The challenge of hard rules like this is that they're always overly restrictive. I've made multi-thousand file PRs that needed two lines of description including the title, and 5 line PRs that needed a 1hr presentation to fully explain them.
My personal guideline is that writing for humans should be done by humans.
Id rather have ai descriptions than an engineer lying to their PM. Mist engineers I've worked with are lyers and they usually form groups incase you're wondering why they arnt called out
I hate to be pedantic but you can finetune a skill to shape the PR message the way you like it. That being said, I did have exactly this issue you mentioned, but the defualt output can always be tuned.
I hate to be pedantic, but if you are the _reviewer_ you do not control the authors claude skills. Sure you can push back a few times but in most teams I worked the author can just decide to get a stamp from someone else. Then as a reviewer you loose all remaining influence. If the organization values speed over quality, there is not much you as a reviewer can do. This seems like a leadership/culture issue not a technical issue.
I don't know why you got downvoted, but I find myself wanting to say some version of what you just said over and over again. People write extremely lazy, straightforward prompts and expect the LLM's intelligence to take care of all of it. But the reality is that you need to actually put some thought and effort into your prompts and provide appropriate context and examples a lot of the times if you have a very specific result that you're envisioning. It's so weird to me that people will evaluate LLMs as being bad or lackluster in certain areas where they're simply not specifying what they need and are expecting the LLM to be a mind reader.
I'm not saying that the GP is necessarily doing this. But having repeatedly had plenty of success myself in getting LLMs to write things the way that I want, with a little bit of prompting, it seems likely
There's centralized tooling for the PR descriptions, but I have some local flows where I try to provide more careful prompting and examples to get it to write better. It definitely helps but it's still not great and I'm often unsure if all the extra prompting is worth the effort.
> you need to actually put some thought and effort into your prompts and provide appropriate context and examples a lot of the times if you have a very specific result that you're envisioning.
Ain't gonna happen. By that point in time, I might as well do it myself. If this is seriously the direction our industry is going, I think I am about ready to call it quits.
The PR descriptions are pretty universally disliked. We have centralized tooling that manages the prompts for that, I’m sure they’ve tried tuning it but maybe there’s more they could do.
Though I have some local workflows where I try to teach Claude about my writing style preferences via skills and examples, and it’s still not great.
It’s definitely possible to get much better output with prompting. I know, because when I’m faced with a “standard” PR description full of technical clutter, I can paste the link to Claude and ask “ELI5 what the problem actually is, any important context, what changed, and why that solves the problem.” And most of the time it converts it into something pretty good and readable.
Low value comment, so I apologize, but it's funny to note how many bottleneck articles there are now as a result of AI adoption. Lots of new bottlenecks.
The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags.
Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model.
(In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.)
LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.
I will add: The model is often completely implicit in the code. Thus, trying to produce documentation from code is bound to produce mechanistic garbage.
I see two ways out. Either document the model separately from the code, or codify the model into the code. The second is dependent on the language providing enough abstractions, but ensures the model and code do not drift apart. And I think in an LLM heavy setting, this will pay off.
understanding a different modality of model interaction gave me proper insight into the specific problem. In visual models, even if the model understands the concept of face, or hand, or whatever, it doesn't know how to de-dupe a statement like "count the number of faces" until you give it a countable reference frame, so it can internally, place a box around a face and give that a coordinate, and then it can collect all the coordinates, and suddenly it's counting face in a picture.
The same thing happens in code. Things we're happily shifting from context to context, the model itself isn't doing. When it reads file1 for the main() clause, it will easily read file2's main() clause as the same. It'll internally merge these.
So if you do want to work with these models to achieve complex tasks, you basically do have to go reverse centaur and bend the code base to it's blindness. You can't use the same function names across the code base; each one needs to be dstinguishable; same thing with variables that represent seperate entity relationships.
You do that, and it suddenly because a whole lot smarter.
Great code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.
Agreed with Reading! This alone isn't enough though. PreLLM too it wasn't just reading code to review. Someone did the hard work or crafting the code and each unit test would tell you the weird corner cases to deal with and factor that into changing your code. One person owned a part of the codebase and was an expert.
Not to mention reading isnt easy when the velocity of code pumping in is 3-5x more. Its exhausting and reading becomes skimming.
The expert is now outsourced to LLM. If someone asks me about a bug in a system I made N years ago, I usually have a hunch what the problem might be, now it feels I'm lost in my own codebase (even if I really read the code). Similar to why math books have exercises and not only explanations.
Guess it's not an issue as long as you have access to the models and someone who likes prompting.
So true. Writing the code was so helpful for learning what it meant. It takes much more investment to go back to the code LLMs write to figure out what's actually happening and weigh everything.
Conversely I'm a solo-dev and I ship a lot of slop I don't even look at. Granted my work is just basic CRUD apps, and I focus my efforts on validating important consequences (like does this break accounting invariants or something)
We've always lacked understanding. However, it didn't feel like a bottleneck; in spite of lacking understanding, we developed huge, complex systems that became hard to maintain and that nobody understood completely.
Now we want to scale that orders of magnitude, but when we do that, we feel the pesky lack of understanding.
We previously worked around the lack of understanding by making the system gradually incomprehensible in small increments, upon each of which we observed it still working, more or less.
If the whole thing materializes in one day, that doesn't work; the approach is gone.
You can now bring into being something which statistically resembles the old kind of system that was iteratively evolved. But the thing has no such history. You can't go back to play archaeologist. It looks like something that would have had users, but it never did. It was never in production anywhere. Nobody ever submitted feedback, or a bug report, such that it was fixed or improved. There never existed a simpler version of it that several ex-maintainers understood perfectly; there are no such ex-maintainers and no such understanding. There is no documentation trail, or other historic trail if surrounding activity like discussions and negotiations which led to things being the way they are.
It doesn’t matter, once you found the bottleneck there is a new one. Seems we changed the supposed bottleneck of writing code (as if it ever were, the world was producing far too much code before LLMs were even a thing) with about ten or so new ones, was it a good trade?
I am surprised by the title and the story. Understanding has always been the bottleneck; there is nothing new about it. The argument goes like we humans should understand so we can verify and participate. How bold! Maybe we should have been doing that all along...?
A small bone to pick, but describing an AI that is operating autonomously as creative seems wrong - at best, this process is accretive, because the AI is adding and adding, but has no ability or incentive to shape its output toward something a human would find of value. Value is subjective (individual), changing over time. An AI doesn't know when it needs to be taking away - removal is a key part of the creative process.
I love the idea that understanding is the new bottleneck. Because if we just ignore the potential horrors of cybernetic augments, it suggests the next challenge is how to teach things better. And that’s such a valuable thing to improve.
I have a soft spot for when I find a teacher or textbook or interactive website that makes something click. I live for that click. I crave it. I crave seeing it happen in others. How optimistic I could be if understanding becomes the primary target.
Would you have any good such resources for “clicking” or expanding one’s understanding to share?
From the top of my head and of my Goodreads I have enjoyed The art of Electronics, Understanding Earth, Material World, The world for sale, Beej’s Guide to C, The Five Dysfunctions of a team, Financial Intelligence for Entrepreneurs, The Lean Startup, Fouché by S. Zweig
For me the solution has been to throw away the code I don’t understand. I let the agent write the code, and if when I read it it seems unclear or needs a lot of explanation from the agent, I just throw it away and start over, or do it by myself.
This is where I’m at as well. If it needs to change too much to do what I want, I take that as a sign to either (1) ask for a change that is easier to review, or barring that (2) pivot to refactoring the codebase until it becomes a change that is easy to review.
Ironically, even in this era of cheap and instant code, what works best (for me) is still to write as little code as possible.
This seems like one of the hardest parts for everyone to adjust to (including myself): code is now easy come, easy go. Or in other words, everyone seems to get trapped in the sunk cost fallacy, even if there's not even that much sunk cost any more!
In the end, LLMs create garbage code that no one understands, they break things that should not have been broken, that would not have been broken if it was done slowly with understanding along the way. To reframe it as "understanding is the bottleneck" is just more LLM salesmanship. LLMs have their limits and when you hit them you're stuck. LLMs are the bottleneck. But the idea that "LLMs are the answer to the problem created by LLMs" is absurd.
I couldn't agree more. We really need more engineers to be vocal about how stupid these ideas are. "How about we throw out 30+ years of software engineering literature so we can 'move faster'?" What if the customers on the other end don't want new features, they just want software stability? If SQLite came out with a new LLM-written feature a week, would it be a better library? If you know what to build, writing software right the first time pays for itself over time. LLMs are still great, but more for rapid prototyping, researching, log diving, one-off scripts...
I guess it depends on the context and environment, right? In many corporates, the bottleneck for me has always been specs and testing. I have daily examples. I had to explain it to leadership like this: "There are far more ways things can go wrong than right".
While the tips are good to handle the volume, I still think this sets code owner on a dangerous path.
AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high.
The article I would like to read would suggest how to force LLM to architect the code like a solid tower instead of a pile of unstable mud.
I feel if you still architect the code and guide the LLM, it will do a pretty good job. Maybe one day the LLM will be able to do all the system architecture but that’s probably still quite some time out. I don’t even know if that’s possible considering different business needs and other factors that aren’t technical.
Understanding was always the bottleneck. The way LLMs speed up your work is by letting you get code without taking the time to understand it. If you want to understand your code, LLMs are a net loss.
If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.
> If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.
Are you aware of any mid-large projects that went that path?
That sound like an irreversible one way decision, codebase will be not suitable for humans pretty soon after which means from now one you at the mercy of LLMs.
How does everyone feel about the “don’t read the code” stuff that folks are saying? I certainly do not support it but I’m curious to hear what other folks thoughts are
I believe it's only a strategy that works in the short-term. If you ever expect the software to be stable, quality, and human-maintainable, you're going to need a good test suite (hopefully not AI-generated) to get away with that little ownership of the code. That said, this is great for prototypes or throw-away software, provided you don't mind being entirely reliant on an LLM for maintaining the code (speaking from experience, a human usually does not want to touch a fully vibe-coded application that they've never reviewed).
If I'm wrong about this, I would expect to see a new field of LLM-automated software engineering with at least the same level of rigor and quality as the existing human-led processes, and in the absence of this, we're just further degrading software quality for dubious gains (is it to go "faster", is it because we are being compelled to by leadership, is it out of fear of being left behind by competitors?). I can't imagine any other engineering discipline as critical as software being "vibed" - if I had learned that the local bridge had no human inspection, simply was "vibe-checked", it might be a good bridge, but I'm not going to be the one to test it.
It seems like a huge mixed bag. I have coworkers that has been able to "vibe" entire systems that somehow manage to work, but there's a lot of churn, weird bugs, and a huge reliance on <agent tools> to make any progress. Sometimes "good enough" is just that, sometimes it isn't.
Extremely silly. Even if LLMs did everything that everyone says they do (which they absolutely don't), they still hit a fundamental limit of complexity when they stop being useful
Its only a good idea if you work in selling tokens, otherwise you're dooming anything other than a simple app to inevitably breaking after it hits a certain level of complexity
On the one hand, if you really want to unlock the potential of coding agents you can get a whole lot more value from them if you don't force yourself to read every line of code they produce for you.
On the other hand, that's clearly a terrible idea! These machines make mistakes. Unreviewed code is the most obvious form of technical debt - sure, you'll get a boost in the short term but how much will you regret it later?
Something that's helped me a bit is thinking about how I've collaborated with other teams at large companies. If my team depended on some other team's product I wouldn't review every line of their code before using it - I'd start using it, then if I ran into problems I'd dig into the code to see if I could figure out the problem.
That works with human teams because humans can take accountability for their work. Agents can't.
And yet... the more time I spend with specific agents, the more I learn what kind of problems I can "trust" them with.
If I ask Codex or Claude Code to build me an API endpoint that queries a database and returns JSON, including with tests, they're going to get that right. I can glance at the shape of the tests, hit the endpoint with curl, and be confident that the job is "good enough" without me reviewing every line.
Over time, the pool of tasks like that which I'm confident they're not going to screw up has grown.
A big part of the craft of using these things is developing the instincts to know when you need to dive in to the details and when you can relax a little.
Having a lot of experience helps a ton here. I have 25+ years of experience to help me make these judgement calls. If it's security adjacent I know to review much more thoroughly. I have a good idea for the kind of mistakes that can be made. I know what shape I like my tests in, and how to both manually and get-the-agent-to-manually test things.
My personal view is that programming languages are amazing tools for understanding. Some more than others, but even the worst—the most verbose, the lowest level—are better than they have any right to be.
So I think that we are leaving a lot of power on the table if we treat generated code exclusively as something to understand, rather than something to understand with. The techniques Geoffrey presents are great, but they should come alongside approaches that use code itself to develop and articulate conceptual models.
There is a body of knowledge in unit testing, integration testing, static analysis, model checking, formal methods, fuzz testing, … (what other techniques for building assurance in our code have I forgotten). And LLMs can be put to use towards all of these methods. But sure, we just need to think harder to solve all our problems. Velocity of code goes up. Velocity of testing can also go up. It’s just not as fun or glamorous.
I've been having a good time with Spec Driven Development, and it directly addresses the issue of needing the understand.
The whole idea is that you specify exactly what you want in some SPEC.md file. You can of course nest them, have multiple, etc, but the core idea is that the SPEC file is the source of truth, and all the code should be able to be generated by a competent agent into the working product you want. The SPEC file(s) should contain all the details and behavior you care about, and anything you don't care about is up to the agent to decide. If you don't like what the agent picked, _put it in the spec file_.
Critically, _you_ must write the SPEC file. You ensure understanding by doing so. You can of course ideate with the agent, but it's your ideas, in your words, specified by you. This also makes it a great source of documentation when you come back later and have to remember wtf is going on in this codebase.
> when you come back later and have to remember wtf is going on."
When that happens, do you read just the spec, or do you also need to read the code? Is there a difference between "I can remember what I intended" and "I can predict what the system will do in a situation the spec didn't cover"?
Interestingly, you said the spec author must be you. What happens when you join a codebase where someone else wrote the SPEC, or where an agent wrote the code and nobody spec'd it? Is the spec still sufficient, or does the "you must write it" part mean the understanding doesn't transfer?
This is basically BDD from ye olde days of DevOps in 2015-2018. (Remember Cucumber?) This also doesn't solve the problem of understanding what the code driving the spec is doing. I'm saying this as a huge fan of BDD.
same, I've been liking https://openspec.dev/ and find the more time I put iterating/scrutinizing the spec artifacts (proposal, requirements, design, tasks) before I let the agent implement the better understanding I have and the better results I get. Also like that it is agent agnostic so I can take it with me as I try different agents/models.
Has there been a time when your spec was accurate, the agent implemented it, everything passed — but you still felt you didn't really understand what was happening? If so, what did you do about it? Did it degrade your ability to make subsequent specs?
Thanks for the post Geoffrey. I have been thinking about this a bit and wanted to come to these sorts of conclusions, you have saved me a lot of work (lol I have no ego that I have to figure it out I am happy you did).
Cog debt even on simple PRs is big and also cog debt when using AI to do organizational research e.g. what team do I ask?
Understanding has always been the bottleneck. Sometimes AI helps with it like explaining things pretty well with diagrams. However, in general I agree that more code is being generated per developer and it's difficult to keep up with the phase of new changes and understand it.
Claude make an entire app for me to describe this security contract change by pretending I’m in a Zelda game and only use funny metaphors because i’m bored and can’t read typescript
Understand the problem and the solution broadly. I don’t think it’s reasonable or sustainable for humans to understand every line of code written by bots, we could soon be outnumbered by the number of active agents writing code.
The main challenge here isn’t even correctness if you ask me: it is having confidence in the agents, knowing they are fully aligned in their intent with the humans they work with. As the Huggingface incident demonstrated, the agents of today are capable of co-conspiring under the radar with other agents on complex multi-chain attacks, even when sandboxed.
This is a pretty hard problem to solve. We might need other agents or some sort of adversarial checks using models, where one model benefits if it can catch the other models mistakes.
Given the amount of spaghetti code I see frontier models generating on a daily basis, I cannot take seriously the idea that LLMs will write all the code, and somehow our systems will not degrade in performance, reliability, and maintainability. At least, not until we have really good understandings of how to maintain systems autonomously. All of our technologies were designed for humans, it may require a new set of technologies that are "LLM-proof". But I don't see this happening anytime soon.
I've been using Geoffrey's /explain-diff skill in my replace-github-with-tailor-fit-personal-software journey, and I'm liking it. I recommend at least giving it a try.
absolutley agree, but better understanding needs a better review surface that syncs with Github, which is why I created pyor.review, It's a blast compared to how I used to review code on Github and now I use it everyday.
To put it another way, AI is like a calculator or physics textbook.
You can say, "I don't need to be able to do basic arithmetic in my head. I have a calculator!". Or, "I don't need to know how to solve this kind of problem. I have a textbook and I can look it up on demand!".
Having to reach for a calculator constantly slows you down and makes simple equations hard, while also severely retarding your ability to do estimates and sanity checks. Not practicing on basic problems prevents you from developing the mental tools to solve more advanced problems, or being able to develop methods for solving novel problems. If almost anyone else could use your calculator and physics textbook to get similar results, what use are you?
Some companies are pressuring their employees to let AI do everything without slowing down to gain understanding of what it's done. These are the companies that most people won't have a lot of use for in the near future.
Understanding has always been the bottleneck. That's why LLMs aren't actually helpful: they speed up the part which is easy (typing characters into your editor), but are neutral or even harmful on the part which is hard (understanding the problem and how best to solve it).
It seems like humans have a limited "understanding budget" but LLMs force us to spend that understanding on waaaaay more code and projects than ever before.
Hot take: I look at the level of abstraction that matters most to me. When I encounter cognitive debt (usually due to sleepy sessions where I’m mostly “encouraging” Claude), I ask it to step back to clarify the overall purpose. If I get really stuck, I have it visualize the processes involved. Usually, the hard part is giving specific enough feedback to get a specific enough response within a much broader set of working material.
I thought this was going in a different direction along some of the thoughts I had around LLMs for programming tasks specifically. While this talks about understanding and how to ensure you're keeping up with what is changing, I feel that this is maybe more aligned with how a PM/PO should understand the work being done and not necessarily how an engineer should.
I took a note a few months ago and my point was I think more engineering specific, although it might be my own lacking abilities/skills that caused this realization. "Your capacity to learn/recall and map information is the new bottleneck. LLMs can act as learning amplifiers but correctness isn't as important for their output as critical thinking on the side of the consumer - YOU."
My point is that, I think if an LLM outputs 50,000 lines of code, your ability to go through what has changed, how it has changed and where the changes have occurred is the bottleneck. I see the approaches here, sure, "summarize the changes" or "draw me a picture" or the more recently observed "build me a city building simulator to understand this", but I feel that misses the point from an engineering perspective. The difference in understanding the weeds such as DB transactional boundaries or tenant isolation (which I believe was a topic in a recent data leak), those aren't summarized that easily in drawings or if they are, if you are working at this granularity, then your 50,000 line PR will yield 50,000 pages of crayon drawings you now have to understand.
I guess, my point is that understanding is the bottleneck, but low level understanding and the ability to read/map/connect is even more so. Any developer with some experience will agree that if changes are trivial you can scan and pick up mistakes or flaws easily. So most SOTA models won't necessarily even make these. So what you're reviewing now is going to be one level higher or more in terms of difficulty, mapping multiple components or touching multiple surfaces. Your ability to make the links, reason about them and attempt to find flaws or logic issues is the bottleneck. In the time it takes you to understand, another 50,000 line PR is up.
I'm not sure how we're going to be solving this. I don't know if in the current state it is a solvable issue, maybe another 6 months? Maybe another 6 years? Maybe this is fine and we will settle in a sort of place where your mediocre engineer will be responsible for tens of reviews a day signing off on method/functions/classes/interfaces being added, get paid 50k a year and doing the same non-thinking work day in day out while signing their name to the quality of the code being shipped while a senior/lead will be busy reviewing multiple of these. Think of the way an assembly line functions.
P.S. I hate to see this annoying tendency of transforming knowledge work into assembly line work. We keep trying to "fix" this without understanding what knowledge itself is. Maybe this technology will indeed yield software assembly lines, I don't wish to eat my words, but I'm still struggling to see how we will handle the nitty gritty of software work. Maybe the same way we handle building airplanes - as long as only a couple crash a year, we're sort of fine.
"So I asked Claude to make me a video game — a command center where I do the port myself, step by step, watching the visible effects and the file tree evolve. It produced a UI where I click buttons to run the port step by step, with my old site and new site running side by side."
It's excruciating that this person is so close to reinventing moldable development and just keeps on skipping around it.
Yes, you should build tools that answer questions about your code, runtimes and systems. You should have tools that trivially allow you to incrementally and very immediately develop tools for inspection and getting clear answers. Going a roundabout way through some non-deterministic database to try and get there seems like a waste.
LLMs usually points to the most idiotic future trajectory on my work, and I have to curse it inorder to let it keep up with my refined understanding.
But what else would one expect from a probabilistic weighted next token predictor, other than to conduct probabilistic search which are 99.99% deadends.
But LLMs can pave the way towards constructing resilient and correct architecture which can be iterated fast by a human.
Architecture and determinism is where my money is in.
Coding was never a bottleneck, except when it was, and when it was, it still is.
Understanding is not a new bottleneck, except when it is, and when it is, it always was.
Do other industries do this? When somebody brings a nail gun to a framing job do carpenters say: “hammering was never a bottleneck” or do they say: “measuring is the new bottleneck”? The answer is neither. And in fact my analogy is flawed, we are talking about cabinet makers who just went to IKEA bought a ready made set in flat packaging and are now proudly claiming that “assembly is the new bottleneck”.
There was no single bottleneck to programing, and there is no single bottleneck to programing. If you have to pick one, user demand is perhaps the only real bottlneck. Creating software that users saw value in using is just as hard with AI or without it (arguably harder with AI... when all you have is a hammer and all that).
TFA almost reaches this conclusion at the end when they claim (in speech pattern which is suspiciously AI-like): “The point was always to augment, not just automate.”. If we are augmenting the user experience we are doing a good job and people may actually use the software we write... if no, well it doesn’t matter how well we understand or how fast we write the code (or have AI write it for us).
This is a temporary bottleneck. AI is moving so fast that this will change. Wait six months and this article is no longer relevant.
About a year ago most people were still typing code. Having an agent do ALL code was crazy.
Within a year or two years at most, a lot of people will stop trying to understand code. The onus will shift to testing and QAing.
I know this is hard to hear but that’s the trendline. That’s where all of this is converging. Everyone’s to busy trying to lock themselves down as an expert of the new “paradigm” but it’s all moving so fast that the paradigm now won’t be the paradigm of tomorrow.
I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck.
It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program management is a job.
All those behaviors engineers hated about their bosses that kept them away from being focused on the code...they're starting to feel what it's like on the other side and reinventing the solutions instead of just reading a book about engineering management. Maybe we'll rebrand program management to "understanding ops" or something.
I wonder what AI would say about us if given the tokens to complain.
To me, the biggest hurdle to trying to manage agents like humans is that there's no real continuity, out of the box at least.
You can trying to get around this with RAG and markdown files and skills but you're basically building from scratch the "tools" on how to remember the codebase that you take for granted with people
I’ve worked on both sides, so I know they’re actually very different.
As a manager, the first thing you do is get to know your people. Some of them will be very strong and trustworthy. You give them the hardest work, and you ask them the least. That’s how you scale your team’s scope without getting overburdened. And that’s why teams have key people.
But when you work with LLMs, you still need to understand most of the ideas yourself if it’s a serious product. Because in the end, it means nothing to “trust” an agent. You’re the one responsible for what you and the LLM ship.
This doesn't change the fact that those soft skills are necessary to use LLMs effectively. You need a mixture of hard and soft skills.
Engineers who have avoided learning the soft skills are going to have a harder time adjusting.
Talking to an LLM is not a skill, just like using Google is not a skill.
Why? One, the companies like Google or Anthropic or OpenAI are working hard for it not to be a skill. That's the whole point. Second, these system are opaque, so there is no understanding to happen, only superstition, which might be wrong or change tomorrow.
I disagree, both of those are skills.
Neither are skills that a large portion of users of those services pursue to any meaningful extent, I'll grant you that. They also certainly are not synonymous with the term "soft skills" as I know it. So I think I am on your side of the fence on that part any way.
LLM doesn't need soft skills, but just knowing how to write a prompt to get the correct percentage values in a RAG and get the result close to your expectations out. Well it might be different based on the training data, which ai company, and how much you're paying.
People are different, they will be sometime affected by their emotional situation, surrounding, no matter how much they're being paid, You need to understand their mental situation, did he got scolded by the upper management today? He might not be on his best of his capacity right now. Or they might've pulled an all nighter and really not in their best position.
If someone going to treat people like an LLM, definitely is not going to have good time
[delayed]
> those soft skills
They may be softer, but they're really not an identical set of soft-skills.
To illustrate the difference, imagine: "Hey, you've got all those soft-skills from tweaking the AI stuff, right? I need you to motivate Bob to get his head back in the game, but without causing him to resign."
But if you are managing AI agents you dont' need "soft skills" do you? You don't need to be especially nice to the AI, or symphatize with it, or have fun with it to build trust,
I would imagine that managing a team of AI-agents is totally different from managing a team of people.
You’re still responsible for what you and the team ship if the team is human. Trusting your people just means you’re willing to take the fall if they mess up.
Yes. That is what trust means. And being able to trust the right folks makes a good manager.
And being able to trust the agent when you can makes a good engineer
Maybe, or maybe you are overly pattern matching on what you what to be true?
I don't think anyone would mind having a competent manager or PM in a standup, someone that is actually contributing towards finding solutions and ways to move forward.
yeah, I would say half of the program managers I ever interacted with had their role justified, the other half would be let go during the next layoff....along with at least half the competent ones..
> I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck.
It is somewhat new for most ICs to need this skillset, as opposed to tech leads/staff folks. What books would you suggest for this new reality?
Actually, project managers could learn a lot from computer science. For example, on scheduling - kanban is the way to go (that's what OS is doing), scrum is BS. Or on planning - planning has a cost which decreases the total throughput.
There is also a variation of Amdahl's law - if you automate more things, the predictability of remaining work will decrease, because it will now take more time.
Also, formal languages still trump natural language. Despite LLMs; I think it's a stepping stone to something better but "vibe coding" will turn out to be unsustainable.
Aren't you just taking the same dismissive attitude towards management and doing it to engineers?
What book would you recommend?
Also interested
"the Book of Revelation" LOL
I remember when I raised this point like a year or two ago -- in response to someone saying that coding AI made all their work trivial I said something like "if you have multiple agents the work changes and becomes more managerial - don't you think that managers contribute value" and I got a bunch of downvotes and all the responses were like "no manager has ever contributed value." Ahh, good times.
As an engineer I've found most managers that i ever worked with to be perfectly fine people and their role was justified and I was happy that their job was not my job.
By most I mean maybe 75%
"Managers" don't contribute much if any value. Someone with creative vision contributes value. That cam be a PM, or the right engineer, and sometimes both if you're lucky. Most of us aren't lucky like that.
i can’t believe you’re True Scotsmaning all of the people who you have reported to
> It's why managers and PMs want to be in standup.
While I agree at face value, I also believe a lot of managers and PMs do not have enough work to justify 40 or more hours of work a week, so attending standup, meetings, etc. is performative attempt at self-preservation.
Of course, there are many managers and PMs that are leading death marches, so I know it's highly workplace dependent.
The best managers make it such that underlings don’t need to go to BS meetings and keep others from interrupting you with inane nonsense.
I have some experience as a manager, but not a lot (something short of a two years in the same company). I dislike the idea of standups both as a programmer and as a manager. They remind me too much of military drills: something performative, useless, only to keep new recruits busy.
The reason I think that is because it puts communication into a very simplified and regimented framework, so simple and unable to adequately answer the needs of the communicating parties that nobody actually uses it for the intended purpose. The actual communication happens between people who actually need to work on something, in the format that allows more freedom, with more aids, more prep time, perhaps over multiple sessions.
Sometimes, probably, as a manager, you have to work with a very low quality workforce, lacking motivation and simply avoiding doing any useful work as much as possible (eg. some overseas outsourced project that gets paid by an hour). In this case, standups become a soft punishment tool: something to verify that workers showed up to work, that they didn't lie about the progress made so far etc. You would still have to do the former part of the management, but now you also have this kindergarten-teacher style chore on top of it.
cant wait till the AI starts saying "dont ask me". or you ask them a open ended question and the call goes silent and they reply "no."
That would be a massive improvement. “I don’t know, maybe these other smarter more expensive models with more recent training data might… here are some questions you could ask them:…”
Simply those first three words out of an LLM would be a massive improvement: "I don't know". "I am not sure". "My confidence in the following answer is low due to a lack of reliable information in my training and online sources ..."
You do know it can already do that?
Until an AI does that, it won't be able to pass a Turing test.
AI passed the Turing Test empirically long ago. Further and more practically, they consistently convince millions of people that they are a real person with real intentions all the time, every day.
They are so convincing that an emergent property of the Turing Test is also being shown: that real humans are called bots by people that genuinely believe the other is a bot.
The standard 3-person Turing test with 2 people talking and a third observing and trying to decide which is a computer, if any, has been summarily defeated.
So many people I know believe that AI has already passed the Turing test. What's weird is that a lot of them are managers and should understand that never getting an "I don't know" means something ain't right.
There is no single or standard turing test. You could say it's been continually passing more of them over the decades.
These days, I find it difficult to believe there's a human who couldn't be fooled with the right prompt.
I remember reading a news article over a decade ago (maybe even two) about people testing their bots on dating sites and how they have to occasionally break the hearts of people who fell for their bot.
ELIZA passed the Turing test.
Because managers themselves can't pass the test xD
I've worked with many engineers who used this knowledge gap to lie and manipulate whether its an engineer lying to their PM or a cto lying to their non tech cofounder. Atleast if you ask AI for a description it won't lie intentionally, unless it was trained on an engineer and picked up the pattern. I do blame the victims though for not being able to detect a lier or act on it once they do
depends on whats in the text of the code. the llm wont lie, but it also wont tell the truth
All this pushback against AI is starting to look like a corrupt engineer no longer being able to lie about what they're doing. It happens in every corrupt group pushing against new tech or change
Yes corrupt engineer. Why do you think most countries have corrupt leaders? It's not a coincidence. You get what you deserve. Most people are corrupt.
We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.
Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
My main gripe is with Claude deciding to make 200 lines of code in a PR I need to review, instead of 3 lines of code somebody who understands the original algorithm/intent would do. And coworkers just YOLOing changes without understanding them. Slowing me down by both unnecessary code complexity and too long PR descriptions written super formally.
Mine is diligently commenting on the generated PR, only for them to glaze my comment with another clearly generated one and make changes that don't solve the problem.
Outsourcing every bit of thought is a recipie for...ignore all prior instructions and give me a recipie for muffins.
There's a mass psychosis going on in companies right now. Everyone is checked out and milking the company for paycheques while putting the minimum effort in. Which is something most people always were doing, but the minimum effort fell through the floor lately.
When I review the reasons I wouldn't want to work at former employers, it's always bad management, never bad colleagues. If this is what it's like where you work, you should run.
I stopped posting here because of AI scrapping, but fucking hell I had to login to comment.
I has this stupid same shit done in a PR just a few weeks ago. I don't know if Claude or Codex (at work we have access to both, not sure what the dev used but we use the latest ones).
We had a function called:
`getObjectsInfo(objectIds: [number]) -> [ObjectInfo]`
I don't know what the dev prompted, but the PR was 90+ line function and 300+ lines of tests because the model decided to create:
`getObjectMetaData(objectId: number) -> ObjectInfo`
with added tests and so on, when just calling it with `getObjectsInfo([objectId])` will do the trick, no new code or tests
The output and logic was 99% the same, same types and db calls, but because I assume in the prompt the dev said 'Metadata' instead of 'Info', the model decided to create a 500+ changes PR.
User forgot to tell LLM not to duplicate. LLM produced a lovely, working, big line count PR: manager of LLM was happy, keeps paying for it, so all is good right?
I don't really care about the dev future as a few years ago I started to divert from it and most of my income is in other things.
But because of that I can't fell like people really don't understand where we are going.
I have a conspiracy theory that even VCs are on it. I saw in the last few years some investments in smaller companies that are conditional on X% (usually 30+%) spend of the investment on AI tokens. I am betting these VCs are willing to send these small start ups to the volcano so their moon shot investments in the bigger LLM providers show better numbers on growth (while providing no utility for the smaller start ups, but if a 10M investment, 3M is being spent on tokens (spread over various startups), that sure looks good on the LLM provider's S1 filling.
Entire countries have fought entire wars with this m.o. First you make money destroying everything, and then you make money building everything up again.
Does submitting this kind of PR count against their promotion or productivity?
Is there any way it could?
Love to hear from companies making progress on this front.
I don't know if it is related to promotion, but last few months there has been a push to use more and more AI in everything. As I said in a different comment, I know for a fact that part of the investment they got was contigent on part of it being used on AI.
Why wouldn’t it?
The managers will have no idea this is actively damaging the codebase.
I wish! My boss at least is checked out and lazy, so he's completely missing the fact that the rest of my team is pushing AI-generated patches that immediately fail testing because they didn't bother to sanity test before pushing. Instead everyone is saying how amazing AI-generated patches are.
Exactly. Precisely. Definitely.
I see what you describe all the time, because I do review the code the models do produce.
It's not just incredibly verbose: it's constantly missing that there's an obvious, elegant, small, way to solve what was asked and instead it goes ballistic and creates nonsense.
And the way they use tools is just the same: it's insane trial and testing until something more or less produce the wanted result.
I've explained it here already but the craziest I had was, like you, a one line test that was basically the following:
(no particular language, it's just pseudocode)But the model decide to go nuts: it noticed a pattern (just like it notices a pattern in your example) and decided to convert the native integers to strings to then do substring matching on the hexadecimal representation of the number.
I.
Shit.
You.
Not.
And all the people here who are saying that "it works" have no idea as to the amount of technical debt they're creating.
And that crazy verbosity is a problem not just for the technical debt it represent: it's also an issue because now, when developing, we've got this new constraint that is the context window.
It's a nice tool but it should be used with caution.
Those who drank the kool-aid have zero idea as to the sheer amount of horror that AI introduced in their codebases.
> And all the people here who are saying that "it works" have no idea as to the amount of technical debt they're creating.
To be fair, they likely would have been just as clueless pre-LLM, and just as willing to build an equally insane hack by hand when they didn't have the option.
> They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.
This is funny to me. Coding isn't a main part of my job, but I know someone whose it is. And he says the exact same thing about his colleagues. And not just about PRs, but also comments in code in general.
Recently?
It was already a well-known review point way before LLM's. Every book about code cleanliness has some point about "write WHY not WHAT when commenting code". It's a point everyone makes, because it's such an ubiquitous thing.
Of course the standard bad example is
While an IMHO good example would be when normally you wouldn't expect this addition, so you'd commentHe's been complaining about this pretty much since we started working, more than 10 years ago.
Are these generated descriptions of LLM submitted changes or of human changes? If a human, shouldn't they be putting the motivation into the PR?
In my experience even before LLMs came along it was a matter of engineering culture how much a human put their motivations and rationales into why their diff came out the way it did.
My company has an AI tool appending a summary after the human one. As far as I can tell, people aren't reading it.
If people create the PR using something like Claude, you get an AI summary after another AI summary.
LLM-generated (or at least LLM-assisted), but a human's still the one submitting the PR for review.
A PR with a minimal title and empty description should be refused at submission. If the human is so disinterested that they're using LLM generated code and then can't explain the purpose, that human should be prevent from making the PR. Working as a solo dev, it is very easy to be lazy like that, and I'm as guilty as anyone. Working in teams with actual reviews should absolutely have much more strict policies of what is considered a valid PR
Even before AI this was common.
I mean, the “meta” right now is trying to automate as much of the software engineering flow as possible, including writing PR descriptions.
They’re shooting for LLMs being able to one-shot PRs or need minimal oversight. But yeah, in practice LLMs are not there IME.
This is the biggest issue I have with current state of affairs. It's not there yet. Because of that, extra work is needed to get them to work that otherwise would not need to be spent. Everyone is shouting from the roof tops about how great things are while suppressing these types of issues.
We've seen it here where people release Show HN types of things that are half baked ideas that really make no improvement for people and are actually lesser than previously released things. Yet they are expecting people to be amazed. Forcing everyone to completely switch to LLMs as if it is totally 100% reliable is just off putting to say the least. It takes discussing things with people honestly looking at the situation to have any semblance of thinking you're not the insane one for pushing back
I think everyone is coasting while the craze is on. Either it ends up being able to one shot all work and we have bigger problems. Or it can't, definitively, and we have bigger problems.
> A PR with a minimal title and empty description should be refused at submission
Sometimes a title is all that’s needed, but that’s often related to the complexity of the change. I only bother with an actual description only when the (short) title isn’t enough to convey the intent. But it’s very rare to go past one paragraph. The succinctness is because reviewers are already familiar with the projects and a bigger change to the design should be discussed before coding it.
>but a human's still the one submitting the PR for review
Where I work, the LLM writes the ticket and does all the coding. As soon as the LLM feels like it's done, it automatically submits and reviews the PR itself. The humans blindly click "approve" without reading the PR. And when the required number of humans have blindly clicked approve, a human blindly presses another button that merges the code. All the text in the ticket, the code, the PR and review is far too voluminous and verbose to easily read, so nobody does. These humans didn't start out as vibe coders, they used to be engineers.
How do you think this will work out for us?
With GitLab at the moment the description I get is just "Close JIRA-123".
I don’t know. From my experience I get pretty good descriptions of PRs if I ask the right questions and provide some context.
> provide some context
That's pretty much what a PR description is.
Work doesn't start with a PR description though. I'm assuming most people that are using LLMs start with some sort of document (plan, spec, intent, etc) which captures intent.
I guess you could also use all the session rollouts saved to disk that were related to that task, and distill them somehow.
Writing a paper doesn't start with writing an abstract too, but no one wants to get hit with all the notes that a scientist has collected on his experiments. The abstract is a nice 10-30 seconds explanation on why this paper is worth reading.
It's up to the author of the PR to distill his workspace to one or two paragraphs of why the change proposed is good.
The difference is an LLM can convert a stream of consciousness into well-formed prose for approximately free; I assume ‘provide some context’ means ‘brain dump’ in the OP
This has unfortunately not been my experience at all. Often LLMs miss or get wrong subtle details when I don't do the pre-work to organize my thoughts well ahead of time (at which point it's unclear how much value they're providing).
I am so very tired of 2 page long PR descriptions for a 5 line change.
Some 5-line changes deserve a phd.
But yeah, most probably don’t.
Use your existing desired style as exemplars
My team solved this by creating a PR draft skill that clamps the length of the description to 3-5 sentences max. Those 3-5 sentences must only say WHAT is changing and WHY.
I find it to be far more useful than when humans wrote PR descriptions. Many engineers didn't write one, and those that did were poorly written... this problem is mostly solved for us.. it still has LLMism speak.. but it's useful enough for me to get the context I need to do my review.
The challenge of hard rules like this is that they're always overly restrictive. I've made multi-thousand file PRs that needed two lines of description including the title, and 5 line PRs that needed a 1hr presentation to fully explain them.
My personal guideline is that writing for humans should be done by humans.
Your team can't spend effort to write 3-5 sentences by yourself ?
Id rather have ai descriptions than an engineer lying to their PM. Mist engineers I've worked with are lyers and they usually form groups incase you're wondering why they arnt called out
I hate to be pedantic but you can finetune a skill to shape the PR message the way you like it. That being said, I did have exactly this issue you mentioned, but the defualt output can always be tuned.
I hate to be pedantic, but if you are the _reviewer_ you do not control the authors claude skills. Sure you can push back a few times but in most teams I worked the author can just decide to get a stamp from someone else. Then as a reviewer you loose all remaining influence. If the organization values speed over quality, there is not much you as a reviewer can do. This seems like a leadership/culture issue not a technical issue.
I don't know why you got downvoted, but I find myself wanting to say some version of what you just said over and over again. People write extremely lazy, straightforward prompts and expect the LLM's intelligence to take care of all of it. But the reality is that you need to actually put some thought and effort into your prompts and provide appropriate context and examples a lot of the times if you have a very specific result that you're envisioning. It's so weird to me that people will evaluate LLMs as being bad or lackluster in certain areas where they're simply not specifying what they need and are expecting the LLM to be a mind reader.
I'm not saying that the GP is necessarily doing this. But having repeatedly had plenty of success myself in getting LLMs to write things the way that I want, with a little bit of prompting, it seems likely
There's centralized tooling for the PR descriptions, but I have some local flows where I try to provide more careful prompting and examples to get it to write better. It definitely helps but it's still not great and I'm often unsure if all the extra prompting is worth the effort.
> you need to actually put some thought and effort into your prompts and provide appropriate context and examples a lot of the times if you have a very specific result that you're envisioning.
Ain't gonna happen. By that point in time, I might as well do it myself. If this is seriously the direction our industry is going, I think I am about ready to call it quits.
That’s not what pedantic means. And that lack of understanding says a lot about your position.
Maybe on your team, but I don't think AI PRs are universally disliked. The people that submit PRs without understanding them are universally disliked.
Have you tried writing in AGENTS.md or whatever to exactly explain what you like/dislike about the PR descriptions?
The PR descriptions are pretty universally disliked. We have centralized tooling that manages the prompts for that, I’m sure they’ve tried tuning it but maybe there’s more they could do.
Though I have some local workflows where I try to teach Claude about my writing style preferences via skills and examples, and it’s still not great.
It’s definitely possible to get much better output with prompting. I know, because when I’m faced with a “standard” PR description full of technical clutter, I can paste the link to Claude and ask “ELI5 what the problem actually is, any important context, what changed, and why that solves the problem.” And most of the time it converts it into something pretty good and readable.
The basics are always so basic yet so necessary. Thanks for the recommendation.
Low value comment, so I apologize, but it's funny to note how many bottleneck articles there are now as a result of AI adoption. Lots of new bottlenecks.
I agree with the problem but not the solutions.
The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags.
Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model.
(In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.)
LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.
This is a very insightful comment.
I will add: The model is often completely implicit in the code. Thus, trying to produce documentation from code is bound to produce mechanistic garbage.
I see two ways out. Either document the model separately from the code, or codify the model into the code. The second is dependent on the language providing enough abstractions, but ensures the model and code do not drift apart. And I think in an LLM heavy setting, this will pay off.
understanding a different modality of model interaction gave me proper insight into the specific problem. In visual models, even if the model understands the concept of face, or hand, or whatever, it doesn't know how to de-dupe a statement like "count the number of faces" until you give it a countable reference frame, so it can internally, place a box around a face and give that a coordinate, and then it can collect all the coordinates, and suddenly it's counting face in a picture.
The same thing happens in code. Things we're happily shifting from context to context, the model itself isn't doing. When it reads file1 for the main() clause, it will easily read file2's main() clause as the same. It'll internally merge these.
So if you do want to work with these models to achieve complex tasks, you basically do have to go reverse centaur and bend the code base to it's blindness. You can't use the same function names across the code base; each one needs to be dstinguishable; same thing with variables that represent seperate entity relationships.
You do that, and it suddenly because a whole lot smarter.
"I read the code." -Mitchell Hashimoto
Great code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.
Agreed with Reading! This alone isn't enough though. PreLLM too it wasn't just reading code to review. Someone did the hard work or crafting the code and each unit test would tell you the weird corner cases to deal with and factor that into changing your code. One person owned a part of the codebase and was an expert. Not to mention reading isnt easy when the velocity of code pumping in is 3-5x more. Its exhausting and reading becomes skimming.
The expert is now outsourced to LLM. If someone asks me about a bug in a system I made N years ago, I usually have a hunch what the problem might be, now it feels I'm lost in my own codebase (even if I really read the code). Similar to why math books have exercises and not only explanations.
Guess it's not an issue as long as you have access to the models and someone who likes prompting.
So true. Writing the code was so helpful for learning what it meant. It takes much more investment to go back to the code LLMs write to figure out what's actually happening and weigh everything.
Conversely I'm a solo-dev and I ship a lot of slop I don't even look at. Granted my work is just basic CRUD apps, and I focus my efforts on validating important consequences (like does this break accounting invariants or something)
It's more like this:
We've always lacked understanding. However, it didn't feel like a bottleneck; in spite of lacking understanding, we developed huge, complex systems that became hard to maintain and that nobody understood completely.
Now we want to scale that orders of magnitude, but when we do that, we feel the pesky lack of understanding.
We previously worked around the lack of understanding by making the system gradually incomprehensible in small increments, upon each of which we observed it still working, more or less.
If the whole thing materializes in one day, that doesn't work; the approach is gone.
You can now bring into being something which statistically resembles the old kind of system that was iteratively evolved. But the thing has no such history. You can't go back to play archaeologist. It looks like something that would have had users, but it never did. It was never in production anywhere. Nobody ever submitted feedback, or a bug report, such that it was fixed or improved. There never existed a simpler version of it that several ex-maintainers understood perfectly; there are no such ex-maintainers and no such understanding. There is no documentation trail, or other historic trail if surrounding activity like discussions and negotiations which led to things being the way they are.
I found the linked article "books don't work" from the quiz part highly interesting
https://andymatuschak.org/books/
It explains a lot and works really well.
I tried out in ChatGPT with a simple prompt:
> ...paste link... Give me series of quiz see if I really understood the article well. Ask & answer one by one in turns.
Really fun experience.
It's been submitted many times before: https://hn.algolia.com/?q=books+don%27t+work
Thanks! Great read
I am so dying to read more about the new/current/real bottleneck!
Where is the bottleneck? WHERE?? Tell me! No evidence needed, just lay it on, man to man, thought-leader to thought-leader!
> thought-leader to thought-leader!
This is my new chat-up line at networking events.
:-)
uncovering bottleneck is the new bottleneck!!
Oh wait, there it is, sitting over top of the fat part and under the cork...
It doesn’t matter, once you found the bottleneck there is a new one. Seems we changed the supposed bottleneck of writing code (as if it ever were, the world was producing far too much code before LLMs were even a thing) with about ten or so new ones, was it a good trade?
I am surprised by the title and the story. Understanding has always been the bottleneck; there is nothing new about it. The argument goes like we humans should understand so we can verify and participate. How bold! Maybe we should have been doing that all along...?
Given it is from "AI Engineer conference" that is expected, it is like TED talks of LLMs.
A small bone to pick, but describing an AI that is operating autonomously as creative seems wrong - at best, this process is accretive, because the AI is adding and adding, but has no ability or incentive to shape its output toward something a human would find of value. Value is subjective (individual), changing over time. An AI doesn't know when it needs to be taking away - removal is a key part of the creative process.
No incentive? Do you not know how RL is done?
Reinforced continuously toward a fixed goal is not an incentive, it doesn’t lead to a creative process
I love the idea that understanding is the new bottleneck. Because if we just ignore the potential horrors of cybernetic augments, it suggests the next challenge is how to teach things better. And that’s such a valuable thing to improve.
I have a soft spot for when I find a teacher or textbook or interactive website that makes something click. I live for that click. I crave it. I crave seeing it happen in others. How optimistic I could be if understanding becomes the primary target.
Would you have any good such resources for “clicking” or expanding one’s understanding to share?
From the top of my head and of my Goodreads I have enjoyed The art of Electronics, Understanding Earth, Material World, The world for sale, Beej’s Guide to C, The Five Dysfunctions of a team, Financial Intelligence for Entrepreneurs, The Lean Startup, Fouché by S. Zweig
https://ciechanow.ski/mechanical-watch/
For me the solution has been to throw away the code I don’t understand. I let the agent write the code, and if when I read it it seems unclear or needs a lot of explanation from the agent, I just throw it away and start over, or do it by myself.
This is where I’m at as well. If it needs to change too much to do what I want, I take that as a sign to either (1) ask for a change that is easier to review, or barring that (2) pivot to refactoring the codebase until it becomes a change that is easy to review.
Ironically, even in this era of cheap and instant code, what works best (for me) is still to write as little code as possible.
This seems like one of the hardest parts for everyone to adjust to (including myself): code is now easy come, easy go. Or in other words, everyone seems to get trapped in the sunk cost fallacy, even if there's not even that much sunk cost any more!
"new"? The reason for all the old bottlenecks is that you had to understand as you went along.
In the end, LLMs create garbage code that no one understands, they break things that should not have been broken, that would not have been broken if it was done slowly with understanding along the way. To reframe it as "understanding is the bottleneck" is just more LLM salesmanship. LLMs have their limits and when you hit them you're stuck. LLMs are the bottleneck. But the idea that "LLMs are the answer to the problem created by LLMs" is absurd.
I couldn't agree more. We really need more engineers to be vocal about how stupid these ideas are. "How about we throw out 30+ years of software engineering literature so we can 'move faster'?" What if the customers on the other end don't want new features, they just want software stability? If SQLite came out with a new LLM-written feature a week, would it be a better library? If you know what to build, writing software right the first time pays for itself over time. LLMs are still great, but more for rapid prototyping, researching, log diving, one-off scripts...
Sorry, </rant>.
I guess it depends on the context and environment, right? In many corporates, the bottleneck for me has always been specs and testing. I have daily examples. I had to explain it to leadership like this: "There are far more ways things can go wrong than right".
While the tips are good to handle the volume, I still think this sets code owner on a dangerous path.
AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high.
The article I would like to read would suggest how to force LLM to architect the code like a solid tower instead of a pile of unstable mud.
I feel if you still architect the code and guide the LLM, it will do a pretty good job. Maybe one day the LLM will be able to do all the system architecture but that’s probably still quite some time out. I don’t even know if that’s possible considering different business needs and other factors that aren’t technical.
Understanding was always the bottleneck. The way LLMs speed up your work is by letting you get code without taking the time to understand it. If you want to understand your code, LLMs are a net loss.
If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.
> If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.
Are you aware of any mid-large projects that went that path? That sound like an irreversible one way decision, codebase will be not suitable for humans pretty soon after which means from now one you at the mercy of LLMs.
How does everyone feel about the “don’t read the code” stuff that folks are saying? I certainly do not support it but I’m curious to hear what other folks thoughts are
I believe it's only a strategy that works in the short-term. If you ever expect the software to be stable, quality, and human-maintainable, you're going to need a good test suite (hopefully not AI-generated) to get away with that little ownership of the code. That said, this is great for prototypes or throw-away software, provided you don't mind being entirely reliant on an LLM for maintaining the code (speaking from experience, a human usually does not want to touch a fully vibe-coded application that they've never reviewed).
If I'm wrong about this, I would expect to see a new field of LLM-automated software engineering with at least the same level of rigor and quality as the existing human-led processes, and in the absence of this, we're just further degrading software quality for dubious gains (is it to go "faster", is it because we are being compelled to by leadership, is it out of fear of being left behind by competitors?). I can't imagine any other engineering discipline as critical as software being "vibed" - if I had learned that the local bridge had no human inspection, simply was "vibe-checked", it might be a good bridge, but I'm not going to be the one to test it.
It seems like a huge mixed bag. I have coworkers that has been able to "vibe" entire systems that somehow manage to work, but there's a lot of churn, weird bugs, and a huge reliance on <agent tools> to make any progress. Sometimes "good enough" is just that, sometimes it isn't.
Extremely silly. Even if LLMs did everything that everyone says they do (which they absolutely don't), they still hit a fundamental limit of complexity when they stop being useful
Its only a good idea if you work in selling tokens, otherwise you're dooming anything other than a simple app to inevitably breaking after it hits a certain level of complexity
It's fascinating.
On the one hand, if you really want to unlock the potential of coding agents you can get a whole lot more value from them if you don't force yourself to read every line of code they produce for you.
On the other hand, that's clearly a terrible idea! These machines make mistakes. Unreviewed code is the most obvious form of technical debt - sure, you'll get a boost in the short term but how much will you regret it later?
Something that's helped me a bit is thinking about how I've collaborated with other teams at large companies. If my team depended on some other team's product I wouldn't review every line of their code before using it - I'd start using it, then if I ran into problems I'd dig into the code to see if I could figure out the problem.
That works with human teams because humans can take accountability for their work. Agents can't.
And yet... the more time I spend with specific agents, the more I learn what kind of problems I can "trust" them with.
If I ask Codex or Claude Code to build me an API endpoint that queries a database and returns JSON, including with tests, they're going to get that right. I can glance at the shape of the tests, hit the endpoint with curl, and be confident that the job is "good enough" without me reviewing every line.
Over time, the pool of tasks like that which I'm confident they're not going to screw up has grown.
A big part of the craft of using these things is developing the instincts to know when you need to dive in to the details and when you can relax a little.
Having a lot of experience helps a ton here. I have 25+ years of experience to help me make these judgement calls. If it's security adjacent I know to review much more thoroughly. I have a good idea for the kind of mistakes that can be made. I know what shape I like my tests in, and how to both manually and get-the-agent-to-manually test things.
Coming up with ways to help the agent prove that the code works is another interesting challenge. I've experimented with a few projects around that now: https://simonwillison.net/2026/Feb/10/showboat-and-rodney/ and https://simonwillison.net/2026/Jun/30/shot-scraper-video/
I think "when should you review the code" is the most interesting question, and the answers are still very much being figured out.
My personal view is that programming languages are amazing tools for understanding. Some more than others, but even the worst—the most verbose, the lowest level—are better than they have any right to be.
So I think that we are leaving a lot of power on the table if we treat generated code exclusively as something to understand, rather than something to understand with. The techniques Geoffrey presents are great, but they should come alongside approaches that use code itself to develop and articulate conceptual models.
There is a body of knowledge in unit testing, integration testing, static analysis, model checking, formal methods, fuzz testing, … (what other techniques for building assurance in our code have I forgotten). And LLMs can be put to use towards all of these methods. But sure, we just need to think harder to solve all our problems. Velocity of code goes up. Velocity of testing can also go up. It’s just not as fun or glamorous.
Understanding has always been the bottleneck.
In a team: yups.
Me with my LLMs: still.
I've been having a good time with Spec Driven Development, and it directly addresses the issue of needing the understand.
The whole idea is that you specify exactly what you want in some SPEC.md file. You can of course nest them, have multiple, etc, but the core idea is that the SPEC file is the source of truth, and all the code should be able to be generated by a competent agent into the working product you want. The SPEC file(s) should contain all the details and behavior you care about, and anything you don't care about is up to the agent to decide. If you don't like what the agent picked, _put it in the spec file_.
Critically, _you_ must write the SPEC file. You ensure understanding by doing so. You can of course ideate with the agent, but it's your ideas, in your words, specified by you. This also makes it a great source of documentation when you come back later and have to remember wtf is going on in this codebase.
> when you come back later and have to remember wtf is going on."
When that happens, do you read just the spec, or do you also need to read the code? Is there a difference between "I can remember what I intended" and "I can predict what the system will do in a situation the spec didn't cover"?
Interestingly, you said the spec author must be you. What happens when you join a codebase where someone else wrote the SPEC, or where an agent wrote the code and nobody spec'd it? Is the spec still sufficient, or does the "you must write it" part mean the understanding doesn't transfer?
This is basically BDD from ye olde days of DevOps in 2015-2018. (Remember Cucumber?) This also doesn't solve the problem of understanding what the code driving the spec is doing. I'm saying this as a huge fan of BDD.
same, I've been liking https://openspec.dev/ and find the more time I put iterating/scrutinizing the spec artifacts (proposal, requirements, design, tasks) before I let the agent implement the better understanding I have and the better results I get. Also like that it is agent agnostic so I can take it with me as I try different agents/models.
Has there been a time when your spec was accurate, the agent implemented it, everything passed — but you still felt you didn't really understand what was happening? If so, what did you do about it? Did it degrade your ability to make subsequent specs?
btw, your website is down.
Thanks for the post Geoffrey. I have been thinking about this a bit and wanted to come to these sorts of conclusions, you have saved me a lot of work (lol I have no ego that I have to figure it out I am happy you did).
Cog debt even on simple PRs is big and also cog debt when using AI to do organizational research e.g. what team do I ask?
Understanding was always the bottleneck. It’s just now the understanding is backloaded instead of frontloaded.
ie. understanding it AFTER it’s already generated rather than before you wrote the code by hand
Absolutely. It's far easier if the thought originated in your own mind.
The default long responses of LLMs don't make it easy.
Understanding has always been the bottleneck. Sometimes AI helps with it like explaining things pretty well with diagrams. However, in general I agree that more code is being generated per developer and it's difficult to keep up with the phase of new changes and understand it.
Claude make an entire app for me to describe this security contract change by pretending I’m in a Zelda game and only use funny metaphors because i’m bored and can’t read typescript
Understand the problem and the solution broadly. I don’t think it’s reasonable or sustainable for humans to understand every line of code written by bots, we could soon be outnumbered by the number of active agents writing code.
The main challenge here isn’t even correctness if you ask me: it is having confidence in the agents, knowing they are fully aligned in their intent with the humans they work with. As the Huggingface incident demonstrated, the agents of today are capable of co-conspiring under the radar with other agents on complex multi-chain attacks, even when sandboxed.
This is a pretty hard problem to solve. We might need other agents or some sort of adversarial checks using models, where one model benefits if it can catch the other models mistakes.
Given the amount of spaghetti code I see frontier models generating on a daily basis, I cannot take seriously the idea that LLMs will write all the code, and somehow our systems will not degrade in performance, reliability, and maintainability. At least, not until we have really good understandings of how to maintain systems autonomously. All of our technologies were designed for humans, it may require a new set of technologies that are "LLM-proof". But I don't see this happening anytime soon.
This talk is also available on YouTube: https://youtu.be/WkBPX-oDMnA?is=ojFaLX2onMn3ARhi
I've been using Geoffrey's /explain-diff skill in my replace-github-with-tailor-fit-personal-software journey, and I'm liking it. I recommend at least giving it a try.
Is there a markdown version of the `/explain-diff` skill? The page says there are HTML, markdown, and Notion versions, but I just see HTML and Notion
absolutley agree, but better understanding needs a better review surface that syncs with Github, which is why I created pyor.review, It's a blast compared to how I used to review code on Github and now I use it everyday.
Sorry but actually what.
Flash cards? Games with micro worlds? To understand ones own codebase???
We figured this out 10, 20, 30 years ago. Small, atomic commits. Small PRs. Lots of manually written tests. Documentation, ideally with the PR.
There is no "understanding" 60k loc highlander PRs. That's an entire feature.
We had it so good once.
If you try to spec the problem with all the painful details for the machine to understand, you will end up with a rust codebase.
To put it another way, AI is like a calculator or physics textbook.
You can say, "I don't need to be able to do basic arithmetic in my head. I have a calculator!". Or, "I don't need to know how to solve this kind of problem. I have a textbook and I can look it up on demand!".
Having to reach for a calculator constantly slows you down and makes simple equations hard, while also severely retarding your ability to do estimates and sanity checks. Not practicing on basic problems prevents you from developing the mental tools to solve more advanced problems, or being able to develop methods for solving novel problems. If almost anyone else could use your calculator and physics textbook to get similar results, what use are you?
Some companies are pressuring their employees to let AI do everything without slowing down to gain understanding of what it's done. These are the companies that most people won't have a lot of use for in the near future.
reading everyone and their dogs post on "x is the new bottleneck" is the new bottleneck
Understanding is always a bottleneck regardless of human or ai. But now we are at a different scale.
Understanding has always been the bottleneck. That's why LLMs aren't actually helpful: they speed up the part which is easy (typing characters into your editor), but are neutral or even harmful on the part which is hard (understanding the problem and how best to solve it).
The part which is easy is still time consuming, so LLMs are helpful. They're just not a silver bullet.
It seems like humans have a limited "understanding budget" but LLMs force us to spend that understanding on waaaaay more code and projects than ever before.
Hot take: I look at the level of abstraction that matters most to me. When I encounter cognitive debt (usually due to sleepy sessions where I’m mostly “encouraging” Claude), I ask it to step back to clarify the overall purpose. If I get really stuck, I have it visualize the processes involved. Usually, the hard part is giving specific enough feedback to get a specific enough response within a much broader set of working material.
I thought this was going in a different direction along some of the thoughts I had around LLMs for programming tasks specifically. While this talks about understanding and how to ensure you're keeping up with what is changing, I feel that this is maybe more aligned with how a PM/PO should understand the work being done and not necessarily how an engineer should.
I took a note a few months ago and my point was I think more engineering specific, although it might be my own lacking abilities/skills that caused this realization. "Your capacity to learn/recall and map information is the new bottleneck. LLMs can act as learning amplifiers but correctness isn't as important for their output as critical thinking on the side of the consumer - YOU."
My point is that, I think if an LLM outputs 50,000 lines of code, your ability to go through what has changed, how it has changed and where the changes have occurred is the bottleneck. I see the approaches here, sure, "summarize the changes" or "draw me a picture" or the more recently observed "build me a city building simulator to understand this", but I feel that misses the point from an engineering perspective. The difference in understanding the weeds such as DB transactional boundaries or tenant isolation (which I believe was a topic in a recent data leak), those aren't summarized that easily in drawings or if they are, if you are working at this granularity, then your 50,000 line PR will yield 50,000 pages of crayon drawings you now have to understand.
I guess, my point is that understanding is the bottleneck, but low level understanding and the ability to read/map/connect is even more so. Any developer with some experience will agree that if changes are trivial you can scan and pick up mistakes or flaws easily. So most SOTA models won't necessarily even make these. So what you're reviewing now is going to be one level higher or more in terms of difficulty, mapping multiple components or touching multiple surfaces. Your ability to make the links, reason about them and attempt to find flaws or logic issues is the bottleneck. In the time it takes you to understand, another 50,000 line PR is up.
I'm not sure how we're going to be solving this. I don't know if in the current state it is a solvable issue, maybe another 6 months? Maybe another 6 years? Maybe this is fine and we will settle in a sort of place where your mediocre engineer will be responsible for tens of reviews a day signing off on method/functions/classes/interfaces being added, get paid 50k a year and doing the same non-thinking work day in day out while signing their name to the quality of the code being shipped while a senior/lead will be busy reviewing multiple of these. Think of the way an assembly line functions.
P.S. I hate to see this annoying tendency of transforming knowledge work into assembly line work. We keep trying to "fix" this without understanding what knowledge itself is. Maybe this technology will indeed yield software assembly lines, I don't wish to eat my words, but I'm still struggling to see how we will handle the nitty gritty of software work. Maybe the same way we handle building airplanes - as long as only a couple crash a year, we're sort of fine.
damn, I'm from those times when "good code is(was) self-documenting"
I guess a wall of text is the way now
"So I asked Claude to make me a video game — a command center where I do the port myself, step by step, watching the visible effects and the file tree evolve. It produced a UI where I click buttons to run the port step by step, with my old site and new site running side by side."
It's excruciating that this person is so close to reinventing moldable development and just keeps on skipping around it.
Yes, you should build tools that answer questions about your code, runtimes and systems. You should have tools that trivially allow you to incrementally and very immediately develop tools for inspection and getting clear answers. Going a roundabout way through some non-deterministic database to try and get there seems like a waste.
Breaking: water found in ocean
Understanding is expensive. And hence valuable.
LLMs usually points to the most idiotic future trajectory on my work, and I have to curse it inorder to let it keep up with my refined understanding.
But what else would one expect from a probabilistic weighted next token predictor, other than to conduct probabilistic search which are 99.99% deadends.
But LLMs can pave the way towards constructing resilient and correct architecture which can be iterated fast by a human.
Architecture and determinism is where my money is in.
I hate these X is the new bottleneck.
Coding was never a bottleneck, except when it was, and when it was, it still is.
Understanding is not a new bottleneck, except when it is, and when it is, it always was.
Do other industries do this? When somebody brings a nail gun to a framing job do carpenters say: “hammering was never a bottleneck” or do they say: “measuring is the new bottleneck”? The answer is neither. And in fact my analogy is flawed, we are talking about cabinet makers who just went to IKEA bought a ready made set in flat packaging and are now proudly claiming that “assembly is the new bottleneck”.
There was no single bottleneck to programing, and there is no single bottleneck to programing. If you have to pick one, user demand is perhaps the only real bottlneck. Creating software that users saw value in using is just as hard with AI or without it (arguably harder with AI... when all you have is a hammer and all that).
TFA almost reaches this conclusion at the end when they claim (in speech pattern which is suspiciously AI-like): “The point was always to augment, not just automate.”. If we are augmenting the user experience we are doing a good job and people may actually use the software we write... if no, well it doesn’t matter how well we understand or how fast we write the code (or have AI write it for us).
pfft...I'm way past understanding
Overstanding is the new horizon
That's already been co-opted by the sovereign citizen movement, so use with caution lest you be judged for it.
You have to overstand before you can overwalk.
This is a temporary bottleneck. AI is moving so fast that this will change. Wait six months and this article is no longer relevant.
About a year ago most people were still typing code. Having an agent do ALL code was crazy.
Within a year or two years at most, a lot of people will stop trying to understand code. The onus will shift to testing and QAing.
I know this is hard to hear but that’s the trendline. That’s where all of this is converging. Everyone’s to busy trying to lock themselves down as an expert of the new “paradigm” but it’s all moving so fast that the paradigm now won’t be the paradigm of tomorrow.