I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt.
I was fully expecting that writing the code will pose no problem for the AI. But i was curious if the AI will realise that my friend is a novice and needs extra help with things like: copy pasting the code into a text file and saving it with an html extension, helping her host the file online so she can share it with others, buying a domain for it, etc. I assumed they will get there eventually, but i also assumed that it will take a lot of stumbling around and misunderstandings.
But i was completely wrong. They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. They were just going around in circles where the AI was brainstorming with her about possible features and getting thints more and more complicated. We terminated the experiment after one and a half hours and many many messages exchanged between her and the LLM.
Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode. In other words in that case the LLM would have rewarded even just a little bit of expertise, but without it there was a confusion about goals between the human and the machine.
I ran a similar test and got completely different results. My girlfriend (hair stylist/artist) with zero coding background mentioned a Telegram bot idea. I asked "Why not build it yourself?"
I gave her a Windows laptop, but she said she wanted what I have instead. So I handed her a USB stick and told her she was on her own now.
Fast forward: she now runs Arch Linux with Hyprland (I use Xorg/i3 though), fully riced with cats. I only interfered on partitioning to preserve my data. She even installed Steam and got Portal running (that was her "watch this" flex after I told her not to even try). She pulled all of this off using a free-tier Gemini chat.
For the bot, a friend gave her a Kimi 2.7 key. She set up their harness and built a working bot in a matter of days. She even got a free Oracle VPS for deployment, though I stopped her there to check security first (still haven't had time, unfortunately).
She uses that laptop daily now and says she enjoys it over Windows by a mile.
There are big differences in what people call "tinkerers". Some are INTERESTED in building things, give it a little shot and find it to be daunting, and push it no further.
True tinkerers have no problem with this, because they enjoy learning how things work. Installing Linux, Steam, Portal are all relatively straightforward tasks for someone who uses computers on the regular - but to some people this is just something they've never done, are scared to do, or just don't want to learn. (Which is fine, but they'll never pick these agents up and run free.)
Barrier to entry used to be blog posts, documentation, watching poor quality Youtube videos of a thing that SEEMS related to what you're trying to do. Now we're getting that spoon fed to our particular case, so the friction is essentially just "follow the AI directives". (However, the depth of understanding probably struggles.)
You can get pretty dang far asking the AI to explain things and show you where it got that info, especially with IT/DEV stuff. Thing that has been very useful to me is specifically asking AI for that vocabulary. AI is suprisingly useful for giving a vague description of something you want and asking for possible words that match. But you have to KNOW to try that. A tinkerer might have started that convo with the chatbot by asking "What are web pages built from" "Can we use those building blocks to make our own simple app" and gotten there from a place of low expertise in web development but juicing out the knowlege from a place of low expertise is itself a skill.
I know you are not arguing from bad faith, but you are making an assumption along cultural lines, which is a mistake of missing the forest through the trees.
The vast majority of people (blanket statement, I know..) do not come from a culture where embracing curiosity, asking questions, or trying to break things down is the norm. Developers, tinkerers, etc., sure... you can reasonably make that assumption. But not everyone. A cultural practice of critical thinking and problem solving is HOW you KNOW to ask WHAT questions need to be answered FIRST, in order to solve a problem or progress toward a solution (if more information is needed).
You might even make the argument that everyone should have these skills, and I would agree with you. But the missing link here is a culture or cultural practice that provides those things (the WHY), and an AI/LLM will not provide those things in absentia, without "prompting", or build up that infrastructure in meatspace for a given set of users. Ignore this at your own risk.
for a lot of lower barrier to entry tinkering, for years i felt it was mostly just google a thing and follow directions. Feel like the only thing that really changed is the initial query can be a mess of kinda nonsense and still give you those nice directions, and you can instantly get clarity on a single direction if you get stuck.
So the barrier is still kinda there to just grab the instructions and follow them, its just incredibly easier to follow them.
To rice something must mean to customize and decorate. I'm guessing that it references the "rice rocket" term, originally a derogatory ethic slur. That term likely originated in the USA in the 70's or 80's, at first referring to Japanese motorcycles (which were fast, rightfully deserving the "rocket" designation: classic Japanese racing bikes are fully-fledged crotch rockets).
Sometime in the late 1990s, a Rice Rockets website appeared, making fun of people decorating their under-powered import econobox cars (not rockets in any sense) to look like racing cars with features like rear wings (on a front-wheel drive, lol), exhaust modifications ("fart cannons"), stripes, stickers, rims, steering wheels, etc.
If the cats decorating the desktop are Hello Kitty, she is really ricing it.
Desktop Linux has a deep and storied culture of real empowerment through learning and teaching. Even if you never sign up for a forum or hop into an IRC channel, that culture pervades a host of informal docs like the Arch wiki, and seems likely to get picked up by LLMs during training.
Maybe web development, despite plentiful tutorials, just isn't quite the same.
I needed a website and did the same. I would say it's a mixed bag to do it from the command line codex. It nailed generating the code but lots of stuff was annoyingly off. Any copy it wrote was terrible but using it to expand and refine my writing was helpful.
I would say we are very close to the end of things like WordPress and templated websites. It's pretty easy to make a custom page.
Do you think "Race Inspired Cosmetic Enhancements" is the origin of "ricing" / "riced-out" or a backronym[1]?
Honest curiosity about language and language use.
In fairness and full disclosure: I have believed until now that the language comes from rice-burners and related, and I have never liked it. If I were still in communities that used it (unix desktop crowd), I might proactively steer newcomers towards your acronym as a kind of reclaiming.
> I might proactively steer newcomers towards your acronym as a kind of reclaiming.
Any negative connotations "rice burner" once had was lost when the term shifted towards referring to cars instead of humans. But now re-recognizing that the enhancements are inspired by the East Asian race turns the connotations back to humans. Isn't that a regression?
Yes, the term originated as derogatory slang towards the Asian population in the early 1900s, and its use towards them became especially popular during the Korean War. There is no question there. However, as before, the term evolved away from people and towards cars. You cannot be derogatory towards an inanimate object. It doesn't have a mechanism to internalize feedback. In that evolution, the term lost its derogatory connotations. The historical usage, while a part of our past, is used no more.
Reintroducing this to be something about a population's race reinstates the derogatoriness. You can be derogatory towards humans. Minimizing the cosmetic enhancements to be being inspired by the East Asian race and not valuable human achievement brings us right back to the same place we were when Japanese cars started being introduced into the North American market, diminishing the human contribution. It is a regression.
She did build a bot that saves cooking ideas with a schedule of dates for when to cook what. We have a lot of friends come over, and we cook various dishes for every occasion. Just a couple of days ago, I asked if she had a recipe for a dessert we tried earlier, and she said, "Yes, of course, let me just check that in the bot."
The bot that runs locally and still helps with the end goal, makes me feel proud of her, even though that was an experiment she was aware of.
>she didn’t find a way to tip the AI into “just do it, write it now” mode
worse, the longer an LLM conversation goes on, but especially with constricted/free models (yes the simple chat interface they are likely using) the harder it is to get an LLM into this mode even *IF* you know the right words to say
at that point the best way forward is to terminate the exchange entirely, and to start off with the right initial message, instantly getting into coding mode. a non technical person will not know this and be stuck in feature theory crafting mode in perpetuity, or worse in an endless "excuses' mode as the LLM diverts ant attempt at coding into reasons why its not going to: "i wont output incomplete/broken code! that would require too many lines of code sorry i wont do it! i wont be able to get it perfect so i wont attempt it! but heres more features and theory crafting"
will a non technical person know to end the conversation and start fresh? not likely unless they have a lot of experience already with LLMs
Also, the correct way to LLM is to constantly trial-and-error in new/branched contexts.
Remember the LLM is not a human employee. You don't have to say "yes and" to whatever crap they produced so as to not hurt their feelings or infringe upon their creative autonomy, nor do you have to defend the correctness of your original instructions so that they don't think less of you for asking them to chase the wrong goose.
I probably generate 20-50 lines of code for every 1 line that I keep.
This is also why I think harnesses and things like Claude Code and OpenCode are false efficiency. The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code.
This is the only way I've been able to get LLMs to produce code I will actually use. Manually selecting the context for them and asking for a specific piece or similar
I started using harnesses because they are good for when something breaks and it's not trivial to investigate so I'll have the agent tell me what's happening, then using that to produce my own change
> The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code.
That’s been the way I do it.
I suppose that it will be considered “quaint,” soon enough, but I have found it to be effective.
A technical person will also have have met those problems related to context and will know to drop a simple
"update an AGENTS file with relevant information"
To be able to navigate that faster on longer tasks. Meanwhile, the lay person does not even conceive of the LLM as a file reading entity. To them, its machinations are its own, so these types of "dumb" (simple) solutions are not even on the deck of cards.
Interesting, but... wrong tool, wrong job. And by tool, I mean web based chat interface, not LLMs in general. (Maybe wrong delivery mechanism, if you like.)
Your friend needed an agent, not a chatbot. I use Claude within VS Code (as per many others) but I certainly wouldn't recommend that for a beginner. They needed a tool that's specifically aimed at people who want to build software but don't know the first thing about how to do it. I think there are a bunch of these now but the one I'm most aware of is Lovable, and I'm pretty surprised you didn't recommend one of these.
An HR person I know was searching for a way get something build, found Lovable, and managed to build a somewhat functional application with it on their first attempt within an hour or two. It was full of holes and far from perfect but they got something working - at least the outline of a potential solution.
As I say, you should have recommended your friend to try building with a tool like that: a tool that they're a member of the target market for. They would have got a lot further. I'm not vouching for the quality of the result, but they would have got something.
Even for experiened engineers, chatbots have always been a pretty grim experience for software development: from the mind-numbing drudgery of endlessly copying and pasting code, commands, and prompts around, to the fact that they just can't see enough of what you're doing to generate the best quality output or advice. You can do software development with a ChatBot but it seriously sucks, and better tools are (a) probably being shoved at you day in, day out via ads, and (b) only a Google search or a ChatGPT recommendation away.
(Obviously, nobody's going to search for "Lovable" without knowing about Lovable, but they might search, or ask ChatGPT or whatever, something like, "How would I build a website without knowing anything about building websites?", which might get them an advert or recommendation.)
The whole point is that the friend wouldn’t know what an agent is. The friend would have heard that AI is changing the world and she would have went into ChatGPT or maybe Claude or Gemini. Presuming that the ideal is that anyone should be able to use these tools for anything (which I do not agree with) the chatbot needs to say “you need to use my agent mode.”
This is not a wrong tool, wrong job issue. The issue is that people think any moron can use an LLM and get professional results.
We're not talking about the friend. I'm talking to the person I responded to: I'm saying they should have recommended a better tool to their friend.
My HR friend found Lovable off her own bat. I imagine she probably Googled or asked ChatGPT something like, "How do I build a website to do BLAH without knowing anything about building websites?"
The point is people talk, they ask questions, they Google, they talk to ChatGPT, and if they have a problem to solve they're often quite motivated to find a solution to that problem off their own backs.
If someone asks me for advice on how to get something built then I'm going to recommend a tool that suits them and their situation, whatever that may be. In this specific situation, if they know nothing about building software, I'm certainly not going to sit them down and have them try to follow the most jank-ass way imaginable of building software with an LLM when I know much better tools exist that are built with people like them in mind.
Seriously, what is with the overly narrow assumptions in the replies I'm getting this morning? You're the third person who's tried to set this same fraying paper tiger on me. Can we all just wake up and think about the issues a bit more in the round, please?
What makes this anecdote hard to believe is that seemingly two things happened simultaneously:
1. The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas.
2. The user was the sycophant in this relationship, never steering the session(-s) into producing something tangible.
The premise supposes that somehow the session(-s) never even tangentially touched implementation/deployment ideas and the user has never typed something like "that's enough, how to make this appear in my browser?". While not impossible, the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
> What makes this anecdote hard to believe is that seemingly two things happened simultaneously
What i described is what happened. I don’t appreciate the undertones where you are insinuating that i’m lying for whatever reason.
> The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas
You call it steering. I would call it falling into that grove. Probably tiny things in the initial message made the first response more likely to be a clarifying/ideating type. And once that happened the conversation was gaining momentum in that direction and neither participant was trying to guide it in a different one.
> the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
Exactly. The LLM itself sidetracked her. They were just talking about cool features they could add, and at no point did she put down her feet and say “stop asking more questions and just write the code”.
It can be a combination of many things. Attitude (some people hate to be rude, and not answering a question feels a bit rude). It can also be that she enjoyed the process of unpacking and elaborating on the idea.
The meaning of the story is not that no lay person can possibly develop using AI. That would be silly, and untrue. I know clear counter examples. The point is that if you don’t know what you don’t know it is harder to steer the AI in the direction you could very easily with the right lingo.
Indeed. It reads like software engineer employment cope by appealing to the lowest common denominator possible. As if solutions for that type of end user weren't already solved for over a decade ago, most of which now have agents built in.
It just is not believable or interesting. Even if it did happen, the reality is it just doesn't matter.
It reads like a discussion between two people sharing detailed experiences of concrete things they've observed, in response to an article where the author did the same. It could be that they're lying to stave off employment anxiety, I suppose, even though nobody involved seemed particularly anxious and everyone involved seems pretty familiar with AI tooling.
It could also be that you've seen a lot of social media memes regarding "cope", observed their ability to provoke strong emotions, and confused this for meaningful insight. I see a lot of AI commentary these days that is clearly being spread for its virality rather than its truth value.
No, not really. My HR friend went out and found Lovable on her own. You're acting like people can't Google, or even ask an LLM via a Chatbot interface for recommendations for tools that would help them build an application as an absolute beginner. They can and do. People have agency, which is exactly why this person's friend asked them for help.
This: https://xkcd.com/2501/ 1 billion people use chatgpt, but only 10M use codex, which is even more impressive when you consider the chatgpt windows app now comes with codex a click away. This means everything that is super obvious to us, is not for the average person
This comment - and many others - misses the OP's original point: what are tools? Do you mean my laptop, and a pen and paper? What's an agent? I.D.E.? none of this is part of the vocabulary or context of someone who has never built things with a computer, let alone AI. Years ago I had a neighbor who was trying to learn how to use their computer; they complained that the book they were using was only telling them all the DOS and none of the DON'TS. This is were most people start.
Isn’t that kinda the point, though? If you already lack the required knowledge to ask a web-based chat UI (which, as an aside, is almost exclusively referring to ChatGPT Free for the average user) to build a HTML page, how could you possibly know to go search for Lovable, let alone figure out agents?
I feel as though the gap between the theoretical power of LLMs and what the average user knows of them and their capability have already widened so far that it’s irreconcilable.
> how could you possibly know to go search for Lovable, let alone figure out agents?
Is this an actually serious question? Am I losing my mind here?
I didn't tell my HR friend about Lovable: she found it on her own. Of course someone's not going to Google for Lovable if they've never heard of it, but they might Google for "how do I build a website without knowing anything about it?", or ask ChatGPT the same question.
They're also, most likely, getting endless ads for AI services that help you build various kinds of software shoved into their faces all the time - these ads may not couch the value ad in exactly these terms, but that's fundamentally what they're advertising.
Not everybody is like this but there are plenty of people in the world who, when they have a problem, are quite motivated to find ways to solve it off their own backs.
Yes, it is. I just Googled that exact question (and I promise I'm not trying to be obstinate when I say I'm Googling the exact phrase!) and it's automated AI response was to use WiX or Squarespace. Prompting it further with "What if I want it to do bespoke things that Squarespace can't do?" it responded with using Figma to design the website UI and then pass it along to either Framer or a "professional developer".
I do genuinely think that this is a discoverability issue. Of course, if you prompt it further with "Could I use AI to do this?" it dutifully responds that it can help with generating HTML, but that's three layers of difficulty to eventually get whatever default model Gemini has for signed-out Google searches to even suggest HTML.
You can see the same thing with restaurants generating their own menus/pictures. Some of them look absolutely terrible, visually ugly, way too information dense, the classic piss filter, etc. idk how they do it, even the most basic prompt I can come up with makes something 10x better, and when I put in my amateur photography knowledge/keywords in it gets pretty close to what I'd consider a good pre LLM quality menu. Literally just adding "make it look nicer/cleaner" seems to get rid of most problems, but people just don't care apparently.
Complete amateurs without these tools were basically limited to making big text in word processors and maybe pasting an image in there.
Being able to create a basically coherent, polished looking image is what you’d get on Fiverr for a few bucks. Mostly hustlers filling in templates, or people that know the tools but never learned design fundamentals.
Actually being a competent professional: Knowing how to visually communicate showing information hierarchy, what purely visual aspects of an image say, how different things read differently among people who might see it— e.g. does an image of an apple communicate fancy computer? teachers/school? Nutrition? Food? Produce?, etc etc etc (Good kerning and type usage, composition, gestalt, etc all come with that for free. Many think that is the point — those are tools someone can wield to do good design, they aren’t themselves good design.)
These tools let amateurs do what the fiverr crowd used to do. Unfortunately, the fiverr crowd is now being pushed into doing what entry-level new graduate professionals used to do, and the job market is kind of fucked.
I think this is about teaching problem solving at a young age and it is an abomination that our education system does so poorly at. The key is to know what kind of questions to ask and knowing when to go deeper and what to pay attention to.
But that is not how our education system aligns us. One typical example of problem solving kids, and I too, learn in school is how to apply a concept in physics to a free-body-diagram(Indian and Chinese education cram schools are famously good at teaching kids how to do this, the usefulness of which I debate). But it all stops at the exam room. No architect or jr structural engineer position for you kiddo.
Another kind of problem solving skill might be how to invest money and understand your own risk appetite to construct portfolios to manage your money. All that is taught in school is a dry compound interest formulae, time discounted cash flows and a black scholes model. Only to find out later I don't need most of it to manage my money.
It's also my opinion that Kids should get to do a lot of real work and experimentation at an early age(13 yo IMO) with real responsibilities and prospects of making income. Knowing the failure mode of most real world problems you can solve is the skill you always want but dont nearly get to do enough of until much later in life(sadly true for a large population of kids in the world).
> They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code.
What harness did you use?
In e.g. claude, there are two modes:
1. Spit out code
2. Draft a plan, ask questions, GOTO 1
You literally have to go out of your way to get it NOT to write code. I keep mine on a tight-ish leash because it modify code way too happily even when there's no intention or instruction to do so
"rawdogging" - as the youth says - weights on local comodore cluster?
They are either using some generic web frontend, ala chatgpt, some local app like claude desktop, or programming app like cursor.
Each of those will detect that you are "building an app" and will spit out code in one form or another. You have to try really hard and be very explicit that you want the output in some other format than code.
How would an absolute newbie know what a harness is? What an IDE is? That code is represented in a set of files? That some of those files are actually metadata? What metadata is? What GOTO means? That you're suggesting a modified SDLC?
None of this is related to intelligence, desire or potential. It's about context and experience. The vast majority of people use their computers as consumption devices, like a TV. If I asked you to "make a movie" where would you start?
I'm pretty sure the idea of the question "what harness did she use" was to ask the expert recounting this anecdote to include more detail, not suggest that he relay the question to his example friend.
That's the terminology they have chosen because they believe they are "in control". It's a purely psychological thing, not different from calling different DB servers "master" and "slave".
When I explain things to people, I say: the LLM is a big mouth, and it is a big mouth without hands. It can only talk. The agent is what gives some hands to the LLM, and then it can do some work.
Harness refers to the tooling that allows you to interact with an LLM. Web chat interface is a harness, CLI coding tools (claude code, codex, opencode, pi, etc.) are harnesses, agent systems (openclaw, hermes, etc.) are harnesses. They present different capabilities to the underlying LLM. Codex, for example, is more likely to write code if you say "I want to build an app" than ChatGPT on web will.
Personally I even refuse to learn what it is in the AI context. Feels like a waste of energy given new ~~~best ways to use YOUR tokens~~~ are discovered every month and things you learn now will be useless next month.
I am doing perfectly fine with the web UI version of these tools... They seem to also not make tokens dissappear as fast as using claude cli tool to automate implementations. Makes my work day more tolerable as well as I actually have something to do over waiting until some implementation can be read through...
As decades-exp SWE I love using Claude Design for any kind of app and web development, because it gives much faster visual feedback loop than changing views within deep framework stack. It much easier to "tell" coding agent what I need instead of writing wall of prose to define visual stuff.
I reminds me old WYSIWYG and unlike Figma it has full HTML/CSS capabilities available.
How I work with it:
- I ask agent to extract part of app into Design, let it even use playwright-cli to get full rendering of the particular view.
- perform design session in Design.
- once design system is perfected I go down to Claude Code dungeons, do /design-sync.
- perform on the stack implementation session.
Actually you don't need Claude Design UI for any of that too. Just ask any coding agent to prepare local mock HTMLs and iterate over them.
I’ve been curious on how to close this loop between engineering and design/product.
We don’t use react, which Claude design seems to trend towards. We use Phoenix / liveview.
We have a shared design system, which keeps the visual elements in line. And then just prototype on design, collab, discuss and arrive at what we want to ship. And then engineering take over and rebuild via hand / claude code.
But the tools aren’t directly connected.
The value has been in the separation. In iterating on the prototype without impacting the codebase, dev cycle, etc. And solving problems/unknowns earlier.
There were always tools for this, but Claude design just feels more accessible and therefore gets used more immediately.
And the fidelity of the outcome (and the assumptions it’s forced the make) are more valuable and faster to achieve than Figma.
I think the way humans divide up design and programming are broadly correct. Working that way with LLMs seems to work well.
I vibe coded an iOS conference schedule app recently, built on top of my own rust UI framework. I started with claude design. I gave it the requirements, and showed it screenshots of other conference schedule apps I like which have features I want to use. I also gave it some visual references for how I want the app styled. It came up with some workable designs. They were a bit 'webby'. But, fine. The high level breakdown of UI screens and navigation between them was excellent.
Then I gave all the HTML files it produced to claude code, along with the documentation for my UI framework and told it to port the code to my UI framework. The first working version was rough. It copied a lot of the unintentional webby look and feel. It worked around missing features in my UI framework by rolling its own janky reimplementations of platform features. For example, instead of using UINavigationController, it rolled its own. It made its own (kinda bad) tab based navigation bar. The app didn't work properly in dark mode, because it was hard-coding a lot of colours. It took a bit of back and forth to fix all of this stuff. But I'm really happy with it now. It looks and feels great.
It's just a pity I couldn't share the app at the conference. Apple took a few days to approve the app in Testflight, and by the time they approved it, the conference was over.
I assume everyone else is playing with the same AI tools that I am, and getting similar results. But a lot of people I talk to seem to have no idea that this is possible right now. They're amazed when I show them my schedule app.
My non-technical cofounder managed to vibe code a holding page with Claude Design and it walked him through deploying it to Netlify.
However for some reason it had him deploy a single HTML file with all the assets encoded as a huge base64 blob in the code that required a massive amount of JavaScript to extract and render.
It's been doing this for our non-technical folk. Giving users a gigantic single file for deployment. We saw one user deploy a JS file with around 3K-5K elements in an array, storing unique IDs of items they wanted to list.
Welcome to Software Development, Lindsey from HR - here's your first database!
People just don't really understand how these things work yet, and they don't know what to ask for, I'm hopeful that they eventually do become more tech-literate, but not sure yet.
The person you responded too didn't mention app though. They just said static. OP was talking about an app but the responded was hypothesizing about something static.
Anyway I'm not so sure "static" is a viable boundary between app and not app. A static page that does any sort of API request doesn't suddenly become an app imo.
If you ask a non-programmer to install Claude Code, just installing it will be a challenge, then opening the shell and interacting with it. Things as simple as copying and pasting can present roadblocks if you've never used a shell before, and things intuitive to programmers like using up-arrow to go back to a previous prompt would never occur to someone in the field.
Claude Code seems so simple and natural of a UI to programmers, it's easy to forget how much it builds on.
Zoom out one step. The experiment should have been searching for "build website with AI", or "build website with <product name>", not "hey use this very specific UI to do something I know will fail"
(FWIW I think people betting their whole companies on AI are trusting shitty one-wish genie goblins, but the terrible irony is that anyone "technical" with years-old knowledge is talking about something else entirely in today's context)
Claude Cowork is the application aimed at non-developers that gives them a lot of the same functionality. My girlfriend uses it and has gotten quite far in producing her own software.
With Claude it's actually not that hard. I set up a system for a startup where devs used a full dev environment and non-devs could use the Claude web UI to open rougher PRs, and one of the non-devs asked me what was involved with getting a full dev environment set up, so Claude could iterate and he could test locally, which was faster than the "push to GitHub, use the Vercel/Supabase preview deployment" workflow he was using.
I gave him a link to Ghostty, a link to the claude code copy/paste pipe to bash thing, told him how to cd/ls/pwd into the folder he had locally from the GitHub app, and he was off to the races. I told him to type `claude` to open Claude Code in the terminal and gave him a prompt to use about being a non-dev getting his environment set up to the point of being able to pnpm dev and test, and Claude took it from there. The repo's readme had a setup section which it followed to install brew, asdf, pnpm, etc. With the GitHub MCP he's now opening PRs the same way devs do.
No, but I bet this has a high likelihood of coming out in preliminary discussions.
What most likely happens is a normie has no idea of how to build something that does nothing first - they just start describing the end state.
I'm going to try this with my wife later today. I bet she'd sort something out since she's been a manager forever and phases out instructions maddeningly
Perhaps! But I do think the vocabulary issue is real and I think LLMs are still sycophantic enough that they won’t really challenge someone or offer alternative ideas on how to implement something unless they explicitly ask.
Interestingly at my work, Claude Code was available before Claude Desktop, so a number of non-technical PMs tried to use it in order to build… anything, with very mixed success.
The “hey guys, check out the website I built with Claude: http://localhost:3000/” joke is real!
In my experience, the whole “the terminal is a scary place” aspect is very real and some non-technical people can feel intimidated by.
I think Claude Code in the desktop app helps alleviate that a bit (perhaps Codex, too, but man what a mess the ‘ol ChatGPT app has become).
But I’m sure there are entire repos of web dev skills that someone could use to put together things with a bit of effort.
Isn't the the powerful, unlimited, unopinionated blank LLM text input waiting for your instructions eerily similar to a scary terminal?
WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
Terminals are scary because it feels insurmountable. What are you supposed to do? If you just type “start python program” it gives you this absurd error that doesn’t make sense. What do you mean start is not in path?
The moment you interact with an LLM it gives you feedback that you’re doing things right. It feels like a gradual climb instead of a series of abrupt jumps. People really don’t like feeling like they don’t know what they’re doing, and the terminal constantly reminds you that you are making mistakes.
Yeah, if you're willing to put in that much effort.
I started on this path literally about as soon as I could read thanks to the family having bought a Commodore 64 for my older siblings, but also perfect timing in that when I got to this age the sibling whose room it was in had just gone off to university.
Most people are not like this, in much the same way that they're not going to read the T&C end-to-end (another thing I've done) or learn enough law to actually understand what those words mean (a step too far even for me).
I've had people criticise me for having had the opportunity to learn in that way, as they did not.
> You also think that you are smarter than the people flooding Ceuta streets these days, don't you?
No, why would I think that? I don't know them, the only thing I can say is in their favour: moving country to better your situation is difficult and them getting as far as they did is a demonstration of putting in a lot of effort of the exact type I praise by default.
> Isn't the the powerful, unlimited, unopinionated blank LLM text input waiting for your instructions eerily similar to a scary terminal?
This is why it has the title (for me currently reading "What can I help with?" but this varies a lot) and the text box itself has the placeholder text "Ask anything". Sometimes I get big friendly suggestions about what to ask it, placed on screen near that text box.
> WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
I don't think it's constraints, per se: almost nobody looks at the font list and goes "oh no, too many options!"
Rather, GUIs are there to organise your options visually, group them in ways easy to intuitively get. There's a bit of fashion-induced rot here, e.g. I'm old enough to remember when it was always unambiguous when you were looking at a checkbox vs. a radio button, and now there's a blurry middle ground of collections of boxes with ticks in them that act mutually exclusive, but the point of a GUI from a UX POV is not the same as how software in general drifted as it got both more users and more developers and more opinionated managers and middle managers and designers who only cared about shiny rather than usability.
I am old enough to have observed non-tech workers using all kinds of text-based interfaces and it was a real pleasure seeing how old ma's would jiggle numbers on the bc-style TUI in the way that would offset any modern CompSci major.
Me too. But did you see them on their first day using it or their thousandth?
(Also you don't need to be that old. Less than 10 years ago I watched a doctor breeze through some clinical system while I was crawling along constantly referring to the manual)
What "democratized computing" was cheaper computing and VisiCalc, not WIMP and GUI?
But again, VisiCalc is intentionally limited, it's not a all-powerful environment, on purpose. It's all about intentional limitations, making computation easier to reason about.
Not really, for making a single-file anything ex nihilo. I suppose the chat window won't be able to run a linter or make and run tests as a typical "eager" agent might, so maybe it will make more mistakes.
Almost all web chatbot providers have code sandboxes that they will run (limited) tooling for you. If you ask for it, it will run deterministic linters, formatters, format converters, tests for you. Older versions of Claude would for example happily try to reverse engineer entire artifacts for you, once you provide a URL.
My experience is similar, people succeed with Loveable or similar platforms because they don't need to know _deployment_ and _runtime_ experience. It's like using Adobe vs Canva, most people are now used to the latter and don't even have a mental model of "runs on a server" or "database and server are two different things". This is the key difference for me compared to earlier Low Code solutions such as OutSystems that still required a SDLC mental model.
However, at some point this breaks, like in your example of feature discussions, because people struggle to test out new ideas _without_ immediately showing them. So it becomes Canva vs Figma, a system-based model. And we don't even have good terminology for that ourselves yet.
We've seen plenty of examples of apps successfully vibe coded by non technical people, including apps making real revenue.
Your friend could start with telling the LLM that they are a non technical person who wants to make an app and it will explain all the successive steps.
Maybe people don't cite specific apps because they like their jobs, and outing apps as vibe-coded is still seen as negative
I personally know of two completely vibe-coded large apps in my professional environment. One by a non-technical manager, made to solve his needs, then sold to customers. Initial development went along great, but by now velocity has greatly slowed down. Also took a lot of engineering hours (of actual software developers) to get permission management from "chaotic and ineffective" to passable. It's still worse than what you would have gotten by just using a couple sentences of the right technical language at the start. Deployment is also a bit of a nightmare. All in all, anything beyond the first rollout phase was delayed by months. Honestly it should have stayed as a prototype that then gets rebuilt from the ground up. But still, it is a real app, making real revenue
The other example was vibe-coded by a software engineer in his free time. Works pretty well, doesn't have too many bugs. Makes some revenue, but a lot less. Solving manager problems just sells better.
I vibecoded a Postman/Insomnia API tester program, and I use it everyday at work now.
But as another software engineer, I remove myself from that comparison, because the idea is to find out if a non technical person can do the same, that's the definition of vibe coding.
A friend of mine, non-technical, is not making money with his apps. But he's creating a street fighter like game. Just for fun.
So there's that.
He can't exactly release it because he uses a lot of copyrighted stuff. It's also meant only for himself. Though, I've been asking if I can play it, it looks fun.
I think we desperately need to start differentiating between "is creating" and "has created". I have a couple of "am creating" projects too, but their proximity to "have created" is directly proportional to how much effort and expertise _I_ am bringing, not so much related to the AI's contribution.
I run into this quite a bit. We have users generating MANY apps at our small company (30 FTE), entirely with Claude. It's great to see people mess around and tinker. It's NOT great to see someone with a GH repo that has 750+ commits for what would be MAYBE 1 week of a developers time. SO these are non-developers now spending hours and hours working on software that is probably going to get thrown out.
We're in this spot where we don't know when to cut our losses on projects like this. (Is it even viable as production software? Does it currrently do what it's supposed to, or are they adding new features? Is there a return on continued development efforts?)
None of these apps they have built are seeing any major usage, and I don't think a single one is what I would call "done" (There was a gold rush stage at the beginning of 2026 where senior leadership wanted everyone to spend some time messing around with Claude). Unfortunately, they never told anyone when to stop messing around with Claude, so the ROI is ever diminishing.
The problem with lovable, from someone with insider knowledge, is that many of the apps existed even before appearing there and where ported to the platform to ride the hype wave.
Every now and then I'll write a single prompt to Claude to see how much of a web app I can build in one shot. I usually pick Django (though I assume any other web framework could suffice) because of the battery included nature, I still sometimes describe that I want users to login / register, have access to x, y, z
I also noticed a friend of mine had way more success by having Claude write code by doing TDD and giving Claude scenarios for things the code should be able to handle, if you do this correctly, and cover edge cases, Claude will work with these in mind.
OTOH my wife's friends got drunk and made "tinder for horse purchases". They prompted to read typical horse advertisements (we're all horse people) and create an app with mock tinder like entries to swipe right and left to buy horses.
A web app was produced with lots of mock "Hi i'm Dominique and i love running through fields and having a bucking good time" type entries complete with silly horse photos. A huge amount of drunken fun even if it boiled a towns water supply and blew through half a subscription to create.
I was looking at the results as a dev with 30 years experience and thinking fuck me. The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim!
I'm wondering: What did she tell you that made you build the website? Did she ask the same thing from the AI? Did you bridge any gaps the AI didn't do for you? I don't know the answer, but I suspect that she treated the AI differently and would've gotten better results if she had asked what she told you. At least if it's an agentic coding assistant like Claude Code or GitHub Copilot. Of course a simple chat will leave manual tasks for you.
I find there's a lot of variation between LLMs / models...
But there's also some psychology in play too; that we (engineers) see a lot: Some people just let their imagination run away and forget to "do"; without someone in the conversation pushing for results and action, the conversation will just stay within imagination and everyone will be happy in the moment but nothing will get done.
This sort of thing has been studied in academic experiments. Although the models studied are now “old”, and I expect the floor is higher, the lack of vocabulary and basic concept familiarity sets the ceiling.
I have the same experience giving my brother an OpenClaw as his personal assistant.
His words were: It feels as if I need to know how to program it.
I was expecting he could say something like "Oh, it seems like you don't remember the people I'm referring to, perhaps you need some kind of CRM system. Can you investigate if there are any easily available CRM systems you can interface with, so we don't need to make one for you?"
Whereas my OpenClaw moment was trying to make it manage its own NixOS installation, so that if I ask it to do something, it doesn't yolo `apt install` commands, but rather improves on the same overview of its own installation.
A lot of people had success making their OpenClaw do things without being Linux experts. But you need a tinkerer's mindset, is what I came to conclude.
Do your friend at least know what Claude Code (or any harness) is?
Of course she wouldn't be able to make a website if she doesn't even know the right tools to use. But I don't think it prove anything. Knowing and installing Claude Code might not be a common sense, but nor is it "expertise" or "skill."
I've seen in first hand that people struggle installing Steam. Yes, "people" in the plural. But just because some people struggle with it, it doesn't mean that installing Steam isn't an objectively easy task. Your friend's experience doesn't change the fact that building a website is something that an average person can do in hours if not minutes.
Knowing what Claude Code is and why you might want it actually is domain knowledge which the op's friend does not have. Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise.
> Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise.
I just said I've seen multiple people struggle installing Steam...
The point is that it's something objectively easy. Once they find (in this case, given by me) the correct instructions and follow through, they can easily do it by themselves again. It's quite different from what are traditionally considered "expertise": for example, even if you followed a master's painting process, stroke by stroke, tomorrow you still don't know how to paint.
Building common apps were more akin to "painting," now it's "installing Steam."
> The point is that it's something objectively easy. Once they find (in this case, given by me) the correct instructions and follow through, they can easily do it by themselves again.
This is true of most things in life. It is very easy to make compost, it is very easy to grow carrots, it is very easy to graft an apple tree onto rootstock, it's very easy to hang a door and it's also very easy to replace the break pads on your car.
Once you've done it, that is. And once you know what tools you need. And how to use those tools. And that you actually have those tools.
Codex, Zed, the like are all tools that you need to know exist and you need to have and you need to know how to use. It's the same thing as a wrench, a break bleeding kit, or some graft tape.
You are assuming that the hypothetical friend has been isolated from any kind of technology from the last few decades. The friend can, presumably, use search engines and the very same chip chipities to get the very rudimentary domain knowledge going.
I'm genuinely a bit floored reading the comments here, but I guess my idea of the average HN commenter's ability to talk to non-technical folks about technical topics is biased because I work and socialize with lots of people who don't have technical expertise (thankfully along with other technical folks who also have lots of experience talking to the former group). Many of them don't even know what Claude is, let alone Claude Code.
Yeah it's a true HN bubble moment. I've got non-technical people in my company (a tech company, where the majority of the employees are engineers) who have absolutely no idea what any of the agentic coding nonsense is, their only exposure to this stuff is Gemini in Google's office apps.
To them this is all just "AI" whether it comes from OpenAI, Anthropic or Google - hell they probably don't even know what an LLM is in the first place, yet alone which company provides what tooling. And these are people whose day-to-day involves talking to at least 1 dev a day, so you would imagine some of the knowledge would materialize via osmosis at the very least.
I don't think this is true. Certainly not in Britain.
I went to 3 weddings last summer and each one had a joke in a speech about using ChatGPT to write it and everyone laughed. 68yo father of the bride is a retired plumber and even he's cracking jokes about AI.
> I've seen in first hand that people struggle installing Steam. Yes, "people" in the plural. But just because some people struggle with it, it doesn't mean that installing Steam isn't an objectively easy task. Your friend's experience doesn't change the fact that building a website is something that an average person can do in hours if not minutes.
I think I would define "easy" with reference to the % of people who can do it. I don't know what that is for Steam, but there's a (now dated) survey of computer literacy in OECD that I keep coming back to in order to set expectations for what "average" looks like:
I have counter evidence of this. My wife with absolutely zero skills in computers was at the terminal doing things that Claude was telling her to do and generated some impressive tools. One was a tool to help her organize her day. It involved scripts, PDF generation, printing. She did it all without even asking me. Maybe it fails in some cases but I'm not sure the anecdote above is the average experience.
did she try saying "I want to make a webpage"? Even ChatGPT will just build, deploy and host a webpage for you with that request. I don't really understand what system she must have been using.
Having a similar experience. Watching a muggle try to build a website with basic functionality, e.g. auth, database, etc. is an enlightening experience. They simply do not have the vocabulary to guide the LLM. My good friend calls me every night frustrated with his results, and when I watch what he is doing it's amazing what we take for granted being in the software industry. Don't get me started on the UX, that's even more mind boggling.
I've seen some designers and product managers get pretty far with LLMs but mostly because they already know how to build apps just from a non-technical perspective.
I have been pondering this and I think it's likely a gap that will get filled sooner or later.
Right now there's just so much value in building LLM tools for experts that everyone is focusing on that. But surely at some point we'll have bespoke harnesses that exist exactly to solve this kind of thing.
I think this can start with constrained problem spaces like "you are a WordPress developer, you solve problems for people with enough expertise to know they are looking for a WordPress developer" and incrementally expand from there. Maybe I'm naive but I think you can probably get pretty far with this today just by writing loads of skills and picking the right technical preferences to encode in them.
Yeah there's lots of companies out there that promise that people can build a website quickly, often using WYSIWYG / drag / drop interfaces, I bet they already have AI integration to speed things up. ("I bet" because I don't actually use those services.)
I think people are better helped by using those services than going a level lower and using LLMs directly.
We already have these, and have for a while. Replit and Bolt exist for exactly this case. Not sure why GP didn't direct their friend to these tools (or if these tools are somehow the ones that failed in this example). In my experience, Replit and Bolt (and other similar tools) are quite good at this kind of 0->1 kind of thing.
him: Oh, I thought I would write something like: "Create football stadium and football players. Start the game when user presses spacebar. Make players have red shirts and white socks."'
Yes, but usually programming knowledge does not outdate so quickly. Now we actually have prompts, what some people always believed that it always worked like this.
Weird because I had a friend that wanted a web app, also no experience, and he just told Claude (on the iOS app) to make it for him and Claude just made an artifact and put everything there.
Buddy sent me a share link and he was just like “dude this is crazy I just told it what I wanted and it just spit it out in a few minutes”. No “experiment” needed he just did it because he knew Claude could do it and it worked exactly like you’d expect. Hell, he did it on the Free tier.
I just gave this a quick try with Sonnet 5 Medium [0] on a free Anthropic account. It's a bit of a contrived example I guess, but it probably isn't too far off what a completely non-technical person seeing this for the first time would do.
The output is EXTREMELY misleading, as all the data here lives purely locally, yet the AI says that you can "just share the link" and other people will see the schedule you set on the generated artifact. Also, what link? To the Claude chat? It doesn't explain what to do with that `Booking` artifact other than "link to it".
I can so easily see someone tapping out a few steps down the line of this once they realize it doesn't work and they have no clue what to do or say to make it work. What do you even ask as a non-technical person at this point? I guess they could explain "The other person doesn't see it", but would the AI actually clarify that it's because it's not fucking hosted anywhere and has no mechanism of persisting the data outside of the current machine, or would it - as I'm almost 100% sure would be the case - not actually point out this error?
I've seen the same thing happen to my brother when he tried making an app with zero experience. Only difference was that he got a front end that didn't work.
same thing as people just entering a question prompt and copying and pasting the response as gospel. e.g. politicians using it to write speeches, or lawyers for testimonials.
the output is programmed to look correct so unless you have some sort of background you won't actually know what errors to look for.
Not only does it look correct, it looks correct with an extremely Subject Matter Expert degree of authority. Often I'll work with an LLM, and it simply just misses so many things. I've worked in all sorts of different domains, software, chemistry, material design, everything from power generation through to physics, and in each and every case I see it missing incredibly important things. Any true subject matter expert would immediately bring up and prompt concerns, but not the LLM.
This makes sense, of course, because these are language models. They were trained on language. Their first and foremost capability is language.
An LLM's true expertise, true subject matter expertness is language.
And so anyone working with LLMs who isn't already highly skilled in the field they're asking questions about, will invariably be led astray and miss extremely important parts of a puzzle that need to be solved.
That is exactly the key or the sign there. Even with a couple of decades of engineering expertise, when I try to do / research something that I don't know enough about, I find myself in the exact position of not having the vocabulary.
To the point that I sometimes have to ask the AI "nicely" to cut the pleasantries and be ruthless against nonsense, whether from its/their side or from mine.
> Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode.
Even as a developer, when I've been using the web chat interface for things which I know the AI can do easily, I've had this happen to me a few times. I was very surprised the first time I saw ChatGPT respond ~"this would be a few thousand tokens, I can't do that".
Even more surprising: ChatGPT was accurate when responding that way this time, despite this being trivial for Claude and well within what ChatGPT could do using the web chat interface 6 months earlier. The ChatGPT output was extremely meh.
Question is if the outcome was LLM related or just a result of a personality trait. After all, talking in circles without really approaching an end-goal is something certain humans do all day long.
I’ve had the absolute opposite experience with a friend of mine. I started by setting her up with a terminal emulator on the web linked to Claude Code, and written a CLAUDE.md that told it how to deploy. These days (with no further intervention by me) she’s running Claude Code natively on her laptop, and she tagged me yesterday on Facebook in some update about how much more she enjoyed using Claude Code than plain Claude.
So maybe your friend need a role reversal:
A prompt guiding the llm to act as a consultant, guiding people in implementing a software project, asking questions, creating a shared understanding, limiting scope or creating milestones.
That seems like something that could be done using an llm, not that complicated probably.
The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot.
People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a replacement for their minds and their senses will struggle.
One of the Claude skills I made Claude itself generate was the 'learning a concept across tiers' skill -- from ELI5 level to a PhD level, and it triggers whenever I ask it a very general question on a complex topic that isn't my bread-and-butter. The fact that I'm able to choose explanation level from a super smart LLM (that's available 24x7) that can explain any topic under the sun would've been mind-bogglingly sci-fi-ish just 4 years ago in 2022.
When I get out of my domain, I always ask it to describe things to me "like I'm a senior CS student who doesn't know any industry terms". I find it really easy to skim past the definitions I know and very useful to get the explicit clarification on unfamiliar terminology. Telling it to talk down to you a little also seems to calm down it's tendency to talk like it's trying really hard to convince you it's smart like a 8th grader trying to use every single vocab word they learned last week.
I've been using a prompt that comes from the opposite direction for non-programming stuff: "Assume the user is an expert in all fields; while this is clearly logically untrue on a literal level, the user prefers to see a field's technical terminology and then ask the AI about terms the user does not understand rather than get an inaccurate statement about some issue."
Whether you have to reassure the LLM that this is obviously untrue, I don't know, but they do have a knowledge baseline to know it's not true and I have a sneaking suspicion it would be less effective without that.
This has ended up in some of the most interesting incidental knowledge exploration I've ever done. A recent example is that I was asking about some stretches and it started talking about how useful they are for the sarcomeres, which I had not heard of. Now I have.
I'm not saying this is better, just that it is different. I think there's a time and a place for both approaches.
Off topic, but stretching is absolutely useless for ... stretching. And vital for retaining and enhancing usable range of motion (not static stretching, still, but dynamic stretching / mobility under tension).
The wildest thing is, there's no evidence that I can find that static stretching does... anything? It increases pain tolerance through the range of motion, but not any more than just asking people to try harder etc. And it doesn't build up, so after you stretch once for a given day, you're done.
Edit: after looking into it a bit further there's actually a cochrane review that stretching does bupkis, fascinating. It's also the first "Good" evidence graded cochrane review I've seen.
Can you share the source? This seems dangerously wrong.
I see a specific review making the very narrow claim that it can have detrimental effects on power/strength activities immediately following static stretching.
But by and large I'm finding a great deal of evidence for a wide range of other benefits, particularly in range of motion and injury prevention. I can't find anything widely damning, and I certainly don't see any reviews contesting the validity of the very vast body of research supporting the many benefits of static stretching.
Most of this comment is not true. Static stretching does increase flexibility over time - it’s not the most efficient tool for it, but it’s certainly effective.
Presumably the Cochrane study you’re referring to is “Stretching to prevent or reduce muscle soreness after exercise”, which as it’s title suggests, investigates a narrow question and does not support you claim that “stretching does bupkis”.
"And vital for retaining and enhancing usable range of motion (not static stretching, still, but dynamic stretching / mobility under tension)."
I'm doing it for knee pain. It's possible it's a placebo because there is a slight strength training component to the whole thing (it's several exercises), although the knee pain has persisted through a lot of strengthening of my leg in general. This falls under "don't care" as the program as a whole works and I'm not worried enough about the details to try to optimize it. All the stretches fall under "dynamic stretching under tension" so maybe it's not something covered by the study you reference last.
It’s not a placebo. I’m not sure exactly which Chocrane review the GP is talking about but if it’s the one I’m thinking of, the headline is that stretching doesn’t help muscle soreness [1] (delayed onset muscle soreness specifically).
You should absolutely keep stretching to help with your injury, and there is decades of “sport science” to back that up.
I can tell you if I don't stretch, my muscles feel bad and stiff. No doubt this is partly because I did karate for a few years, where stretching to increase your range of motion is absolutely required. You're not going to "just try harder" into a high kick.
Sometimes when I want AI to explain something technical, I say "explain it like I'm a junior engineer" -- just to get it to start with the high level like a human being would.
> People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a replacement for their minds and their senses will struggle.
Claude code is entirely vibed. Someone posted some of the prompts they used: barely comprehensible typo-addled half phrases.
I am sure some variant of the infinite monkey theorem applies. Anthropic devs have unlimited access to Claude, so they are no doubt lazy when it comes to prompt writing and just brute-force the agent to do what they want.
This was true some time ago but nowadays I don't get this impression. Seems like whatever style I type in, the LLM is already pre-prompted to respond in "its" "own" "style".
As an unapologetic generalist[1] this has also been my experience. Many tools that would have been "eh maybe if I get bored over Thanksgiving holiday" have become "hold on, gimme fifteen minutes".
Tiny, isolated, but awesomely useful CLI scriptlets, for me, seem to be the sweet spot. Little shining rays spreading out from the veins of my own familiarity.
The downside, the Achilles Heel of LLMs, so far as I can tell, is using the system to assist in maintaining large, sprawling, and largely pointless legacy codebases. Somewhere you have to keep many many many stupid things alive. I swear I can almost sense Claude's frustration with some of this shit. Then you get frustrated, and then Claude wants to agree with you so it acts even more frustrated, and the gyre thus widens. You're just cussing at everything with a machine. Which can be fun - Claude is often surprisingly funny - but not productive.
[1] I know absolutely nothing about positively everything, but have the attention span of a squirrel.
I've found Claude Code absolutely amazing for the sorts of 100-500 line data cleaning/analysis/visualization tasks that used to take me a couple hours to knock out. They're often self contained (boss wants a graphic for a slide or some numbers), and I tell it which packages I would prefer it to use.
On the other hand, I've been using it to make small changes to a ~4000 line codebase, and it takes a lot of wrangling to keep changes in scope.
Today I'm translating a 5,000 line VB6 codebase to C#, and I've been spending the day chopping up the job, passing it to Claude, and manually validating it.
I went to VB.NET first using a previous generation of LLM's (that was quite manual back then) and then from VB.NET to C# or just keeping the VB.NET around worked very well. The code was not highly complex but more than just CRUD. The porting from VB6 to VB.NET included building tests which helped.
Your LLM is not "smart" and it cannot "explain", it is a program that can output a statistically average text of an explanation. Don't mistake the capabilities of these models for intelligence.
It explained something to me today. So your point is disproven.
What is intelligence then if not explaining things. They can be intelligent without consciousness if that’s what you mean
It explained something to me today. So your point is disproven.
No. It produced a statistically likely answer.
What is intelligence then if not explaining things.
"the ability to learn, understand, and make judgments or have opinions that are based on reason"[0]
LLMs by definition cannot learn, understand, make judgements, or have opinions. If you find the model to be a tool that is useful to you, by all means. But do not anthropomorphize.
You're just talking semantics. To me if I read text that helps me understand something, that's an explanation. The black box it comes from is irrelevant.
LLMs do learn, understand, and make judgments. You're just pushing back on the underlying mechanism, which instead of reason/sentience is instead statistics and weights. I'd call that intelligence but I don't really care if you want to call it something else.
The amplifying mirror is not just a characteristic, but the fundamental driving force of LLMs. With every token it consumes, its primary goal is to understand who you are and what you intend. If you type Hola thats sufficient to tell it so much more than if you type hi.
Not the OP but you can whack this into your prompt and get most of the way there: "no jargon goes unearned, nothing gets dumbed down, every abstraction touches ground"
This is something that really needs to be formally studied.
I'm inclined to say that this matches my own experience, but I can't rule out confirmation bias on my part.
As a meticulous person generally looking for a very specific code outcome, I prompt in a way intended to get exactly the thing I have in mind, and my results reflect that. But on the other hand, I have coworkers who type ten-word prompts with very limited specificity, and they seem to get results that way as well, and that makes me wonder.
It would certainly be beneficial for my career and financial well-being for the assertion to be true, because it means I don't have to worry about being pushed out of my job by an army of $15/hr vibe coders. But the convenience of that assumption is exactly why I think it's important to be skeptical.
Meanwhile all I do is vibecode. I get the outcomes I want though.
I see vibe coded apps as requirement documents. Rarely do I have to engineer.
If my job gave me some actual tasks, then maybe I'd engineer something. But at home? Vibe coding all the way. I'm open to engineering, but I need a compelling reason such as: the app is fundamentally broken and an LLM is going in circles. When the only user is me, there are not many performance issues to think about or fix, so that helps. Moreover, certain systems don't need to exist (though they might soon since now I have a smattering of apps that I need to manage).
I suck at writing code, so strictly speaking: Everything I do is vibecoded.
The stuff that I produce in this way would probably be considered by many to be unusable trash. But it solves the problems I have, and it does so with exactly the amount of precision that I demand.
When I built a PWM fan controller for a pro audio amplifier, I was very particular about some aspects. I wanted maximum resolution from the DS1820B temperature sensors (which is a relatively slow mode where reads take ~750ms, and often the bot is primarily interested in fast), and resolutely-consistent PWM output (so software PWM was a non-starter).
It was very important to me that the fan speed ramp smoothly and without audibly-discernible steps, so the target output goes through a low-pass filter to smooth things out and the final PWM value gets recalculated at a completely-overkilled rate of 1KHz.
Power consumption was a very deliberate non-concern: The power used by the MCU is ~nothing compared to that of the whole of the system, so optimizing towards reducing it was never my goal.
At the end, it's a rewarding little project that is all wrapped into a state machine that burns clock cycles like they're free (they are free!), and it works very well.
There's parts of this thing that I do not understand at all, and that I have no desire to understand.
But if I hadn't been so particular about the parts I did care about, then: An underspecified one-shot prompt seems like it would probably have just produced a loop with a lazy 1-second sleep at the end, since being sleepy and power-efficient was a feature that the bot kept working to reintroduce.
I spent a lot of time working to dismantle the bot's proclivities to be this way, and I probably would not be happy with the end result if I had just let it do its thing.
Differently-stated: It could have been an unsupervised one-shot prompt, and the result almost certainly would have done the job of keeping the amplifier cool. (I just would not like it.)
Both the article and some of the discussions here share a lot of commonalities with doctors taking a medical history. There is a certain skill in guiding the conversation towards useful outputs, while not dictating the exact outputs to a patient who is eager to please with their responses.
E.g., medical history taking protocol always says to start with open ended (albeit structured) questions, and converge towards more closed/specific ones when you're sure you've extracted the broader surface and you now want to close in on a differential diagnosis.
If you start open and go with the flow but then just let the patient talk without any structure or subsequent attempt to converge, there's a risk that the patient might spend 60 minutes taking about their fluffy dog at home, which wastes time, and doesn't get you anywhere nearer the diagnosis. But, if you skip the open questions and go straight to yes/no diagnostic questions, you will definitely miss the fact that they have a dog at home that they're worried about, and that they'll be self-discharging against medical advice in the next hour to go tend to their dog.
So while to an outsider, the conversation might look effortless, in reality the doctor requires considerable skill to be able to strike a balance between open vs closed prompts, as well as the ability to critically sift through the outputs, and decide which outputs are relevant to pursue further and lead to a fruitful direction, versus those that can be safely discarded to remove potentially distracting noise from the conversation (and all while attempting to keep this interaction within a limited number of prompts due to operational time constraints).
Counterpoint, some doctors will zoom in on the most likely problem and misdiagnose. This is in part due to pressure on the health care system (where I live anyway); you can only get a GP appointment for 10 minute blocks, which really isn't a lot.
But when a 30-some year old shows up at a rheumatologist with joint pain they will likely go to unusual (at that age) but not unheard of rheumatism/arthritis, not hypermobile spectrum disorder. When a woman goes to a GP with period pain they will be prescribed mild pain killers or anticonception pills and fobbed off, until a decade and much suffering / many more issues later they get diagnosed with endometriosis.
Indeed, I didn't mean to imply that all doctors possess that skill in great amounts (or that, in any case, circumstances allow for full utilisation of the skill even if they do) or that medical history taking is therefore always of the highest standard. Obviously some doctors will be better at it than others, or more at liberty to use this skill than others (as you say, I could imagine a GP who only has 10 minutes per patient won't be as keen to start with fully open questions as a junior doctor in a ward who can clerk patients somewhat more freely).
Note that this isn't too different from, say, how software engineers are expected to be good at, and make good use of unit tests. But most probably don't (either because they never really cared to fully develop that skill, or their organisations applied contrary pressures leading to tech debt). But it is a recognised skill.
My main point was that, it is, in theory, a skill that doctors are expected to train (or at least pick up on during their practice), and therefore the same prompting principles that seem to apply here in the context of LLMs also interestingly seem to apply to medicine and history taking when "prompting" and interacting with humans.
You can’t run expensive, time consuming, and potentially harmful tests on every patient. Someone comes in with a headache, you tell them to go home and take an aspirin. You don’t send them for a a CT scan.
If it doesn’t go away and they come back, you start considering more serious issues.
It’s expected that uncommon non-emergent diseases will be diagnosed over multiple visits.
I think endometriosis is common enough that it would be caught earlier ("Oh, does your joint pain get worse right before you menstruate? Tell me more"), but your point is reasonable.
I counter with the platitude that common things are common - especially in fields like primary care, the amount of wasted effort one would expend in pursuing unusual explanations for every presenting symptom is considerable. We thus have to examine patients over time and trust that they will tell us if things have indeed not followed the course of the initial diagnosis.
While this is true as a platitude, I do find that medical colleagues seem to treat this platitude rather overzealously at times, often completely disregarding the posterior probabilities involved. I like to call this the prior-over-posterior fallacy/bias (though one could argue this is simply a variant of the normalcy bias)
E.g., I get the whole "if you hear galloping think horses not zebras" adage, but I guarantee you, if someone comes and says "hey when I was in Africa I saw a black and white striped animal galloping really loud", I bet you an uncomfortably large percentage of the "horses not zebras" crowd would still favour a horse over a zebra diagnosis, despite the overwhelming posterior.
Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests, and it becomes a big problem, whereby uncommon conditions are treated as common, thereby often escaping detection altogether, and driving down their apparent prior probability even lower, causing a diagnostic vicious cycle.
I hear the complaint, all I can say is that everyone I speak to about this wants to diagnose things correctly while not wasting anybody's time or energy on bad diagnostic workups.
> Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests
There isn't a way around this: if you order the test and a value is abnormal, you are now committing yourself to treat a thing. We should not be ordering tests if we aren't ready to follow their results to the logical conclusion. So I would disagree that this is a problem in the way you've framed it.
I'm not disagreeing with what you say here, but I think we're arguing about subtly different things.
E.g. you'll note my zebra example was not about whether one should additionally request a photo of the animal just to make sure it is indeed a rare animal. It was arguing that given existing differentiating information that points to an uncommon condition in the first place, one should not dismiss this on the basis that horses are still more common than zebras in the general case regardless. Under this uncommon information, the prior of thinking about horses should go out the window, and one should absolutely focus on zebras (at least as an additional differential). I assume you would also agree with this conclusion.
But of course, in real medicine things are not as simple as this contrived example. So the point I'm making is that, from what I have observed, there seems to be a bias towards decisions based on "prior" rather than "posterior" probabilities, even in the presence of additional differentiating information which should have prompted additional differentials to be considered. But this is different to saying one should constantly seek additional evidence to include or exclude additional differentials that are unlikely in the first place. That, I agree, would be a waste of time and resources (and potentially harmful for the patient).
Having said that, I somewhat disagree with the phrasing that we should not be ordering tests if we aren't ready to follow their results to the logical conclusion. This is a bit like saying, "I don't want to check for X because if I do and it checks out it will derail my current management plan"; but then this is putting the cart before the horse, since it's the findings that need to dictate management, not the other way round. I do think it is reasonable to say that one shouldn't be ordering a test if the expectation that it will change management is sufficiently low to justify the time/cost expended for it -- and where this expectation is a function of both the likelihood of the finding (given current information!), as well as its potential to change management. But this doesn't mean that if you do find an unrelated inconsequential incidental finding you are now required to divert all resources to it.
Conversely, if an incidental findings does prove worthy of treatment, then I don't see the problem with committing to treat it, as long as you don't lose sight of the original complaint either. Obviously that doesn't mean one should go looking for incidental findings willy-nilly though.
You'd think, but I have 4 friends who have been diagnosed, and they all have horror stories. This is in 3 countries, none of which are the US, but from reading Reddit, it seems the US has the same issue.
> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems.
In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasonable to actually apply to it.
This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise? Sure you can ask questions about the codebase, but IMHO that falls under surface-level information, and the devil is often in the deeper details. Hmm.
We are having trouble onboarding engineers with AI now. Some still struggle after their first year with very basic concepts/patterns we use and make the same mistakes again and again in their PRs because they just ask Claude to fix it and never internalize it. I think using LLMs feels good at first because you can get tickets out the door faster but you never develop enough knowledge to make a big impact or become an expert in the code or business.
I imagine they are still good hires - however there is a real cognitive debt that happens when we code with LLMs in my experience.
It’s harder to internalise concepts because you don’t go through the struggle of understanding them and finding the mental links you need to remember later.
I notice this with people around me - all of them are doing more things, but I am also catching more issues when reviewing docs and code.
Honestly you should probably invest in baking the basic patterns and concepts of your codebase into shared skills that are applied to everyone's Claude sessions.
I think this is only a partial answer but I've been surprised by how familiar dev leads become with the app even if they are not in the code.
They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.
It's not for everyone, but I write all the code and then I ask the LLM for a review. It catches something minor thing every single time (out-of-date comments on a function or method, typo, etc.), something that's clear a mistake maybe a third of the time and, occasionally, points out something that looked fine to me but actually won't work because of some other code that I forgot about. I'm a person, I can't remember everything I guess!
I have to say on those occasional times where it finds something that I totally missed or misunderstood, those are for sure the most productive sessions. I find I'm actually working with the model, while I read the code it's pointing me at, and getting a good solution together. Often the model suggest something that's maybe too simple or, weirdly way way too complicated but it's definitely helps me zero in on a decent solution.
3. When you hit a bit you don't care about, go back to the model and ask it for the pertinent details
4. When you hit a really confusing bit, ask the model for hypotheses about what's going on. (I always phrase it as "give me some hypotheses" not "what is going on here". I dunno if this changes the output but I think it helps me stay in a mindset of uncertainty, it's important to avoid locking in any misunderstandings. Anyway I find the models do well at this task, and when they bullshit here it has a strong smell).
Before AI, parts 1 and 3 could be insanely time consuming, sometimes it felt like a infinite breadth-first-search. And part 4 was basically: either you find a human who knows the code, or you just make a mental note and hope that later on you find something that makes you go "oh, THAT'S why they <do weird thing that should 100% have a comment>!".
So yeah even though you're still reading code with your wetware the AI makes you dramatically more powerful.
This is also extremely helpful for unpicking undocumented API contracts. E.g. you can say "the x86 implementation of this API is safe to call under a spinlock, go read the other arch versions and tell me if that's true there too".
the devil is always in the details. those details are on every level you look at: human minds, nature around us, space. so if your inputs are vague, you should only expect outputs that are vague and generalized
I’ve found that planning tickets with granular details (like semi specific code changes needed) is one of the best ways to get that deep knowledge of the system. Even if ultimately I delegate most of the implementation to an LLM. I also heavily verify the changes, but I find that that’s less impactful than planning a feature / tickets.
If it performs the overall function but the code is a mess, then you can ask the model to describe parts of it, the structures and what the structures are for. When it shows you something that is messed up, suggest a better framing for the same thing and ask the model to implement the better framing without regressing on already passing tests.
I'm going through this right now on a very difficult to implement task, the original was python and very verbose. But had facilitated a rust implementation that produced byte identical outputs. Then I asked it for what data was being passed around, placing restrictions on what passed between interfaces I could tell it what parts should be immutable and what parts should have no presence outside it's local context. Placing those limitations while having a exemplar of what it should be doing gives it little choice but to make better code if it meets the conditions set while at the same time not regressing.
I say things like 'this field is a implementation detail in a declarative data structure, it should not exist here.
This can mean hours of work with no observable change in program function, yet it is directly addressing the limitations that prevents it from being used in larger tasks.
Yes, how does the LLM make a difference there? Inexperienced developers write bad code. Hell, experienced developers who are outside their domain write bad code too, then you later realize that it's hard to read, or that it leads to architectural problems or such, and hopefully learn from the experience.
> This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise?
Learning from LLM written code is significantly and meaningfully different from struggling on your own or learning from more experienced human co-contributors. Especially in a large, complex, iteratively developed codebase.
Worse, once you're in that situation you are now at the juxtaposition of:
"I did this. I understand what my reasoning was, and now I know why it is wrong and how to fix it" vs. "An LLM did this, I don't know why it did this, I'm not sure what it was trying to do or what pattern it followed and I'm not sure how to make it better because I didn't write or understand the original implementation either"
Is it impossible to learn and gain experience this way? Not at all. But it's definitely not equivalent.
> If we're pushed to delegate most work to them, how do you build that expertise?
Have you ever pair programmed with someone? It's the same idea. You can be an active enough participant in the process if you wish to be and can be just as knowledgeable even if some of that knowledge lies in transactive memory. https://en.wikipedia.org/wiki/Transactive_memory As long as you have the map, and the map to the map, you don't need to retain every fact about the landscape.
This reminded me of Gaussian Processes. You start out with n-dimensional unconstrained (but strongly correlated) gaussians. As soon as constraints (data) are added (mathematically it's called conditioning), the thing goes more and more into shape.
Prompting feels a lot like this conditioning phase to me. You start with an LLM in unconstrained mode, basically just a "soup" of knowledge. If you prompt wisely, you immediately condition the LLM into "your space of (domain) knowledge".
What comes out is an extended version of your existing knowledge.
Prompting is conditioning, that is what it is. The visual of a GP (like the thing you get if you google image search “Gaussian process”) is a great metaphor for what prompting an LLM is doing.
The output of the LLM is the logits which is sampled - plucking out tokens from a distribution. The input of the LLM is data which constrains the logits. That is what it is.
That’s also how you know that AI will never “solve” intelligence (the way the boosters say it will) without some general mechanism for this conditioning process. The ultimate mechanism would be embodiment; the crappy mechanism we have now is something like openCLAW.
I do find that "signalling expertise" is important. "I have a significant background in biblical scholarship. You can assume I've read the most important works in NT studies in particular. Do not translate Greek, Latin, Hebrew, or Syriac. Now, I would like to know . . ." That changes things significantly. So does telling it you have 20+ years of experience with C programming, that you have a robust understanding of machine organization, memory layouts, embedded systems, etc.
For sure. On a personal coding project I said "I'm a professional software engineer, and while this is a hobby project I'm not just vibe-coding and want to build reliable software" and the agent suddenly started suggesting all kinds of things to make its code more robust.
LLMs skew toward over-focusing on things that you mention.
The reason "the agent suddenly started suggesting all kinds of things to make its code more robust" is because you said you "want to build reliable software".
It's not a signal of good judgment or understanding. It's just how LLM attention works.
I thought exactly the same at first. But then i wondered if that still holds true with today's advanced thinking, RLHF involved, frontier models. I guess to a certain extend it did indeed behave better, as a reaction to his self description into account.
EDIT: I mean, those systems accumulated so much complexity around the attention based next token predictor.
Training the LLM to do things that the user didn’t explicitly ask for is a good way to get complaints from the users. Doesn’t matter if those things are best practices.
this feels like "make no mistakes" level of prompting. reliable software isn't as simple as making it reliable, it's about choosing the trade-offs in the areas that don't matter as much as the areas that do. if you keep prompting the LLM to make your software more robust it will keep giving you things to do. they aren't all good things. eventually you'll end up needing kubernetes to run a calculator app.
Not sure I agree with this. The math guy at anthropic's prompts are essentially:
"suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, everything depends on it. think really hard, and try to come up with a bunch of ideas to try. but remember to trust yourself and not necessarily in conventional wisdom!!"
https://claude.ai/share/25740bd5-aa97-4bd7-bf58-c4df3793fda7
https://xcancel.com/__alpoge__/status/2083855298239078748
Tao's chat was for him to gain intuition, not to solve the problem from the outset.
What's funny is that every other person gets a different conclusion about who these models reward/empower. I've seen people say that the generalist stands to gain the most and others say that it's the experts. Like all of life, maybe the "winner" is the person who just does stuff.
It depends on the levels. People with differing fitness levels and ages run at very different paces. Now, do cars make them more equal or less? On the bottom end, the tide lifts all boats. Most healthy people can learn to drive and will drive "fine", they get from A to B. Out there in the city streets the car flattens the differences, everyone roughly takes the same time to get from A to B in a car.
But at the top of top, the gap probably widens. A professional F1 driver will drive laps around some random guy. It amplifies reflexes etc, because at that speed little differences in timing make a big difference.
Now, AI coding isn't exactly analogous, but I think it also has these two regimes. It flattens things for simple tasks. If your task is to shovel data, do some trivial compiler wrangling staring at badly designed error messages, looking through GitHub issues hunting for the comment with many tadaa emojis to fix an issue etc, those things can now be done by anyone. Just as grandpa can also drive to the grocery store. But if you're pushing at things on a higher level, now only your above-AI ability matters. If all the things that AI can do well are subtracted out, how much other expertise do you have left? This will be proportionally a bigger and bigger difference between different people.
So you're saying that it depends whether you're measuring "success at task X" (where in average everyone improves) vs comparative success (where people with knowledge can do far and away better at specific tasks).
Exactly. The post read to me as another variation of the denial that people with expertise are reaching for right now. My sense is that we as programmers went through it over a year ago already (perhaps not all of us, but at least anyone paying attention), and so it's easy to overlook that it's still new to people who do other forms of "knowledge work," i.e. people whose identity is bound up with their expertise.
My working theory at the moment is that for programmers it was relatively "clean" and took the form of an inside-out transformation of the work, where AIs directly produced the central work product more or less adequately and relatively early on, but for other forms of work it will appear as some mixture of inside-out (in which case it will appear similarly first as a tool, then as something more than mere tool) and outside-in (the things surrounding their work and the supports their work processes rely on will be progressively automated). This is going to give rise to all sorts of pathologies in the white collar world, we'll get all kinds of variations on denial/negotiation, and so on, until it fully transforms the division of labor.
One interesting point of reference here: Yuval Harari gave a talk recently about the radical changes that will take place relatively quickly, in which he noted the AIs are not quite as good at writing as he is yet, although he expects they will be relatively soon. He then gave the timeline for what he considered "soon": 10 years! So we find the denial ("I still have time, they're not as good as me yet, maybe in 10 years...") even among the most vocal "prophets," among those supposedly most wised-up to what's going on and where the capability frontier lies.
Have you considered the possibility that you're simply not as good as the experts, and that your experience of LLMs being capable of performing your work up to your standards doesn't imply that experts are necessarily in denial?
Being fairly at expert level in a "solved" domain (for some definition of solved) but also having near-expert level proficiency in a non-technical as-yet "unsolved" domain and watching the process repeat there (and watching how people react as inroads are made progressively deeper) is basically my own standpoint. But I am intentionally using "solved" and "unsolved" very loosely here: you can get stuck in the thicket of arguments about verifiable domains, what it means for a domain to be solved, whether a set of evals can tell us something has been solved or not, and so on. One way to avoid that (as my original comment pointed to) and focus on what matters for us is to look instead at the effects being produced in the work process and on the division of labor as a whole.
He's saying: "Expertise improves AI-assisted work, experts can steer better and get more out of models."
The obvious conclusion for anyone is "therefore experts will remain the indispensable and specially rewarded center of the production process."
That this is appearing exactly now, and in this form, strikes me as extremely suspicious. I don't doubt the author's sincerity on the surface. What I suspect is that anxiety over the possibility that the (unstated) conclusion might be false (!) motivates the argument in the first place.
I'm asking the question, "Why is this argument appearing now?" At least one reason seems to me to be, "because we're afraid of what the world could look like if it's not true."
However, I personally agree with the author and I don't think his argument is necessarily motivated out of an anxious fear. On the contrary I think it may be motivated out of a sense of extreme exhilaration and empowerment.
Because experts (like myself as a programmer for 15+ years) who are using AI in many fields are suddenly empowered and much more useful than we were before AI. My employability and value has gone up and not down, precisely because of being able to apply my expertise with AI, which people without expertise simply cannot do. I am a professional programmer and also owner of my own startup.
Let me give you a concrete example that I am dealing with at my startup. I'm a small business owner. Before AI if i wanted to produce production quality video for marketing it would taken such a huge budget and such a large team of people (or an expensive agency) that I wouldn't even have considered it due to the enormous cost. I'm talking about Apple quality video production which takes millions of dollars to produce.
Not anymore. A single competent person with AI can replace an entire marketing video production department or agency. But expertise is key here: knowledge of film terminology to be able to describe the effect you want, and ability to use video editing tools effectively, as well aesthetic taste. I as a programmer with no filmmaking experience don't even know how to write the prompt which makes the video that i want because I don't even have the terminology. But a person with that expertise has suddenly become more employable and more valuable to my business because I as a small business now have the capability to create Apple quality marketing videos.
So AI actually created a new job for an expert that would have otherwise not existed because it was outside the budget of small businesses. Previously somebody like that would have been employable to only a few large production studios but now they become employable by almost any small business.
Did you maybe consider that the 10 year timeline is not denial, but actually well educated reasoning based on Yuval’s experience and understanding of the problem space? You shouldn’t dismiss people’s thoughts as denial just because they don’t match your perspective.
I would guess if asked Harari would actually revise that lower. The point was: even those arguing most strongly that AI is an autonomous historical force can slide back into the mere abstract recognition of it. In context, Harari was saying "I'm speaking to you as a writer now, but my own standpoint will eventually be undermined." I'm saying "yes, and your 10-year timeline suggests you along with all of us are not taking your own thesis seriously enough."
This works better for math because math is self-verifiable. Once you have a proof it needs no outside evidence.
Expertise is needed to evaluate model outputs where it can't verify itself, or at the very least one's expertise can help steer the model in the right direction.
However this is irrelevant if models themselves are better at evaluating/leveraging expertise/information.
Corollary to this is an important part of LLM usage is what I call pinning it to reality. That is, designing verification steps that interact with the real world in some way not easy to hallucinate or work around. This means things like having code that interacts with the physical world, round trip tests, arriving at the same result using different paths, interoperability / replication with external libraries / competing products, performance improvement projects that start with robust performance test suites, and similar sorts of things that reduce to "how do I provide evidence that's difficult to fool myself about".
This includes things like "before you start fixing this bug, write two tests that fail proving it exists".
Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.
But the guy who writes the “just do it” prompt can neither formulate the conjecture in the first place, nor come up with any follow-up questions to build on the result.
Yeah, the people who say no expertise is needed for these things confuse me somewhat. This is indeed the case if you want to be a meat wrapper around an LLM, understanding neither your inputs nor your outputs. But at that point, what is the point of you versus going to the LLM myself? Expertise is necessary because it adds understanding and structure to the blob of text produced by an LLM. Progress can only be built on such understanding.
I am tempted to say (uncharitably) that the 'No knowledge needed! Just add LLMs!' byline is wishful thinking by non-experts who do not want to confront the reality that they will ultimately need to learn things.
> But at that point, what is the point of you versus going to the LLM myself?
reply
You're missing one word, and that word explains why everyone running the companies is so excited. The word is 'paying'. "But at that point, what is the point paying of you versus going to the LLM myself?"
And, yes, I think that LLMs make it a lot easier to hire a minimally trained stooge and get them productive. It's worked for me, and I appreciate being able to pretend to be productive and walk away from the job a bit early every day. I don't think software engineering is likely to be a high status, high income job for very long.
It's actually refreshing when a model is sure about something because it actually tested it and has the receipts. Opus 5 seems really good about testing its own knowledge with experiments. Scientific method ftw.
There was one math proof that was AI generated going around Twitter and the chat transcript to generate it was basically the human prompting "keep going" until it solved it.
Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.
Yup I linked that in my other comment but copy-pasted here for everyone's convenience:
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample"
Yes, we're in the low hanging fruit stage where folks can just say "here's the problem" and "keep going" until a result is found and it will work sometimes.
The low hanging fruit will run short. Ultimately mathematics is a field of subjective selections of problems and proofs as beautiful and interesting. Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof.
> Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof
And why do you think this would be the case? I'm not talking about today but in 1-2 years. For reference o1 was released less than 2 years ago, and we've had reasonable coding agents for 9 months or so.
Mathematics is ultimately an aesthetic pursuit. Outside of a well defined goal ML models don't have any sense of taste and regardless of the scaling that's been enabled in the last year or so of capability if they haven't memorized the process of doing something they have the same limitations of inability to make choices about unknowns not trained into them.
Real synthetic intelligence seems to me to be still very far away and not a matter of making models bigger or more efficient.
LLMs are a collection of biases. Humans are also a collection of biases. So we project our biases as input through the biases of an LLM and get an output. Hence why I think getting optimal output requires being an optimal person. And in that sentence there are many points of expression.
Finally, we train our LLMs on who we are. Another reinforcement of biases.
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample" lol
I am not sure. Why do you need domain expertise beyond being able to draft a verifier for the problem? Once you have the verifier, it's just a matter of compute. You can argue that being a domain expert allows you to narrow down the search space for the LLM and save time/compute costs. This is partially true, but LLMs are getting better and better at search (they can already do end-to-end performance optimization faster than performance experts at a FAANG company I work at), and compute to maintain the same intelligence level is getting cheaper.
A concrete example: GPU performance optimization for a kernel. This was (and still is) a very niche domain with not many top-notch experts. But kernel performance and characteristics are easily verifiable. You can run the agent in a closed loop for it to improve iteratively (and people are already doing it, coming up with kernels better than human-written ones).
You see Tao's example because:
1. He is curious (so he asks detailed questions, which are not necessarily needed in a closed-loop optimization).
2. Verification in math is harder. Many math tasks used in RL are easily verifiable. But for advanced open conjectures that require long proofs, you cannot trust the proof directly from the LLM (so it's not as easily verifiable as basic math problems or code). The model needs to write it in Lean, and you still need to make sure the Lean implementation correctly captures the specification of the problem. So you still need a human for verification in advanced math. But I don't see why you would need this in domains like performance improvement.
The short version I give to non-technical people who ask me about whether "AI will replace coding" is this: it accelerates you. You can get much further much more quickly.
If you don't know where you're going or how to get there, or even if you're just not paying enough attention, it will get you very far in the wrong direction before you've realised.
> The model outputs are much more concise than when I try and talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians” mode, not “explaining-to-amateurs” mode
I believe this works in two different ways.
First, information compression. The use of professional language helps describe problems more densely with minimal information loss/distortions. Verbose output by LLMs (e.g. ELI5) tend to incorporate local chat context, which can destabilize the context (e.g. out-of-topic, irrelevant nitpicking on writing style and wordings) and lead to faulty logic and even hallucination. LLMs are not good enough to look through all the noise, so, sometimes, it's helpful to refine the input data before performing actual tasks.
Second, boosting logical pattern-matching. Using professional language helps drive logical reasoning through simpler pattern-matching b/w texts. This is not about whether LLMs can reason or not; it's about how high-level reasoning is guided by preconception. Even humans tend to consume only textual surface of highly complicated theories (e.g. Adam Smith's "invisible hand"), and use them casually during conversation. It's similar for LLMs: if the conversation is conducted entirely in professional language, LLMs can easily incorporate external professional information into its reasoning. If the text is written in amateurish tongue, translating it into professional language can introduce errors and distortions.
So, yeah, keep your conversation professional, tidy and tight. A large volume of unprofessional text helps no one.
This is why the chat interface is ultimately not the best option for non-expert users, because they require the user to bring knowledge with them. You can call this the “query” method: you have to know what to ask to get the answer you want.
A real world example might be: I can find any movie DVD you want from our warehouse, but you need to tell me the name of it. Don’t know the name? Tough luck.
Contrast this with a “browse” interface: the options available are presented to you, and you can pick from them. Relevant contextual information is already on-site. The DVD store has shelves of potential movies you can rent, and you don’t need to know their names ahead of time.
The interfaces of future AI will be more browse oriented, with a query viewer available in the settings for advanced users.
Picking a DVD to watch is a rather inconsequential decision. LLMs already do this sometimes, asking you to pick one of a few options, but without domain expertise you will invariably make worse decisions, but if all the n-th order consequences were first explained to you, that would result in you having built domain expertise, but also erasing most of the speed advantage LLMs give you. Moreover, you will never know about the options that are never presented.
Inevitably, this is the new tradeoff to make, above average quality comes from asking for more, and knowing what to ask for comes from expertise.
> In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet.
You could read some general reference/guide/tutorial documentation on CSS, and then probably solve your problem (without searching for "how to center a div", or whatever your exact problem was, and copy&pasting the answer and moving on), also becoming more knowledgeable in the process.
The rest of the short blog post has some good points, but the first sentence sounds like it's targeted at the percentage of developers who did StackOverflow copy&paste to close Jira tickets, never becoming experts.
Delegating to LLM-ish AI is just a natural evolution of that. The question is whether they can still add value if kept in the loop.
The article author suggests that the answer is to be expert, and is addressing people who... "either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet."
> You could read some general reference/guide/tutorial documentation on CSS, and then probably solve your problem
Hours + Hours of reading and a lot of trial-and-error. The loop was so long and sooo slow. Now it's instant. As if your very first Google search just solved the problem for you immediately.
I lost my desire to learn anything about web development twenty years ago and specialized my career in a completely different direction. I had not touched HTML at all since then until earlier this year, when I discovered that AI robots could do it for me. Now I am happy to whip up HTML/CSS/JavaScript visualizers, explorers, and even one-off interactive report presentations whenever they might be useful, precisely because I don't have to learn any of that crap to do it. My time & attention are far more productively spent focused on the work I am actually good at and interested in doing.
There was some learning, but also a lot of waste. As a self-learner, I've never been able to learn so fast as I can now with an LLM to instantly answer my specific questions, and incrementally correct and grow my mental model. And while with some subjects you need to worry about accuracy, LLMs are generally very good with programming questions, and (for most types of questions) you can verify their claims yourself.
The problem with CSS is it's a) all broad and no depth and b) constantly adding major new paradigms. If you don't get up to speed and stay there, you lose it.
Pre-AI I'd say I've gotten over the hump 3 times with CSS, only to lose it again by the next time I had to use it in volume.
It's inefficient though. With LLMs you can get a correct answer instantly for asking the right question, and if it's not perfect you can iterate on it. With your typical googling loop you wasted a ton of time on things you didn't need.
Why do you think thats worth learning? Doing a few times manually sure… but to force yourself to solely rely on it is foolish. Just use the calculator.
I am a frontend engineer and I see so many bugs from vibe-coded CSS from people who think just like you. CSS is very powerful and easy to get wrong if you don't understand it. Since most CSS code on the internet is not good, LLMs do not generate good CSS code and create a lot of bugs ranging from bad performance to completely broken reflow.
The people who make the tools that generate your assembly instructions need to learn it. Just like the people who make the browser rendering engine and push CSS forward still need to learn it.
The people who don't need to learn asm never needed to learn it. If you wouldn't code in asm now, you wouldn't have ever.
I just want to point out the exception - the person who the AI learns from.
Someone has to know it. And to advance it to the next level requires domain level expertise. Or are we allowing LLM's to determine the next CSS snapshot/version/level/whatever?
How familiar? Do they ever look at the .S files? know what ar and nm do? Deal with objdump? How important are those details for day-to-day compiling of C code for regular programmers?
Yes, they know what ar and nm do. Whether these things are relevant for day-to-day compiling of C code is watching the goalposts sail over the horizon; the original statement being mocked here is "we don't need to learn ASM because GCC is so good", which completely betrays ignorance of why people learn still learn and deal with ASM to this day. Even if LLMs were comparable to compilers either in predictability (they're not) or code quality (they're not), the people who know what they're doing still take pains to understand how their tech stacks work, because that makes them better programmers, and this will continue to be true.
Not sure what your point is here since wrong code doesn't mean non-deterministic. How many of these yield different results with the same input and architecture?
Honestly, we don't really need to learn any more. These days, I mostly copy and paste from tickets to the LLM, and then rubber stamp the output. I have people in my team that try to care about what they're making, but they're generating less code, and filling tickets more slowly, than I am. Even when it comes to system design and architecture, the LLMs aren't great at it, but honestly, they're better than my coworkers (I'd say system architecture is probably at the same readiness level that coding was last year). Expertise wouldn't really help you get things done faster.
You could probably replace me with a minimum wage worker to do some manual testing and copy-paste errors from the console into the LLM, and still be fine.
Let's see how long it is before the next round of layoffs, I guess. For now, the money's fine and the work's boring but ok.
But, no, the LLMs rewarding expertise line is pure cope. Software is not really skilled labor any more.
(obviously, throwaway -- I'm not interested in potentially advertising this to my employer)
> Honestly, we don't really need to learn any more.
Perhaps for your task of closing Jira tickets that’s true - but the you have a decision to make: wait until you’re eliminated or try to anticipate where future competitiveness comes from.
Yeah, that's the thing; future competitiveness isn't going to come from learning, at least not from learning about software. The LLMs have that covered, and software is becoming low status work anyways.
It's largely going to come from interpersonal skills and being likeable enough to be the face of the projects and companies. Technical skill is rapidly dropping in value.
All? obviously not, but enough that I can ship more than my colleagues while not really paying attention to the output or fully understanding it. Technical skills aren't unnecessary, they're just becoming something that nobody will pay for.
LLMs debug extremely well, generate code well enough, and do high level design acceptably, but fail in weird ways, so my actual work these days (outside of delegating to the LLM) is mostly manual testing. I need to give them nudges sometimes, but a year ago they needed constant nudges. In another year, I expect that it'll be good enough that pasting in a bug report with the right jargon will be all the nudging I need to get something that customers will accept if it gets shipped. They're already close.
It's obviously going to remain pretty manual without some breakthroughs, but it's largely not work that needs a ton of intelligence or skill. The world hasn't caught up with software development being a few steps up from burger flipping, so I'm still getting paid well to do it, and I'll keep collecting that paycheck as long as it lasts. But learning about code is rapidly becoming relatively valueless.
I wish it was different, I actually enjoyed getting paid to learn, but I play with the cards I'm dealt.
Yes, but sometimes I don't need and want to learn. One example from my recent experience in research -- building custom dashboard pages for results of scientific analyses. Each analysis is bespoke, and building interactive webpages is simply not the skill many researchers have (and it's boring IMO). But here with LLM you could easily explore the results visually/share them with collaborators etc. There are plenty examples like that.
But certainly there are cases where learning is required.
All the boring stuff you skipped to get right to the answer you want was just as important whether you want to admit it or not.
Having to read through a structured resource describing something to figure something out has intrinsic value that an LLM is not going to provide you with.
> All the boring stuff you skipped to get right to the answer you want was just as important
This is obviously not the case. There are mountains and mountains of boring, unimportant stuff that LLMs can do for us now.
For example, at work I can now make a nestJS dashboard without having any idea what nestJS is. I can just tell the LLM what I want, and it makes it so. This allows me to make my tool useful to people without having to become an expert on something unimportant.
Yes? My point is that for what I do - low level systems and embedded stuff - nestJS is really not something that I need or want to know about. I just want a good dashboard. Now thanks to LLMs, I can focus on the fun and interesting stuff.
You seem to be hung up on the part about "learning". Do you know how x86 registers work? Or atomic instructions available on ARM, SPARC, or POWER architectures?
No, of course not. Because all of that got abstracted to higher-level instructions decades ago.
> Do you know how x86 registers work? Or atomic instructions available on ARM, SPARC, or POWER architectures?
Yeah, I have a pretty good idea of how we go from code -> IR registers -> x86 registers and how the underlying hardware has more registers than that, etc.
As for SPARC and POWER, I have not yet gotten my hands on one! But as someone who writes compilers for fun I did learn about most of that, yes.
I write C# in my dayjob, and having written high-ish performance C++ and Rust for a couple years before that, and because of all the curiosity-driven projects I've done (like writing useless compilers), I did learn a lot.
This has helped me massively in my job, especially when optimizing our C# sensor data ingestion backend to process hundreds of thousands of measurements per seconds with explicit backpressure etc. it was great fun because I know what I'm doing.
I'm not trying to "uhm actually" you, but when you call yourself a software engineer, I do expect that you know some of these things after some years in the industry. The only way you wouldn't is a lack of curiosity, or because you genuinely had no idea that you CAN learn these things.
Well that's great for you. That's some knowledge that you will possess for the rest of your life.
I want a green lawn and big bushy shrubs in front of my house. Do I need to know the intricate biology of my soil and habitat? No, of course not. I just do the surface-level things that make the lawn and shrubs thrive.
>No, of course not. I just do the surface-level things that make the lawn and shrubs thrive.
And others want the green lawn and big bushy shrubs, but don't want to learn all of the surface level things to make their lawn thrive so they hire a service to do it for them. And there's nothing wrong with that - not everyone enjoys the yard work, but they still want the thriving lawn and shrubs... and they are happy to write a check to OpenAI... err...Lawn Doctor every month to get that result since the time they save by not dealing with their lawn, they can pursue things they do want to do.
Not everyone wants to (or needs to) learn every detail along the way of getting the results they want.
I understand your point, but also imagine applying that across the whole spectrum of computing.
It's very useful that there's an intermediary that knows how that stuff works so I can build things without thinking about it in excruciating detail the whole time, I can dip down lower and learn stuff when it's relevant (like cache access and nested arrays) but I can also not do that in many situations.
There's also the argument that you can do engineering without understanding the underlying science as seen in th pyramids, the beautiful old european churches, etc
More productive? If the result can be trash, of course the AI work wins. In industries where quality and performance BOTH need to be upheld, knowing these details lets you guide the LLMs in a way that they will simply never do by themselves, and lets you meaningfully review performance critical code.
This reduces the cycle time on issues and reduces bugs and performance regressions. If I pitched that last sentence to your boss, chances are he'd take it.
But maybe there's a limit to the usefulness of this abstraction. Perhaps forgetting too many basic skills lower down isn't always as useful as going from Assembly to C. I'd argue that there's a law of diminishing returns there, and it's not an endless series of benefits.
So, since you care so much and caring requires knowing everything about everything that even remotely intersects with your project, as (presumably) a software developer presumably you understand in detail how every bit of the CPUs you're targeting, plus the lithography, chemistry, and how solid state physics, and the underlying quantum mechanics work?
You seem to be massively exaggerating here for effect.
I think the main point is, if you're building frontend web apps, you should probably know how HTML, CSS, and JavaScript interact to some decently high level. Can you look stuff up on the margins via AI, or StackOverflow, or whatever? Sure.
If you're building <something else>, you should probably know the core tools and concepts at the appropriate level to build them.
No we do not need to know every aspect of CPU branch prediction and whatnot to make a webpage.
I think it's pretty obvious from the context of "if you had technical gaps (say, you couldn’t write CSS)" that this isn't about someone that regularly works on frontend web apps, it's about someone that is having to do something that is not normal for them. E.g. I mainly work on HPC code, maybe once in a year or two I need to throw together an animated poster to accompany a printed one. It's obviously ridiculous to claim that this means I have to learn frontend web dev.
I think you're talking about a different type of expertise from TFA. Consider this: What if I never enjoyed frontend programming and so I never wanted to be an expert on that?
In fact, I never enjoyed frontend programming because it was such a pain to deal with matters I considered trivial yet so frustratingly hard to do right... like centering a div. And yet the slightest misalignment is visually jarring and forces me to get a bit OCD about fixing it, which made it even more frustrating.
I questioned the whole premise of the situation: is working around a bad developer experience something worth spending my time on? Unless I actively wanted to get in there and fix the situation, not really. So yes, in those cases I would outsource my problem to a colleague or StackOverflow and move on. And as a career choice, I preferred to do more backend dev.
I would posit that that was the type of expertise that did not matter. The type of expertise that really matters here is good UI design. That is entirely orthogonal to the drudgery that is implementing and debugging webpage rendering, and I am eternally grateful to LLMs for freeing us from it.
You can extend that line of thought to the entire article. What really matters (and what LLMs reward) is domain expertise rather than technical expertise.
I don't think AI use is supposed to replace foundational learning such as reading a C++ book or Python book or CSS tutorial when you're a beginner. You still have to do those things if you want to be a professional or a strong amateur. But many people just want to get the thing done. They don't want to become a mechanic, they just want to drive from A to B.
> They don't want to become a mechanic, they just want to drive from A to B.
I'm fairly certain the article is directed at professionals, or at least the AI companies are basing their valuations off of directly taking a slice of that professional "productivity".
I've seen junior engineers be productive on their first day in the industry because of AI, so I don't think the article is the whole truth.
The example math is boundary-pushing and definitely not a solved problem. But most of us work on CRUD backends with a React frontend. Those are more or less solved problems that have well-documented solutions. For those kinds of tasks, LLMs just reward usage.
I can count on one hand the number of times in my career I've needed to solve a problem that's not described on Stack Overflow.
Except LLMs will only tend to share the most common or average of what it knows as the standard and deviating from it (including new ways) it can be resistant to.
An expert can lay a different kind of frame to prevent the llm to fell out of its way of being generally too verbose, and that can transfer as well to code generation and complication.
I think this is extremely true when it comes to prompting, but not only in this way. I would add that this also applies to an LLM's output on deep enough topics.
Anyone can point at a public GitHub repo and have an LLM write the documentation for it. Whether that documentation is good requires understanding that codebase.
There's no way around expertise unless we're talking about strictly mechanical tasks.
I do think LLMs are incredible at helping to build your expertise. You could point it at a codebase and say: "Explain how this API works" and interrogating the LLM until you get an explanation at exactly your level of understanding.
I don’t understand why this is such a revelation. Anyone who has listened to a good/great interview knows the skill of the interviewer plays a big part. To ask good questions, to understand what the other person is saying (AI or human) - that requires skill, expertise and patience.
Someone with less skill or expertise might still get good results, sure. It would just take longer and it wouldn’t be pretty
There's a growing and fascinating divide between people who see LLMs as more of a "bicycle for the mind" in the vein of Jobs vs those who see them as whollly supplanting the role of human intelligence. I can't help but wonder if these aren't primarily two human archetypes more than anything - the LLMs can be both and they erect a mirror of the human using them. Some humans really don't want deep individual expertise and intelligence to be the deciding factor because they don't identify with that. Others are completely the opposite. We really can't tell which will be more effective yet, because LLMs are very good in both modes. But most of the predictions currently are people executing on wishful thinking about what they hope will be the outcome.
I've been equating them to graphing calculators since the first LLM launched. It's an amazing tool if you know how to use it. If you don't know how to use it, it's still a tool, but you won't be doing anything amazing with it.
maybe outing myself as a dinosaur, but "back in my day" the calculator came with a book that detailed exactly how to use it. Both the high level basic language and the low level system language. Not knowing how to use it is simply a failure to Read The Fucking Manual.
You can read the manual all you want, but if you don't know basic algebra, trig, calculus, etc, you are not going to have any idea how to apply or use much of anything that the manual describes with regards to actually doing math with a graphing calculator. There is a base level of knowledge required.
It was a general purpose computer, just a small one. Anything you could do on a "real" computer could be done on a calculator, albeit with tighter constraints. It might help to have some higher math objective to accomplish, because that would better utilize the preloaded system software. But in terms of the hardware? Probably not super relevant.
I made a lot of use of the TI-89 era CAS in college. But IMO the TI-83 era manuals taught me more about both math and computers than the subsequent generations could have.
I totally see this. I just did 3 hours of bot sitting to put together some thrash loops that thrash our provisioning working flow for a BLE gadget we make. It was pretty straightforward and productive. But then, I have a lot of experience with BLE, and a quite a bit of experience with python and shell scripting. So I was able to guide the process through stages, do some intermediate testing, make some adjustments, and proceed. Domain experience made this really easy and straightforward. Me two junior engineers who have only superficial/high level knowledge of BLE and some of the other pieces, couldn't have done this as effectively.
Where my angst comes, is worrying that no one will ever get that experience anymore. They might have had some eventual success, who knows what monstrosity a much less guided LLM would have done, but experential learning may be mostly a thing of the past. And it creates a real tension between the person with experience and the person without.
>Where my angst comes, is worrying that no one will ever get that experience anymore.
I am a fairly inexperienced python developer using LLMs to build software and find that I still learn a lot just from prompting and tinkering. Maybe that's less true once you reach a certain level of competence, but at my intermediate level I am still learning a lot even leaning heavily on LLMs.
The LLM is like the death star. If you don't know exactly where to point it, you will likely miss your target and have no/negative effect. The further away the target, the more accurate your firing solution needs to be. If all you need to do is add something like a dark mode theme to an existing product, this is probably a point blank shot in this metaphor. Building an entire codebase from zero, or even refactoring a legacy codebase into a new codebase, are lightyears away by comparison. You can still land the shot, but you need to deeply understand the metrology and astrodynamics.
The information system required to encode the aesthetic preferences needed to make a technology experience not suck is likely in excess of what any near-term solution will offer. Knowing when to say "no" is perhaps the most important skill here. You can't just say it arbitrarily either. You really have to mean it and be willing to fight other humans for it.
That's why when people like Pieter Levels tweet "I cancelled and then vibecoded 100% of my SaaS subscriptions", you need to take it with a huge grain of salt because you're not Pieter Levels, you cannot vibe code your SaaS subscriptions.
I haven’t experienced this - or maybe the training data for aviation is limited. But ask a an llm for aircraft performance / flight planning data and it’s scary how bad the advice /feed back is.
For anything where "the answer" wasn't already in the training data you just get some arbitrary correlated tokens out, as that's all a LLM can do. Of course the meaning of these tokens is just random.
(And even for things that were in the training data you don't have any guaranty they will be reproduced correctly, there is just some chance something meaningful comes out, or it doesn't, it's random.)
The whole idea to use a next token predictor as "answer machine" is completely flawed. This can't work like advertised, and that's by construction.
Most Hacker News are working on simple JavaScript applications. LLMs quickly become unusable on any sort of specialist discipline except for the math marketing releases we’ve seen recently.
This matches my experience. Just Talk To It is the best method for working with LLMs if you're an expert.
I've seen this at work (as eng manager/lead/principal/whoevenknowsanymore) – all the big APIs give you stats. We see how much people burn in tokens and we know how much output they produce. There is a pretty strong inverse correlation between token burn and output.
The more tokens people burn, the less likely they are to produce a good outcome.
LLMs are language models. So much of this can be reduced to a simple heuristic: If you can't think clearly, LLMs will not help you. If you don't know what you're asking for or how to express it precisely, it really should not be a surprise that the output is garbage. Failures of LLMs are more often failures of our own brain to consider the problem clearly. That's harder to admit than just blaming the AI.
It's important to note though, from the perspective of the LLM's objective function, that this is not a failure at all! LLMs are designed to match patterns. You give a jumbled mess of incoherent ideas, it will faithfully reproduce a token stream of incoherent ideas. It's only when the model output is subjected to the real world that it fails.
The more cracked you are in a domain, the more you can squeeze out of an LLM. If you already know what "good" looks like, you can steer it, call out its BS, and iterate way faster than someone who's using it to learn the domain itself.
The interesting thing is, most messages were ending with just one question of his. Examples:
- ..Does this polynomial map have any symmetry or other structure that makes this cancelation less miraculous?
- Given this structure can you see the non injectivity in a transparent way?
- ..But why is the jacobian from x u r to P Q R just a monomial?
- ..Is there a general theory of such twisted jacobians and do you have any sense why those particular dilation weights were used?
- Given this weight structure, why exactly is x given by a cubic equation from P,Q,R?
Also, looks like Terence Tao was doing lot of work and asking LLM to verify. This is inverse of the LLM trend, where LLM does the work and humans verify.
This has been my experience. I’ve been working on an app (highly successfully) since February, with the help of an LLM (ChatGPT). It has not been used as an author. Rather, it’s been a “coding partner.” I’ve been the one that has submitted the work to VCS, and run the tests.
I’ve found the most utility in having it do “small stuff that I could do, myself, but it’s faster to have the LLM do it,” and in analyzing intractable bugs, like memory and threading problems. It’s really good at analyzing a bunch of code, and seeing a small typo that results in something like a strong reference.
In both these cases, my own expertise is vital. I’m asking it to act as a consultant; to give me advice and material to be integrated into a whole that I am architecting.
I guess part of it, is that I haven’t been able to completely “give in,” and wholly trust the LLM, like I hear many people do (profitably, I guess). I’m used to having my sleeves rolled up, and my hands in the dough. Catching some pretty severe mistakes, from time to time, has reinforced this perception, on my part. I wouldn’t catch these, if I didn’t know what I was doing.
I am feeling this a lot lately. Getting the most out of agents seems to require being able to ask the right question. And how can you ask the right questions without deep domain expertise?
Yes sometimes it’s a matter of just using the right word. You can talk to an agent about a general concept for hours and hours and it may never mention $Concept_X, but you mention $Keyword_Y and all of a sudden the AI is going on about how $Concept_X is foundational to understanding the whole thing.
I started developing webapps back in the late 90s when I was in high school using Perl, and I've worked with tons of technologies up till around 2014 or so when I shifted into almost pure backend work and lost touch with modern frontend development.
I'm now learning how modern frontend development is done (for both personal and professional projects), so I may not know the specific tools, technologies, or terms but I can say "whatever the equivalent of XYZ is" and the models will translate for me. If I say "run pytype" it will tell me "we're using mypy - i'll run that checker for you".
If you can express what problem you're trying to solve, that will get you most of the way - and then you can refine by asking questions. "I think I need something like Redis for caching things - do people still use that? Is there a simpler more modern version that is the new standard? Do we already have company docs suggesting what to use?"
A good moment to remind everyone that if we took the promise for granted, that AI will in fact prevail and prompting is the one skill that will rule them all... we'll lose all domain experts in one generation.
It's less of "signaling expertise" and more about actually having said "expertise".
In my experience with LLMs it's not uncommon to be having a deep conversation about making pasta, only to be told, after asking for a sample recipe, to get a bucket of paint and a bag of concrete. Of course these hallucinations are way more subtle and easy to miss for someone who doesn't have deep domain knowledge.
Sometimes after a larger source draft with AI before I even touch the program I start to pick up on some of the holes in my prompts caused bugs/unintentional mechanics to be potentially woven in. Knowing what you want and how it should be made is half of it, but if you don't supply the bot with extra guardrails eg. don't modify the contents of x, because of y don't allow z. etc. They will do what you ask of them, usually less than you'd hope.
When I use Claude to write code for my own projects, the code it generates is exactly the same code that I would have written had I done it by hand. If there is any deviation, I ask it to adjust but that is rare. At least 95% of the time, it's like it read my mind... Which is quite impressive when it outputs like 500+ lines from a single prompt and then it works straight away without any debugging necessary.
I don't even debug anymore on those projects. If Claude tries to add debugging logic in my code, I tell it not to and just provide additional information and it can usually find the solution faster that way.
This is when working on my own projects. When working on projects created by other people, it's a different story and I have to fight it constantly to stop it from implementing hacks and workarounds... It uses much more tokens to implement basic features. It's more work for both the AI agent and myself.
The project's existing code makes up most of the context so if the code is not great, you have to write long detailed prompts to set it on the right path. You have to make it clear that the existing code isn't good enough and your expectation is higher.
In this case, it usually gets better with more back-and-forth... At the beginning, it can't do anything because you keep pointing out a problem whenever it tries anything at all, but eventually, after a lot of criticism, it starts becoming more careful and adapting to your standards.
So yeah, even same person doing the prompting can lead to two very different experiences depending on who built the foundation.
So my conclusion is that the expertise comes from both the existing codebase and from the person doing the prompting... And TBH, I would say the codebase/foundation carries more weight than the person doing the prompting.
Pretty sure I could put an idiot on one of my codebases with Claude Code and they'd do a decent job.
At work we call this implicit steering. To use webdev metaphor, if a non-technical person describes making a web page with a big block at the top and some things to click on and then my pictures below that, that will eventually get somewhere. Meanwhile, if you know industry jargon, you might describe a hero, with call to action buttons, and then below a 3x3 grid of images of my portfolio photos—that's likely going to generate something entirely different and likely richer. It can assume things about you (it doesn't think), it can ask you specific questions a web personal might know, it can infer domain context that is otherwise omitted with a basic conversation. Everyone wants to capitalize on corporate vibe coding but the tech literacy is hardly there, let alone more advanced topics.
Opus, assume i know nothing about web development. if i wanted to design a new webpage, with a good design, what are some of the terms of art, some best practices? Like if i wanted a big block at the top, some things to click on and some pictures below that, is there terminology for that?
>Yes. Nearly everything you described has a standard name. Here is the vocabulary, organized by what part of the page it describes...
Goes on to identify Header, Navbar, Stucky header, hamburger menu, hero, CTA, Above the fold etc.
>So your described page is: header/nav -> hero with CTA -> card grid -> footer. That is the single most common landing page structure in existence, and that is fine. Being conventional is a feature, not a failure.
I've had the same conversation with an electrician wiring a car charger: we are more likely to succeed if I use his terminology.
That first sentence already uses a ton of jargon that non-developers don't use, "web development", "new webpage", "good design", "best practices", "big block",
2. AI starts being able to do something, but one needs to prompt it carefully, so one needs to be an expert, see, we will always need human experts <--- this article is here
3. AI just one-shots it.
Why do people still need to say this for each and every task? It's just reliving the bitter lesson over and over again.
Now explain how someone with no expertise would even know about the Dinitz conjecture, or even a conjecture at all for that matter, and how that would be useful to them.
This why those RL env startups are able to charge frontier labs so much for their work. LLMs still generalize poorly outside of self-verifiable tasks like coding and math.
Labs have to compensate with post-training in RL env that embeds these expertise well, which is non-trivial both in terms of domain knowledge and technical expertise.
The actual "prompting trick" that dramatically improves your results is often to add just two or three words, like "use library foo", "write in <language>", "<bar> algorithm". To know which two or three words apply in your situation, you need a deep understanding of both the problem and the solution space.
Your prompt might look almost the same as the one from somebody with a good understanding of the requirements but no technical competency, plus maybe one or two sentences. Those one or two sentences dramatically change the results, and what those sentences are differs from prompt to prompt.
I consider myself senior engineer. When talking with junior colleagues they often are surprised how little I care about some things and how much I care about others.
These internal "attention weights" are highly influential parameters of how I work with LLM.
E.g. when working with Rust I often hold strict control over structures and lifetimes. But when lately I've been doing token-based bind generation I didn't care about anything outside of high level patterns like RAII and ultimately - API ergonomics which was verified in consumer app.
I've been in position of porting real-code to vibe-code platform and seeing non-technical people prompt-stream (they were shared across accounts) I know why they engaged engineer to run this work.
Their efforts took 6 weeks, I ported app within 4 days and (to be honest with myself) without LLM I that'd be 3M+ work pre-LLM.
If you go back to Alan Turings' paper, its all about chastisement! honestly, last pages are all about postive and negative (child!) reenforcement - 1950's style (I do not condone ... etc). Simple as that. I conduct high level litgation in the courts, and win because of a good LLM, with a good version of me, keeping it in line! also crypto and cyber sec. Of course, child rearing, and dealing with former spouses is also very useful. The orginal paper 1950 https://tinyurl.com/yuszahpw (punish is mentioned six times). Just say-ing-like. TTFN.
Im beginning to think buzzword bingo is a viable job interview method.
If you use the b words the LLM would be somewhat constrained to content that has them. If you don't use them it will use content that doesn't have them.
The company website (subject) will never become a static html document. If you request exactly what static html does it should probably point you to a wysiwyg website builder.
> Because my friend didn’t have the vocabulary to ask the AI to write code
Is it possible that the effectiveness of an LLM user with respect to the expertise of the user is like a sigmoid function or at least a step function in that shape? That is, one has to know something like the basic concepts and the vocabulary to bootstrap a programming project, but one does not have to know too much to do lots of meaningful work, and then again one needs to be en expert to build something extraordinary. Since most of the work is somewhere middle, most of us mere mortals are still concerned or stressed out for the possibility that LLMs will squeeze out too many job opportunities.
> The model outputs are much more concise than when I try and talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians”
Maybe, but FWIW my first thought when I skimmed Tao's session was that he probably has a personal system prompt requesting this style.
E.g. even if you get it into "talking to an expert" mode I've found AI waffling through filler like "given your background in Linux kernel engineering, I'll skip the surface level and go straight to the technical meat". You do have to explicitly tell them if you don't want this.
"The most important skill in the AI era may not be prompting. It may be learning how to solve problems using the right kind of help." [1]
Context - I have over 25+ years in software, and I have this observation - being introduced to a new codebase as a human is difficult, especially depending on the scale/size and complexity of it.
Yes, you do start to learn it as you work through it, but if the scale is truly huge, it may just not be possible to fully read and understand all the code and paths etc.
I have found systems-thinkers (I believe I am one, sometimes they are architects) to be able to kind of "see the whole picture" while not knowing all the details, to the point of being able to guess how the system/software should be behaving, even if it is not actually yet. This is a hugely valuable skill and I think takes a certain kind of brain too.
That said, I think recently I may have realized something - we rely on statistics and confidence levels in order to make statements about larger populations. If we can represent a codebase as a, perhaps stratified population of code, interfaces, docs, etc. etc. etc. we may be able to take a valid random sample, review portions of the code, and make some kind of assertions about the state of the larger system - potentially, from that.
I am trying to implement this as a side project right now to see if there is anything to it, basically, a combination of AI/LLM + stats/sampling + facilitated expert human review.
I'd be interested to know if anybody is doing anything similar.
A fascinating thing about the LLM/AI blogosphere and X is watching memetics in real time. An idea like this one propagates on the order of days until everyone that speaks publicly or in workplace meetings about AI is repeating it.
I actually don't feel like Tao's recently published conversation is the best example of this idea. As intelligent as Dr. Tao is, and surely more so than me, I got the feeling that he wasn't running up against failure states of the model, which I'm not sure you could attribute entirely to his expertise. I honestly think it was more a matter of luck that the model apparently had so much training data on the topic or that it was architecturally so well suited for it. On the other hand, I've had really surprising moments where Claude was just failing terribly to execute simple dev ops tasks having to do with log processing. And I'd be so bold to say that I don't think it could have been explained by a lack of expertise on my part, or even a misuse of the model.
So yeah, sometimes LLMs reward expertise, sometimes they don't. I guess either way it helps to have it.
I think this is just the nature of LLMs as predictive generators.
The model is predicting the type/level of conversation based on what the other party is saying.
The most typical types of conversation are of two peers, so by default the LLM is likely to respond to you at your own level, unless you ask it to behave differently.
As always, prediction goes deep. The best response to Terrance Tao is Tao-level math.
It reminds me of reading how LLMs continue chess games if given a partial game - they have learnt to assess player strength based on the moves they make, and will predict game continuations based on the perceived strength of each player, predicting (generating) poor quality moves for a weaker player. This isn't an AI playing chess to win - it's an expert predictor predicting what comes next.
I like it. And if you don't know a codebase, like normal, spend some time learning in. Use the LLM to query it, create your own architecture diagrams. Get homey with it before making sweeping changes. Maybe make the first simple changes by hand.
Of course they do. They have such a huge parameter maps. You need to be able to guide it through the map, so it starts making the right connections. Even in the Sonnet 3.7 days, it became clear to me, that if I have want efficient code out of it, I need to really take care of the context. If I just let it research a problem, it will mess up most of the time. If I tell it to study A, B, C and then present problem D, it will solve it perfectly. And it's true even with the current top models.
Well, nice post. Actually, there may be some truth behind it, but basically, it captures what I—as a programmer—want to read: expertise will remain valuable.
But how I am observing is different, though. Since LLMs the gap between experts and non-experts has been shrinking. And yes, there is still a gap, but vanishing.
I agree with the premise that LLMs reward experstise, but people without expertise can very eaily learn to prompt correctly and get to a result that is very good. I remember somebody proved a mathematical conjecture by just asking 'keep going' in plain english without a mathematics background.
But then what's the point of the proof? I guess it's cool that it's possible, but given enough tokens, you could take someone who's never written a line of code and have them prompt AI to turn their vibe-coded meal prep app into a highly available distributed system with multi-region failover and immutable audit logs.
They would probably get something that checks those boxes in one way or another, but what does it do for them?
Yes. I agree with most, if not all of this. For instance, I am seeing folks either relying in the LLM as an _assumed_ expert or, assuming someone - who knows the structure of skill definitions - also has some expertise (in the area of the skill).
It's a difficult situation; there is not much point in explaining _why_ the LLM output or skill in use (on a domain problem) isn't what the person actually _needs_ to address the domain problem, because the person isn't a domain expert or indeed, adjacent to domain expertise.
But, it is an interesting experiemnt to arm folk with little domain expertise with the _skill_ necessary to be able to extract the right solution from the model.
This is true for output but as well for learning, if you speak to an LLM trying to get it to give you a certain answer, it’ll find a way to tell you you’re right. If you’re truth seeking and attempting to understand it step by step as it’s going, you’ll likely learn what it’s doing as it’s doing it, meaning you’re basically distilling that information into your own local LLM (also know as the brain).
Similarly there is research that shows the quality of LLM outputs strongly correlate with the education level (in the field) of the person promoting them.
This is true today and has been my experience as well -- both to write software as well as doing computational physics.
The interesting question then is to ask how long will this stay true? As the models get better will they eventually not need the human expertise to start adding value?
I very much agree, but at the same time I feel like those type of shortcomings are fundamental and will be somehow fixed within the next year. The AI companies would just go bankrupt otherwise...
This is also why people's experience with LLMs/AI varies so much, because some people can see a use for AI for their needs and go about using the tool, while others do not as it relates to whatever they're working on and so they may say "LLMs/AI are useless" (it doesn't mean they're not experts though, although some people who have totally no expertise might also see no use for AI for themselves).
In my experience (scientific programming) AI is a giant multiplier for people with specialized knowledge.
But it’s also a giant devaluer for that same knowledge as people with no idea what they’re doing can clog the field with plausible bullshit.
It’s now the case that if someone tells me they’ve done something, and I look into it and find out it’s completely AI slop, then I will have spent more time on the project than the person who “made” it. The situation is completely untenable and only serves to drain time and resources from people with better things to do.
this will have educational consequences (that I'm trying to solve). I don't think that we can adjust without rapid education and making extreme specialists of us all.
This requires coordination, certification, licensing, and other tiers of authenticity. False experts can ruin sample gathering, can ruin training. False expertise is exemplified by the current American Administration. Look at Robert F. Kennedy Jr.; he's a false expert. He is responsible for the measles outbreak. He is responsible for ivermectin abuse by humans. False expertise is overtaking real expertise. And the results are continuously disastrous and large-scale.
If LLMs are good at busy work, and it is expertise that lets you distinguish between busy work and valuable one, then it makes sense that they reward expertise.
I agree. When I talk to LLMs about fields I am familiar with, I can push back on bad suggestions and ignore faulty/incorrect advice and assumptions, which is much harder for unfamiliar subjects.
Of course, simple common sense and extremely basic Googling on unfamiliar subjects can produce similar results, but it's much faster if you are truly understanding what the AI is suggesting.
I have lengthy conversations with my LLM, almost like an interview. I agree on the expertise part, because I wouldn't be able to go in depth on a subject with it if I lacked the expertise.
Some work is a result of design and negotiations in those designs.
I don't think Tao's style works with everyone/thing, especially if we don't know what style he's tuned his LLM on.
I often do my best to represent a genuine interest in the subject at hand and learning in general to models. Imagine the model's response prose and mannerisms being on the other polar end of answering questions simply to get the correct answers as they're often scoped for on quantitative benchmarks. Not sure I explained this well, sorry. An LLM could help
recently i start to rewrite a core part of one of my iOS VPN app to rust, which previously use fd.io vpp as it's networking core, the original vpp port is 1.5 years ago manually by myself, given i know a lot about how the vpp does and how vpp coroutine and runtime scheduling works.
the rewrite is in good shape and solve many issues such as pre allocated memory heap using mmap apis and some scheduling issue of back2back tcp session terminated in the vpp host stack.
also by addressing the issus, i am now can easily integrated tailscale as a addon interface for moving in/out l3 packets between tailscale and the core.
All those i think cannot be done easily without domain knowledge about those networking and system stuffs.
So we could run a lighter LLM in front of humans, which translates from 'no domain knowledge' to 'domain expert' and in turn prompts over to the larger LLM.
Then the larger LLM gets all the right lights on, yields better outputs and we translate back into user domain.
I kinda thought the chain-of-thought reasoning already did this, no?
It's pretty obvious that for some questions a novice wont be able to drive the conversation towards an "answer". A novice may also not be able to understand an answer either. But there's a more subtle failure mode.
The vernacular used by an expert and novice to describe the exact same problem lead to different traversals of the information space.
For example, I recently asked ChatGPT a medical question using plain english. It gave me an imprecise vague response and told me to call 911. Repeated prodding did not fix this, so I asked the exact same question using medical jargon and in one shot I got what I wanted.
Might be a very noob question but in this era of LLMs, let me ask the reverse, how do you gain expertise? It seems this rewards people who had expertise pre LLMs, but what about people who don’t have that in a specific domain? What approaches are viable now in this current system?
this 100%. Skills are gained via effort. Not too much effort that it discourages you, but enough so it's a challenge and engages you. Ofc it helps to have wonderful teachers, coaches, mentors, books, even ai to help guide you, but YOU must put in the effort. You don't get something for nothing.
The same as it's been! Make things without using LLMs. Don't debug with them, don't use them to research things, just do it yourself. It'll be painful and that pain is learning.
But getting an answer is not the same thing as understanding why that is the correct answer, or going deeper and learning more about the subject.
IMHO, the people who genuinely desire to learn will trudge through whatever they need to in order to grow their understanding - be it through reading books, original research papers or what have you.
If, OTOH, all you seek is the answers and that alone is satisfying to you, then of course you simply will not be motivated to do it the old school way anyway. But that's hardly different now in the age of AI.
Domain knowledge will stand alone as the sole differentiator. Because LLM benefits can be reaped by almost anyone and it’s a force multiplier. Now those who have the strongest initial force will have a far bigger edge than before.
Real world domain knowledge and experience cuts through the chaff too. LLMs are going to have people reinventing the wheel and wasting tonnes of time on stuff that won't work out. If you're a domain expert you are going to be much more aware of how to focus effort in the right places, and what's actually needed or been tried before in your niche.
A lot of this domain knowledge is not in any training data, it's locked up in companies in the industry. I suspect it will get even more important to guard it.
Love this idea of reading prompts that lead to new discoveries and figuring out how the person got the LLM there. It truly is an art and I’m always reminded of “I, Robot” and the scene about “you must ask the right questions.”
Agree so much with this! In domains I know well I get much better results then someone who doesn't know the domain because I know where to challenge the LLM. LLMs need to be pushed because otherwise their answers are typically average.
I do agree. An LLM is like a motorboat that's tends to drift off course. If you know where you want to go, and can steer it to keep it on course, you will get there very fast.
Overall, I agree, when I ask things I'm an expert in and do professionally every day. I get very good useful answers. When, for example, our marketing people, ask about the science, they often get confusing and wrong answers.
I agree with this post's gist, and I've certainly noticed how LLMs change their interaction with me once I demonstrate some knowledge. I've often started a technical conversation very vaguely and only once I challenge the LLM on its simplifications does it start to actually get to the meat of issues. Often there is a perceptible moment where the LLM seems to recognize my level of ability and how it communicates clearly changes.
But another thing I have found is that I get significantly better results from the LLM by treating it like an intelligent independent agent. All of the "you are a senior dev ..." or "your starving kids depend on the correctness of this answer ..." kind of prompting has been mostly useless. In general, I find being honest and clear to be the best strategy.
You can't "pretend" to be a senior software engineer. If I can root you out of an interview process then you aren't going to fool the LLM. But if you clearly state your level of expertise and your desired outcome, then the LLM does a very good job of meeting you where you are.
There is also a strange ephemeral attitude I get from agents sometimes, like they don't like to be called out for being wrong. But in the same way that human's show this trait, they also seem to warm up over time as they gain trust. It is almost like social positioning, once they realize they aren't actually expert they morph into a support role stance pretty seamlessly. That is also why they can still feel sycophantic, because once they realize they aren't actually driving the discussion they can actually feel like enthusiastic passengers, wanting to see where the conversation leads as much as the prompter.
It seems possible, in fact reasonable, to understand my observation as following from your assumed "correlation". I gave it context that it is having a conversation with an expert, it correlates it's output with the context. A non-expert user can't fake that context so they never see similar correlated output. This context-correlation doesn't happen if one uses "You are a expert ..." prompting "tricks".
My experience has been, if you speak like an expert who is speaking to an expert you get better results. I'm not sure how that is anthropomorphizing. It is just "if I do these things, I get these results".
i find this post re-assuring (as who doesn't like to feel like they are an expert at something and llm definitely strips that away)... but it still feels like you are rewarded just as much for being a 6/10 expert as you are for being a 9/10 expert. It definitely is an equalizer it is just a question of to what degree.
Not the best way to test this, but I am working on a blog post, where I'll implement a problem statement with the dumbest/cheapest AI model while someone non technical person will vibe code end to end and compare both of them.
If you are not skilled they are not chaos, just more distracted, wasting tokens in being nice to humans. It could still teach one very well so that one improves its domain expertise.
Maybe LLMs don't usher in the end of software engineering but they definitely end the whole "made with love (and coffee) in X". Nobody cares if you put effort into something software related. "Does it work? Yes? Ok build the next thing." Its the same with the notion of "taste" (see "sometimes tasteless computer code" from the Goedecke article), your colleague who is also a SE might respect your choices as good taste. But it ends there. This was also the case pre-LLMs I would argue. What is worse now is that communicating any uncertainty in decisions related to implementation will result in an immediate "Have you asked Claude?".
The entire issue is that when you ask it to do something, you are leaving it to chance they may or may not do it properly, either on look/feel, performance, security, scalability etc. It compounds as you layer a new prompt output over that project.
For example, if you write iOS apps, you ask it to do some animations. It will do it for you, but you are at it's mercy of writing complete custom code or you can specify they use a certain apple supplied API for a native experience. It's one of thousand things that will get a vibe coder if they don't ask. Or you may get lucky and LLM chooses to use a native method.
Many, including myself, report having a lot of success with braindumping and not structuring anything. Just talking into speech recognition for 2-10 minutes as a stream of consciousness about what my context is, what I want, what I know already, what I have a vague hunch about, how it fits into a bigger picture, what aspects are most important to me, any footguns I already know about, really like having a chat with a person on the phone, with someone you have to guide remotely because they have to implement the thing right now but you have to be out of office and so your only interface is speech. Except you can be more structureless because the AI won't be offended. Just keep on rambling, and press enter, don't even correct mistranscriptions. It will understand it anyway.
Now, the key is, that while rambling without structure, you do have to drop the key facts into your speech, and you have to know what you're talking about in at least a good portion of it.
I think people are afraid of doing it, because it seems "not the right way" or "not scientific" or whatnot. They want to believe there is some magic to writing the right prompt. So let me tell you, it works.
I don't completely disagree with the concept of giving a free association thought process ramble into context. But I also bet that when you start getting it to actually generate code and link modules of things together, subroutines, functions, code structure and filenames, you still pay attention to what it does and you guide it into the architecture that makes logical sense to you.
For real work yes. For personal projects, less and less since Fable came out (probably the same if true of the other frontier models). You can get a lot done if it's just some one off, or a personal tool, even without looking at the code, just trying the application. Frontier models now automatically test it before handing the thing to you, they take screenshots, they fix the superficial issues themselves. To get something up and running, it's enough to send chat messages.
Unfortunately the software industry is saying things like "don't look at the code", "LLMs have made developers 10-100x faster", etc. The only way they can make such claims is by saying what you said above: "Claude, make me Microsoft Flight Simulator, make no mistakes".
Additionally engineers are facing pressures via deadlines to work in the paradigm of "Claude, make me Microsoft Flight Simulator, make no mistakes"...
The question i wonder about is, when will an event come along that persuades everyone that human understanding is still required? Or will it never come?
I wonder the same thing. I think we've already seen some of this happening, however the consequences haven't been large enough to the organization, for example:
Yeah, this will never happen until governments step in and cause these companies real pain. I mean just look at Crowdstrike. They caused billions and billions in economic damage due to their incompetence, and nothing happened. In fact, their stock is close to an all time high.
You're assuming that LLMs entered a world of people who understood how the systems they're inside of work, why they're setup that way, and that LLMs are displacing them. I sadly don't think that's the case in... well... a lot of the cases.
Such an event would have to be pretty catastrophic at this point to slow down the inertia. Perhaps the tech debt will just pile up until someone's product implodes, or there's a massive safety issue that causes loss of life, or some big hedge fund goes bust.
The event could be when fair pricing comes from the model providers. We're still at the cash burning stage. When the economy crashes a little and departments start monitoring their spending, and the prices for inference are 10x what they are, there will be less tolerance for employees to substitute constant AI usage for understanding.
The fact that Claude knows I wrote the Virgil compiler makes it be on its best behavior when working on it. I force it to not write too much code, and to write more tests. I push back on slop and just adding another special case. It has a surprisingly deep understanding of floating point.
I'm surprised nobody mentioned (including the author) the Gell-Mann Amnesia Effect [1]... Just substitute "LLM" for "journalist" and there you have it.
And to be honest, I have seen it, as I'm sure (almost) everyone has, who has demonstrated experience/expertise in their own fields, and correct the LLM's responses one time or another...
Same thing I personally am not a front end guy but I have dabbled with it in the past but I am writing a front end app and a chrome extension, but besides a few pages of code I have reviewed I really do not know what the fck is written (its for an MVP I am building) and I am feeling really conflicted as to what the fuck do I do. At work, the stuff I write has a decent mix of my code, AI code and a few things I do the old way of copying from stackoverflow and seeing what works/doesn't work.
The LLM industry is deliberately consuming human expertise on a grand scale, so that eventually knowledge work is delegated to any machine, yet I have to gain some sense of comfort knowing that for the present point in time it still rewards personal skill?
Regardless of whether you agree with the claim or not, it's definitely not the endgame.
You got to know how to use the model+harness+prompt to achieve the results you want, but honestly for many projects and questions all the models already pump out their same best version of an answer.
Sometimes it is really akin to a git clone, although it was a LLM request.
This rewarding expertise is somewhat wishful thinking.
At the end of a day, it feels and is more like gambling, even with the recommended expertise and a good approach, don't delude yourself you're simply pulling the lever too, as any novice.
I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt.
I was fully expecting that writing the code will pose no problem for the AI. But i was curious if the AI will realise that my friend is a novice and needs extra help with things like: copy pasting the code into a text file and saving it with an html extension, helping her host the file online so she can share it with others, buying a domain for it, etc. I assumed they will get there eventually, but i also assumed that it will take a lot of stumbling around and misunderstandings.
But i was completely wrong. They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. They were just going around in circles where the AI was brainstorming with her about possible features and getting thints more and more complicated. We terminated the experiment after one and a half hours and many many messages exchanged between her and the LLM.
Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode. In other words in that case the LLM would have rewarded even just a little bit of expertise, but without it there was a confusion about goals between the human and the machine.
I ran a similar test and got completely different results. My girlfriend (hair stylist/artist) with zero coding background mentioned a Telegram bot idea. I asked "Why not build it yourself?" I gave her a Windows laptop, but she said she wanted what I have instead. So I handed her a USB stick and told her she was on her own now. Fast forward: she now runs Arch Linux with Hyprland (I use Xorg/i3 though), fully riced with cats. I only interfered on partitioning to preserve my data. She even installed Steam and got Portal running (that was her "watch this" flex after I told her not to even try). She pulled all of this off using a free-tier Gemini chat.
For the bot, a friend gave her a Kimi 2.7 key. She set up their harness and built a working bot in a matter of days. She even got a free Oracle VPS for deployment, though I stopped her there to check security first (still haven't had time, unfortunately). She uses that laptop daily now and says she enjoys it over Windows by a mile.
There are big differences in what people call "tinkerers". Some are INTERESTED in building things, give it a little shot and find it to be daunting, and push it no further.
True tinkerers have no problem with this, because they enjoy learning how things work. Installing Linux, Steam, Portal are all relatively straightforward tasks for someone who uses computers on the regular - but to some people this is just something they've never done, are scared to do, or just don't want to learn. (Which is fine, but they'll never pick these agents up and run free.)
Barrier to entry used to be blog posts, documentation, watching poor quality Youtube videos of a thing that SEEMS related to what you're trying to do. Now we're getting that spoon fed to our particular case, so the friction is essentially just "follow the AI directives". (However, the depth of understanding probably struggles.)
You can get pretty dang far asking the AI to explain things and show you where it got that info, especially with IT/DEV stuff. Thing that has been very useful to me is specifically asking AI for that vocabulary. AI is suprisingly useful for giving a vague description of something you want and asking for possible words that match. But you have to KNOW to try that. A tinkerer might have started that convo with the chatbot by asking "What are web pages built from" "Can we use those building blocks to make our own simple app" and gotten there from a place of low expertise in web development but juicing out the knowlege from a place of low expertise is itself a skill.
I know you are not arguing from bad faith, but you are making an assumption along cultural lines, which is a mistake of missing the forest through the trees.
The vast majority of people (blanket statement, I know..) do not come from a culture where embracing curiosity, asking questions, or trying to break things down is the norm. Developers, tinkerers, etc., sure... you can reasonably make that assumption. But not everyone. A cultural practice of critical thinking and problem solving is HOW you KNOW to ask WHAT questions need to be answered FIRST, in order to solve a problem or progress toward a solution (if more information is needed).
You might even make the argument that everyone should have these skills, and I would agree with you. But the missing link here is a culture or cultural practice that provides those things (the WHY), and an AI/LLM will not provide those things in absentia, without "prompting", or build up that infrastructure in meatspace for a given set of users. Ignore this at your own risk.
for a lot of lower barrier to entry tinkering, for years i felt it was mostly just google a thing and follow directions. Feel like the only thing that really changed is the initial query can be a mess of kinda nonsense and still give you those nice directions, and you can instantly get clarity on a single direction if you get stuck.
So the barrier is still kinda there to just grab the instructions and follow them, its just incredibly easier to follow them.
Sounds like OP’s friend was looking for a website, and your girlfriend was looking for a hobby.
> Arch Linux with Hyprland (I use Xorg/i3 though), fully riced with cats
I think I lack the vocabulary to understand "fully riced with cats" in context here.
To rice something must mean to customize and decorate. I'm guessing that it references the "rice rocket" term, originally a derogatory ethic slur. That term likely originated in the USA in the 70's or 80's, at first referring to Japanese motorcycles (which were fast, rightfully deserving the "rocket" designation: classic Japanese racing bikes are fully-fledged crotch rockets).
Sometime in the late 1990s, a Rice Rockets website appeared, making fun of people decorating their under-powered import econobox cars (not rockets in any sense) to look like racing cars with features like rear wings (on a front-wheel drive, lol), exhaust modifications ("fart cannons"), stripes, stickers, rims, steering wheels, etc.
If the cats decorating the desktop are Hello Kitty, she is really ricing it.
Desktop Linux has a deep and storied culture of real empowerment through learning and teaching. Even if you never sign up for a forum or hop into an IRC channel, that culture pervades a host of informal docs like the Arch wiki, and seems likely to get picked up by LLMs during training.
Maybe web development, despite plentiful tutorials, just isn't quite the same.
I needed a website and did the same. I would say it's a mixed bag to do it from the command line codex. It nailed generating the code but lots of stuff was annoyingly off. Any copy it wrote was terrible but using it to expand and refine my writing was helpful.
I would say we are very close to the end of things like WordPress and templated websites. It's pretty easy to make a custom page.
Damn you, Poe's law!
What does "fully riced with cats" mean?
I understand "ricing" to mean customising the appearance of a Linux machine, see e.g https://old.reddit.com/r/unixporn/top/?sort=top&t=month (SFW). So in this case I assume the UI has a cat theme or cat icons.
I think the etymology is from "rice burner" cars [0]:
> Riced out is an adjective denigrating a badly customized sports car, "usually with oversized or ill-matched exterior appointments".
[0]: https://en.wikipedia.org/wiki/Rice_burner
Rice stands for Race Inspired Cosmetic Enhancements.
Race-inspired to look cool, but not actually functional. Picture putting one of those giant fake air intakes on the hood of your car.
Do you think "Race Inspired Cosmetic Enhancements" is the origin of "ricing" / "riced-out" or a backronym[1]?
Honest curiosity about language and language use.
In fairness and full disclosure: I have believed until now that the language comes from rice-burners and related, and I have never liked it. If I were still in communities that used it (unix desktop crowd), I might proactively steer newcomers towards your acronym as a kind of reclaiming.
[1]: https://en.wikipedia.org/wiki/Backronym
> I might proactively steer newcomers towards your acronym as a kind of reclaiming.
Any negative connotations "rice burner" once had was lost when the term shifted towards referring to cars instead of humans. But now re-recognizing that the enhancements are inspired by the East Asian race turns the connotations back to humans. Isn't that a regression?
Nah, I remember back then, it was definitely intended to be derogatory to the cars, to asians, and to people who like the cars.
Yes, the term originated as derogatory slang towards the Asian population in the early 1900s, and its use towards them became especially popular during the Korean War. There is no question there. However, as before, the term evolved away from people and towards cars. You cannot be derogatory towards an inanimate object. It doesn't have a mechanism to internalize feedback. In that evolution, the term lost its derogatory connotations. The historical usage, while a part of our past, is used no more.
Reintroducing this to be something about a population's race reinstates the derogatoriness. You can be derogatory towards humans. Minimizing the cosmetic enhancements to be being inspired by the East Asian race and not valuable human achievement brings us right back to the same place we were when Japanese cars started being introduced into the North American market, diminishing the human contribution. It is a regression.
Old enough to watch this cycle from racism, to community, to memes, to racism, and now back to community
Speaking of flexing..
> mentioned a Telegram bot idea
> Fast forward: she now runs Arch Linux with Hyprland (I use Xorg/i3 though), fully riced with cats.
Did she end up building the bot or did the LLM just lead her down a desktop Linux rabbit hole?
She did build a bot that saves cooking ideas with a schedule of dates for when to cook what. We have a lot of friends come over, and we cook various dishes for every occasion. Just a couple of days ago, I asked if she had a recipe for a dessert we tried earlier, and she said, "Yes, of course, let me just check that in the bot." The bot that runs locally and still helps with the end goal, makes me feel proud of her, even though that was an experiment she was aware of.
>she didn’t find a way to tip the AI into “just do it, write it now” mode
worse, the longer an LLM conversation goes on, but especially with constricted/free models (yes the simple chat interface they are likely using) the harder it is to get an LLM into this mode even *IF* you know the right words to say
at that point the best way forward is to terminate the exchange entirely, and to start off with the right initial message, instantly getting into coding mode. a non technical person will not know this and be stuck in feature theory crafting mode in perpetuity, or worse in an endless "excuses' mode as the LLM diverts ant attempt at coding into reasons why its not going to: "i wont output incomplete/broken code! that would require too many lines of code sorry i wont do it! i wont be able to get it perfect so i wont attempt it! but heres more features and theory crafting"
will a non technical person know to end the conversation and start fresh? not likely unless they have a lot of experience already with LLMs
Also, the correct way to LLM is to constantly trial-and-error in new/branched contexts.
Remember the LLM is not a human employee. You don't have to say "yes and" to whatever crap they produced so as to not hurt their feelings or infringe upon their creative autonomy, nor do you have to defend the correctness of your original instructions so that they don't think less of you for asking them to chase the wrong goose.
I probably generate 20-50 lines of code for every 1 line that I keep.
This is also why I think harnesses and things like Claude Code and OpenCode are false efficiency. The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code.
This is the only way I've been able to get LLMs to produce code I will actually use. Manually selecting the context for them and asking for a specific piece or similar
I started using harnesses because they are good for when something breaks and it's not trivial to investigate so I'll have the agent tell me what's happening, then using that to produce my own change
> The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code.
That’s been the way I do it.
I suppose that it will be considered “quaint,” soon enough, but I have found it to be effective.
And the best part is, I don't spend more than $20 a month on LLMs. Going manual and constantly branching keeps the contexts super lean.
I’m likely to switch to the $100/month sub, but I want to finish this project on the $20 one first, as a “proof of concept.”
I think it’s valuable enough to justify the price, and I want it to use the better model, as much as possible.
A technical person will also have have met those problems related to context and will know to drop a simple
"update an AGENTS file with relevant information"
To be able to navigate that faster on longer tasks. Meanwhile, the lay person does not even conceive of the LLM as a file reading entity. To them, its machinations are its own, so these types of "dumb" (simple) solutions are not even on the deck of cards.
Interesting, but... wrong tool, wrong job. And by tool, I mean web based chat interface, not LLMs in general. (Maybe wrong delivery mechanism, if you like.)
Your friend needed an agent, not a chatbot. I use Claude within VS Code (as per many others) but I certainly wouldn't recommend that for a beginner. They needed a tool that's specifically aimed at people who want to build software but don't know the first thing about how to do it. I think there are a bunch of these now but the one I'm most aware of is Lovable, and I'm pretty surprised you didn't recommend one of these.
An HR person I know was searching for a way get something build, found Lovable, and managed to build a somewhat functional application with it on their first attempt within an hour or two. It was full of holes and far from perfect but they got something working - at least the outline of a potential solution.
As I say, you should have recommended your friend to try building with a tool like that: a tool that they're a member of the target market for. They would have got a lot further. I'm not vouching for the quality of the result, but they would have got something.
Even for experiened engineers, chatbots have always been a pretty grim experience for software development: from the mind-numbing drudgery of endlessly copying and pasting code, commands, and prompts around, to the fact that they just can't see enough of what you're doing to generate the best quality output or advice. You can do software development with a ChatBot but it seriously sucks, and better tools are (a) probably being shoved at you day in, day out via ads, and (b) only a Google search or a ChatGPT recommendation away.
(Obviously, nobody's going to search for "Lovable" without knowing about Lovable, but they might search, or ask ChatGPT or whatever, something like, "How would I build a website without knowing anything about building websites?", which might get them an advert or recommendation.)
The whole point is that the friend wouldn’t know what an agent is. The friend would have heard that AI is changing the world and she would have went into ChatGPT or maybe Claude or Gemini. Presuming that the ideal is that anyone should be able to use these tools for anything (which I do not agree with) the chatbot needs to say “you need to use my agent mode.”
This is not a wrong tool, wrong job issue. The issue is that people think any moron can use an LLM and get professional results.
We're not talking about the friend. I'm talking to the person I responded to: I'm saying they should have recommended a better tool to their friend.
My HR friend found Lovable off her own bat. I imagine she probably Googled or asked ChatGPT something like, "How do I build a website to do BLAH without knowing anything about building websites?"
The point is people talk, they ask questions, they Google, they talk to ChatGPT, and if they have a problem to solve they're often quite motivated to find a solution to that problem off their own backs.
If someone asks me for advice on how to get something built then I'm going to recommend a tool that suits them and their situation, whatever that may be. In this specific situation, if they know nothing about building software, I'm certainly not going to sit them down and have them try to follow the most jank-ass way imaginable of building software with an LLM when I know much better tools exist that are built with people like them in mind.
Seriously, what is with the overly narrow assumptions in the replies I'm getting this morning? You're the third person who's tried to set this same fraying paper tiger on me. Can we all just wake up and think about the issues a bit more in the round, please?
That would be poisoning the test with outside knowledge. Essentially boosting their skills with some expertise, exactly what was undesirable.
You are not wrong, just that was not the intent of what the person was trying to test for.
What makes this anecdote hard to believe is that seemingly two things happened simultaneously:
1. The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas.
2. The user was the sycophant in this relationship, never steering the session(-s) into producing something tangible.
The premise supposes that somehow the session(-s) never even tangentially touched implementation/deployment ideas and the user has never typed something like "that's enough, how to make this appear in my browser?". While not impossible, the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
> What makes this anecdote hard to believe is that seemingly two things happened simultaneously
What i described is what happened. I don’t appreciate the undertones where you are insinuating that i’m lying for whatever reason.
> The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas
You call it steering. I would call it falling into that grove. Probably tiny things in the initial message made the first response more likely to be a clarifying/ideating type. And once that happened the conversation was gaining momentum in that direction and neither participant was trying to guide it in a different one.
> the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
Exactly. The LLM itself sidetracked her. They were just talking about cool features they could add, and at no point did she put down her feet and say “stop asking more questions and just write the code”.
It can be a combination of many things. Attitude (some people hate to be rude, and not answering a question feels a bit rude). It can also be that she enjoyed the process of unpacking and elaborating on the idea.
The meaning of the story is not that no lay person can possibly develop using AI. That would be silly, and untrue. I know clear counter examples. The point is that if you don’t know what you don’t know it is harder to steer the AI in the direction you could very easily with the right lingo.
Indeed. It reads like software engineer employment cope by appealing to the lowest common denominator possible. As if solutions for that type of end user weren't already solved for over a decade ago, most of which now have agents built in.
It just is not believable or interesting. Even if it did happen, the reality is it just doesn't matter.
It reads like a discussion between two people sharing detailed experiences of concrete things they've observed, in response to an article where the author did the same. It could be that they're lying to stave off employment anxiety, I suppose, even though nobody involved seemed particularly anxious and everyone involved seems pretty familiar with AI tooling.
It could also be that you've seen a lot of social media memes regarding "cope", observed their ability to provoke strong emotions, and confused this for meaningful insight. I see a lot of AI commentary these days that is clearly being spread for its virality rather than its truth value.
Circular dependency detected. You must first know the thing to be able to know the thing.
No, not really. My HR friend went out and found Lovable on her own. You're acting like people can't Google, or even ask an LLM via a Chatbot interface for recommendations for tools that would help them build an application as an absolute beginner. They can and do. People have agency, which is exactly why this person's friend asked them for help.
This: https://xkcd.com/2501/ 1 billion people use chatgpt, but only 10M use codex, which is even more impressive when you consider the chatgpt windows app now comes with codex a click away. This means everything that is super obvious to us, is not for the average person
They were literally in front of an LLM. Did they ask for tools?
edit: Lovable is a web app too, not an agent.
This comment - and many others - misses the OP's original point: what are tools? Do you mean my laptop, and a pen and paper? What's an agent? I.D.E.? none of this is part of the vocabulary or context of someone who has never built things with a computer, let alone AI. Years ago I had a neighbor who was trying to learn how to use their computer; they complained that the book they were using was only telling them all the DOS and none of the DON'TS. This is were most people start.
Isn’t that kinda the point, though? If you already lack the required knowledge to ask a web-based chat UI (which, as an aside, is almost exclusively referring to ChatGPT Free for the average user) to build a HTML page, how could you possibly know to go search for Lovable, let alone figure out agents?
I feel as though the gap between the theoretical power of LLMs and what the average user knows of them and their capability have already widened so far that it’s irreconcilable.
> how could you possibly know to go search for Lovable, let alone figure out agents?
Is this an actually serious question? Am I losing my mind here?
I didn't tell my HR friend about Lovable: she found it on her own. Of course someone's not going to Google for Lovable if they've never heard of it, but they might Google for "how do I build a website without knowing anything about it?", or ask ChatGPT the same question.
They're also, most likely, getting endless ads for AI services that help you build various kinds of software shoved into their faces all the time - these ads may not couch the value ad in exactly these terms, but that's fundamentally what they're advertising.
Not everybody is like this but there are plenty of people in the world who, when they have a problem, are quite motivated to find ways to solve it off their own backs.
> Is this an actually serious question?
Yes, it is. I just Googled that exact question (and I promise I'm not trying to be obstinate when I say I'm Googling the exact phrase!) and it's automated AI response was to use WiX or Squarespace. Prompting it further with "What if I want it to do bespoke things that Squarespace can't do?" it responded with using Figma to design the website UI and then pass it along to either Framer or a "professional developer".
I do genuinely think that this is a discoverability issue. Of course, if you prompt it further with "Could I use AI to do this?" it dutifully responds that it can help with generating HTML, but that's three layers of difficulty to eventually get whatever default model Gemini has for signed-out Google searches to even suggest HTML.
You are very much mistaken.
Chatgpt can absolutely do the things here: it can give you files, it can integrate and show web pages that you develop, etc.
There is no need for an "agent", the chat version works just fine and is actually easier for novices.
You can see the same thing with restaurants generating their own menus/pictures. Some of them look absolutely terrible, visually ugly, way too information dense, the classic piss filter, etc. idk how they do it, even the most basic prompt I can come up with makes something 10x better, and when I put in my amateur photography knowledge/keywords in it gets pretty close to what I'd consider a good pre LLM quality menu. Literally just adding "make it look nicer/cleaner" seems to get rid of most problems, but people just don't care apparently.
Complete amateurs without these tools were basically limited to making big text in word processors and maybe pasting an image in there.
Being able to create a basically coherent, polished looking image is what you’d get on Fiverr for a few bucks. Mostly hustlers filling in templates, or people that know the tools but never learned design fundamentals.
Actually being a competent professional: Knowing how to visually communicate showing information hierarchy, what purely visual aspects of an image say, how different things read differently among people who might see it— e.g. does an image of an apple communicate fancy computer? teachers/school? Nutrition? Food? Produce?, etc etc etc (Good kerning and type usage, composition, gestalt, etc all come with that for free. Many think that is the point — those are tools someone can wield to do good design, they aren’t themselves good design.)
These tools let amateurs do what the fiverr crowd used to do. Unfortunately, the fiverr crowd is now being pushed into doing what entry-level new graduate professionals used to do, and the job market is kind of fucked.
I think this is about teaching problem solving at a young age and it is an abomination that our education system does so poorly at. The key is to know what kind of questions to ask and knowing when to go deeper and what to pay attention to.
But that is not how our education system aligns us. One typical example of problem solving kids, and I too, learn in school is how to apply a concept in physics to a free-body-diagram(Indian and Chinese education cram schools are famously good at teaching kids how to do this, the usefulness of which I debate). But it all stops at the exam room. No architect or jr structural engineer position for you kiddo.
Another kind of problem solving skill might be how to invest money and understand your own risk appetite to construct portfolios to manage your money. All that is taught in school is a dry compound interest formulae, time discounted cash flows and a black scholes model. Only to find out later I don't need most of it to manage my money.
It's also my opinion that Kids should get to do a lot of real work and experimentation at an early age(13 yo IMO) with real responsibilities and prospects of making income. Knowing the failure mode of most real world problems you can solve is the skill you always want but dont nearly get to do enough of until much later in life(sadly true for a large population of kids in the world).
> They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code.
What harness did you use?
In e.g. claude, there are two modes:
1. Spit out code 2. Draft a plan, ask questions, GOTO 1
You literally have to go out of your way to get it NOT to write code. I keep mine on a tight-ish leash because it modify code way too happily even when there's no intention or instruction to do so
> my friend is a novice and needs extra help with things like: copy pasting the code into a text file
I don't think this is using an agent harness.
"rawdogging" - as the youth says - weights on local comodore cluster?
They are either using some generic web frontend, ala chatgpt, some local app like claude desktop, or programming app like cursor.
Each of those will detect that you are "building an app" and will spit out code in one form or another. You have to try really hard and be very explicit that you want the output in some other format than code.
How would an absolute newbie know what a harness is? What an IDE is? That code is represented in a set of files? That some of those files are actually metadata? What metadata is? What GOTO means? That you're suggesting a modified SDLC?
None of this is related to intelligence, desire or potential. It's about context and experience. The vast majority of people use their computers as consumption devices, like a TV. If I asked you to "make a movie" where would you start?
Why would they need to know any of that? Tons of barely technically literate people at my job crank out webshit POC/demos with Claude desktop.
How should his friend know what a harness is?
Having an experienced friend looking over your shoulder and taking notes of every step might help a bit, I reckon
I thought the idea was to test if she gets along without any help
I'm pretty sure the idea of the question "what harness did she use" was to ask the expert recounting this anecdote to include more detail, not suggest that he relay the question to his example friend.
The whole point of the anecdote was to showcase that it shouldn’t matter what harness she used.
That’s why the question only makes sense if it was relayed to the laywoman, in this case.
Of course everyone knows what a harness is. It is the vest they put on their dogs. Also, construction workers use one. You can buy one on Amazon.
As for a coding harness, I prefer the term agentic coding.
Harness? I am an avid HN reader but even I don't yet fully understand how this word is used in the AI context. Which is exactly the point of the OP.
It all boils down to naming things and cache invalidation, /s
That's the terminology they have chosen because they believe they are "in control". It's a purely psychological thing, not different from calling different DB servers "master" and "slave". When I explain things to people, I say: the LLM is a big mouth, and it is a big mouth without hands. It can only talk. The agent is what gives some hands to the LLM, and then it can do some work.
Harness refers to the tooling that allows you to interact with an LLM. Web chat interface is a harness, CLI coding tools (claude code, codex, opencode, pi, etc.) are harnesses, agent systems (openclaw, hermes, etc.) are harnesses. They present different capabilities to the underlying LLM. Codex, for example, is more likely to write code if you say "I want to build an app" than ChatGPT on web will.
Which is an abstraction layer that the non-technical should need to know, if we follow GP's line of reasoning.
My question is do we harness a bootstrap or bootstrap a harness?
You strap your harness to your bootstraps, and then you can pull yourself up by your own bootstraps... right?
You bootstrap css and harness js
You forgot off by one errors ;)
Personally I even refuse to learn what it is in the AI context. Feels like a waste of energy given new ~~~best ways to use YOUR tokens~~~ are discovered every month and things you learn now will be useless next month.
I am doing perfectly fine with the web UI version of these tools... They seem to also not make tokens dissappear as fast as using claude cli tool to automate implementations. Makes my work day more tolerable as well as I actually have something to do over waiting until some implementation can be read through...
I’d love to see this experiment executed with Claude design.
Particularly with something static, I don’t think they’d fail to get a result.
But without domain knowledge I think they’d misunderstand prototype with finished product.
Without knowing what it’s doing, it’s hard to know what it’s not doing.
As decades-exp SWE I love using Claude Design for any kind of app and web development, because it gives much faster visual feedback loop than changing views within deep framework stack. It much easier to "tell" coding agent what I need instead of writing wall of prose to define visual stuff.
I reminds me old WYSIWYG and unlike Figma it has full HTML/CSS capabilities available.
How I work with it:
- I ask agent to extract part of app into Design, let it even use playwright-cli to get full rendering of the particular view.
- perform design session in Design.
- once design system is perfected I go down to Claude Code dungeons, do /design-sync.
- perform on the stack implementation session.
Actually you don't need Claude Design UI for any of that too. Just ask any coding agent to prepare local mock HTMLs and iterate over them.
I’ve been curious on how to close this loop between engineering and design/product.
We don’t use react, which Claude design seems to trend towards. We use Phoenix / liveview.
We have a shared design system, which keeps the visual elements in line. And then just prototype on design, collab, discuss and arrive at what we want to ship. And then engineering take over and rebuild via hand / claude code.
But the tools aren’t directly connected.
The value has been in the separation. In iterating on the prototype without impacting the codebase, dev cycle, etc. And solving problems/unknowns earlier.
There were always tools for this, but Claude design just feels more accessible and therefore gets used more immediately.
And the fidelity of the outcome (and the assumptions it’s forced the make) are more valuable and faster to achieve than Figma.
I think the way humans divide up design and programming are broadly correct. Working that way with LLMs seems to work well.
I vibe coded an iOS conference schedule app recently, built on top of my own rust UI framework. I started with claude design. I gave it the requirements, and showed it screenshots of other conference schedule apps I like which have features I want to use. I also gave it some visual references for how I want the app styled. It came up with some workable designs. They were a bit 'webby'. But, fine. The high level breakdown of UI screens and navigation between them was excellent.
Then I gave all the HTML files it produced to claude code, along with the documentation for my UI framework and told it to port the code to my UI framework. The first working version was rough. It copied a lot of the unintentional webby look and feel. It worked around missing features in my UI framework by rolling its own janky reimplementations of platform features. For example, instead of using UINavigationController, it rolled its own. It made its own (kinda bad) tab based navigation bar. The app didn't work properly in dark mode, because it was hard-coding a lot of colours. It took a bit of back and forth to fix all of this stuff. But I'm really happy with it now. It looks and feels great.
It's just a pity I couldn't share the app at the conference. Apple took a few days to approve the app in Testflight, and by the time they approved it, the conference was over.
I assume everyone else is playing with the same AI tools that I am, and getting similar results. But a lot of people I talk to seem to have no idea that this is possible right now. They're amazed when I show them my schedule app.
My non-technical cofounder managed to vibe code a holding page with Claude Design and it walked him through deploying it to Netlify.
However for some reason it had him deploy a single HTML file with all the assets encoded as a huge base64 blob in the code that required a massive amount of JavaScript to extract and render.
It's been doing this for our non-technical folk. Giving users a gigantic single file for deployment. We saw one user deploy a JS file with around 3K-5K elements in an array, storing unique IDs of items they wanted to list.
Welcome to Software Development, Lindsey from HR - here's your first database!
People just don't really understand how these things work yet, and they don't know what to ask for, I'm hopeful that they eventually do become more tech-literate, but not sure yet.
Interesting.
I often ask it export a single html file, for an external collaborator or simpler sharing. But I wouldn’t deploy that to production.
I wonder if they asked it to deploy a html file.
But this is exactly the kind of hidden domain knowledge / expertise that changes how you use the tool.
> Particularly with something static
Apps ain’t static.
They probably mean static site, in the sense of static front end, no backend.
Yes. That's not an app.
It could be an app.
Minesweeper is an app right? Unit conversion? Color palette designer? Metronome?
The person you responded too didn't mention app though. They just said static. OP was talking about an app but the responded was hypothesizing about something static.
Anyway I'm not so sure "static" is a viable boundary between app and not app. A static page that does any sort of API request doesn't suddenly become an app imo.
Not really, is a offline chess page not an app? It seems like it would be closer to an app.
What does this hypothetical chess page do? And how’s does it do it if it is static?
You misunderstand the meaning of static web pages. Here's a short explanation of the terms: https://developer.mozilla.org/en-US/docs/Learn_web_developme...
It uses javascript. Still a static file. Lets you play chess
Don’t you update the DOM to render the pieces as they’re moved?
A static website is one that doesn’t have an associated backend API server, just serves as one or more self contained file assets.
The files you serve to the browser are static, not the contents of the page itself
Updating the dom can happen with only individual assets, so it’s a static site
when someone describes a static site/app they generally mean there is no backend. not that the frontend is a static image
for example you can service static sites from S3 that have HTML/CSS/JS but no API or DB
Are you describing a "chat window" experience here? This is apples to oranges.
As opposed to what?
If you ask a non-programmer to install Claude Code, just installing it will be a challenge, then opening the shell and interacting with it. Things as simple as copying and pasting can present roadblocks if you've never used a shell before, and things intuitive to programmers like using up-arrow to go back to a previous prompt would never occur to someone in the field.
Claude Code seems so simple and natural of a UI to programmers, it's easy to forget how much it builds on.
Zoom out one step. The experiment should have been searching for "build website with AI", or "build website with <product name>", not "hey use this very specific UI to do something I know will fail"
(FWIW I think people betting their whole companies on AI are trusting shitty one-wish genie goblins, but the terrible irony is that anyone "technical" with years-old knowledge is talking about something else entirely in today's context)
Claude Cowork is the application aimed at non-developers that gives them a lot of the same functionality. My girlfriend uses it and has gotten quite far in producing her own software.
I got my girlfriend to install Claude Code and she was happily able to create software with it completely independently of me.
With Claude it's actually not that hard. I set up a system for a startup where devs used a full dev environment and non-devs could use the Claude web UI to open rougher PRs, and one of the non-devs asked me what was involved with getting a full dev environment set up, so Claude could iterate and he could test locally, which was faster than the "push to GitHub, use the Vercel/Supabase preview deployment" workflow he was using.
I gave him a link to Ghostty, a link to the claude code copy/paste pipe to bash thing, told him how to cd/ls/pwd into the folder he had locally from the GitHub app, and he was off to the races. I told him to type `claude` to open Claude Code in the terminal and gave him a prompt to use about being a non-dev getting his environment set up to the point of being able to pnpm dev and test, and Claude took it from there. The repo's readme had a setup section which it followed to install brew, asdf, pnpm, etc. With the GitHub MCP he's now opening PRs the same way devs do.
The terminal? No way, the normal Claude or ChatGPT desktop app.
Open any chat window and ask for a simple SPA with startup instructions. It's fine, it's fine, it works.
And you think a random John/Jane Doe knows what a SPA is?
No, but I bet this has a high likelihood of coming out in preliminary discussions.
What most likely happens is a normie has no idea of how to build something that does nothing first - they just start describing the end state.
I'm going to try this with my wife later today. I bet she'd sort something out since she's been a manager forever and phases out instructions maddeningly
Yeah I wonder if they had given their friend Claude code or Codex, would it have been more likely to create what she wanted?
Perhaps! But I do think the vocabulary issue is real and I think LLMs are still sycophantic enough that they won’t really challenge someone or offer alternative ideas on how to implement something unless they explicitly ask.
Interestingly at my work, Claude Code was available before Claude Desktop, so a number of non-technical PMs tried to use it in order to build… anything, with very mixed success.
The “hey guys, check out the website I built with Claude: http://localhost:3000/” joke is real!
In my experience, the whole “the terminal is a scary place” aspect is very real and some non-technical people can feel intimidated by.
I think Claude Code in the desktop app helps alleviate that a bit (perhaps Codex, too, but man what a mess the ‘ol ChatGPT app has become).
But I’m sure there are entire repos of web dev skills that someone could use to put together things with a bit of effort.
> the terminal is a scary place
Isn't the the powerful, unlimited, unopinionated blank LLM text input waiting for your instructions eerily similar to a scary terminal?
WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
Terminals are scary because it feels insurmountable. What are you supposed to do? If you just type “start python program” it gives you this absurd error that doesn’t make sense. What do you mean start is not in path?
The moment you interact with an LLM it gives you feedback that you’re doing things right. It feels like a gradual climb instead of a series of abrupt jumps. People really don’t like feeling like they don’t know what they’re doing, and the terminal constantly reminds you that you are making mistakes.
Scary but then liberating once you get to know what you have to type in to reach your objective.
Yeah, if you're willing to put in that much effort.
I started on this path literally about as soon as I could read thanks to the family having bought a Commodore 64 for my older siblings, but also perfect timing in that when I got to this age the sibling whose room it was in had just gone off to university.
Most people are not like this, in much the same way that they're not going to read the T&C end-to-end (another thing I've done) or learn enough law to actually understand what those words mean (a step too far even for me).
What an ode to exceptionalism /s
You also think that you are smarter than the people flooding Ceuta streets these days, don't you?
> What an ode to exceptionalism
I've had people criticise me for having had the opportunity to learn in that way, as they did not.
> You also think that you are smarter than the people flooding Ceuta streets these days, don't you?
No, why would I think that? I don't know them, the only thing I can say is in their favour: moving country to better your situation is difficult and them getting as far as they did is a demonstration of putting in a lot of effort of the exact type I praise by default.
> Isn't the the powerful, unlimited, unopinionated blank LLM text input waiting for your instructions eerily similar to a scary terminal?
This is why it has the title (for me currently reading "What can I help with?" but this varies a lot) and the text box itself has the placeholder text "Ask anything". Sometimes I get big friendly suggestions about what to ask it, placed on screen near that text box.
> WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
I don't think it's constraints, per se: almost nobody looks at the font list and goes "oh no, too many options!"
Rather, GUIs are there to organise your options visually, group them in ways easy to intuitively get. There's a bit of fashion-induced rot here, e.g. I'm old enough to remember when it was always unambiguous when you were looking at a checkbox vs. a radio button, and now there's a blurry middle ground of collections of boxes with ticks in them that act mutually exclusive, but the point of a GUI from a UX POV is not the same as how software in general drifted as it got both more users and more developers and more opinionated managers and middle managers and designers who only cared about shiny rather than usability.
I am old enough to have observed non-tech workers using all kinds of text-based interfaces and it was a real pleasure seeing how old ma's would jiggle numbers on the bc-style TUI in the way that would offset any modern CompSci major.
Me too. But did you see them on their first day using it or their thousandth?
(Also you don't need to be that old. Less than 10 years ago I watched a doctor breeze through some clinical system while I was crawling along constantly referring to the manual)
> WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering
Are you kidding? WIMP and GUI democratized computing!
What "democratized computing" was cheaper computing and VisiCalc, not WIMP and GUI?
But again, VisiCalc is intentionally limited, it's not a all-powerful environment, on purpose. It's all about intentional limitations, making computation easier to reason about.
Democracy clearly disempowered the monarchs. :)
Not really, for making a single-file anything ex nihilo. I suppose the chat window won't be able to run a linter or make and run tests as a typical "eager" agent might, so maybe it will make more mistakes.
Almost all web chatbot providers have code sandboxes that they will run (limited) tooling for you. If you ask for it, it will run deterministic linters, formatters, format converters, tests for you. Older versions of Claude would for example happily try to reverse engineer entire artifacts for you, once you provide a URL.
My experience is similar, people succeed with Loveable or similar platforms because they don't need to know _deployment_ and _runtime_ experience. It's like using Adobe vs Canva, most people are now used to the latter and don't even have a mental model of "runs on a server" or "database and server are two different things". This is the key difference for me compared to earlier Low Code solutions such as OutSystems that still required a SDLC mental model. However, at some point this breaks, like in your example of feature discussions, because people struggle to test out new ideas _without_ immediately showing them. So it becomes Canva vs Figma, a system-based model. And we don't even have good terminology for that ourselves yet.
We've seen plenty of examples of apps successfully vibe coded by non technical people, including apps making real revenue.
Your friend could start with telling the LLM that they are a non technical person who wants to make an app and it will explain all the successive steps.
>We've seen plenty of examples of apps successfully vibe coded by non technical people, including apps making real revenue.
Have we? Or is this just something that people say now, without citation?
Maybe people don't cite specific apps because they like their jobs, and outing apps as vibe-coded is still seen as negative
I personally know of two completely vibe-coded large apps in my professional environment. One by a non-technical manager, made to solve his needs, then sold to customers. Initial development went along great, but by now velocity has greatly slowed down. Also took a lot of engineering hours (of actual software developers) to get permission management from "chaotic and ineffective" to passable. It's still worse than what you would have gotten by just using a couple sentences of the right technical language at the start. Deployment is also a bit of a nightmare. All in all, anything beyond the first rollout phase was delayed by months. Honestly it should have stayed as a prototype that then gets rebuilt from the ground up. But still, it is a real app, making real revenue
The other example was vibe-coded by a software engineer in his free time. Works pretty well, doesn't have too many bugs. Makes some revenue, but a lot less. Solving manager problems just sells better.
I vibecoded a Postman/Insomnia API tester program, and I use it everyday at work now.
But as another software engineer, I remove myself from that comparison, because the idea is to find out if a non technical person can do the same, that's the definition of vibe coding.
> Honestly it should have stayed as a prototype that then gets rebuilt from the ground up.
And that’s a natural process for many products. In the journey from discovery to prototype to MVP to product, it should be rebuilt multiple times.
Particularly with LLM’s to assist, the process of rebuilding from a new context and understanding of the desired goal requires even less effort.
The hardest part is managing any real users, their expectations, and any data / workflows they’ve come to require from what came before.
A friend of mine, non-technical, is not making money with his apps. But he's creating a street fighter like game. Just for fun.
So there's that.
He can't exactly release it because he uses a lot of copyrighted stuff. It's also meant only for himself. Though, I've been asking if I can play it, it looks fun.
I think we desperately need to start differentiating between "is creating" and "has created". I have a couple of "am creating" projects too, but their proximity to "have created" is directly proportional to how much effort and expertise _I_ am bringing, not so much related to the AI's contribution.
I run into this quite a bit. We have users generating MANY apps at our small company (30 FTE), entirely with Claude. It's great to see people mess around and tinker. It's NOT great to see someone with a GH repo that has 750+ commits for what would be MAYBE 1 week of a developers time. SO these are non-developers now spending hours and hours working on software that is probably going to get thrown out.
We're in this spot where we don't know when to cut our losses on projects like this. (Is it even viable as production software? Does it currrently do what it's supposed to, or are they adding new features? Is there a return on continued development efforts?)
None of these apps they have built are seeing any major usage, and I don't think a single one is what I would call "done" (There was a gold rush stage at the beginning of 2026 where senior leadership wanted everyone to spend some time messing around with Claude). Unfortunately, they never told anyone when to stop messing around with Claude, so the ROI is ever diminishing.
I’m afraid this isn’t surprising anymore.
Plinq.
Plinq was made on Lovable, https://www.aieatingtheworld.com/articles/non-technical-foun...
Couple more on https://buildthedamnthing.com/resources/articles/case-studie...
The problem with lovable, from someone with insider knowledge, is that many of the apps existed even before appearing there and where ported to the platform to ride the hype wave.
Wow, I am not sure that I want my safety app to be videcoded by someone without experience
I don’t think we’re talking about this kind of website.
.
That was kinda harnessed and prompted by a team of security experts, so there's that.
No we haven't
Every now and then I'll write a single prompt to Claude to see how much of a web app I can build in one shot. I usually pick Django (though I assume any other web framework could suffice) because of the battery included nature, I still sometimes describe that I want users to login / register, have access to x, y, z
I also noticed a friend of mine had way more success by having Claude write code by doing TDD and giving Claude scenarios for things the code should be able to handle, if you do this correctly, and cover edge cases, Claude will work with these in mind.
OTOH my wife's friends got drunk and made "tinder for horse purchases". They prompted to read typical horse advertisements (we're all horse people) and create an app with mock tinder like entries to swipe right and left to buy horses.
A web app was produced with lots of mock "Hi i'm Dominique and i love running through fields and having a bucking good time" type entries complete with silly horse photos. A huge amount of drunken fun even if it boiled a towns water supply and blew through half a subscription to create.
I was looking at the results as a dev with 30 years experience and thinking fuck me. The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim!
> The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim!
Yes, but the premise of the article is you should be able to outdo a bunch of drunk people with your 30 years experience, if you use AI too.
The article doesn't hold the universal truth.
I'm wondering: What did she tell you that made you build the website? Did she ask the same thing from the AI? Did you bridge any gaps the AI didn't do for you? I don't know the answer, but I suspect that she treated the AI differently and would've gotten better results if she had asked what she told you. At least if it's an agentic coding assistant like Claude Code or GitHub Copilot. Of course a simple chat will leave manual tasks for you.
I find there's a lot of variation between LLMs / models...
But there's also some psychology in play too; that we (engineers) see a lot: Some people just let their imagination run away and forget to "do"; without someone in the conversation pushing for results and action, the conversation will just stay within imagination and everyone will be happy in the moment but nothing will get done.
This sort of thing has been studied in academic experiments. Although the models studied are now “old”, and I expect the floor is higher, the lack of vocabulary and basic concept familiarity sets the ceiling.
https://www.feldmanmolly.com/chiwork2024-author-version.pdf
I have the same experience giving my brother an OpenClaw as his personal assistant.
His words were: It feels as if I need to know how to program it.
I was expecting he could say something like "Oh, it seems like you don't remember the people I'm referring to, perhaps you need some kind of CRM system. Can you investigate if there are any easily available CRM systems you can interface with, so we don't need to make one for you?"
Whereas my OpenClaw moment was trying to make it manage its own NixOS installation, so that if I ask it to do something, it doesn't yolo `apt install` commands, but rather improves on the same overview of its own installation.
A lot of people had success making their OpenClaw do things without being Linux experts. But you need a tinkerer's mindset, is what I came to conclude.
Curiosity and a tinkerer's / engineer's mindset is basically the only thing we can hire for right now
Do your friend at least know what Claude Code (or any harness) is?
Of course she wouldn't be able to make a website if she doesn't even know the right tools to use. But I don't think it prove anything. Knowing and installing Claude Code might not be a common sense, but nor is it "expertise" or "skill."
I've seen in first hand that people struggle installing Steam. Yes, "people" in the plural. But just because some people struggle with it, it doesn't mean that installing Steam isn't an objectively easy task. Your friend's experience doesn't change the fact that building a website is something that an average person can do in hours if not minutes.
Knowing what Claude Code is and why you might want it actually is domain knowledge which the op's friend does not have. Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise.
> Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise.
I just said I've seen multiple people struggle installing Steam...
The point is that it's something objectively easy. Once they find (in this case, given by me) the correct instructions and follow through, they can easily do it by themselves again. It's quite different from what are traditionally considered "expertise": for example, even if you followed a master's painting process, stroke by stroke, tomorrow you still don't know how to paint.
Building common apps were more akin to "painting," now it's "installing Steam."
> The point is that it's something objectively easy. Once they find (in this case, given by me) the correct instructions and follow through, they can easily do it by themselves again.
This is true of most things in life. It is very easy to make compost, it is very easy to grow carrots, it is very easy to graft an apple tree onto rootstock, it's very easy to hang a door and it's also very easy to replace the break pads on your car.
Once you've done it, that is. And once you know what tools you need. And how to use those tools. And that you actually have those tools.
Codex, Zed, the like are all tools that you need to know exist and you need to have and you need to know how to use. It's the same thing as a wrench, a break bleeding kit, or some graft tape.
You are assuming that the hypothetical friend has been isolated from any kind of technology from the last few decades. The friend can, presumably, use search engines and the very same chip chipities to get the very rudimentary domain knowledge going.
I assume no such thing.
I'm genuinely a bit floored reading the comments here, but I guess my idea of the average HN commenter's ability to talk to non-technical folks about technical topics is biased because I work and socialize with lots of people who don't have technical expertise (thankfully along with other technical folks who also have lots of experience talking to the former group). Many of them don't even know what Claude is, let alone Claude Code.
Yeah it's a true HN bubble moment. I've got non-technical people in my company (a tech company, where the majority of the employees are engineers) who have absolutely no idea what any of the agentic coding nonsense is, their only exposure to this stuff is Gemini in Google's office apps.
To them this is all just "AI" whether it comes from OpenAI, Anthropic or Google - hell they probably don't even know what an LLM is in the first place, yet alone which company provides what tooling. And these are people whose day-to-day involves talking to at least 1 dev a day, so you would imagine some of the knowledge would materialize via osmosis at the very least.
The average person does not even know what ChatGPT is and has not interacted with an LLM ever.
I don't think this is true. Certainly not in Britain.
I went to 3 weddings last summer and each one had a joke in a speech about using ChatGPT to write it and everyone laughed. 68yo father of the bride is a retired plumber and even he's cracking jokes about AI.
> I've seen in first hand that people struggle installing Steam. Yes, "people" in the plural. But just because some people struggle with it, it doesn't mean that installing Steam isn't an objectively easy task. Your friend's experience doesn't change the fact that building a website is something that an average person can do in hours if not minutes.
I think I would define "easy" with reference to the % of people who can do it. I don't know what that is for Steam, but there's a (now dated) survey of computer literacy in OECD that I keep coming back to in order to set expectations for what "average" looks like:
https://www.nngroup.com/articles/computer-skill-levels/
I have counter evidence of this. My wife with absolutely zero skills in computers was at the terminal doing things that Claude was telling her to do and generated some impressive tools. One was a tool to help her organize her day. It involved scripts, PDF generation, printing. She did it all without even asking me. Maybe it fails in some cases but I'm not sure the anecdote above is the average experience.
did she try saying "I want to make a webpage"? Even ChatGPT will just build, deploy and host a webpage for you with that request. I don't really understand what system she must have been using.
Having a similar experience. Watching a muggle try to build a website with basic functionality, e.g. auth, database, etc. is an enlightening experience. They simply do not have the vocabulary to guide the LLM. My good friend calls me every night frustrated with his results, and when I watch what he is doing it's amazing what we take for granted being in the software industry. Don't get me started on the UX, that's even more mind boggling.
I've seen some designers and product managers get pretty far with LLMs but mostly because they already know how to build apps just from a non-technical perspective.
I have been pondering this and I think it's likely a gap that will get filled sooner or later.
Right now there's just so much value in building LLM tools for experts that everyone is focusing on that. But surely at some point we'll have bespoke harnesses that exist exactly to solve this kind of thing.
I think this can start with constrained problem spaces like "you are a WordPress developer, you solve problems for people with enough expertise to know they are looking for a WordPress developer" and incrementally expand from there. Maybe I'm naive but I think you can probably get pretty far with this today just by writing loads of skills and picking the right technical preferences to encode in them.
Yeah there's lots of companies out there that promise that people can build a website quickly, often using WYSIWYG / drag / drop interfaces, I bet they already have AI integration to speed things up. ("I bet" because I don't actually use those services.)
I think people are better helped by using those services than going a level lower and using LLMs directly.
We already have these, and have for a while. Replit and Bolt exist for exactly this case. Not sure why GP didn't direct their friend to these tools (or if these tools are somehow the ones that failed in this example). In my experience, Replit and Bolt (and other similar tools) are quite good at this kind of 0->1 kind of thing.
Reminds me of watching someone who has no idea how to use a search engine try to use a search engine
I am reminded of this one:
https://thedailywtf.com/articles/Could-You-Explain-Programmi...
The comments feel so outdated:
'him: Keywords? Variables? ...
me: (Explaining what programming actually is)
him: Oh, I thought I would write something like: "Create football stadium and football players. Start the game when user presses spacebar. Make players have red shirts and white socks."'
They are from 2008 is why. lol
Yes, but usually programming knowledge does not outdate so quickly. Now we actually have prompts, what some people always believed that it always worked like this.
Weird because I had a friend that wanted a web app, also no experience, and he just told Claude (on the iOS app) to make it for him and Claude just made an artifact and put everything there.
Buddy sent me a share link and he was just like “dude this is crazy I just told it what I wanted and it just spit it out in a few minutes”. No “experiment” needed he just did it because he knew Claude could do it and it worked exactly like you’d expect. Hell, he did it on the Free tier.
I just gave this a quick try with Sonnet 5 Medium [0] on a free Anthropic account. It's a bit of a contrived example I guess, but it probably isn't too far off what a completely non-technical person seeing this for the first time would do.
The output is EXTREMELY misleading, as all the data here lives purely locally, yet the AI says that you can "just share the link" and other people will see the schedule you set on the generated artifact. Also, what link? To the Claude chat? It doesn't explain what to do with that `Booking` artifact other than "link to it".
I can so easily see someone tapping out a few steps down the line of this once they realize it doesn't work and they have no clue what to do or say to make it work. What do you even ask as a non-technical person at this point? I guess they could explain "The other person doesn't see it", but would the AI actually clarify that it's because it's not fucking hosted anywhere and has no mechanism of persisting the data outside of the current machine, or would it - as I'm almost 100% sure would be the case - not actually point out this error?
[0] https://claude.ai/share/0cbfe698-3886-4d4d-86e4-7c697b61dc00
I've seen the same thing happen to my brother when he tried making an app with zero experience. Only difference was that he got a front end that didn't work.
same thing as people just entering a question prompt and copying and pasting the response as gospel. e.g. politicians using it to write speeches, or lawyers for testimonials.
the output is programmed to look correct so unless you have some sort of background you won't actually know what errors to look for.
And look correct is accurate.
Not only does it look correct, it looks correct with an extremely Subject Matter Expert degree of authority. Often I'll work with an LLM, and it simply just misses so many things. I've worked in all sorts of different domains, software, chemistry, material design, everything from power generation through to physics, and in each and every case I see it missing incredibly important things. Any true subject matter expert would immediately bring up and prompt concerns, but not the LLM.
This makes sense, of course, because these are language models. They were trained on language. Their first and foremost capability is language.
An LLM's true expertise, true subject matter expertness is language.
And so anyone working with LLMs who isn't already highly skilled in the field they're asking questions about, will invariably be led astray and miss extremely important parts of a puzzle that need to be solved.
It's just like working with engineers.
...didn't have the vocabulary...
That is exactly the key or the sign there. Even with a couple of decades of engineering expertise, when I try to do / research something that I don't know enough about, I find myself in the exact position of not having the vocabulary.
To the point that I sometimes have to ask the AI "nicely" to cut the pleasantries and be ruthless against nonsense, whether from its/their side or from mine.
Interesting, thanks for sharing.
> Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode.
Even as a developer, when I've been using the web chat interface for things which I know the AI can do easily, I've had this happen to me a few times. I was very surprised the first time I saw ChatGPT respond ~"this would be a few thousand tokens, I can't do that".
Even more surprising: ChatGPT was accurate when responding that way this time, despite this being trivial for Claude and well within what ChatGPT could do using the web chat interface 6 months earlier. The ChatGPT output was extremely meh.
Question is if the outcome was LLM related or just a result of a personality trait. After all, talking in circles without really approaching an end-goal is something certain humans do all day long.
What terminology is required to make Claude Code write code?
My experience with friends has been the opposite. A PM friend made a custom tool. A friend who has never written a line of computer code has an app.
If you tell Claude Code "I want a website that does X, Y, and Z" it will write code.
Especially if the path is well trodden.
"Make me a kanban board for construction tasks. I want you to walk me through the process of hosting it" worked for me.
I’ve had the absolute opposite experience with a friend of mine. I started by setting her up with a terminal emulator on the web linked to Claude Code, and written a CLAUDE.md that told it how to deploy. These days (with no further intervention by me) she’s running Claude Code natively on her laptop, and she tagged me yesterday on Facebook in some update about how much more she enjoyed using Claude Code than plain Claude.
copying and pasting code? a few months ago?
2025 called and wants its test back
Exactly, sounds to me like OP is the one with the LLM skill issue.
So maybe your friend need a role reversal: A prompt guiding the llm to act as a consultant, guiding people in implementing a software project, asking questions, creating a shared understanding, limiting scope or creating milestones.
That seems like something that could be done using an llm, not that complicated probably.
And maybe, in other fields as well.
The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot.
People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a replacement for their minds and their senses will struggle.
One of the Claude skills I made Claude itself generate was the 'learning a concept across tiers' skill -- from ELI5 level to a PhD level, and it triggers whenever I ask it a very general question on a complex topic that isn't my bread-and-butter. The fact that I'm able to choose explanation level from a super smart LLM (that's available 24x7) that can explain any topic under the sun would've been mind-bogglingly sci-fi-ish just 4 years ago in 2022.
When I get out of my domain, I always ask it to describe things to me "like I'm a senior CS student who doesn't know any industry terms". I find it really easy to skim past the definitions I know and very useful to get the explicit clarification on unfamiliar terminology. Telling it to talk down to you a little also seems to calm down it's tendency to talk like it's trying really hard to convince you it's smart like a 8th grader trying to use every single vocab word they learned last week.
I've been using a prompt that comes from the opposite direction for non-programming stuff: "Assume the user is an expert in all fields; while this is clearly logically untrue on a literal level, the user prefers to see a field's technical terminology and then ask the AI about terms the user does not understand rather than get an inaccurate statement about some issue."
Whether you have to reassure the LLM that this is obviously untrue, I don't know, but they do have a knowledge baseline to know it's not true and I have a sneaking suspicion it would be less effective without that.
This has ended up in some of the most interesting incidental knowledge exploration I've ever done. A recent example is that I was asking about some stretches and it started talking about how useful they are for the sarcomeres, which I had not heard of. Now I have.
I'm not saying this is better, just that it is different. I think there's a time and a place for both approaches.
Off topic, but stretching is absolutely useless for ... stretching. And vital for retaining and enhancing usable range of motion (not static stretching, still, but dynamic stretching / mobility under tension).
The wildest thing is, there's no evidence that I can find that static stretching does... anything? It increases pain tolerance through the range of motion, but not any more than just asking people to try harder etc. And it doesn't build up, so after you stretch once for a given day, you're done.
Edit: after looking into it a bit further there's actually a cochrane review that stretching does bupkis, fascinating. It's also the first "Good" evidence graded cochrane review I've seen.
I think you probably just don't like to stretch.
Can you share the source? This seems dangerously wrong.
I see a specific review making the very narrow claim that it can have detrimental effects on power/strength activities immediately following static stretching.
But by and large I'm finding a great deal of evidence for a wide range of other benefits, particularly in range of motion and injury prevention. I can't find anything widely damning, and I certainly don't see any reviews contesting the validity of the very vast body of research supporting the many benefits of static stretching.
https://www.researchgate.net/publication/377653878_The_benef...
Most of this comment is not true. Static stretching does increase flexibility over time - it’s not the most efficient tool for it, but it’s certainly effective.
Presumably the Cochrane study you’re referring to is “Stretching to prevent or reduce muscle soreness after exercise”, which as it’s title suggests, investigates a narrow question and does not support you claim that “stretching does bupkis”.
"And vital for retaining and enhancing usable range of motion (not static stretching, still, but dynamic stretching / mobility under tension)."
I'm doing it for knee pain. It's possible it's a placebo because there is a slight strength training component to the whole thing (it's several exercises), although the knee pain has persisted through a lot of strengthening of my leg in general. This falls under "don't care" as the program as a whole works and I'm not worried enough about the details to try to optimize it. All the stretches fall under "dynamic stretching under tension" so maybe it's not something covered by the study you reference last.
It’s not a placebo. I’m not sure exactly which Chocrane review the GP is talking about but if it’s the one I’m thinking of, the headline is that stretching doesn’t help muscle soreness [1] (delayed onset muscle soreness specifically).
You should absolutely keep stretching to help with your injury, and there is decades of “sport science” to back that up.
[1] https://www.cochrane.org/evidence/CD004577_stretching-preven... - there’s another one about joint deformities which is far more interesting, but less relevant to this discussion
I can tell you if I don't stretch, my muscles feel bad and stiff. No doubt this is partly because I did karate for a few years, where stretching to increase your range of motion is absolutely required. You're not going to "just try harder" into a high kick.
That's very interesting. It kind of mirrors the experience of having a very smart friend.
Edit: https://xkcd.com/2501/
Sometimes when I want AI to explain something technical, I say "explain it like I'm a junior engineer" -- just to get it to start with the high level like a human being would.
This was part of my system prompt for a while, but the LLM would just make forced, unhelpful CS analogies.
> People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a replacement for their minds and their senses will struggle.
Claude code is entirely vibed. Someone posted some of the prompts they used: barely comprehensible typo-addled half phrases.
This seems to be the opposite of your experience.
I am sure some variant of the infinite monkey theorem applies. Anthropic devs have unlimited access to Claude, so they are no doubt lazy when it comes to prompt writing and just brute-force the agent to do what they want.
This was true some time ago but nowadays I don't get this impression. Seems like whatever style I type in, the LLM is already pre-prompted to respond in "its" "own" "style".
As an unapologetic generalist[1] this has also been my experience. Many tools that would have been "eh maybe if I get bored over Thanksgiving holiday" have become "hold on, gimme fifteen minutes".
Tiny, isolated, but awesomely useful CLI scriptlets, for me, seem to be the sweet spot. Little shining rays spreading out from the veins of my own familiarity.
The downside, the Achilles Heel of LLMs, so far as I can tell, is using the system to assist in maintaining large, sprawling, and largely pointless legacy codebases. Somewhere you have to keep many many many stupid things alive. I swear I can almost sense Claude's frustration with some of this shit. Then you get frustrated, and then Claude wants to agree with you so it acts even more frustrated, and the gyre thus widens. You're just cussing at everything with a machine. Which can be fun - Claude is often surprisingly funny - but not productive.
[1] I know absolutely nothing about positively everything, but have the attention span of a squirrel.
I've found Claude Code absolutely amazing for the sorts of 100-500 line data cleaning/analysis/visualization tasks that used to take me a couple hours to knock out. They're often self contained (boss wants a graphic for a slide or some numbers), and I tell it which packages I would prefer it to use.
On the other hand, I've been using it to make small changes to a ~4000 line codebase, and it takes a lot of wrangling to keep changes in scope.
Today I'm translating a 5,000 line VB6 codebase to C#, and I've been spending the day chopping up the job, passing it to Claude, and manually validating it.
I went to VB.NET first using a previous generation of LLM's (that was quite manual back then) and then from VB.NET to C# or just keeping the VB.NET around worked very well. The code was not highly complex but more than just CRUD. The porting from VB6 to VB.NET included building tests which helped.
Your LLM is not "smart" and it cannot "explain", it is a program that can output a statistically average text of an explanation. Don't mistake the capabilities of these models for intelligence.
It explained something to me today. So your point is disproven. What is intelligence then if not explaining things. They can be intelligent without consciousness if that’s what you mean
LLMs by definition cannot learn, understand, make judgements, or have opinions. If you find the model to be a tool that is useful to you, by all means. But do not anthropomorphize.
[0] https://dictionary.cambridge.org/dictionary/english/intellig...
You're just talking semantics. To me if I read text that helps me understand something, that's an explanation. The black box it comes from is irrelevant.
LLMs do learn, understand, and make judgments. You're just pushing back on the underlying mechanism, which instead of reason/sentience is instead statistics and weights. I'd call that intelligence but I don't really care if you want to call it something else.
The amplifying mirror is not just a characteristic, but the fundamental driving force of LLMs. With every token it consumes, its primary goal is to understand who you are and what you intend. If you type Hola thats sufficient to tell it so much more than if you type hi.
LLMs do not have goals and cannot understand. Please don't anthropomorphize.
Mind sharing the skill?
ELI5 is the way to go. I have Claude break down high level physics "as if I'm a farmer standing in a field" - works beautifully.
I tried that but Claude thought I was already outstanding in my field.
You literally made me Laughing out Loud :D
> I have Claude break down high level physics "as if I'm a farmer standing in a field" - works beautifully.
Careful now: https://www.youtube.com/watch?v=TMoz3gSXBcY
There's an irony here in using Gemini to summarize this 40 minute video right from the page.
What about breaking down farming as if I'm a physicist standing in a lab?
Farming is physics, silly.
Does it? ChatGPT had no problem telling me that dark energy is the Higgs VEV.
I am really curious about that skill now, would you mind sharing it?
That sounds useful. Can you share that skill?
Not the OP but you can whack this into your prompt and get most of the way there: "no jargon goes unearned, nothing gets dumbed down, every abstraction touches ground"
A punchy tricolon containing 2 analogies that don’t quite make sense. That’s some S-tier AI-mimicking. Nice!
One could say that it being an LLMism is...load-bearing :)
Here's the kicker: it's not just load-bearing — it's a whole new paradigm support column.
And honestly, it's amazing.
Clean.
One I got "That settles it, and it flips the picture"
well not very settled then is it
I have been using this project recently to make myself tutorials on things I want to learn: https://github.com/devenjarvis/lathe
>learning a concept across tiers can u share this pls? thx.
This is something that really needs to be formally studied.
I'm inclined to say that this matches my own experience, but I can't rule out confirmation bias on my part.
As a meticulous person generally looking for a very specific code outcome, I prompt in a way intended to get exactly the thing I have in mind, and my results reflect that. But on the other hand, I have coworkers who type ten-word prompts with very limited specificity, and they seem to get results that way as well, and that makes me wonder.
It would certainly be beneficial for my career and financial well-being for the assertion to be true, because it means I don't have to worry about being pushed out of my job by an army of $15/hr vibe coders. But the convenience of that assumption is exactly why I think it's important to be skeptical.
Meanwhile all I do is vibecode. I get the outcomes I want though.
I see vibe coded apps as requirement documents. Rarely do I have to engineer.
If my job gave me some actual tasks, then maybe I'd engineer something. But at home? Vibe coding all the way. I'm open to engineering, but I need a compelling reason such as: the app is fundamentally broken and an LLM is going in circles. When the only user is me, there are not many performance issues to think about or fix, so that helps. Moreover, certain systems don't need to exist (though they might soon since now I have a smattering of apps that I need to manage).
I suck at writing code, so strictly speaking: Everything I do is vibecoded.
The stuff that I produce in this way would probably be considered by many to be unusable trash. But it solves the problems I have, and it does so with exactly the amount of precision that I demand.
When I built a PWM fan controller for a pro audio amplifier, I was very particular about some aspects. I wanted maximum resolution from the DS1820B temperature sensors (which is a relatively slow mode where reads take ~750ms, and often the bot is primarily interested in fast), and resolutely-consistent PWM output (so software PWM was a non-starter).
It was very important to me that the fan speed ramp smoothly and without audibly-discernible steps, so the target output goes through a low-pass filter to smooth things out and the final PWM value gets recalculated at a completely-overkilled rate of 1KHz.
Power consumption was a very deliberate non-concern: The power used by the MCU is ~nothing compared to that of the whole of the system, so optimizing towards reducing it was never my goal.
At the end, it's a rewarding little project that is all wrapped into a state machine that burns clock cycles like they're free (they are free!), and it works very well.
There's parts of this thing that I do not understand at all, and that I have no desire to understand.
But if I hadn't been so particular about the parts I did care about, then: An underspecified one-shot prompt seems like it would probably have just produced a loop with a lazy 1-second sleep at the end, since being sleepy and power-efficient was a feature that the bot kept working to reintroduce.
I spent a lot of time working to dismantle the bot's proclivities to be this way, and I probably would not be happy with the end result if I had just let it do its thing.
Differently-stated: It could have been an unsupervised one-shot prompt, and the result almost certainly would have done the job of keeping the amplifier cool. (I just would not like it.)
Both the article and some of the discussions here share a lot of commonalities with doctors taking a medical history. There is a certain skill in guiding the conversation towards useful outputs, while not dictating the exact outputs to a patient who is eager to please with their responses.
E.g., medical history taking protocol always says to start with open ended (albeit structured) questions, and converge towards more closed/specific ones when you're sure you've extracted the broader surface and you now want to close in on a differential diagnosis.
If you start open and go with the flow but then just let the patient talk without any structure or subsequent attempt to converge, there's a risk that the patient might spend 60 minutes taking about their fluffy dog at home, which wastes time, and doesn't get you anywhere nearer the diagnosis. But, if you skip the open questions and go straight to yes/no diagnostic questions, you will definitely miss the fact that they have a dog at home that they're worried about, and that they'll be self-discharging against medical advice in the next hour to go tend to their dog.
So while to an outsider, the conversation might look effortless, in reality the doctor requires considerable skill to be able to strike a balance between open vs closed prompts, as well as the ability to critically sift through the outputs, and decide which outputs are relevant to pursue further and lead to a fruitful direction, versus those that can be safely discarded to remove potentially distracting noise from the conversation (and all while attempting to keep this interaction within a limited number of prompts due to operational time constraints).
Counterpoint, some doctors will zoom in on the most likely problem and misdiagnose. This is in part due to pressure on the health care system (where I live anyway); you can only get a GP appointment for 10 minute blocks, which really isn't a lot.
But when a 30-some year old shows up at a rheumatologist with joint pain they will likely go to unusual (at that age) but not unheard of rheumatism/arthritis, not hypermobile spectrum disorder. When a woman goes to a GP with period pain they will be prescribed mild pain killers or anticonception pills and fobbed off, until a decade and much suffering / many more issues later they get diagnosed with endometriosis.
Indeed, I didn't mean to imply that all doctors possess that skill in great amounts (or that, in any case, circumstances allow for full utilisation of the skill even if they do) or that medical history taking is therefore always of the highest standard. Obviously some doctors will be better at it than others, or more at liberty to use this skill than others (as you say, I could imagine a GP who only has 10 minutes per patient won't be as keen to start with fully open questions as a junior doctor in a ward who can clerk patients somewhat more freely).
Note that this isn't too different from, say, how software engineers are expected to be good at, and make good use of unit tests. But most probably don't (either because they never really cared to fully develop that skill, or their organisations applied contrary pressures leading to tech debt). But it is a recognised skill.
My main point was that, it is, in theory, a skill that doctors are expected to train (or at least pick up on during their practice), and therefore the same prompting principles that seem to apply here in the context of LLMs also interestingly seem to apply to medicine and history taking when "prompting" and interacting with humans.
You can’t run expensive, time consuming, and potentially harmful tests on every patient. Someone comes in with a headache, you tell them to go home and take an aspirin. You don’t send them for a a CT scan.
If it doesn’t go away and they come back, you start considering more serious issues.
It’s expected that uncommon non-emergent diseases will be diagnosed over multiple visits.
I think endometriosis is common enough that it would be caught earlier ("Oh, does your joint pain get worse right before you menstruate? Tell me more"), but your point is reasonable.
I counter with the platitude that common things are common - especially in fields like primary care, the amount of wasted effort one would expend in pursuing unusual explanations for every presenting symptom is considerable. We thus have to examine patients over time and trust that they will tell us if things have indeed not followed the course of the initial diagnosis.
While this is true as a platitude, I do find that medical colleagues seem to treat this platitude rather overzealously at times, often completely disregarding the posterior probabilities involved. I like to call this the prior-over-posterior fallacy/bias (though one could argue this is simply a variant of the normalcy bias)
E.g., I get the whole "if you hear galloping think horses not zebras" adage, but I guarantee you, if someone comes and says "hey when I was in Africa I saw a black and white striped animal galloping really loud", I bet you an uncomfortably large percentage of the "horses not zebras" crowd would still favour a horse over a zebra diagnosis, despite the overwhelming posterior.
Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests, and it becomes a big problem, whereby uncommon conditions are treated as common, thereby often escaping detection altogether, and driving down their apparent prior probability even lower, causing a diagnostic vicious cycle.
I hear the complaint, all I can say is that everyone I speak to about this wants to diagnose things correctly while not wasting anybody's time or energy on bad diagnostic workups.
> Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests
There isn't a way around this: if you order the test and a value is abnormal, you are now committing yourself to treat a thing. We should not be ordering tests if we aren't ready to follow their results to the logical conclusion. So I would disagree that this is a problem in the way you've framed it.
I'm not disagreeing with what you say here, but I think we're arguing about subtly different things.
E.g. you'll note my zebra example was not about whether one should additionally request a photo of the animal just to make sure it is indeed a rare animal. It was arguing that given existing differentiating information that points to an uncommon condition in the first place, one should not dismiss this on the basis that horses are still more common than zebras in the general case regardless. Under this uncommon information, the prior of thinking about horses should go out the window, and one should absolutely focus on zebras (at least as an additional differential). I assume you would also agree with this conclusion.
But of course, in real medicine things are not as simple as this contrived example. So the point I'm making is that, from what I have observed, there seems to be a bias towards decisions based on "prior" rather than "posterior" probabilities, even in the presence of additional differentiating information which should have prompted additional differentials to be considered. But this is different to saying one should constantly seek additional evidence to include or exclude additional differentials that are unlikely in the first place. That, I agree, would be a waste of time and resources (and potentially harmful for the patient).
Having said that, I somewhat disagree with the phrasing that we should not be ordering tests if we aren't ready to follow their results to the logical conclusion. This is a bit like saying, "I don't want to check for X because if I do and it checks out it will derail my current management plan"; but then this is putting the cart before the horse, since it's the findings that need to dictate management, not the other way round. I do think it is reasonable to say that one shouldn't be ordering a test if the expectation that it will change management is sufficiently low to justify the time/cost expended for it -- and where this expectation is a function of both the likelihood of the finding (given current information!), as well as its potential to change management. But this doesn't mean that if you do find an unrelated inconsequential incidental finding you are now required to divert all resources to it.
Conversely, if an incidental findings does prove worthy of treatment, then I don't see the problem with committing to treat it, as long as you don't lose sight of the original complaint either. Obviously that doesn't mean one should go looking for incidental findings willy-nilly though.
You'd think, but I have 4 friends who have been diagnosed, and they all have horror stories. This is in 3 countries, none of which are the US, but from reading Reddit, it seems the US has the same issue.
> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems.
In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasonable to actually apply to it.
This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise? Sure you can ask questions about the codebase, but IMHO that falls under surface-level information, and the devil is often in the deeper details. Hmm.
We are having trouble onboarding engineers with AI now. Some still struggle after their first year with very basic concepts/patterns we use and make the same mistakes again and again in their PRs because they just ask Claude to fix it and never internalize it. I think using LLMs feels good at first because you can get tickets out the door faster but you never develop enough knowledge to make a big impact or become an expert in the code or business.
These would have been good hires before AI but now they struggle?
I imagine they are still good hires - however there is a real cognitive debt that happens when we code with LLMs in my experience.
It’s harder to internalise concepts because you don’t go through the struggle of understanding them and finding the mental links you need to remember later.
I notice this with people around me - all of them are doing more things, but I am also catching more issues when reviewing docs and code.
Obviously YMMV.
My guess is they would take longer to ramp up but then have a better understanding of the codebase and domain.
But there’s a lot of pressure to get up to speed as a new hire and it’s easy to move fast with AI
Honestly you should probably invest in baking the basic patterns and concepts of your codebase into shared skills that are applied to everyone's Claude sessions.
Or documentation that everyone could read and refer to
I think this is only a partial answer but I've been surprised by how familiar dev leads become with the app even if they are not in the code.
They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.
They also review a huge amount of the code changes and rubber duck/plan/troubleshoot a lot
It's not for everyone, but I write all the code and then I ask the LLM for a review. It catches something minor thing every single time (out-of-date comments on a function or method, typo, etc.), something that's clear a mistake maybe a third of the time and, occasionally, points out something that looked fine to me but actually won't work because of some other code that I forgot about. I'm a person, I can't remember everything I guess!
I have to say on those occasional times where it finds something that I totally missed or misunderstood, those are for sure the most productive sessions. I find I'm actually working with the model, while I read the code it's pointing me at, and getting a good solution together. Often the model suggest something that's maybe too simple or, weirdly way way too complicated but it's definitely helps me zero in on a decent solution.
In the Linux kernel I've had a lot of luck with:
1. "find the code that does X"
2. go read that code
3. When you hit a bit you don't care about, go back to the model and ask it for the pertinent details
4. When you hit a really confusing bit, ask the model for hypotheses about what's going on. (I always phrase it as "give me some hypotheses" not "what is going on here". I dunno if this changes the output but I think it helps me stay in a mindset of uncertainty, it's important to avoid locking in any misunderstandings. Anyway I find the models do well at this task, and when they bullshit here it has a strong smell).
Before AI, parts 1 and 3 could be insanely time consuming, sometimes it felt like a infinite breadth-first-search. And part 4 was basically: either you find a human who knows the code, or you just make a mental note and hope that later on you find something that makes you go "oh, THAT'S why they <do weird thing that should 100% have a comment>!".
So yeah even though you're still reading code with your wetware the AI makes you dramatically more powerful.
This is also extremely helpful for unpicking undocumented API contracts. E.g. you can say "the x86 implementation of this API is safe to call under a spinlock, go read the other arch versions and tell me if that's true there too".
the devil is always in the details. those details are on every level you look at: human minds, nature around us, space. so if your inputs are vague, you should only expect outputs that are vague and generalized
I’ve found that planning tickets with granular details (like semi specific code changes needed) is one of the best ways to get that deep knowledge of the system. Even if ultimately I delegate most of the implementation to an LLM. I also heavily verify the changes, but I find that that’s less impactful than planning a feature / tickets.
Read the code.
What if the code sucks, because it was vibe coded by an LLM over a dozen disparate sessions?
If it performs the overall function but the code is a mess, then you can ask the model to describe parts of it, the structures and what the structures are for. When it shows you something that is messed up, suggest a better framing for the same thing and ask the model to implement the better framing without regressing on already passing tests.
I'm going through this right now on a very difficult to implement task, the original was python and very verbose. But had facilitated a rust implementation that produced byte identical outputs. Then I asked it for what data was being passed around, placing restrictions on what passed between interfaces I could tell it what parts should be immutable and what parts should have no presence outside it's local context. Placing those limitations while having a exemplar of what it should be doing gives it little choice but to make better code if it meets the conditions set while at the same time not regressing.
I say things like 'this field is a implementation detail in a declarative data structure, it should not exist here.
This can mean hours of work with no observable change in program function, yet it is directly addressing the limitations that prevents it from being used in larger tasks.
This sounds like coding manually would take less time.
I think you would be incorrect. While AI could not write what what I have so far on its own. Neither could I.
But coding manually would use fewer tokens.
Then that's the point. You know the code sucks, the guy who vibecoded it either didn't know or didn't care. That's the added value.
But how would you know? If you're an inexperienced developer you may well end up ingraining bad patterns without even realizing it.
Yes, how does the LLM make a difference there? Inexperienced developers write bad code. Hell, experienced developers who are outside their domain write bad code too, then you later realize that it's hard to read, or that it leads to architectural problems or such, and hopefully learn from the experience.
In the context of what I was replying to:
> This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise?
Learning from LLM written code is significantly and meaningfully different from struggling on your own or learning from more experienced human co-contributors. Especially in a large, complex, iteratively developed codebase.
Worse, once you're in that situation you are now at the juxtaposition of: "I did this. I understand what my reasoning was, and now I know why it is wrong and how to fix it" vs. "An LLM did this, I don't know why it did this, I'm not sure what it was trying to do or what pattern it followed and I'm not sure how to make it better because I didn't write or understand the original implementation either"
Is it impossible to learn and gain experience this way? Not at all. But it's definitely not equivalent.
Claude, make this codebase less ass
That will actually work pretty well.
I legitimately caught Claude calling things "ass" while a friend was using it earlier, which I think is pretty funny.
Then fix it using the llm
Reading it is good, but working with it more directly seems to help retention.
> If we're pushed to delegate most work to them, how do you build that expertise?
Have you ever pair programmed with someone? It's the same idea. You can be an active enough participant in the process if you wish to be and can be just as knowledgeable even if some of that knowledge lies in transactive memory. https://en.wikipedia.org/wiki/Transactive_memory As long as you have the map, and the map to the map, you don't need to retain every fact about the landscape.
This reminded me of Gaussian Processes. You start out with n-dimensional unconstrained (but strongly correlated) gaussians. As soon as constraints (data) are added (mathematically it's called conditioning), the thing goes more and more into shape.
Prompting feels a lot like this conditioning phase to me. You start with an LLM in unconstrained mode, basically just a "soup" of knowledge. If you prompt wisely, you immediately condition the LLM into "your space of (domain) knowledge".
What comes out is an extended version of your existing knowledge.
Yes I love this comment! GPs are awesome.
Prompting is conditioning, that is what it is. The visual of a GP (like the thing you get if you google image search “Gaussian process”) is a great metaphor for what prompting an LLM is doing.
The output of the LLM is the logits which is sampled - plucking out tokens from a distribution. The input of the LLM is data which constrains the logits. That is what it is.
That’s also how you know that AI will never “solve” intelligence (the way the boosters say it will) without some general mechanism for this conditioning process. The ultimate mechanism would be embodiment; the crappy mechanism we have now is something like openCLAW.
It's this https://youtube.com/shorts/OPVp_FDEkdA
I do find that "signalling expertise" is important. "I have a significant background in biblical scholarship. You can assume I've read the most important works in NT studies in particular. Do not translate Greek, Latin, Hebrew, or Syriac. Now, I would like to know . . ." That changes things significantly. So does telling it you have 20+ years of experience with C programming, that you have a robust understanding of machine organization, memory layouts, embedded systems, etc.
For sure. On a personal coding project I said "I'm a professional software engineer, and while this is a hobby project I'm not just vibe-coding and want to build reliable software" and the agent suddenly started suggesting all kinds of things to make its code more robust.
LLMs skew toward over-focusing on things that you mention.
The reason "the agent suddenly started suggesting all kinds of things to make its code more robust" is because you said you "want to build reliable software".
It's not a signal of good judgment or understanding. It's just how LLM attention works.
I thought exactly the same at first. But then i wondered if that still holds true with today's advanced thinking, RLHF involved, frontier models. I guess to a certain extend it did indeed behave better, as a reaction to his self description into account.
EDIT: I mean, those systems accumulated so much complexity around the attention based next token predictor.
Yeah, in my experience, there's nothing about:
1. LLM thinking 2. RLHF 3. The latest frontier models
that does anything to change this fundamental "suggestibility" of LLMs.
But who knows, maybe I'm wrong.
Training the LLM to do things that the user didn’t explicitly ask for is a good way to get complaints from the users. Doesn’t matter if those things are best practices.
>build reliable software
this feels like "make no mistakes" level of prompting. reliable software isn't as simple as making it reliable, it's about choosing the trade-offs in the areas that don't matter as much as the areas that do. if you keep prompting the LLM to make your software more robust it will keep giving you things to do. they aren't all good things. eventually you'll end up needing kubernetes to run a calculator app.
Not sure I agree with this. The math guy at anthropic's prompts are essentially:
Tao's chat was for him to gain intuition, not to solve the problem from the outset.What's funny is that every other person gets a different conclusion about who these models reward/empower. I've seen people say that the generalist stands to gain the most and others say that it's the experts. Like all of life, maybe the "winner" is the person who just does stuff.
It depends on the levels. People with differing fitness levels and ages run at very different paces. Now, do cars make them more equal or less? On the bottom end, the tide lifts all boats. Most healthy people can learn to drive and will drive "fine", they get from A to B. Out there in the city streets the car flattens the differences, everyone roughly takes the same time to get from A to B in a car.
But at the top of top, the gap probably widens. A professional F1 driver will drive laps around some random guy. It amplifies reflexes etc, because at that speed little differences in timing make a big difference.
Now, AI coding isn't exactly analogous, but I think it also has these two regimes. It flattens things for simple tasks. If your task is to shovel data, do some trivial compiler wrangling staring at badly designed error messages, looking through GitHub issues hunting for the comment with many tadaa emojis to fix an issue etc, those things can now be done by anyone. Just as grandpa can also drive to the grocery store. But if you're pushing at things on a higher level, now only your above-AI ability matters. If all the things that AI can do well are subtracted out, how much other expertise do you have left? This will be proportionally a bigger and bigger difference between different people.
So you're saying that it depends whether you're measuring "success at task X" (where in average everyone improves) vs comparative success (where people with knowledge can do far and away better at specific tasks).
In short being a high agency person is now more important than ever.
Exactly. The post read to me as another variation of the denial that people with expertise are reaching for right now. My sense is that we as programmers went through it over a year ago already (perhaps not all of us, but at least anyone paying attention), and so it's easy to overlook that it's still new to people who do other forms of "knowledge work," i.e. people whose identity is bound up with their expertise.
My working theory at the moment is that for programmers it was relatively "clean" and took the form of an inside-out transformation of the work, where AIs directly produced the central work product more or less adequately and relatively early on, but for other forms of work it will appear as some mixture of inside-out (in which case it will appear similarly first as a tool, then as something more than mere tool) and outside-in (the things surrounding their work and the supports their work processes rely on will be progressively automated). This is going to give rise to all sorts of pathologies in the white collar world, we'll get all kinds of variations on denial/negotiation, and so on, until it fully transforms the division of labor.
One interesting point of reference here: Yuval Harari gave a talk recently about the radical changes that will take place relatively quickly, in which he noted the AIs are not quite as good at writing as he is yet, although he expects they will be relatively soon. He then gave the timeline for what he considered "soon": 10 years! So we find the denial ("I still have time, they're not as good as me yet, maybe in 10 years...") even among the most vocal "prophets," among those supposedly most wised-up to what's going on and where the capability frontier lies.
Have you considered the possibility that you're simply not as good as the experts, and that your experience of LLMs being capable of performing your work up to your standards doesn't imply that experts are necessarily in denial?
Being fairly at expert level in a "solved" domain (for some definition of solved) but also having near-expert level proficiency in a non-technical as-yet "unsolved" domain and watching the process repeat there (and watching how people react as inroads are made progressively deeper) is basically my own standpoint. But I am intentionally using "solved" and "unsolved" very loosely here: you can get stuck in the thicket of arguments about verifiable domains, what it means for a domain to be solved, whether a set of evals can tell us something has been solved or not, and so on. One way to avoid that (as my original comment pointed to) and focus on what matters for us is to look instead at the effects being produced in the work process and on the division of labor as a whole.
What sort of denial are you reading into the author's post? I don't read it as denial. He is fully acknowledging the usefulness of AI.
He's saying: "Expertise improves AI-assisted work, experts can steer better and get more out of models."
The obvious conclusion for anyone is "therefore experts will remain the indispensable and specially rewarded center of the production process."
That this is appearing exactly now, and in this form, strikes me as extremely suspicious. I don't doubt the author's sincerity on the surface. What I suspect is that anxiety over the possibility that the (unstated) conclusion might be false (!) motivates the argument in the first place.
I'm asking the question, "Why is this argument appearing now?" At least one reason seems to me to be, "because we're afraid of what the world could look like if it's not true."
Okay, I get your point.
However, I personally agree with the author and I don't think his argument is necessarily motivated out of an anxious fear. On the contrary I think it may be motivated out of a sense of extreme exhilaration and empowerment.
Because experts (like myself as a programmer for 15+ years) who are using AI in many fields are suddenly empowered and much more useful than we were before AI. My employability and value has gone up and not down, precisely because of being able to apply my expertise with AI, which people without expertise simply cannot do. I am a professional programmer and also owner of my own startup.
Let me give you a concrete example that I am dealing with at my startup. I'm a small business owner. Before AI if i wanted to produce production quality video for marketing it would taken such a huge budget and such a large team of people (or an expensive agency) that I wouldn't even have considered it due to the enormous cost. I'm talking about Apple quality video production which takes millions of dollars to produce.
Not anymore. A single competent person with AI can replace an entire marketing video production department or agency. But expertise is key here: knowledge of film terminology to be able to describe the effect you want, and ability to use video editing tools effectively, as well aesthetic taste. I as a programmer with no filmmaking experience don't even know how to write the prompt which makes the video that i want because I don't even have the terminology. But a person with that expertise has suddenly become more employable and more valuable to my business because I as a small business now have the capability to create Apple quality marketing videos.
So AI actually created a new job for an expert that would have otherwise not existed because it was outside the budget of small businesses. Previously somebody like that would have been employable to only a few large production studios but now they become employable by almost any small business.
Did you maybe consider that the 10 year timeline is not denial, but actually well educated reasoning based on Yuval’s experience and understanding of the problem space? You shouldn’t dismiss people’s thoughts as denial just because they don’t match your perspective.
I would guess if asked Harari would actually revise that lower. The point was: even those arguing most strongly that AI is an autonomous historical force can slide back into the mere abstract recognition of it. In context, Harari was saying "I'm speaking to you as a writer now, but my own standpoint will eventually be undermined." I'm saying "yes, and your 10-year timeline suggests you along with all of us are not taking your own thesis seriously enough."
This works better for math because math is self-verifiable. Once you have a proof it needs no outside evidence.
Expertise is needed to evaluate model outputs where it can't verify itself, or at the very least one's expertise can help steer the model in the right direction.
However this is irrelevant if models themselves are better at evaluating/leveraging expertise/information.
Corollary to this is an important part of LLM usage is what I call pinning it to reality. That is, designing verification steps that interact with the real world in some way not easy to hallucinate or work around. This means things like having code that interacts with the physical world, round trip tests, arriving at the same result using different paths, interoperability / replication with external libraries / competing products, performance improvement projects that start with robust performance test suites, and similar sorts of things that reduce to "how do I provide evidence that's difficult to fool myself about".
This includes things like "before you start fixing this bug, write two tests that fail proving it exists".
Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.
But the guy who writes the “just do it” prompt can neither formulate the conjecture in the first place, nor come up with any follow-up questions to build on the result.
Yeah, the people who say no expertise is needed for these things confuse me somewhat. This is indeed the case if you want to be a meat wrapper around an LLM, understanding neither your inputs nor your outputs. But at that point, what is the point of you versus going to the LLM myself? Expertise is necessary because it adds understanding and structure to the blob of text produced by an LLM. Progress can only be built on such understanding.
I am tempted to say (uncharitably) that the 'No knowledge needed! Just add LLMs!' byline is wishful thinking by non-experts who do not want to confront the reality that they will ultimately need to learn things.
> But at that point, what is the point of you versus going to the LLM myself? reply
You're missing one word, and that word explains why everyone running the companies is so excited. The word is 'paying'. "But at that point, what is the point paying of you versus going to the LLM myself?"
And, yes, I think that LLMs make it a lot easier to hire a minimally trained stooge and get them productive. It's worked for me, and I appreciate being able to pretend to be productive and walk away from the job a bit early every day. I don't think software engineering is likely to be a high status, high income job for very long.
So how many conjectures have you proved in your spare time?...
As the old joke goes, a mechanic charges you $5 for hitting it with a wrench and $495 for knowing what and where to hit.
It's not contradictory to say that expertise is a multiplier, and that models are systematically underconfident in themselves.
It's actually refreshing when a model is sure about something because it actually tested it and has the receipts. Opus 5 seems really good about testing its own knowledge with experiments. Scientific method ftw.
Hacker News is the biggest collection of idiots who think they are geniuses, so they will upvote anything that makes them feel smarter.
Who's end state took / is going to take more tokens / money, however?
"LLMs reward expertise" is the title, not that "LLMs only make things possible for those with expertise"
And what about problems that cannot be one-shotted but helped along?
There was one math proof that was AI generated going around Twitter and the chat transcript to generate it was basically the human prompting "keep going" until it solved it.
Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.
Yup I linked that in my other comment but copy-pasted here for everyone's convenience:
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample"
https://x.com/DmitryRybin1/status/2079904005652893709
https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
Yes, we're in the low hanging fruit stage where folks can just say "here's the problem" and "keep going" until a result is found and it will work sometimes.
The low hanging fruit will run short. Ultimately mathematics is a field of subjective selections of problems and proofs as beautiful and interesting. Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof.
> Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof
And why do you think this would be the case? I'm not talking about today but in 1-2 years. For reference o1 was released less than 2 years ago, and we've had reasonable coding agents for 9 months or so.
> why do you think this would be the case?
Mathematics is ultimately an aesthetic pursuit. Outside of a well defined goal ML models don't have any sense of taste and regardless of the scaling that's been enabled in the last year or so of capability if they haven't memorized the process of doing something they have the same limitations of inability to make choices about unknowns not trained into them.
Real synthetic intelligence seems to me to be still very far away and not a matter of making models bigger or more efficient.
While they may struggle on what to study, we'll be able to spin up millions of them.
LLMs are a collection of biases. Humans are also a collection of biases. So we project our biases as input through the biases of an LLM and get an output. Hence why I think getting optimal output requires being an optimal person. And in that sentence there are many points of expression.
Finally, we train our LLMs on who we are. Another reinforcement of biases.
> Like all of life, maybe the "winner" is the person who just does stuff.
Someone who just does stuff still has to be able to deal with errors and failures. That’s where an expert or a generalist may have an advantage.
> who these models reward/empower
The easy, straightforward answer is "the people who own the models". Who else benefits feels like a more complex question and we'll have to see...
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample" lol
https://x.com/DmitryRybin1/status/2079904005652893709
https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
I am not sure. Why do you need domain expertise beyond being able to draft a verifier for the problem? Once you have the verifier, it's just a matter of compute. You can argue that being a domain expert allows you to narrow down the search space for the LLM and save time/compute costs. This is partially true, but LLMs are getting better and better at search (they can already do end-to-end performance optimization faster than performance experts at a FAANG company I work at), and compute to maintain the same intelligence level is getting cheaper.
A concrete example: GPU performance optimization for a kernel. This was (and still is) a very niche domain with not many top-notch experts. But kernel performance and characteristics are easily verifiable. You can run the agent in a closed loop for it to improve iteratively (and people are already doing it, coming up with kernels better than human-written ones).
You see Tao's example because:
1. He is curious (so he asks detailed questions, which are not necessarily needed in a closed-loop optimization).
2. Verification in math is harder. Many math tasks used in RL are easily verifiable. But for advanced open conjectures that require long proofs, you cannot trust the proof directly from the LLM (so it's not as easily verifiable as basic math problems or code). The model needs to write it in Lean, and you still need to make sure the Lean implementation correctly captures the specification of the problem. So you still need a human for verification in advanced math. But I don't see why you would need this in domains like performance improvement.
The short version I give to non-technical people who ask me about whether "AI will replace coding" is this: it accelerates you. You can get much further much more quickly.
If you don't know where you're going or how to get there, or even if you're just not paying enough attention, it will get you very far in the wrong direction before you've realised.
> The model outputs are much more concise than when I try and talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians” mode, not “explaining-to-amateurs” mode
I believe this works in two different ways.
First, information compression. The use of professional language helps describe problems more densely with minimal information loss/distortions. Verbose output by LLMs (e.g. ELI5) tend to incorporate local chat context, which can destabilize the context (e.g. out-of-topic, irrelevant nitpicking on writing style and wordings) and lead to faulty logic and even hallucination. LLMs are not good enough to look through all the noise, so, sometimes, it's helpful to refine the input data before performing actual tasks.
Second, boosting logical pattern-matching. Using professional language helps drive logical reasoning through simpler pattern-matching b/w texts. This is not about whether LLMs can reason or not; it's about how high-level reasoning is guided by preconception. Even humans tend to consume only textual surface of highly complicated theories (e.g. Adam Smith's "invisible hand"), and use them casually during conversation. It's similar for LLMs: if the conversation is conducted entirely in professional language, LLMs can easily incorporate external professional information into its reasoning. If the text is written in amateurish tongue, translating it into professional language can introduce errors and distortions.
So, yeah, keep your conversation professional, tidy and tight. A large volume of unprofessional text helps no one.
This is why the chat interface is ultimately not the best option for non-expert users, because they require the user to bring knowledge with them. You can call this the “query” method: you have to know what to ask to get the answer you want.
A real world example might be: I can find any movie DVD you want from our warehouse, but you need to tell me the name of it. Don’t know the name? Tough luck.
Contrast this with a “browse” interface: the options available are presented to you, and you can pick from them. Relevant contextual information is already on-site. The DVD store has shelves of potential movies you can rent, and you don’t need to know their names ahead of time.
The interfaces of future AI will be more browse oriented, with a query viewer available in the settings for advanced users.
Picking a DVD to watch is a rather inconsequential decision. LLMs already do this sometimes, asking you to pick one of a few options, but without domain expertise you will invariably make worse decisions, but if all the n-th order consequences were first explained to you, that would result in you having built domain expertise, but also erasing most of the speed advantage LLMs give you. Moreover, you will never know about the options that are never presented.
Inevitably, this is the new tradeoff to make, above average quality comes from asking for more, and knowing what to ask for comes from expertise.
> In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet.
You could read some general reference/guide/tutorial documentation on CSS, and then probably solve your problem (without searching for "how to center a div", or whatever your exact problem was, and copy&pasting the answer and moving on), also becoming more knowledgeable in the process.
The rest of the short blog post has some good points, but the first sentence sounds like it's targeted at the percentage of developers who did StackOverflow copy&paste to close Jira tickets, never becoming experts.
Delegating to LLM-ish AI is just a natural evolution of that. The question is whether they can still add value if kept in the loop.
The article author suggests that the answer is to be expert, and is addressing people who... "either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet."
> You could read some general reference/guide/tutorial documentation on CSS, and then probably solve your problem
Hours + Hours of reading and a lot of trial-and-error. The loop was so long and sooo slow. Now it's instant. As if your very first Google search just solved the problem for you immediately.
>Hours + Hours of reading and a lot of trial-and-error. The loop was so long and sooo slow.
But that's how you learn...
I lost my desire to learn anything about web development twenty years ago and specialized my career in a completely different direction. I had not touched HTML at all since then until earlier this year, when I discovered that AI robots could do it for me. Now I am happy to whip up HTML/CSS/JavaScript visualizers, explorers, and even one-off interactive report presentations whenever they might be useful, precisely because I don't have to learn any of that crap to do it. My time & attention are far more productively spent focused on the work I am actually good at and interested in doing.
There was some learning, but also a lot of waste. As a self-learner, I've never been able to learn so fast as I can now with an LLM to instantly answer my specific questions, and incrementally correct and grow my mental model. And while with some subjects you need to worry about accuracy, LLMs are generally very good with programming questions, and (for most types of questions) you can verify their claims yourself.
The problem with CSS is it's a) all broad and no depth and b) constantly adding major new paradigms. If you don't get up to speed and stay there, you lose it.
Pre-AI I'd say I've gotten over the hump 3 times with CSS, only to lose it again by the next time I had to use it in volume.
It's inefficient though. With LLMs you can get a correct answer instantly for asking the right question, and if it's not perfect you can iterate on it. With your typical googling loop you wasted a ton of time on things you didn't need.
Why do you think thats worth learning? Doing a few times manually sure… but to force yourself to solely rely on it is foolish. Just use the calculator.
It is not the only way to learn.
We don't need to learn CSS anymore. Just like we don't need to learn ASM since GCC does a great job generating it from higher-level code.
I am a frontend engineer and I see so many bugs from vibe-coded CSS from people who think just like you. CSS is very powerful and easy to get wrong if you don't understand it. Since most CSS code on the internet is not good, LLMs do not generate good CSS code and create a lot of bugs ranging from bad performance to completely broken reflow.
Funnily enough, I seem to remember some people saying that compiler generated assembly wasn’t as good as handmade assembly.
For early compilers in certain contexts, this was once true.
the question is, what's the business cost of some amount of css bugs vs paying specialists or people to learn these details.
the generalists win overall, except of course for specific cases where specialists are great
Define "we".
The people who make the tools that generate your assembly instructions need to learn it. Just like the people who make the browser rendering engine and push CSS forward still need to learn it.
The people who don't need to learn asm never needed to learn it. If you wouldn't code in asm now, you wouldn't have ever.
This " logic" is so irrational.
I just want to point out the exception - the person who the AI learns from.
Someone has to know it. And to advance it to the next level requires domain level expertise. Or are we allowing LLM's to determine the next CSS snapshot/version/level/whatever?
Experts work on gcc so everyone else doesn’t have to.
Yes
Learning ASM will actually teach you a lot about how the computer works, and I doubt learning CSS deeply teaches you much about design fundamentals.
I want to believe this is sarcasm, but it's so hard to tell these days.
For many cases like the one described, its true.
It's not. The majority of people regularly using GCC to develop C code (IOW, the primary use case of GCC) are familiar with assembly.
How familiar? Do they ever look at the .S files? know what ar and nm do? Deal with objdump? How important are those details for day-to-day compiling of C code for regular programmers?
Yes, they know what ar and nm do. Whether these things are relevant for day-to-day compiling of C code is watching the goalposts sail over the horizon; the original statement being mocked here is "we don't need to learn ASM because GCC is so good", which completely betrays ignorance of why people learn still learn and deal with ASM to this day. Even if LLMs were comparable to compilers either in predictability (they're not) or code quality (they're not), the people who know what they're doing still take pains to understand how their tech stacks work, because that makes them better programmers, and this will continue to be true.
It's not sarcasm.
We've all accepted that code-generation has been required and accepted for decades.
Code generation (by compiler) has one major difference: it's deterministic.
Here are the reported miscompilation bugs in GCC so far in 2026. The ones labeled "wrong-code".
https://gcc.gnu.org/bugzilla/buglist.cgi?chfield=%5BBug%20cr
I count 500+ of them.
That's not what "deterministic" means.
Not sure what your point is here since wrong code doesn't mean non-deterministic. How many of these yield different results with the same input and architecture?
Honestly, we don't really need to learn any more. These days, I mostly copy and paste from tickets to the LLM, and then rubber stamp the output. I have people in my team that try to care about what they're making, but they're generating less code, and filling tickets more slowly, than I am. Even when it comes to system design and architecture, the LLMs aren't great at it, but honestly, they're better than my coworkers (I'd say system architecture is probably at the same readiness level that coding was last year). Expertise wouldn't really help you get things done faster.
You could probably replace me with a minimum wage worker to do some manual testing and copy-paste errors from the console into the LLM, and still be fine.
Let's see how long it is before the next round of layoffs, I guess. For now, the money's fine and the work's boring but ok.
But, no, the LLMs rewarding expertise line is pure cope. Software is not really skilled labor any more.
(obviously, throwaway -- I'm not interested in potentially advertising this to my employer)
> Honestly, we don't really need to learn any more.
Perhaps for your task of closing Jira tickets that’s true - but the you have a decision to make: wait until you’re eliminated or try to anticipate where future competitiveness comes from.
Yeah, that's the thing; future competitiveness isn't going to come from learning, at least not from learning about software. The LLMs have that covered, and software is becoming low status work anyways.
It's largely going to come from interpersonal skills and being likeable enough to be the face of the projects and companies. Technical skill is rapidly dropping in value.
This claim can only be true if LLMs have completely mastered all intellectual aspects of creating and shipping software.
That's so far from being true it's like saying that humans are currently an interplanetary, post-scarcity species.
All? obviously not, but enough that I can ship more than my colleagues while not really paying attention to the output or fully understanding it. Technical skills aren't unnecessary, they're just becoming something that nobody will pay for.
LLMs debug extremely well, generate code well enough, and do high level design acceptably, but fail in weird ways, so my actual work these days (outside of delegating to the LLM) is mostly manual testing. I need to give them nudges sometimes, but a year ago they needed constant nudges. In another year, I expect that it'll be good enough that pasting in a bug report with the right jargon will be all the nudging I need to get something that customers will accept if it gets shipped. They're already close.
It's obviously going to remain pretty manual without some breakthroughs, but it's largely not work that needs a ton of intelligence or skill. The world hasn't caught up with software development being a few steps up from burger flipping, so I'm still getting paid well to do it, and I'll keep collecting that paycheck as long as it lasts. But learning about code is rapidly becoming relatively valueless.
I wish it was different, I actually enjoyed getting paid to learn, but I play with the cards I'm dealt.
Yes, but sometimes I don't need and want to learn. One example from my recent experience in research -- building custom dashboard pages for results of scientific analyses. Each analysis is bespoke, and building interactive webpages is simply not the skill many researchers have (and it's boring IMO). But here with LLM you could easily explore the results visually/share them with collaborators etc. There are plenty examples like that. But certainly there are cases where learning is required.
All the boring stuff you skipped to get right to the answer you want was just as important whether you want to admit it or not.
Having to read through a structured resource describing something to figure something out has intrinsic value that an LLM is not going to provide you with.
> All the boring stuff you skipped to get right to the answer you want was just as important
This is obviously not the case. There are mountains and mountains of boring, unimportant stuff that LLMs can do for us now.
For example, at work I can now make a nestJS dashboard without having any idea what nestJS is. I can just tell the LLM what I want, and it makes it so. This allows me to make my tool useful to people without having to become an expert on something unimportant.
You’re not the only one who doesn’t know what nestJS is
Yes? My point is that for what I do - low level systems and embedded stuff - nestJS is really not something that I need or want to know about. I just want a good dashboard. Now thanks to LLMs, I can focus on the fun and interesting stuff.
My point is that it’s not even called nestJS
maybe you are confusing the fact that 2 bad frameworks exist with similar names, nextjs and nestjs
Well, no, it might not be important. Maybe you won't use that knowledge about CSS grids and flexbox for another year or two, or maybe ever.
This "instant" loop is so fast because it doesn't involve the part where you learn
You seem to be hung up on the part about "learning". Do you know how x86 registers work? Or atomic instructions available on ARM, SPARC, or POWER architectures?
No, of course not. Because all of that got abstracted to higher-level instructions decades ago.
> Do you know how x86 registers work? Or atomic instructions available on ARM, SPARC, or POWER architectures?
Yeah, I have a pretty good idea of how we go from code -> IR registers -> x86 registers and how the underlying hardware has more registers than that, etc.
As for SPARC and POWER, I have not yet gotten my hands on one! But as someone who writes compilers for fun I did learn about most of that, yes.
I write C# in my dayjob, and having written high-ish performance C++ and Rust for a couple years before that, and because of all the curiosity-driven projects I've done (like writing useless compilers), I did learn a lot.
This has helped me massively in my job, especially when optimizing our C# sensor data ingestion backend to process hundreds of thousands of measurements per seconds with explicit backpressure etc. it was great fun because I know what I'm doing.
I'm not trying to "uhm actually" you, but when you call yourself a software engineer, I do expect that you know some of these things after some years in the industry. The only way you wouldn't is a lack of curiosity, or because you genuinely had no idea that you CAN learn these things.
Yes, I do. And it wouldn't take you that long to learn, either.
Well that's great for you. That's some knowledge that you will possess for the rest of your life.
I want a green lawn and big bushy shrubs in front of my house. Do I need to know the intricate biology of my soil and habitat? No, of course not. I just do the surface-level things that make the lawn and shrubs thrive.
>No, of course not. I just do the surface-level things that make the lawn and shrubs thrive.
And others want the green lawn and big bushy shrubs, but don't want to learn all of the surface level things to make their lawn thrive so they hire a service to do it for them. And there's nothing wrong with that - not everyone enjoys the yard work, but they still want the thriving lawn and shrubs... and they are happy to write a check to OpenAI... err...Lawn Doctor every month to get that result since the time they save by not dealing with their lawn, they can pursue things they do want to do.
Not everyone wants to (or needs to) learn every detail along the way of getting the results they want.
I understand your point, but also imagine applying that across the whole spectrum of computing.
It's very useful that there's an intermediary that knows how that stuff works so I can build things without thinking about it in excruciating detail the whole time, I can dip down lower and learn stuff when it's relevant (like cache access and nested arrays) but I can also not do that in many situations.
There's also the argument that you can do engineering without understanding the underlying science as seen in th pyramids, the beautiful old european churches, etc
Who is more productive? A python dev who learns about x86 registers in his free time or one that uses AI at work?
The one who uses AI at work will eventually be replaced by someone cheaper who uses AI at work.
The one who learns about x86 registers is much more likely to continue to evolve
More productive? If the result can be trash, of course the AI work wins. In industries where quality and performance BOTH need to be upheld, knowing these details lets you guide the LLMs in a way that they will simply never do by themselves, and lets you meaningfully review performance critical code.
This reduces the cycle time on issues and reduces bugs and performance regressions. If I pitched that last sentence to your boss, chances are he'd take it.
But maybe there's a limit to the usefulness of this abstraction. Perhaps forgetting too many basic skills lower down isn't always as useful as going from Assembly to C. I'd argue that there's a law of diminishing returns there, and it's not an endless series of benefits.
In a world where there are endless things to do and only limited time to get them done, not every issue needs to be a learning matter.
If you are making something that you care about you should probably understand how it works. I didn't know this was such a hot take.
So, since you care so much and caring requires knowing everything about everything that even remotely intersects with your project, as (presumably) a software developer presumably you understand in detail how every bit of the CPUs you're targeting, plus the lithography, chemistry, and how solid state physics, and the underlying quantum mechanics work?
You seem to be massively exaggerating here for effect.
I think the main point is, if you're building frontend web apps, you should probably know how HTML, CSS, and JavaScript interact to some decently high level. Can you look stuff up on the margins via AI, or StackOverflow, or whatever? Sure.
If you're building <something else>, you should probably know the core tools and concepts at the appropriate level to build them.
No we do not need to know every aspect of CPU branch prediction and whatnot to make a webpage.
I think it's pretty obvious from the context of "if you had technical gaps (say, you couldn’t write CSS)" that this isn't about someone that regularly works on frontend web apps, it's about someone that is having to do something that is not normal for them. E.g. I mainly work on HPC code, maybe once in a year or two I need to throw together an animated poster to accompany a printed one. It's obviously ridiculous to claim that this means I have to learn frontend web dev.
Learn what? The thing the tool can do instantly? Take the win and spend your energy on bigger problems.
If you don't learn anything, you can be replaced with zero friction
I think you're talking about a different type of expertise from TFA. Consider this: What if I never enjoyed frontend programming and so I never wanted to be an expert on that?
In fact, I never enjoyed frontend programming because it was such a pain to deal with matters I considered trivial yet so frustratingly hard to do right... like centering a div. And yet the slightest misalignment is visually jarring and forces me to get a bit OCD about fixing it, which made it even more frustrating.
I questioned the whole premise of the situation: is working around a bad developer experience something worth spending my time on? Unless I actively wanted to get in there and fix the situation, not really. So yes, in those cases I would outsource my problem to a colleague or StackOverflow and move on. And as a career choice, I preferred to do more backend dev.
I would posit that that was the type of expertise that did not matter. The type of expertise that really matters here is good UI design. That is entirely orthogonal to the drudgery that is implementing and debugging webpage rendering, and I am eternally grateful to LLMs for freeing us from it.
You can extend that line of thought to the entire article. What really matters (and what LLMs reward) is domain expertise rather than technical expertise.
> like centering a div
We'll of course you're gonna be frustrated if you start with the hardest problem in HTML :/
Just in HTML?? I think it's up there with naming and cache invalidation!
I don't think AI use is supposed to replace foundational learning such as reading a C++ book or Python book or CSS tutorial when you're a beginner. You still have to do those things if you want to be a professional or a strong amateur. But many people just want to get the thing done. They don't want to become a mechanic, they just want to drive from A to B.
> They don't want to become a mechanic, they just want to drive from A to B.
I'm fairly certain the article is directed at professionals, or at least the AI companies are basing their valuations off of directly taking a slice of that professional "productivity".
I've seen junior engineers be productive on their first day in the industry because of AI, so I don't think the article is the whole truth.
The example math is boundary-pushing and definitely not a solved problem. But most of us work on CRUD backends with a React frontend. Those are more or less solved problems that have well-documented solutions. For those kinds of tasks, LLMs just reward usage.
I can count on one hand the number of times in my career I've needed to solve a problem that's not described on Stack Overflow.
As they said in the 80s or maybe earlier RTFM. I think if you got a good enough duster TFM was still readable in 2010.
the author forgot you could also do a secret third thing: learn!
dont really see the point when LLM compiles english
Except LLMs will only tend to share the most common or average of what it knows as the standard and deviating from it (including new ways) it can be resistant to.
An expert can lay a different kind of frame to prevent the llm to fell out of its way of being generally too verbose, and that can transfer as well to code generation and complication.
I think this is extremely true when it comes to prompting, but not only in this way. I would add that this also applies to an LLM's output on deep enough topics.
Anyone can point at a public GitHub repo and have an LLM write the documentation for it. Whether that documentation is good requires understanding that codebase.
There's no way around expertise unless we're talking about strictly mechanical tasks.
I do think LLMs are incredible at helping to build your expertise. You could point it at a codebase and say: "Explain how this API works" and interrogating the LLM until you get an explanation at exactly your level of understanding.
I don’t understand why this is such a revelation. Anyone who has listened to a good/great interview knows the skill of the interviewer plays a big part. To ask good questions, to understand what the other person is saying (AI or human) - that requires skill, expertise and patience.
Someone with less skill or expertise might still get good results, sure. It would just take longer and it wouldn’t be pretty
There's a growing and fascinating divide between people who see LLMs as more of a "bicycle for the mind" in the vein of Jobs vs those who see them as whollly supplanting the role of human intelligence. I can't help but wonder if these aren't primarily two human archetypes more than anything - the LLMs can be both and they erect a mirror of the human using them. Some humans really don't want deep individual expertise and intelligence to be the deciding factor because they don't identify with that. Others are completely the opposite. We really can't tell which will be more effective yet, because LLMs are very good in both modes. But most of the predictions currently are people executing on wishful thinking about what they hope will be the outcome.
There ARE two types of people. Those who ride bicycles and those who prefer a self-driving car.
I've been equating them to graphing calculators since the first LLM launched. It's an amazing tool if you know how to use it. If you don't know how to use it, it's still a tool, but you won't be doing anything amazing with it.
Nice analogy.
I loved graphing calculators until I learned tools like Mathematica and Matlab. Still waiting for the Mathematica version of LLMs.
Agents / loop engineering / whatever is hot with the AI Twitter kids still isn’t it.
LLMs are golden retrievers that have memorized all of wikipedia
maybe outing myself as a dinosaur, but "back in my day" the calculator came with a book that detailed exactly how to use it. Both the high level basic language and the low level system language. Not knowing how to use it is simply a failure to Read The Fucking Manual.
You can read the manual all you want, but if you don't know basic algebra, trig, calculus, etc, you are not going to have any idea how to apply or use much of anything that the manual describes with regards to actually doing math with a graphing calculator. There is a base level of knowledge required.
Difference is, it can teach you.
It was a general purpose computer, just a small one. Anything you could do on a "real" computer could be done on a calculator, albeit with tighter constraints. It might help to have some higher math objective to accomplish, because that would better utilize the preloaded system software. But in terms of the hardware? Probably not super relevant.
I made a lot of use of the TI-89 era CAS in college. But IMO the TI-83 era manuals taught me more about both math and computers than the subsequent generations could have.
I totally see this. I just did 3 hours of bot sitting to put together some thrash loops that thrash our provisioning working flow for a BLE gadget we make. It was pretty straightforward and productive. But then, I have a lot of experience with BLE, and a quite a bit of experience with python and shell scripting. So I was able to guide the process through stages, do some intermediate testing, make some adjustments, and proceed. Domain experience made this really easy and straightforward. Me two junior engineers who have only superficial/high level knowledge of BLE and some of the other pieces, couldn't have done this as effectively.
Where my angst comes, is worrying that no one will ever get that experience anymore. They might have had some eventual success, who knows what monstrosity a much less guided LLM would have done, but experential learning may be mostly a thing of the past. And it creates a real tension between the person with experience and the person without.
>Where my angst comes, is worrying that no one will ever get that experience anymore.
I am a fairly inexperienced python developer using LLMs to build software and find that I still learn a lot just from prompting and tinkering. Maybe that's less true once you reach a certain level of competence, but at my intermediate level I am still learning a lot even leaning heavily on LLMs.
The LLM is like the death star. If you don't know exactly where to point it, you will likely miss your target and have no/negative effect. The further away the target, the more accurate your firing solution needs to be. If all you need to do is add something like a dark mode theme to an existing product, this is probably a point blank shot in this metaphor. Building an entire codebase from zero, or even refactoring a legacy codebase into a new codebase, are lightyears away by comparison. You can still land the shot, but you need to deeply understand the metrology and astrodynamics.
The information system required to encode the aesthetic preferences needed to make a technology experience not suck is likely in excess of what any near-term solution will offer. Knowing when to say "no" is perhaps the most important skill here. You can't just say it arbitrarily either. You really have to mean it and be willing to fight other humans for it.
That's why when people like Pieter Levels tweet "I cancelled and then vibecoded 100% of my SaaS subscriptions", you need to take it with a huge grain of salt because you're not Pieter Levels, you cannot vibe code your SaaS subscriptions.
I haven’t experienced this - or maybe the training data for aviation is limited. But ask a an llm for aircraft performance / flight planning data and it’s scary how bad the advice /feed back is.
For anything where "the answer" wasn't already in the training data you just get some arbitrary correlated tokens out, as that's all a LLM can do. Of course the meaning of these tokens is just random.
(And even for things that were in the training data you don't have any guaranty they will be reproduced correctly, there is just some chance something meaningful comes out, or it doesn't, it's random.)
The whole idea to use a next token predictor as "answer machine" is completely flawed. This can't work like advertised, and that's by construction.
Most Hacker News are working on simple JavaScript applications. LLMs quickly become unusable on any sort of specialist discipline except for the math marketing releases we’ve seen recently.
This matches my experience. Just Talk To It is the best method for working with LLMs if you're an expert.
I've seen this at work (as eng manager/lead/principal/whoevenknowsanymore) – all the big APIs give you stats. We see how much people burn in tokens and we know how much output they produce. There is a pretty strong inverse correlation between token burn and output.
The more tokens people burn, the less likely they are to produce a good outcome.
LLMs are language models. So much of this can be reduced to a simple heuristic: If you can't think clearly, LLMs will not help you. If you don't know what you're asking for or how to express it precisely, it really should not be a surprise that the output is garbage. Failures of LLMs are more often failures of our own brain to consider the problem clearly. That's harder to admit than just blaming the AI.
It's important to note though, from the perspective of the LLM's objective function, that this is not a failure at all! LLMs are designed to match patterns. You give a jumbled mess of incoherent ideas, it will faithfully reproduce a token stream of incoherent ideas. It's only when the model output is subjected to the real world that it fails.
I've experienced this firsthand and 100% agree.
The more cracked you are in a domain, the more you can squeeze out of an LLM. If you already know what "good" looks like, you can steer it, call out its BS, and iterate way faster than someone who's using it to learn the domain itself.
The interesting thing is, most messages were ending with just one question of his. Examples:
- ..Does this polynomial map have any symmetry or other structure that makes this cancelation less miraculous?
- Given this structure can you see the non injectivity in a transparent way?
- ..But why is the jacobian from x u r to P Q R just a monomial?
- ..Is there a general theory of such twisted jacobians and do you have any sense why those particular dilation weights were used?
- Given this weight structure, why exactly is x given by a cubic equation from P,Q,R?
Also, looks like Terence Tao was doing lot of work and asking LLM to verify. This is inverse of the LLM trend, where LLM does the work and humans verify.
This has been my experience. I’ve been working on an app (highly successfully) since February, with the help of an LLM (ChatGPT). It has not been used as an author. Rather, it’s been a “coding partner.” I’ve been the one that has submitted the work to VCS, and run the tests.
I’ve found the most utility in having it do “small stuff that I could do, myself, but it’s faster to have the LLM do it,” and in analyzing intractable bugs, like memory and threading problems. It’s really good at analyzing a bunch of code, and seeing a small typo that results in something like a strong reference.
In both these cases, my own expertise is vital. I’m asking it to act as a consultant; to give me advice and material to be integrated into a whole that I am architecting.
I guess part of it, is that I haven’t been able to completely “give in,” and wholly trust the LLM, like I hear many people do (profitably, I guess). I’m used to having my sleeves rolled up, and my hands in the dough. Catching some pretty severe mistakes, from time to time, has reinforced this perception, on my part. I wouldn’t catch these, if I didn’t know what I was doing.
I am feeling this a lot lately. Getting the most out of agents seems to require being able to ask the right question. And how can you ask the right questions without deep domain expertise?
Yes sometimes it’s a matter of just using the right word. You can talk to an agent about a general concept for hours and hours and it may never mention $Concept_X, but you mention $Keyword_Y and all of a sudden the AI is going on about how $Concept_X is foundational to understanding the whole thing.
I started developing webapps back in the late 90s when I was in high school using Perl, and I've worked with tons of technologies up till around 2014 or so when I shifted into almost pure backend work and lost touch with modern frontend development.
I'm now learning how modern frontend development is done (for both personal and professional projects), so I may not know the specific tools, technologies, or terms but I can say "whatever the equivalent of XYZ is" and the models will translate for me. If I say "run pytype" it will tell me "we're using mypy - i'll run that checker for you".
If you can express what problem you're trying to solve, that will get you most of the way - and then you can refine by asking questions. "I think I need something like Redis for caching things - do people still use that? Is there a simpler more modern version that is the new standard? Do we already have company docs suggesting what to use?"
A good moment to remind everyone that if we took the promise for granted, that AI will in fact prevail and prompting is the one skill that will rule them all... we'll lose all domain experts in one generation.
It's less of "signaling expertise" and more about actually having said "expertise".
In my experience with LLMs it's not uncommon to be having a deep conversation about making pasta, only to be told, after asking for a sample recipe, to get a bucket of paint and a bag of concrete. Of course these hallucinations are way more subtle and easy to miss for someone who doesn't have deep domain knowledge.
Sometimes after a larger source draft with AI before I even touch the program I start to pick up on some of the holes in my prompts caused bugs/unintentional mechanics to be potentially woven in. Knowing what you want and how it should be made is half of it, but if you don't supply the bot with extra guardrails eg. don't modify the contents of x, because of y don't allow z. etc. They will do what you ask of them, usually less than you'd hope.
Agree with this. LLMs multiply the human user's ability. More ability, more impact!
And unfortunately, more ineptitude, more chaos.
When I use Claude to write code for my own projects, the code it generates is exactly the same code that I would have written had I done it by hand. If there is any deviation, I ask it to adjust but that is rare. At least 95% of the time, it's like it read my mind... Which is quite impressive when it outputs like 500+ lines from a single prompt and then it works straight away without any debugging necessary.
I don't even debug anymore on those projects. If Claude tries to add debugging logic in my code, I tell it not to and just provide additional information and it can usually find the solution faster that way.
This is when working on my own projects. When working on projects created by other people, it's a different story and I have to fight it constantly to stop it from implementing hacks and workarounds... It uses much more tokens to implement basic features. It's more work for both the AI agent and myself.
The project's existing code makes up most of the context so if the code is not great, you have to write long detailed prompts to set it on the right path. You have to make it clear that the existing code isn't good enough and your expectation is higher.
In this case, it usually gets better with more back-and-forth... At the beginning, it can't do anything because you keep pointing out a problem whenever it tries anything at all, but eventually, after a lot of criticism, it starts becoming more careful and adapting to your standards.
So yeah, even same person doing the prompting can lead to two very different experiences depending on who built the foundation.
So my conclusion is that the expertise comes from both the existing codebase and from the person doing the prompting... And TBH, I would say the codebase/foundation carries more weight than the person doing the prompting.
Pretty sure I could put an idiot on one of my codebases with Claude Code and they'd do a decent job.
At work we call this implicit steering. To use webdev metaphor, if a non-technical person describes making a web page with a big block at the top and some things to click on and then my pictures below that, that will eventually get somewhere. Meanwhile, if you know industry jargon, you might describe a hero, with call to action buttons, and then below a 3x3 grid of images of my portfolio photos—that's likely going to generate something entirely different and likely richer. It can assume things about you (it doesn't think), it can ask you specific questions a web personal might know, it can infer domain context that is otherwise omitted with a basic conversation. Everyone wants to capitalize on corporate vibe coding but the tech literacy is hardly there, let alone more advanced topics.
Opus, assume i know nothing about web development. if i wanted to design a new webpage, with a good design, what are some of the terms of art, some best practices? Like if i wanted a big block at the top, some things to click on and some pictures below that, is there terminology for that?
>Yes. Nearly everything you described has a standard name. Here is the vocabulary, organized by what part of the page it describes...
Goes on to identify Header, Navbar, Stucky header, hamburger menu, hero, CTA, Above the fold etc.
>So your described page is: header/nav -> hero with CTA -> card grid -> footer. That is the single most common landing page structure in existence, and that is fine. Being conventional is a feature, not a failure.
I've had the same conversation with an electrician wiring a car charger: we are more likely to succeed if I use his terminology.
That first sentence already uses a ton of jargon that non-developers don't use, "web development", "new webpage", "good design", "best practices", "big block",
Every thing passes through the following stages:
1. AI cannot do something.
2. AI starts being able to do something, but one needs to prompt it carefully, so one needs to be an expert, see, we will always need human experts <--- this article is here
3. AI just one-shots it.
Why do people still need to say this for each and every task? It's just reliving the bitter lesson over and over again.
Why not not show these kickass one-shots and prove how awesome AI can be?
You need to define what "one-shotting" is. Some examples would help too.
https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
Now explain how someone with no expertise would even know about the Dinitz conjecture, or even a conjecture at all for that matter, and how that would be useful to them.
What you are proposing is a barrier no AI and no human and no intelligence would pass.
The idea of an AI that one-shots a task presumes that the one requesting the task already knows what the task is, and no additional expertise.
Similarly, when you ask an AI "please summarize this text" it means you already know what summarization is as a concept.
The breakthroughs are coming from simple prompts, some made by people with no math training:
https://www.newscientist.com/article/2580932-extremely-basic...
The referenced Terence Tao chat did not lead to new breakthroughs.
This why those RL env startups are able to charge frontier labs so much for their work. LLMs still generalize poorly outside of self-verifiable tasks like coding and math.
Labs have to compensate with post-training in RL env that embeds these expertise well, which is non-trivial both in terms of domain knowledge and technical expertise.
The actual "prompting trick" that dramatically improves your results is often to add just two or three words, like "use library foo", "write in <language>", "<bar> algorithm". To know which two or three words apply in your situation, you need a deep understanding of both the problem and the solution space.
Your prompt might look almost the same as the one from somebody with a good understanding of the requirements but no technical competency, plus maybe one or two sentences. Those one or two sentences dramatically change the results, and what those sentences are differs from prompt to prompt.
I consider myself senior engineer. When talking with junior colleagues they often are surprised how little I care about some things and how much I care about others.
These internal "attention weights" are highly influential parameters of how I work with LLM.
E.g. when working with Rust I often hold strict control over structures and lifetimes. But when lately I've been doing token-based bind generation I didn't care about anything outside of high level patterns like RAII and ultimately - API ergonomics which was verified in consumer app.
I've been in position of porting real-code to vibe-code platform and seeing non-technical people prompt-stream (they were shared across accounts) I know why they engaged engineer to run this work.
Their efforts took 6 weeks, I ported app within 4 days and (to be honest with myself) without LLM I that'd be 3M+ work pre-LLM.
In short: I observed same effect as claimed.
You're basically playing the role of team lead to the LLM's junior dev.
I love larping as a vacant scrum master
If you go back to Alan Turings' paper, its all about chastisement! honestly, last pages are all about postive and negative (child!) reenforcement - 1950's style (I do not condone ... etc). Simple as that. I conduct high level litgation in the courts, and win because of a good LLM, with a good version of me, keeping it in line! also crypto and cyber sec. Of course, child rearing, and dealing with former spouses is also very useful. The orginal paper 1950 https://tinyurl.com/yuszahpw (punish is mentioned six times). Just say-ing-like. TTFN.
Prompt an image or video generator without knowledge in photography or art skills and your results will look sloppy.
Im beginning to think buzzword bingo is a viable job interview method.
If you use the b words the LLM would be somewhat constrained to content that has them. If you don't use them it will use content that doesn't have them.
The company website (subject) will never become a static html document. If you request exactly what static html does it should probably point you to a wysiwyg website builder.
> Because my friend didn’t have the vocabulary to ask the AI to write code
Is it possible that the effectiveness of an LLM user with respect to the expertise of the user is like a sigmoid function or at least a step function in that shape? That is, one has to know something like the basic concepts and the vocabulary to bootstrap a programming project, but one does not have to know too much to do lots of meaningful work, and then again one needs to be en expert to build something extraordinary. Since most of the work is somewhere middle, most of us mere mortals are still concerned or stressed out for the possibility that LLMs will squeeze out too many job opportunities.
> The model outputs are much more concise than when I try and talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians”
Maybe, but FWIW my first thought when I skimmed Tao's session was that he probably has a personal system prompt requesting this style.
E.g. even if you get it into "talking to an expert" mode I've found AI waffling through filler like "given your background in Linux kernel engineering, I'll skip the surface level and go straight to the technical meat". You do have to explicitly tell them if you don't want this.
"The most important skill in the AI era may not be prompting. It may be learning how to solve problems using the right kind of help." [1]
Context - I have over 25+ years in software, and I have this observation - being introduced to a new codebase as a human is difficult, especially depending on the scale/size and complexity of it.
Yes, you do start to learn it as you work through it, but if the scale is truly huge, it may just not be possible to fully read and understand all the code and paths etc.
I have found systems-thinkers (I believe I am one, sometimes they are architects) to be able to kind of "see the whole picture" while not knowing all the details, to the point of being able to guess how the system/software should be behaving, even if it is not actually yet. This is a hugely valuable skill and I think takes a certain kind of brain too.
That said, I think recently I may have realized something - we rely on statistics and confidence levels in order to make statements about larger populations. If we can represent a codebase as a, perhaps stratified population of code, interfaces, docs, etc. etc. etc. we may be able to take a valid random sample, review portions of the code, and make some kind of assertions about the state of the larger system - potentially, from that.
I am trying to implement this as a side project right now to see if there is anything to it, basically, a combination of AI/LLM + stats/sampling + facilitated expert human review.
I'd be interested to know if anybody is doing anything similar.
[1] https://www.actinginbalance.com/p/the-right-tool-rule
A fascinating thing about the LLM/AI blogosphere and X is watching memetics in real time. An idea like this one propagates on the order of days until everyone that speaks publicly or in workplace meetings about AI is repeating it.
I actually don't feel like Tao's recently published conversation is the best example of this idea. As intelligent as Dr. Tao is, and surely more so than me, I got the feeling that he wasn't running up against failure states of the model, which I'm not sure you could attribute entirely to his expertise. I honestly think it was more a matter of luck that the model apparently had so much training data on the topic or that it was architecturally so well suited for it. On the other hand, I've had really surprising moments where Claude was just failing terribly to execute simple dev ops tasks having to do with log processing. And I'd be so bold to say that I don't think it could have been explained by a lack of expertise on my part, or even a misuse of the model.
So yeah, sometimes LLMs reward expertise, sometimes they don't. I guess either way it helps to have it.
It is very funny that you felt the need to say this, "and surely more so than me"
I think this is just the nature of LLMs as predictive generators.
The model is predicting the type/level of conversation based on what the other party is saying.
The most typical types of conversation are of two peers, so by default the LLM is likely to respond to you at your own level, unless you ask it to behave differently.
As always, prediction goes deep. The best response to Terrance Tao is Tao-level math.
It reminds me of reading how LLMs continue chess games if given a partial game - they have learnt to assess player strength based on the moves they make, and will predict game continuations based on the perceived strength of each player, predicting (generating) poor quality moves for a weaker player. This isn't an AI playing chess to win - it's an expert predictor predicting what comes next.
They may reward it but they don't build it, and therein lies the paradox
I like it. And if you don't know a codebase, like normal, spend some time learning in. Use the LLM to query it, create your own architecture diagrams. Get homey with it before making sweeping changes. Maybe make the first simple changes by hand.
Of course they do. They have such a huge parameter maps. You need to be able to guide it through the map, so it starts making the right connections. Even in the Sonnet 3.7 days, it became clear to me, that if I have want efficient code out of it, I need to really take care of the context. If I just let it research a problem, it will mess up most of the time. If I tell it to study A, B, C and then present problem D, it will solve it perfectly. And it's true even with the current top models.
Well, nice post. Actually, there may be some truth behind it, but basically, it captures what I—as a programmer—want to read: expertise will remain valuable.
But how I am observing is different, though. Since LLMs the gap between experts and non-experts has been shrinking. And yes, there is still a gap, but vanishing.
This is an interesting problem. A similar approach that worked for us was to only load the tools the agent actually needs for each task.
This feels like a moment in time, not the end state of AI.
Like I read there was a time when teams of people + AI could beat pure AI at chess. But that these days, pure AI wins.
For all the things people say about "how AI works" you have to add the missing piece "how current AI works".
These LLM articles are so boring. Most of them are like shower thoughts with no data to back up and only the writers experience.
I agree with the premise that LLMs reward experstise, but people without expertise can very eaily learn to prompt correctly and get to a result that is very good. I remember somebody proved a mathematical conjecture by just asking 'keep going' in plain english without a mathematics background.
But then what's the point of the proof? I guess it's cool that it's possible, but given enough tokens, you could take someone who's never written a line of code and have them prompt AI to turn their vibe-coded meal prep app into a highly available distributed system with multi-region failover and immutable audit logs.
They would probably get something that checks those boxes in one way or another, but what does it do for them?
Yes. I agree with most, if not all of this. For instance, I am seeing folks either relying in the LLM as an _assumed_ expert or, assuming someone - who knows the structure of skill definitions - also has some expertise (in the area of the skill). It's a difficult situation; there is not much point in explaining _why_ the LLM output or skill in use (on a domain problem) isn't what the person actually _needs_ to address the domain problem, because the person isn't a domain expert or indeed, adjacent to domain expertise. But, it is an interesting experiemnt to arm folk with little domain expertise with the _skill_ necessary to be able to extract the right solution from the model.
I swear the ever living shit out of LLMs for even the tiniest of logical mistakes they commit.
Correcting LLMs with extreme swearing that they dare never make it again.
I make otherworldly progress with kimi, Gemini, Chatgpt, Deepseek and Claude.
Claude now stops the session. Hence Claude is now useless for me.
Swearing is nothing personal. Its a correctness enforcer.
This is true for output but as well for learning, if you speak to an LLM trying to get it to give you a certain answer, it’ll find a way to tell you you’re right. If you’re truth seeking and attempting to understand it step by step as it’s going, you’ll likely learn what it’s doing as it’s doing it, meaning you’re basically distilling that information into your own local LLM (also know as the brain).
Similarly there is research that shows the quality of LLM outputs strongly correlate with the education level (in the field) of the person promoting them.
> The most important skill in prompting is expertise in the domain you’re prompting for.
Amen. AI is a tool, a powerful tool indeed, but if you don't know how to apply it, the quality is seriously impacted.
This is true today and has been my experience as well -- both to write software as well as doing computational physics.
The interesting question then is to ask how long will this stay true? As the models get better will they eventually not need the human expertise to start adding value?
I very much agree, but at the same time I feel like those type of shortcomings are fundamental and will be somehow fixed within the next year. The AI companies would just go bankrupt otherwise...
This is also why people's experience with LLMs/AI varies so much, because some people can see a use for AI for their needs and go about using the tool, while others do not as it relates to whatever they're working on and so they may say "LLMs/AI are useless" (it doesn't mean they're not experts though, although some people who have totally no expertise might also see no use for AI for themselves).
This is true but also false.
In my experience (scientific programming) AI is a giant multiplier for people with specialized knowledge.
But it’s also a giant devaluer for that same knowledge as people with no idea what they’re doing can clog the field with plausible bullshit.
It’s now the case that if someone tells me they’ve done something, and I look into it and find out it’s completely AI slop, then I will have spent more time on the project than the person who “made” it. The situation is completely untenable and only serves to drain time and resources from people with better things to do.
we are slowly punishing reading comprehension
this will have educational consequences (that I'm trying to solve). I don't think that we can adjust without rapid education and making extreme specialists of us all.
This requires coordination, certification, licensing, and other tiers of authenticity. False experts can ruin sample gathering, can ruin training. False expertise is exemplified by the current American Administration. Look at Robert F. Kennedy Jr.; he's a false expert. He is responsible for the measles outbreak. He is responsible for ivermectin abuse by humans. False expertise is overtaking real expertise. And the results are continuously disastrous and large-scale.
Keep telling yourself that, right up to the layoff.[1]
[1] https://www.linkedin.com/posts/ademola-adelakun_pov-you-get-...
I think that’s an influencer making a joke.
If LLMs are good at busy work, and it is expertise that lets you distinguish between busy work and valuable one, then it makes sense that they reward expertise.
I agree. When I talk to LLMs about fields I am familiar with, I can push back on bad suggestions and ignore faulty/incorrect advice and assumptions, which is much harder for unfamiliar subjects.
Of course, simple common sense and extremely basic Googling on unfamiliar subjects can produce similar results, but it's much faster if you are truly understanding what the AI is suggesting.
I have lengthy conversations with my LLM, almost like an interview. I agree on the expertise part, because I wouldn't be able to go in depth on a subject with it if I lacked the expertise. Some work is a result of design and negotiations in those designs.
I don't think Tao's style works with everyone/thing, especially if we don't know what style he's tuned his LLM on.
I often do my best to represent a genuine interest in the subject at hand and learning in general to models. Imagine the model's response prose and mannerisms being on the other polar end of answering questions simply to get the correct answers as they're often scoped for on quantitative benchmarks. Not sure I explained this well, sorry. An LLM could help
agreed on this.
recently i start to rewrite a core part of one of my iOS VPN app to rust, which previously use fd.io vpp as it's networking core, the original vpp port is 1.5 years ago manually by myself, given i know a lot about how the vpp does and how vpp coroutine and runtime scheduling works.
the rewrite is in good shape and solve many issues such as pre allocated memory heap using mmap apis and some scheduling issue of back2back tcp session terminated in the vpp host stack.
also by addressing the issus, i am now can easily integrated tailscale as a addon interface for moving in/out l3 packets between tailscale and the core.
All those i think cannot be done easily without domain knowledge about those networking and system stuffs.
So we could run a lighter LLM in front of humans, which translates from 'no domain knowledge' to 'domain expert' and in turn prompts over to the larger LLM.
Then the larger LLM gets all the right lights on, yields better outputs and we translate back into user domain.
I kinda thought the chain-of-thought reasoning already did this, no?
This has been my experience as well. I’ve also been thinking a lot about Terrence Tao and his chats and presentation.
canine - dog = expertise
It's pretty obvious that for some questions a novice wont be able to drive the conversation towards an "answer". A novice may also not be able to understand an answer either. But there's a more subtle failure mode.
The vernacular used by an expert and novice to describe the exact same problem lead to different traversals of the information space.
For example, I recently asked ChatGPT a medical question using plain english. It gave me an imprecise vague response and told me to call 911. Repeated prodding did not fix this, so I asked the exact same question using medical jargon and in one shot I got what I wanted.
I cant have an overall opinion but in my personal experience i have analysed that LLMs do reward concreteness
Also, one can improve domain expertise with the help of the LLM, to becomme a better part of the LLM harness.
Might be a very noob question but in this era of LLMs, let me ask the reverse, how do you gain expertise? It seems this rewards people who had expertise pre LLMs, but what about people who don’t have that in a specific domain? What approaches are viable now in this current system?
Same way you get strong in an age of heavy machinery: Lift heavy weights yourself.
Skills will have to be built through artificial constraints. Pen & paper, reading books, not using AI, etc.
this 100%. Skills are gained via effort. Not too much effort that it discourages you, but enough so it's a challenge and engages you. Ofc it helps to have wonderful teachers, coaches, mentors, books, even ai to help guide you, but YOU must put in the effort. You don't get something for nothing.
The same as it's been! Make things without using LLMs. Don't debug with them, don't use them to research things, just do it yourself. It'll be painful and that pain is learning.
I'm not sure I understand the question. What would prevent you from doing what these people did now that LLMs are here?
Those people had no choice. In my opinion, it’s harder to grind through problems knowing very well an answer is a prompt away.
But getting an answer is not the same thing as understanding why that is the correct answer, or going deeper and learning more about the subject.
IMHO, the people who genuinely desire to learn will trudge through whatever they need to in order to grow their understanding - be it through reading books, original research papers or what have you.
If, OTOH, all you seek is the answers and that alone is satisfying to you, then of course you simply will not be motivated to do it the old school way anyway. But that's hardly different now in the age of AI.
Domain knowledge will stand alone as the sole differentiator. Because LLM benefits can be reaped by almost anyone and it’s a force multiplier. Now those who have the strongest initial force will have a far bigger edge than before.
Real world domain knowledge and experience cuts through the chaff too. LLMs are going to have people reinventing the wheel and wasting tonnes of time on stuff that won't work out. If you're a domain expert you are going to be much more aware of how to focus effort in the right places, and what's actually needed or been tried before in your niche.
A lot of this domain knowledge is not in any training data, it's locked up in companies in the industry. I suspect it will get even more important to guard it.
Love this idea of reading prompts that lead to new discoveries and figuring out how the person got the LLM there. It truly is an art and I’m always reminded of “I, Robot” and the scene about “you must ask the right questions.”
Agree so much with this! In domains I know well I get much better results then someone who doesn't know the domain because I know where to challenge the LLM. LLMs need to be pushed because otherwise their answers are typically average.
I do agree. An LLM is like a motorboat that's tends to drift off course. If you know where you want to go, and can steer it to keep it on course, you will get there very fast.
LLMs raise the floor, but you determine the ceiling.
Overall, I agree, when I ask things I'm an expert in and do professionally every day. I get very good useful answers. When, for example, our marketing people, ask about the science, they often get confusing and wrong answers.
My LLM said this was a good article :)
I agree with this post's gist, and I've certainly noticed how LLMs change their interaction with me once I demonstrate some knowledge. I've often started a technical conversation very vaguely and only once I challenge the LLM on its simplifications does it start to actually get to the meat of issues. Often there is a perceptible moment where the LLM seems to recognize my level of ability and how it communicates clearly changes.
But another thing I have found is that I get significantly better results from the LLM by treating it like an intelligent independent agent. All of the "you are a senior dev ..." or "your starving kids depend on the correctness of this answer ..." kind of prompting has been mostly useless. In general, I find being honest and clear to be the best strategy.
You can't "pretend" to be a senior software engineer. If I can root you out of an interview process then you aren't going to fool the LLM. But if you clearly state your level of expertise and your desired outcome, then the LLM does a very good job of meeting you where you are.
There is also a strange ephemeral attitude I get from agents sometimes, like they don't like to be called out for being wrong. But in the same way that human's show this trait, they also seem to warm up over time as they gain trust. It is almost like social positioning, once they realize they aren't actually expert they morph into a support role stance pretty seamlessly. That is also why they can still feel sycophantic, because once they realize they aren't actually driving the discussion they can actually feel like enthusiastic passengers, wanting to see where the conversation leads as much as the prompter.
That's some of the most weird anthropomorphization of a next token predictor I've read in a while…
This things output token which are correlated with the context given. That's all!
If you feed it some context the parrot will answer with the same.
It does not "sense your expertise level"—it just outputs correlated tokens… Is this really so hard to understand?
It seems possible, in fact reasonable, to understand my observation as following from your assumed "correlation". I gave it context that it is having a conversation with an expert, it correlates it's output with the context. A non-expert user can't fake that context so they never see similar correlated output. This context-correlation doesn't happen if one uses "You are a expert ..." prompting "tricks".
My experience has been, if you speak like an expert who is speaking to an expert you get better results. I'm not sure how that is anthropomorphizing. It is just "if I do these things, I get these results".
They reward expertise but not for long. Let's not kid ourselves into coping for a little longer.
What makes you think that they will get good at driving themselves? Can they now train on their own outputs? Doesn't seem like it
Did you ever hear of this neat thing called "The Bitter Lesson" ?
Sure. Not to speak about the ones who own the LLMs.
They just rewarded with a FREE SUBSCRIPTION one hundred thousands or so professional mathematicians! Undoubtedly, to advance mathematics! Hurrah!
This is absolutely a case where you can't get any output that better than the input, and the input is you.
The LLM knows how to solve problems in any way you want. This is not a good thing
i find this post re-assuring (as who doesn't like to feel like they are an expert at something and llm definitely strips that away)... but it still feels like you are rewarded just as much for being a 6/10 expert as you are for being a 9/10 expert. It definitely is an equalizer it is just a question of to what degree.
Not the best way to test this, but I am working on a blog post, where I'll implement a problem statement with the dumbest/cheapest AI model while someone non technical person will vibe code end to end and compare both of them.
They’re a force multiplier if you are skilled and chaos if you are not.
If you are not skilled they are not chaos, just more distracted, wasting tokens in being nice to humans. It could still teach one very well so that one improves its domain expertise.
Skilled use may or may not matter, depending on the task. Do you need to do what Terence Tao is doing?
I think if we have a large population of generalists, then none of them are generalists after all
I believe they would still be called generalists.
Maybe LLMs don't usher in the end of software engineering but they definitely end the whole "made with love (and coffee) in X". Nobody cares if you put effort into something software related. "Does it work? Yes? Ok build the next thing." Its the same with the notion of "taste" (see "sometimes tasteless computer code" from the Goedecke article), your colleague who is also a SE might respect your choices as good taste. But it ends there. This was also the case pre-LLMs I would argue. What is worse now is that communicating any uncertainty in decisions related to implementation will result in an immediate "Have you asked Claude?".
Terminology is a crazy lever with AI
LLMs are good on things I know little about, but fall short on things I'm good in.
No, they fail all the time.
You just don't notice all the bullshit if you're not already an expert in the field you generate LLM output for.
Hah. Clever.
The problem is LLMs reward no expertise and stupidity also.
What's this LLM's generated yapping?
The entire issue is that when you ask it to do something, you are leaving it to chance they may or may not do it properly, either on look/feel, performance, security, scalability etc. It compounds as you layer a new prompt output over that project.
For example, if you write iOS apps, you ask it to do some animations. It will do it for you, but you are at it's mercy of writing complete custom code or you can specify they use a certain apple supplied API for a native experience. It's one of thousand things that will get a vibe coder if they don't ask. Or you may get lucky and LLM chooses to use a native method.
Yes. This is called the Matthew Principle:
> For to every one who has will more be given, and he will have abundance; but from him who has not, even what he has will be taken away.
LLMs reward architecture knowledge of how to structure things and how to not just say "Claude, make me Microsoft Flight Simulator, make no mistakes".
Many, including myself, report having a lot of success with braindumping and not structuring anything. Just talking into speech recognition for 2-10 minutes as a stream of consciousness about what my context is, what I want, what I know already, what I have a vague hunch about, how it fits into a bigger picture, what aspects are most important to me, any footguns I already know about, really like having a chat with a person on the phone, with someone you have to guide remotely because they have to implement the thing right now but you have to be out of office and so your only interface is speech. Except you can be more structureless because the AI won't be offended. Just keep on rambling, and press enter, don't even correct mistranscriptions. It will understand it anyway.
Now, the key is, that while rambling without structure, you do have to drop the key facts into your speech, and you have to know what you're talking about in at least a good portion of it.
I think people are afraid of doing it, because it seems "not the right way" or "not scientific" or whatnot. They want to believe there is some magic to writing the right prompt. So let me tell you, it works.
I don't completely disagree with the concept of giving a free association thought process ramble into context. But I also bet that when you start getting it to actually generate code and link modules of things together, subroutines, functions, code structure and filenames, you still pay attention to what it does and you guide it into the architecture that makes logical sense to you.
For real work yes. For personal projects, less and less since Fable came out (probably the same if true of the other frontier models). You can get a lot done if it's just some one off, or a personal tool, even without looking at the code, just trying the application. Frontier models now automatically test it before handing the thing to you, they take screenshots, they fix the superficial issues themselves. To get something up and running, it's enough to send chat messages.
Unfortunately the software industry is saying things like "don't look at the code", "LLMs have made developers 10-100x faster", etc. The only way they can make such claims is by saying what you said above: "Claude, make me Microsoft Flight Simulator, make no mistakes". Additionally engineers are facing pressures via deadlines to work in the paradigm of "Claude, make me Microsoft Flight Simulator, make no mistakes"...
The question i wonder about is, when will an event come along that persuades everyone that human understanding is still required? Or will it never come?
I wonder the same thing. I think we've already seen some of this happening, however the consequences haven't been large enough to the organization, for example:
- https://www.theguardian.com/technology/2026/mar/20/meta-ai-a...
- https://tech.yahoo.com/articles/ai-code-wreaked-havoc-amazon...
- https://alexeyondata.substack.com/p/how-i-dropped-our-produc...
We can only hope that engineers working in safety critical systems haven't fallen to these working conditions.
As long as people keep gaslighting by sayting those events are just "skill issues," I doubt there will sadly be a catalyst.
Yeah, this will never happen until governments step in and cause these companies real pain. I mean just look at Crowdstrike. They caused billions and billions in economic damage due to their incompetence, and nothing happened. In fact, their stock is close to an all time high.
You're assuming that LLMs entered a world of people who understood how the systems they're inside of work, why they're setup that way, and that LLMs are displacing them. I sadly don't think that's the case in... well... a lot of the cases.
Such an event would have to be pretty catastrophic at this point to slow down the inertia. Perhaps the tech debt will just pile up until someone's product implodes, or there's a massive safety issue that causes loss of life, or some big hedge fund goes bust.
That question makes me think about Boeing. Or NASA. Or Enron. Reality always wins, no matter what management and Investor Relations says.
It makes me think about The Terminator.
The event could be when fair pricing comes from the model providers. We're still at the cash burning stage. When the economy crashes a little and departments start monitoring their spending, and the prices for inference are 10x what they are, there will be less tolerance for employees to substitute constant AI usage for understanding.
The fact that Claude knows I wrote the Virgil compiler makes it be on its best behavior when working on it. I force it to not write too much code, and to write more tests. I push back on slop and just adding another special case. It has a surprisingly deep understanding of floating point.
LLMs are like fire: great servant, terrible master
I'm surprised nobody mentioned (including the author) the Gell-Mann Amnesia Effect [1]... Just substitute "LLM" for "journalist" and there you have it.
And to be honest, I have seen it, as I'm sure (almost) everyone has, who has demonstrated experience/expertise in their own fields, and correct the LLM's responses one time or another...
[1] https://en.wikipedia.org/wiki/Michael_Crichton#%22Gell-Mann_...
Same thing I personally am not a front end guy but I have dabbled with it in the past but I am writing a front end app and a chrome extension, but besides a few pages of code I have reviewed I really do not know what the fck is written (its for an MVP I am building) and I am feeling really conflicted as to what the fuck do I do. At work, the stuff I write has a decent mix of my code, AI code and a few things I do the old way of copying from stackoverflow and seeing what works/doesn't work.
The LLM industry is deliberately consuming human expertise on a grand scale, so that eventually knowledge work is delegated to any machine, yet I have to gain some sense of comfort knowing that for the present point in time it still rewards personal skill?
Regardless of whether you agree with the claim or not, it's definitely not the endgame.
What a nothing-burger. Garbage in garbage out. Hasn't everyone who worked with LLM's experienced this?
Now everyone who feels rewarded by LLMs will conclude that it demonstrates their expertise. ;)
Is someone keeping a list of the excuses and varying instructions on how to hold it right? It would be fascinating historic documentation
Yes. If you use the right technical terms together it’s lights up more specific feature spaces to your task.
Specificity matters to LLMs a lot.
You got to know how to use the model+harness+prompt to achieve the results you want, but honestly for many projects and questions all the models already pump out their same best version of an answer.
Sometimes it is really akin to a git clone, although it was a LLM request.
This rewarding expertise is somewhat wishful thinking.
At the end of a day, it feels and is more like gambling, even with the recommended expertise and a good approach, don't delude yourself you're simply pulling the lever too, as any novice.
Wow
Cows
OK
https://youtu.be/FavUpD_IjVY