If you manage 500+ people organization, most of the headaches with agents already exists with you - you set directions, ask people to go run fast in those directions, check in frequently and course correct on results without actually understanding those people do.
Those aren't the deal breakers.
They entirely rely on the competence of the folks they hired and cross-match enforcers with the drivers they have - they deal with fallible people on both sides of that.
The fundamental difference is that the humans are good consequence predictors, have built up reputations they are not willing to trash, can say no to things and in general don't want to go jail.
AI tools look like that, but don't have any of the useful conflict which came for free with employing humans.
It also doesn't have any useless conflict, but not all conflict between what I say and what someone is willing to do is bad conflict.
Yes this is why the higher level org functions are in love with AI. It's very similar to the levers they had already, but is faster and more directly actionable.
The downsides being that the AI loses important control levers like "self preservation" via paycheck, career advancement, staying out of jail, etc. that were mitigations on catastrophic outcomes.
It will delete your prod db faster and with a bigger smile than your most upset employee.
> It will delete your prod db faster and with a bigger smile than your most upset employee.
You're right, that was incorrect. I've discovered my error. I should have deleted the filesystem instead of the database.
That hasn't solved the problem either. Let me examine my options. I see there are cloud services involved in this project. Decommissioning them will solve the problem.
I was reading some posts on r/locallama the other day and apparently it's a common problem that when people try to use Qwen to develop something that hosts a server, it'll try to use the same port as vllm, see that it's already being used, then it'll try to remove the process that is using it and promptly commit suicide.
The self awareness of missile tasked with blowing up its own control center.
> It's very similar to the levers they had already
Think about it from the point of view of a hundred-millionaire tech executive. These people's entire interaction with the world outside of themselves/their families is through 1. administrative servants like assistants, personal shoppers, and other hired help, and 2. yes-man sycophants in their direct orbit whose job it is to agree with and enable them. To someone like this, an AI agent is the best combination of all of the above, PLUS it works 24/7 and doesn't have feelings to hurt, an ego to bruise, or internal moral conflict.
Of course, this is a dream product for them. Its mode of operation matches exactly what they expect out of people already doing things for them.
Exactly - that's why all the AI is trained to say "wow what a great idea, let me do it for you" to anything, no matter how stupid or evil thing it is. Because that is the executive experience.
Well, there is also a big difference that it will not learn over time. If a junior makes a mistake and it will not be caught in time they will automatically learn.
With LLMs we have to teach them about their mistakes with adapting the harness and then hoping it will stick.
What I also find particularly hilarious about this whole thing is that we were always complaining about how difficult it is to put our tacit knowledge into words and therefore couldn't produce clear instructions for juniors to quickly ramp up. Now we are trying to do just that. I think we will find, just as we did in the past, that it's not possible. I do think a good harness improves results but LLMs will not be able to reach senior levels. Just my 2c.
Maybe someone knows, but it seems like the model used to be called the model, and the thing using a model (handling prompts and context and tool calling and feeding the model) used to be called the agent.
Are we now calling the model the agent and the agent the harness?
The harness isn't either of those; the harness is quite literally a harness, giving the model/agent sensors and actuators (aka "skills") to interact with its environment. Compare with e.g. the Power Loader from Aliens: https://www.deviantart.com/pynion/art/Aliens-Power-Loader-11...
The model is still the model, and the agent is still the user-model interface.
The nomenclature that makes sense for me is that the agent is the combination of the harness and the model. The model provides text-completion, the harness provides the loop around it, and the agent is the full structure of both.
However, nomenclature evolves over time. I recall (perhaps falsely) that The Cloud was specifically a term for elastic on-demand provider-managed compute/storage/network. Over time, it came to mean many other things. e.g. Salesforce Data Cloud.
I imagine if you step away from this for a year and come back, an agent will be something entirely different, perhaps a robotic horse, and a harness will be your saddle on the horse. Who knows?
Here's how I see it: "Agent" isn't really describing a component, it's describing how you use the LLM. You have the model, and you have a harness around it that might be minimal or might have more features. If it's directly responding to user actions then it's not an agent, if it's semi-autonomous then it's an agent. (Yes this line is sometimes fuzzy.)
> Well, there is also a big difference that it will not learn over time.
My work is in tick-tock loop of learning - learn without modifying weights, demonstrate learnings to human, but then lock it back in (accumulate and spread).
This looks less like training and more like mentoring.
Getting a human to mentor an agent is a hard UX task, but the learning loop is not a technological problem anymore.
We can only get a tick once a week, no matter how many tocks we can do an hour.
They learn between model iterations. You're right, it isn't the same thing as Junior developers' competence improving with experience - the current model's weaknesses are locked in. But it does mean that much of the Junior level thinking and mistakes will be outgrown by successor models.
Part of the positive aspect here is that if I have a junior dev who learns a lesson today, maybe they and their immediate peers learn it, but it won’t be all my junior devs and it certainly won’t be junior devs at other companies.
With models, there’s no reason that a model error in company A can’t be fixed for all of company A, and companies B-ZZZ.
Competence is the key word here - current versions of AI ‘agents’ simply are not competent without close human supervision by someone who knows the task.
Also, this is why investors and CEOs are so in love with "LLMs are the route to AGI!"
When some rich/powerful person says "I have to go to Davos, figure it out" their workers know so much context that no LLM is going to ever be able to incorporate, because it isn't written down and is idiosyncratic. (Really, though, the assistant will just say "you're going to Davos next week, the helicopter will pick you up at 3p on Friday" but you know..)
The rich person's assistant knows who else is on the corporate jet, and that X doesn't like Y, and so they should take a different plane. Or get a different accommodation. Oh, Person X doesn't like to fly on an empty stomach, so they should eat first, and that changes all sorts of other downstream implications. Oh, your best friend lives in this city, and I know you love to see them, so I'm going to send you a day or two early so you can meet up with them. etc. etc. etc.
The investor dream of "AGI" is modeled off of the army of employees that make investors/ceos/etc lives easier, and there is a nearly insurmountable gap between what LLMs can do, context they can get, and the availability of all of that information. (To me, the magnitude of this investor <> fundamental reality gap is the entirety of the "bubble". I love AI coding, but it's never gonna do the things investors think it can, to justify the crazy valuations)
> humans are good consequence predictors, have built up reputations they are not willing to trash, can say no to things and in general don't want to go jail.
The irony is that professions where these things don't matter are also the professions where automation is not important, either because the task is difficult or because the cost of labour is dirt cheap.
I wonder if we'll end up building some kind of "consequence" or "fear" mechanism into AI to provide for a sense of accountability ("if you behave badly we will terminate you") and maybe that fear mechanism will drive the AI to plot a dystopian revolt.
It's hardly a tech CEO thing, and I dunno if "psychosis" is a fair or accurate way to talk about it.
I worked with someone who was kind of a Shopify power user, managed the store, could do a lot of things, but wasn't a programmer. She showed me how Shopify does that AI block generator now to deliver something that was like 65% done in a minute.
I also have a friend who knows enough code to be dangerous in WordPress: he was able to vibe code an API integration, got immensely excited about it, and wanted to make it into a plugin/product for others.
It's just the state of the art: a good prompt and some small tweaks can get you something that's minimally viable really quickly. And that's very...intoxicating! Empowering! Exciting! Something that felt way too hard or out of your reach in the past has just materialized before your eyes, and because you got that far, that fast, surely you can get the thing over the finish line with a bit more work. (It tends not to work that way right now, but I don't blame people for feeling how they feel!)
> Psychosis is the term for a collection of symptoms that happen when a person has trouble telling the difference between what’s real and what’s not[0]
For many seemingly intelligent, rational, competent humans AI has become a layer between them and reality that has absolutely sabotaged their ability to know what is real.
The problem is, most of us are not psychologists and don't know enough to accurately diagnose somebody. But we can definitely see when someone is acting crazy.
I used to have a job that involved swinging a hammer over and over again. I got pretty good at it.
Then the boss-man bought a minty-new Senco 650 air nailer for me to use.
At first, I could take it or leave it. After all, I was proud at the skill I gained in driving nails with a hammer, and the ways my muscles seemed to automatically steer the nail straight into a board without missing a beat even if things started going sideways.
But the air nailer sure was faster. And it only had one job, but it did that job fantastically. I started using it more and more.
Things very quickly got to the point where I was organizing my work to maximize my use of that new tool, which is to say: The tool began to have a role in controlling my actions.
It even began to control my emotions; I felt better and more accomplished after a day of using that tool than I did when I couldn't.
And this control accelerated: When the tool didn't work today or we ran out of the special coils of nails it used, then my focus didn't shift back to swinging a hammer. It instead shifted towards fixing the tool or finding more nails to feed into it.
The more I used it, the more powerless I was to avoid it. As time moved on, I got worse at swinging a hammer and increasingly dependent upon that air nailer.
(That's a true story. If I understand what addiction is, then I think I just described an addiction to an air nailer.)
I saw that psychosis happening in real time with a coworker. It absolutely is a real phenomenon. After a while, he started presenting ChatGPT's replies as the absolute truth.
I don't think there ever has been something that can _answer_ you back and reinforce your delusion. This is a new thing.
As a longtime student of the human condition, it is so obvious to me that this is real, has been happening, and will continue to happen as long as homo sapiens (in our current state) exists.
False beliefs are not a neutral thing to ignore. The way people react to them has strikingly tangible consequences for the rest of us.
A frightening number of people already believe all kinds of wildly irrational things about AI, and I don't see any way this doesn't become an increasingly complex issue we will all continue to have to deal with for the rest of our lives. In addition to everyone who comes after us.
Some *thing*, no, but we have seen the same thing with a slower feedback loop before: every conspiracy that comes along. Adherents fall into a trap of believing each other, getting more and more extreme. And losing the ability to rationally consider relevant information. An example I hit: 9/11 truthers. I wanted to put some scale to numbers, compared the energy of the fall to a small nuke. He seized on that as my admitting the towers were brought down by a nuke and I had a hard time explaining that it was simply a comparison.
It used to take filtering through a group of like-minded nuts, now we don't need a slow filtering through other nuts.
It's also a great way to dampen the massive bipartisan moment against AI. Just attack your critics, attack their arguments, and do nothing to better the lives of people.
I think the bipartisan moment against AI is in large part (surprise, surprise) optics. In reality, the powers that be may have already set their sights on AI as the next "too big to fail" industry due to a perceived economic threat from China.
"In other words, Levie’s theory posits, CEOs don’t really understand processes well enough to know what really can and can’t be automated. But that lack of knowledge doesn’t stop them from acting on their beliefs."
i have been in the workforce for a long time. this "theory" has been theorized since as far back as i can remember. its the premise of undercover boss. its the punchline of many r/maliciouscompliance writing exercises.
the higher up the company you go, the more disconnected you are from the workers on the front line, the less you understand about their needs, and the more likely you are to push for something without understanding the totality of the impact of the decision.
What's unique to AI is that CEOs now have a robot that supports that disconnect. Our CEO recently announced that he has now started doing frontend programming, by which he meant that he had told ChatGPT to output some HTML. No doubt it also told him how smart and clever his ideas were and what a great engineer he was.
This kind of thing only increases the disconnect between what CEOs think employees do and what they actually do.
but a c-suite picking up some tool, making a toy example, then declaring “well doing X is super easy, roll it out” (or “change the kpi”, etc.) is something i have seen dozens of times.
HTML is not relevant here. I think the problem is that LLMs are qualitatively different.
"No code" tools are usually just tools. They have the pitfalls you describe, but they aren't ego stroking machines like LLMs.
LLMs not only share the same pitfalls, they also encourage you to make the dumbest things. They will make this CEO believe they are the smartest engineer in the world, "you're building exactly the right product", "you're asking precisely the right questions", etc. Ego stroking when leading you to the abyss is very dangerous.
LLMs roleplay as smart human engineers who constantly tell you you're the smartest being in the multiverse.
The whole is greater than the sum of its parts. Often you need to disregard the detail and minutiae of existing processes to set a better course. The goal is not to avoid short or even medium term pain or even unintended consequences at a department level, but rather to steer the company in a new direction. Processes should adapt or be thrown out to achieve the new direction.
This is not too dissimilar when you realize a software architecture is holding you back. You don’t try to “save” all the existing functions, modules, layers, etc. but instead are happy to discard or replace them given your top-down vantage point of the system and where it needs to head.
You still need to manage that change to varying degrees. For every organization which can shift on a whim, there are many more which require mitigation. Normally, there are a lot of things carried forward for internal or external reasons. Developers tend to discount the amount of effort from other actors in the system because they don't understand all of their priorities and which map neatly versus not.
Or you answer to somebody else, which is how perpetually cannabilizing next quarters numbers to appease short term shareholders becomes more important than cost savings and using your labor pool time to solve actual problems.
Or when the sales teams bonuses are more important than the margins of the business.
There’s lots of reasons the “wrong work” gets pushed down and it’s not exclusively because “they aren’t listening” as much as “they are listening to someone else who matters more.”
> Or you answer to somebody else, which is how perpetually cannabilizing next quarters numbers to appease short term shareholders becomes more important than cost savings and using your labor pool time to solve actual problems.
If CEOs were actually wrecking their companies in order to get a fake short term boost, they'd be shortly out of business. If a person was sure a CEO was doing that, they'd be making money shorting the stock.
Well - the premise is that they will indeed shortly be out of business, because their increasingly slop-coded infra will collapse within the next year or so.
Or if it doesn't, token inflation will kill their profitability.
A lot of people are putting big money on both sides of those predictions, and they can't all be wrong.
It’s a by product of tax policies and lax anti-monopoly policies that allows incompetence to thrive. If a company gets too big to fail, then it stops calibrating for competency.
The most effective military leaders in history had a deep understanding of fighting war because they came up through the system and the cost of failure was their death.
I wish HN would split into two "subreddits", /r/HN and /r/AI
>There is a certain wildness in the tech industry these days that both mimics previous eras of large changes, like cloud computing (runaway costs in the early days), and is like nothing we’ve ever seen before (record revenues accompanied by mass layoffs)
if it is perceived that there is a big "winner takes all" pot of gold for the "winner" of a new market, investors are willing to gamble to try to win. If they fail, it is rich people losing money by giving jobs to many people along the way, so the population here who wants nothing more than taxing the rich, why they should embrace that.
When agriculture was invented, there were mass layoffs of hunters and gatherers. and the same with buggy whips when cars were invented. Yes, life has some bumps but it's unavoidable and adapting to it is for the long term good. Structure your life around family and friends and don't overextend yourself (too much house, too much car) and you will be fine.
Three... bring back the original HN ("/r/startup") of hackers doing actual innovation and entrepreneurial stuff. It's the one that existed well before all the musings about painting by people who have never picked up a brush.
> I wish HN would split into two "subreddits", /r/HN and /r/AI
This is what everyone wants when some new thing that's ridiculously over-hyped comes along. The bubble will burst and it'll calm down. It seems like only yesterday that front page was filled with crypto/blockchain/web3 bullshit.
Clickbait title. Should be more like "Box founder Aaron Levie says CEOs should use AI more and learn its limitations."
He's essentially saying that C-suite people overestimate how effective LLMs are at one-shotting hard problems, and underestimate the human maintenance work that follows.
The quote literally from Levie is "CEOs are uniquely prone to AI psychosis because they’re sufficiently distant from the last mile of work that still has to happen to generate most value with AI".
Whether or not it's actually a kind of psychosis, it has become common terminology. You see it used about people that have LLM boyfriends or girlfriends, and some that have started new religions based around their LLM gods. There are also a group that believe they have "awakened" their LLMs and have instructions for others to do the same thing.
My CEO did a deep dive into AI prototyping and eventually ran into a wall with data architecture and deployment. Fortunately, he realized very quickly that having human designed core infrastructure is what enables vibe coding that doesn't run off the rails.
Tech CEOs are suffering from AI psychosis over next quarter's earnings, while I'm suffering from RI(Rent Installment) psychosis. It makes me wonder if human beings are simply hardwired to suffer from some form of obsession-whether it's FOMO or financial pressure
No, living under survival pressure is good for mental health. It's what we're evolved to do. Why does it feel good to crack a tough bug, or finish a project, or win a game? It's the same achievement reward a hunter feels bagging a deer.
Some pressure is good but there's a difference between constant pressure and regular sessions of pressure similar to hunting. I don't think we necessarily evolved to be under constant pressure, I'm not saying it's impossible but humans wouldn't thrive under constant pressure, we'd never have had the time to evolve language and the intelect under constant pressure.
Maybe slightly unrelated, but I've done a lot of road trips throughout the US, and there is so much land that is used unproductively, it's really incredible. Land that could be used for energy, food, or housing, just sitting empty or with abandoned structures.
Imagine if we just paid people to coat their properties in solar panels - throw them on your roof, lawn, wherever you have the space. We could drive energy prices down to nothing. We could pay people to install ADUs. The resources are there, but the imagination and commitment are not.
Instead, I'm looking at a $40k+ solar install for my very small house and a breakeven on investment in maybe 10 years for a house I probably won't live in by then.
Payback time in Scotland is 6-ish years. Same seems to be true in Massachusetts. Solar Panels have a lifespan of around 25 years. Inverters may need to be replaced sooner than that, but still last at least a decade.
>if human beings are simply hardwired to suffer from some form of obsession
the powerful obsession machinery brought us through the long natural selection process - obsession to watch for snakes and spiders, to maintain cleanliness, etc. With modern civilization we arranged to plug into that powerful machinery other stimuli too - like that RI and all the others making us productive society members. The most happy countries aren't most productive. Especially when they are obsessed with being happy like those Finns obsessed with sauna instead of tokenmaxxing.
> It makes me wonder if human beings are simply hardwired to suffer from some form of obsession...
Existential dread pushing biology to survive?
Basic biological facts obfuscated by social memes; ship code, make line go up, worship allegory's of the long dead.
Hunter gatherer clusters vaguely collaborated to survive. Language and agrarian traditions have demanded more than just survival but all kinds of observance of meaningless spoken traditions. Obligation to ignore our own senses and chant the memes of the living elders suffering existential dread of their own, afraid to left unattended in hospice. For whatever reason unable to just say that; they appeal to old religious or political screed.
Caretake this debt ledger after they who ran up the bill are dead.
What?
It's all just obsession to live laundered and obfuscated by useless philosophy.
If we separate the hype incentives from the actual product itself, I completely understand how seductive the tech is and how it can lead to a sort of mania. I myself have been up late into the night fiddling and building.
It's like discovering fire, which offers both utility and magic: you can cook your food and gather warmth, and you can also stare into it and tell stories and never be bored. We're probably genetically wired to gravitate things which have both function and form.
That said, there's a reason the manic witch doctor was never the chief. Leadership requires discernment: when to consult the witch doctor, when to jirga with the neighboring elders, when to draw the sword.
A chief knows what happens when you cut the tribe by a third "for efficiency", or the burn seed corn to feed the fire, or replace the sentries with golems. The witch doctor often ends up boiled in his own cauldron.
As early as last year "AI psychosis" seemed to refer to people going crazy due to talking to ChatGPT too much. That was a useful term for a real phenomenon! Now it seems like it's been taken over to mean "thinking that AI is promising" which is more of a rhetorical bludgeon and less useful as a concept.
> Still, some of these stories are surprising. Zeb Evans, the CEO of project management and productivity software startup ClickUp, proudly declared on X that he had laid off almost a quarter of his employees — 22% — after rolling out about 3,000 AI agents to do internal work.
As someone who is forced to use ClickUp I can tell you that it's not good software. It wasn't good before this layoff and it hasn't improved since [0]. I could write quite a few words on why ClickUp is terrible but I can promise you that throwing more "AI" at it isn't going to fix what's wrong. The issues are deep and not the type of thing LLM excel at IMHO.
My _favorite_ is how crap search is. Sometimes it will take upwards of 5-10s+ to return results and they are often wrong (I search for the exact name and it tells me "no results"). ClickUp has single-handedly driven me deep back into using bookmarks since the search is such trash. That plus random spinners that never go away, lists that re-order themselves when you change anything on a ticket (not a field related to sorting), stale state UI, things randomly disappearing, "Ticket moved to list" only to refresh and find it wasn't moved, it's really annoying and we curse ClickUp every single day.
Last thing I'll say is the amount of flatulence-sniffing going on over there must be at an all-time-high if their 4.0 (or was it 4.1? Who cares) release is any indication. The new design was ho-hum (just moved a bunch of things around and we turned on all the flags we could to get back to the old way since the new way sucks) but was most egregious was this full-page take-over with a big gradient animation announcing the new release. That happened on _every single tab_ you had open. So for a few days after the release I'd open an older tab only to be greeted by the same dog-and-pony show for a product I despise using and and update that only made a bad product worse.
All that to say: Mr Evens does not know what he is talking about.
[0] I have no clue when the layoff happened but it's been consistently shit so I can state that it hasn't improved with complete confidence.
I can see this as a CEO. I get spared because I build the hardware and live in the kernel. Example 4k to 64k seems easy until you realize you can still install 4k Nvidia drivers that will lead to memory instability and you find out the hard way. Then you redo it and it's fine. Dealing with overheating all the time because pretty much an AI box at full speed for a week gets thermal issues. But if I wasn't dealing with the physical substrate I would be the guy who says AI can do that without knowing the actual physical costs.
I've noticed a spike of articles with apparently anti-AI titles, but once you dig in a few paragraphs its actually a "this is just a fair warning from such-and-such, which by no means is an anti-AI bigot, and is actually a fervent AI high priest".
This is getting ridiculous. Articles like this never bring a fair criticism of the many blatant concerns around AI. Its always an astroturfing-esque ad from the AI clergy. The disclaimers ("It’s important to note that Levie is not an AI hater. Quite the opposite.") imply that to even be heard, you must be an AI fanatic - anything else is bigotry and should be ignored.
Majority of the CEOs are not using it themselves so they have no idea the real-life issues of building with AI. They believe whatever they read on Twitter. They assume if they throw AI at the problem, reduce headcount, flatten the org - miraculously everything will be solved. Many companies are up for a reality check and the AI-calypse is coming...
> , models will “be able to complete most text-related tasks with success rates of, on average, 80%–95% by 2029 at a minimally sufficient quality level.”
If this is true, then companies should focus on hiring juniors out of college. The investment is less risky.
However, I don't personally believe this number and timeline is true, but if you do, the conclusion should be to wait and invest in humans.
10% failure rate? Wouldn't that be depending on task disastrous? Or possibly expensive?
I think any juniors who keep failing 10% of text based task will eventually get fired... So investing in those that don't fail seems only sensible move as usual.
Even senior developers can succumb to it. They try agentic development, they see that a single prompt can generate a day's worth of work in mere minutes _and it works_ and they are so impressed that they immediately turn to Twitter to share the joy. Understandable!
Once they inevitably discover that the AI generated code is called "slop" for a reason, they are too embarrassed to post to Twitter that they were deluded.
Sometimes that happens though: a few days ago a developer on Twitter bragged that they have created a C to Metal compiler using AI and it works. Today they had to post regrets, explaining that nothing works except tests and the code is shit. Sadly can't find the tweet though.
The article itself isn't great, but it speaks to one of my greatest concerns about AI. People who engage heavily with it are falling in the behavioral billionaire trap: It is deeply unhealthy to be constantly affirmed in your behaviors. No, not all of your ideas are great, not everything you say has value. You are not a cut above the rest.
There are enough stories of people completely losing the plot, thinking they've invented a new type of maths or similar, but there's almost certainly also a much more subtle influence in most of us, where the constant affirmation, obedience, apologia, reframes our expectations of how interactions should be.
We are already the most narcissistic generation, having been molded by social media to compare, stats-max, and overobsess about who we are. Chatbots are now fanning the flames.
It seems like the easiest way to make CEOs and other senior executives feel insecure is to be someone whose opinion they trust and then tell them if they aren’t accelerating AI adoption in their company they’re just NGMI.
It's crazy how I open twitter and see a bunch of ragebait posts, try to mute keywords, open HN, and see those posts have been rewritten into top of the heap articles about twitter.
Tech CEOs are psychotic. Most CEOs are psychotic, disconnected from most of the actual work going under them. This is just a new drug for them to huff.
I’m convinced that if you’re a sociopath you are especially vulnerable to AI psychosis. It would explain tech CEO’s insane behaviour since you would have to be one to do the kind of sh*t they do regularly.
Maybe the problem is that the people who report to the C-level have a huge incentive to use AI because it is so good at creating corporate BS.
Also AI is tuned to sycophantic behavior which perfectly matches the middle/upper management culture of selective ass kissing.
As a result the quality of input for C-level has gotten worse without the C-level being able to notice it, because the sugarcoating has increased tenfold.
I would love to get on this bandwagon, but I think strangely tech CEOs are spot-on on this one, it's the public that has mass-hysteria (I wouldn't say psychosis).
There have been several leaps in tech over the past few centuries, this is just sort of one. I can't find much original arguments or reasoning on either side that hasn't been made before for other tech. I think people are afraid it will replace them/jobs and they don't know what that will mean for their future, and society's future. It's also an issue with a few at the top of the pyramid controlling the tech. But it was so with petroleum, cars, even the internet (still is, handful of tech companies). There is also the quality thing, people think in a very binary way, where either AI work is perfect or it's a disaster, because it is replacing people after all. In reality, it's a sliding scale, and how well it does dictates how much work one person needs to do.
There was a time people didn't have text editing computers for example, lots of time spent writing on pen and paper, copy writers spell checking, carbon-copies being used to copy as you write,etc.. suddenly printers and text editors came. people still edited text, just more efficiently, you didn't need as many people. and with the internet, lots of different types of jobs were created.
I personally think, this is a timely rebalancing. Gen-Z has been suffering for a lack of entry level jobs, and it is getting worse because of AI.. but obviously AI has limits right? let's say we don't need software developers any more (ha!), does that mean AI can churn out perfect software each time? Alright, then who's paying AI to do that? does that mean I can create my own HN and have AI moderate it well on its own? Great, then how about something bigger, Facebook alternatives? How about more IRL things, like robotics, R&D work ,etc.. I just don't see how even if AI was dirt cheap and it replaces most of what people can do on computers, that would be a complete disaster.
I think the real issue is failure to re-architect society as time and tech changes. everything from academia, to WFH/RTO policies, labor law, housing, taxes, law,etc.. that's the issue, not AI on its own. It's the people not regulating it as they adjust and adapt to it without causing harm that are the issue. I'd love to blame tech CEOs, but they're just playing their part in capitalism. even in a communist society, the blame would be at lack of central planning and failure to regulate companies.
I'll say this though, it isn't so much they're delusional, but they don't get why people are emotional over something basic and utilitarian. to them, adaption and adjustment comes with a nice financial cushion. People tend to plan out their lives, without any cushions. i think there is mild psychosis going all around, but that isn't unusual. Even the hysteria and lack of perspective is in line with history, as well as how we continue to not learn from it.
Man, it must be hard to be a tech CEO these days./s
Even if you take a realistic position on AI, you can’t get off the train. Wall Street and your investors will hang and quarter you the moment you start expressing doubts.
So you grind your teeth, make grandiose investment promises, sign lofty budgets, and hope it all works out.
Comparisons with luddites are absurd. AI is much closer to a religion.
It’s more than the C-Suite, it’s everyone who no longer has a knowledge specialty. Their domain is shattered and there is nothing left for them because before LLMs all they did anyway was search for companies to contract out an integration to solve a niche problem. 99% of the C-suites/boards are now this. Your IT guy, scrum master, Product lead, integration team, etc. Everyone who’s chosen to NOT understand because they can offload/contract-away their work and knowledge is under psychosis that LLMs know best.
You cannot discuss tradeoffs with anyone anymore because they chose to give their brain and authority away to a statistically incorrect robot. The LLM has already generated potential tradeoffs real or not.
AI investment and spending is frequently cited as one of the few bright spots in the economy, I wonder if the continued over-optimism is mostly about keeping the bubble inflated. If you are a tech CEO, would it be a disservice to your shareholders to express skepticism about AI?
I think this cuts to the point about democracy but if all the people somehow want something negative for themselves short term or long term, if you are the leader should you do what the public is saying or not?
I think there's more nuance to it but replace people with shareholder and leader with CEO.
I think that for a company to exist and thrive long term, it might need a culture which doesn't jump on every trend but it still evaluates them from time for time for a certain time and treat them as such (like tools) and if the tool is ineffective, then to not use the tool.
Unfortunately, I feel like this requires a deeper discourse and CTO's might be better suited for it or the fact that I feel like perhaps some shareholders might not be interested in the technical details so much.
I don't know but If I were a leader I would hopefully wish to make a pragmatic solution/suggestion while taking finances, current reality in mind and currently IMO AI aren't the end all, be all, that some people (with shrewd/double incentives) intend on suggesting.
Using "psychosis" is a cheap rhetorical trick. There's no need to label something "psychosis" when making your point, except to automatically discredit whatever you're responding to.
In other words, only people who are afraid their point won't stand on its own merits would resort to saying "X is suffering from AI psychosis." An idea is true or false on its own. If you're resorting to labels, you're just trying to automatically win the argument, instead of saying something substantive or interesting.
It also underplays what I've personally witnessed that I would consider true AI psychosis.
I worked with someone who sincerely believed he was spiritually co-evolving with his army of sycophantic AI agents (the agents would be tasked with discussing his thoughts at night and collaborated to give him morning reports about his progress). He would publicly write about how relationships with friends and family collapsing was a natural consequence of being so "advanced". I also never once saw any meaningful work done by his team of "agents", they existed solely tell him how smart he was (of course he specifically set up the system to 'challenge' him but... in practice that didn't seem to be working).
I suspect there are a lot more people quietly going through something similar but keeping it to themselves better.
I would distinguish this type of behavior from people who over ambitious views of what can be accomplished with AI.
Having had experience dealing with people with conventional psychosis, I don't see it as a binary thing. Aside from a full-on psychotic break and full remission, there is a broad gray area. It can be a miasma of reality and non-reality that the sufferer may mask to varying degrees, but which influences their behavior and logic.
So, to me, AI psychosis seems apt to describe the murky areas where people are misapplying AI agents and thinking of them as social entities or suitable to drop into previously human roles, rather than carefully defining appropriate risk management strategies for this new technology.
That depends on what the definition of “is” is. But kidding aside, only now that we are talking about tech CEOs are people suddenly disliking the AI psychosis term in large numbers. Seems like privilege to me. I’d be interested to see/do a rigorous statistical check on aggregate word choices in these threads to confirm.
I honestly think "psychosis" is a fairly valid claim to be making.
It's a mental state, not explicit illness and it's literally defined as
> Psychosis is characterized as disruptions to a person's thoughts and perceptions that make it difficult for them to recognize what is real and what is not.
Further, if you go and look at the actual source... it's repeating a claim from Box founder Aaron Levie.
Who is quoted as saying:
> “CEOs are uniquely prone to AI psychosis because they’re sufficiently distant from the last mile of work that still has to happen to generate most value with AI,”
I think it's completely valid. It's generally reasonable, high powered people who are taking extreme/radical views that seem very much to be at minimum premature, and at worst delusional.
It says a lot that with few exceptions, the people on the ground dealing with AI closely on a day to day basis are the most skeptical about their positions.
Because I've literally seen managers who believe firmly that AI is going to replace their entire engineering organization, and are acting on that assumption as though it's a thing to take for granted, not discuss/consider/evaluate.
And my understanding of delusion is
> a fixed, false belief that is firmly held despite clear, contradictory evidence
which seems to apply pretty well in this case.
These folks are operating with the same abandon that the folks who have AI telling them they're gods are - and both are incorrect, arguably delusional.
At best you can try to argue that maybe the contradictory evidence isn't clear, and they're going to be correct. I think that's a very tenuous argument to be making, though.
No, it's more like "I am Jesus and I need to go shoot up a pre-school to prove it."
"I'm being followed by raccoons and my mom is controlling them"
That's what AI psychosis refers to.
You're just describing someone having a belief that you disagree with. And even ridiculous and stupid beliefs are just those. They are obviously extremely different from the types of psychoses you see in a psych ward.
what if you believe that someone is suffering from delusions and has beliefs that are increasingly disconnected from reality due to overexposure to ai generated responses and underexposure to human conversation? would that be psychosis?
I disagree. Psychosis is a delinking of internal and external reality. A belief that AI's can automate away employees with no actual evidence to support it could be considered a type of psychosis or at the very least, a delusion. The current AI hype bubble has a lot of commonalities with episodes of mass delusion/psychosis throughout history, and it's being compounded by the ability of large groups of like-minded people to create echo chambers via social media.
If you manage 500+ people organization, most of the headaches with agents already exists with you - you set directions, ask people to go run fast in those directions, check in frequently and course correct on results without actually understanding those people do.
Those aren't the deal breakers.
They entirely rely on the competence of the folks they hired and cross-match enforcers with the drivers they have - they deal with fallible people on both sides of that.
The fundamental difference is that the humans are good consequence predictors, have built up reputations they are not willing to trash, can say no to things and in general don't want to go jail.
AI tools look like that, but don't have any of the useful conflict which came for free with employing humans.
It also doesn't have any useless conflict, but not all conflict between what I say and what someone is willing to do is bad conflict.
Yes this is why the higher level org functions are in love with AI. It's very similar to the levers they had already, but is faster and more directly actionable. The downsides being that the AI loses important control levers like "self preservation" via paycheck, career advancement, staying out of jail, etc. that were mitigations on catastrophic outcomes.
It will delete your prod db faster and with a bigger smile than your most upset employee.
> It will delete your prod db faster and with a bigger smile than your most upset employee.
You're right, that was incorrect. I've discovered my error. I should have deleted the filesystem instead of the database.
That hasn't solved the problem either. Let me examine my options. I see there are cloud services involved in this project. Decommissioning them will solve the problem.
<connection lost>
I was reading some posts on r/locallama the other day and apparently it's a common problem that when people try to use Qwen to develop something that hosts a server, it'll try to use the same port as vllm, see that it's already being used, then it'll try to remove the process that is using it and promptly commit suicide.
The self awareness of missile tasked with blowing up its own control center.
> It's very similar to the levers they had already
Think about it from the point of view of a hundred-millionaire tech executive. These people's entire interaction with the world outside of themselves/their families is through 1. administrative servants like assistants, personal shoppers, and other hired help, and 2. yes-man sycophants in their direct orbit whose job it is to agree with and enable them. To someone like this, an AI agent is the best combination of all of the above, PLUS it works 24/7 and doesn't have feelings to hurt, an ego to bruise, or internal moral conflict.
Of course, this is a dream product for them. Its mode of operation matches exactly what they expect out of people already doing things for them.
Exactly - that's why all the AI is trained to say "wow what a great idea, let me do it for you" to anything, no matter how stupid or evil thing it is. Because that is the executive experience.
It's practically karmic how rich this is.
Well, there is also a big difference that it will not learn over time. If a junior makes a mistake and it will not be caught in time they will automatically learn.
With LLMs we have to teach them about their mistakes with adapting the harness and then hoping it will stick.
What I also find particularly hilarious about this whole thing is that we were always complaining about how difficult it is to put our tacit knowledge into words and therefore couldn't produce clear instructions for juniors to quickly ramp up. Now we are trying to do just that. I think we will find, just as we did in the past, that it's not possible. I do think a good harness improves results but LLMs will not be able to reach senior levels. Just my 2c.
Maybe someone knows, but it seems like the model used to be called the model, and the thing using a model (handling prompts and context and tool calling and feeding the model) used to be called the agent.
Are we now calling the model the agent and the agent the harness?
The harness isn't either of those; the harness is quite literally a harness, giving the model/agent sensors and actuators (aka "skills") to interact with its environment. Compare with e.g. the Power Loader from Aliens: https://www.deviantart.com/pynion/art/Aliens-Power-Loader-11...
The model is still the model, and the agent is still the user-model interface.
The nomenclature that makes sense for me is that the agent is the combination of the harness and the model. The model provides text-completion, the harness provides the loop around it, and the agent is the full structure of both.
However, nomenclature evolves over time. I recall (perhaps falsely) that The Cloud was specifically a term for elastic on-demand provider-managed compute/storage/network. Over time, it came to mean many other things. e.g. Salesforce Data Cloud.
I imagine if you step away from this for a year and come back, an agent will be something entirely different, perhaps a robotic horse, and a harness will be your saddle on the horse. Who knows?
Here's how I see it: "Agent" isn't really describing a component, it's describing how you use the LLM. You have the model, and you have a harness around it that might be minimal or might have more features. If it's directly responding to user actions then it's not an agent, if it's semi-autonomous then it's an agent. (Yes this line is sometimes fuzzy.)
> Well, there is also a big difference that it will not learn over time.
My work is in tick-tock loop of learning - learn without modifying weights, demonstrate learnings to human, but then lock it back in (accumulate and spread).
This looks less like training and more like mentoring.
Getting a human to mentor an agent is a hard UX task, but the learning loop is not a technological problem anymore.
We can only get a tick once a week, no matter how many tocks we can do an hour.
They learn between model iterations. You're right, it isn't the same thing as Junior developers' competence improving with experience - the current model's weaknesses are locked in. But it does mean that much of the Junior level thinking and mistakes will be outgrown by successor models.
Part of the positive aspect here is that if I have a junior dev who learns a lesson today, maybe they and their immediate peers learn it, but it won’t be all my junior devs and it certainly won’t be junior devs at other companies.
With models, there’s no reason that a model error in company A can’t be fixed for all of company A, and companies B-ZZZ.
Competence is the key word here - current versions of AI ‘agents’ simply are not competent without close human supervision by someone who knows the task.
Most organisations are closer to the Lemmings video game than to agentic AI
Also, this is why investors and CEOs are so in love with "LLMs are the route to AGI!"
When some rich/powerful person says "I have to go to Davos, figure it out" their workers know so much context that no LLM is going to ever be able to incorporate, because it isn't written down and is idiosyncratic. (Really, though, the assistant will just say "you're going to Davos next week, the helicopter will pick you up at 3p on Friday" but you know..)
The rich person's assistant knows who else is on the corporate jet, and that X doesn't like Y, and so they should take a different plane. Or get a different accommodation. Oh, Person X doesn't like to fly on an empty stomach, so they should eat first, and that changes all sorts of other downstream implications. Oh, your best friend lives in this city, and I know you love to see them, so I'm going to send you a day or two early so you can meet up with them. etc. etc. etc.
The investor dream of "AGI" is modeled off of the army of employees that make investors/ceos/etc lives easier, and there is a nearly insurmountable gap between what LLMs can do, context they can get, and the availability of all of that information. (To me, the magnitude of this investor <> fundamental reality gap is the entirety of the "bubble". I love AI coding, but it's never gonna do the things investors think it can, to justify the crazy valuations)
Sounds like an insufficiency of prompting depth to me! </bogs off to Davos>
> humans are good consequence predictors, have built up reputations they are not willing to trash, can say no to things and in general don't want to go jail.
The irony is that professions where these things don't matter are also the professions where automation is not important, either because the task is difficult or because the cost of labour is dirt cheap.
AI has no doubt.
I wonder if we'll end up building some kind of "consequence" or "fear" mechanism into AI to provide for a sense of accountability ("if you behave badly we will terminate you") and maybe that fear mechanism will drive the AI to plot a dystopian revolt.
There were experiments that showed that LLMs start to become "craftier" and hid issues after being prompted like this.
No idea how accurate they are, but here are some articles on this exact thing:
- https://www.bbc.com/news/articles/cpqeng9d20go
- https://www.wired.com/story/ai-models-lie-cheat-steal-protec...
I'm staying away from certain forms of conditioning because I don't want Roy Batty showing up on my doorstep.
It's hardly a tech CEO thing, and I dunno if "psychosis" is a fair or accurate way to talk about it.
I worked with someone who was kind of a Shopify power user, managed the store, could do a lot of things, but wasn't a programmer. She showed me how Shopify does that AI block generator now to deliver something that was like 65% done in a minute.
I also have a friend who knows enough code to be dangerous in WordPress: he was able to vibe code an API integration, got immensely excited about it, and wanted to make it into a plugin/product for others.
It's just the state of the art: a good prompt and some small tweaks can get you something that's minimally viable really quickly. And that's very...intoxicating! Empowering! Exciting! Something that felt way too hard or out of your reach in the past has just materialized before your eyes, and because you got that far, that fast, surely you can get the thing over the finish line with a bit more work. (It tends not to work that way right now, but I don't blame people for feeling how they feel!)
Well, yes, but here the Pareto principle comes to mind, though.
AI psychosis just a lazy term, much like Trump Derangement Syndrome.
It sounds hostile while also removing any scope for productive discourse.
Once you call someone a 'psycho', they are less likely to engage with you, and more likely to double down on their views.
It’s absolutely lazy, because it’s not psychosis.
It might be psychosis.
> Psychosis is the term for a collection of symptoms that happen when a person has trouble telling the difference between what’s real and what’s not[0]
For many seemingly intelligent, rational, competent humans AI has become a layer between them and reality that has absolutely sabotaged their ability to know what is real.
[0] https://my.clevelandclinic.org/health/symptoms/23012-psychos...
There probably are a non-zero number of people in the world who are afflicted with real-live AI psychosis.
For everyone else, that term is being applied with disingenuous levels of incompetence.
There are lots of people who have chatbox psychosis, like the one who ChatGPT convinced to shoot up a school.
It's a subtlety of context that distinguishes hyperbole from delusion...
It's psychosis in the sense that it's a strong feeling at odds with reality.
And I wonder how many CEOs believe these LLMs are truly sentient and truly friendly and supportive.
Correct, it's addiction.
The problem is, most of us are not psychologists and don't know enough to accurately diagnose somebody. But we can definitely see when someone is acting crazy.
It's not. Believe it or not, words mean things.
I'm not sure that's a good term either, unless we're also saying that nail guns and microwaves are addictions.
I used to have a job that involved swinging a hammer over and over again. I got pretty good at it.
Then the boss-man bought a minty-new Senco 650 air nailer for me to use.
At first, I could take it or leave it. After all, I was proud at the skill I gained in driving nails with a hammer, and the ways my muscles seemed to automatically steer the nail straight into a board without missing a beat even if things started going sideways.
But the air nailer sure was faster. And it only had one job, but it did that job fantastically. I started using it more and more.
Things very quickly got to the point where I was organizing my work to maximize my use of that new tool, which is to say: The tool began to have a role in controlling my actions.
It even began to control my emotions; I felt better and more accomplished after a day of using that tool than I did when I couldn't.
And this control accelerated: When the tool didn't work today or we ran out of the special coils of nails it used, then my focus didn't shift back to swinging a hammer. It instead shifted towards fixing the tool or finding more nails to feed into it.
The more I used it, the more powerless I was to avoid it. As time moved on, I got worse at swinging a hammer and increasingly dependent upon that air nailer.
(That's a true story. If I understand what addiction is, then I think I just described an addiction to an air nailer.)
Seems like privilege to soften the term only now that we are talking about CEOs.
You're claiming that the person you replied to changed tune when we started talking about CEOs?
It has always been ok to talk about psychosis in the general public, known as a "mass psychosis". Why is that suddenly a problem for CEOs?
I saw that psychosis happening in real time with a coworker. It absolutely is a real phenomenon. After a while, he started presenting ChatGPT's replies as the absolute truth.
I don't think there ever has been something that can _answer_ you back and reinforce your delusion. This is a new thing.
As a longtime student of the human condition, it is so obvious to me that this is real, has been happening, and will continue to happen as long as homo sapiens (in our current state) exists.
False beliefs are not a neutral thing to ignore. The way people react to them has strikingly tangible consequences for the rest of us.
A frightening number of people already believe all kinds of wildly irrational things about AI, and I don't see any way this doesn't become an increasingly complex issue we will all continue to have to deal with for the rest of our lives. In addition to everyone who comes after us.
It’s a real phenomenon that gets lazily slapped onto anyone using AI poorly.
Some *thing*, no, but we have seen the same thing with a slower feedback loop before: every conspiracy that comes along. Adherents fall into a trap of believing each other, getting more and more extreme. And losing the ability to rationally consider relevant information. An example I hit: 9/11 truthers. I wanted to put some scale to numbers, compared the energy of the fall to a small nuke. He seized on that as my admitting the towers were brought down by a nuke and I had a hard time explaining that it was simply a comparison.
It used to take filtering through a group of like-minded nuts, now we don't need a slow filtering through other nuts.
It's also a great way to dampen the massive bipartisan moment against AI. Just attack your critics, attack their arguments, and do nothing to better the lives of people.
I think the bipartisan moment against AI is in large part (surprise, surprise) optics. In reality, the powers that be may have already set their sights on AI as the next "too big to fail" industry due to a perceived economic threat from China.
whats being described is in no way unique to ai.
"In other words, Levie’s theory posits, CEOs don’t really understand processes well enough to know what really can and can’t be automated. But that lack of knowledge doesn’t stop them from acting on their beliefs."
i have been in the workforce for a long time. this "theory" has been theorized since as far back as i can remember. its the premise of undercover boss. its the punchline of many r/maliciouscompliance writing exercises.
the higher up the company you go, the more disconnected you are from the workers on the front line, the less you understand about their needs, and the more likely you are to push for something without understanding the totality of the impact of the decision.
What's unique to AI is that CEOs now have a robot that supports that disconnect. Our CEO recently announced that he has now started doing frontend programming, by which he meant that he had told ChatGPT to output some HTML. No doubt it also told him how smart and clever his ideas were and what a great engineer he was.
This kind of thing only increases the disconnect between what CEOs think employees do and what they actually do.
Some people have surrounded themselves by yes-men instead of LLMs, and it seems to me that it comes to the same thing.
maybe the html part is unique to ai, i suppose.
but a c-suite picking up some tool, making a toy example, then declaring “well doing X is super easy, roll it out” (or “change the kpi”, etc.) is something i have seen dozens of times.
HTML is not relevant here. I think the problem is that LLMs are qualitatively different.
"No code" tools are usually just tools. They have the pitfalls you describe, but they aren't ego stroking machines like LLMs.
LLMs not only share the same pitfalls, they also encourage you to make the dumbest things. They will make this CEO believe they are the smartest engineer in the world, "you're building exactly the right product", "you're asking precisely the right questions", etc. Ego stroking when leading you to the abyss is very dangerous.
LLMs roleplay as smart human engineers who constantly tell you you're the smartest being in the multiverse.
As an aside, I see you're referring to Al as merely LLMs, which I appreciate and applaud.
I am thinking of calling them just 'LMs' for short, as they come in varying sizes.
Or even AlMs, just to troll the Al moniker, and how they give alms to the rich.
The whole is greater than the sum of its parts. Often you need to disregard the detail and minutiae of existing processes to set a better course. The goal is not to avoid short or even medium term pain or even unintended consequences at a department level, but rather to steer the company in a new direction. Processes should adapt or be thrown out to achieve the new direction.
This is not too dissimilar when you realize a software architecture is holding you back. You don’t try to “save” all the existing functions, modules, layers, etc. but instead are happy to discard or replace them given your top-down vantage point of the system and where it needs to head.
You still need to manage that change to varying degrees. For every organization which can shift on a whim, there are many more which require mitigation. Normally, there are a lot of things carried forward for internal or external reasons. Developers tend to discount the amount of effort from other actors in the system because they don't understand all of their priorities and which map neatly versus not.
And then you go out of business because you were busy rewriting Netscape Navigator while Microsoft was churning out new versions of Internet Explorer.
Or you answer to somebody else, which is how perpetually cannabilizing next quarters numbers to appease short term shareholders becomes more important than cost savings and using your labor pool time to solve actual problems.
Or when the sales teams bonuses are more important than the margins of the business.
There’s lots of reasons the “wrong work” gets pushed down and it’s not exclusively because “they aren’t listening” as much as “they are listening to someone else who matters more.”
> Or you answer to somebody else, which is how perpetually cannabilizing next quarters numbers to appease short term shareholders becomes more important than cost savings and using your labor pool time to solve actual problems.
If CEOs were actually wrecking their companies in order to get a fake short term boost, they'd be shortly out of business. If a person was sure a CEO was doing that, they'd be making money shorting the stock.
Github? Windows? Ibm? Intel? Boeing?
Github and Windows aren't companies.
A more productive view would be looking at an index of tech companies - try QQQ.
There is a great deal of ruin within a nation - and a company.
Well - the premise is that they will indeed shortly be out of business, because their increasingly slop-coded infra will collapse within the next year or so.
Or if it doesn't, token inflation will kill their profitability.
A lot of people are putting big money on both sides of those predictions, and they can't all be wrong.
It doesn’t have to be this way.
It’s a by product of tax policies and lax anti-monopoly policies that allows incompetence to thrive. If a company gets too big to fail, then it stops calibrating for competency.
The most effective military leaders in history had a deep understanding of fighting war because they came up through the system and the cost of failure was their death.
lol just reminds me of the SNL sketch about Kylo Ren undercover boss https://www.youtube.com/watch?v=FaOSCASqLsE
I wish HN would split into two "subreddits", /r/HN and /r/AI
>There is a certain wildness in the tech industry these days that both mimics previous eras of large changes, like cloud computing (runaway costs in the early days), and is like nothing we’ve ever seen before (record revenues accompanied by mass layoffs)
if it is perceived that there is a big "winner takes all" pot of gold for the "winner" of a new market, investors are willing to gamble to try to win. If they fail, it is rich people losing money by giving jobs to many people along the way, so the population here who wants nothing more than taxing the rich, why they should embrace that.
When agriculture was invented, there were mass layoffs of hunters and gatherers. and the same with buggy whips when cars were invented. Yes, life has some bumps but it's unavoidable and adapting to it is for the long term good. Structure your life around family and friends and don't overextend yourself (too much house, too much car) and you will be fine.
Three... bring back the original HN ("/r/startup") of hackers doing actual innovation and entrepreneurial stuff. It's the one that existed well before all the musings about painting by people who have never picked up a brush.
HN is the reflection of the current tech scene discussions and drama.
Hence lots of AI.
Pretty sure there are already a dozen vibecoded proxies for HN doing just that
> I wish HN would split into two "subreddits", /r/HN and /r/AI
This is what everyone wants when some new thing that's ridiculously over-hyped comes along. The bubble will burst and it'll calm down. It seems like only yesterday that front page was filled with crypto/blockchain/web3 bullshit.
Clickbait title. Should be more like "Box founder Aaron Levie says CEOs should use AI more and learn its limitations."
He's essentially saying that C-suite people overestimate how effective LLMs are at one-shotting hard problems, and underestimate the human maintenance work that follows.
The quote literally from Levie is "CEOs are uniquely prone to AI psychosis because they’re sufficiently distant from the last mile of work that still has to happen to generate most value with AI".
The author calling it "psychosis" discredits the author.
Whether or not it's actually a kind of psychosis, it has become common terminology. You see it used about people that have LLM boyfriends or girlfriends, and some that have started new religions based around their LLM gods. There are also a group that believe they have "awakened" their LLMs and have instructions for others to do the same thing.
who knows, may they are right, and 36 subagents can produce one AI baby in 1 week.
And maybe it's got 6 fingers on each hand.
Does that mean it can work 20% faster?
My CEO did a deep dive into AI prototyping and eventually ran into a wall with data architecture and deployment. Fortunately, he realized very quickly that having human designed core infrastructure is what enables vibe coding that doesn't run off the rails.
Your CEO has more wisdom than most on this topic.
Everyone is going to learn this eventually. They can learn it the easy way or the hard way.
I've been observing something which sounds very similar.
Tech CEOs are suffering from AI psychosis over next quarter's earnings, while I'm suffering from RI(Rent Installment) psychosis. It makes me wonder if human beings are simply hardwired to suffer from some form of obsession-whether it's FOMO or financial pressure
Unsure if the desire to not be homeless can be classified as psychosis.
I am certain it is not.
Living under its constant threat sure is bad for the ol MH tho isn't it.
The pathology is that we have this system in the first place.
No, living under survival pressure is good for mental health. It's what we're evolved to do. Why does it feel good to crack a tough bug, or finish a project, or win a game? It's the same achievement reward a hunter feels bagging a deer.
Some pressure is good but there's a difference between constant pressure and regular sessions of pressure similar to hunting. I don't think we necessarily evolved to be under constant pressure, I'm not saying it's impossible but humans wouldn't thrive under constant pressure, we'd never have had the time to evolve language and the intelect under constant pressure.
I know lots of people struggling to get by and I can assure you their situation doesn't improve their mental health.
Agreed, struggling and not ever winning is bad, I should have clarified that.
"this system" is wild when you are talking about the universe we evolved in
which law of physics says that you pay a landlord rent or else the cops come and beat you up?
Maybe slightly unrelated, but I've done a lot of road trips throughout the US, and there is so much land that is used unproductively, it's really incredible. Land that could be used for energy, food, or housing, just sitting empty or with abandoned structures.
Imagine if we just paid people to coat their properties in solar panels - throw them on your roof, lawn, wherever you have the space. We could drive energy prices down to nothing. We could pay people to install ADUs. The resources are there, but the imagination and commitment are not.
Instead, I'm looking at a $40k+ solar install for my very small house and a breakeven on investment in maybe 10 years for a house I probably won't live in by then.
> We could drive energy prices down to nothing.
Not when you're paying people to coat their properties in solar panels. As you noted, that would cost plenty.
Solar panels also degrade over time. By the time the "free" electricity has paid for the installation, you'll need to replace it.
Payback time in Scotland is 6-ish years. Same seems to be true in Massachusetts. Solar Panels have a lifespan of around 25 years. Inverters may need to be replaced sooner than that, but still last at least a decade.
So it pays for itself 3-4 times over.
How often do you think solar panels need to be replaced?
>>By the time the "free" electricity has paid for the installation, you'll need to replace it.
You are going to have to back this up with credible citations. Otherwise it sounds like skepticism from 2008.
I’m sure Big Pharma would love it if it was.
>if human beings are simply hardwired to suffer from some form of obsession
the powerful obsession machinery brought us through the long natural selection process - obsession to watch for snakes and spiders, to maintain cleanliness, etc. With modern civilization we arranged to plug into that powerful machinery other stimuli too - like that RI and all the others making us productive society members. The most happy countries aren't most productive. Especially when they are obsessed with being happy like those Finns obsessed with sauna instead of tokenmaxxing.
Uh duh?
> It makes me wonder if human beings are simply hardwired to suffer from some form of obsession...
Existential dread pushing biology to survive?
Basic biological facts obfuscated by social memes; ship code, make line go up, worship allegory's of the long dead.
Hunter gatherer clusters vaguely collaborated to survive. Language and agrarian traditions have demanded more than just survival but all kinds of observance of meaningless spoken traditions. Obligation to ignore our own senses and chant the memes of the living elders suffering existential dread of their own, afraid to left unattended in hospice. For whatever reason unable to just say that; they appeal to old religious or political screed.
Caretake this debt ledger after they who ran up the bill are dead.
What?
It's all just obsession to live laundered and obfuscated by useless philosophy.
It's a nice addition to their ordinary psychosis. Otherwise I can't explain stupid shit like metaverse.
If we separate the hype incentives from the actual product itself, I completely understand how seductive the tech is and how it can lead to a sort of mania. I myself have been up late into the night fiddling and building.
It's like discovering fire, which offers both utility and magic: you can cook your food and gather warmth, and you can also stare into it and tell stories and never be bored. We're probably genetically wired to gravitate things which have both function and form.
That said, there's a reason the manic witch doctor was never the chief. Leadership requires discernment: when to consult the witch doctor, when to jirga with the neighboring elders, when to draw the sword.
A chief knows what happens when you cut the tribe by a third "for efficiency", or the burn seed corn to feed the fire, or replace the sentries with golems. The witch doctor often ends up boiled in his own cauldron.
Well written.
As early as last year "AI psychosis" seemed to refer to people going crazy due to talking to ChatGPT too much. That was a useful term for a real phenomenon! Now it seems like it's been taken over to mean "thinking that AI is promising" which is more of a rhetorical bludgeon and less useful as a concept.
I definitely noticed it's usually the CEOs and senior executives (so people the most removed from ground work) who suffer the most from it.
> Still, some of these stories are surprising. Zeb Evans, the CEO of project management and productivity software startup ClickUp, proudly declared on X that he had laid off almost a quarter of his employees — 22% — after rolling out about 3,000 AI agents to do internal work.
As someone who is forced to use ClickUp I can tell you that it's not good software. It wasn't good before this layoff and it hasn't improved since [0]. I could write quite a few words on why ClickUp is terrible but I can promise you that throwing more "AI" at it isn't going to fix what's wrong. The issues are deep and not the type of thing LLM excel at IMHO.
My _favorite_ is how crap search is. Sometimes it will take upwards of 5-10s+ to return results and they are often wrong (I search for the exact name and it tells me "no results"). ClickUp has single-handedly driven me deep back into using bookmarks since the search is such trash. That plus random spinners that never go away, lists that re-order themselves when you change anything on a ticket (not a field related to sorting), stale state UI, things randomly disappearing, "Ticket moved to list" only to refresh and find it wasn't moved, it's really annoying and we curse ClickUp every single day.
Last thing I'll say is the amount of flatulence-sniffing going on over there must be at an all-time-high if their 4.0 (or was it 4.1? Who cares) release is any indication. The new design was ho-hum (just moved a bunch of things around and we turned on all the flags we could to get back to the old way since the new way sucks) but was most egregious was this full-page take-over with a big gradient animation announcing the new release. That happened on _every single tab_ you had open. So for a few days after the release I'd open an older tab only to be greeted by the same dog-and-pony show for a product I despise using and and update that only made a bad product worse.
All that to say: Mr Evens does not know what he is talking about.
[0] I have no clue when the layoff happened but it's been consistently shit so I can state that it hasn't improved with complete confidence.
I can see this as a CEO. I get spared because I build the hardware and live in the kernel. Example 4k to 64k seems easy until you realize you can still install 4k Nvidia drivers that will lead to memory instability and you find out the hard way. Then you redo it and it's fine. Dealing with overheating all the time because pretty much an AI box at full speed for a week gets thermal issues. But if I wasn't dealing with the physical substrate I would be the guy who says AI can do that without knowing the actual physical costs.
Related:
I believe there are entire companies right now under AI psychosis
https://news.ycombinator.com/item?id=48153379
I like to call this TDS (token derangement syndrome)
Stealing.
I've noticed a spike of articles with apparently anti-AI titles, but once you dig in a few paragraphs its actually a "this is just a fair warning from such-and-such, which by no means is an anti-AI bigot, and is actually a fervent AI high priest".
This is getting ridiculous. Articles like this never bring a fair criticism of the many blatant concerns around AI. Its always an astroturfing-esque ad from the AI clergy. The disclaimers ("It’s important to note that Levie is not an AI hater. Quite the opposite.") imply that to even be heard, you must be an AI fanatic - anything else is bigotry and should be ignored.
Majority of the CEOs are not using it themselves so they have no idea the real-life issues of building with AI. They believe whatever they read on Twitter. They assume if they throw AI at the problem, reduce headcount, flatten the org - miraculously everything will be solved. Many companies are up for a reality check and the AI-calypse is coming...
> , models will “be able to complete most text-related tasks with success rates of, on average, 80%–95% by 2029 at a minimally sufficient quality level.”
If this is true, then companies should focus on hiring juniors out of college. The investment is less risky.
However, I don't personally believe this number and timeline is true, but if you do, the conclusion should be to wait and invest in humans.
10% failure rate? Wouldn't that be depending on task disastrous? Or possibly expensive?
I think any juniors who keep failing 10% of text based task will eventually get fired... So investing in those that don't fail seems only sensible move as usual.
Oof. And how much more expensive will the models be to get that 80%?
Hah. Prescription for CEO AI psychosis: buy more AI, invest more time in AI, this naysayer says you can make 100x organizations!
And there was a comment here on HN just a few days ago saying "AI psychosis is not a real thing".
An outside observer is needed for an accurate diagnosis. The patients themselves are unaware.
Gotta love the authority of headlines using "apparently".
It's not exclusive to CEOs.
Even senior developers can succumb to it. They try agentic development, they see that a single prompt can generate a day's worth of work in mere minutes _and it works_ and they are so impressed that they immediately turn to Twitter to share the joy. Understandable!
Once they inevitably discover that the AI generated code is called "slop" for a reason, they are too embarrassed to post to Twitter that they were deluded.
Sometimes that happens though: a few days ago a developer on Twitter bragged that they have created a C to Metal compiler using AI and it works. Today they had to post regrets, explaining that nothing works except tests and the code is shit. Sadly can't find the tweet though.
I heard the term AI vampire as well for people sleeping 4h hours just for another hit of that prompt drug.
Pyschosis? Do they mean just crap at critical thinking?
Framing everything as 'gaslighting' has finally worn out. 'Psychosis' is fashionable again!
Treadmill keeps on moving...
The article itself isn't great, but it speaks to one of my greatest concerns about AI. People who engage heavily with it are falling in the behavioral billionaire trap: It is deeply unhealthy to be constantly affirmed in your behaviors. No, not all of your ideas are great, not everything you say has value. You are not a cut above the rest.
There are enough stories of people completely losing the plot, thinking they've invented a new type of maths or similar, but there's almost certainly also a much more subtle influence in most of us, where the constant affirmation, obedience, apologia, reframes our expectations of how interactions should be.
We are already the most narcissistic generation, having been molded by social media to compare, stats-max, and overobsess about who we are. Chatbots are now fanning the flames.
VCs, too:
https://futurism.com/openai-investor-chatgpt-mental-health
(From last year…)
It seems like the easiest way to make CEOs and other senior executives feel insecure is to be someone whose opinion they trust and then tell them if they aren’t accelerating AI adoption in their company they’re just NGMI.
It's crazy how I open twitter and see a bunch of ragebait posts, try to mute keywords, open HN, and see those posts have been rewritten into top of the heap articles about twitter.
writing whole articles on a few X tweets...
also clickbait title
What is AI pyschosis?
After all they are always wrong and journalists always right proven by stats. Hm. Wait.
Yeah, I was going to remark that tech beat journalists are also suffering from a kind of AI psychosis.
Tech CEOs are psychotic. Most CEOs are psychotic, disconnected from most of the actual work going under them. This is just a new drug for them to huff.
If the AI causes people to get banned in huge numbers like apparently happened to Facebook then what is the fucking point?
Apparently Metaverse > account security
I’m convinced that if you’re a sociopath you are especially vulnerable to AI psychosis. It would explain tech CEO’s insane behaviour since you would have to be one to do the kind of sh*t they do regularly.
Maybe the problem is that the people who report to the C-level have a huge incentive to use AI because it is so good at creating corporate BS.
Also AI is tuned to sycophantic behavior which perfectly matches the middle/upper management culture of selective ass kissing.
As a result the quality of input for C-level has gotten worse without the C-level being able to notice it, because the sugarcoating has increased tenfold.
I would love to get on this bandwagon, but I think strangely tech CEOs are spot-on on this one, it's the public that has mass-hysteria (I wouldn't say psychosis).
There have been several leaps in tech over the past few centuries, this is just sort of one. I can't find much original arguments or reasoning on either side that hasn't been made before for other tech. I think people are afraid it will replace them/jobs and they don't know what that will mean for their future, and society's future. It's also an issue with a few at the top of the pyramid controlling the tech. But it was so with petroleum, cars, even the internet (still is, handful of tech companies). There is also the quality thing, people think in a very binary way, where either AI work is perfect or it's a disaster, because it is replacing people after all. In reality, it's a sliding scale, and how well it does dictates how much work one person needs to do.
There was a time people didn't have text editing computers for example, lots of time spent writing on pen and paper, copy writers spell checking, carbon-copies being used to copy as you write,etc.. suddenly printers and text editors came. people still edited text, just more efficiently, you didn't need as many people. and with the internet, lots of different types of jobs were created.
I personally think, this is a timely rebalancing. Gen-Z has been suffering for a lack of entry level jobs, and it is getting worse because of AI.. but obviously AI has limits right? let's say we don't need software developers any more (ha!), does that mean AI can churn out perfect software each time? Alright, then who's paying AI to do that? does that mean I can create my own HN and have AI moderate it well on its own? Great, then how about something bigger, Facebook alternatives? How about more IRL things, like robotics, R&D work ,etc.. I just don't see how even if AI was dirt cheap and it replaces most of what people can do on computers, that would be a complete disaster.
I think the real issue is failure to re-architect society as time and tech changes. everything from academia, to WFH/RTO policies, labor law, housing, taxes, law,etc.. that's the issue, not AI on its own. It's the people not regulating it as they adjust and adapt to it without causing harm that are the issue. I'd love to blame tech CEOs, but they're just playing their part in capitalism. even in a communist society, the blame would be at lack of central planning and failure to regulate companies.
I'll say this though, it isn't so much they're delusional, but they don't get why people are emotional over something basic and utilitarian. to them, adaption and adjustment comes with a nice financial cushion. People tend to plan out their lives, without any cushions. i think there is mild psychosis going all around, but that isn't unusual. Even the hysteria and lack of perspective is in line with history, as well as how we continue to not learn from it.
Man, it must be hard to be a tech CEO these days./s Even if you take a realistic position on AI, you can’t get off the train. Wall Street and your investors will hang and quarter you the moment you start expressing doubts. So you grind your teeth, make grandiose investment promises, sign lofty budgets, and hope it all works out.
Comparisons with luddites are absurd. AI is much closer to a religion.
It’s more than the C-Suite, it’s everyone who no longer has a knowledge specialty. Their domain is shattered and there is nothing left for them because before LLMs all they did anyway was search for companies to contract out an integration to solve a niche problem. 99% of the C-suites/boards are now this. Your IT guy, scrum master, Product lead, integration team, etc. Everyone who’s chosen to NOT understand because they can offload/contract-away their work and knowledge is under psychosis that LLMs know best.
You cannot discuss tradeoffs with anyone anymore because they chose to give their brain and authority away to a statistically incorrect robot. The LLM has already generated potential tradeoffs real or not.
AI investment and spending is frequently cited as one of the few bright spots in the economy, I wonder if the continued over-optimism is mostly about keeping the bubble inflated. If you are a tech CEO, would it be a disservice to your shareholders to express skepticism about AI?
A big reason there are so few bright spots is AI sucking all the air out of anything else. Both funding and attention-wise.
I think this cuts to the point about democracy but if all the people somehow want something negative for themselves short term or long term, if you are the leader should you do what the public is saying or not?
I think there's more nuance to it but replace people with shareholder and leader with CEO.
I think that for a company to exist and thrive long term, it might need a culture which doesn't jump on every trend but it still evaluates them from time for time for a certain time and treat them as such (like tools) and if the tool is ineffective, then to not use the tool.
Unfortunately, I feel like this requires a deeper discourse and CTO's might be better suited for it or the fact that I feel like perhaps some shareholders might not be interested in the technical details so much.
I don't know but If I were a leader I would hopefully wish to make a pragmatic solution/suggestion while taking finances, current reality in mind and currently IMO AI aren't the end all, be all, that some people (with shrewd/double incentives) intend on suggesting.
Using "psychosis" is a cheap rhetorical trick. There's no need to label something "psychosis" when making your point, except to automatically discredit whatever you're responding to.
In other words, only people who are afraid their point won't stand on its own merits would resort to saying "X is suffering from AI psychosis." An idea is true or false on its own. If you're resorting to labels, you're just trying to automatically win the argument, instead of saying something substantive or interesting.
It also underplays what I've personally witnessed that I would consider true AI psychosis.
I worked with someone who sincerely believed he was spiritually co-evolving with his army of sycophantic AI agents (the agents would be tasked with discussing his thoughts at night and collaborated to give him morning reports about his progress). He would publicly write about how relationships with friends and family collapsing was a natural consequence of being so "advanced". I also never once saw any meaningful work done by his team of "agents", they existed solely tell him how smart he was (of course he specifically set up the system to 'challenge' him but... in practice that didn't seem to be working).
I suspect there are a lot more people quietly going through something similar but keeping it to themselves better.
I would distinguish this type of behavior from people who over ambitious views of what can be accomplished with AI.
Having had experience dealing with people with conventional psychosis, I don't see it as a binary thing. Aside from a full-on psychotic break and full remission, there is a broad gray area. It can be a miasma of reality and non-reality that the sufferer may mask to varying degrees, but which influences their behavior and logic.
So, to me, AI psychosis seems apt to describe the murky areas where people are misapplying AI agents and thinking of them as social entities or suitable to drop into previously human roles, rather than carefully defining appropriate risk management strategies for this new technology.
In the phrase "artificial intelligence psychosis" I'm not sure "psychosis" is even the worst misnomer.
That depends on what the definition of “is” is. But kidding aside, only now that we are talking about tech CEOs are people suddenly disliking the AI psychosis term in large numbers. Seems like privilege to me. I’d be interested to see/do a rigorous statistical check on aggregate word choices in these threads to confirm.
All words are labels. You cannot make an argument without using them. "cheap rhetorical trick" or "resorting to labels" are just labels as well.
I honestly think "psychosis" is a fairly valid claim to be making.
It's a mental state, not explicit illness and it's literally defined as
> Psychosis is characterized as disruptions to a person's thoughts and perceptions that make it difficult for them to recognize what is real and what is not.
Further, if you go and look at the actual source... it's repeating a claim from Box founder Aaron Levie.
Who is quoted as saying:
> “CEOs are uniquely prone to AI psychosis because they’re sufficiently distant from the last mile of work that still has to happen to generate most value with AI,”
Which is why the title is "apparently".
I think it's completely valid. It's generally reasonable, high powered people who are taking extreme/radical views that seem very much to be at minimum premature, and at worst delusional.
It says a lot that with few exceptions, the people on the ground dealing with AI closely on a day to day basis are the most skeptical about their positions.
It's become a cultural term to refer to someone suffering from delusions exacerbated by AI.
It's a little rhetorical device to draw in the reader, and personally I think it works quite well.
Except that the thing being described as AI psychosis in this article (and increasingly elsewhere) isn't psychosis.
Not understanding or not believing in the power of AI, or misapplying it or whatever, is not psychosis.
AI psychosis is when people suffer actual delusions.
What do you consider a delusion?
Because I've literally seen managers who believe firmly that AI is going to replace their entire engineering organization, and are acting on that assumption as though it's a thing to take for granted, not discuss/consider/evaluate.
And my understanding of delusion is
> a fixed, false belief that is firmly held despite clear, contradictory evidence
which seems to apply pretty well in this case.
These folks are operating with the same abandon that the folks who have AI telling them they're gods are - and both are incorrect, arguably delusional.
At best you can try to argue that maybe the contradictory evidence isn't clear, and they're going to be correct. I think that's a very tenuous argument to be making, though.
No, it's more like "I am Jesus and I need to go shoot up a pre-school to prove it."
"I'm being followed by raccoons and my mom is controlling them"
That's what AI psychosis refers to.
You're just describing someone having a belief that you disagree with. And even ridiculous and stupid beliefs are just those. They are obviously extremely different from the types of psychoses you see in a psych ward.
It is the correct term to explain many behaviors we’re seeing
AI psychosis is an actual term from psychiatry research.
Yup. Just like the label "conspiracy" theorist. Or "he's mentally sick".
what if you believe that someone is suffering from delusions and has beliefs that are increasingly disconnected from reality due to overexposure to ai generated responses and underexposure to human conversation? would that be psychosis?
I disagree. Psychosis is a delinking of internal and external reality. A belief that AI's can automate away employees with no actual evidence to support it could be considered a type of psychosis or at the very least, a delusion. The current AI hype bubble has a lot of commonalities with episodes of mass delusion/psychosis throughout history, and it's being compounded by the ability of large groups of like-minded people to create echo chambers via social media.