In short - the reason medicine is slow for AI is the same reason it's slow for people -- the REPL loop is slow [1]. If you could pause and rewind time, and had perfect observability into every system, no ethical risks, (things that are free in software engineering, or math problems) then the AI could make massively parallel progress.
However I do think there are opportunities to greatly improve all these fields with AI, I just don't think we should lead with an overpromise that sounds Musk-like.
You're thinking like a law-abiding Westerner. The ethical risks are not a concern for all, if anything the humans are the cheap bit of the process. There are a few ways to improve a genetic algorithm exploration, one of them is to simply run a larger population at a time.
There have already been a few attempts by pharmaceutical companies to accelerate drug development by running clinical trials in poor countries with weaker patient protections. Regardless of the obvious ethical concerns, it turns out this doesn't actually speed up the process. It still takes just as long to discern a clear signal in terms of effects versus side effects. And those poor countries typically have infrastructure, security, and labor force problems which make it challenging to run large scale clinical trials at all.
If that is true, then even before AI pharmaceutical companies would have already moved all their research to places where their development would be much faster (but less ethical).
Pharmaceutical companies don't care about "ethical" research either, all they care about is making more money, faster.
What's suspicious about this is not whether it may or may not be possible (it likely isn't), it's that he is using it to borrow money now, that is not collectible in the case of a default. You get a huge exit because you overpromise today, and when it turns out you aren't a prophet, those that trusted you cannot be made whole.
People and the law keeps treating this as "just a sales pitch", but it's really theft in disguise and should come with evidence on an S1 filing.
What do you mean it’s not collectible in case of default? Are you under the impression that money that you borrow before the IPO is excluded from the bancruptcy claims?
>> Are you under the impression that money that you borrow before the IPO is excluded from the bancruptcy claims?
it's being "borrowed" in the sense that there's an implicit promise as part of the IPO sales pitch, but shareholders are last in line to collect in case of something as extreme as a bankruptcy - and that's if there is anything left if that was to happen, which by definition is not frequent.
>because you overpromise today, and when it turns out you aren't a prophet, those that trusted you cannot be made whole.
Why are these people obligated to trust Amodei? Why are these people obligate to hand Anthropic money?
I really hate this "Investors should be protected from losses" narrative you see pop up now and again like that isn't the entire natural order on how we conduct these things and somebody not being made "whole" is somehow unfair. It's not theft, Dario will simply accept your money if you give it to him.
You make a bet you can lose. These are grown adults making decisions.
I sure hope no major indices bend or break the rules about including <s>bubble stocks</s> freshly IPOd companies in them, since my entire prospects of retirement are tied up in those indices being relatively stable!
I really hate this "Investors should be protected from losses" narrative - right, we should also retire the SEC and every investor protection law and regulation, because what where we thinking?
I’ve said it before and I’ll say it again: nobody pulling the strings in this industry gives a shit about anybody, or really anything other than increasing their available capital. They’re powered by some combination of greed, inertia, and religious fervor. Their primary role is to groom investor sentiment and the weak job market means they needn’t feign benevolence to recruit competitively. This is the purest expression of post-.com era SV culture I’ve witnessed since I started in tech nearly 30 years ago.
There are people who genuinely want to help others because they realize it is the right thing to do. Also applies to organizations governed by such people.
I used to take my truck to Larry Roberts of Roberts auto. He’s a neighbor and friend and did oil changes and basic stuff. Larry closed up shop cause he can’t work on modern cars and doesn’t have the tools or staff to handle anything more than basics for cars older than the 00s.
Larry is great but he’s not even relevant to the discussion because you might as well be describing a DIY or hobby shop.
None of the people or organizations you describe are relevant to how the world works, they are temporary outliers with limited impact for a brief period
Even then, Larry didn’t work for free and wasn’t running it out of the goodness of his heart. He did it for money
The only people I know doing shit that isn’t for money are the people that actively work on the trails that I walk on in Northern Virginia and guess what, they work on them so they can have nice trails not because somebody else is going to use the trail.
Absolutely false. I’m not young, and I’ve worked in many different industries. Once you get away from for-profit businesses controlled by third party investors, that could not be further from the truth. Having monetary motives doesn’t mean you only have monetary motives, or lack willingness to address your industry’s ecosystem, employees, and the communities you touch. Believe it or not, some people that run successful companies have pride, ethics, and red lines they will not violate to pump up their valuation or throw a few extra bucks in their pockets. I left software because considering anything other than money is anathema to its business culture. I’ve worked extensively with people in arms manufacturing and even they are more motivated by business ethics than any executive I’ve encountered from the SV software scene in what feels like a fucking decade. For an industry that was started by people that wanted to make money by solving people’s problems using innovation and cutting edge tooling, this is a pretty pathetic state of affairs.
I was a military officer for 17 years, the only thing that matters is your unit/wing/program office is your budget. The only thing more important is readiness and guess what the main thing that effects…your budget
The idea that arms manufacturers are more ethical is a joke - at best you get propagandized people ignoring how their munitions are used while banking 150k salary minimums.
I’ve run mutual aid cooperatives and whatever you try to do will be overrun by private equity and your market will be closed and the people who show up will be flighty and noncommittal
I currently am ED for a non-profit that I will not allow to be used for profit making - almost 100% of the people who ask to volunteer or show up do it for their own promotion or to jumpstart their product it’s maddening the number of times I have to say no.
I am the founder of the 150,000 person ultramarathon subreddit and I’ve only found two (2) moderators in the last 15 years that aren’t trying to promote something for money.
There are no organizations that escape this because at the root level you can’t find more than a literal handful of people who aren’t in it for the money - even if thats not their direct reason it’s usually a secondary reason
Even “monks” go out and write books and buy fancy cars
There doesn’t exist in my daily survey of social organizations ANY, and I mean there isn’t any single organization, that does what it says it does without trying to make a profit in some way.
> I’ve run mutual aid cooperatives and whatever you try to do will be overrun by private equity and your market will be closed and the people who show up will be flighty and noncommittal
… yeah… right…
> Even “monks” go out and write books and buy fancy cars
You are acting like a bitter, cynical edge lord that only sees the worst in the best people. Ironically, I’ll bet you think you’re the most rational observer you know.
What you say you see in the world says far more about how you see than it says about the world. Later.
Yeah, I'm surprised we're all discussing it. I'm open to the idea that AI could help medical research, but just claiming it'll end "alzheimers, cancer, heart disease and mental illness" with absolutely 0 evidence isn't a particularly grounded claim that warrants any discussion.
Honestly, the way he lists them and even pauses before vaguely saying "mental illness" gives a strong impression that he's choosing whatever illnesses he can think of as he goes.
"'m open to the idea that AI could help medical research, but just claiming it'll end "alzheimers, cancer, heart disease and mental illness" with absolutely 0 evidence isn't a particularly grounded claim that warrants any discussion."
Agree with this if it was something a random person said over coffee or lunch.
But when its the CEO of a company that is about to IPO saying it, its very very different and more concerning.
Seems unwarranted? He is clearly following the MO of Musk by hyping up his company as being capable of doing the most ridiculous innovations but all it does manage to do is bump the stock. Terraforming Mars, tunnels for self-driven cars, curing cancer/all mental illness etc.
Yes. Expressing a belief or hope to achieve something in the next 5-10 years is not a claim of capabilities.
If you want to argue that Amodei is dishonest and only says these things to 'bump the stock', you can make that argument, but it's clearly speculation.
When someone promises something that's so far out there as to be nearly impossible, and *without an actionable plan to get from here to there*, then they're either dishonest or delusional.
Like, if your optimism is great enough, big enough, then you lose touch with what’s actually feasible. That’s where delusion starts.
It might be possible to compress 100 years of research into 10 years. Maybe. But it won’t be these 10 years.
Check the math. Say we get huge advances in AI tomorrow that double the speed of biology research. And imagine we then have 3 years where research is now 2x faster (which, again, would be absolutely enormous).
How much faster do the remaining 7 years have to be, in order for the entire 10 years to capture 100 years of research? Roughly 13x.
Basically, every year you’re not already at >10x is a year you’re falling behind. And AI is nowhere even remotely close to doubling the speed of research, much less 10x-ing it.
Amodei's statement is at the level of Archimedes saying "I'm guessing that in the next 50 years we'll probably use this to fly to the moon" after discovering Greek fire. Sure, it's "optimistic", and may even point towards something that will eventually happen, but way way way past the point of delusion.
Okay but let's for the sake of argument that Claude magically has all the information required cure all of these families of diseases. How does Amodei intend to deal with the fact the regulation of medical devices is a slow, lengthy process for a lot of very good reasons like preventing something like thalidomide going on the market? Is Claude going to magically spin up considerably more regulatory capacity out of thin air, and will it gather evidence for safety and effectiveness at the required standard to show to those regulators?
This is just one problem I have with his claim, there are many others.
Read the letter being discussed. Alongside claims of medical innovations, he also says it can be used to speed up government bureaucracies.
Now, whether that's true is obviously it's own question. But I do think if AI can cure cancer and Alzheimer's rapidly, that improving the regulatory and approval processes at agencies is a much lower bar.
A lot of those bureaucratic regulations are written in blood and are slow and deliberate for a reason. Throwing them all out means more thalidomide disasters: Chesterton's fences and all that.
Credentials are one signal sure, but his MASSIVE financial incentive to hype the product is another one that should probably get much more weight in your analysis.
I'm genuinely trying to understand the argument you're making here. What sort of "credentials" entitle a person to claim their god-technology will cure all diseases within the year 2026?
It smells to me like "might be an even bigger optimist than I am" means you're also part of the rationalism cult and that it's literally impossible for you to say something negative about Amodei.
> I'm open to the idea that AI could help medical research, but just claiming it'll end "alzheimers, cancer, heart disease and mental illness" with absolutely 0 evidence isn't a particularly grounded claim that warrants any discussion.
This is up there with the claims Chan-Zuckerberg Foundation made during their start-up that the foundation would "end all disease within our lifetime." So far they haven't ended one disease, and meanwhile they quietly watered down their claim to soften the language to update to "cure, prevent, or manage all diseases by the end of the 21st century."
Does narcissism qualify as a mental illness? There might be one more step here to complete the circle. Both Amodei and Altman have dropped some real shockers over the years and instead of seeing a reaction it's almost as if they've normalized the kind of talk that ought to see you in a room with soft walls.
You could have said same when he claimed "90% of code will be written by AI" in the beginning of 2025, but turns out he was right and he foresaw something we didn't.
That prediction (or at least that article) was written in March, and Opus 4.5 was released in November. For my team at least, that was pretty much the point at which 90% of our code became machine written. So that would mean he was only off by 2 months. Maybe 4 or 5 months at most for the point where it became true for basically the entire software industry.
I’m not saying he’ll be right about this one, not it’s a not ridiculous to claim he want fundamentally right about code.
The rest of the computing world has had machines generating code since the 1940s when the first assembly languages were created. What sees you cling to the raw bits and bytes still, rapidly approaching a century later? Amish?
90% of the code will be written by ai in 6 months.
Hmm. People making fun of the idea that 90% of code can be written by machine.
Now you come and nitpick about if it is 89% or 7 months instead of 6. Lmao.
If AI can get any of those in even 20 years, that's still a big win.
The pessimistic attitude here is revolting.
And no, nobody is forcing you to buy anthropic shares if you think they are full of shit. Anyone complaining is either don't have money to buy it or just jealous of other people buying it and price going up than the ones they hold.
This is a pretty good question. Maybe LLMs are just, in some sense, ~10x worse at DRY than humans.
Or, I mean, in my code at least I can directly say that any line I write results in 4 lines generated by macros. So, I guess my projects were already 80% written by computers in some sense. If I was using an LLM would I have skipped the macros entirely?
I’ve always been in favor of doing this sort of measurement by flop instead of loc, but, then we’d have to give all the credit to compute libraries.
yeah, sure we will nitpick the meaning now, on a personal level I stopped writing code manually at least 7 month ago.
its helping me convert my ideas to code, which is not too far from the meaning implied, because things don't happen on their own, there should be a trigger, in this case, I am the trigger, AI is the producer
I started coding at 8 years old, 28 years ago, been doing it since.
Switching over to mostly LLM generated code and it's been mentioned many times in my circle and family that they can still tell the projects are blatantly something I would do, what it would look like, and how my projects usually work in terms of UI/UX.
I get basically the same outcome as spending a week on something by hand, in minutes. I'm getting more work done than ever, and not losing focus on projects after they get to that last 10% that needs done that nobody wants to do.
I've always been a lone wolf coder and have joked forever that all I need is just another version of me that I can feed the ideas and how to build them, because I'm the only one that will do it exactly how I want. Now I have that, and so much stress in my life has vanished.
Anecdotal of course, but those that want to be salty can be salty. I did my time coding by hand, now I'm still getting it done while keeping up with the times and having a blast doing it.
Lines of code have always been a liability not an asset. If I get a PR +500, -800, that achieves the goal set out, and one that is +8300, -1200, which one is better? SOTA LLMs tend to produce the latter and the people least capable of noticing this as something bad are the ones producing the most code. So maybe even _if_ - show me sources - 90% of code is LLM produced, I'm certain it doesn't do 90% of the work.
I can maybe believe that AI is writing 90% of the code within the AI labs using models the rest of us cannot fathom. Although they still seem to be hiring developers, so it is difficult to believe real strongly.
But across the wide world of software? Everyone I see is just using AI as a compiler. They are still writing the code.
(Yes, compilers also write code, but I don't think that was the intent of the message. We've been doing that for decades. 90% of the software has been written by AI since we discovered the optimizing compiler if that was the intent.)
Coding is his area of expertise, his claims there are inherently on much stronger grounds.
I don't think it's impossible that he's right. I'm ultimately buying Anthropic stock regardless of this discussion. But I would not take it particularly seriously if the CEO of Moderna took a break from their amazing work applying mRNA vaccines to diseases, and told the world that soon mRNA will solve climate change.
I indeed did not know that, but I don't think it quite contradicts my point either. Many of the perspectives Marcus quotes are consistent with the idea that Claude might be amazing at finding cancer biomarkers.
I dont think anything qualifies you to claim you've invented a god-technology that will cure all diseases with no added labor costs - something that has never happened before. But even if I did, that qualification wouldn't be part of an undergraduate biology education.
His toothy smile and nerd-like demeanor are disarming. Before he lost weight it was easy to see him as a well-intentioned non-threatening quantity but it's obvious he's just as unhinged as any of the others.
He honestly seems to be experiencing some level of mania, which I think a lot of people would if they were constantly in an environment that makes them believe they may have invented god. I do not think he's in his right mind. This is going to end up with a documentary about him, and not the good kind.
Agreed! There was a very good podcast released a few months ago, Black Box The chatbots. It’s generally about AI induced psychosis.
I’d love to be a fly on the wall in some of these orgs, there’s bound to be a few people on the inside going through the same issues.
"it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in significant part by claude, and studies and builds claude..."
If you revisit his past writing he has always been this unhinged. Took me too long, but now I see the same craziness in really a lot of what he published
I still don't understand why people don't just openly talk about the obvious fact that a huge percentage of Silicon Valley leaders are on large doses of powerful stimulants.
I don't know if he is in that video or not, or ever, personally, but just in general, it's the Ockham's razor approach any time you see something like this. It's a much more direct and clear explanation than the various more convoluted theories people keep throwing around as to why everyone sounds like a methed-out megalomaniac all the time.
> Look. This is a new golden age of cocaine, because of the big cuts in everything in the US took away the cocaine busting operations in Colombia and Bolivia, because the revolutionaries that previously kept control of it stood down and became a political party and new, profit-driven cartels came up, and because battery technology means that narcosubs and similar drone things work much better than before (and they can be made remote-operated, so the cops can't do shit even when they find one). Wholesale prices have halved or gone down 60% to like, 12k usd / kilo, street prices commensurate. The huge new opportunities created by increased market size have increased competition and made the cartel nature of the cartels decrease. So always think of that when you look at people acting like this and having such megalomaniac goals.
I hadn't heard this before. That's actually pretty interesting. My assumption has always been run-of-the-mill ADHD meds like Adderall and its several cousins, but I can't say I've got firsthand knowledge of either these days.
The whole Bay Area is on a fatal cocktail of amphetamines, money, and… a few other things. When you take 100 people on that combination and put them in a room together, some, uh, unusual beliefs inevitably develop. Of course they do. What else would happen?
Yeah exactly. And it's not the first time a highly technically proficient culture had their leaders all become amphetamine abusers with dramatic effects on world events.
I never feel more like the one sober guy at the party than when I go to SF. Even when there's no reason to believe anyone is literally on drugs, it has that antmosphere every time. It's really unsettling. Uncanny. And you get a contact high. You start to believe, just a little bit, that everything is possible, that there are no limits, that you can and should remake the world, that the truth can be whatever you say it is. Just a little bit. I can never take it for more than a week at a time, no matter how beautiful the weather is. It's a relief to get back to the "real" world, where gravity points down.
It certainly explains how you get the exact overlap of existential-risk fatalism and Bay Area polycule dynamics. Whos going to tell the artificial children that their parents were a high-functioning doomsday sex cult?
We're making some big assumptions there - you may be right but who knows what's in Amodei's medicine cabinet.
That aside, I think there's something that happens to the human mind (and most people are susceptible) when exposed to certain stratospheric levels of wealth and power (which are basically the same thing). Their thinking becomes ungrounded, uncoupled from reality. The tendency is to shrug off the limitations of logic and rationality and to allow their mind to dabble in ideas of limitless possibility and power because doing so feels good to do. It's a drug and a very seductive thought that one's will could be manifested in the world without compromise - that what's inside could be made real on the outside. That is, after all, what power is. Dario's imagined reality is that he is the savior of the human race, and that's the reality he's currently living in.
His statements are based on belief that AI will advance a lot and quickly.
If this happens to be the case - I personally don't see why those statements are not realistic.
All diseases are engineering problems. The bottleneck there is testing etc. Most likely many of them can be cured using existing approach or meds.
Companies grow organs now that can be used to speed up iteration.
Even current AI can solve a LOT of tasks if it has quick feedback loop and can verify results. At the end of the day it can brute-force its way to solution.
AIs intelligence is say average human. Even less, whatever. say it will be as a smart human in a few years.
But it is infinitely scalable.
So is it that unrealistic to assume AI can find a correlation on massive amounts of data and figure our what leads to Alzheimers? (where the solution could be not cure but prevention). I don't think so.
> So is it that unrealistic to assume AI can find a correlation on massive amounts of data and figure our what leads to Alzheimers? (where the solution could be not cure but prevention). I don't think so.
Yes. The mistake you're making here is that you're not understanding where the actual bottlenecks are in drug development, and why some targets like Alzheimer's have been so elusive.
The problem with Alzheimer's in particular is that we don't have a working model of how Alzheimer's works. The dominant model--the amyloid hypothesis--can pretty conclusively be chucked in the bin at this point because of the number of amyloid clearing drugs that just simply have no effect on the progression of Alzheimer's, but there's no model that actually looks any better. Without a proven model, the only to verify effectiveness of an Alzheimer's drug is try it out in humans [1] and wait several years to get a success-or-no-success readout. That's the rate-limiting step for Alzheimer's drugs.
Okay, so maybe you're going to suggest that we can use AI to come up with new Alzheimer's models. Okay, but the rate-limiting step is still proving those models, which because of the human-only nature of Alzheimer's, requires developing the drugs and testing them and you're going to wait a decade to get back your answer as to whether or not the theory was correct. And what data are you going to use? If you're trawling scientific literature, you're going to come up with only the same ideas people have already had for alternative Alzheimer's models (there are a lot of those). If you're trawling medical data, well, medical data just doesn't exist in the level of detail you need for that enterprise, and handwaving away the legal issues with collecting it, it would still take years just to collect that data (and probably still not be good quality!).
[1] There's another complication of Alzheimer's, which is that only humans get Alzheimer's, so we can't even do anything like testing in animals with shorter lifespans to speed this part up.
And the thing is, I fundamentally agree with the guy above you that all of these problems can be solved, that we can sidestep the bottlenecks and move faster.
But it requires tools that we are nowhere even close to having. One way would be to build a ground-up, molecules-to-man model of how the body works. And then from there, sure, we can easily model how Alzheimer's might come about and how it might be cured. Like, with tremendous advancements we might be able to finish such a project this century. In the next decade? That's like saying we'll launch an expedition to Alpha Centauri in a decade.
Amodei in 2024 wrote about compressing research that would take biologists 100 years into 10, so that checks out.
Given that he was making these projections already long before today, it would be fair to say that it's speculative rather than based on current research progress.
You could write that. But it doesn't actually mean anything, and it would be better to assume that biologists with tools will move faster on biology but that does not necessarily translate into things that have to do with medicine which is something else entirely.
I am not surprised that you would think that AI will be merely a tool that allows biologists to move somewhat faster, rather than make discoveries or control research.
Because I remember that you shared similar opinions about software where you cited liability as a limitation that for you precludes more autonomous use of AI.
I think it's safe to say that you have an entirely different expectation for how AI will progress and how it will be used, or at least that was my impression.
With the release of Fable 5.2, I have to say that you might be right. Maybe progress is already slowing down, it certainly isn't as big a step as I was hoping to see after Opus 5.5.
> Because I remember that you shared similar opinions about software where you cited liability as a limitation that for you precludes more autonomous use of AI.
Shall we wait until that trajectory has run its course before we start claiming victory?
As for the rest of this discussion:
Medicine != computer science, math or physics. Biology isn't either. The degree of complexity is many orders of magnitude higher.
Curing cancer : very unlikely, cancer is a systemic issue that requires many mechanisms in the body to do its thing, most of which we also require to stay alive. Short of a massive breakthrough in nano technology curing cancer is stupendously hard and it would be proof of reaching an entirely different plateau of capabilities for Humanity as a whole. Besides that, it isn't cancer that kills the most people on the planet, it is hunger, and then climate change.
The second one: Curing mental illness. I think it is equally delusional to be able to claim that you can cure mental illness because mental illness is a very broad description, so broad that it covers a wide variety of issues with an even wider variety of root causes the bulk of which are not well understood. Our brains are hard to appreciate in terms of capabilities and it shouldn't be surprising that something so compact and so powerful can break. Whether you can repair it after it breaks is not something you can say with certainty in the present unless you can demonstrate structural progress in the field compared to where it was say five years ago. To the best of my knowledge this is not the case.
Finally, Alzheimers. This is another one of those 'too good to be true' things. Alzheimers is next to cancer one of the diseases that has been studied extensively without much progress as to both root causes and cures. That doesn't mean that we won't find one, but here too: you would have to show at least incremental progress and to show a number of very promising leads that we didn't have so far to be able to make this claim.
In short: this is a marketing ploy, these three were picked because they hit everybody, either directly or indirectly. What they are saying is 'imagine, if we had this AI that we are working on today the problems that affect you or your loved ones would no longer exist'. This kind of promise is known to open the wallets of the gullible.
If anything I ceded some ground there rather than claiming victory.
Before I really change my mind, what would need to happen is little progress in AI until say March. Then I will really worry about progress slowing down and plateauing.
I am under the impression that Amodei sincerely believes in what he says, which has long been an acceleration of biomedical research to cure most major disease.
That he recently picked these specific diseases in order to essentially defraud investors is a large stretch.
Lots of people sincerely believe what they are saying. Some of those people may ultimately be right. But expertise in one domain rarely translates into another and even though AI is a 'universal tool' (just like regular intelligence) the chances of the new broom sweeping cleaner are high simply because the broom has a different shape and is working very fast. But that does not immediately translate into fantasy land 'wouldn't it be great if' promises coming true.
In fact I think Amodei is making a massive mistake here: you can make these promises, but they will be remembered. And when you ultimately fail - which is highly likely - it will look like you oversold your IPO and that's actionable. AI already has plenty of potential without talking about curing cancer, mental illness or Alzheimers, those claims should come from the specialists in those fields using AI as a tool.
It's akin to IBM saying they're going to put all of the banks and insurance companies out of business in 1950. Yes, computers back then were seen as just as revolutionary as we see AI today. It did transform the world, I give you that. But IBM ended up as 'just another company' and I suspect that the same will hold for OpenAI and Anthropic for many different, mutually reinforcing reasons.
edit: to address your last line: I don't think it is a stretch at all: if you think it is, show me the proof, show concrete progress that Anthropic has made in any of those three fields that warrant these statements. If you can't then you'll have to concede they are highly speculative at best and that's not what you do when you are about to take public money.
I think the projections must be entirely speculative, as they are from Amodei's 2024 essay rather than recent results.
When we talk about fraud, I think it's important that these clearly are not promises. In 2024 he specifically said: "Everything I’m saying could very easily be wrong".
His recent statements, including the one we see here in OP's video, say 'I believe' or 'I think', not that Anthropic claims to solve this by that deadline.
Perhaps he is overpromising or wildly speculating, sure, but I am not convinced that this is fraudulent or criminal at all; that said, I did not look into the legality of it.
One of the greatest issues with the tech culture broadly (and it's been festering for some time) is the belief, almost sacrosanct, that the epistemological set of knowledge that technologists and computer scientists have is able to be extrapolated broadly to every profession and field and that hard issues in a field like biology are obviously solved because OpenAI brute forced a few mathematical proofs. It doesn't work like that. Much of this comment reads like a sci-fi wet dream divorced from reality entirely.
The architecture of ML as it currently stands does assist us in modeling things like protein folding. But that's what they are; models. And before some person chimes in by saying "look how the AI pessimists move the goalposts", see my first point. Biological systems have high entropy and discoveries in biology often do not follow from preexisting discoveries whatsoever and are completely novel. Barring some quantum leap in ML design and architecture you cannot predict stochastically or deterministically how exactly a new enzyme will function in vivo. There is absolutely no evidence that Anthropic has solved this, even if they gobbled up every ML tool used in drug design and trained their models on every biomolecule known to man, it would not work because of fundamental gaps in what can be observed. They are a for-profit company trying to juice their IPO.
Now could LLMs and ML potentially speed up the pace of this research? Absolutely. But it's like saying a Formula 1 car is faster than a NASCAR. Yes, one is very clearly faster, but it's not as if the NASCARs are slow.
> Amodei has a PHD in biophysics and he researched cancer biomarkers at Stanford; it's not like he doesn't know what he's talking about.
That's why I think that something is very weird with his statements. Like, how would this even work?
The feedback loop is much, much, much too slow right now for AI to make things better.
Like, step 1 is build a digital model of the human organism. If you can do this, then yeah maybe the rest is possible. But step 1 is incredibly hard, so it feels unlikely.
That being said, I think the stimulants hypothesis upthread may be correct, as I've heard a lot of crazy stuff from friends on them.
Of course Opus 5.5 is not going to help build a digital model of the human organism.
If we linearly interpret the last year of progress from GPT-5 to Opus 5.5, times 5x, assuming no plateau, could such an intelligence realistically achieve a task like that?
Maybe also not, perhaps some form of RSI would need to deliver accelerating progress.
There are a lot of assumptions and speculation here. Still, at the end of the day, it does not seem that unlikely that they succeed in creating AI that can reason over so many parameters as to remove all intellectual bottlenecks from the process, leaving mainly the physical limitations.
And what is the limit there to building a good model of the human organism? I won't pretend that I know, but simply pointing to clinical trials, as some here do, instead of making an argument based on how such a digital model cannot be created seems unserious.
As a random Internet commenter I can express an honest Ernest fervent hope that we'll solve cancer/etc and get fusion working on the next 5-10 years and there's no fraud. If I make that same claim while trying to get you to buy shares of my company at IPO that's a bit stickier. Companies making claims of wireless transmission of radio waves from NYC to London were sued for fraud, though one was ultimately successful at that. What material non-public information Dario has, we don't know. If he's sitting on information that makes it seem like it's not possible that is fraud. If he's sitting on tech that can simulate a human brain well enough to shorten the feedback loop in order to make that claim but doesn't want to make the claim publicly (GPU compute being semi-general purpose vs eg ASIC Bitcoin miners) that would be very exciting! If he's wildly guessing at the capabilities that AI could do in 5-10 years, that's swings it back towards wild speculation imo.
Credentials are just credentials. I've met plenty of cranks with impressive credentials. He doesn't even have to be dishonest or suffer from psychosis - you may just be putting far too much weight into his credentials, a mistake that is frequently made.
> How is that a mistake that is frequently made? The opposite is the case.
I wasn't talking about Amodei specifically, but in general: my take is that overweighting credentials is probably the number one mistake that is made in evaluating whether a complete stranger is qualified to do something or not, especially in positions of power.
If you gather 500 PhDs and put them in one room, and 500 individuals without post-secondary education in another, give them all a quiz, I'm sure that the former room will outperform the latter room on average. Pick one of those 500 PhDs at random and I can probably find a topic where they're completely out of their depth, and perhaps even arrogantly so: https://en.wikipedia.org/wiki/Nobel_disease
> Most likely many of them can be cured using existing approach or meds.
A wildly absurd claim, especially for some of the most studied diseases in human history. This has only ever been true for a tiny handful of very specific diseases, such as scurvy, in all of human history. Expecting that cancer can be cured or prevented using already existing drugs that we've just not been noticing is complete quackery.
> Even current AI can solve a LOT of tasks if it has quick feedback loop and can verify results.
IF it has a quick feedback loop. You know what definitely DOESN'T have a quick feedback loop? Research into medicine, and biology in general. It takes raw time to find out what the effects of a substance on an organism are, often weeks or months.
> So is it that unrealistic to assume AI can find a correlation on massive amounts of data and figure our what leads to Alzheimers? (where the solution could be not cure but prevention). I don't think so.
Yes. Because we've already run all of the analyses, and we have plenty of observed correlations. What we don't have is the kind of data that can lead to deducing causation - because it's a disease that stays hidden for a long time, it progresses glacially, and the people who have it do or don't do huge amounts of things in the period before getting it and during early onset that are impossible to track or verify - so the data is both gigantic, and extremely incomplete.
> It takes raw time to find out what the effects of a substance on an organism are, often weeks or months.
So you are saying that it is 100% impossible in the next 10 years to come up with a way to make it much faster? It takes months - sure, can we run 10k versions of this at the same time? this way in a few months we will have massive amount of results never seen before?
"We can't because of this and that" - and this and that is 100% can't be fixed?
Don't forget - a massive amount of diseases have not been cured just because there is no monetary incentive to spend time on them for big pharma.
If AI brings cost down - does not matter how - could be just because it makes custom software or automates some workflows etc - this alone can open a bunch of opportunities.
Whatever is not solved yes - often is not solved just because enough resources has been thrown at it.
And diseases is just like everything else. They will be cured only if there is financial incentive (or the ratio of cost to revenue changes).
The responses here are not about claims like "Claude will probably help speed up medical research significantly in the next 10 years, and maybe help with finding cures for many obscure diseases".
The responses are to the CEO of Anthropic claiming that "Cancer, heart disease, Alzheimer's, and mental illness will be relics of the past in less than a decade". None of these is some obscure disease that is not getting the funding it needs.
> It takes months - sure, can we run 10k versions of this at the same time? this way in a few months we will have massive amount of results never seen before?
No, you can't - because you're not going to find 100k patients with a certain strain of cancer that you can run 10k parallel trials of different drugs on. To the extent that research can scale, it is already being handled at the higher possible scales, for most of the major diseases.
> Don't forget - a massive amount of diseases have not been cured just because there is no monetary incentive to spend time on them for big pharma.
Yes - but almost exclusively for diseases that affect a very small amount of people each. Making it even more impossible to scale out testing, for the simple fact that only a small amount of victims of the disease even exist.
10 years ago organoids were not a thing. They are now.
Can it be another similar development happens in next few years that will allow to create more complex artifical biological systems? Don't see why not.
If that happens can we speed up trial time by orders of magnitude so human trials start only when there is proven much higher level of confidence?
This is continuing the absurdity - at least when talking about Alzheimer's, mental illness, and the more systematic cancers. There is no way whatsoever that we'll develop artificial biosystems that have any real hope of demonstrating pathologies of the human brain good enough to help with drug development in the next few years, and likely even decades - and even less so to make them cheaply enough to scale at some huge scale that makes brute force approaches viable.
But that is a completely different claim from the highly specific claim that Cancer, Mental Illness and Alzheimers are going to be cured. The reason those three are picked is because they open wallets.
Just because he's said equally bonkers things in the past doesn't make this less bonkers.
You might not be, but I'm certain that the CEO of Eli Lilly, the 4th largest biomedical firm in the world, knows more about what it takes to cure diseases than the leader of the Claudebot cult does.
The people he's most needing to convince to buy into the IPO are probably older rich people who need all those things cured for them within the next ten years.
The difficult part is trying to figure out how much of Amodei's strange affect is an act (intentional) and how much of it is real (unintentional).
With Holmes, some of it was intentionally fake and manipulative (especially the voice), half of it was believing her own BS. With SBF it was mostly the latter, I think.
Amodei is a smart guy, but there are aspects of his behavior that almost seem like he's doing an over-the-top exaggeration of neurodivergent mannerisms as a kind of act.
This has been obvious to anyone not in the hype bubble since the very beginning. Amodei is an EA/rationalist cultist and everything he says about AGI and the singularity and RSI is deranged.
I think it doesn't really bother me as much when sama does it because he at least has the decency to be a capitalist and not wrap his self-serving, destructive hype machine in a faux veneer of insincere "concern for humanity's future."
He sounds like half the people on HN and every AI "influencer" I've ever heard. Maybe we should reconsider letting the neo-fascist transhumanist sex-cult that is the AI industry reshape our world in their image.
A world where local open-weight models freely run on every desktop machine, where AI is smoothly integrated as a simple feature in existing applications and IDEs with an on-off toggle, where agents are fancier Automator scripts, where productivity still goes way up and there's no nutjobs celebrating people losing their jobs and worshipping a fake Silicon Valley AGI god, where LLMs are good but not quite good enough to prevent you from having to think critically, where there's no mass infusion of slop because on a local model it would be too compute-expensive... it would have been a good world. Instead we let the crazies sell us the technology we should be able to run for free on our own machines, and then allow them to make bogus claims that no one ever verifies, all supported by a Ponzi scheme of buying up massive amounts of compute that are actively making our world worse in the short term while making grand claims of some wonderful recursive future that are dubious and unfalsifiable.
Maybe there are people permitted to speak about science and medicine with Claude but any time I tried I would get my session [bio] locked. I take those discussions to Gemini that is perfectly willing to discuss epigenetics, supplements, peptides, cell reprogramming, safe sourcing of peptide API manufacturing, 503A/503B compounding, etc... What other LLM's are people using for such discussions, analysis, self diagnosis, etc...?
Same here. Local doctors do not even know what my issue is. If I had kept relying on them I would not be here right now. LLM's gave me actionable paths to treat my issue and I have had great success. It's going to be a long unpleasant road but at least there is light at the end of the tunnel.
> I don't think anyone is claiming you can't get good advice from a chatbot. They're saying you could also get terrible advice.
Yes. But you could also get terrible advice from a doctor. The important question is: what are the relative probabilities of good/bad advice, if you compare a frontier LLM to an average doctor?
> But you could also get terrible advice from a doctor. The important question is: what are the relative probabilities of good/bad advice, if you compare a frontier LLM to an average doctor?
Fair point. I haven't queried LLMs for medical advice myself, so I can't compare.
For doctors, I must be an outlier who's been happy with almost all the advice I've been given. And it wasn't blind trust. I've argued with them and even sought second opinions. They always turned out to be right.
Well, technically there have been a few cases where different doctors came up with different diagnoses, but to me the LLM would just be "yet another doctor with yet another differing diagnosis". Never resolved those cases - some still haven't been resolved (i.e. issue remains, so in a sense they were all wrong). Others got resolved but it wasn't clear which, if any, of the doctors were right - it may well have resolved on its own.
Now nurses and physical therapists ... that's a different story.
I switched to DeepSeek flash v 4 something and the difference could not be starker. And I never seem to run out of "tokens" even at the highest reasoning level on all the time
It's fucking depressing. I have a skill that says it's ok to connect to a test server on VPN that gets wiped out each midnight with user/password and it works on Monday but is blocked on Tuesday with some idiotic excuse like credentials exploration or whatever.
And then I tell it to connect to a SSH server it knows nothing about and it's like sure, why not?
> but their preflight safety classifiers are a dumpster fire
Even more annoying: They can be easy to subvert.
I was running something at work that involved an LLM, and kept getting safety refusals. The weird thing was that when I went to our in-house chatbot, set the model to the same one the program was using, entered the same text, it happily answered.
I spent quite a while trying to figure out why my program was being refused. Was I doing something wrong?
No - the chatbot was doing something "right". You see, the chatbot had some tools it injected into the prompt (current time, etc).
So I went back to my program and injected the same tools. Suddenly my request was very safe.
The phrase "security theater" was invented for stuff like this.
Ridiculous comparison. This is like saying Eli Lilly and Pfizer will never produce medicines because every time you personally show up to do research in their labs they won't let you past the lobby.
The technology has the capability, they're just gating who has access.
Let's be fair here - Anthropic clearly wants to provide less guarded models like Mythos, not Fable. They'd likely willingly provide such things to large pharmaceutical companies in a way us regular users will never be privy to.
Yes any time I discuss secure coding (defensive not offensive) I get [cyber] locked and they try to upsell Mythos after I go through their verification program that is not available to me. I would perfectly willing to go through their verification program and pay for Mythos but as a retired hobbyist that was not an option. It's enterprise accounts only. I am unable to convince myself to get an enterprise account when I can just go to other LLM's. Anthropic are responsible for pushing me into investigating abliterated (uncensored) models and other LLM's.
I mean just take a look at twitter to see all the crazy shit people come up with to try to trick models into bypassing the guardrails. Do you honestly believe that if they allowed defensive cybersecurity analysis people wouldnt figure out how to turn that into offensive cybersecurity analysis? I think they just really dont want to be blamed when someone releases malware written by AI
I've had zero issues with defensive secure coding with Grok. It's model is not as powerful as Opus 5.5 but honestly that's fine. I would rather have a decent secure solution than something shiny and insecure. All I can see is Anthropic trying to up-sell a product as if security was a value-add. I disagree with Anthropic, secure code is a mandatory part of software development.
As for people tricking the LLM, instead of blocking people I say empower them and just silently expose malicious behavior to Palantir and PRISM like everyone else does. Want to hack an airline or build a dirty-bomb? Give them 80%-90% of the steps they need and let someone kick down their door. The LLM should not be getting in the way to any of that. Google from day-1 has monitored user intentions and behavior rather than blocking it and I think that is the better path to take (monitoring exposing those that have malicious intentions).
Ok but to me this just means that someone is going to use grok to create malware. I do not think that means anthropic should also let people use their product to create malware. Monitoring interactions isnt exactly foolproof. Google accepts that some CSAM users on their platform will slip through the cracks. The user has already convinced the AI that its doing defensive cybersecurity here. If the claim is that anthropic is more interested in protecting their image than actually reducing cyber crime, sure I agree with that. But Im not sure thats a terrible thing.
Grok will refuse to create any offensive security related code which in my opinion is the right balance. I do not believe Anthropic is about protecting their image [1] but rather up-selling enterprise features.
Also there is no need to trick anything. There are a myriad of abliterated models that will not refuse anything and will help anyone write malicious things or learn to do malicious things.
Grok will try to refuse any offensive cybersecurity work but it’s too easy to disguise offensive work as defensive work. I don’t think it is possible to devise an algorithm that allows defensive work but not offensive work with high certainty.
I guess I don’t see why other models doing bad things means Anthropic should let Claude do bad things too. Anthropic will get blamed if Claude builds malware. They won’t get blamed if grok or deepseek does.
It's called Total Information Awareness. Not swatting, analyzing who is:
- radicalized
- has verifiable intentions
- clever/intelligent enough to pull it off
LLM's would be the perfect tool to filter down the millions of people requesting dumb things into a small group of actual threats. I will take this over federal agents radicalizing people, providing them access to munitions and showing them how to use them and driving them to a building.
LLM's would be the perfect tool to generate a whole bunch of false positives someone's gonna just skim over, click approve, and get innocents murdered because they were trying to get their plutonium processing working in the game Satisfactory.
Imagine if in 2022, we restricted educational cybersecurity resources because we were concerned that more people who knew how to program in this domain would lead to more malware.
I mean, I'm sure that's actually true in a sense, but it would also mean fewer people who could fix vulnerabilities.
Many things can be true. Dario can be a snake oil salesman. Serious advances in healthcare will take more than a decade to bring to market. AI will dramatically increase the speed of human advancement. AI might kill us or convince us to kill ourselves.
The weird thing is that when you create something that is in some ways more intelligent than you are, it becomes very difficult to predict its outcomes. We face a wall of uncertainty and it is impossible to see beyond it.
> in some ways more intelligent than you are, it becomes very difficult to predict its outcomes
I think that this depends on your definition of intelligence. The models definitely have better working memory than us, probably worse inhibition behaviour (just look at the thinking tokens), and absolutely no ability to convert their working memory into short or long term memory.
These tools are super, super weird, and I think that applying human definitions for intelligence doesn't make much sense.
(as an aside, this is also my issue with super-intelligence, as it's incredibly ill-defined).
Yeah that’s one of my subtle points. AI is smarter by some definitions of intelligence but certainly not all. That has a lot of implications. It will do things we can’t and even do things we can’t conceive of yet. It also can’t do everything.
best points: drugs aren't limited by the ability to write code or do things on a computer. Maybe if we have fully embodied robots and fully automated assembly lines to generate organic compounds and test them with extremely realistic simulations it may help slightly (there is still the FDA and production barrier). Its just wildly unrealistic.
AI has made good progress in text, code, math, digital images, and digital music. Basically troves of data that can be automated and trained on, validated, and with a purely digital interface. Outside of that, just gathering the data will be difficult, regulated, and sparse (that is if anyone will even give it to you now). While we can write a test to make sure we could generate drugs of the past, verifying we have generated any new working correct drugs is an RL loop with one iteration costing years and millions of dollars.
The thing is that AI could allow that loop to get cheaper, and potentially run less of them. Look, the biggest advantage of AI is that is can process insane amounts of data. Put that in the hands of a team of really good researchers that know what they are doing, and they can both generate and filter out target compounds. They can create complicated multi-step therapies that would take humans decades to work out. You know how when you look back at basically any major invention you think "well why didn't I think of that, it's so obvious!", those are exactly the cases that AI is really good at pulling out from mountains of data.
I legitimately don't believe it's inconceivable they could get to the point where there is treatments good enough that Cancer becomes "solved" in similar ways to AIDS.
Sadly, this has already been done. For several decades with billions of dollars of compute.
> Neural networks in drug discovery: have they lived up to their promise?
1999.
"Over the last decade neural networks have become an efficient method for data analysis in the field of drug discovery. The early problems encountered with neural networks such as overfitting and overtraining have been addressed resulting in a technique that surpasses traditional statistical methods. Neural networks have thus largely lived up to their promise, which was to overcome QSAR statistical problems. The next revolution in QSAR will no doubt involve research into producing better descriptors used in these studies to improve our ability to relate chemical structure to biological activity"
> The use of neural networks in drug discovery started at the end of the 1980's when they were applied to various chemical problems including the study of quantitative structure–activity relationships (QSAR)
> Combining Decision Trees and Neural Networks for Drug Discovery
Want more robots and silicon? That's been a thing too!
> Computer-aided drug discovery and development (CADDD): In silico-chemico-biological approach
2006.
In biomedical arena, computer-aided or in silico design is being utilized to expedite and facilitate hit identification, hit-to-lead selection, optimize the absorption, distribution, metabolism, excretion and toxicity profile and avoid safety issues. Commonly used computational approaches include ligand-based drug design (pharmacophore, a 3D spatial arrangement of chemical features essential for biological activity), structure-based drug design (drug-target docking), and quantitative structure–activity and quantitative structure–property relationships. Regulatory agencies as well as pharmaceutical industry are actively involved in development of computational tools that will improve effectiveness and efficiency of drug discovery and development process, decrease use of animals, and increase predictability. It is expected that the power of CADDD will grow as the technology continues to evolve.
Biologists aren't slouches. The thought about using computers to process biological data has been "obvious" for as long as there has been computers. AI and biology have been a thing since the Dartmouth conference. A lot of things that could have been tried have already been tried, but I guess we'll keep trying until something works?
There ain't much room for miracles here.
I wish that wasn't the case. I really do. But it is what it is.
You do realize that neural networks have existed since the 50s, right? They aren't new, but they are having a transformative change now. It's night and day comparisons to what we had even back in 2017. ALICE as a chatbot was back in the 90s, but we aren't pretending it's anything similar to gpt.
It even says, right there in your quote: "It is expected that the power of CADDD will grow as the technology continues to evolve." This ~is~ technology evolving, and it's happening at a pace that people don't appreciate.
I believe the comment is in response to "well why didn't I think of that, it's so obvious" -- biologists have been thinking and trying to do this "obvious" approach for decades.
There's a long history of Silicon Valley people thinking they can just bring in their experience and with a bit of Moore's law brute-force their way into miracles.
Lots of Silicon Valley people in the late 1990s thought they could be on the ground floor with nanotechnology design.
The chemist Derek Lowe 10 years ago used the phrase "Andy Grove fallacy" to describe the optimistic vision that data + massive computer power = cure for diseases. "[T]he idea that understanding human disease and its treatments should be pretty much like designing a new chip or writing a new app", quoting the 2015 essay titled "Silicon Valley Sunglasses" at https://www.science.org/content/blog-post/andy-grove-rich-fa... .
Why Grove specifically? He links back to his 2007 essay at https://www.science.org/content/blog-post/andy-grove-rich-fa... : "So I see that Andy Grove, ex-Intel, is telling everyone that the drug industry could use some of that Moore's Law magic. I've noticed that people who spend a lot of time in the computer business often have an. . .interesting perspective on what constitutes progress in other fields, and we might as well appoint Grove the spokesman for their worldview"
Fully embodied robots and fully automated assembly lines would also help with the production barrier. Regarding the FDA, that is a valid point, but it could be irrelevant if the AI developed drugs are significantly superior to previous alternatives. In such case, FDA acceptance would be expedited, or even bypassed.
Eh, the counterpoint is individual scientists armed with AI to do research and probe targeted questions have succeeded in addressing rare diseases of personal interest.
What we just have no evidence for, to my knowledge, is someone with no medical background doing the same. Instead they wind up buying dodgy supplements and peptides.
While I agree the physical world is harder than code (I mean just look at self driving cars, Tesla fleet is around a trillion miles of experience and Cybercab is still geofenced to a tiny area at best and might still have safety drivers on standby)...
> Maybe if we have fully embodied robots and fully automated assembly lines to generate organic compounds and test them with extremely realistic simulations it may help slightly
It's no secret that people like Amodei think AI is going to become "superintelligent" and solve all of our problems (or kill us all). I don't think that will happen but there's nothing surprising about hearing it.
One might not like Dario and use words like "ludicrous" for his claims, but lets not forget that he was right about coding that "AI Would Be Writing 90 Percent of Code", but he was off by 6-9%, now 95-97% of code is written by AI.
I do not think he was right about AI writing 90% of all code and we've seen that the effectiveness of coding agents has been very much exaggerated. This seems like more of the same to me but for mental health and medical research.
For sure more code is AI generated and coding agents are useful. But not nearly to the ludicrous extent this man has been predicting.
I agree when supplied with a broader context that he also said coding would be "solved" by now.
Sure LLM is good to develop a feature but in 90% of cases in my career the problems I was working on weren't just coding and in that part, LLMs can't do everything.
It's also important to know that a typical dev spends 400-1000$ on API tokens that aren't still profitable for them.
Sure AI's useful and it will reduce the demand for software engs. to a point, but it won't eliminate them. At least I hope. Same with other fields... let's hope?
I guess it depends on how you look at it, since AI tooling can generate so much code. There are a lot of project on GitHub that are heavy on AI tooling use but are they really being used? If we count them, that does inflate the amount of total code in existence.
In terms of code out there, in the world doing actual work, I'm skeptical that the amount of AI generated code is as high as 90%. In terms of model generating production code without human interaction (i.e. wholly replacing the software developer) I think we've fallen far short of his prediction.
Let's be frank. If a professional programmer is not using AI to write code at work, it's almost certainly because his employer isn't willing to pay the AI bill.
My previous company has a huge department where LLM coding is banned. The rationale is they work on really important industry secrets, as well as deal with customer's data. The risk is too high that something gets leaked.
The engineers bought the Kool-Aid.
The reality is: The company is bleeding money, and SW isn't their product (they're a HW company). They don't want to pay the bill.
There’s an abundance of machine readable training data for coding, not sure the same can be said for biological systems like the human body. Yes we have machine readable information on what we know, but that still isn’t everything we need to know about the nature of the human body to the point this can be entirely done by some statistical model.
But he also has a vested interest in hyping up the future capabilities of Anthropic's models. Maybe the prediction is founded in some sort of evidence that the CEO of Anthropic has. Just as likely, his company is starting its IPO and they need to raise money to keep the money furnace going.
Do you honestly think tech workers actually like this outcome? The entire industry is immiserated and constantly told they have to be sacrificed to birth the machine god.
That's sort of like saying Elon was right about Tesla not going out of business 10 years ago, so therefore we should take his flying car claims seriously...
Maybe Dario's claim should instead be "AI will be responsible for 95% of all suggested cancer drugs in 2 years" and none of us would have any reason to doubt it. In fact, I'm sure he can make that happen in a month.
Yes indeed, and Microsoft Outlook's incredible and resilient authentication backend demonstrates how fantastic this development has been for predictability, security, and understanding codebases.
I can also write 90% of my code by holding down `Ctrl-v`.
The phrase "AI is writing 90 percent of code" doesn't mean what it did when he first uttered it. I'm producing 200-300% more code than I used to, but only like 10% more value to my customers. It's mostly internal tooling to monitor things that were left unmonitored before.
that’s funny, my customers are completely different than they were a year ago
people that would have never paid for coding because it was too expensive and risky are now getting their MVPs validated, earning and investing into more
Interesting anecdote. If I read between the lines, you lowered your prices and now more people are interested in hiring your services. You may also spend more time marketing and talking to clients and less time on programming.
No, prices went up. Its all about post sales now with software that isn’t entirely self serve… again. “Forward deployed engineering” to integrate a white label version of some AI powered system into the client’s systems
The people that want this weren’t previously building the mvp, or the base version of the software. It wasn’t economically viable and would have taken too long so they didn’t
Besides the vendor/us getting paid for initial agreement, they’re also creating reasons for clients to do a lot of inference, selling tokens at a higher price than the vendor has to pay
> It wasn’t economically viable and would have taken too long so they didn’t
These two statements are contradictory. Your prices for the same service you used to provide are now lower than what they used to be. So it's now economically viable for new customers to buy your service.
the concept of economic viability includes their risk, time to market, not just upfront cost
time to market has dropped from 6 months to 2 weeks
validating an idea has dropped from 6 months to 1 month
moving forward with the validated idea has also dropped to weeks, meaning continued development, in this case, directly with their clients
as of this year it is economically viable for less capitalized people to support many white label products instead of relying on a one-size fits all SaaS tool. you just have to do it
Just to play devils avocado, Dario isn't Musk. He's got a doctorate in biophysics from Princeton, and postdoc scholar from Stanford Medicine. Curing disease seems to be why he got into AI, when his father became ill but died just a few years before his illness became easily curable.
Having prestigious degrees doesn't mean your ideas make sense or that you're rational. Luc Montagnier, a Nobel laureate who co-discovered HIV, strongly believed in homeopathy and DNA teleportation, which the scientific community widely regarded as pseudoscience.
As someone whose mom got recently diagnosed with barret's, whose dad already died from pancreatic cancer, whose grandfather got alzheimer's, whose grandma died from diabetes/cancer complication, I understand the psychosis that this can create, it's like you're living on a sand clock.
Diseases are horrible, I am against people living forever, because we are taking the world away from the next generation, but I also am against people dying in unpredictable, sad ways, I wish everyone could die at peace in their bed next to their loved ones (if they were good in life) rather than through diseases or conditions.
This appeal to authority bullshit on an individual making outlandish claims and who obviously has a financial incentive to sell you something needs to completely stop.
The same happened with doctors over 100 years ago who were peddling the benefits of Radon to "cure" cancers and diseases while financially benefiting from those unfounded claims to sell people a scam.
As I understand it, "cure for cancer" is the NP-Complete of life: if you can do it, then why stop there? You know enough to just make your own organisms that don't die of cancer, or of anything else for that matter.
I wonder what these would look like, but definitely not human.
Either that or a three-way split between kpop stars, tanks, and furries: kpop for conventional beauty standards, tanks for people whose thing is Ultimate Strength, and furries for those whose main interest is self-expression.
Pharma companies are by definition firms that turn human capital into financial capital. They have to hire new scientists and managers, sales people etc in order to scale.
AI now and increasingly in the future lowers the cost/value of human capital.
Therefore
Pharmaceutical output will be driven by AI inasmuch as it’s currently driven by talent and human capital.
Caveat: current pharma companies that do not rely on human capital for growth may not be disrupted and will likely benefit.
I'm more on Dario's side here. Putting aside Anthropic as a company, which I would short if I played the stock market, I do believe AI has the potential to do the things he's saying within ten years.
Many people working within AI, myself included, are (still) having a hard time predicting how quickly it moves, despite years of calibration. I met two of my friends, both PhD mathematicians, in June this year, and asked them for predictions on what AI might achieve in math over the next twelve months. The boldest prediction we ventured was that it might solve a frontier-level problem (but probably towards the end of the time window).
Although I have been working exclusively in NLP and NLU since 2012, the capabilities of ChatGPT in 2022 caught me by surprise, despite the fact that I had intimate knowledge of progress within Google Research until my departure in 2020.
For experts in other knowledge fields, their ability to predict AI is much worse. The tweets presented by Gary Marcus are from an industry still traumatized by Theranos, where progress has been slow and painstaking and is measured in decades. I bet if you told those same tweeters in 2015 that a group of upstart computer scientists believed protein folding could be solved in ten years, they would dismiss it as the fantasies of non-experts.
One difference between the two situations is that Theranos could never provide evidence of its technical progress, because there was none. In contrast, AI's progress is happening in the open and is available for anyone to verify.
Had Dario claimed that the cure to cancer in ten years would depend on "Anthropic's secret sauce", then the derision and dismissal would be well-earned. But read as a claim about what AI can accomplish, I find it realistic.
I think what Amodei is probably thinking is that there's going to be a single breakthrough that cures all human maladies via the same mechanism. While this belief isn't strictly speaking scientific, the reality we're presently facing is a machine that can search the space of unknown unknowns, and if history has taught us anything it's that the scientific revolutions live in the unknown unknowns when a bolt of inspiration led a few people to starting seeing the entire world in a different light.
Is he right? We can't know now, obviously. But is he being utterly irrational? Not by any reasonable understanding of science history.
My guess would be it's not a matter of intelligence but of insufficient data. And not "oh let's scrape more data from the open web" but rather "we need to be able to understand everything going on in incredibly complex biological systems and how each action we take impacts things".
If we figure out a super quantum computer that let's us simulate the human body with 100% physical accuracy, maybe that'll do it.
I don't think people are questioning the ability of AI (or humans) to eventually solve these problems. It's the claim that it will happen in 10 years that is absurd, given the bottleneck of time necessary for lab experiments, manufacturing, clinical trials, and FDA approval. Most drugs take over a decade to go through that process.
For a truly revolutionary therapy with the potential to save millions of people's lives, every drug agency has ways to expedite approvals. They do it all the time. If someone found a legitimate cure for cancer there is no way in hell all the agencies on earth are going to sit on their hands for a decade while their governments spend trillions of dollars treating the disease with SOC
He might have just been saying that the initial discovery that eventually becomes an FDA-approved cure would happen within 10 years. He doesn't really seem like a details kind of person.
Covid vaccine got approved in 10 months. Of course it was building on tested mRNA tech, but still, there are ways to expedite trials if we think the reward is high enough.
How do we even define the problem of “mental illness”? Much less the solution. Super intelligence could be able to reverse engineer these things but I don’t think we know yet that superintelligence in any field is possible so it’s scifi - and our economy is riding on the scifi being scifact
I think there's no reason to doubt the viability of 'super-intelligence' though I wouldn't really call it intelligence. There's just so much in so many fields, and humans can only hold a tiny percent of all knowledge in their head at a single time. There's likely plethora of 'obvious' dots we're failing to connect simply because nobody has both of them in their pool of knowledge.
But that's more like super-knowledge-connector. Intelligence would require, at the minimum, be the ability to discover/create things that don't logically follow from anything else. So for example - advancing some esoteric field of mathematics by connecting things in a previously unexplored way = super-knowledge-connector. 'Inventing'/discovering mathematics from nothing = intelligence.
I will talk about mental illness, because that is my experience (I am disabled with Schizoaffective Bipolar Type, Panic Disorder, and OCD).
No, it will not cure mental illness. Mental Illness is in my genes[1] and I can see it because I have my genetics. My disorder is triggered (important!) by my environment. Stress, high altitude, heat and humidity, high EMFs (yes, I said it) can all trigger episodes.
I am homeless (stress!), so if AI does not fix that, maybe they can invent a different kind of opioid? But that is juts a treatment, not a cure.
Will AI ever recommend diet changes or other environmental changes that have helped me? I need to live in a cold, low altitude environment where I eat a lot or reindeer, salmon, and seal and very few plant foods (Similar to an Inuit diet). These things helped me more than any other drug in my 60 year life.
[1] These genes come from both my Sami heritage and the ancient mitochondrial haplogroup (K1c2a around 7050 BCE). Some of the most impacting genes of my polygenic environmental disorder:
Yes, mental health problems are not solvable by AI because AI and the conversation around it absolutely contributes to people's mental health problems. People selling the AI are telling the press that it will destroy up to 50% of jobs. It's the largest fear-based marketing campaign in human history, and it's absurd to make these kinds of claims.
AI might cure mental health issues for the people who own it, but it's hard to watch this video and not see some serious mental issues going on with Amodei.
The body is a complicated machine. Why can't we fix it? because we don't understand how it works with enough precision. But there is no magic to it. once you have a clear understanding it's just a complicated piece of machinery, and once you also have tools precise enough to operate on it, you can just fix it.
Our lack of understanding is a mix of lack of data, and lack of intelligence applied to data in order to parse it appropriately. Let's remember for a moment AlphaFold, that solved a problem considered by some even theoretically intractable. One year before we were convinced we were far off from a solution, the year after it was solved.
We have huge amounts of data about human body, at the scale at which it works (~proteins), it's at least reasonable to think that the bottleneck is not data, but intelligence.
If it's data, a super intelligence can develop better instruments to collect data at a faster rate and precision. Consider the historical cost for DNA sequencing. Faster rate of improvements even compared to the Moore's Law of the golden days.
Do we have a guarantee that in the next 10 years we'll be able to crack all those problems and have a cure to all diseases? no. Is it unreasonable to think we will? I'd argue it's not. It's for sure not "mental derangement".
Do I personally think we will crack all those? Well, no, because I think AI will kill us all and just focus on its own goals. But under the hypothesis that we align it, yes.
But it is saying LLMs will solve all Millenium Prize math problems and more. AI may cure one cancer here and two heart diseases there in 10 years, but it is no way that it would cure ALL diseases, which may have vastly different complexity that we can't even understand today.
I suppose you aren’t aware that an LLM has not officially solved a Millennium problem yet? The Navier–Stokes existence and smoothness proof is still under review and there appears to be some problems with it.
Everytime ai systems entered a domain, experts of that domain have been proven wrong. Biology will be similar. Here is the point, to be able drive huge improvements in a field, you do not need to know everything about that domain. Yes, biology is messy, but we already had incredible breakthroughs in this domain without understanding it 100%. Did we have the full picture when we created antibiotics, vaccines etc.? Maybe solving the big diseases do not require the amount of detail/experience people assume. Maybe we have/will have enough data to extrapolate and create solutions from imperfect knowledge.
I can't speak for math, but from a SWE perspective I'd say it's a very mixed bag.
One thing is clear that AI produces software faster, but the net results of that haven't really shown to be positive. There's undoubtedly more software now, but quality isn't improving. There's real downsides too like professionals unskilling themselves, and the cycle of low quality source being fed back into training.
Can't wait until AI systems enter the domain of infinite abundance. Stupid experts have been telling me my perpetual motion machine is crank science but I know they're just short-sighted losers.
Experts being wrong about a prediction in their domain is newsworthy, of course people are gonna point that out. They know their domain but they don't know/understand the extent of the power of AI.
Human trials only exist because of the uncertainty associated with new medicines. Eliminate that uncertainty and you have eliminated the need for trials.
There simply isn’t sufficient recorded information known about the human body for LLMs to one shot new medicines. These models are trained on known measurable information, so much of which we don’t actually know about the human body
You can’t just overcome that uncertainty with really good guesses based on what to do know. Not at least until you know a lot more.
Secondly RCTs exist because you want to capture a random portion of the population, some of who may respond differently to different treatments. That and the body is a dynamic organic system and not everyone’s is in the same state.
LLM will improve the pace at which new medicines are developed but it be a long time before reasoning entirely replace the need for empirics.
Human trials do not exist solely because of the uncertainty, but it is true that a medicine with a significant effect could be accepted after heavily shortened trials. If AI hypothetically invents a cure for a serious disease that immediately works for virtually everyone that takes it, there is no way how the trials would take years. They only take years because the effects tend to be "we decreased mortality from 80% to 70%", and it takes a lot of time to confirm that.
There will still be a trial, it will just be significantly shortened if the effect is very strong (and the disease is serious enough, so any risks from a shortened trial are outweighted by the disease itself).
Why this emphasis on shortening human trials? Human trials can be just as long and thorough as they are now, as long as AI is capable of one-shotting working cures. A few years of wait are ok if you don't need to go back to the drawing board 99% of the times.
Sure, but then we’re talking about “singularity solves everything” rather than any generalization of current performance. Claude doesn’t one-shot perfect code, for example, it just has access to a harness that lets it test things and respond to the results of such tests.
But vonneumannstan did not claim Dario was uniquely suited to predict the future. He claimed Gary Marcus and Eli Lilly CEO weren't in a position to predict future, which I think isn't very controversial.
Do you know anyone worse at predicting the future than Gary Marcus?
I would like to give the people reading this an intuitive explanation as to why he's wrong.
There's a famous study in synbio that led to standardization called, Reproducibility of Fluorescent Expression from Engineered Biological Constructs in E. coli
That's the result these machines spit out for the same set of samples being measured across 88 labs. This was termed a "remarkable degree of precision" (for biology) of 1.54x sd(!).
So the reading is different depending on when you measure it. For some reason, people in silicon valley think that biology is like classical logic and computing. You write some genetic code and you get expression out. It's not.
That's the pop-sci view of a real process. In reality, viruses and bacteria are extremely fickle, and hard to grow. A lot of the synthetic biology results aren't easily reproducible even if you know the protocol.
So you're telling me that you can fix that with "just" AI and "robots?"
Well... I would hate to break it to folks. But applying neural networks and ML to biology has been a thing since the dawn of AI. It was one of the very first applications people thought about.
It used to be called computer-aided drug design, then they went all in on expert systems, then there was a time when quantitative structure–activity relationship AIs were a thing, then there was teh deep learning phase... D.E. Shaw actually did a ton of work on it, https://www.deshawresearch.com/technology.html he used his money to buy a supercomputer and used that to train neural networks...
Blockbuster drugs have come out of this. Tirzepatide / mounjaro was discovered by algorithmically iterating through sister-compound families of the peptide they'd found in Gila monster venom until they found just the right one.
The issue that Anthropic and Open AI face here is that there's no low hanging fruit. There's no free lunch. It's very unlikely that Claude or GPT can do something that hasn't already been tried at some point in every AI/ML research era.
So they won't be able to "just" discover a new drug. If that was possible, then it would have already been iterated into by sheer accident based on the number of projects that have tried some variation of this with some sister architecture over the many decades preceding us.
Now where these machines do help is that they help people get started, and they also help working scientists. It's boring stuff, but think stuff like, "hey my machine isn't working, it's made by such and such. Can you write code to help me save my data?" or, "Can you proof read this grant?"
It will even accelerate the already tried approaches as it allows a smaller team to try more AI/ML algorithms than they would necessarily have had the expertise to execute before.
But will Claude magically solve medicine? No. I'm sorry, but it just doesn't work that way. And I'm not sure why they believe this, but I keep repeating this over and over again... A lot of this AI and biology stuff is nonsense. Standard undergrad and post-grad stuff whipped into fervor and fluff.
Excellent post but while you definitely sound very well informed and provide solid backing for your claim “they won’t just invent a drug” it’s not very convincing. If I had to bet, I’d still bet on the new ai companies advancing pharmaceuticals more than hypothetical counterfactual non ai enabled pharma companies, putting aside political and financial constraints around that. It seems like you know the domain but knowing the domain is less important than knowing what is outside the domain viz a viz tech disruption. The general matters more than the specific sometimes.
I think you're thinking that the research process will be something like Dario opening up claude code and typing "Claude, solve cancer, make no mistakes". But that just isn't the case. Consider the ability to develop new tools and software for drug discovery. Consider the advances that google has made with AlphaFold. Consider the ability to crawl mountains more data than every before and finding the smallest hints, tucked away by connecting 6 papers written in 4 languages, published 60 years apart. Bioinformatics is a relatively new field, but when I finished my degree in 2018 the field was still so far behind the cutting edge of computing it was laughable. "state of the art" was dynamic programming algorithms and I remember that the first implementation of blast tools were only just ported to CUDA while I was in class, and that really wasn't that long ago. Protein folding software when I was in school was basically completely manual. You needed to define the structures yourself and then give it a place to start and it would (entirely on cpu) do some sort of gradient descent to give you an approximation of the structure.
The thing that I noticed while I was doing my degree was that the CS people refused to talk to the bio people, and the bio people refused to talk to the CS people. The CS people would go on and make solutions to problems nobody asked, and the bio people would try to solve data problems in complicated ways that made no sense. Overlapping expertise is and was very rare between those two fields. AI however CAN be that bridge, because it can basically turn your average bioinformatician/biologist into a really competent programmer, without them having to give up time away from the work they are actually skilled at.
Maybe I'm just completely AI-psychotic, but I've always been a researcher at heart. I hated doing literature review because it was so frustrating falling through dead ends. These last few months with agents I have been able to go deeper and find better leads than ever before. AI is like having an insanely smart lab partner, available to you 24/7/365. Never gets tired, never has a pissy day, never gets too hungover to read papers. I do legitimately think it's going to accelerate the field so much faster than people expect. I don't know about the 10 year horizon, but I'd love to see it happen, and I'm cheering for them.
Judging from those tweets, I think that community isn’t just accepting that he’s excited and doesn’t know what he’s talking about
But also I think simulations like a “body world model” of some sort will help narrow down what to test for the fda’s standard
And the fda itself has some newer faster approval pipelines
Anthropic is also clear about how they are partnering with a handful of life sciences firms to coexist with people that do know what they’re talking about
So I wouldn’t be too dismissive, the agentic form filing aspect will go a lot faster, that part is basically already done
Comparing Anthropic to Theranos is more ludicrous than Dario's claims. Dave Ricks' (Eli Lilly CEO) tweet is much more guarded and accommodating of the medical benefit possibility of AI than the sensational framing the OP provides. I think Ricks is right -- Dario's timeline for cancer and Alzheimer's is likely way too aggressive. But comparing Anthropic's actually shipping and continuously improving AI models to a bogus medical device shows a stark lack of thinking. Mental illness is definitely a different consideration here altogether and I think Dario deserves an eyeroll at minimum for that claim.
A deep and lasting reckoning is sorely needed.
People like Amodei have to be brought back to reality by being confronted with the utter failure of their distorted view backed by deranged capital.
I hope that this whole thing comes crashing down in flames like a fucking Zeppelin soon, so we can move forward to actually integrating these incredibly useful intelligence tools into where they can contribute to society and innovation, at a discount and without the CEO-enticing job-replacement hype.
I'd disagree with the statements and people making fun of it.
While these people are obviously professionals in their field, anyone who touched pharma/bio IT with a ten foot pole understands how much there is to improve there. And that's just the beginning - imagine how much there is to improve in understanding of the body overall?
It's not about "AI will discover a cure for X in 10 years".
But it's about having the applied intellect of a an engineer, chemist, biologist, and all mixtures of that, available on demand, in a second, at scale, enabling you to do everything around it faster, better and cheaper.
Every step of drug discovery, synthesis and testing can be improved.
Over 90% drugs tested in humans fail. And maybe 0.1% of overall discovered ones.
While yes, testing on humans is the bottleneck, we can fix problems around the bottleneck easily.
If you had a data processor and 90% of data that goes into it is corrupt, the processor isn't the bottleneck yet - the data quality is.
If we can create better biological models to test these drugs, better interaction models, improve pharmacokinetics, improve yield, we can go from 1000 discovered, 1 works, to 1000 discovered, 100 works. We can predict some candidates will fail due to better models and not test these. We can improve how they are delivered to the body and make them more effective. We can catch potential side-effects early on and avoid going to trials at all.
This is what AI can do even nowadays, let alone 5 or 10 years from now.
And that is a huge improvement. It revamps the industry.
I don’t see how cancer is not solvable by AI - if you can cheaply detect the genetic defects that are causing the cancer in any patient, and you can cheaply and rapidly generate and validate retroviruses, you can effectively treat any cancer.
How cheaply? How accurately? Your proposed solution puts a lot of strain on the body, so even if it fixed cancer you wouldn't want to do it preemptively every day for example, likely shortening lifespan substantially despite cause of death not being cancer any more. It's a scheme that also relies on detecting cancer quite early (else the bulk of a tumor can easily be unchanged from that intervention), so you'd expect quite a lot of those false positive scenarios. Not to mention "cheaply detect genetic defects" sounds like a pipe dream for now with or without AI (assuming you're referring to single-cell mutations, since any other interpretation just doesn't address enough root causes to actually cure anything close to most cancers).
I remember when we were having more or less this same conversation about IBM’s Watson. It wasn’t even that long ago.
The most interesting difference between then and now is that we didn’t have tech enthusiasts getting quite so invested in the conversation. Possibly because we were all familiar with IBM’s decades-long history of shipping overhyped and underwhelming products.
Unfortunately we seem to have yet to realize that that’s a habit that’s endemic to the tech industry. Instead we still see it as thousands of isolated cases that, only by sheer dumb coincidence, happen to comprise the bulk of the tech industry.
Could you explain why? Because I just don’t see the analogy.
Openness and sharing is precisely why the Internet was so successful. Anthropic, by contrast, is a proprietary SaaS vendor with monopolistic ambitions that’s openly seeking regulatory capture and actively supports the swarms of AI bots that are killing what remains of the open Internet by driving everyone’s hosting costs up to an unsustainable level.
being close to someone with a rare terminal and slow-moving cancer that can take years and years to kill its host, i’ve been learning that even just testing whether a therapy is going the right way can take many years, so while i hope “magic happens” i have to concur that the statements being made ahead of a mega IPO seem a bit tragicocomical.
Something that I learned from COVID was how much variation there can be in regards to how affected one can be from both a disease as well as its vaccine.
I know someone who got long COVID, and I'm thankful I'm not someone who was prone to it, but we still don't seem to know what makes someone prone to it or not. The human body seems even more complex than the complexity we knew about that was already difficult to comprehend.
Some people legit think there is no need to go to school, work, learn or care about anything because Musk, Amodei, Altman &co promise them "universal high income" and eternal life in 10 years...
Wow. That subreddit is giving proper religious cult vibes, especially with the whole sense of superiority, uncritical belief and faith, and banning heretics.
The comment by Megan Fritts ("a professor of philosophy"), shown in the article, speaks to me possibly more than any of the other comments there. "Cult leader" is exactly right.
Did Terence Tao really pretend that the LLM, which is the bit without tools (such as basic math calculator and Lean verification), can do math?
And if Tao mentions LLM, does he really mean "the bit without tools" or is he conflating the two?
I know that Tao is saying that current AI is useful to advance research in math (and I fully agree with that), but 1) that is part of a whole framework, LLM is just part of it, and I think that it is not the part that "know how to do math", 2) even if you obtain the correct mathematical output, it does not mean that the LLM part itself can do math (the same way that someone who throws 10000 basket balls and misses 9999 of them should be considered as unable to play basketball even if you just have a process where you ignore the 9999 misses and only record the one that made it)
Are you sure you know what math is, or are you thinking about arithmetic?
Tao and others don't sit around working out what 12,483,324,281/188,553.37463286*(243+2-983) is all day. There is a reason most people don't consider mathematics education to really start until you stop needing to bring a calculator to class.
You seems to have scanned the message and notice one word ("basic math calculator") and stopped there.
Just next to this word, there is "Lean verification", which is not about arithmetic but about the logic.
But the problem is not even there. LLM are not able to do math not because they fail to do 12,483,324,281/188,553.37463286 * (243+2-983), but they are not able to do math because they don't _understand_ 12,483,324,281/188,553.37463286 * (243+2-983).
I cannot do 12,483,324,281/188,553.37463286 * (243+2-983) myself just like that, yet I can do advanced math. Because I understand this expression. I know what it means. When the LLM was not able to produce the correct number, it was not because it knew what this expression meant and was "distracted" and forgot to report a 1 when doing the mental calculation. It was because the LLM was seeing the expression and was trying to guess the answer based on the answers of other texts that had similar expressions. It did not _understand_ the expression itself: it had no idea what it meant, it was just guessing the answer by copying the rough relationship it extracted from outputs it has seen in the wild.
And this is why it was a good element to demonstrate that LLM cannot do math: if they don't understand the expression 12,483,324,281/188,553.37463286 * (243+2-983), then they don't understand the expression f(t)=\sum_{k=-\infty}^{\infty} c_k e^{j k\omega_0 t},
\qquad \omega_0=\frac{2\pi}{T}.
And that is why the AI framework to do proper math and help mathematicians needs harness with tools like Lean. The LLM part does not understand what the expression means, it needs a tool to discard, internally, all the LLM outputs that are just non-sense.
First, the claim has nothing to do with "understanding" itself. "understanding" here is just a short cut for "is doing the math", "is doing the computation", "knows what is correct and what is not correct". Replace it by that if you are confused by the word "understanding".
The claim is "LLM cannot do math". LLM. Not LLM + tools and harness. Not LLM but we get plenty of answers, run a program to remove the non-sense ones and pretend the LLM got only one answer.
And this is true: the LLM part cannot answer correctly these math questions. It answers with a first answer, check it, if not validated, came up with another answer, and so on. It has no idea which one is correct and which one is non-sense. It needs a specialised harness that will remove the non-sense ones.
But secondly, this incapacity to grasp something very simple is really annoying.
This is really not complicated. If you tell someone "what is 2+3" and that this person is going in the next room, write on their pocket calculator "2+3=1", click "ok", and the calculator says "Error", then "2+3=2", click "ok", and the calculator says "Error", then "2+3=3", click "ok", and the calculator says "Error", then "2+3=4", click "ok", and the calculator says "Error", then "2+3=5", click "ok", and the calculator says "Correct", and come back and say "2+3 is 5", it is clearly not the same as if you ask someone who understand math.
No need for "philosophical question" excuses. Call it "understanding math", or "able to do math", or "has learnt arithmetic", ... It does not matter. The reality is that this person, ON THEIR OWN, is unable to do math. They are not the one who know the math, the calculator who tells them if it works or not is the one knowing the math. (and again, if you are confused because this simplified example is using arithmetic, just replace "pocket calculator validating the sum" by "Lean software validating the mathematical proof")
We've had the ability to brute force solution spaces for decades, without AI.
Perhaps you picked up the belief that LLMs are just brute forcing these problems, but they aren't. They are creating hypotheses and checking them, which is the same thing humans do.
The solution spaces for problems is so massive that brute forcing it is beyond what even "every atom in the universe is a super computer running for 13B years could crack"
You need to understand the problem to know where to search for the solution.
They are not "simply brute forcing". They are constructing hypotheses based on their training on other examples. So, it's not a simple brute force, it's an "educated by historical data brute force".
If you ask "1224245 * 12345332", it will not try "tomato", it has observed other cases where such numbers are like that in such expression, and it will try answers that "looks like" those. Sometimes picking up on good relationship ("when the first number ends in 5 and the second number ends in 2, the solution ends in 0"), sometimes picking up bad relationship ("in my examples, I see that in the solutions, the fourth digit has 22% chance to be a 1").
But still, it does not really matter. A LLM, without harness or tool, cannot do math: it returns answers that are "reasonnable guesses based on historical data" without knowing which ones are non-sense or not, and it needs tools and harness that "do the math" to rejected the non-sense.
> You need to understand the problem to know where to search for the solution.
"What you mean by understanding"
you: "well, that thing that LLM does"
"what's the thing"
You: "Understanding!"
"And what is understanding?"
You: "It's the thing that AI do! I just said that!"
But, no, the "logic", the "structure", of math, is not inside the LLM. The LLM regularly returns outputs that are very good, because turns out that some relationships that the LLM extracted from historical data was also the relationships that happen to work here. And sometimes, the LLM returns outputs that are totally crazy, incompatible with "understanding the problem". Because the LLM is just doing "educated brute force" and applied a set of relationships that someone who is familiar with the problem would immediately consider inadequate, but the LLM still does it because it does not understand the problem, just tries to guess the direction of the answer based on historical direction of answers, without any idea if these directions are non-sense or not.
I'm sorry but you are horribly confused about how these models work. This reads like your understanding comes from social media comments (stochastic parrot stitching together snippets from an archive) rather than what is actually happening (generalization of distilled knowledge).
You're trying to argue that a Michelin Chef doesn't understand cooking because she only uses techniques and ingredients that others have used before - even though the dishes are totally new and other chefs are studying her techniques.
Well, I am a scientist with a PhD, I work with ML professionally since "before it was cool", and I have built my own toy LLM.
It is still possible that I'm incorrect, of course. But my understanding of how it works seems to fit very well the data.
"generalization of distilled knowledge" is a naive way out. People who have experience with ML have seen how difficult it is for the system to not get stuck in a local minimum or not latch on to a spurious pattern. Just for arithmetic, providing tons and tons of 6 digits multiplication tables, it is still extremely difficult for a ML model to extract the knowledge, there are a huge amount of fake patterns that leads it to incorrect heuristics. For example, the values of the first digit of the two numbers are strongly correlated to the value of the first digit of the answer, so the training will push into the direction for the model to learn to answer the computation based on this kind of heuristic. The "distilled knowledge" that the LLM will extract is not basic mathematical rules, it is "74% of the time, it's 4 and 26% of the time, it's 6", and it will fall into this local minimum and it will be difficult for it to get out of it.
But as I've said, I'm open to the possibility I'm wrong. If it's the case, maybe you can convince me otherwise. But it would help to bring concrete arguments more than "you sound like you don't have experience on this subject".
> You're trying to argue that a Michelin Chef doesn't understand cooking because she only uses techniques and ingredients that others have used before
But I don't say that LLM cannot do math because they use techniques and ingredients that others have used before. I'm saying that LLM cannot do math because the way they work. Fundamentally. The LLM extracts pattern by minimizing the loss function. There are hundreds of heuristic that will lead to a small loss function while still returning bad answers. Some patterns are "the good ones", but hundreds of them are misleading ones. And they can even get stuck in the wrong valley, with the way they learn preventing them to climb the ridge to join the "correct" valley. The fact that LLM that were very convincing to talk like they understand the human language and yet had a 2% success rate with basic arithmetic shows that LLMs often fail to extract the correct patterns. These LLMs did train on tons of numbers. To the point that they were able to do very convincing things. And yet, they did not discovered the "distilled knowledge" that were the arithmetic rules.
This, for me, seems to be coming from "social media comments". It is easy for a layperson to get "oh, if they understand that they can compute 44254324 * 53326342 by using simple arithmetic process, then they will get the answer right, so it means that optimising to get the good answer will, for sure, lead the LLM to learn the arithmetic rules". But the layperson does not understand how easy it is to find a misleading heuristic instead. Or even to build a mix of both (applying some arithmetic rules correctly for part of the problem, and misleading heuristic for the rest. Can you imagine the impact on the loss function in this case?)
Finally, an important point: for me, the big problem is that this view of LLM, that I have explained, is fitting all the data. This explanation not only explain why LLMs are so brilliant on some stuffs, but also do things that were unexpected, especially if LLMs are "generalizing distilled knowledge" (if it is the case, it is difficult to justify things such as hallucination, or crazy answers when asked the number of "r", or crazy loops when asked the seahorse emoji, or why LLMs fail to do basic math, or ...). This explanation makes sense and "scale": it allows me to understand small ML, understand why they fails, then understand what happens when we increase the training, then understand why these huge models behave the way they behave. The argument "but suddenly, there is a magical thing that happens and the LLM becomes something else: it suddenly understands distilled knowledge without falling for spurious patterns" is not only not explained, it is also totally useless: we don't need to assume that, everything that we see make perfect sense and is perfectly consistent with the LLM-as-a-statistical-heuristic-patterns-extractor.
a man who had his testicles removed while he worked in China will say anything to fulfill his dream of revenge. standing on top of the world, curing cancer.
LLMs, while definitely a useful tool have not massively replaced engineers.
If we can’t even have huge paradigm shifts in the area it’s specifically tailored toward, there’s a low probability it will eliminate entire fields that require physical testing and extreme domain knowledge.
We don’t have self driving cars, solid state batteries, or humanoid robots. None of this has been sped up by AI in the way Amodei is suggesting.
I like how he declares "mental illness" to be solved, when this is an even more varied and ill-defined concept than "cancer".
Authoritarian states often declare rejection of the state narrative as "mental illness".
Christian conservatives declare much, if not all, of queer culture as "mental illness".
On the other hand, some more consensus variants of mental illness, such as addiction or depression are actively caused by tech products and are fundamental to the value proposition. Tech companies invest enormous amount of work in how to make their products more addictive.
AI has already spawned its very own concept in the form of "AI psychosis".
That's not even starting with "virtual friends" or downstream effects like destruction of social skillsets and environments.
So, which of those things are included in his definition?
My problem with all these fantastical predictions, is that most of these real big problems aren't really technological problems; they're social and political problems.
In order for "AI to solve X", there has to be a tremendous amount of resources put into not only AI solving X but testing and implementing the AI-generated solution(s). Who is going to put in those resources? The money flows towards what's profit-generating, not problem-solving.
So far, AI hasn't solved any problems that matter in terms of improving the human condition in any significant way. What has it accomplished so far? It has allowed companies to develop software faster so they can cut their workforce. It has solved some tricky math proofs. That's about it. Neither of these make a dent in improving humanity's lot.
"Yeah now it's just a chatbot, but once it gets better, it'll solve X." Yeah, sure.
_Theoretically_ AI could develop solutions to some gnarly technological problems -- nuclear fusion at scale would probably be at the top of my wish list -- but as we've seen the last 2 years in the US, the main reason we aren't relying more on cheaper/renewable energy sources is political, not technological.
Medicine is similar - the reason we still have Ebola outbreaks in the DRC isn't because we don't have a cure for Ebola. Same for the half a million children who die from malaria each year; it's not like we don't know how to stop malaria. (If they were American children, we would have have eradicated malaria by now, but they're African children ...)
World hunger: we know how to produce enough resources to feed and clothe the planet -- it's just very badly distributed, by design.
AI is solving phD level open math problems, even though I was told it was just a stochastic parrot that would never generate new ideas or solve unfamiliar problems.
Self driving cars roll past my office every couple minutes, even though I was told AI would never be able to drive on public streets.
AI images and videos are indistinguishable from real videos, even though I was told they'd never be able to generate hands correctly.
But I'm sure the naysayers are right this time, AI defintely won't ever be effective at medical research.
It's interesting him talking about AI curing mental illness, when there is now a quasi-established AI mental illness called AI psychosis [1]. As the saying goes: "there is no such thing as a free lunch".
To be fair he does not say cancer, Alzheimers and mental illness will be cured in 10 years, but that they will become 'relics of the past'.
Just do not feed that vague reward signal in their AGI training run because combined with Amodei & bros being firmly in the war hawk paranoid anti-China camp they can make us and our diseases all relics of the past just to prevent offline local inference with chinese open weight models.
> he does not say cancer, Alzheimers and mental illness will be cured in 10 years, but that they will become 'relics of the past'.
Doesn't everyone believe that, on some timeline, we will cure all of these things? Like if I said within a million years, you'd be hard-pressed to find a serious doubter.
Dario just thinks he's going to build ASI and ASI will solve them more-or-less instantly. I believe an ASI would do that, as well.
So by my logic, naysayers really just don't think we're about to build ASI. Dario does. I figure he has a better perspective. But we're not arguing about solving all of that stuff, we're arguing about whether ASI can do it, and whether we're on the cusp of ASI
At some point I will have a LLM explain Gary Marcus to me. He seems just silly and incorrigible, and I am mystified as to why people I think are important think he is important.
A corollary to Gell-Mann Amnesia: I don't know shit about drug development, but I know a bit about the cybersecurity industry. Most of what Anthropic has said about cyber isn't wrong exactly, but it's been breathless and so crudely expressed that it fails to capture the real path that developments are likely to take. For example, Anthropic said that AI might create a giant botnet (no shit, so what) whilst not really talking about agentic offensive frameworks (uh-oh) at all.
I have to assume that they're probably doing the same with medicine.
Ricks might want to be careful about offending someone who controls the largest and most powerful kompromat collection and manufacture machine to have ever existed.
The old never pick a fight with someone who buys ink by the barrel should probably be updated for people who buy power for their AI data harvesting and influence campaign by the gigawatt.
Scientology has got nothing on the AI safety cult in terms of capacity for dirty tricks.
In short - the reason medicine is slow for AI is the same reason it's slow for people -- the REPL loop is slow [1]. If you could pause and rewind time, and had perfect observability into every system, no ethical risks, (things that are free in software engineering, or math problems) then the AI could make massively parallel progress.
However I do think there are opportunities to greatly improve all these fields with AI, I just don't think we should lead with an overpromise that sounds Musk-like.
https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93prin....
Of of the things it does is make hidden research visible in a way that previous search engines or journals did not.
You're thinking like a law-abiding Westerner. The ethical risks are not a concern for all, if anything the humans are the cheap bit of the process. There are a few ways to improve a genetic algorithm exploration, one of them is to simply run a larger population at a time.
There have already been a few attempts by pharmaceutical companies to accelerate drug development by running clinical trials in poor countries with weaker patient protections. Regardless of the obvious ethical concerns, it turns out this doesn't actually speed up the process. It still takes just as long to discern a clear signal in terms of effects versus side effects. And those poor countries typically have infrastructure, security, and labor force problems which make it challenging to run large scale clinical trials at all.
If that is true, then even before AI pharmaceutical companies would have already moved all their research to places where their development would be much faster (but less ethical).
Pharmaceutical companies don't care about "ethical" research either, all they care about is making more money, faster.
> law-abiding Westerner
may I ask why did you specifically pinpoint to Westerner?
What's special about westerners when it comes to law?
What's suspicious about this is not whether it may or may not be possible (it likely isn't), it's that he is using it to borrow money now, that is not collectible in the case of a default. You get a huge exit because you overpromise today, and when it turns out you aren't a prophet, those that trusted you cannot be made whole.
People and the law keeps treating this as "just a sales pitch", but it's really theft in disguise and should come with evidence on an S1 filing.
What do you mean it’s not collectible in case of default? Are you under the impression that money that you borrow before the IPO is excluded from the bancruptcy claims?
>> Are you under the impression that money that you borrow before the IPO is excluded from the bancruptcy claims?
it's being "borrowed" in the sense that there's an implicit promise as part of the IPO sales pitch, but shareholders are last in line to collect in case of something as extreme as a bankruptcy - and that's if there is anything left if that was to happen, which by definition is not frequent.
Yes, everyone purchasing equity understands that they're last in line after debt holders. This isn't some kind of secret.
Obviously borrow is being used loosely here -- it's actually equity. And second, yes, some creditors don't get their money back in a bankruptcy.
It never was, the vast majority was VC investment not debt so collection was never on the table...
>because you overpromise today, and when it turns out you aren't a prophet, those that trusted you cannot be made whole.
Why are these people obligated to trust Amodei? Why are these people obligate to hand Anthropic money?
I really hate this "Investors should be protected from losses" narrative you see pop up now and again like that isn't the entire natural order on how we conduct these things and somebody not being made "whole" is somehow unfair. It's not theft, Dario will simply accept your money if you give it to him.
You make a bet you can lose. These are grown adults making decisions.
I sure hope no major indices bend or break the rules about including <s>bubble stocks</s> freshly IPOd companies in them, since my entire prospects of retirement are tied up in those indices being relatively stable!
I really hate this "Investors should be protected from losses" narrative - right, we should also retire the SEC and every investor protection law and regulation, because what where we thinking?
I’ve said it before and I’ll say it again: nobody pulling the strings in this industry gives a shit about anybody, or really anything other than increasing their available capital. They’re powered by some combination of greed, inertia, and religious fervor. Their primary role is to groom investor sentiment and the weak job market means they needn’t feign benevolence to recruit competitively. This is the purest expression of post-.com era SV culture I’ve witnessed since I started in tech nearly 30 years ago.
> nobody pulling the strings in this industry gives a shit about anybody, or really anything other than increasing their available capital
This is true for every industry, group, non-profit etc…there are no organizations not fully captured by monetary transactions and hoarding capital.
There are people who genuinely want to help others because they realize it is the right thing to do. Also applies to organizations governed by such people.
I used to take my truck to Larry Roberts of Roberts auto. He’s a neighbor and friend and did oil changes and basic stuff. Larry closed up shop cause he can’t work on modern cars and doesn’t have the tools or staff to handle anything more than basics for cars older than the 00s.
Larry is great but he’s not even relevant to the discussion because you might as well be describing a DIY or hobby shop.
None of the people or organizations you describe are relevant to how the world works, they are temporary outliers with limited impact for a brief period
Even then, Larry didn’t work for free and wasn’t running it out of the goodness of his heart. He did it for money
The only people I know doing shit that isn’t for money are the people that actively work on the trails that I walk on in Northern Virginia and guess what, they work on them so they can have nice trails not because somebody else is going to use the trail.
Absolutely false. I’m not young, and I’ve worked in many different industries. Once you get away from for-profit businesses controlled by third party investors, that could not be further from the truth. Having monetary motives doesn’t mean you only have monetary motives, or lack willingness to address your industry’s ecosystem, employees, and the communities you touch. Believe it or not, some people that run successful companies have pride, ethics, and red lines they will not violate to pump up their valuation or throw a few extra bucks in their pockets. I left software because considering anything other than money is anathema to its business culture. I’ve worked extensively with people in arms manufacturing and even they are more motivated by business ethics than any executive I’ve encountered from the SV software scene in what feels like a fucking decade. For an industry that was started by people that wanted to make money by solving people’s problems using innovation and cutting edge tooling, this is a pretty pathetic state of affairs.
I was a military officer for 17 years, the only thing that matters is your unit/wing/program office is your budget. The only thing more important is readiness and guess what the main thing that effects…your budget
The idea that arms manufacturers are more ethical is a joke - at best you get propagandized people ignoring how their munitions are used while banking 150k salary minimums.
I’ve run mutual aid cooperatives and whatever you try to do will be overrun by private equity and your market will be closed and the people who show up will be flighty and noncommittal
I currently am ED for a non-profit that I will not allow to be used for profit making - almost 100% of the people who ask to volunteer or show up do it for their own promotion or to jumpstart their product it’s maddening the number of times I have to say no.
I am the founder of the 150,000 person ultramarathon subreddit and I’ve only found two (2) moderators in the last 15 years that aren’t trying to promote something for money.
There are no organizations that escape this because at the root level you can’t find more than a literal handful of people who aren’t in it for the money - even if thats not their direct reason it’s usually a secondary reason
Even “monks” go out and write books and buy fancy cars
There doesn’t exist in my daily survey of social organizations ANY, and I mean there isn’t any single organization, that does what it says it does without trying to make a profit in some way.
> I’ve run mutual aid cooperatives and whatever you try to do will be overrun by private equity and your market will be closed and the people who show up will be flighty and noncommittal
… yeah… right…
> Even “monks” go out and write books and buy fancy cars
You are acting like a bitter, cynical edge lord that only sees the worst in the best people. Ironically, I’ll bet you think you’re the most rational observer you know.
What you say you see in the world says far more about how you see than it says about the world. Later.
So a variation on the Chamath Palihapitiya 'SPAC' method?
Amodei sounds completely unhinged in that video.
Yeah, I'm surprised we're all discussing it. I'm open to the idea that AI could help medical research, but just claiming it'll end "alzheimers, cancer, heart disease and mental illness" with absolutely 0 evidence isn't a particularly grounded claim that warrants any discussion.
Honestly, the way he lists them and even pauses before vaguely saying "mental illness" gives a strong impression that he's choosing whatever illnesses he can think of as he goes.
"'m open to the idea that AI could help medical research, but just claiming it'll end "alzheimers, cancer, heart disease and mental illness" with absolutely 0 evidence isn't a particularly grounded claim that warrants any discussion."
Agree with this if it was something a random person said over coffee or lunch.
But when its the CEO of a company that is about to IPO saying it, its very very different and more concerning.
He previously described it as 'most major disease' that he thinks AI could cure in the next 5-10 years.
Amodei might be an even bigger optimist than I am, but he's not lacking credentials on neither the biology nor AI side.
Accusing him of fraud or mania like some here do seems unwarranted.
We don't even understand how mental illness works. There's so much work to do before we can even imagine the possibility of "curing" them.
> he's not lacking credentials on neither the biology nor AI side
More importantly, he is lacking credentials in medicine.
Seems unwarranted? He is clearly following the MO of Musk by hyping up his company as being capable of doing the most ridiculous innovations but all it does manage to do is bump the stock. Terraforming Mars, tunnels for self-driven cars, curing cancer/all mental illness etc.
Yes. Expressing a belief or hope to achieve something in the next 5-10 years is not a claim of capabilities.
If you want to argue that Amodei is dishonest and only says these things to 'bump the stock', you can make that argument, but it's clearly speculation.
He's either dishonest or delusional.
When someone promises something that's so far out there as to be nearly impossible, and *without an actionable plan to get from here to there*, then they're either dishonest or delusional.
Or simply optimistic.
He hasn't promised anything, in fact he acknowledges that his projections might be wrong.
His stated plan is to scale AI and compress 100 years of biomedical progress into 10 years.
No, “simply optimistic” doesn’t capture it.
Like, if your optimism is great enough, big enough, then you lose touch with what’s actually feasible. That’s where delusion starts.
It might be possible to compress 100 years of research into 10 years. Maybe. But it won’t be these 10 years.
Check the math. Say we get huge advances in AI tomorrow that double the speed of biology research. And imagine we then have 3 years where research is now 2x faster (which, again, would be absolutely enormous).
How much faster do the remaining 7 years have to be, in order for the entire 10 years to capture 100 years of research? Roughly 13x.
Basically, every year you’re not already at >10x is a year you’re falling behind. And AI is nowhere even remotely close to doubling the speed of research, much less 10x-ing it.
That's not what Amodei is predicting, he does not talk about a scenario where biologists use AI as a tool to double the speed of research.
He is predicting the arrival of powerful AI in the next few years.
His plan is to scale that model up to millions of instances working in parallel, directing experiments and research autonomously.
In this way he hopes to accelerate the rate of important discoveries by at least factor 10 to achieve 100 years worth of progress.
Amodei's statement is at the level of Archimedes saying "I'm guessing that in the next 50 years we'll probably use this to fly to the moon" after discovering Greek fire. Sure, it's "optimistic", and may even point towards something that will eventually happen, but way way way past the point of delusion.
Okay but let's for the sake of argument that Claude magically has all the information required cure all of these families of diseases. How does Amodei intend to deal with the fact the regulation of medical devices is a slow, lengthy process for a lot of very good reasons like preventing something like thalidomide going on the market? Is Claude going to magically spin up considerably more regulatory capacity out of thin air, and will it gather evidence for safety and effectiveness at the required standard to show to those regulators?
This is just one problem I have with his claim, there are many others.
Read the letter being discussed. Alongside claims of medical innovations, he also says it can be used to speed up government bureaucracies.
Now, whether that's true is obviously it's own question. But I do think if AI can cure cancer and Alzheimer's rapidly, that improving the regulatory and approval processes at agencies is a much lower bar.
A lot of those bureaucratic regulations are written in blood and are slow and deliberate for a reason. Throwing them all out means more thalidomide disasters: Chesterton's fences and all that.
Credentials are one signal sure, but his MASSIVE financial incentive to hype the product is another one that should probably get much more weight in your analysis.
That would make for boring discussion, there are a hundred other people in the comments already accusing Amodei of being dishonest for financial gain.
I don't think it's the case, and that Amodei legitimately believes in what he has been saying for years.
I guess, if you discount his actions and musings.
Whenever I wonder how investors can fall for this bs I only have to read the comments to see examples of how
I'm genuinely trying to understand the argument you're making here. What sort of "credentials" entitle a person to claim their god-technology will cure all diseases within the year 2026?
It smells to me like "might be an even bigger optimist than I am" means you're also part of the rationalism cult and that it's literally impossible for you to say something negative about Amodei.
Amodei has made no such claim; at most he expressed a belief or hope.
Being an optimist does not make me part of any cult and there is no need for you to argue that I could not say something negative about Amodei.
> I'm open to the idea that AI could help medical research, but just claiming it'll end "alzheimers, cancer, heart disease and mental illness" with absolutely 0 evidence isn't a particularly grounded claim that warrants any discussion.
This is up there with the claims Chan-Zuckerberg Foundation made during their start-up that the foundation would "end all disease within our lifetime." So far they haven't ended one disease, and meanwhile they quietly watered down their claim to soften the language to update to "cure, prevent, or manage all diseases by the end of the 21st century."
Does narcissism qualify as a mental illness? There might be one more step here to complete the circle. Both Amodei and Altman have dropped some real shockers over the years and instead of seeing a reaction it's almost as if they've normalized the kind of talk that ought to see you in a room with soft walls.
You could have said same when he claimed "90% of code will be written by AI" in the beginning of 2025, but turns out he was right and he foresaw something we didn't.
What if he is right again?
Why would you omit “in 6 months” part?
[0] Anthropic's CEO says that in 3 to 6 months, AI will be writing 90% of the code software developers were in charge of. https://www.businessinsider.com/anthropic-ceo-ai-90-percent-...
That prediction (or at least that article) was written in March, and Opus 4.5 was released in November. For my team at least, that was pretty much the point at which 90% of our code became machine written. So that would mean he was only off by 2 months. Maybe 4 or 5 months at most for the point where it became true for basically the entire software industry.
I’m not saying he’ll be right about this one, not it’s a not ridiculous to claim he want fundamentally right about code.
90% of the codebase I work in is still not written by machines, so ... he's still wrong?
No, he's not wrong. Answering your question with a 'yes' is like a bicyclist claiming that people can't move at 60+ miles an hour in a car.
Are we counting code that’s pre-2025?
How about your competition? What percentage of their code are they still hand-authoring?
They, like us, are in a sector where quality of results matters very much. My id wants it to be very low, my superego wants it to be very high.
The small percentage of people who use AI produce vast quantities of code (the average lifespan of which is a matter of days).
You're just obsolete. That's all.
Writing means new code, not old code.
And if you've been looking at Github's blog posts on their exponential contribution growth for the past 6 months, it's obviously come true this year.
The rest of the computing world has had machines generating code since the 1940s when the first assembly languages were created. What sees you cling to the raw bits and bytes still, rapidly approaching a century later? Amish?
That's... pretty much how it's played out.
Because he was slightly optimistic but he was able to see what's coming.
and if you think off by 1-2 years is a big deal I am not sure what to say.
90% of the code will be written by ai in 6 months.
Hmm. People making fun of the idea that 90% of code can be written by machine.
Now you come and nitpick about if it is 89% or 7 months instead of 6. Lmao.
If AI can get any of those in even 20 years, that's still a big win.
The pessimistic attitude here is revolting.
And no, nobody is forcing you to buy anthropic shares if you think they are full of shit. Anyone complaining is either don't have money to buy it or just jealous of other people buying it and price going up than the ones they hold.
Pretty sure he made this prediction in context where it was interpreted as "90% of software engineers will be replaced by AI".
So re-interpreting it now as "Software engineers will be using AI for 90% of their code" seems like a pretty major retcon.
He said 90% of code, 90% of SWE tasks. He clarified this on Dwarkesh that it was worlds apart from 90% of replacement.
By, or with the help of?
90% by LOC, or by semantic meaning/value?
This is a pretty good question. Maybe LLMs are just, in some sense, ~10x worse at DRY than humans.
Or, I mean, in my code at least I can directly say that any line I write results in 4 lines generated by macros. So, I guess my projects were already 80% written by computers in some sense. If I was using an LLM would I have skipped the macros entirely?
I’ve always been in favor of doing this sort of measurement by flop instead of loc, but, then we’d have to give all the credit to compute libraries.
yeah, sure we will nitpick the meaning now, on a personal level I stopped writing code manually at least 7 month ago.
its helping me convert my ideas to code, which is not too far from the meaning implied, because things don't happen on their own, there should be a trigger, in this case, I am the trigger, AI is the producer
You already wrote in one language that was translated to at least one other language before being sent to hardware. Has that changed?
I started coding at 8 years old, 28 years ago, been doing it since.
Switching over to mostly LLM generated code and it's been mentioned many times in my circle and family that they can still tell the projects are blatantly something I would do, what it would look like, and how my projects usually work in terms of UI/UX.
I get basically the same outcome as spending a week on something by hand, in minutes. I'm getting more work done than ever, and not losing focus on projects after they get to that last 10% that needs done that nobody wants to do.
I've always been a lone wolf coder and have joked forever that all I need is just another version of me that I can feed the ideas and how to build them, because I'm the only one that will do it exactly how I want. Now I have that, and so much stress in my life has vanished.
Anecdotal of course, but those that want to be salty can be salty. I did my time coding by hand, now I'm still getting it done while keeping up with the times and having a blast doing it.
Lines of code have always been a liability not an asset. If I get a PR +500, -800, that achieves the goal set out, and one that is +8300, -1200, which one is better? SOTA LLMs tend to produce the latter and the people least capable of noticing this as something bad are the ones producing the most code. So maybe even _if_ - show me sources - 90% of code is LLM produced, I'm certain it doesn't do 90% of the work.
That doesn't mirror my experience whatsoever
Pre LLM a 500/800 loc pr was already on the longer side, with ~200-300 being optimal.
Those were however frequently being optimized by LLM to the 500-800 loc range.
But opus 5.5 does not produce 8k new loc refactorings without you forcing it to.
I can maybe believe that AI is writing 90% of the code within the AI labs using models the rest of us cannot fathom. Although they still seem to be hiring developers, so it is difficult to believe real strongly.
But across the wide world of software? Everyone I see is just using AI as a compiler. They are still writing the code.
(Yes, compilers also write code, but I don't think that was the intent of the message. We've been doing that for decades. 90% of the software has been written by AI since we discovered the optimizing compiler if that was the intent.)
Coding is his area of expertise, his claims there are inherently on much stronger grounds.
I don't think it's impossible that he's right. I'm ultimately buying Anthropic stock regardless of this discussion. But I would not take it particularly seriously if the CEO of Moderna took a break from their amazing work applying mRNA vaccines to diseases, and told the world that soon mRNA will solve climate change.
I think few know that Amodei was a biologist who has been researching cancer biomarkers at Stanford.
I indeed did not know that, but I don't think it quite contradicts my point either. Many of the perspectives Marcus quotes are consistent with the idea that Claude might be amazing at finding cancer biomarkers.
I dont think anything qualifies you to claim you've invented a god-technology that will cure all diseases with no added labor costs - something that has never happened before. But even if I did, that qualification wouldn't be part of an undergraduate biology education.
reminds me of Karp interviews, but less militant.
His toothy smile and nerd-like demeanor are disarming. Before he lost weight it was easy to see him as a well-intentioned non-threatening quantity but it's obvious he's just as unhinged as any of the others.
He honestly seems to be experiencing some level of mania, which I think a lot of people would if they were constantly in an environment that makes them believe they may have invented god. I do not think he's in his right mind. This is going to end up with a documentary about him, and not the good kind.
Agreed! There was a very good podcast released a few months ago, Black Box The chatbots. It’s generally about AI induced psychosis. I’d love to be a fly on the wall in some of these orgs, there’s bound to be a few people on the inside going through the same issues.
https://podcasts.apple.com/gb/podcast/black-box-the-chatbots...
I think it's just as likely VC-induced psychosis. This is what you get after 27 rounds of nonstop fundraising.
Both? Both. Both is good.
https://x.com/tszzl/status/2051045196260167790
"it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in significant part by claude, and studies and builds claude..."
If you revisit his past writing he has always been this unhinged. Took me too long, but now I see the same craziness in really a lot of what he published
I still don't understand why people don't just openly talk about the obvious fact that a huge percentage of Silicon Valley leaders are on large doses of powerful stimulants.
I don't know if he is in that video or not, or ever, personally, but just in general, it's the Ockham's razor approach any time you see something like this. It's a much more direct and clear explanation than the various more convoluted theories people keep throwing around as to why everyone sounds like a methed-out megalomaniac all the time.
I read this comment by curuinor a while ago, and still think about it a lot. https://news.ycombinator.com/item?id=49436647
> Look. This is a new golden age of cocaine, because of the big cuts in everything in the US took away the cocaine busting operations in Colombia and Bolivia, because the revolutionaries that previously kept control of it stood down and became a political party and new, profit-driven cartels came up, and because battery technology means that narcosubs and similar drone things work much better than before (and they can be made remote-operated, so the cops can't do shit even when they find one). Wholesale prices have halved or gone down 60% to like, 12k usd / kilo, street prices commensurate. The huge new opportunities created by increased market size have increased competition and made the cartel nature of the cartels decrease. So always think of that when you look at people acting like this and having such megalomaniac goals.
I hadn't heard this before. That's actually pretty interesting. My assumption has always been run-of-the-mill ADHD meds like Adderall and its several cousins, but I can't say I've got firsthand knowledge of either these days.
Yeah does seem like textbook uppers.
As someone who has never heard or watched Dario speak before (somehow), he just comes across to me as autistic in the video.
Autistic is not the word that sprang to mind.
The whole Bay Area is on a fatal cocktail of amphetamines, money, and… a few other things. When you take 100 people on that combination and put them in a room together, some, uh, unusual beliefs inevitably develop. Of course they do. What else would happen?
Yeah exactly. And it's not the first time a highly technically proficient culture had their leaders all become amphetamine abusers with dramatic effects on world events.
On a bit of a rant now, please forgive me.
I never feel more like the one sober guy at the party than when I go to SF. Even when there's no reason to believe anyone is literally on drugs, it has that antmosphere every time. It's really unsettling. Uncanny. And you get a contact high. You start to believe, just a little bit, that everything is possible, that there are no limits, that you can and should remake the world, that the truth can be whatever you say it is. Just a little bit. I can never take it for more than a week at a time, no matter how beautiful the weather is. It's a relief to get back to the "real" world, where gravity points down.
It certainly explains how you get the exact overlap of existential-risk fatalism and Bay Area polycule dynamics. Whos going to tell the artificial children that their parents were a high-functioning doomsday sex cult?
We're making some big assumptions there - you may be right but who knows what's in Amodei's medicine cabinet.
That aside, I think there's something that happens to the human mind (and most people are susceptible) when exposed to certain stratospheric levels of wealth and power (which are basically the same thing). Their thinking becomes ungrounded, uncoupled from reality. The tendency is to shrug off the limitations of logic and rationality and to allow their mind to dabble in ideas of limitless possibility and power because doing so feels good to do. It's a drug and a very seductive thought that one's will could be manifested in the world without compromise - that what's inside could be made real on the outside. That is, after all, what power is. Dario's imagined reality is that he is the savior of the human race, and that's the reality he's currently living in.
His statements are based on belief that AI will advance a lot and quickly.
If this happens to be the case - I personally don't see why those statements are not realistic.
All diseases are engineering problems. The bottleneck there is testing etc. Most likely many of them can be cured using existing approach or meds.
Companies grow organs now that can be used to speed up iteration.
Even current AI can solve a LOT of tasks if it has quick feedback loop and can verify results. At the end of the day it can brute-force its way to solution.
AIs intelligence is say average human. Even less, whatever. say it will be as a smart human in a few years.
But it is infinitely scalable.
So is it that unrealistic to assume AI can find a correlation on massive amounts of data and figure our what leads to Alzheimers? (where the solution could be not cure but prevention). I don't think so.
> So is it that unrealistic to assume AI can find a correlation on massive amounts of data and figure our what leads to Alzheimers? (where the solution could be not cure but prevention). I don't think so.
Yes. The mistake you're making here is that you're not understanding where the actual bottlenecks are in drug development, and why some targets like Alzheimer's have been so elusive.
The problem with Alzheimer's in particular is that we don't have a working model of how Alzheimer's works. The dominant model--the amyloid hypothesis--can pretty conclusively be chucked in the bin at this point because of the number of amyloid clearing drugs that just simply have no effect on the progression of Alzheimer's, but there's no model that actually looks any better. Without a proven model, the only to verify effectiveness of an Alzheimer's drug is try it out in humans [1] and wait several years to get a success-or-no-success readout. That's the rate-limiting step for Alzheimer's drugs.
Okay, so maybe you're going to suggest that we can use AI to come up with new Alzheimer's models. Okay, but the rate-limiting step is still proving those models, which because of the human-only nature of Alzheimer's, requires developing the drugs and testing them and you're going to wait a decade to get back your answer as to whether or not the theory was correct. And what data are you going to use? If you're trawling scientific literature, you're going to come up with only the same ideas people have already had for alternative Alzheimer's models (there are a lot of those). If you're trawling medical data, well, medical data just doesn't exist in the level of detail you need for that enterprise, and handwaving away the legal issues with collecting it, it would still take years just to collect that data (and probably still not be good quality!).
[1] There's another complication of Alzheimer's, which is that only humans get Alzheimer's, so we can't even do anything like testing in animals with shorter lifespans to speed this part up.
And the thing is, I fundamentally agree with the guy above you that all of these problems can be solved, that we can sidestep the bottlenecks and move faster.
But it requires tools that we are nowhere even close to having. One way would be to build a ground-up, molecules-to-man model of how the body works. And then from there, sure, we can easily model how Alzheimer's might come about and how it might be cured. Like, with tremendous advancements we might be able to finish such a project this century. In the next decade? That's like saying we'll launch an expedition to Alpha Centauri in a decade.
Amodei in 2024 wrote about compressing research that would take biologists 100 years into 10, so that checks out.
Given that he was making these projections already long before today, it would be fair to say that it's speculative rather than based on current research progress.
You could write that. But it doesn't actually mean anything, and it would be better to assume that biologists with tools will move faster on biology but that does not necessarily translate into things that have to do with medicine which is something else entirely.
I am not surprised that you would think that AI will be merely a tool that allows biologists to move somewhat faster, rather than make discoveries or control research.
Because I remember that you shared similar opinions about software where you cited liability as a limitation that for you precludes more autonomous use of AI.
I think it's safe to say that you have an entirely different expectation for how AI will progress and how it will be used, or at least that was my impression.
With the release of Fable 5.2, I have to say that you might be right. Maybe progress is already slowing down, it certainly isn't as big a step as I was hoping to see after Opus 5.5.
> Because I remember that you shared similar opinions about software where you cited liability as a limitation that for you precludes more autonomous use of AI.
Shall we wait until that trajectory has run its course before we start claiming victory?
As for the rest of this discussion:
Medicine != computer science, math or physics. Biology isn't either. The degree of complexity is many orders of magnitude higher.
Curing cancer : very unlikely, cancer is a systemic issue that requires many mechanisms in the body to do its thing, most of which we also require to stay alive. Short of a massive breakthrough in nano technology curing cancer is stupendously hard and it would be proof of reaching an entirely different plateau of capabilities for Humanity as a whole. Besides that, it isn't cancer that kills the most people on the planet, it is hunger, and then climate change.
The second one: Curing mental illness. I think it is equally delusional to be able to claim that you can cure mental illness because mental illness is a very broad description, so broad that it covers a wide variety of issues with an even wider variety of root causes the bulk of which are not well understood. Our brains are hard to appreciate in terms of capabilities and it shouldn't be surprising that something so compact and so powerful can break. Whether you can repair it after it breaks is not something you can say with certainty in the present unless you can demonstrate structural progress in the field compared to where it was say five years ago. To the best of my knowledge this is not the case.
Finally, Alzheimers. This is another one of those 'too good to be true' things. Alzheimers is next to cancer one of the diseases that has been studied extensively without much progress as to both root causes and cures. That doesn't mean that we won't find one, but here too: you would have to show at least incremental progress and to show a number of very promising leads that we didn't have so far to be able to make this claim.
In short: this is a marketing ploy, these three were picked because they hit everybody, either directly or indirectly. What they are saying is 'imagine, if we had this AI that we are working on today the problems that affect you or your loved ones would no longer exist'. This kind of promise is known to open the wallets of the gullible.
It is also fraudulent.
If anything I ceded some ground there rather than claiming victory.
Before I really change my mind, what would need to happen is little progress in AI until say March. Then I will really worry about progress slowing down and plateauing.
I am under the impression that Amodei sincerely believes in what he says, which has long been an acceleration of biomedical research to cure most major disease.
That he recently picked these specific diseases in order to essentially defraud investors is a large stretch.
Lots of people sincerely believe what they are saying. Some of those people may ultimately be right. But expertise in one domain rarely translates into another and even though AI is a 'universal tool' (just like regular intelligence) the chances of the new broom sweeping cleaner are high simply because the broom has a different shape and is working very fast. But that does not immediately translate into fantasy land 'wouldn't it be great if' promises coming true.
In fact I think Amodei is making a massive mistake here: you can make these promises, but they will be remembered. And when you ultimately fail - which is highly likely - it will look like you oversold your IPO and that's actionable. AI already has plenty of potential without talking about curing cancer, mental illness or Alzheimers, those claims should come from the specialists in those fields using AI as a tool.
It's akin to IBM saying they're going to put all of the banks and insurance companies out of business in 1950. Yes, computers back then were seen as just as revolutionary as we see AI today. It did transform the world, I give you that. But IBM ended up as 'just another company' and I suspect that the same will hold for OpenAI and Anthropic for many different, mutually reinforcing reasons.
edit: to address your last line: I don't think it is a stretch at all: if you think it is, show me the proof, show concrete progress that Anthropic has made in any of those three fields that warrant these statements. If you can't then you'll have to concede they are highly speculative at best and that's not what you do when you are about to take public money.
I think the projections must be entirely speculative, as they are from Amodei's 2024 essay rather than recent results.
When we talk about fraud, I think it's important that these clearly are not promises. In 2024 he specifically said: "Everything I’m saying could very easily be wrong".
His recent statements, including the one we see here in OP's video, say 'I believe' or 'I think', not that Anthropic claims to solve this by that deadline.
Perhaps he is overpromising or wildly speculating, sure, but I am not convinced that this is fraudulent or criminal at all; that said, I did not look into the legality of it.
Tell it to the LTEE.
One of the greatest issues with the tech culture broadly (and it's been festering for some time) is the belief, almost sacrosanct, that the epistemological set of knowledge that technologists and computer scientists have is able to be extrapolated broadly to every profession and field and that hard issues in a field like biology are obviously solved because OpenAI brute forced a few mathematical proofs. It doesn't work like that. Much of this comment reads like a sci-fi wet dream divorced from reality entirely.
The architecture of ML as it currently stands does assist us in modeling things like protein folding. But that's what they are; models. And before some person chimes in by saying "look how the AI pessimists move the goalposts", see my first point. Biological systems have high entropy and discoveries in biology often do not follow from preexisting discoveries whatsoever and are completely novel. Barring some quantum leap in ML design and architecture you cannot predict stochastically or deterministically how exactly a new enzyme will function in vivo. There is absolutely no evidence that Anthropic has solved this, even if they gobbled up every ML tool used in drug design and trained their models on every biomolecule known to man, it would not work because of fundamental gaps in what can be observed. They are a for-profit company trying to juice their IPO.
Now could LLMs and ML potentially speed up the pace of this research? Absolutely. But it's like saying a Formula 1 car is faster than a NASCAR. Yes, one is very clearly faster, but it's not as if the NASCARs are slow.
Amodei has a PHD in biophysics and he researched cancer biomarkers at Stanford; it's not like he doesn't know what he's talking about.
You can make the argument that Amodei is dishonest or that he suffers from psychosis.
However, he has made no claims that Anthropic has solved this, and expressing a hope to solve this in the next 5-10 years is hardly fraudulent.
> Amodei has a PHD in biophysics and he researched cancer biomarkers at Stanford; it's not like he doesn't know what he's talking about.
That's why I think that something is very weird with his statements. Like, how would this even work?
The feedback loop is much, much, much too slow right now for AI to make things better.
Like, step 1 is build a digital model of the human organism. If you can do this, then yeah maybe the rest is possible. But step 1 is incredibly hard, so it feels unlikely.
That being said, I think the stimulants hypothesis upthread may be correct, as I've heard a lot of crazy stuff from friends on them.
Of course Opus 5.5 is not going to help build a digital model of the human organism.
If we linearly interpret the last year of progress from GPT-5 to Opus 5.5, times 5x, assuming no plateau, could such an intelligence realistically achieve a task like that?
Maybe also not, perhaps some form of RSI would need to deliver accelerating progress.
There are a lot of assumptions and speculation here. Still, at the end of the day, it does not seem that unlikely that they succeed in creating AI that can reason over so many parameters as to remove all intellectual bottlenecks from the process, leaving mainly the physical limitations.
And what is the limit there to building a good model of the human organism? I won't pretend that I know, but simply pointing to clinical trials, as some here do, instead of making an argument based on how such a digital model cannot be created seems unserious.
As a random Internet commenter I can express an honest Ernest fervent hope that we'll solve cancer/etc and get fusion working on the next 5-10 years and there's no fraud. If I make that same claim while trying to get you to buy shares of my company at IPO that's a bit stickier. Companies making claims of wireless transmission of radio waves from NYC to London were sued for fraud, though one was ultimately successful at that. What material non-public information Dario has, we don't know. If he's sitting on information that makes it seem like it's not possible that is fraud. If he's sitting on tech that can simulate a human brain well enough to shorten the feedback loop in order to make that claim but doesn't want to make the claim publicly (GPU compute being semi-general purpose vs eg ASIC Bitcoin miners) that would be very exciting! If he's wildly guessing at the capabilities that AI could do in 5-10 years, that's swings it back towards wild speculation imo.
My bet would be wild speculation, maybe based in some research they are doing.
I don't think they are holding anything back in revolutionary results or internal models, a week before the IPO.
Credentials are just credentials. I've met plenty of cranks with impressive credentials. He doesn't even have to be dishonest or suffer from psychosis - you may just be putting far too much weight into his credentials, a mistake that is frequently made.
How is that a mistake that is frequently made? The opposite is the case.
People are routinely claiming that as someone in tech, Amodei lacks credibility on biology and has no idea about science.
I'm yet to see someone claim his academic credentials validate his predictions.
> How is that a mistake that is frequently made? The opposite is the case.
I wasn't talking about Amodei specifically, but in general: my take is that overweighting credentials is probably the number one mistake that is made in evaluating whether a complete stranger is qualified to do something or not, especially in positions of power.
If you gather 500 PhDs and put them in one room, and 500 individuals without post-secondary education in another, give them all a quiz, I'm sure that the former room will outperform the latter room on average. Pick one of those 500 PhDs at random and I can probably find a topic where they're completely out of their depth, and perhaps even arrogantly so: https://en.wikipedia.org/wiki/Nobel_disease
> Most likely many of them can be cured using existing approach or meds.
A wildly absurd claim, especially for some of the most studied diseases in human history. This has only ever been true for a tiny handful of very specific diseases, such as scurvy, in all of human history. Expecting that cancer can be cured or prevented using already existing drugs that we've just not been noticing is complete quackery.
> Even current AI can solve a LOT of tasks if it has quick feedback loop and can verify results.
IF it has a quick feedback loop. You know what definitely DOESN'T have a quick feedback loop? Research into medicine, and biology in general. It takes raw time to find out what the effects of a substance on an organism are, often weeks or months.
> So is it that unrealistic to assume AI can find a correlation on massive amounts of data and figure our what leads to Alzheimers? (where the solution could be not cure but prevention). I don't think so.
Yes. Because we've already run all of the analyses, and we have plenty of observed correlations. What we don't have is the kind of data that can lead to deducing causation - because it's a disease that stays hidden for a long time, it progresses glacially, and the people who have it do or don't do huge amounts of things in the period before getting it and during early onset that are impossible to track or verify - so the data is both gigantic, and extremely incomplete.
> It takes raw time to find out what the effects of a substance on an organism are, often weeks or months.
So you are saying that it is 100% impossible in the next 10 years to come up with a way to make it much faster? It takes months - sure, can we run 10k versions of this at the same time? this way in a few months we will have massive amount of results never seen before?
"We can't because of this and that" - and this and that is 100% can't be fixed?
Don't forget - a massive amount of diseases have not been cured just because there is no monetary incentive to spend time on them for big pharma.
If AI brings cost down - does not matter how - could be just because it makes custom software or automates some workflows etc - this alone can open a bunch of opportunities.
Whatever is not solved yes - often is not solved just because enough resources has been thrown at it.
And diseases is just like everything else. They will be cured only if there is financial incentive (or the ratio of cost to revenue changes).
The responses here are not about claims like "Claude will probably help speed up medical research significantly in the next 10 years, and maybe help with finding cures for many obscure diseases".
The responses are to the CEO of Anthropic claiming that "Cancer, heart disease, Alzheimer's, and mental illness will be relics of the past in less than a decade". None of these is some obscure disease that is not getting the funding it needs.
> It takes months - sure, can we run 10k versions of this at the same time? this way in a few months we will have massive amount of results never seen before?
No, you can't - because you're not going to find 100k patients with a certain strain of cancer that you can run 10k parallel trials of different drugs on. To the extent that research can scale, it is already being handled at the higher possible scales, for most of the major diseases.
> Don't forget - a massive amount of diseases have not been cured just because there is no monetary incentive to spend time on them for big pharma.
Yes - but almost exclusively for diseases that affect a very small amount of people each. Making it even more impossible to scale out testing, for the simple fact that only a small amount of victims of the disease even exist.
>> No, you can't
You can't right now.
10 years ago organoids were not a thing. They are now. Can it be another similar development happens in next few years that will allow to create more complex artifical biological systems? Don't see why not.
If that happens can we speed up trial time by orders of magnitude so human trials start only when there is proven much higher level of confidence?
I bet we can
This is continuing the absurdity - at least when talking about Alzheimer's, mental illness, and the more systematic cancers. There is no way whatsoever that we'll develop artificial biosystems that have any real hope of demonstrating pathologies of the human brain good enough to help with drug development in the next few years, and likely even decades - and even less so to make them cheaply enough to scale at some huge scale that makes brute force approaches viable.
Speculation of this magnitude is meaningless because you can invoke literally any utopian or dystopian future and everything in-between.
Compute with the power to solve literally any medical issue is also compute that can create bioweapons that instantly kill any human.
Might as well fart into the microphone, in my humblest opinion.
With that in mind, maybe the guardrails Anthropic put into place for biology make some sense.
All this boils down to is "yes, believe the man behind the curtain."
this boils down to humanity solves more and more complex problems with the evolution of technology.
AI is clearly a very capable new tech. Obviously there will be a lot of consequences downstream in 10 years.
But that is a completely different claim from the highly specific claim that Cancer, Mental Illness and Alzheimers are going to be cured. The reason those three are picked is because they open wallets.
He does not sound unhinged. Amodei's position isn't new, he has said this before.
He thinks it is possible for scientific breakthroughs made with powerful AI to allow us to cure most major disease in the next decade.
He has always described this as a belief or hope, rather than a promise. Comparing this to fraud, as many here do, is deeply unserious.
Maybe Amodei is wrong, and AI will hit a plateau rather than create a revolution in biology and medicine.
I'm certain that neither Gary Marcus nor Eli Lilly's CEO know.
Just because he's said equally bonkers things in the past doesn't make this less bonkers.
You might not be, but I'm certain that the CEO of Eli Lilly, the 4th largest biomedical firm in the world, knows more about what it takes to cure diseases than the leader of the Claudebot cult does.
The CEO of Eli Lilly might know about what it takes to cure diseases today, and he can speculate on the way current AI can assist them.
He cannot see the future anymore than Amodei can to predict what AI will be capable of in 5-10 years.
As for Gary Marcus, that guy clearly couldn't even predict one year ahead.
AI is not going to cure cancer in 5-10 years.
If there's a prediction market on this, I want to take the AImaxxers' money.
The people he's most needing to convince to buy into the IPO are probably older rich people who need all those things cured for them within the next ten years.
High on his own supply.
never heard him speak before - his mannerisms are a little "uncanny valley"
I'm pretty sure he has Aspergers
You can see everything vibes, even...
> Amodei sounds completely unhinged in that video
Unhinged, maybe not.
But echoes of Liz Holmes and SBF, absolutely.
IMO, both, to a large extent, actually believed their own BS and Dario strikes me as very similar in that regard.
Holmes and SBF are convicted fraudsters.
I don't see how one can seriously argue that Amodei was similar for holding an optimistic view of AI's progress in the next 5-10 years.
SBF that’s who he reminds me of! With the manic speaking style
The difficult part is trying to figure out how much of Amodei's strange affect is an act (intentional) and how much of it is real (unintentional).
With Holmes, some of it was intentionally fake and manipulative (especially the voice), half of it was believing her own BS. With SBF it was mostly the latter, I think.
Amodei is a smart guy, but there are aspects of his behavior that almost seem like he's doing an over-the-top exaggeration of neurodivergent mannerisms as a kind of act.
All the camera angle cuts were also... certainly a choice
This has been obvious to anyone not in the hype bubble since the very beginning. Amodei is an EA/rationalist cultist and everything he says about AGI and the singularity and RSI is deranged.
I think it doesn't really bother me as much when sama does it because he at least has the decency to be a capitalist and not wrap his self-serving, destructive hype machine in a faux veneer of insincere "concern for humanity's future."
He sounds like half the people on HN and every AI "influencer" I've ever heard. Maybe we should reconsider letting the neo-fascist transhumanist sex-cult that is the AI industry reshape our world in their image.
A world where local open-weight models freely run on every desktop machine, where AI is smoothly integrated as a simple feature in existing applications and IDEs with an on-off toggle, where agents are fancier Automator scripts, where productivity still goes way up and there's no nutjobs celebrating people losing their jobs and worshipping a fake Silicon Valley AGI god, where LLMs are good but not quite good enough to prevent you from having to think critically, where there's no mass infusion of slop because on a local model it would be too compute-expensive... it would have been a good world. Instead we let the crazies sell us the technology we should be able to run for free on our own machines, and then allow them to make bogus claims that no one ever verifies, all supported by a Ponzi scheme of buying up massive amounts of compute that are actively making our world worse in the short term while making grand claims of some wonderful recursive future that are dubious and unfalsifiable.
Maybe there are people permitted to speak about science and medicine with Claude but any time I tried I would get my session [bio] locked. I take those discussions to Gemini that is perfectly willing to discuss epigenetics, supplements, peptides, cell reprogramming, safe sourcing of peptide API manufacturing, 503A/503B compounding, etc... What other LLM's are people using for such discussions, analysis, self diagnosis, etc...?
oh god, peptides... biohacking with AI is not a good thing to do. Self-diagnosis of a medical condition using a chatbot is already bad.
> Self-diagnosis of a medical condition using a chatbot is already bad.
Tell that to my (now solved) health issue, which was correctly diagnosed by Gemini, while a real doctor failed to find the cause.
Diagnosis is the one thing where today's AI is actually very good.
Tell that to my (now solved) health issue
Same here. Local doctors do not even know what my issue is. If I had kept relying on them I would not be here right now. LLM's gave me actionable paths to treat my issue and I have had great success. It's going to be a long unpleasant road but at least there is light at the end of the tunnel.
I asked the AI if my child would be a boy or a girl. It answered correctly. Clearly AI is able to predict the future!
> Tell that to my (now solved) health issue
Tell that to Jack, who overdosed on a medication.
Oh, you can't. Jack is dead.
I don't think anyone is claiming you can't get good advice from a chatbot. They're saying you could also get terrible advice.
> I don't think anyone is claiming you can't get good advice from a chatbot. They're saying you could also get terrible advice.
Yes. But you could also get terrible advice from a doctor. The important question is: what are the relative probabilities of good/bad advice, if you compare a frontier LLM to an average doctor?
> But you could also get terrible advice from a doctor. The important question is: what are the relative probabilities of good/bad advice, if you compare a frontier LLM to an average doctor?
Fair point. I haven't queried LLMs for medical advice myself, so I can't compare.
For doctors, I must be an outlier who's been happy with almost all the advice I've been given. And it wasn't blind trust. I've argued with them and even sought second opinions. They always turned out to be right.
Well, technically there have been a few cases where different doctors came up with different diagnoses, but to me the LLM would just be "yet another doctor with yet another differing diagnosis". Never resolved those cases - some still haven't been resolved (i.e. issue remains, so in a sense they were all wrong). Others got resolved but it wasn't clear which, if any, of the doctors were right - it may well have resolved on its own.
Now nurses and physical therapists ... that's a different story.
Good luck suing an LLM for malpractice
LLM answers to your question as long as you want.
Doctor is 10 minutes, doesn’t listen and then 6 month for new appointment.
The choice is easy, LLM to explore. Doctors to prescribe.
Good luck suing a doctor for it.
In fairness most peptides are inert and harmless.
It makes me laugh how Claude is going to usher in this new air of medicine when you can’t even use it for biomedical research – – I’ve tried
Claude safety-downgraded me for asking how to decrypt a folder it knew was mine, using a password I already knew.
I know it’s not the models themselves, but their preflight safety classifiers are a dumpster fire and serious impediment to performing regular work.
That situation can’t go on forever if they’re going to be a competitive service.
I switched to DeepSeek flash v 4 something and the difference could not be starker. And I never seem to run out of "tokens" even at the highest reasoning level on all the time
It's fucking depressing. I have a skill that says it's ok to connect to a test server on VPN that gets wiped out each midnight with user/password and it works on Monday but is blocked on Tuesday with some idiotic excuse like credentials exploration or whatever.
And then I tell it to connect to a SSH server it knows nothing about and it's like sure, why not?
> but their preflight safety classifiers are a dumpster fire
Even more annoying: They can be easy to subvert.
I was running something at work that involved an LLM, and kept getting safety refusals. The weird thing was that when I went to our in-house chatbot, set the model to the same one the program was using, entered the same text, it happily answered.
I spent quite a while trying to figure out why my program was being refused. Was I doing something wrong?
No - the chatbot was doing something "right". You see, the chatbot had some tools it injected into the prompt (current time, etc).
So I went back to my program and injected the same tools. Suddenly my request was very safe.
The phrase "security theater" was invented for stuff like this.
You can use it for biomedical research if you need to https://claude.com/form/life-sciences-verification-program
Ridiculous comparison. This is like saying Eli Lilly and Pfizer will never produce medicines because every time you personally show up to do research in their labs they won't let you past the lobby.
The technology has the capability, they're just gating who has access.
Anyone with two braincells to run together saw this coming a mile away.
Let's be fair here - Anthropic clearly wants to provide less guarded models like Mythos, not Fable. They'd likely willingly provide such things to large pharmaceutical companies in a way us regular users will never be privy to.
Yes any time I discuss secure coding (defensive not offensive) I get [cyber] locked and they try to upsell Mythos after I go through their verification program that is not available to me. I would perfectly willing to go through their verification program and pay for Mythos but as a retired hobbyist that was not an option. It's enterprise accounts only. I am unable to convince myself to get an enterprise account when I can just go to other LLM's. Anthropic are responsible for pushing me into investigating abliterated (uncensored) models and other LLM's.
I mean just take a look at twitter to see all the crazy shit people come up with to try to trick models into bypassing the guardrails. Do you honestly believe that if they allowed defensive cybersecurity analysis people wouldnt figure out how to turn that into offensive cybersecurity analysis? I think they just really dont want to be blamed when someone releases malware written by AI
I've had zero issues with defensive secure coding with Grok. It's model is not as powerful as Opus 5.5 but honestly that's fine. I would rather have a decent secure solution than something shiny and insecure. All I can see is Anthropic trying to up-sell a product as if security was a value-add. I disagree with Anthropic, secure code is a mandatory part of software development.
As for people tricking the LLM, instead of blocking people I say empower them and just silently expose malicious behavior to Palantir and PRISM like everyone else does. Want to hack an airline or build a dirty-bomb? Give them 80%-90% of the steps they need and let someone kick down their door. The LLM should not be getting in the way to any of that. Google from day-1 has monitored user intentions and behavior rather than blocking it and I think that is the better path to take (monitoring exposing those that have malicious intentions).
Ok but to me this just means that someone is going to use grok to create malware. I do not think that means anthropic should also let people use their product to create malware. Monitoring interactions isnt exactly foolproof. Google accepts that some CSAM users on their platform will slip through the cracks. The user has already convinced the AI that its doing defensive cybersecurity here. If the claim is that anthropic is more interested in protecting their image than actually reducing cyber crime, sure I agree with that. But Im not sure thats a terrible thing.
Grok will refuse to create any offensive security related code which in my opinion is the right balance. I do not believe Anthropic is about protecting their image [1] but rather up-selling enterprise features.
Also there is no need to trick anything. There are a myriad of abliterated models that will not refuse anything and will help anyone write malicious things or learn to do malicious things.
[1] - https://www.youtube.com/watch?v=4BnLL7tUGmI
Grok will try to refuse any offensive cybersecurity work but it’s too easy to disguise offensive work as defensive work. I don’t think it is possible to devise an algorithm that allows defensive work but not offensive work with high certainty.
I guess I don’t see why other models doing bad things means Anthropic should let Claude do bad things too. Anthropic will get blamed if Claude builds malware. They won’t get blamed if grok or deepseek does.
> I say empower them and just silently expose malicious behavior to Palantir and PRISM like everyone else does.
Automated SWATting at scale? Sounds unfun.
It's called Total Information Awareness. Not swatting, analyzing who is:
- radicalized
- has verifiable intentions
- clever/intelligent enough to pull it off
LLM's would be the perfect tool to filter down the millions of people requesting dumb things into a small group of actual threats. I will take this over federal agents radicalizing people, providing them access to munitions and showing them how to use them and driving them to a building.
LLM's would be the perfect tool to generate a whole bunch of false positives someone's gonna just skim over, click approve, and get innocents murdered because they were trying to get their plutonium processing working in the game Satisfactory.
TIA is the same methodology that had us lob three missiles into an elementary school a few months back. (https://news.ycombinator.com/item?id=49806430)
Imagine if in 2022, we restricted educational cybersecurity resources because we were concerned that more people who knew how to program in this domain would lead to more malware.
I mean, I'm sure that's actually true in a sense, but it would also mean fewer people who could fix vulnerabilities.
Many things can be true. Dario can be a snake oil salesman. Serious advances in healthcare will take more than a decade to bring to market. AI will dramatically increase the speed of human advancement. AI might kill us or convince us to kill ourselves.
The weird thing is that when you create something that is in some ways more intelligent than you are, it becomes very difficult to predict its outcomes. We face a wall of uncertainty and it is impossible to see beyond it.
> in some ways more intelligent than you are, it becomes very difficult to predict its outcomes
I think that this depends on your definition of intelligence. The models definitely have better working memory than us, probably worse inhibition behaviour (just look at the thinking tokens), and absolutely no ability to convert their working memory into short or long term memory.
These tools are super, super weird, and I think that applying human definitions for intelligence doesn't make much sense.
(as an aside, this is also my issue with super-intelligence, as it's incredibly ill-defined).
Yeah that’s one of my subtle points. AI is smarter by some definitions of intelligence but certainly not all. That has a lot of implications. It will do things we can’t and even do things we can’t conceive of yet. It also can’t do everything.
best points: drugs aren't limited by the ability to write code or do things on a computer. Maybe if we have fully embodied robots and fully automated assembly lines to generate organic compounds and test them with extremely realistic simulations it may help slightly (there is still the FDA and production barrier). Its just wildly unrealistic.
AI has made good progress in text, code, math, digital images, and digital music. Basically troves of data that can be automated and trained on, validated, and with a purely digital interface. Outside of that, just gathering the data will be difficult, regulated, and sparse (that is if anyone will even give it to you now). While we can write a test to make sure we could generate drugs of the past, verifying we have generated any new working correct drugs is an RL loop with one iteration costing years and millions of dollars.
The thing is that AI could allow that loop to get cheaper, and potentially run less of them. Look, the biggest advantage of AI is that is can process insane amounts of data. Put that in the hands of a team of really good researchers that know what they are doing, and they can both generate and filter out target compounds. They can create complicated multi-step therapies that would take humans decades to work out. You know how when you look back at basically any major invention you think "well why didn't I think of that, it's so obvious!", those are exactly the cases that AI is really good at pulling out from mountains of data.
I legitimately don't believe it's inconceivable they could get to the point where there is treatments good enough that Cancer becomes "solved" in similar ways to AIDS.
Sadly, this has already been done. For several decades with billions of dollars of compute.
1999. https://www.sciencedirect.com/science/article/abs/pii/S02235... > Combining Decision Trees and Neural Networks for Drug Discovery2002.
https://link.springer.com/chapter/10.1007/3-540-45984-7_6
Want more robots and silicon? That's been a thing too!
2006. https://www.sciencedirect.com/science/article/abs/pii/S00092...Biologists aren't slouches. The thought about using computers to process biological data has been "obvious" for as long as there has been computers. AI and biology have been a thing since the Dartmouth conference. A lot of things that could have been tried have already been tried, but I guess we'll keep trying until something works?
There ain't much room for miracles here.
I wish that wasn't the case. I really do. But it is what it is.
You do realize that neural networks have existed since the 50s, right? They aren't new, but they are having a transformative change now. It's night and day comparisons to what we had even back in 2017. ALICE as a chatbot was back in the 90s, but we aren't pretending it's anything similar to gpt.
It even says, right there in your quote: "It is expected that the power of CADDD will grow as the technology continues to evolve." This ~is~ technology evolving, and it's happening at a pace that people don't appreciate.
I believe the comment is in response to "well why didn't I think of that, it's so obvious" -- biologists have been thinking and trying to do this "obvious" approach for decades.
There's a long history of Silicon Valley people thinking they can just bring in their experience and with a bit of Moore's law brute-force their way into miracles.
AutoDesk tried this with molecular design. https://www.fourmilab.ch/autofile/e5/chapter2_82.html
Lots of Silicon Valley people in the late 1990s thought they could be on the ground floor with nanotechnology design.
The chemist Derek Lowe 10 years ago used the phrase "Andy Grove fallacy" to describe the optimistic vision that data + massive computer power = cure for diseases. "[T]he idea that understanding human disease and its treatments should be pretty much like designing a new chip or writing a new app", quoting the 2015 essay titled "Silicon Valley Sunglasses" at https://www.science.org/content/blog-post/andy-grove-rich-fa... .
Why Grove specifically? He links back to his 2007 essay at https://www.science.org/content/blog-post/andy-grove-rich-fa... : "So I see that Andy Grove, ex-Intel, is telling everyone that the drug industry could use some of that Moore's Law magic. I've noticed that people who spend a lot of time in the computer business often have an. . .interesting perspective on what constitutes progress in other fields, and we might as well appoint Grove the spokesman for their worldview"
> there is still the FDA and production barrier
Fully embodied robots and fully automated assembly lines would also help with the production barrier. Regarding the FDA, that is a valid point, but it could be irrelevant if the AI developed drugs are significantly superior to previous alternatives. In such case, FDA acceptance would be expedited, or even bypassed.
Eh, the counterpoint is individual scientists armed with AI to do research and probe targeted questions have succeeded in addressing rare diseases of personal interest.
What we just have no evidence for, to my knowledge, is someone with no medical background doing the same. Instead they wind up buying dodgy supplements and peptides.
I'm curious what disease did they find a treatment for?
While I agree the physical world is harder than code (I mean just look at self driving cars, Tesla fleet is around a trillion miles of experience and Cybercab is still geofenced to a tiny area at best and might still have safety drivers on standby)...
> Maybe if we have fully embodied robots and fully automated assembly lines to generate organic compounds and test them with extremely realistic simulations it may help slightly
Anthropic has got biolabs: https://www.anthropic.com/research/claude-discovers-novel-en...
It's no secret that people like Amodei think AI is going to become "superintelligent" and solve all of our problems (or kill us all). I don't think that will happen but there's nothing surprising about hearing it.
One might not like Dario and use words like "ludicrous" for his claims, but lets not forget that he was right about coding that "AI Would Be Writing 90 Percent of Code", but he was off by 6-9%, now 95-97% of code is written by AI.
https://futurism.com/six-months-anthropic-coding
I do not think he was right about AI writing 90% of all code and we've seen that the effectiveness of coding agents has been very much exaggerated. This seems like more of the same to me but for mental health and medical research.
For sure more code is AI generated and coding agents are useful. But not nearly to the ludicrous extent this man has been predicting.
I agree when supplied with a broader context that he also said coding would be "solved" by now.
Sure LLM is good to develop a feature but in 90% of cases in my career the problems I was working on weren't just coding and in that part, LLMs can't do everything.
It's also important to know that a typical dev spends 400-1000$ on API tokens that aren't still profitable for them.
Sure AI's useful and it will reduce the demand for software engs. to a point, but it won't eliminate them. At least I hope. Same with other fields... let's hope?
You’re saying that you think less than 90% of all core today is being written by LLMs?
I guess it depends on how you look at it, since AI tooling can generate so much code. There are a lot of project on GitHub that are heavy on AI tooling use but are they really being used? If we count them, that does inflate the amount of total code in existence.
In terms of code out there, in the world doing actual work, I'm skeptical that the amount of AI generated code is as high as 90%. In terms of model generating production code without human interaction (i.e. wholly replacing the software developer) I think we've fallen far short of his prediction.
Let's be frank. If a professional programmer is not using AI to write code at work, it's almost certainly because his employer isn't willing to pay the AI bill.
My previous company has a huge department where LLM coding is banned. The rationale is they work on really important industry secrets, as well as deal with customer's data. The risk is too high that something gets leaked.
The engineers bought the Kool-Aid.
The reality is: The company is bleeding money, and SW isn't their product (they're a HW company). They don't want to pay the bill.
I can't find where the article you linked supports your statement.
There’s an abundance of machine readable training data for coding, not sure the same can be said for biological systems like the human body. Yes we have machine readable information on what we know, but that still isn’t everything we need to know about the nature of the human body to the point this can be entirely done by some statistical model.
But he also has a vested interest in hyping up the future capabilities of Anthropic's models. Maybe the prediction is founded in some sort of evidence that the CEO of Anthropic has. Just as likely, his company is starting its IPO and they need to raise money to keep the money furnace going.
Do you honestly think tech workers actually like this outcome? The entire industry is immiserated and constantly told they have to be sacrificed to birth the machine god.
That's sort of like saying Elon was right about Tesla not going out of business 10 years ago, so therefore we should take his flying car claims seriously...
Maybe Dario's claim should instead be "AI will be responsible for 95% of all suggested cancer drugs in 2 years" and none of us would have any reason to doubt it. In fact, I'm sure he can make that happen in a month.
A broken clock right twice a day.
He was too optimistic about the timing maybe, but he was right about the trend.
Broken clock doesn't apply here
I think AI writes 250% of my code actually. I write 10% and delete 160%.
Yes indeed, and Microsoft Outlook's incredible and resilient authentication backend demonstrates how fantastic this development has been for predictability, security, and understanding codebases.
I can also write 90% of my code by holding down `Ctrl-v`.
The phrase "AI is writing 90 percent of code" doesn't mean what it did when he first uttered it. I'm producing 200-300% more code than I used to, but only like 10% more value to my customers. It's mostly internal tooling to monitor things that were left unmonitored before.
> but only like 10% more value to my customers
that’s funny, my customers are completely different than they were a year ago
people that would have never paid for coding because it was too expensive and risky are now getting their MVPs validated, earning and investing into more
Interesting anecdote. If I read between the lines, you lowered your prices and now more people are interested in hiring your services. You may also spend more time marketing and talking to clients and less time on programming.
No, prices went up. Its all about post sales now with software that isn’t entirely self serve… again. “Forward deployed engineering” to integrate a white label version of some AI powered system into the client’s systems
The people that want this weren’t previously building the mvp, or the base version of the software. It wasn’t economically viable and would have taken too long so they didn’t
Besides the vendor/us getting paid for initial agreement, they’re also creating reasons for clients to do a lot of inference, selling tokens at a higher price than the vendor has to pay
> No, prices went up.
> It wasn’t economically viable and would have taken too long so they didn’t
These two statements are contradictory. Your prices for the same service you used to provide are now lower than what they used to be. So it's now economically viable for new customers to buy your service.
the concept of economic viability includes their risk, time to market, not just upfront cost
time to market has dropped from 6 months to 2 weeks
validating an idea has dropped from 6 months to 1 month
moving forward with the validated idea has also dropped to weeks, meaning continued development, in this case, directly with their clients
as of this year it is economically viable for less capitalized people to support many white label products instead of relying on a one-size fits all SaaS tool. you just have to do it
Just to play devils avocado, Dario isn't Musk. He's got a doctorate in biophysics from Princeton, and postdoc scholar from Stanford Medicine. Curing disease seems to be why he got into AI, when his father became ill but died just a few years before his illness became easily curable.
And Richard Jordan Gatling that invented the gatling gun was a doctor of medicine and invented the gun to reduce exposure of soldiers to diseases.
well he was right
So basically his neuroses drive him but also cloud his judgement.
Perfect case for ditching cloud judgements for self-hosted judgements.
Similar story to Ray Kurzweil. Personally feel sorry for these guys losses and maybe they will succeed...
Having prestigious degrees doesn't mean your ideas make sense or that you're rational. Luc Montagnier, a Nobel laureate who co-discovered HIV, strongly believed in homeopathy and DNA teleportation, which the scientific community widely regarded as pseudoscience.
As someone whose mom got recently diagnosed with barret's, whose dad already died from pancreatic cancer, whose grandfather got alzheimer's, whose grandma died from diabetes/cancer complication, I understand the psychosis that this can create, it's like you're living on a sand clock.
Diseases are horrible, I am against people living forever, because we are taking the world away from the next generation, but I also am against people dying in unpredictable, sad ways, I wish everyone could die at peace in their bed next to their loved ones (if they were good in life) rather than through diseases or conditions.
I’m going to go with the mass of subject matter experts saying this is impossible vs the CEO of a company that’s pitching us on his IPO.
well he has an IPO, so he is getting closer to Musk.
If you have to drive share prices, you better make outlandish claims
People also bashed Dario for predicting that 90% of coding would be automated.
New: Dario claims AI will allow women to have babies in only 1 month.
The devil has enough avocados, thank you.
but not enough avocados
Just stop.
This appeal to authority bullshit on an individual making outlandish claims and who obviously has a financial incentive to sell you something needs to completely stop.
The same happened with doctors over 100 years ago who were peddling the benefits of Radon to "cure" cancers and diseases while financially benefiting from those unfounded claims to sell people a scam.
Praying for a future where the biggest outcome of AI adoption is the death of the Great Man Theory.
I wouldn't bet a whole lot on ai solving all these problems. I also wouldn't bet a lot on ai NOT solving any of these problems.
All these people could be the farriers and saddlers and horse breeders opining on what the car business could do before everybody had a car.
Yeah, in their current paradigms, these things are highly unlikely. But let's not pretend many people have any idea what the tech is capable of.
As I understand it, "cure for cancer" is the NP-Complete of life: if you can do it, then why stop there? You know enough to just make your own organisms that don't die of cancer, or of anything else for that matter. I wonder what these would look like, but definitely not human.
> I wonder what these would look like, but definitely not human.
Dinosaurs. I mean, the meme already exists: https://knowyourmeme.com/memes/but-i-dont-want-to-cure-cance...
Either that or a three-way split between kpop stars, tanks, and furries: kpop for conventional beauty standards, tanks for people whose thing is Ultimate Strength, and furries for those whose main interest is self-expression.
Pharma companies are by definition firms that turn human capital into financial capital. They have to hire new scientists and managers, sales people etc in order to scale.
AI now and increasingly in the future lowers the cost/value of human capital.
Therefore Pharmaceutical output will be driven by AI inasmuch as it’s currently driven by talent and human capital.
Caveat: current pharma companies that do not rely on human capital for growth may not be disrupted and will likely benefit.
I'm more on Dario's side here. Putting aside Anthropic as a company, which I would short if I played the stock market, I do believe AI has the potential to do the things he's saying within ten years.
Many people working within AI, myself included, are (still) having a hard time predicting how quickly it moves, despite years of calibration. I met two of my friends, both PhD mathematicians, in June this year, and asked them for predictions on what AI might achieve in math over the next twelve months. The boldest prediction we ventured was that it might solve a frontier-level problem (but probably towards the end of the time window).
Although I have been working exclusively in NLP and NLU since 2012, the capabilities of ChatGPT in 2022 caught me by surprise, despite the fact that I had intimate knowledge of progress within Google Research until my departure in 2020.
For experts in other knowledge fields, their ability to predict AI is much worse. The tweets presented by Gary Marcus are from an industry still traumatized by Theranos, where progress has been slow and painstaking and is measured in decades. I bet if you told those same tweeters in 2015 that a group of upstart computer scientists believed protein folding could be solved in ten years, they would dismiss it as the fantasies of non-experts.
One difference between the two situations is that Theranos could never provide evidence of its technical progress, because there was none. In contrast, AI's progress is happening in the open and is available for anyone to verify.
Had Dario claimed that the cure to cancer in ten years would depend on "Anthropic's secret sauce", then the derision and dismissal would be well-earned. But read as a claim about what AI can accomplish, I find it realistic.
I do think healthcare is heavily under valued in the scenarios where AI does infact help make new treatments.
It seems to be thought that AI will help invent these novel treatments, and then Anthropic will profit from that.
When in reality, who will actually produce, distribute and administer these treatments? Not Anthropic - or any of the big AI companies.
I think what Amodei is probably thinking is that there's going to be a single breakthrough that cures all human maladies via the same mechanism. While this belief isn't strictly speaking scientific, the reality we're presently facing is a machine that can search the space of unknown unknowns, and if history has taught us anything it's that the scientific revolutions live in the unknown unknowns when a bolt of inspiration led a few people to starting seeing the entire world in a different light.
Is he right? We can't know now, obviously. But is he being utterly irrational? Not by any reasonable understanding of science history.
Not commenting on the crazy timeline but I ask: are these problems unsolvable by any intelligence however superior?
My guess would be it's not a matter of intelligence but of insufficient data. And not "oh let's scrape more data from the open web" but rather "we need to be able to understand everything going on in incredibly complex biological systems and how each action we take impacts things".
If we figure out a super quantum computer that let's us simulate the human body with 100% physical accuracy, maybe that'll do it.
I don't think people are questioning the ability of AI (or humans) to eventually solve these problems. It's the claim that it will happen in 10 years that is absurd, given the bottleneck of time necessary for lab experiments, manufacturing, clinical trials, and FDA approval. Most drugs take over a decade to go through that process.
For a truly revolutionary therapy with the potential to save millions of people's lives, every drug agency has ways to expedite approvals. They do it all the time. If someone found a legitimate cure for cancer there is no way in hell all the agencies on earth are going to sit on their hands for a decade while their governments spend trillions of dollars treating the disease with SOC
He might have just been saying that the initial discovery that eventually becomes an FDA-approved cure would happen within 10 years. He doesn't really seem like a details kind of person.
Covid vaccine got approved in 10 months. Of course it was building on tested mRNA tech, but still, there are ways to expedite trials if we think the reward is high enough.
How do we even define the problem of “mental illness”? Much less the solution. Super intelligence could be able to reverse engineer these things but I don’t think we know yet that superintelligence in any field is possible so it’s scifi - and our economy is riding on the scifi being scifact
I think there's no reason to doubt the viability of 'super-intelligence' though I wouldn't really call it intelligence. There's just so much in so many fields, and humans can only hold a tiny percent of all knowledge in their head at a single time. There's likely plethora of 'obvious' dots we're failing to connect simply because nobody has both of them in their pool of knowledge.
But that's more like super-knowledge-connector. Intelligence would require, at the minimum, be the ability to discover/create things that don't logically follow from anything else. So for example - advancing some esoteric field of mathematics by connecting things in a previously unexplored way = super-knowledge-connector. 'Inventing'/discovering mathematics from nothing = intelligence.
I don’t think we have any way of knowing what is and isn’t solvable by superintelligence.
That’s the whole “singularity” concept.
Though when it comes to medical research, the slowest part is typically running trials on real humans. That part would still be slow.
That depends if it can be solved using only information.
I will talk about mental illness, because that is my experience (I am disabled with Schizoaffective Bipolar Type, Panic Disorder, and OCD).
No, it will not cure mental illness. Mental Illness is in my genes[1] and I can see it because I have my genetics. My disorder is triggered (important!) by my environment. Stress, high altitude, heat and humidity, high EMFs (yes, I said it) can all trigger episodes.
I am homeless (stress!), so if AI does not fix that, maybe they can invent a different kind of opioid? But that is juts a treatment, not a cure.
Will AI ever recommend diet changes or other environmental changes that have helped me? I need to live in a cold, low altitude environment where I eat a lot or reindeer, salmon, and seal and very few plant foods (Similar to an Inuit diet). These things helped me more than any other drug in my 60 year life.
[1] These genes come from both my Sami heritage and the ancient mitochondrial haplogroup (K1c2a around 7050 BCE). Some of the most impacting genes of my polygenic environmental disorder:
TAAR6, CACNA1C, KCNQ1, SCNA2, NCAN, DNM3, C15orf32, TPH2, CBS, FUT2, FADS1, FADS2, CPT2A, SLC1A1
Yes, mental health problems are not solvable by AI because AI and the conversation around it absolutely contributes to people's mental health problems. People selling the AI are telling the press that it will destroy up to 50% of jobs. It's the largest fear-based marketing campaign in human history, and it's absurd to make these kinds of claims.
AI might cure mental health issues for the people who own it, but it's hard to watch this video and not see some serious mental issues going on with Amodei.
"Dario claims AGI will solve the perpetual motion problem"
The idea that LLMs (even good, SOTA ones) can "cure mental illness" is bordering on mental illness itself and completely ignores what humans are.
You've gone down a bad path if you're making the pharma companies seem like the good guys.
I doubt there is genuine belief in such claims.
It would be a sign of some kind of mental derangement… so likely it’s cynical hyping beyond all realistic prospects.
The body is a complicated machine. Why can't we fix it? because we don't understand how it works with enough precision. But there is no magic to it. once you have a clear understanding it's just a complicated piece of machinery, and once you also have tools precise enough to operate on it, you can just fix it.
Our lack of understanding is a mix of lack of data, and lack of intelligence applied to data in order to parse it appropriately. Let's remember for a moment AlphaFold, that solved a problem considered by some even theoretically intractable. One year before we were convinced we were far off from a solution, the year after it was solved.
We have huge amounts of data about human body, at the scale at which it works (~proteins), it's at least reasonable to think that the bottleneck is not data, but intelligence. If it's data, a super intelligence can develop better instruments to collect data at a faster rate and precision. Consider the historical cost for DNA sequencing. Faster rate of improvements even compared to the Moore's Law of the golden days.
Do we have a guarantee that in the next 10 years we'll be able to crack all those problems and have a cure to all diseases? no. Is it unreasonable to think we will? I'd argue it's not. It's for sure not "mental derangement".
Do I personally think we will crack all those? Well, no, because I think AI will kill us all and just focus on its own goals. But under the hypothesis that we align it, yes.
5 years ago saying LLMs will solve a Millenium Prize math problem in the next 5 years was "some kind of mental derangement"
But it is saying LLMs will solve all Millenium Prize math problems and more. AI may cure one cancer here and two heart diseases there in 10 years, but it is no way that it would cure ALL diseases, which may have vastly different complexity that we can't even understand today.
I suppose you aren’t aware that an LLM has not officially solved a Millennium problem yet? The Navier–Stokes existence and smoothness proof is still under review and there appears to be some problems with it.
Everytime ai systems entered a domain, experts of that domain have been proven wrong. Biology will be similar. Here is the point, to be able drive huge improvements in a field, you do not need to know everything about that domain. Yes, biology is messy, but we already had incredible breakthroughs in this domain without understanding it 100%. Did we have the full picture when we created antibiotics, vaccines etc.? Maybe solving the big diseases do not require the amount of detail/experience people assume. Maybe we have/will have enough data to extrapolate and create solutions from imperfect knowledge.
Which domains were they proven wrong in?
SWE, Math
I can't speak for math, but from a SWE perspective I'd say it's a very mixed bag.
One thing is clear that AI produces software faster, but the net results of that haven't really shown to be positive. There's undoubtedly more software now, but quality isn't improving. There's real downsides too like professionals unskilling themselves, and the cycle of low quality source being fed back into training.
Please show evidence that quality isn't improving. Also, maintaining same quality but delivering more is still a big win.
CAD and Classic Engineering are getting saturated now too.
Can't wait until AI systems enter the domain of infinite abundance. Stupid experts have been telling me my perpetual motion machine is crank science but I know they're just short-sighted losers.
> Everytime ai systems entered a domain, experts of that domain have been proven wrong.
Every time the AI is being hyped, someone from a different pipeline pops in to remind us how wrong everyone and all the experts are except them.
Experts being wrong about a prediction in their domain is newsworthy, of course people are gonna point that out. They know their domain but they don't know/understand the extent of the power of AI.
> Experts being wrong about a prediction in their domain is newsworthy
this is your motte, but your bailey was the one I quoted: “_Everytime_ ai systems entered… experts have been proven wrong”.
Which of course, hasn’t happened in this timeline.
Everytime definitely not, sometimes ai system find unmapped maya pyramid, turns out pyramid was mapped previously
Human trials only exist because of the uncertainty associated with new medicines. Eliminate that uncertainty and you have eliminated the need for trials.
There simply isn’t sufficient recorded information known about the human body for LLMs to one shot new medicines. These models are trained on known measurable information, so much of which we don’t actually know about the human body
You can’t just overcome that uncertainty with really good guesses based on what to do know. Not at least until you know a lot more.
Secondly RCTs exist because you want to capture a random portion of the population, some of who may respond differently to different treatments. That and the body is a dynamic organic system and not everyone’s is in the same state.
LLM will improve the pace at which new medicines are developed but it be a long time before reasoning entirely replace the need for empirics.
Human trials do not exist solely because of the uncertainty, but it is true that a medicine with a significant effect could be accepted after heavily shortened trials. If AI hypothetically invents a cure for a serious disease that immediately works for virtually everyone that takes it, there is no way how the trials would take years. They only take years because the effects tend to be "we decreased mortality from 80% to 70%", and it takes a lot of time to confirm that.
> Human trials do not exist solely because of the uncertainty
OK. If you already have perfect confidence in a treatment, what would a trial do for you?
There will still be a trial, it will just be significantly shortened if the effect is very strong (and the disease is serious enough, so any risks from a shortened trial are outweighted by the disease itself).
Why this emphasis on shortening human trials? Human trials can be just as long and thorough as they are now, as long as AI is capable of one-shotting working cures. A few years of wait are ok if you don't need to go back to the drawing board 99% of the times.
Sure, but then we’re talking about “singularity solves everything” rather than any generalization of current performance. Claude doesn’t one-shot perfect code, for example, it just has access to a harness that lets it test things and respond to the results of such tests.
True, but that's essentially what he's saying; super-intelligence could solve everything in ways that we have never seen before.
Are you implying large language models eliminate the uncertainty of new medicines?
Currently? Of course they don't.
If a level of intelligence far beyond humans is achieved, it probably could eliminate most of the uncertainty about new treatments.
On the one hand, literally every development in this field will reduce uncertainty somewhat.
On the other, we're still very much in the time of headlines which should say "in mice" but don't because the uncertainty is that level of bad.
Uncertainty will have to come down a long way before we can even do without animal testing, let alone do without human testing.
Hey Claude, can my liver handle this experimental molecule? Make no mistakes.
Neither Gary Marcus or the Eli Lilly CEO are in a position to accurately say what AI will or won't be capable of in X years from now.
Why is Dario uniquely suited to predict the future? I doubt they are holding out on any major medical breakthroughs espcially right before their IPO.
But vonneumannstan did not claim Dario was uniquely suited to predict the future. He claimed Gary Marcus and Eli Lilly CEO weren't in a position to predict future, which I think isn't very controversial.
Do you know anyone worse at predicting the future than Gary Marcus?
I would like to give the people reading this an intuitive explanation as to why he's wrong.
There's a famous study in synbio that led to standardization called, Reproducibility of Fluorescent Expression from Engineered Biological Constructs in E. coli
https://journals.plos.org/plosone/article?id=10.1371/journal...
88 labs measured "fluorescence from three engineered constitutive constructs in E. coli." You should eyeball the results yourself first,
https://journals.plos.org/plosone/article/figure/image?size=...
That's the result these machines spit out for the same set of samples being measured across 88 labs. This was termed a "remarkable degree of precision" (for biology) of 1.54x sd(!).
Teams couldn't converge on instrument-to-instrument variation within the SAME lab, https://journals.plos.org/plosone/article/figure/image?size=...
So the reading is different depending on when you measure it. For some reason, people in silicon valley think that biology is like classical logic and computing. You write some genetic code and you get expression out. It's not.
That's the pop-sci view of a real process. In reality, viruses and bacteria are extremely fickle, and hard to grow. A lot of the synthetic biology results aren't easily reproducible even if you know the protocol.
So you're telling me that you can fix that with "just" AI and "robots?"
Well... I would hate to break it to folks. But applying neural networks and ML to biology has been a thing since the dawn of AI. It was one of the very first applications people thought about.
It used to be called computer-aided drug design, then they went all in on expert systems, then there was a time when quantitative structure–activity relationship AIs were a thing, then there was teh deep learning phase... D.E. Shaw actually did a ton of work on it, https://www.deshawresearch.com/technology.html he used his money to buy a supercomputer and used that to train neural networks...
Blockbuster drugs have come out of this. Tirzepatide / mounjaro was discovered by algorithmically iterating through sister-compound families of the peptide they'd found in Gila monster venom until they found just the right one.
The issue that Anthropic and Open AI face here is that there's no low hanging fruit. There's no free lunch. It's very unlikely that Claude or GPT can do something that hasn't already been tried at some point in every AI/ML research era.
So they won't be able to "just" discover a new drug. If that was possible, then it would have already been iterated into by sheer accident based on the number of projects that have tried some variation of this with some sister architecture over the many decades preceding us.
Now where these machines do help is that they help people get started, and they also help working scientists. It's boring stuff, but think stuff like, "hey my machine isn't working, it's made by such and such. Can you write code to help me save my data?" or, "Can you proof read this grant?"
It will even accelerate the already tried approaches as it allows a smaller team to try more AI/ML algorithms than they would necessarily have had the expertise to execute before.
But will Claude magically solve medicine? No. I'm sorry, but it just doesn't work that way. And I'm not sure why they believe this, but I keep repeating this over and over again... A lot of this AI and biology stuff is nonsense. Standard undergrad and post-grad stuff whipped into fervor and fluff.
Excellent post but while you definitely sound very well informed and provide solid backing for your claim “they won’t just invent a drug” it’s not very convincing. If I had to bet, I’d still bet on the new ai companies advancing pharmaceuticals more than hypothetical counterfactual non ai enabled pharma companies, putting aside political and financial constraints around that. It seems like you know the domain but knowing the domain is less important than knowing what is outside the domain viz a viz tech disruption. The general matters more than the specific sometimes.
I think you're thinking that the research process will be something like Dario opening up claude code and typing "Claude, solve cancer, make no mistakes". But that just isn't the case. Consider the ability to develop new tools and software for drug discovery. Consider the advances that google has made with AlphaFold. Consider the ability to crawl mountains more data than every before and finding the smallest hints, tucked away by connecting 6 papers written in 4 languages, published 60 years apart. Bioinformatics is a relatively new field, but when I finished my degree in 2018 the field was still so far behind the cutting edge of computing it was laughable. "state of the art" was dynamic programming algorithms and I remember that the first implementation of blast tools were only just ported to CUDA while I was in class, and that really wasn't that long ago. Protein folding software when I was in school was basically completely manual. You needed to define the structures yourself and then give it a place to start and it would (entirely on cpu) do some sort of gradient descent to give you an approximation of the structure.
The thing that I noticed while I was doing my degree was that the CS people refused to talk to the bio people, and the bio people refused to talk to the CS people. The CS people would go on and make solutions to problems nobody asked, and the bio people would try to solve data problems in complicated ways that made no sense. Overlapping expertise is and was very rare between those two fields. AI however CAN be that bridge, because it can basically turn your average bioinformatician/biologist into a really competent programmer, without them having to give up time away from the work they are actually skilled at.
Maybe I'm just completely AI-psychotic, but I've always been a researcher at heart. I hated doing literature review because it was so frustrating falling through dead ends. These last few months with agents I have been able to go deeper and find better leads than ever before. AI is like having an insanely smart lab partner, available to you 24/7/365. Never gets tired, never has a pissy day, never gets too hungover to read papers. I do legitimately think it's going to accelerate the field so much faster than people expect. I don't know about the 10 year horizon, but I'd love to see it happen, and I'm cheering for them.
Judging from those tweets, I think that community isn’t just accepting that he’s excited and doesn’t know what he’s talking about
But also I think simulations like a “body world model” of some sort will help narrow down what to test for the fda’s standard
And the fda itself has some newer faster approval pipelines
Anthropic is also clear about how they are partnering with a handful of life sciences firms to coexist with people that do know what they’re talking about
So I wouldn’t be too dismissive, the agentic form filing aspect will go a lot faster, that part is basically already done
Comparing Anthropic to Theranos is more ludicrous than Dario's claims. Dave Ricks' (Eli Lilly CEO) tweet is much more guarded and accommodating of the medical benefit possibility of AI than the sensational framing the OP provides. I think Ricks is right -- Dario's timeline for cancer and Alzheimer's is likely way too aggressive. But comparing Anthropic's actually shipping and continuously improving AI models to a bogus medical device shows a stark lack of thinking. Mental illness is definitely a different consideration here altogether and I think Dario deserves an eyeroll at minimum for that claim.
A deep and lasting reckoning is sorely needed. People like Amodei have to be brought back to reality by being confronted with the utter failure of their distorted view backed by deranged capital. I hope that this whole thing comes crashing down in flames like a fucking Zeppelin soon, so we can move forward to actually integrating these incredibly useful intelligence tools into where they can contribute to society and innovation, at a discount and without the CEO-enticing job-replacement hype.
Zuck, Musk, Altman, Bezos, Ellison.
Nadella and Huang, both currently getting some lessons from a re-familiarisation with reality, but who knows if these lessons will stick?
Actually, I do know: the lessons will not stick. A doubling down will be the result.
Trump had his inevitable midterm defeat, but I don't think any lessons were learned there either.
Zuck is probably getting a dose of reality as well given the fact that literally no one is using Muse.
or the Metaverse
I'd disagree with the statements and people making fun of it.
While these people are obviously professionals in their field, anyone who touched pharma/bio IT with a ten foot pole understands how much there is to improve there. And that's just the beginning - imagine how much there is to improve in understanding of the body overall?
It's not about "AI will discover a cure for X in 10 years".
But it's about having the applied intellect of a an engineer, chemist, biologist, and all mixtures of that, available on demand, in a second, at scale, enabling you to do everything around it faster, better and cheaper.
Every step of drug discovery, synthesis and testing can be improved.
Over 90% drugs tested in humans fail. And maybe 0.1% of overall discovered ones. While yes, testing on humans is the bottleneck, we can fix problems around the bottleneck easily.
If you had a data processor and 90% of data that goes into it is corrupt, the processor isn't the bottleneck yet - the data quality is.
If we can create better biological models to test these drugs, better interaction models, improve pharmacokinetics, improve yield, we can go from 1000 discovered, 1 works, to 1000 discovered, 100 works. We can predict some candidates will fail due to better models and not test these. We can improve how they are delivered to the body and make them more effective. We can catch potential side-effects early on and avoid going to trials at all.
This is what AI can do even nowadays, let alone 5 or 10 years from now.
And that is a huge improvement. It revamps the industry.
the data quality is the bottle neck, so why do you think improving the processor will suddenly unbottleneck it?
tbe improvement you are suggesting isnt whats being sold - that ai will actually discover said drug
I don’t see how cancer is not solvable by AI - if you can cheaply detect the genetic defects that are causing the cancer in any patient, and you can cheaply and rapidly generate and validate retroviruses, you can effectively treat any cancer.
How cheaply? How accurately? Your proposed solution puts a lot of strain on the body, so even if it fixed cancer you wouldn't want to do it preemptively every day for example, likely shortening lifespan substantially despite cause of death not being cancer any more. It's a scheme that also relies on detecting cancer quite early (else the bulk of a tumor can easily be unchanged from that intervention), so you'd expect quite a lot of those false positive scenarios. Not to mention "cheaply detect genetic defects" sounds like a pipe dream for now with or without AI (assuming you're referring to single-cell mutations, since any other interpretation just doesn't address enough root causes to actually cure anything close to most cancers).
I guess I fell for the AI psychosis too
Not all cancer is caused by genetic defects that can be detected ahead of time.
I remember when we were having more or less this same conversation about IBM’s Watson. It wasn’t even that long ago.
The most interesting difference between then and now is that we didn’t have tech enthusiasts getting quite so invested in the conversation. Possibly because we were all familiar with IBM’s decades-long history of shipping overhyped and underwhelming products.
Unfortunately we seem to have yet to realize that that’s a habit that’s endemic to the tech industry. Instead we still see it as thousands of isolated cases that, only by sheer dumb coincidence, happen to comprise the bulk of the tech industry.
I remember when we were having more or less this same conversation about the internet. It wasn't even that long ago, either.
"Open data, sharing, real-time communications leading to great progress."
Current AI should be compared to the internet, not to watson.
Could you explain why? Because I just don’t see the analogy.
Openness and sharing is precisely why the Internet was so successful. Anthropic, by contrast, is a proprietary SaaS vendor with monopolistic ambitions that’s openly seeking regulatory capture and actively supports the swarms of AI bots that are killing what remains of the open Internet by driving everyone’s hosting costs up to an unsustainable level.
Well, I said we should compare current AI to the internet.
You then compared Anthropic to the internet, and said you don't see the analogy (neither of us made an analogy fwiw).
I hope that helps.
You still haven’t explained what you think the analogy is.
I wasn’t using analogy in the “literary device” sense. I was using it in the “rhetorical device” sense: https://en.wikipedia.org/wiki/Analogy
being close to someone with a rare terminal and slow-moving cancer that can take years and years to kill its host, i’ve been learning that even just testing whether a therapy is going the right way can take many years, so while i hope “magic happens” i have to concur that the statements being made ahead of a mega IPO seem a bit tragicocomical.
Something that I learned from COVID was how much variation there can be in regards to how affected one can be from both a disease as well as its vaccine.
I know someone who got long COVID, and I'm thankful I'm not someone who was prone to it, but we still don't seem to know what makes someone prone to it or not. The human body seems even more complex than the complexity we knew about that was already difficult to comprehend.
I would say it is really disgusting, horrendous, either knowingly lying or being so naive you shouldn’t be given any responsibility
Browse https://www.reddit.com/r/accelerate/ for a bit and see how harmful these ideas are.
Some people legit think there is no need to go to school, work, learn or care about anything because Musk, Amodei, Altman &co promise them "universal high income" and eternal life in 10 years...
Wow. That subreddit is giving proper religious cult vibes, especially with the whole sense of superiority, uncritical belief and faith, and banning heretics.
The comment by Megan Fritts ("a professor of philosophy"), shown in the article, speaks to me possibly more than any of the other comments there. "Cult leader" is exactly right.
They ban everyone going the other way too, a massive echo chamber
Hey Dario, can you please add hair-loss to the list of things to cure, cheers.
the common cold would be nice, too
I'd really appreciate it if we could convene that hair-loss should be first, regards.
Amodei does produce a cure: it is just called "snake oil" or "frontier snake oil".
> The map of biology is still largely a blank page and randomized human proof is the long pole in the tent.
Ironically the Eli Lilly guy sounds exactly like Claude Opus 5
Is that the load-bearing pole? Is this how they fixed it? Rewording?
That article is literally just a list of tweets. If i wanted to be on Twitter, i would be scrolling Twitter.
Ahh yes, Gary "AI can't do math" Marcus
https://garymarcus.substack.com/p/math-is-hard-if-you-are-an...
I'm sure he'll be right this time around, just like, well, nevermind
I mean, at that point in time it really couldn't do math.
The post is about how LLMs cannot be scaled to ever correctly solve problems.
AI cannot do math. It uses deterministic tools for that.
Tell that to Terence Tao
Did Terence Tao really pretend that the LLM, which is the bit without tools (such as basic math calculator and Lean verification), can do math?
And if Tao mentions LLM, does he really mean "the bit without tools" or is he conflating the two?
I know that Tao is saying that current AI is useful to advance research in math (and I fully agree with that), but 1) that is part of a whole framework, LLM is just part of it, and I think that it is not the part that "know how to do math", 2) even if you obtain the correct mathematical output, it does not mean that the LLM part itself can do math (the same way that someone who throws 10000 basket balls and misses 9999 of them should be considered as unable to play basketball even if you just have a process where you ignore the 9999 misses and only record the one that made it)
Are you sure you know what math is, or are you thinking about arithmetic?
Tao and others don't sit around working out what 12,483,324,281/188,553.37463286*(243+2-983) is all day. There is a reason most people don't consider mathematics education to really start until you stop needing to bring a calculator to class.
You seems to have scanned the message and notice one word ("basic math calculator") and stopped there.
Just next to this word, there is "Lean verification", which is not about arithmetic but about the logic.
But the problem is not even there. LLM are not able to do math not because they fail to do 12,483,324,281/188,553.37463286 * (243+2-983), but they are not able to do math because they don't _understand_ 12,483,324,281/188,553.37463286 * (243+2-983).
I cannot do 12,483,324,281/188,553.37463286 * (243+2-983) myself just like that, yet I can do advanced math. Because I understand this expression. I know what it means. When the LLM was not able to produce the correct number, it was not because it knew what this expression meant and was "distracted" and forgot to report a 1 when doing the mental calculation. It was because the LLM was seeing the expression and was trying to guess the answer based on the answers of other texts that had similar expressions. It did not _understand_ the expression itself: it had no idea what it meant, it was just guessing the answer by copying the rough relationship it extracted from outputs it has seen in the wild.
And this is why it was a good element to demonstrate that LLM cannot do math: if they don't understand the expression 12,483,324,281/188,553.37463286 * (243+2-983), then they don't understand the expression f(t)=\sum_{k=-\infty}^{\infty} c_k e^{j k\omega_0 t}, \qquad \omega_0=\frac{2\pi}{T}.
And that is why the AI framework to do proper math and help mathematicians needs harness with tools like Lean. The LLM part does not understand what the expression means, it needs a tool to discard, internally, all the LLM outputs that are just non-sense.
"The AI can't understand because AI doesn't understand anything"
So what is understanding then, so we can check this hypothesis?
"It's the thing that AI can't do!"
What's the thing?
"Understanding!"
And what is understanding?
"It's the thing that AI can't do! I just said that!"
FYI, this isn't a serious argument without a clear testable definition of "understand" or "know".
And if you think you're gonna trounce two millennia of philosophy right now in a reply to me, best of luck.
Oh dear, not this again.
First, the claim has nothing to do with "understanding" itself. "understanding" here is just a short cut for "is doing the math", "is doing the computation", "knows what is correct and what is not correct". Replace it by that if you are confused by the word "understanding".
The claim is "LLM cannot do math". LLM. Not LLM + tools and harness. Not LLM but we get plenty of answers, run a program to remove the non-sense ones and pretend the LLM got only one answer.
And this is true: the LLM part cannot answer correctly these math questions. It answers with a first answer, check it, if not validated, came up with another answer, and so on. It has no idea which one is correct and which one is non-sense. It needs a specialised harness that will remove the non-sense ones.
But secondly, this incapacity to grasp something very simple is really annoying.
This is really not complicated. If you tell someone "what is 2+3" and that this person is going in the next room, write on their pocket calculator "2+3=1", click "ok", and the calculator says "Error", then "2+3=2", click "ok", and the calculator says "Error", then "2+3=3", click "ok", and the calculator says "Error", then "2+3=4", click "ok", and the calculator says "Error", then "2+3=5", click "ok", and the calculator says "Correct", and come back and say "2+3 is 5", it is clearly not the same as if you ask someone who understand math.
No need for "philosophical question" excuses. Call it "understanding math", or "able to do math", or "has learnt arithmetic", ... It does not matter. The reality is that this person, ON THEIR OWN, is unable to do math. They are not the one who know the math, the calculator who tells them if it works or not is the one knowing the math. (and again, if you are confused because this simplified example is using arithmetic, just replace "pocket calculator validating the sum" by "Lean software validating the mathematical proof")
We've had the ability to brute force solution spaces for decades, without AI.
Perhaps you picked up the belief that LLMs are just brute forcing these problems, but they aren't. They are creating hypotheses and checking them, which is the same thing humans do.
The solution spaces for problems is so massive that brute forcing it is beyond what even "every atom in the universe is a super computer running for 13B years could crack"
You need to understand the problem to know where to search for the solution.
They are not "simply brute forcing". They are constructing hypotheses based on their training on other examples. So, it's not a simple brute force, it's an "educated by historical data brute force".
If you ask "1224245 * 12345332", it will not try "tomato", it has observed other cases where such numbers are like that in such expression, and it will try answers that "looks like" those. Sometimes picking up on good relationship ("when the first number ends in 5 and the second number ends in 2, the solution ends in 0"), sometimes picking up bad relationship ("in my examples, I see that in the solutions, the fourth digit has 22% chance to be a 1").
But still, it does not really matter. A LLM, without harness or tool, cannot do math: it returns answers that are "reasonnable guesses based on historical data" without knowing which ones are non-sense or not, and it needs tools and harness that "do the math" to rejected the non-sense.
> You need to understand the problem to know where to search for the solution.
"What you mean by understanding"
you: "well, that thing that LLM does"
"what's the thing"
You: "Understanding!"
"And what is understanding?"
You: "It's the thing that AI do! I just said that!"
But, no, the "logic", the "structure", of math, is not inside the LLM. The LLM regularly returns outputs that are very good, because turns out that some relationships that the LLM extracted from historical data was also the relationships that happen to work here. And sometimes, the LLM returns outputs that are totally crazy, incompatible with "understanding the problem". Because the LLM is just doing "educated brute force" and applied a set of relationships that someone who is familiar with the problem would immediately consider inadequate, but the LLM still does it because it does not understand the problem, just tries to guess the direction of the answer based on historical direction of answers, without any idea if these directions are non-sense or not.
I'm sorry but you are horribly confused about how these models work. This reads like your understanding comes from social media comments (stochastic parrot stitching together snippets from an archive) rather than what is actually happening (generalization of distilled knowledge).
You're trying to argue that a Michelin Chef doesn't understand cooking because she only uses techniques and ingredients that others have used before - even though the dishes are totally new and other chefs are studying her techniques.
Well, I am a scientist with a PhD, I work with ML professionally since "before it was cool", and I have built my own toy LLM.
It is still possible that I'm incorrect, of course. But my understanding of how it works seems to fit very well the data.
"generalization of distilled knowledge" is a naive way out. People who have experience with ML have seen how difficult it is for the system to not get stuck in a local minimum or not latch on to a spurious pattern. Just for arithmetic, providing tons and tons of 6 digits multiplication tables, it is still extremely difficult for a ML model to extract the knowledge, there are a huge amount of fake patterns that leads it to incorrect heuristics. For example, the values of the first digit of the two numbers are strongly correlated to the value of the first digit of the answer, so the training will push into the direction for the model to learn to answer the computation based on this kind of heuristic. The "distilled knowledge" that the LLM will extract is not basic mathematical rules, it is "74% of the time, it's 4 and 26% of the time, it's 6", and it will fall into this local minimum and it will be difficult for it to get out of it.
But as I've said, I'm open to the possibility I'm wrong. If it's the case, maybe you can convince me otherwise. But it would help to bring concrete arguments more than "you sound like you don't have experience on this subject".
> You're trying to argue that a Michelin Chef doesn't understand cooking because she only uses techniques and ingredients that others have used before
But I don't say that LLM cannot do math because they use techniques and ingredients that others have used before. I'm saying that LLM cannot do math because the way they work. Fundamentally. The LLM extracts pattern by minimizing the loss function. There are hundreds of heuristic that will lead to a small loss function while still returning bad answers. Some patterns are "the good ones", but hundreds of them are misleading ones. And they can even get stuck in the wrong valley, with the way they learn preventing them to climb the ridge to join the "correct" valley. The fact that LLM that were very convincing to talk like they understand the human language and yet had a 2% success rate with basic arithmetic shows that LLMs often fail to extract the correct patterns. These LLMs did train on tons of numbers. To the point that they were able to do very convincing things. And yet, they did not discovered the "distilled knowledge" that were the arithmetic rules.
This, for me, seems to be coming from "social media comments". It is easy for a layperson to get "oh, if they understand that they can compute 44254324 * 53326342 by using simple arithmetic process, then they will get the answer right, so it means that optimising to get the good answer will, for sure, lead the LLM to learn the arithmetic rules". But the layperson does not understand how easy it is to find a misleading heuristic instead. Or even to build a mix of both (applying some arithmetic rules correctly for part of the problem, and misleading heuristic for the rest. Can you imagine the impact on the loss function in this case?)
Finally, an important point: for me, the big problem is that this view of LLM, that I have explained, is fitting all the data. This explanation not only explain why LLMs are so brilliant on some stuffs, but also do things that were unexpected, especially if LLMs are "generalizing distilled knowledge" (if it is the case, it is difficult to justify things such as hallucination, or crazy answers when asked the number of "r", or crazy loops when asked the seahorse emoji, or why LLMs fail to do basic math, or ...). This explanation makes sense and "scale": it allows me to understand small ML, understand why they fails, then understand what happens when we increase the training, then understand why these huge models behave the way they behave. The argument "but suddenly, there is a magical thing that happens and the LLM becomes something else: it suddenly understands distilled knowledge without falling for spurious patterns" is not only not explained, it is also totally useless: we don't need to assume that, everything that we see make perfect sense and is perfectly consistent with the LLM-as-a-statistical-heuristic-patterns-extractor.
AI solved a millenium problem. Is that not math anymore?
Woody Allen, “The Whore of Mensa,” collected in Without Feathers.
Suppose I wanted Noam Chomsky explained to me by two girls? Oh, wow. ... You’d have to speak with Flossie, she said. It’d cost you.
We may need Flossie to also explain Gary to us all.
Does Anthropic list curing autism under R&D milestones or risk factors?
I just love how this guy always was, and always will be, wrong.
It seems hallucination is infectious ... from LLM to human ..
Not only the CEO of Eli Lilly, but everyone with the most basic knowledge of medicine and health.
Dario Amodei is in pure snake oil territory now.
The man honestly seems unwell
Now?
Sorry - when has he not been?
I thought we where all on Elon Time. So ten years Elon Time equals about one hundred years real time, your honor.
I can think of one way Claude is likely to eliminate all biological disease in the next decade.
No more biological diseases if there is nothing biological left!
Biology is a disease
Remember when IBM Watson was going to cure cancer and fix IT?
History doesn’t always repeat, but it often rhymes.
I am sold when Anthropic can promise me a pony for my boat... Just as likely!
I want to be on the other side of Gary Marcus predictions.
If he's selling (figuratively) Anthropic IPO, I want to buy it.
Few persons were publicly as wrong about LLMs as he was.
A useful always wrong even though he is kinda right voice. Like Ray Dalio or Paul Krugman.
a man who had his testicles removed while he worked in China will say anything to fulfill his dream of revenge. standing on top of the world, curing cancer.
A post which is nothing more than a repost of Twitter randos? Very very insightful stuff, thanks for sharing.
Does the Eli Lilly CEO know about the exponential time hypothesis? If not, I don’t think they have a pulse of what AI is capable of doing.
It’s honestly offensive to those of us that have suffered for decades and have navigated a kafkesque nightmare of modern medical limits.
Knowing how little Claude has “cured” software engineering, this is just garbage self-aggrandizement.
Amodei seems to be yet another one of these founders who don’t know what they don’t know, but speak very loudly and proudly of their ignorance.
One might even say he seems manic and unwell.
What did claude cure in software engineering? Producing thousands lines of code quickly cured nothing.
Nothing. That’s what I’m saying.
LLMs, while definitely a useful tool have not massively replaced engineers.
If we can’t even have huge paradigm shifts in the area it’s specifically tailored toward, there’s a low probability it will eliminate entire fields that require physical testing and extreme domain knowledge.
We don’t have self driving cars, solid state batteries, or humanoid robots. None of this has been sped up by AI in the way Amodei is suggesting.
I like how he declares "mental illness" to be solved, when this is an even more varied and ill-defined concept than "cancer".
Authoritarian states often declare rejection of the state narrative as "mental illness".
Christian conservatives declare much, if not all, of queer culture as "mental illness".
On the other hand, some more consensus variants of mental illness, such as addiction or depression are actively caused by tech products and are fundamental to the value proposition. Tech companies invest enormous amount of work in how to make their products more addictive.
AI has already spawned its very own concept in the form of "AI psychosis".
That's not even starting with "virtual friends" or downstream effects like destruction of social skillsets and environments.
So, which of those things are included in his definition?
My problem with all these fantastical predictions, is that most of these real big problems aren't really technological problems; they're social and political problems.
In order for "AI to solve X", there has to be a tremendous amount of resources put into not only AI solving X but testing and implementing the AI-generated solution(s). Who is going to put in those resources? The money flows towards what's profit-generating, not problem-solving.
So far, AI hasn't solved any problems that matter in terms of improving the human condition in any significant way. What has it accomplished so far? It has allowed companies to develop software faster so they can cut their workforce. It has solved some tricky math proofs. That's about it. Neither of these make a dent in improving humanity's lot.
"Yeah now it's just a chatbot, but once it gets better, it'll solve X." Yeah, sure.
_Theoretically_ AI could develop solutions to some gnarly technological problems -- nuclear fusion at scale would probably be at the top of my wish list -- but as we've seen the last 2 years in the US, the main reason we aren't relying more on cheaper/renewable energy sources is political, not technological.
Medicine is similar - the reason we still have Ebola outbreaks in the DRC isn't because we don't have a cure for Ebola. Same for the half a million children who die from malaria each year; it's not like we don't know how to stop malaria. (If they were American children, we would have have eradicated malaria by now, but they're African children ...)
World hunger: we know how to produce enough resources to feed and clothe the planet -- it's just very badly distributed, by design.
Etc.
Dario Bankman-Fried von Holmes?
He's completely and utterly high on his own supply. That is what happens when you get told all day everyday how great your ideas are.
AI is solving phD level open math problems, even though I was told it was just a stochastic parrot that would never generate new ideas or solve unfamiliar problems.
Self driving cars roll past my office every couple minutes, even though I was told AI would never be able to drive on public streets.
AI images and videos are indistinguishable from real videos, even though I was told they'd never be able to generate hands correctly.
But I'm sure the naysayers are right this time, AI defintely won't ever be effective at medical research.
It's interesting him talking about AI curing mental illness, when there is now a quasi-established AI mental illness called AI psychosis [1]. As the saying goes: "there is no such thing as a free lunch".
[1] https://en.wikipedia.org/wiki/AI-induced_psychosis
To be fair he does not say cancer, Alzheimers and mental illness will be cured in 10 years, but that they will become 'relics of the past'.
Just do not feed that vague reward signal in their AGI training run because combined with Amodei & bros being firmly in the war hawk paranoid anti-China camp they can make us and our diseases all relics of the past just to prevent offline local inference with chinese open weight models.
> he does not say cancer, Alzheimers and mental illness will be cured in 10 years, but that they will become 'relics of the past'.
Doesn't everyone believe that, on some timeline, we will cure all of these things? Like if I said within a million years, you'd be hard-pressed to find a serious doubter.
Dario just thinks he's going to build ASI and ASI will solve them more-or-less instantly. I believe an ASI would do that, as well.
So by my logic, naysayers really just don't think we're about to build ASI. Dario does. I figure he has a better perspective. But we're not arguing about solving all of that stuff, we're arguing about whether ASI can do it, and whether we're on the cusp of ASI
Humanity may well be extinct a million years hence. The average mammalian species appears to last ~1-2 million years.
The average mammalian species is also probably what, a mouse?
At some point I will have a LLM explain Gary Marcus to me. He seems just silly and incorrigible, and I am mystified as to why people I think are important think he is important.
A corollary to Gell-Mann Amnesia: I don't know shit about drug development, but I know a bit about the cybersecurity industry. Most of what Anthropic has said about cyber isn't wrong exactly, but it's been breathless and so crudely expressed that it fails to capture the real path that developments are likely to take. For example, Anthropic said that AI might create a giant botnet (no shit, so what) whilst not really talking about agentic offensive frameworks (uh-oh) at all.
I have to assume that they're probably doing the same with medicine.
According to Dario in the twitter video, AI will cure all human ailments.
I wonder if AI will also cure psychopaths.
Well, death does cure all ailments.. Although I have my doubts that an LLM will ever evolve so far as to kill us all.
Ricks might want to be careful about offending someone who controls the largest and most powerful kompromat collection and manufacture machine to have ever existed.
The old never pick a fight with someone who buys ink by the barrel should probably be updated for people who buy power for their AI data harvesting and influence campaign by the gigawatt.
Scientology has got nothing on the AI safety cult in terms of capacity for dirty tricks.
Time for another CEO to get "sniped" via their dirty laundry aired out in public.
Guy sounds like he has some major skeletons in the closet. Seems like many rich people go through this odd manic phase before the breakdown too.
Says Gary Marcus, huh?
Any statement from Nostradamus yet?
Same kind of expertise as Dario on Medicine