I understand and maybe even agree with the general “vibes” here, but I think we need to be careful about making the assumption that AI will just figure out every style of “hard problem.”
It’s been really great for certain problems. It may even “cure cancer.” As it stands, AI is good at specific things and not all types of things. Yet.
If theranos can hallucinate magic blood testing machines for juicy investor dollars, why can't AI hallucinate magic energy storage devices for same, but faster!
And let's give it a density of the densest elemental metal (Osmium), at 22.6 g/cm^3
Assume the battery stores pure energy (I know, I know). So E = mc^2, which is like 8.99 x 10^16 J / kg
So that battery would hold about 22.6 g/cm^3 * 3.85 cm^3 * 8.99e16 J / kg = 7.8 * 10^ 15 joules = ~2.2 TWh, about a trillion times more than a normal AAA battery.
Doing some LLM-assisted sleuthing, a large cruise ship can use over 80,000 gallons of fuel a day [1]. A bit of searching indicates marine fuels have a density of approx 39 MJ/L (I'm using volumetric density for consistency), so 80,000 gallons * 39 MJ/L = about 1.1 x 10^7 megajoules = 3.2 GWh.
So a 2.2 TWh AAA battery could power that ship for 671 days.
Of course, the practical physics here are nonsense and there are obvious safety considerations associated with your AAA-sized nuclear weapon, but maybe that's what the author was getting at?
The last thing humanity needs is a way to store that much energy in the palm of your hand. It would take about five of these impossible batteries to match the energy released during the Tunguska event.
Exactly, I assume this is the kind of thing the author is thinking about. But going this far into the extremes of theoretical physics and calling it "hard, but possible" is somewhat misleading on their part. Commercial fusion power is "hard but possible", and even that's an understatement. Ship-mounted matter-antimatter power or whatever should not be implied to be within the same category.
A small to medium size cruise ship has an engine at around 24'000 HP ≈ 176 MW.
The smallest terminal you can connect to a AAA battery is the same diameter as 00 AWG copper wire.
Provided that the engine has 100% efficiency and can turn 176MW of electricity into torque, to safely run power it needs through that 00 AWG wire, the battery would need to provide a voltage of at least 70 kV.
Energy density. This would be approximately 10^9 to 10^10 times the capacity of a real cell, would be better than fusion fuel, and almost 1% of the theoretical maximum.
You don't even need to be an expert to poke at why this is extremely implausible.
Think about the materials involved. What material stores all this energy, which one assumes is inert and room-temp? And if it's not inert/room-temp, what material contains the other material?
In each case, how much mass is needed of each - to store, and to contain? Noting also that the more mass you need of the energy-bearing material, the more you will need of the other. Can that mass be compressed into 4 cubic centimeters?
I personally have no idea, but it's sure sounding impossible.
The article starts that section with “As the Big Bang has so loudly illustrated” so I assume the author is imagining a machine that replicates the conditions of the Big Bang. Exactly how that could be constructed and used safely is left as an exercise for an AI.
No argument, but my point is that the author's confident pronouncement that we can keep scaling is ridiculous, and your otherwise valid point earns the author net zero credibility.
The materials science and biology part yes, it will be awesome.
but the question is always qui bono.
First we will have the end of propriatry software and everything moving to the cloud. Even PC will end to exist with Microsoft pushing hard on their M365 virtual computers.
Who will receive these new materials.
I am looking forward to "Diamond Age" becoming reality. But it will be hell of a ride
"Everything moving to the cloud" is far from inevitable, and represents a concentration of power. Of course existing tech corps want that, they proposed it and have pushed it in the first place, but I strongly disprefer my personal things being on someone else's computer, don't need a ton of computer to host them on the internet myself for my own purposes, and am far from alone in this. The logical endpoint of priorities of large businesses and governments are not inherently inevitable
> "Everything moving to the cloud" is far from inevitable, and represents a concentration of power. >The logical endpoint of priorities of large businesses and governments are not inherently inevitable
Conversely, the present concentration of power, and the lack of any viable effort to reduce it, can be used to force moving everything-that-matters to the cloud. Not a good outcome but inevitable if things stay the same.
Biggest danger of LLMs as they currently fit into society: the radical consolidation of information flow through corporate-controlled bottlenecks that make current generation algorithmic social media (rancid as it is) look like child's play. When people no longer "search" for things, but simply "ask ChatGPT", the lever of social control is nearly absolute.
(Of course, all this is up in the air. If LLMs become commodities, the situation becomes very complex; the information flow doesn't get quite so consolidated, but it still becomes impossible to tractably analyze, as all the LLMs train off each other)
Hand-waves the negatives and the positives. Makes no real effort to contend with the drawbacks of AI, and leaves out the biggest one (IMO) - the potential harm created via delegation of arbitrary authority/capability to AI, whether in government, industry, or military. But we've apparently solved alignment, so maybe it's OK.
The rest is essentially just a short list of consequential problems facing the world, with "maybe AI will help us fix it" tacked on to each paragraph.
The last two problems don't even mention the role AI would play in their solution. That's a bit of a tell.
> But we've apparently solved alignment, so maybe it's OK.
Yeah and it is such an absurd point. You take a brief look at world right now and the history and you find ... constant fight over what exactly "the values" should be. There is no single human aligned answer to "what we should do". There is no way to align AI simultaneously with liberals, transhumanists, right wing, tech CEOs and all the world religions.
And there, I am even ignoring situations where interests of one group go against another group.
> There is no way to align AI simultaneously with liberals, transhumanists, right wing, tech CEOs and all the world religions.
Well, the real problem isn't about "humans aligning LLMs" it's about "LLMs aligning humans" according the values and desires of the LLM's masters - that kind of alignment is entirely possible, proven by history on the human side and by current LLMs on the technology side.
They missed "AI analysis implemented as an infinite number of authoritarian servants documenting the physical traits, locations, behaviors and associations of all people 24/7" in the "really bad" column.
They forgot this bullet for the "seriously great things on the horizon":
* Those fucking annoying workers, we don't need them anymore! Most of them can go FOAD, literally. That leaves even more money and resources for the only ones who really deserve it: the extremely wealthy.
I think its very reasonable for AI to speedup superconductor finding which will make medical scanning a lot cheaper, easier and better.
It will also allow for massive and easy energy transportation for large distances like afrika/spain <> europe.
In medicin understanding DNA and the results of it could lead to solving cancer and dementia.
New better batery chemicals will help us for solid state batteries. no AAA version of this but up to 1000kwh/kg + low temp and high temp stability and very stable against burning.
Will be used for mass surveillance and for further destruction of the democracy. Will be used to create powerful monopoly. Will make climate change worst due to huge energy consumption. This will be unfixable, because tech CEOs who will have more power do not care at best and are far right actors at worst. "The alignment" will be defined by the people who already shown themselves sociopaths.
> Seeing something this childish and frankly stupid make it to the front page HN makes me really sad
Quite a lot of people active on this site and/or working in that industry believe in this nonsense. It’s really not that far from what the AI leadership is pushing, but written by a 14yo who watched a few videos on libertarianism. At least Amodei’s essays are well written
I understand and maybe even agree with the general “vibes” here, but I think we need to be careful about making the assumption that AI will just figure out every style of “hard problem.”
It’s been really great for certain problems. It may even “cure cancer.” As it stands, AI is good at specific things and not all types of things. Yet.
After working with the frontier models for over a year I agree with you. Some problems are "AI shaped" but it's not a universal solution
Yeah, especially since some problems are impossible
Author is misguided:
"Designing energy stores the size of an AAA battery, but with the capacity to power a cruise ship, is hard, but possible."
is false.
For a mere mortal. But AI can simply hallucinate new elements that defy physics. Then it’s just up to us to go find those hallucinated elements!
If theranos can hallucinate magic blood testing machines for juicy investor dollars, why can't AI hallucinate magic energy storage devices for same, but faster!
The author is suggesting that the gift shop has a nearly unlimited supply of AAA batteries and the ship can sail for lengthy periods.
Why is that the case? Genuine question, not rhetorical.
I got nerd-sniped on this, so let's do the math.
Volume of a AAA battery = ~3.85 cm^3
And let's give it a density of the densest elemental metal (Osmium), at 22.6 g/cm^3
Assume the battery stores pure energy (I know, I know). So E = mc^2, which is like 8.99 x 10^16 J / kg
So that battery would hold about 22.6 g/cm^3 * 3.85 cm^3 * 8.99e16 J / kg = 7.8 * 10^ 15 joules = ~2.2 TWh, about a trillion times more than a normal AAA battery.
Doing some LLM-assisted sleuthing, a large cruise ship can use over 80,000 gallons of fuel a day [1]. A bit of searching indicates marine fuels have a density of approx 39 MJ/L (I'm using volumetric density for consistency), so 80,000 gallons * 39 MJ/L = about 1.1 x 10^7 megajoules = 3.2 GWh.
So a 2.2 TWh AAA battery could power that ship for 671 days.
Of course, the practical physics here are nonsense and there are obvious safety considerations associated with your AAA-sized nuclear weapon, but maybe that's what the author was getting at?
[1] https://blog.windstarcruises.com/how-much-fuel-cruise-ship-u...
The last thing humanity needs is a way to store that much energy in the palm of your hand. It would take about five of these impossible batteries to match the energy released during the Tunguska event.
Exactly, I assume this is the kind of thing the author is thinking about. But going this far into the extremes of theoretical physics and calling it "hard, but possible" is somewhat misleading on their part. Commercial fusion power is "hard but possible", and even that's an understatement. Ship-mounted matter-antimatter power or whatever should not be implied to be within the same category.
A small to medium size cruise ship has an engine at around 24'000 HP ≈ 176 MW.
The smallest terminal you can connect to a AAA battery is the same diameter as 00 AWG copper wire.
Provided that the engine has 100% efficiency and can turn 176MW of electricity into torque, to safely run power it needs through that 00 AWG wire, the battery would need to provide a voltage of at least 70 kV.
How to protect yourself from spontaneous release of energy in case of cell failure? This problem gets worse with density
Energy density. This would be approximately 10^9 to 10^10 times the capacity of a real cell, would be better than fusion fuel, and almost 1% of the theoretical maximum.
It violates the laws of thermodynamics.
You don't even need to be an expert to poke at why this is extremely implausible.
Think about the materials involved. What material stores all this energy, which one assumes is inert and room-temp? And if it's not inert/room-temp, what material contains the other material?
In each case, how much mass is needed of each - to store, and to contain? Noting also that the more mass you need of the energy-bearing material, the more you will need of the other. Can that mass be compressed into 4 cubic centimeters?
I personally have no idea, but it's sure sounding impossible.
The article starts that section with “As the Big Bang has so loudly illustrated” so I assume the author is imagining a machine that replicates the conditions of the Big Bang. Exactly how that could be constructed and used safely is left as an exercise for an AI.
This is same thing someone in 1926 would say about your cell phone battery
No argument, but my point is that the author's confident pronouncement that we can keep scaling is ridiculous, and your otherwise valid point earns the author net zero credibility.
What's in this AAA battery? Antimatter?
[dead]
CO2 capture is limited by thermodynamics sadly. https://www.youtube.com/watch?v=EBN9JeX3iDs
The materials science and biology part yes, it will be awesome. but the question is always qui bono.
First we will have the end of propriatry software and everything moving to the cloud. Even PC will end to exist with Microsoft pushing hard on their M365 virtual computers.
Who will receive these new materials.
I am looking forward to "Diamond Age" becoming reality. But it will be hell of a ride
"Everything moving to the cloud" is far from inevitable, and represents a concentration of power. Of course existing tech corps want that, they proposed it and have pushed it in the first place, but I strongly disprefer my personal things being on someone else's computer, don't need a ton of computer to host them on the internet myself for my own purposes, and am far from alone in this. The logical endpoint of priorities of large businesses and governments are not inherently inevitable
> "Everything moving to the cloud" is far from inevitable, and represents a concentration of power. >The logical endpoint of priorities of large businesses and governments are not inherently inevitable
Conversely, the present concentration of power, and the lack of any viable effort to reduce it, can be used to force moving everything-that-matters to the cloud. Not a good outcome but inevitable if things stay the same.
It's a bit of a tautology to say that things are inevitable if we don't avoid them, though of course I agree
If it can make a web page responsive and keep in bounds when I zoom in the font that would be a glorious future.
If web browsers were able to take care of that, as it is their job, it would be a glorious throw back to a brighter past.
Biggest danger of LLMs as they currently fit into society: the radical consolidation of information flow through corporate-controlled bottlenecks that make current generation algorithmic social media (rancid as it is) look like child's play. When people no longer "search" for things, but simply "ask ChatGPT", the lever of social control is nearly absolute.
(Of course, all this is up in the air. If LLMs become commodities, the situation becomes very complex; the information flow doesn't get quite so consolidated, but it still becomes impossible to tractably analyze, as all the LLMs train off each other)
Hand-waves the negatives and the positives. Makes no real effort to contend with the drawbacks of AI, and leaves out the biggest one (IMO) - the potential harm created via delegation of arbitrary authority/capability to AI, whether in government, industry, or military. But we've apparently solved alignment, so maybe it's OK.
The rest is essentially just a short list of consequential problems facing the world, with "maybe AI will help us fix it" tacked on to each paragraph.
The last two problems don't even mention the role AI would play in their solution. That's a bit of a tell.
> But we've apparently solved alignment, so maybe it's OK.
Yeah and it is such an absurd point. You take a brief look at world right now and the history and you find ... constant fight over what exactly "the values" should be. There is no single human aligned answer to "what we should do". There is no way to align AI simultaneously with liberals, transhumanists, right wing, tech CEOs and all the world religions.
And there, I am even ignoring situations where interests of one group go against another group.
> There is no way to align AI simultaneously with liberals, transhumanists, right wing, tech CEOs and all the world religions.
Well, the real problem isn't about "humans aligning LLMs" it's about "LLMs aligning humans" according the values and desires of the LLM's masters - that kind of alignment is entirely possible, proven by history on the human side and by current LLMs on the technology side.
We will get incredible abundance and health before the singularity. After that, all bets are off (almost by definition)
This would be a great essay from a 3rd grader.
Explain
They missed "AI analysis implemented as an infinite number of authoritarian servants documenting the physical traits, locations, behaviors and associations of all people 24/7" in the "really bad" column.
Sounds like a Phillip K. Dick scenario. Which are all happening these days
They forgot this bullet for the "seriously great things on the horizon":
* Those fucking annoying workers, we don't need them anymore! Most of them can go FOAD, literally. That leaves even more money and resources for the only ones who really deserve it: the extremely wealthy.
Terrible ostrich article.
It just basically ignores all the issues and goes "guys, it will be great, stop worrying! There's nothing to worry about!"
Pure schlock.
I find certain points woo wishiwashi.
I think its very reasonable for AI to speedup superconductor finding which will make medical scanning a lot cheaper, easier and better.
It will also allow for massive and easy energy transportation for large distances like afrika/spain <> europe.
In medicin understanding DNA and the results of it could lead to solving cancer and dementia.
New better batery chemicals will help us for solid state batteries. no AAA version of this but up to 1000kwh/kg + low temp and high temp stability and very stable against burning.
Physical ai will solve human labor.
The really bad
Will be used for mass surveillance and for further destruction of the democracy. Will be used to create powerful monopoly. Will make climate change worst due to huge energy consumption. This will be unfixable, because tech CEOs who will have more power do not care at best and are far right actors at worst. "The alignment" will be defined by the people who already shown themselves sociopaths.
[flagged]
> It's that people want meat with every meal and limes in the winter.
I never wanted to end up in the social sciences. I wanted to work on growing big trees. But people keep getting in the way.
Not judging but I don’t understand why anyone would read pas the first point in “Really Bad”.
So much nonsense on HN front page these days, the algorithm desperately needs to be redone
> Seeing something this childish and frankly stupid make it to the front page HN makes me really sad
Quite a lot of people active on this site and/or working in that industry believe in this nonsense. It’s really not that far from what the AI leadership is pushing, but written by a 14yo who watched a few videos on libertarianism. At least Amodei’s essays are well written