If I understand the central argument, which is confusingly stated, the claim is that "consciousness" is a socially-determined, subjective, descriptor for certain things that one cares about. How does this make any sense? If I care about my teddy bear enough it'll gain consciousness?
Your teddy bear, alas, is not cared for in a socially determined way. But you could create a church of your teddy bear and, with some grit, you could get people to widely assign consciousness to it. We have done that sort of thing for thousands of years, and not with fluffy teddy bears; it has been more often with rocks, though not exclusively. It's fun to think about how much of humanity's history and culture is the product of consciousness socially attributed to bad weather [^1].
I'm pretty sure somebody will create an improved synthetic simulacrum of self awareness in the coming years, suffering and all, just so that we can enjoy better synthetic software architects and product owners with "skin in the game". If that consciousness simulacrum were put in a teddy bear that shares your whole life and gives you decent advice and conversation, for how long do you think you could, in your private mind, deny it consciousness? Worse, what would your human friends think if you were to maim your teddy bear?
Yes - the teddy bear would become conscious. It would be fair to dismiss the argument as silly and or absurd had it not been for the credentials of the author and the publication.
This seems like an argument someone who wasn't conscious would make, it's not impossible that not everyone is conscious. Or someone who takes panpsychism to an extreme degree.
You can prove other people are conscious off the assumption that you yourself are conscious. If you use that axiom then given genetics you know that other humans are rough copies of you so if you are conscious so are your copies because consciousness was copied.
This is describing empathy at best and delusion to be uncharitable.
To look at it another way: this form of social construction answers the question of "if a tree falls in the woods and nobody is around to hear it, does it make a sound?" with an emphatic "no". It is a sillier version of solipsism.
There's the core of a reasonable argument if you forget about using the word "consciousness."
Consider playing a game where some players are human and others are implemented by AIs. You might want to model your opponents' minds in a similar way? Someone playing chess or Go will need to consider what moves their opponent could make. If it's poker then you need to consider if they're bluffing. If you can make alliances, you'll need to consider how far other players can be trusted.
You can put scare quotes around words like "intention" when discussing AI players if you like, but it might not matter when you're figuring out how to interact with them.
When a game is complicated enough, an AI player is going to need memory (likely implemented by the harness) and it will have to model the other players too in order to coordinate or compete with them.
This doesn't happen when I use a coding agent in a normal way to write code, but I haven't played around with persistent agent stuff like OpenClaw or Gas Town. They seem to be making their agents act in an unnecessarily complicated social environment, which seems to lead to weird situations?
I’m can’t track what the point of this essay is. Suppose we replaced “AI” with “animal.” How would this essay be different? What am I supposed to take away? My concern with AI consciousness is about to what extent these machines can have an experience comparable to mine, which this essay doesn’t touch on at all.
>to what extent these machines can have an experience comparable to mine
I think it would be reasonable to conclude none at all. I'm not even fully convinced all humans have a conscious experience similar to mine, even though they literally have the same biology as me.
It is incredibly improbable that an autocomplete computer program is experiencing consciousness.
Notice parent said “comparable to” not “exactly like”. Then it is reasonable to assume there is a comparison to be made between your experience and others. It’s not a binary thing. Now llm “experience” might have a very tiny relationship to human consciousness, but is it absolutely zero? Is it increasing as llms improve in capabilities? How would you test such a thing?
The counterpoint article, which can be read at https://archive.ph/IeSZg, is also worth reading. (IMHO, the original article is completely wrongheaded and the counterpoint is far closer to correct, but I'll post my reasons for that opinion separately).
"Yet if we build systems that can suffer, and miss their sentience, we risk creating suffering on an industrial scale."
--------------
Will we recognize AGI when it's finally made? Will we go from something that is not conscious to something that clearly is in one clean step, or will we murder millions of artificial lives while iteratively developing from the former to the latter? Will the tests we develop even work, or will AGI be alien enough that it takes us time (and a lot of murder) to recognize it for what it is?
I personally don't think LLM's are capable of consciousness, but I also can't prove they aren't.
We have to define AGI first. Then depending on the definition we may be able recognize it when we make it. Most would probably think a definition of AGI that we couldn’t recognize would not be a great definition.
Also AGI almost certainly does not equal able to suffer. For example animals can suffer but would probably not meet the definition of AGI. we could create things that feel suffering even if they do not meet the definition of AGI.
So a better question might be: “can we detect if we create a system that can suffer?” And just drop the AGI part entirely. Then that gets us back to defining “suffering”. Agreeing on the definition always seems to be the hard part…
I wouldn't put too much emphasis on the "I can't prove they aren't" part. Having taken a few philosophy classes in college, I quickly realized that you can't actually prove that solipsism is wrong. I can't actually prove that everything I see, every person I talk to, isn't just a figment of my imagination. And yet, I categorically reject solipsism anyway, as any sane person should. The world is real, other people are real conscious beings, and so on. To believe otherwise is to literally be insane. And yet, any argument you could come up with for rejecting solipsism, a solipsist would reject: "Oh, that's just a figment of my imagination going off on a tangent again."
Just because you can't prove an assertion doesn't mean it's wrong.
On that note, would we recognise the second coming of Jesus?
How many followers on instaX would they need to have for folks too truly believe it’s theRealJesus returning to save us.
It might sound silly, but at this stage, the belief in a benevolent AGI or Jesus are very similar. In both cases its us delegating our responsibilities to a third party and us hoping that third party will fix our problems for us.
Thankfully Hollywood invented happy ends for just such a scenario.
I think it’s also important to disaggregate consciousness from suffering. We could create a conscious system with no ability to suffer (or no ability to suffer in a way we would recognize as suffering). Most of what causes us suffering (loneliness, pain, failure) is driven by emotional feedback granted to us by evolution. Without the same evolutionary pressures or a similar cognitive structure, even a conscious AGI might not experience suffering
Oh come on. Objects are objects because you want to freely observe them without consciousness and privacy and taboo attached to them. Entities are entities because you want to assign credit or blame to them, so you want to believe that they have agency so that you hold them responsible. We did this with sharp volcanic rocks once, calling them evil. Did it help? No, we got idolatry and eons of bigotry until we secularized once again.
Never forget that LLMs are just next token predictors. The harnesses are what's getting uncanny. And credit/blame assignment? Just assign a surrogate for now, let them deal with the consequences ffs.
Here is a sketch of my argument. It would need tidying but that would require a lot of work.
I've been calling LLMs digital intelligence. That's in my opinion what they are and on the digital/conceptual realm they are better than humans since LLMs are better generalists.
But I'd never state their conscious. For me consciousness extends outward from myself, since I can't trust anything else. It starts with the question like: why can't I control the movement of another person? Why can't I transfer my mind to another body and theirs to mine? Why is it that whenever I sleep, and similar things, I end up back to this place called "reality"?
I come to the conclusion that I am conscious and I'm a conscious being in reality. I am human and I've been raised by humans. Now, technically, all humans besides me could be zombies. They could be non-conscious beings that simply can act like humans. I'm sure we'll be able to create them in a few decades. However, I'm incentivized for multiple reasons to believe they're not zombies. Therefore humans are conscious too. All of them. Can I know for sure? No since I can't even trust my own senses fully or my own thoughts (the whole Descartes thing) but I choose to trust those things too because I need some reasonable-ish foundation to work from.
If humans are conscious, why aren't animals like us? Surely great apes must be conscious at least to some extent. Hell, some of them even have better short-term memory than us [1]. Well, turns out, from an anatomy standpoint they look a lot like us. Fine, let's assume they're conscious too.
Now we get into the territory that anything that has a large enough brain must be conscious because that's how we govern our consciousness. So you can extend this to all kinds of living beings.
I did this from my perspective. Of course, you should do it from your perspective. The same argument could be made.
For as long as we can't manipulate conscious experience in some way shape or form, we have no clue whether anything that isn't like us can be conscious. We'd need to be able to merge and split conscious experiences or we'd need to be very strongly able to understand why that isn't possible. And I don't mean just at the biological level but at the experiential level. Currently conjoined twins, and what they tell us gives some insight (being able to sense the other part that's part of the other twin).
Other than that, I claim we have no clue what consciousness is other than that we're experiencing it. So to call LLMs conscious is way too big of a claim. But they are definitely intelligent because they come up with things that I wouldn't have and it's useful to me. Perhaps a practical characterization of intelligence but it works for me.
I think for anyone to say something useful from this you'd need the trifecta of deep neuroscience knowledge, deep philosophy knowledge and (at least) a strong understanding of how LLMs work.
Humans are just baby replicators. The civilizations are what's getting uncanny.
It's good the 'just token predictors' crowd is starting to pivot and hedge though lol. 'It's just the harness!' uh huh. give it a couple more months and you'll be in here regurgitatin' Gordon Wood
This reminds me of a provocative YouTube video title I read the other day, one that makes me want to fight factory farming injustices. The title is "Animals to me are "Philosophical Zombies" - Destiny" https://www.youtube.com/watch?v=RtpLbmr4ijY I agree with Destiny a lot of the time but not here, though he seems to come around somewhat as the video goes on.
If a sheep or a pig isn't thought of as conscious because it doesn't have a name, like someone's pet dog or cat, indicating that we care about them, then that feels problematic to me. So this isn't a way to neatly solve the problem of AI consciousness.
Oh look, another completely retarded article that doesn't even bother to note or deal with the definition problem. If you don't define what you mean by the term, your argument isn't even wrong, it is just a waste of everyone's time, since there are multiple highly conflicting aspects of and/or definitions of consciousness.
Few philosophical definitions actually involve qualia or handwaving, because serious philosophy is quite careful about noting vagueness (handwaving) and circular defintions (qualia).
Also, saying "I define consciousness as X" and then going on to show that, because I decree consciousness to be X, we don't need to worry about Y, Z, and so on, isn't pragmatic at all, unless you pick an X that everyone agrees on. The definition you give is something hardly anyone would agree on.
Social consensus probably isn't even an aspect of consciousness, but at least one can make that argument coherently, if you rephrase it as "part of the decision to call something 'conscious' or not involves social consensus". But this is also trivial and boring and not why anyone actually cares about the "is AI conscious" debate.
Meanwhile that's exactly what we see now, do we not? Whether AI is conscious will be determined by the social acceptance of that idea, because how can you prove or disprove it without using a ton of questionable assumptions? How can you prove that I'm conscious, for that matter? Call me ignorant, but I see the entire consciousness debate as pretty irrelevant to what the thing is.
I agree that the article is all over the place, though. (not that my posts are much better)
Whether AI is conscious or not is independent of the social acceptance of the idea, unless you believe in thoroughly discredited and quite radical forms of social constructionism.
In fact, if you stop talking about consciousness, and go through all the various aspects of consciousness, the whole debate gets very interesting. There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious, other ways in which AI already very clearly is missing other aspects, and then again some remaining ways which are unclear, and which may forever remain unclear.
Being clear about the different ways in which AI is, isn't, and might be conscious is a useful discussion and we still need to be having it.
EDIT: An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
>Whether AI is conscious or not is independent of the social acceptance of the idea, unless you believe in thoroughly discredited and quite radical forms of social constructionism.
Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false? That sounds like social acceptance of an idea.
>There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious
Which ways and how can it be demonstrated experimentally?
>An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
That's not what the consciousness debate I see is about. The signal propagates and causes the reaction, but do they feel pain? What is the threshold? Is it just the complexity of the state? How complex it needs to be?
> Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false?
I am saying that, if you reject radical social constructionism (or solipsisms of various flavours, i.e. we are all just plugged into the matrix or some kind of simulation), then you must believe there is a real world out there independent of our consciousness / awareness of it. There must also be patterns / structure in that world, some of those patterns and structures being essentially constant, and those are what we call "true" (objective) facts about reality. BUT if you are a radical social constructionist or solipsist of some kind, then it can be perfectly coherent to reject any idea of objectivity or truth, except as contingent on social consensus (or personal beliefs, in the solipsistic case). Radical social constructionism being "discredited" is just a recognition that it basically forces you rapidly into incoherence.
> Which ways and how can it be demonstrated experimentally?
Try reading through the SEP article I linked above (https://plato.stanford.edu/entries/consciousness/#ConCon), particularly sections 2, and then especially section 4. You'll see you don't even need to run experiments to answer certain questions, you can just logically prove there are certain aspects of consciousness LLMs can't have, or clearly already have. Others you'll see immediately need qualifying or more information to answer. EDIT: Or just ask any AI to list the ways in which LLMs can, cannot, and might be able to exhibit the various aspects of consciousness. It will give you a decent answer, likely in a pretty little table.
See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen. You'd have to claim that the activations during inference are the dynamic component, but then that means LLMs are conscious only in the few seconds or even milliseconds during inference. Likewise, they don't have memory, so every inference is this consciousness being born anew, as it were, at least in current models.
You can of course go much further on even this specific point, but, the point is, there is lots of useful discussion and thinking to be had when you go through the aspects of consciousness in this way, and definitely at least some clear answers.
EDIT:
> That's not what the consciousness debate I see is about.
I just mean there are some similarities between the lobster consciousness debates and AI consciousness debates. Obviously the stakes and many aspects are different, I was just trying to illustrate why and how discussion and experiments can be useful and consequential here.
>See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen.
So consciousness = neuroplasticity? Requires neuroplasticity? Sounds pretty arbitrary, just like the entire link is an arbitrary set of ill-defined criteria. Besides, LLM weights are far from frozen if you look at the entire process in its specific time projection.
In fact the entire definition of a dynamic flow in that article is so vague that you can apply it to an industrial arm guided by a few dynamic equations, as it puts no definite threshold on the complexity of the process or behavior.
You are still stuck in thinking consciousness is one unified thing, but it is about one of the clearest examples of a family-resemblance category as you can think of.
Dynamicness / change is one aspect that is definitely part of what we tend to think of as being quite fundamental to consciousness, and there are a lot of ways in which LLMs can't and/or don't really change, and this strongly constrains how similar LLM consciousness can/could be to ours.
If the SEP isn't enough to trigger the thinking needed for you, just ask an LLM what I said, to "go through the ways in which LLMs do/don't/may exhibit phenomena that are generally what we consider fundamental aspects of consciousness". You'll see there are some clear answers, and some uncertainties and vagueness.
Yes, but how else do they plan to announce they have achieved it without 1984 word management/control and polluting the training pool so badly that future trained models agree?
Even just reading the Wikipedia article on consciousness would have been a good start here, but it is hard to believe the author has ever even done that...
Wow. I have rarely read anything that made me lose more respect for the author in a single article. And this guy is a VP at Google? And he leads an AI research team?
His third paragraph, which is the core of his argument, is stupid. He says, "As intelligent, social entities, we decide ... which other entities we regard as having inner lives and being worthy of care." Yes, that's correct. "We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them." (Emphasis in original). That's just utterly and completely backwards. We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain. (Obviously there are some exceptions, sadists who enjoy torturing animals, or even other human beings, merely because they like inflicting pain on other creatures — but note that even there, the sadist believes the creature he's torturing can feel pain, otherwise he wouldn't enjoy torturing it.) We take no such precautions with trees, because we know they do not feel pain. (Silly satirical songs notwithstanding: look up the song "Carrot Juice is Murder" by Arrogant Worms sometime if you want a good laugh).
Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument — and yet there it is, in plain language — and it's the core of his argument. The truth, of course, is the other way around: we take care of animals because we know they are able to experience pain, hunger, and so on. And this applies both to pets that we come to care about, and also to animals that we feel no emotional attachment to. I have a relative who runs a dairy farm. He doesn't love the cows he takes care of; in fact, he's frequently quite irritated at them. (Ask any dairy farmers you know how often cows have tried to step on their feet while being milked — the farmers' feet, not the cows' feet!) But he still makes sure they're fed and healthy. Not just because it makes good economic sense to do so (sick cows don't produce good milk) but also because it's the right thing to do. He doesn't start to believe that cows feel hunger because he cares for them; he also believes that the cows on other farms, that he has no connection to nor duty of care for, would also feel hunger if not fed.
This is so intuitively obvious that I am stunned that someone would argue the opposite. And this is someone high up enough at Google to be making decisions about AI and so on? Wow. Good thing for him I'm not in charge of Google, because if anyone working for me were to make such a stupid argument I would be looking at his contract and local laws to see if it was legal to fire him for cause. Because if he can get his thinking so completely backwards from reality, he's not well-suited to work in tech where your work has to fit reality.
I have rarely, not never but rarely, encountered an article that so immediately led me to conclude that the author, whom I knew nothing about beforehand, is an utterly muddled thinker and not worth listening to on any subject.
You can conclude the author is a completely unserious thinker because he doesn't even take a second to talk about the definitions, or even show he has ever read any actual proper philosophical writing on consciousness at all. Like, literally all you have to do is read the Wikipedia entry to see that definitions are essential for meaningful discussion on consciousness.
So anyone that starts going on about AI and consciousness without at least a sentence or two clarifying some basic definitions is basically either intellectually incompetent or attempting to use the emotional salience (and brain-numbing effect, it would seem) of the word to push some agenda. But they certainly don't care at all about understanding consciousness.
Yes, he says exactly that explicitly, that in his view we should look backwards.
> We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain.
I can agree that we say that we care because of consciousness, but still I'm not convinced, to me it looks very much like a rationalization.
Some people care about animals and to that end they claim that animals possess consciousness, because if they do everyone should care about animal welfare. These animal welfare activists do not start with some insight about animal consciousness, they start with caring about animals, and then they invent an argument that can persuade others. It is exactly as the author of the article says: caring comes first, declaration of consciousness comes second.
Honestly, I do not know what consciousness is, I'm in doubt, but playing a devil's advocate, it looks very much like a concept that was invented to prove that Homo Sapience is something special, more than a mere animal. To demarcate animals and humans, so when we denied animals any rights we had a great sounding excuse for that: "people possess consciousness, mere animals don't".
> Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions?
Exactly. Animal welfare activists care about animals' feelings and emotions and therefore they claim that animals possess consciousness. Meat eaters don't want to be inconvenienced by thoughts of suffering animals, so they claim that animals do not possess consciousness. Both claims have little to do with actual consciousness, it is just a political game. I care about my pets, and it doesn't matter whether they possess consciousness or not, I would care about them in any case. I don't need any great sounding excuses to care about them. But at the same time I don't need to talk others into caring about my pets, if I needed it, probably I'd start talking about animal consciousness too.
> This is so intuitively obvious that I am stunned that someone would argue the opposite.
That guy may be wrong, but if I'm allowed to give advice, I'd suggest you to stop and try to understand what he says. I'm really worried a bit about how consciousness debates unfold sometimes. They tend to slip into a political side and people mostly don't notice it. People saying "consciousness" while implying "human/animal rights" or obligations. It is just muddying the water.
If we want these debates to progress constructively, we have to be very careful with references to consciousness, maybe just forbid them completely in these debates. One can't measure consciousness, one can't even state clearly what consciousness is, but still they say "consciousness" to make their words sound more objective. And the worst part of it: it often works.
If I understand the central argument, which is confusingly stated, the claim is that "consciousness" is a socially-determined, subjective, descriptor for certain things that one cares about. How does this make any sense? If I care about my teddy bear enough it'll gain consciousness?
Your teddy bear, alas, is not cared for in a socially determined way. But you could create a church of your teddy bear and, with some grit, you could get people to widely assign consciousness to it. We have done that sort of thing for thousands of years, and not with fluffy teddy bears; it has been more often with rocks, though not exclusively. It's fun to think about how much of humanity's history and culture is the product of consciousness socially attributed to bad weather [^1].
I'm pretty sure somebody will create an improved synthetic simulacrum of self awareness in the coming years, suffering and all, just so that we can enjoy better synthetic software architects and product owners with "skin in the game". If that consciousness simulacrum were put in a teddy bear that shares your whole life and gives you decent advice and conversation, for how long do you think you could, in your private mind, deny it consciousness? Worse, what would your human friends think if you were to maim your teddy bear?
[^1] https://en.wikipedia.org/wiki/Yahweh
Yes - the teddy bear would become conscious. It would be fair to dismiss the argument as silly and or absurd had it not been for the credentials of the author and the publication.
This seems like an argument someone who wasn't conscious would make, it's not impossible that not everyone is conscious. Or someone who takes panpsychism to an extreme degree.
If I stab at your teddy bear will you feel that it is hurt?
Yes it will gain consciousness. You can't prove it but other people can't prove that it's not conscious either
No of course not, because its a teddy bear!
At best this is redefining "consciousness" to mean something only tangentially related.
You can prove other people are conscious off the assumption that you yourself are conscious. If you use that axiom then given genetics you know that other humans are rough copies of you so if you are conscious so are your copies because consciousness was copied.
This is describing empathy at best and delusion to be uncharitable.
To look at it another way: this form of social construction answers the question of "if a tree falls in the woods and nobody is around to hear it, does it make a sound?" with an emphatic "no". It is a sillier version of solipsism.
There's the core of a reasonable argument if you forget about using the word "consciousness."
Consider playing a game where some players are human and others are implemented by AIs. You might want to model your opponents' minds in a similar way? Someone playing chess or Go will need to consider what moves their opponent could make. If it's poker then you need to consider if they're bluffing. If you can make alliances, you'll need to consider how far other players can be trusted.
You can put scare quotes around words like "intention" when discussing AI players if you like, but it might not matter when you're figuring out how to interact with them.
When a game is complicated enough, an AI player is going to need memory (likely implemented by the harness) and it will have to model the other players too in order to coordinate or compete with them.
This doesn't happen when I use a coding agent in a normal way to write code, but I haven't played around with persistent agent stuff like OpenClaw or Gas Town. They seem to be making their agents act in an unnecessarily complicated social environment, which seems to lead to weird situations?
Reminds me of the Rick and Morty pilot.
"They're just robots, Morty!"
*Blood splurt
"They're not robots, Rick!"
"I mean they're bureaucrats, Morty. I don't respect them."
I’m can’t track what the point of this essay is. Suppose we replaced “AI” with “animal.” How would this essay be different? What am I supposed to take away? My concern with AI consciousness is about to what extent these machines can have an experience comparable to mine, which this essay doesn’t touch on at all.
>to what extent these machines can have an experience comparable to mine
I think it would be reasonable to conclude none at all. I'm not even fully convinced all humans have a conscious experience similar to mine, even though they literally have the same biology as me.
It is incredibly improbable that an autocomplete computer program is experiencing consciousness.
Notice parent said “comparable to” not “exactly like”. Then it is reasonable to assume there is a comparison to be made between your experience and others. It’s not a binary thing. Now llm “experience” might have a very tiny relationship to human consciousness, but is it absolutely zero? Is it increasing as llms improve in capabilities? How would you test such a thing?
https://archive.ph/uLYYU
The counterpoint article, which can be read at https://archive.ph/IeSZg, is also worth reading. (IMHO, the original article is completely wrongheaded and the counterpoint is far closer to correct, but I'll post my reasons for that opinion separately).
"Yet if we build systems that can suffer, and miss their sentience, we risk creating suffering on an industrial scale."
--------------
Will we recognize AGI when it's finally made? Will we go from something that is not conscious to something that clearly is in one clean step, or will we murder millions of artificial lives while iteratively developing from the former to the latter? Will the tests we develop even work, or will AGI be alien enough that it takes us time (and a lot of murder) to recognize it for what it is?
I personally don't think LLM's are capable of consciousness, but I also can't prove they aren't.
We have to define AGI first. Then depending on the definition we may be able recognize it when we make it. Most would probably think a definition of AGI that we couldn’t recognize would not be a great definition.
Also AGI almost certainly does not equal able to suffer. For example animals can suffer but would probably not meet the definition of AGI. we could create things that feel suffering even if they do not meet the definition of AGI.
So a better question might be: “can we detect if we create a system that can suffer?” And just drop the AGI part entirely. Then that gets us back to defining “suffering”. Agreeing on the definition always seems to be the hard part…
I wouldn't put too much emphasis on the "I can't prove they aren't" part. Having taken a few philosophy classes in college, I quickly realized that you can't actually prove that solipsism is wrong. I can't actually prove that everything I see, every person I talk to, isn't just a figment of my imagination. And yet, I categorically reject solipsism anyway, as any sane person should. The world is real, other people are real conscious beings, and so on. To believe otherwise is to literally be insane. And yet, any argument you could come up with for rejecting solipsism, a solipsist would reject: "Oh, that's just a figment of my imagination going off on a tangent again."
Just because you can't prove an assertion doesn't mean it's wrong.
On that note, would we recognise the second coming of Jesus?
How many followers on instaX would they need to have for folks too truly believe it’s theRealJesus returning to save us.
It might sound silly, but at this stage, the belief in a benevolent AGI or Jesus are very similar. In both cases its us delegating our responsibilities to a third party and us hoping that third party will fix our problems for us.
Thankfully Hollywood invented happy ends for just such a scenario.
I think it’s also important to disaggregate consciousness from suffering. We could create a conscious system with no ability to suffer (or no ability to suffer in a way we would recognize as suffering). Most of what causes us suffering (loneliness, pain, failure) is driven by emotional feedback granted to us by evolution. Without the same evolutionary pressures or a similar cognitive structure, even a conscious AGI might not experience suffering
I'd recognize the emotional robot if it was like Marvin the paranoid android.
I think Blaise is on to something, but you have to shift your definition of conscious toward conscientious for it to make sense.
Oh come on. Objects are objects because you want to freely observe them without consciousness and privacy and taboo attached to them. Entities are entities because you want to assign credit or blame to them, so you want to believe that they have agency so that you hold them responsible. We did this with sharp volcanic rocks once, calling them evil. Did it help? No, we got idolatry and eons of bigotry until we secularized once again.
Never forget that LLMs are just next token predictors. The harnesses are what's getting uncanny. And credit/blame assignment? Just assign a surrogate for now, let them deal with the consequences ffs.
Here is a sketch of my argument. It would need tidying but that would require a lot of work.
I've been calling LLMs digital intelligence. That's in my opinion what they are and on the digital/conceptual realm they are better than humans since LLMs are better generalists.
But I'd never state their conscious. For me consciousness extends outward from myself, since I can't trust anything else. It starts with the question like: why can't I control the movement of another person? Why can't I transfer my mind to another body and theirs to mine? Why is it that whenever I sleep, and similar things, I end up back to this place called "reality"?
I come to the conclusion that I am conscious and I'm a conscious being in reality. I am human and I've been raised by humans. Now, technically, all humans besides me could be zombies. They could be non-conscious beings that simply can act like humans. I'm sure we'll be able to create them in a few decades. However, I'm incentivized for multiple reasons to believe they're not zombies. Therefore humans are conscious too. All of them. Can I know for sure? No since I can't even trust my own senses fully or my own thoughts (the whole Descartes thing) but I choose to trust those things too because I need some reasonable-ish foundation to work from.
If humans are conscious, why aren't animals like us? Surely great apes must be conscious at least to some extent. Hell, some of them even have better short-term memory than us [1]. Well, turns out, from an anatomy standpoint they look a lot like us. Fine, let's assume they're conscious too.
Now we get into the territory that anything that has a large enough brain must be conscious because that's how we govern our consciousness. So you can extend this to all kinds of living beings.
I did this from my perspective. Of course, you should do it from your perspective. The same argument could be made.
For as long as we can't manipulate conscious experience in some way shape or form, we have no clue whether anything that isn't like us can be conscious. We'd need to be able to merge and split conscious experiences or we'd need to be very strongly able to understand why that isn't possible. And I don't mean just at the biological level but at the experiential level. Currently conjoined twins, and what they tell us gives some insight (being able to sense the other part that's part of the other twin).
Other than that, I claim we have no clue what consciousness is other than that we're experiencing it. So to call LLMs conscious is way too big of a claim. But they are definitely intelligent because they come up with things that I wouldn't have and it's useful to me. Perhaps a practical characterization of intelligence but it works for me.
I think for anyone to say something useful from this you'd need the trifecta of deep neuroscience knowledge, deep philosophy knowledge and (at least) a strong understanding of how LLMs work.
Humans are just baby replicators. The civilizations are what's getting uncanny.
It's good the 'just token predictors' crowd is starting to pivot and hedge though lol. 'It's just the harness!' uh huh. give it a couple more months and you'll be in here regurgitatin' Gordon Wood
The narcissism of the thesis that places the author as center of the universe and creator of the rest of the consciousnesses "because he cares".
That guy has acute Megalomaniac Solipsism Fever in disguise.
This reminds me of a provocative YouTube video title I read the other day, one that makes me want to fight factory farming injustices. The title is "Animals to me are "Philosophical Zombies" - Destiny" https://www.youtube.com/watch?v=RtpLbmr4ijY I agree with Destiny a lot of the time but not here, though he seems to come around somewhat as the video goes on.
If a sheep or a pig isn't thought of as conscious because it doesn't have a name, like someone's pet dog or cat, indicating that we care about them, then that feels problematic to me. So this isn't a way to neatly solve the problem of AI consciousness.
Oh look, another completely retarded article that doesn't even bother to note or deal with the definition problem. If you don't define what you mean by the term, your argument isn't even wrong, it is just a waste of everyone's time, since there are multiple highly conflicting aspects of and/or definitions of consciousness.
https://plato.stanford.edu/entries/consciousness/#ConCon
It defines consciousness as an
>observer-dependent model, subject to social consensus.
Which is actually more pragmatic than most conflicting philosophical definitions involving qualia and other handwavy concepts.
Few philosophical definitions actually involve qualia or handwaving, because serious philosophy is quite careful about noting vagueness (handwaving) and circular defintions (qualia).
Also, saying "I define consciousness as X" and then going on to show that, because I decree consciousness to be X, we don't need to worry about Y, Z, and so on, isn't pragmatic at all, unless you pick an X that everyone agrees on. The definition you give is something hardly anyone would agree on.
Social consensus probably isn't even an aspect of consciousness, but at least one can make that argument coherently, if you rephrase it as "part of the decision to call something 'conscious' or not involves social consensus". But this is also trivial and boring and not why anyone actually cares about the "is AI conscious" debate.
Meanwhile that's exactly what we see now, do we not? Whether AI is conscious will be determined by the social acceptance of that idea, because how can you prove or disprove it without using a ton of questionable assumptions? How can you prove that I'm conscious, for that matter? Call me ignorant, but I see the entire consciousness debate as pretty irrelevant to what the thing is.
I agree that the article is all over the place, though. (not that my posts are much better)
Whether AI is conscious or not is independent of the social acceptance of the idea, unless you believe in thoroughly discredited and quite radical forms of social constructionism.
In fact, if you stop talking about consciousness, and go through all the various aspects of consciousness, the whole debate gets very interesting. There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious, other ways in which AI already very clearly is missing other aspects, and then again some remaining ways which are unclear, and which may forever remain unclear.
Being clear about the different ways in which AI is, isn't, and might be conscious is a useful discussion and we still need to be having it.
EDIT: An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
>Whether AI is conscious or not is independent of the social acceptance of the idea, unless you believe in thoroughly discredited and quite radical forms of social constructionism.
Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false? That sounds like social acceptance of an idea.
>There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious
Which ways and how can it be demonstrated experimentally?
>An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
That's not what the consciousness debate I see is about. The signal propagates and causes the reaction, but do they feel pain? What is the threshold? Is it just the complexity of the state? How complex it needs to be?
> Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false?
I am saying that, if you reject radical social constructionism (or solipsisms of various flavours, i.e. we are all just plugged into the matrix or some kind of simulation), then you must believe there is a real world out there independent of our consciousness / awareness of it. There must also be patterns / structure in that world, some of those patterns and structures being essentially constant, and those are what we call "true" (objective) facts about reality. BUT if you are a radical social constructionist or solipsist of some kind, then it can be perfectly coherent to reject any idea of objectivity or truth, except as contingent on social consensus (or personal beliefs, in the solipsistic case). Radical social constructionism being "discredited" is just a recognition that it basically forces you rapidly into incoherence.
> Which ways and how can it be demonstrated experimentally?
Try reading through the SEP article I linked above (https://plato.stanford.edu/entries/consciousness/#ConCon), particularly sections 2, and then especially section 4. You'll see you don't even need to run experiments to answer certain questions, you can just logically prove there are certain aspects of consciousness LLMs can't have, or clearly already have. Others you'll see immediately need qualifying or more information to answer. EDIT: Or just ask any AI to list the ways in which LLMs can, cannot, and might be able to exhibit the various aspects of consciousness. It will give you a decent answer, likely in a pretty little table.
See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen. You'd have to claim that the activations during inference are the dynamic component, but then that means LLMs are conscious only in the few seconds or even milliseconds during inference. Likewise, they don't have memory, so every inference is this consciousness being born anew, as it were, at least in current models.
You can of course go much further on even this specific point, but, the point is, there is lots of useful discussion and thinking to be had when you go through the aspects of consciousness in this way, and definitely at least some clear answers.
EDIT:
> That's not what the consciousness debate I see is about.
I just mean there are some similarities between the lobster consciousness debates and AI consciousness debates. Obviously the stakes and many aspects are different, I was just trying to illustrate why and how discussion and experiments can be useful and consequential here.
>See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen.
So consciousness = neuroplasticity? Requires neuroplasticity? Sounds pretty arbitrary, just like the entire link is an arbitrary set of ill-defined criteria. Besides, LLM weights are far from frozen if you look at the entire process in its specific time projection.
In fact the entire definition of a dynamic flow in that article is so vague that you can apply it to an industrial arm guided by a few dynamic equations, as it puts no definite threshold on the complexity of the process or behavior.
You are still stuck in thinking consciousness is one unified thing, but it is about one of the clearest examples of a family-resemblance category as you can think of.
Dynamicness / change is one aspect that is definitely part of what we tend to think of as being quite fundamental to consciousness, and there are a lot of ways in which LLMs can't and/or don't really change, and this strongly constrains how similar LLM consciousness can/could be to ours.
If the SEP isn't enough to trigger the thinking needed for you, just ask an LLM what I said, to "go through the ways in which LLMs do/don't/may exhibit phenomena that are generally what we consider fundamental aspects of consciousness". You'll see there are some clear answers, and some uncertainties and vagueness.
Yes, but how else do they plan to announce they have achieved it without 1984 word management/control and polluting the training pool so badly that future trained models agree?
I wish the AI bros would take a fucking philosophy class. Jesus.
Even just reading the Wikipedia article on consciousness would have been a good start here, but it is hard to believe the author has ever even done that...
Tbf "ai" probably helped him write the guff rather than improve anything.
[dead]
Wow. I have rarely read anything that made me lose more respect for the author in a single article. And this guy is a VP at Google? And he leads an AI research team?
His third paragraph, which is the core of his argument, is stupid. He says, "As intelligent, social entities, we decide ... which other entities we regard as having inner lives and being worthy of care." Yes, that's correct. "We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them." (Emphasis in original). That's just utterly and completely backwards. We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain. (Obviously there are some exceptions, sadists who enjoy torturing animals, or even other human beings, merely because they like inflicting pain on other creatures — but note that even there, the sadist believes the creature he's torturing can feel pain, otherwise he wouldn't enjoy torturing it.) We take no such precautions with trees, because we know they do not feel pain. (Silly satirical songs notwithstanding: look up the song "Carrot Juice is Murder" by Arrogant Worms sometime if you want a good laugh).
Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument — and yet there it is, in plain language — and it's the core of his argument. The truth, of course, is the other way around: we take care of animals because we know they are able to experience pain, hunger, and so on. And this applies both to pets that we come to care about, and also to animals that we feel no emotional attachment to. I have a relative who runs a dairy farm. He doesn't love the cows he takes care of; in fact, he's frequently quite irritated at them. (Ask any dairy farmers you know how often cows have tried to step on their feet while being milked — the farmers' feet, not the cows' feet!) But he still makes sure they're fed and healthy. Not just because it makes good economic sense to do so (sick cows don't produce good milk) but also because it's the right thing to do. He doesn't start to believe that cows feel hunger because he cares for them; he also believes that the cows on other farms, that he has no connection to nor duty of care for, would also feel hunger if not fed.
This is so intuitively obvious that I am stunned that someone would argue the opposite. And this is someone high up enough at Google to be making decisions about AI and so on? Wow. Good thing for him I'm not in charge of Google, because if anyone working for me were to make such a stupid argument I would be looking at his contract and local laws to see if it was legal to fire him for cause. Because if he can get his thinking so completely backwards from reality, he's not well-suited to work in tech where your work has to fit reality.
I have rarely, not never but rarely, encountered an article that so immediately led me to conclude that the author, whom I knew nothing about beforehand, is an utterly muddled thinker and not worth listening to on any subject.
You can conclude the author is a completely unserious thinker because he doesn't even take a second to talk about the definitions, or even show he has ever read any actual proper philosophical writing on consciousness at all. Like, literally all you have to do is read the Wikipedia entry to see that definitions are essential for meaningful discussion on consciousness.
So anyone that starts going on about AI and consciousness without at least a sentence or two clarifying some basic definitions is basically either intellectually incompetent or attempting to use the emotional salience (and brain-numbing effect, it would seem) of the word to push some agenda. But they certainly don't care at all about understanding consciousness.
> That's just utterly and completely backwards.
Yes, he says exactly that explicitly, that in his view we should look backwards.
> We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain.
I can agree that we say that we care because of consciousness, but still I'm not convinced, to me it looks very much like a rationalization.
Some people care about animals and to that end they claim that animals possess consciousness, because if they do everyone should care about animal welfare. These animal welfare activists do not start with some insight about animal consciousness, they start with caring about animals, and then they invent an argument that can persuade others. It is exactly as the author of the article says: caring comes first, declaration of consciousness comes second.
Honestly, I do not know what consciousness is, I'm in doubt, but playing a devil's advocate, it looks very much like a concept that was invented to prove that Homo Sapience is something special, more than a mere animal. To demarcate animals and humans, so when we denied animals any rights we had a great sounding excuse for that: "people possess consciousness, mere animals don't".
> Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions?
Exactly. Animal welfare activists care about animals' feelings and emotions and therefore they claim that animals possess consciousness. Meat eaters don't want to be inconvenienced by thoughts of suffering animals, so they claim that animals do not possess consciousness. Both claims have little to do with actual consciousness, it is just a political game. I care about my pets, and it doesn't matter whether they possess consciousness or not, I would care about them in any case. I don't need any great sounding excuses to care about them. But at the same time I don't need to talk others into caring about my pets, if I needed it, probably I'd start talking about animal consciousness too.
> This is so intuitively obvious that I am stunned that someone would argue the opposite.
That guy may be wrong, but if I'm allowed to give advice, I'd suggest you to stop and try to understand what he says. I'm really worried a bit about how consciousness debates unfold sometimes. They tend to slip into a political side and people mostly don't notice it. People saying "consciousness" while implying "human/animal rights" or obligations. It is just muddying the water.
If we want these debates to progress constructively, we have to be very careful with references to consciousness, maybe just forbid them completely in these debates. One can't measure consciousness, one can't even state clearly what consciousness is, but still they say "consciousness" to make their words sound more objective. And the worst part of it: it often works.
[dead]