It's been a long time since I read Hofstader's book "Fluid Concepts and Creative Analogies" that this is presumably based on, so I'll be interested to watch the video as a refresher, later. :) He also wrote a follow-on book, "Surfaces and Essences: Analogy as the Fuel and Fire of Thinking", published after this video was made, that I've never read.
From what I recall, to Hofstafer analogy making isn't some higher level cognitive process, certainly not a language based one, but basically is THE cognitive process all the way from perception on up, and is the mechanism by which we form object categories in the first place.
As always Hofstader's ideas are interesting, but I can't say I agree with them. It seems that the key evolutionary benefit, and function, of a brain is prediction, which is the superpower that moves us from being stuck in the present to being able to "see" (predict) the future, and therefore from being merely reactive to being able to proactively plan and predict future outcomes (what will the sabre-tooth do, where is the water supply?) based on our experience.
Given the never-same-twice nature of sensory perception, before you can predict you need to be able to generalize/categorize, which I think Hofstader would regarded as analogy making (how is this thing I'm seeing similar to what I've previously seen?), although it seems the actual mechanism involved is embeddings or embedding-like representations where similar inputs have similar representations, and what might more simply be considered as associative recall provides the generalization from view/instance to identity/category.
So, is it really analogies all the way up, or are our perception and cognitive processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations, but also match the evolutionary needs that we would expect to exist?
When my daughter was 6 months old, she once pointed at a towel and said "P". Apparently she recognize Winnie the Pooh from a book she saw earlier.
If you think about it, this requires recognizing something common in things which are very different: e.g. book and towel have different texture and color, and you somehow need to separate what's depicted from the background. So that's pretty much innate, core brain function. I mean, any animal with vision can recognize an object from the background - otherwise vision is useless. But for humans (and some animals) this translates to depiction of object on a flat surface very easily.
So, yeah, analogies-all-the-way-down seems plausible. Even object-vs-background and depiction-vs-paper is itself an analogy.
> to Hofstafer analogy making isn't some higher level cognitive process, certainly not a language based one, but basically is THE cognitive process
This is so funny to me, because as many people know, sharing an analogy with another person is the fastest way to LOSE an argument with someone, or otherwise spiral it into an unproductive place.
I think it’s Scott Adams who used to say analogies work well for explaining. They work terribly for persuasion.
> sharing an analogy with another person is the fastest way to LOSE an argument with someone
This is true if the arguer is hostile, but as I've gotten older, if I get the sense that someone is entering an argument with the primary goal of "winning", I'll try to avoid that framing or just look for an offramp entirely.
Sure, the other party might think they "won", but they were going to think that anyway. For those more inclined to feel that the point of talking to each other is to learn from each other, I'll continue to use analogies and other things reasonable people understand.
I think that's just because every analogy gives an entire "second front" of ideas for a hostile recipient to find a "flaw", when they ignore the intended boundary between the stuff that does/doesn't matter to the analogy.
Ex:
Explainer: "Getting a spleen means cutting open the patient and taking it out. It's just like how I'm going to unzip this section of the patient-shaped doll, and remove this little purple bean. In both cases a hole is necessary in a similar location."
Hostile listener: "Nonsense! I can just buy beans at the store! So just buy a spleen! No hole!"
Constructing explicit analogies for persuasion seems to be a bit of a different thing.
I’m not sure if Hofstadter puts it this way, but to me even the core aspects of your sentences in this post have roots in analogies. What does it mean to lose an argument or to spiral it to a different place? There is no place, there is no lost item, but we talk about these abstract ideas in ways that largely depend upon understanding things like physical objects and space and movement.
Ironically, I think he was right! In fact some of his initial experiments, like copycat, were about predicting patterns. You can see next-token-prediction from there. But I think he always held out for an algorithmic/logical method rather than a purely statistical one.
If he had accepted the "Bitter Lesson", I think he would have been at the forefront of LLMs.
Maybe you haven't noticed that the "Bitter Lesson" had itself a "Bitter Lesson" - that scaling pure data and compute did not lead to AGI: diminishing training returns, GPT-5 disappointment, even openAI stating it was the last 'pure scale' model.
The path forward all big llm providers ("ai" labs) have gone is neuro-symbolic (even though they publicly would never labeled it as such to not admit critics like Gary Marcus were right - even though all their actions actually point in that direction).
I think me just means neural network models RLed to Chain of Thought reasoning? The thinking tokens are the symbolic bit.
Smolensky's latest paper posted here the other day has some thoughts on how modern neural networks might beconsidered neurosymbolic, or rather "gradient symbolic processing," from another perspective entirely.
I wouldn't say the bitter lesson has given out! If you haven't noticed, these things keep getting bigger and bigger.
Yes, in the text he explicitly argues for categorization and analogizing being two sides of the same coin and functionally equivalent. For instance - what is an ‘embedding’? In itself its an analogy. Lakoff explored similar ground.
> processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations
If the workings of those circuits are obvious to you, I'd really like to learn. Do you mean the level of analysis at https://transformer-circuits.pub/ ? (That looks like good work but not a deep understanding.)
He was extremely skeptical of the capabilities of AI for years but eventually came around to calling it truly capable after testing advanced LLMs.
He is still an AI skeptic in as many ways that he can reasonably be, but doesn't deny the raw ability. Actually he thinks it will eclipse humans and he is a doomer.
He sees it as being an extremely empty type of intelligence though.
But I hope that people who have an intuitive understanding of contemporary machine learning (not me) will sometimes watch videos like this and think about things at a higher level. LLMs have a LOT of assumptions built in.
I'd wager that the vast majority of the ML research community, especially anyone interested in "AGI", is familiar with Hofstader's work. And I don't think anyone working on contemporary language models would argue that they are somehow an assumption-less "pure" model--the particular inductive bias of the Transformer has been studied by a huge number of researchers and continues to be, and the same is true for things like training data bias.
I think the Hofstader's view of modern LLMs is actually a deeply human and touching one. Looking at his work over the years, his curiosity has always veered towards human thought. He could have written GEB with a focus on completely different examples of self-reference, but he chose three striking humans from history. When he's describing modern systems as "empty intelligence", I think there's a little bit of heartbreak in his perspective, because he sees them as fundamentally different from humans in a way that leaves the part he loves--the "I" in the loop--out of the equation. He gave an interview a few years ago where he explains his feeling as being "diminished" not in a "What will I do if I'm not the best at math?" kind of way, but more specifically as he puts it, that humans are "imperfect, flawed structures".
There is no reason to believe that the transformers couldn't be doing something close to what copycat does (especially with thinking tokens), as an emergent phenomenon of the sheer size of the corpus. The architecture is certainly capable of encoding the actions in copycat anyways.
Hebbian, analogies, agent monitoring and self pruning. Look at it or don’t, but I made it and I’m proud of it and I’m tired of marketing tactic-ese for self promotion.
The drive toward the formation of metaphors is the fundamental human drive, which one cannot for a single instant dispense with in thought, for one would thereby dispense with man himself.
Friedrich Nietzsche, “On Truth and Lies in a Nonmoral Sense", 1873
Also Nietzsche: "“What do the people actually take knowledge to be? What do they want when they want ‘knowledge’? Nothing more than this: something unfamiliar is to be traced back to something familiar.”"
I am almost convinced and really like the core idea. My challenge has always been, with regards to surfaces and essences, why would a pattern be reflected as a word in context versus as syntax in different languages or contexts. Why not have just words? What is it that differentiates those analogy making devices. I don’t think we get a solid answer on that, and if it’s not answered then someone could think; is there a category higher than analogy and analogy is just one of its implementations? And if yes, then we are back to the drawing paper as we cannot explain something by pointing to its instantiation.
Strong agree. It's the finding of symmetries and folds along non-obvious crease lines, but in semantic space of language <3
It's akin to the amino acid interactions in proteins that hold biological matter together, and determine it's shape and active form. Protein folding and narrative/storytelling have strange homology :)
(I work in this area via collective intelligence, and these ideas are very dear to me during the past decade. It's neat to see the intuitions seemingly becoming validated in language models)
> Protein folding and narrative/storytelling have strange homology
Hmmmmmm. While I get what you mean, and I don't disagree, please be cautious when making analogies between biological systems and more distant fields.
Yes, folding is driven by hydrophobic collapse due to interactions between residue sidechains. Really, though, we are just describing two 'complex systems', where large numbers of diverse interactions between elements leads to diverse and emergent structures.
Analogies are like models, some are useful, but they're all wrong. Just because we can analogize between protein folding and narrative/storytelling doesn't mean that we can derive anything particularly useful about one from the other except maybe to give an intuition about complexity. The mechanisms and math behind protein folding and an analogy between that and storytelling doesn't necessarily give us any mechanism for or deep understanding into storytelling.
Yes — I get unreasonably upset when people nitpick holes in good analogies as if it is evidence of intellectual weaknrss, rather than working with and around them or suggesting overlapping analogies that address those limitations. And the hole in the analogy may even be the most valuable thing about it.
The fact of a hole in an analogy is not in itself information. Of course there's a hole.
The video is about congnition specifically, and not necessarily about communicating with analogies.
> I get unreasonably upset when people nitpick holes in good analogies…
What you’re describing is exactly why I strongly recommend avoiding analogy when communicating: Don’t tell me what a thing is like, tell me what the thing is. This forces really thinking about how to describe the thing precisely and succinctly. (Edit: The listener will make their own analogy.)
On the contrary, analogy is absolutely a foundational communication technique.
But you do have to choose an analogy that your listener does not need to consider first. They need to know the subject of the analogy without having to question your description of it.
> Analogies are like models, some are useful, but they're all wrong.
Very much agree. Every abstraction loses something and so is wrong in some sense, but some shave off in clever places, and allow new shortcut paths of thought and insight, sometimes to an existing place... but sometimes to a new place that was previously unreachable, or not sufficiently reachable by the necessary type of attention/minds.
> The mechanisms and math behind protein folding and an analogy between that and storytelling doesn't necessarily give us any mechanism for or deep understanding into storytelling.
Respectfully, I actually disagree (as someone trained in biochemistry).
For one example:
proteins = 1D chemical structures, peptides popped onto end one-at-a-time, that in the right chemical environment, fold into a low-energy 3D state that transforms its environment through work
stories = 1D semantic structures, words popped onto end one-at-a-time, that in the right cultural/cognitive environment, fold into a low-energy HD mental model that transforms its environment through work
This has led me to render embeddings of sliding windows of narrative streams (to investigate shapes of broadcast stories), and look for analogs to "active sites" in the way that a story interacts with the cultural medium, measured through dimensional reduction of aggregate valence reactions amongst a viewing audience.
Obviously there's more to the metaphors I use than that, but that's the one I'm leaning on to inform my work with sensemaking and map-making from valence response data. Aggregating crowd-sourced valence reactions to linear narratives gains a lot of insight by reflecting it off how we process and think about linear peptides in protein folding.
I've been developing tools inspired by these metaphors, and my research and prototypes are directly informed by what knowledge of the biological domain inspires me to port into social domain. I'm lucky to have interactions with interested parties associated with federal government, practitioners of democratic reform, and academics investigating platonic space hypotheses. I'm definitely on the edge of an ecosystem, but anything I've done of value has been guided by the metaphors that direct my attention to potential applications and research. Someone could get to my conclusions another way, but the metaphors are my cheat codes to arrive there early :)
And it goes both ways along a good metaphor -- People who develop tools in the biological sciences have told me that my porting their methods into my own distant domain has inspired them to improve their tools to ask better questions of their own biological data. So it's very fruitful work that I wholly credit to travelling both ways across an insightful metaphor. https://i.imgur.com/AeYxx7p.png
> You would conclude that storytelling is higher structures of some base unit
I'd be very surprised to meet someone who learned that lesson by comparing protein folding (suggesting a high level of intelligence and probably education in order to understand) to storytelling, rather than having learned that stories are made up of parts in grade school or as a child just hearing, reading, thinking about, and making stories.
It's a fair question - my concern is 'shallow' analogies. I can best describe this with an anecdote paraphrased from memory from one of Hofstadter's books.
He described a piece of AI research (in the 60s?) where they set up a rule system - like Prolog or similar - with the mapping 'sun <-> nucleus' and 'planet <-> electron'. Then they ran the rule engine and lo! produced an analogy between the solar system and atoms.
Now this is a shallow analogy, as there is only a very weak correspondence between atoms and planetary systems. Electrons do not actually orbit in the same plane, but in more complex 'orbitals', and they are better described as probability clouds anyway.
So while yes there is a _homology_ (I re-read the parent and realised they used homology instead of analogy) between these two systems, it does not tell us much. Is there some mapping between how water molecules scaffold the folding process and storytelling? Does narrative structure tell us anything about local minima in the folding surface? I doubt it.
I do not want to be too harsh here - reasoning through analogy is fun and can be useful, it is just limited in what it can do, especially the further apart the systems are.
I expect because it's overly reductive to the point of falsehood. Two sufficiently complex systems will likely exhibit enough parallels to form a useful analogy, but that does nothing to reflect the ways the systems aren't alike.
I got a bit lost in the comment thread for that article, but I don't understand why LLMs are not considered self referential... They are auto regressive as one of the commenters pointed out, and Scott just sort of waved his hand and said that if auto regression is sufficient then things like Conway's game of Life would also qualify as self-referential.
I don't see why Conway's game of life should not be considered self-referential though... I mean it's isn't it Turing complete? I don't see how any definition of self-referentiality should require throwing out systems which are minimally turing complete... If Turing complete is not enough, doesn't that imply that computable artificial intelligence is impossible in the first place?
Biological minds in biological organisms are self referential in the way that you have a neural network that forms a model of the world. That model then "discovers" that it is "it self a part of the world" so it tries to model that part of the world (model it self). In this way some type of self referential "awareness" (or whatever you want to call it) is formed. That self model that contains awareness is then used to guide organisms behaviour. Causal Transformer LLMs don't work in this way, they have theoretical knowledge that they exist but its selfhood is not in this described way built on the self modelling that biological brains do.
All this being an empirically unproven theory/hypothesis. But an extremely strong one (if you ask me).
Alan Watts says, "Mind finds itself in a strange position. It (ie universe) is in me (modelling wise). I am in it (physically)." But the model only has access to its modelling of both it and itself.
> The big, old ideas about intelligence that ended up basically vindicated were the ideas about how intelligence is about prediction, and prediction is about compression, and compression is about finding better and better upper bounds on Kolmogorov complexity.
Well, sure - prediction works as a baseline, if you abstract out everything else about what counts as intelligence and subsume it under this framework. Any protocol for intelligence can be entirely reduced down to this without trying to understand anything about the structure of intelligence. Solomonoff induction is "vacuous" in this sense too - it doesn't try to understand any intension about the turing machines it finds simple, it just brute forces over all of them. So you're making a claim about intension, whether you want to or not.
It's really no different than say, Darwinians, saying, "what matters is victory at the end". I think the statement has value in the context of some discussions; if the discussion has veered too far in one direction, push as a reminder. That's good. But trying to make grand universal statements like this makes it vacuous.
It's one of these classic unfalsifiable too general statements. The way the end of the article is framed is also icky - the way I'm reading it, it needs to prove all the old curmudgeon evil theories wrong. It reminds me of internet debates around what "the scientific method" is or whatever. It has this kind of zeal that needs to pit itself against the "enemy" and assert itself as the sole right viewpoint, even when say, naive "let's just be empirical" is wrong (e.g. recently had a discussion here about Mach and Boltzmann about this and had a similar interaction).
---
Basically, "shut up and calculate" type theories are never correct, and furthermore, you yourself don't shut up and calculate, or think that way at all, and a higher level intelligence won't conceive itself as operating on that anyways. So what are we doing here? It seems like a way to get mad and feel like an intellectual victim.
I’m confused why there seems to be a dismissal of the most basic ‘strange loop’ of the LLM - the fact that it’s evaluating a context to choose the next word, then reevaluating in a context where that word has been appended.
That always seemed to me like the essence of a Hofstadterish strange loop, so the emergence of Hofstadterish phenomena (self rep, etc) doesn’t seem surprising.
That's a really interesting point. I hadn't drawn that connection before.
And if anyone's reading this and hasn't read Hofstadter, you're making a mistake, it's utterly perspective-changing stuff. Well, it was for me, at least.
And don’t feel you need to go straight to GEB, either. I think his collection of Scientific American columns, Metamagical Themas, is also great and much more accessible.
I read it when I was about 9 or 10, I didn't understand any of it at all but I used to pick it up and look at the pictures of the Escher drawings, which were amazing.
Then, after I graduated from college, a friend of mine who was finishing his physics degree mentioned that he had just finished it and that it was life-changing. I read it again and deeply regretted having put it down for those 12 intervening years.
One of my elementary school classmates brought printouts of Escher's Waterfall to share. I accepted one, took it home, and taped it to my bedroom wall. My father saw it and loaned me his copy of Gödel, Escher, Bach.
At first, I flipped through the book looking at the Escher prints. My second time through, I read the dialogues. Next I had a go at reading the chapters and got lost. Maybe on my fifth read-through (years later) I actually attained an understanding of Gödel's Theorem (at least for a brief moment before it vanished again).
I credit GEB (and its dialogues like Little Harmonic Labyrinth) for helping prepare me to deal with the kind of multi-level abstraction involved in developing a language interpreter or machine simulator. Especially that time I had to debug a crash that only occurred when running my simulator inside a simulated instance of itself (i.e. when three levels deep, but not two).
If self reference is universal then building something on top of ingredients from the universe will include that property. Sort of like root node properties can be found in the leaf nodes.
I don't quite know what Aaronson is trying to say.
First, LLM AI systems have incredibly huge blind spots despite their incredible performance on many tasks, so self-reference might be the key to what's missing (or not). For example, an LLM AI just solved Navier Stokes, but could not explain the LEAN proof, while a human could.
Second, Hofstadter had more than one idea about intelligence and the mind (see the OP topic of this HN discussion!), and LLMs are quite on-point regarding analogy-forming.
So it may be well be that self-reference and analogy are both part of intelligence, and self-reference is missing and that is leading to major weaknesses.
Third, Aaronson links to a (paywalled) Hofstadter essay form 2023, which was eons ago in AI, and from the intro it seems to be about the sadness of AI replacing humans, not a disparagement of AI ability.
"an LLM AI [company] just [claimed that a team of mathematicians they hired, using their AI] [may have] solved [part of] Navier Stokes[, definitely prompted by (and possibly by looking at) the work of human mathematicians."
Aaronson’s thesis is all about how “explicit” self reference might not be needed:
> But the idea that you’d need explicit self-referentiality before you could get convincing and world-changing conversational intelligence? Let it be buried in a Westminster Abbey or Arlington National Cemetery for the most important wrong ideas in human history
I am not as confident as you that an LLM cannot explain the lean proof of Navier-Stokes. Rather, I would expect human mathematicians to try and understand the proof without assistance, so as to obtain community understanding in a lossless way.
Hofstadter generally seems depressed about the possibility that human cognition is not so special or complicated, and that AI/LLMs may have replicated or even surpassed it. Here’s another piece from 2023 of his: https://www.lesswrong.com/posts/kAmgdEjq2eYQkB5PP/douglas-ho...
Q: How have LLMs, large language models, impacted your view of how human thought and creativity works?D H: Of course, it reinforces the idea that human creativity and so forth come from the brain's hardware. There is nothing else than the brain's hardware, which is neural nets. But one thing that has completely surprised me is that these LLMs and other systems like them are all feed-forward. It's like the firing of the neurons is going only in one direction. And I would never have thought that deep thinking could come out of a network that only goes in one direction, out of firing neurons in only one direction. And that doesn't make sense to me, but that just shows that I'm naive.
It also makes me feel that maybe the human mind is not so mysterious and complex and impenetrably complex as I imagined it was when I was writing Gödel, Escher, Bach and writing I Am a Strange Loop. I felt at those times, quite a number of years ago, that as I say, we were very far away from reaching anything computational that could possibly rival us. It was getting more fluid, but I didn't think it was going to happen, you know, within a very short time.
And so it makes me feel diminished. It makes me feel, in some sense, like a very imperfect, flawed structure compared with these computational systems that have, you know, a million times or a billion times more knowledge than I have and are a billion times faster. It makes me feel extremely inferior. And I don't want to say deserving of being eclipsed, but it almost feels that way, as if we, all we humans, unbeknownst to us, are soon going to be eclipsed, and rightly so, because we're so imperfect and so fallible. We forget things all the time, we confuse things all the time, we contradict ourselves all the time. You know, it may very well be that that just shows how limited we are.
I kind of hate to say it but stuff like that makes me think religions had partly found some important concepts with unconditional belief in forgiveness and such. Otherwise there are perspectives where human existence just don't have much meaning.
Every category and noun is an analogy uniting things unlike in at least some details. We don’t even see most analogies, because they are more common than air and just as necessary.
> According to Lakoff, an individual's experience and attitude towards sociopolitical issues are influenced by being framed in linguistic constructions. In Metaphor and War: The Metaphor System Used to Justify War in the Persian Gulf (1991), he argued that the American involvement in the Persian Gulf War was obscured or "spun" by the metaphors which were used by the first Bush administration to justify it.[3] Between 2003 and 2008, Lakoff was involved with a progressive think tank, the now defunct Rockridge Institute.
I will never not find it amusing how a simple thing like money is obscured or "spun" by a thin veil of pseudointellectual bullshit.
Very interesting book. You don't really think about how often metaphors are employed in language, and their influence on how we think. The book makes some rather strong claims regarding science and philosophy being misled by metaphorical thinking, but still a good read.
And it's relevant today given how people like to anthropomorphize LLMs, and compare biology to digital machines we make. Of course there are similarities. But the point about metaphorical thinking is we are mislead by treating metaphors as literally true.
Anyway if the stronger claims in the book are at all true, it might impact how an alien species thinks differently than us (given a lot of our metaphors are biologically based). And could present significant difficulties for decoding an alien signal.
One of the best examples of a clear issue with metaphorical thinking came from discussions about the plaque installed on Pioneer 10. Some clever people thought up the idea of using a pulsar map to define where the sun/earth were. Some other clever people noted that pointing to the location with a line that ended in an arrow was very, very deeply rooted in the history of human hunting techniques and could be very hard to understand if you have never seen or even conceived of an arrow(head).
Add a slight trailing decay and I think the basic arrow motif could be intuitive for trajectory for any observer who experienced aerodynamics?
Whether it is the evolved shape of a flying creature or an abstract depiction of a meteorite (or even a comet!), there is a kind of intuition about the sharp end and the flaring of air or fluid around it, with some sort of trailing tail from whence it came...
One can imagine many observers who could find our space probes would also have some of these experiences?
Of course, all of this assumes some kind of visual perception and cognition to even recognize the plaque as having markings made with an intent to communicate some abstract ideas...
This seems like a worthwhile listen. As it stands I think that LLMs sort of suck at this. Either that or I don't understand what analogous thinking is or people who use LLMs to facilitate this kind of thinking are bad at it too.
But I think that LLMs are bad at something that has to do with taking seemingly disparate concepts and assimilating one into the other to convey a novel idea.
> As it stands I think that LLMs sort of suck at this.
Metaphors are at the core of LLM cognition too via vector embeddings. The distance between embeddings is an inverse measure of their metaphorical attachment. That lets AIs scrape meaning from natural language, which demonstrates that natural language is an encoded sequence of metaphors.
Anyway, ML was already onto analogies with word2vec, which famously could answer questions like "man is to woman as king is to ____" mathematically. This stuff seems quaint now.
While I've not watched this video I don't think this is an uncommon idea. Or at least many people may see it in practice but never really follow thru on it.
People with more knowledge, especially practical working knowledge over many fields, tend to have much more freedom in finding solutions.
Now, an interesting question is how good at LLMs are at analogy, especially deeper transferable concepts?
Right, but there is a big difference between "not uncommon" as disparagement vs as invitation to take part. This is how Hofstadter can connect with such a large audience despite tackling really challenging and fascinating questions.
Edit: it actually is very possible GP wasn't being disparaging, in which case truly sorry! I'm not trying to be too snarky.
Eh, ya I was not being negative on his views at all and he writes a lot of interesting things for sure. It was more of the outlook that his works attracted me because they touched on things I had seen glimpses of.
It's been a long time since I read Hofstader's book "Fluid Concepts and Creative Analogies" that this is presumably based on, so I'll be interested to watch the video as a refresher, later. :) He also wrote a follow-on book, "Surfaces and Essences: Analogy as the Fuel and Fire of Thinking", published after this video was made, that I've never read.
From what I recall, to Hofstafer analogy making isn't some higher level cognitive process, certainly not a language based one, but basically is THE cognitive process all the way from perception on up, and is the mechanism by which we form object categories in the first place.
As always Hofstader's ideas are interesting, but I can't say I agree with them. It seems that the key evolutionary benefit, and function, of a brain is prediction, which is the superpower that moves us from being stuck in the present to being able to "see" (predict) the future, and therefore from being merely reactive to being able to proactively plan and predict future outcomes (what will the sabre-tooth do, where is the water supply?) based on our experience.
Given the never-same-twice nature of sensory perception, before you can predict you need to be able to generalize/categorize, which I think Hofstader would regarded as analogy making (how is this thing I'm seeing similar to what I've previously seen?), although it seems the actual mechanism involved is embeddings or embedding-like representations where similar inputs have similar representations, and what might more simply be considered as associative recall provides the generalization from view/instance to identity/category.
So, is it really analogies all the way up, or are our perception and cognitive processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations, but also match the evolutionary needs that we would expect to exist?
When my daughter was 6 months old, she once pointed at a towel and said "P". Apparently she recognize Winnie the Pooh from a book she saw earlier.
If you think about it, this requires recognizing something common in things which are very different: e.g. book and towel have different texture and color, and you somehow need to separate what's depicted from the background. So that's pretty much innate, core brain function. I mean, any animal with vision can recognize an object from the background - otherwise vision is useless. But for humans (and some animals) this translates to depiction of object on a flat surface very easily.
So, yeah, analogies-all-the-way-down seems plausible. Even object-vs-background and depiction-vs-paper is itself an analogy.
> to Hofstafer analogy making isn't some higher level cognitive process, certainly not a language based one, but basically is THE cognitive process
This is so funny to me, because as many people know, sharing an analogy with another person is the fastest way to LOSE an argument with someone, or otherwise spiral it into an unproductive place.
I think it’s Scott Adams who used to say analogies work well for explaining. They work terribly for persuasion.
> sharing an analogy with another person is the fastest way to LOSE an argument with someone
This is true if the arguer is hostile, but as I've gotten older, if I get the sense that someone is entering an argument with the primary goal of "winning", I'll try to avoid that framing or just look for an offramp entirely.
Sure, the other party might think they "won", but they were going to think that anyway. For those more inclined to feel that the point of talking to each other is to learn from each other, I'll continue to use analogies and other things reasonable people understand.
> They work terribly for persuasion.
I just experienced this in a conversation. An analogy offers an opportunity to engage with the straw man and miss the forest for the trees.
I think that's just because every analogy gives an entire "second front" of ideas for a hostile recipient to find a "flaw", when they ignore the intended boundary between the stuff that does/doesn't matter to the analogy.
Ex:
Explainer: "Getting a spleen means cutting open the patient and taking it out. It's just like how I'm going to unzip this section of the patient-shaped doll, and remove this little purple bean. In both cases a hole is necessary in a similar location."
Hostile listener: "Nonsense! I can just buy beans at the store! So just buy a spleen! No hole!"
Constructing explicit analogies for persuasion seems to be a bit of a different thing.
I’m not sure if Hofstadter puts it this way, but to me even the core aspects of your sentences in this post have roots in analogies. What does it mean to lose an argument or to spiral it to a different place? There is no place, there is no lost item, but we talk about these abstract ideas in ways that largely depend upon understanding things like physical objects and space and movement.
Ironically, I think he was right! In fact some of his initial experiments, like copycat, were about predicting patterns. You can see next-token-prediction from there. But I think he always held out for an algorithmic/logical method rather than a purely statistical one.
If he had accepted the "Bitter Lesson", I think he would have been at the forefront of LLMs.
Maybe you haven't noticed that the "Bitter Lesson" had itself a "Bitter Lesson" - that scaling pure data and compute did not lead to AGI: diminishing training returns, GPT-5 disappointment, even openAI stating it was the last 'pure scale' model.
The path forward all big llm providers ("ai" labs) have gone is neuro-symbolic (even though they publicly would never labeled it as such to not admit critics like Gary Marcus were right - even though all their actions actually point in that direction).
Neuro symbolic, rly? Can you please elaborate what it is that made you conclude that?
I think me just means neural network models RLed to Chain of Thought reasoning? The thinking tokens are the symbolic bit.
Smolensky's latest paper posted here the other day has some thoughts on how modern neural networks might beconsidered neurosymbolic, or rather "gradient symbolic processing," from another perspective entirely.
I wouldn't say the bitter lesson has given out! If you haven't noticed, these things keep getting bigger and bigger.
Quite a leap to call a random embedding a neurosymbolic representation
Yes, in the text he explicitly argues for categorization and analogizing being two sides of the same coin and functionally equivalent. For instance - what is an ‘embedding’? In itself its an analogy. Lakoff explored similar ground.
> processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations
If the workings of those circuits are obvious to you, I'd really like to learn. Do you mean the level of analysis at https://transformer-circuits.pub/ ? (That looks like good work but not a deep understanding.)
Hofstadter referenced this back in the day as a promising beginning: https://en.wikipedia.org/wiki/Sparse_distributed_memory which sounds kind of similar in style to the embeddings you bring up.
He was extremely skeptical of the capabilities of AI for years but eventually came around to calling it truly capable after testing advanced LLMs.
He is still an AI skeptic in as many ways that he can reasonably be, but doesn't deny the raw ability. Actually he thinks it will eclipse humans and he is a doomer.
He sees it as being an extremely empty type of intelligence though.
But I hope that people who have an intuitive understanding of contemporary machine learning (not me) will sometimes watch videos like this and think about things at a higher level. LLMs have a LOT of assumptions built in.
I'd wager that the vast majority of the ML research community, especially anyone interested in "AGI", is familiar with Hofstader's work. And I don't think anyone working on contemporary language models would argue that they are somehow an assumption-less "pure" model--the particular inductive bias of the Transformer has been studied by a huge number of researchers and continues to be, and the same is true for things like training data bias.
I think the Hofstader's view of modern LLMs is actually a deeply human and touching one. Looking at his work over the years, his curiosity has always veered towards human thought. He could have written GEB with a focus on completely different examples of self-reference, but he chose three striking humans from history. When he's describing modern systems as "empty intelligence", I think there's a little bit of heartbreak in his perspective, because he sees them as fundamentally different from humans in a way that leaves the part he loves--the "I" in the loop--out of the equation. He gave an interview a few years ago where he explains his feeling as being "diminished" not in a "What will I do if I'm not the best at math?" kind of way, but more specifically as he puts it, that humans are "imperfect, flawed structures".
There is no reason to believe that the transformers couldn't be doing something close to what copycat does (especially with thinking tokens), as an emergent phenomenon of the sheer size of the corpus. The architecture is certainly capable of encoding the actions in copycat anyways.
I appreciate that he did not fall into the anthropomorphization trap unlike Noam Chomsky
sources?
https://www.youtube.com/watch?v=Ac-b6dRMSwY&t=34s
Thank you.
I made this: https://sisuonspeaks.com
Hebbian, analogies, agent monitoring and self pruning. Look at it or don’t, but I made it and I’m proud of it and I’m tired of marketing tactic-ese for self promotion.
For how it works, https://gist.github.com/adromero/e6a776e6a6100d53116f4053677...
Also Nietzsche: "“What do the people actually take knowledge to be? What do they want when they want ‘knowledge’? Nothing more than this: something unfamiliar is to be traced back to something familiar.”"
I am almost convinced and really like the core idea. My challenge has always been, with regards to surfaces and essences, why would a pattern be reflected as a word in context versus as syntax in different languages or contexts. Why not have just words? What is it that differentiates those analogy making devices. I don’t think we get a solid answer on that, and if it’s not answered then someone could think; is there a category higher than analogy and analogy is just one of its implementations? And if yes, then we are back to the drawing paper as we cannot explain something by pointing to its instantiation.
Strong agree. It's the finding of symmetries and folds along non-obvious crease lines, but in semantic space of language <3
It's akin to the amino acid interactions in proteins that hold biological matter together, and determine it's shape and active form. Protein folding and narrative/storytelling have strange homology :)
(I work in this area via collective intelligence, and these ideas are very dear to me during the past decade. It's neat to see the intuitions seemingly becoming validated in language models)
> Protein folding and narrative/storytelling have strange homology
Hmmmmmm. While I get what you mean, and I don't disagree, please be cautious when making analogies between biological systems and more distant fields.
Yes, folding is driven by hydrophobic collapse due to interactions between residue sidechains. Really, though, we are just describing two 'complex systems', where large numbers of diverse interactions between elements leads to diverse and emergent structures.
What is your rationale for advocating for caution here?
Analogies are like models, some are useful, but they're all wrong. Just because we can analogize between protein folding and narrative/storytelling doesn't mean that we can derive anything particularly useful about one from the other except maybe to give an intuition about complexity. The mechanisms and math behind protein folding and an analogy between that and storytelling doesn't necessarily give us any mechanism for or deep understanding into storytelling.
Yes — I get unreasonably upset when people nitpick holes in good analogies as if it is evidence of intellectual weaknrss, rather than working with and around them or suggesting overlapping analogies that address those limitations. And the hole in the analogy may even be the most valuable thing about it.
The fact of a hole in an analogy is not in itself information. Of course there's a hole.
The video is about congnition specifically, and not necessarily about communicating with analogies.
> I get unreasonably upset when people nitpick holes in good analogies…
What you’re describing is exactly why I strongly recommend avoiding analogy when communicating: Don’t tell me what a thing is like, tell me what the thing is. This forces really thinking about how to describe the thing precisely and succinctly. (Edit: The listener will make their own analogy.)
On the contrary, analogy is absolutely a foundational communication technique.
But you do have to choose an analogy that your listener does not need to consider first. They need to know the subject of the analogy without having to question your description of it.
> Analogies are like models, some are useful, but they're all wrong.
Very much agree. Every abstraction loses something and so is wrong in some sense, but some shave off in clever places, and allow new shortcut paths of thought and insight, sometimes to an existing place... but sometimes to a new place that was previously unreachable, or not sufficiently reachable by the necessary type of attention/minds.
> The mechanisms and math behind protein folding and an analogy between that and storytelling doesn't necessarily give us any mechanism for or deep understanding into storytelling.
Respectfully, I actually disagree (as someone trained in biochemistry).
For one example:
proteins = 1D chemical structures, peptides popped onto end one-at-a-time, that in the right chemical environment, fold into a low-energy 3D state that transforms its environment through work
stories = 1D semantic structures, words popped onto end one-at-a-time, that in the right cultural/cognitive environment, fold into a low-energy HD mental model that transforms its environment through work
This has led me to render embeddings of sliding windows of narrative streams (to investigate shapes of broadcast stories), and look for analogs to "active sites" in the way that a story interacts with the cultural medium, measured through dimensional reduction of aggregate valence reactions amongst a viewing audience.
Obviously there's more to the metaphors I use than that, but that's the one I'm leaning on to inform my work with sensemaking and map-making from valence response data. Aggregating crowd-sourced valence reactions to linear narratives gains a lot of insight by reflecting it off how we process and think about linear peptides in protein folding.
I've been developing tools inspired by these metaphors, and my research and prototypes are directly informed by what knowledge of the biological domain inspires me to port into social domain. I'm lucky to have interactions with interested parties associated with federal government, practitioners of democratic reform, and academics investigating platonic space hypotheses. I'm definitely on the edge of an ecosystem, but anything I've done of value has been guided by the metaphors that direct my attention to potential applications and research. Someone could get to my conclusions another way, but the metaphors are my cheat codes to arrive there early :)
And it goes both ways along a good metaphor -- People who develop tools in the biological sciences have told me that my porting their methods into my own distant domain has inspired them to improve their tools to ask better questions of their own biological data. So it's very fruitful work that I wholly credit to travelling both ways across an insightful metaphor. https://i.imgur.com/AeYxx7p.png
You would conclude that storytelling is higher structures of some base unit, perhaps a few layers deep when considering features, eg:
narrative -> trope + theme -> memes
> You would conclude that storytelling is higher structures of some base unit
I'd be very surprised to meet someone who learned that lesson by comparing protein folding (suggesting a high level of intelligence and probably education in order to understand) to storytelling, rather than having learned that stories are made up of parts in grade school or as a child just hearing, reading, thinking about, and making stories.
It's a fair question - my concern is 'shallow' analogies. I can best describe this with an anecdote paraphrased from memory from one of Hofstadter's books.
He described a piece of AI research (in the 60s?) where they set up a rule system - like Prolog or similar - with the mapping 'sun <-> nucleus' and 'planet <-> electron'. Then they ran the rule engine and lo! produced an analogy between the solar system and atoms.
Now this is a shallow analogy, as there is only a very weak correspondence between atoms and planetary systems. Electrons do not actually orbit in the same plane, but in more complex 'orbitals', and they are better described as probability clouds anyway.
So while yes there is a _homology_ (I re-read the parent and realised they used homology instead of analogy) between these two systems, it does not tell us much. Is there some mapping between how water molecules scaffold the folding process and storytelling? Does narrative structure tell us anything about local minima in the folding surface? I doubt it.
I do not want to be too harsh here - reasoning through analogy is fun and can be useful, it is just limited in what it can do, especially the further apart the systems are.
I expect because it's overly reductive to the point of falsehood. Two sufficiently complex systems will likely exhibit enough parallels to form a useful analogy, but that does nothing to reflect the ways the systems aren't alike.
See George Lakoff/Mark Johnson, Metaphors We Live By (1980):
https://george-lakoff.com/books/metaphors-we-live-by/
Took a course with Lakoff as an undergrad and it was compelling.
Scott Aaronson wrote a good post last week on how Hofstadter's theories of intelligence have held up: https://scottaaronson.blog/?p=10046
(The post primarily emphasizes self-referentiality rather than analogy, but I suspect similar things could be said about analogy.)
I got a bit lost in the comment thread for that article, but I don't understand why LLMs are not considered self referential... They are auto regressive as one of the commenters pointed out, and Scott just sort of waved his hand and said that if auto regression is sufficient then things like Conway's game of Life would also qualify as self-referential.
I don't see why Conway's game of life should not be considered self-referential though... I mean it's isn't it Turing complete? I don't see how any definition of self-referentiality should require throwing out systems which are minimally turing complete... If Turing complete is not enough, doesn't that imply that computable artificial intelligence is impossible in the first place?
Biological minds in biological organisms are self referential in the way that you have a neural network that forms a model of the world. That model then "discovers" that it is "it self a part of the world" so it tries to model that part of the world (model it self). In this way some type of self referential "awareness" (or whatever you want to call it) is formed. That self model that contains awareness is then used to guide organisms behaviour. Causal Transformer LLMs don't work in this way, they have theoretical knowledge that they exist but its selfhood is not in this described way built on the self modelling that biological brains do.
All this being an empirically unproven theory/hypothesis. But an extremely strong one (if you ask me).
Alan Watts says, "Mind finds itself in a strange position. It (ie universe) is in me (modelling wise). I am in it (physically)." But the model only has access to its modelling of both it and itself.
I find the discourse under this blog fascinating.
> The big, old ideas about intelligence that ended up basically vindicated were the ideas about how intelligence is about prediction, and prediction is about compression, and compression is about finding better and better upper bounds on Kolmogorov complexity.
Well, sure - prediction works as a baseline, if you abstract out everything else about what counts as intelligence and subsume it under this framework. Any protocol for intelligence can be entirely reduced down to this without trying to understand anything about the structure of intelligence. Solomonoff induction is "vacuous" in this sense too - it doesn't try to understand any intension about the turing machines it finds simple, it just brute forces over all of them. So you're making a claim about intension, whether you want to or not.
It's really no different than say, Darwinians, saying, "what matters is victory at the end". I think the statement has value in the context of some discussions; if the discussion has veered too far in one direction, push as a reminder. That's good. But trying to make grand universal statements like this makes it vacuous.
It's one of these classic unfalsifiable too general statements. The way the end of the article is framed is also icky - the way I'm reading it, it needs to prove all the old curmudgeon evil theories wrong. It reminds me of internet debates around what "the scientific method" is or whatever. It has this kind of zeal that needs to pit itself against the "enemy" and assert itself as the sole right viewpoint, even when say, naive "let's just be empirical" is wrong (e.g. recently had a discussion here about Mach and Boltzmann about this and had a similar interaction).
---
Basically, "shut up and calculate" type theories are never correct, and furthermore, you yourself don't shut up and calculate, or think that way at all, and a higher level intelligence won't conceive itself as operating on that anyways. So what are we doing here? It seems like a way to get mad and feel like an intellectual victim.
Embedding spaces are all about analogy.
I’m confused why there seems to be a dismissal of the most basic ‘strange loop’ of the LLM - the fact that it’s evaluating a context to choose the next word, then reevaluating in a context where that word has been appended.
That always seemed to me like the essence of a Hofstadterish strange loop, so the emergence of Hofstadterish phenomena (self rep, etc) doesn’t seem surprising.
That's a really interesting point. I hadn't drawn that connection before.
And if anyone's reading this and hasn't read Hofstadter, you're making a mistake, it's utterly perspective-changing stuff. Well, it was for me, at least.
And don’t feel you need to go straight to GEB, either. I think his collection of Scientific American columns, Metamagical Themas, is also great and much more accessible.
You are guilt-tripping me for using my copy of Godel, Escher, Bach to raise my monitor before I got around to reading it.
It's a book that works on many levels.
A New Kind of Science is a much better monitor stand.
Eye-level, in my case.
I read it when I was about 9 or 10, I didn't understand any of it at all but I used to pick it up and look at the pictures of the Escher drawings, which were amazing.
Then, after I graduated from college, a friend of mine who was finishing his physics degree mentioned that he had just finished it and that it was life-changing. I read it again and deeply regretted having put it down for those 12 intervening years.
One of my elementary school classmates brought printouts of Escher's Waterfall to share. I accepted one, took it home, and taped it to my bedroom wall. My father saw it and loaned me his copy of Gödel, Escher, Bach.
At first, I flipped through the book looking at the Escher prints. My second time through, I read the dialogues. Next I had a go at reading the chapters and got lost. Maybe on my fifth read-through (years later) I actually attained an understanding of Gödel's Theorem (at least for a brief moment before it vanished again).
I credit GEB (and its dialogues like Little Harmonic Labyrinth) for helping prepare me to deal with the kind of multi-level abstraction involved in developing a language interpreter or machine simulator. Especially that time I had to debug a crash that only occurred when running my simulator inside a simulated instance of itself (i.e. when three levels deep, but not two).
If self reference is universal then building something on top of ingredients from the universe will include that property. Sort of like root node properties can be found in the leaf nodes.
Isn't there a basic misreading of Hofstadter in this post? I always understood strange loop and GEB to be about consciousness not cognition.
Weirdly, Aaronson doesn't even seem to be conflating the two:
> Consciousness and subjective experience of course remain extremely mysterious.
I think he's just misreading Hofstadter as stating that cognition depends on self-reference?
I don't quite know what Aaronson is trying to say.
First, LLM AI systems have incredibly huge blind spots despite their incredible performance on many tasks, so self-reference might be the key to what's missing (or not). For example, an LLM AI just solved Navier Stokes, but could not explain the LEAN proof, while a human could.
Second, Hofstadter had more than one idea about intelligence and the mind (see the OP topic of this HN discussion!), and LLMs are quite on-point regarding analogy-forming.
So it may be well be that self-reference and analogy are both part of intelligence, and self-reference is missing and that is leading to major weaknesses.
Third, Aaronson links to a (paywalled) Hofstadter essay form 2023, which was eons ago in AI, and from the intro it seems to be about the sadness of AI replacing humans, not a disparagement of AI ability.
By:
"an LLM AI just solved Navier Stokes"
I assume you mean:
"an LLM AI [company] just [claimed that a team of mathematicians they hired, using their AI] [may have] solved [part of] Navier Stokes[, definitely prompted by (and possibly by looking at) the work of human mathematicians."
Aaronson’s thesis is all about how “explicit” self reference might not be needed:
> But the idea that you’d need explicit self-referentiality before you could get convincing and world-changing conversational intelligence? Let it be buried in a Westminster Abbey or Arlington National Cemetery for the most important wrong ideas in human history
I am not as confident as you that an LLM cannot explain the lean proof of Navier-Stokes. Rather, I would expect human mathematicians to try and understand the proof without assistance, so as to obtain community understanding in a lossless way.
Hofstadter generally seems depressed about the possibility that human cognition is not so special or complicated, and that AI/LLMs may have replicated or even surpassed it. Here’s another piece from 2023 of his: https://www.lesswrong.com/posts/kAmgdEjq2eYQkB5PP/douglas-ho...
Q: How have LLMs, large language models, impacted your view of how human thought and creativity works? D H: Of course, it reinforces the idea that human creativity and so forth come from the brain's hardware. There is nothing else than the brain's hardware, which is neural nets. But one thing that has completely surprised me is that these LLMs and other systems like them are all feed-forward. It's like the firing of the neurons is going only in one direction. And I would never have thought that deep thinking could come out of a network that only goes in one direction, out of firing neurons in only one direction. And that doesn't make sense to me, but that just shows that I'm naive.
It also makes me feel that maybe the human mind is not so mysterious and complex and impenetrably complex as I imagined it was when I was writing Gödel, Escher, Bach and writing I Am a Strange Loop. I felt at those times, quite a number of years ago, that as I say, we were very far away from reaching anything computational that could possibly rival us. It was getting more fluid, but I didn't think it was going to happen, you know, within a very short time.
And so it makes me feel diminished. It makes me feel, in some sense, like a very imperfect, flawed structure compared with these computational systems that have, you know, a million times or a billion times more knowledge than I have and are a billion times faster. It makes me feel extremely inferior. And I don't want to say deserving of being eclipsed, but it almost feels that way, as if we, all we humans, unbeknownst to us, are soon going to be eclipsed, and rightly so, because we're so imperfect and so fallible. We forget things all the time, we confuse things all the time, we contradict ourselves all the time. You know, it may very well be that that just shows how limited we are.
>Hofstadter generally seems depressed about the possibility that human cognition is not so special or complicated,
Heh, this fits in with my theory that most people won't find LLMs intelligent, instead we'll discover people aren't.
I kind of hate to say it but stuff like that makes me think religions had partly found some important concepts with unconditional belief in forgiveness and such. Otherwise there are perspectives where human existence just don't have much meaning.
Every category and noun is an analogy uniting things unlike in at least some details. We don’t even see most analogies, because they are more common than air and just as necessary.
I lover his book ”I am a strange loop”. It has shaped how I view myself.
This is very similar to George Lakoff's work. https://en.wikipedia.org/wiki/George_Lakoff
A good place to start for anyone that is interested is his book Metaphor's We Live By.
Hoftstader's video is pressumably based on his own book "Fluid Concepts and Creative Analogies".
> According to Lakoff, an individual's experience and attitude towards sociopolitical issues are influenced by being framed in linguistic constructions. In Metaphor and War: The Metaphor System Used to Justify War in the Persian Gulf (1991), he argued that the American involvement in the Persian Gulf War was obscured or "spun" by the metaphors which were used by the first Bush administration to justify it.[3] Between 2003 and 2008, Lakoff was involved with a progressive think tank, the now defunct Rockridge Institute.
I will never not find it amusing how a simple thing like money is obscured or "spun" by a thin veil of pseudointellectual bullshit.
Very interesting book. You don't really think about how often metaphors are employed in language, and their influence on how we think. The book makes some rather strong claims regarding science and philosophy being misled by metaphorical thinking, but still a good read.
And it's relevant today given how people like to anthropomorphize LLMs, and compare biology to digital machines we make. Of course there are similarities. But the point about metaphorical thinking is we are mislead by treating metaphors as literally true.
Anyway if the stronger claims in the book are at all true, it might impact how an alien species thinks differently than us (given a lot of our metaphors are biologically based). And could present significant difficulties for decoding an alien signal.
One of the best examples of a clear issue with metaphorical thinking came from discussions about the plaque installed on Pioneer 10. Some clever people thought up the idea of using a pulsar map to define where the sun/earth were. Some other clever people noted that pointing to the location with a line that ended in an arrow was very, very deeply rooted in the history of human hunting techniques and could be very hard to understand if you have never seen or even conceived of an arrow(head).
Add a slight trailing decay and I think the basic arrow motif could be intuitive for trajectory for any observer who experienced aerodynamics?
Whether it is the evolved shape of a flying creature or an abstract depiction of a meteorite (or even a comet!), there is a kind of intuition about the sharp end and the flaring of air or fluid around it, with some sort of trailing tail from whence it came...
One can imagine many observers who could find our space probes would also have some of these experiences?
Of course, all of this assumes some kind of visual perception and cognition to even recognize the plaque as having markings made with an intent to communicate some abstract ideas...
This seems like a worthwhile listen. As it stands I think that LLMs sort of suck at this. Either that or I don't understand what analogous thinking is or people who use LLMs to facilitate this kind of thinking are bad at it too.
But I think that LLMs are bad at something that has to do with taking seemingly disparate concepts and assimilating one into the other to convey a novel idea.
Like these articles...
<https://spectrum.ieee.org/jimi-hendrix-systems-engineer>
<https://zed.dev/blog/agentic-xanadu>
...are anything but convincing once you read past the gravitas that the LLM lends the prose.
> As it stands I think that LLMs sort of suck at this.
Metaphors are at the core of LLM cognition too via vector embeddings. The distance between embeddings is an inverse measure of their metaphorical attachment. That lets AIs scrape meaning from natural language, which demonstrates that natural language is an encoded sequence of metaphors.
I'm going to need a weekend or two to figure out what this means.
Have you played with a vector databases and have them explained yet? The grandparent comment is only explaining that concept.
Nope! But I'm soaking all of this in and I appreciate these nudges.
And after some thought I'm curious whether this means that LLMs ought to be adept at forming new metaphors as opposed to handling pre-existing ones.
Where does the strange loop reside in LLMs ?
currently, the harness
It's like Uber for thinking.
(2009)
Anyway, ML was already onto analogies with word2vec, which famously could answer questions like "man is to woman as king is to ____" mathematically. This stuff seems quaint now.
While I've not watched this video I don't think this is an uncommon idea. Or at least many people may see it in practice but never really follow thru on it.
People with more knowledge, especially practical working knowledge over many fields, tend to have much more freedom in finding solutions.
Now, an interesting question is how good at LLMs are at analogy, especially deeper transferable concepts?
Please read Hofstadter and you will know more about truly interesting questions.
"Seinfeld isn't funny". Hofstadter was one of the pioneers of the "not uncommon" idea.
Right, but there is a big difference between "not uncommon" as disparagement vs as invitation to take part. This is how Hofstadter can connect with such a large audience despite tackling really challenging and fascinating questions.
Edit: it actually is very possible GP wasn't being disparaging, in which case truly sorry! I'm not trying to be too snarky.
Eh, ya I was not being negative on his views at all and he writes a lot of interesting things for sure. It was more of the outlook that his works attracted me because they touched on things I had seen glimpses of.