Sounds reasonable... That the model is sometimes learning a lossy vector representation of something symbolic in nature... Sure, a NN can approximate a function?
They say this holds in... Some examples they found?
The math and core experimentation here is beyond my abilities, but what I think I understand is that there are possible deeper patterns of representation that exist in LLMs that are distillations of core conceptual relations in grammar that we can get our heads around in a mathematical sense rather than apparent layer-smeared noise that somehow, un-interpretably (in a meaningful sense), resolve to correct grammar/inferences.
That's pretty cool. I hope I've got that kinda-right.
I haven't read this in depth yet, though I plan to. If this general line of research is interesting to you, I'd recommend checking out some of the lines of research it touches upon--they're really rich and fascinating, and some are pretty approachable mathematically even if ML research papers aren't usually your thing. The related works section here seems pretty well stocked, but mechanistic interpretability is a pretty interesting peephole into this general vein: https://transformer-circuits.pub/
It's like Neo says "You get used to it, though. Your brain does the translating. I don't even see the code." He was referring to something like a K, Q, V vector at the time I believe.
Sounds reasonable... That the model is sometimes learning a lossy vector representation of something symbolic in nature... Sure, a NN can approximate a function?
They say this holds in... Some examples they found?
I don't enough about this area
The math and core experimentation here is beyond my abilities, but what I think I understand is that there are possible deeper patterns of representation that exist in LLMs that are distillations of core conceptual relations in grammar that we can get our heads around in a mathematical sense rather than apparent layer-smeared noise that somehow, un-interpretably (in a meaningful sense), resolve to correct grammar/inferences.
That's pretty cool. I hope I've got that kinda-right.
I haven't read this in depth yet, though I plan to. If this general line of research is interesting to you, I'd recommend checking out some of the lines of research it touches upon--they're really rich and fascinating, and some are pretty approachable mathematically even if ML research papers aren't usually your thing. The related works section here seems pretty well stocked, but mechanistic interpretability is a pretty interesting peephole into this general vein: https://transformer-circuits.pub/
It's like Neo says "You get used to it, though. Your brain does the translating. I don't even see the code." He was referring to something like a K, Q, V vector at the time I believe.
Cypher says that, and he's clearly referring to a blonde, a brunette, and a redhead.