> “An Interdisciplinary Synthesis Across Economics, Psychology, Biology, Philosophy, Game Theory, and Spiritual Tradition.”
If one were inclined, they could shorten the title to "AI Synthesis Across Economics, Psychology, Biology, Philosophy, Game Theory, and Spiritual Tradition." Maybe that's an Easter egg.
AI submissions, soon AI reviewers, and shortly after only AI readers.
The guy's name is "nosloP" spelled backwards. Maybe spelled normally it's supposed to signify the opposite.
> AI submissions, soon AI reviewers, and shortly after only AI readers
This is already happening. Write code with an AI agent, agent creates PR, other agents review, coding agent iterates, PR gets merged, and hopefully real humans use the application and provide feedback
The latter step is also now staring to be replaced by AI (user directs agent to access app, get data, perform actions)
As much as I’m confident this reasercher is just a skinsuit for a slop machine I don’t exactly trust pangram. I’ve apparently been flagged for possible AI slop comments based on Pangram results before.
Oddly it seems to have happened on longer posts? Don’t know if that may play a part.
That the name in no slop backwards is too wild to be real, but it's a real person.
I mean it's everywhere now. I complained already at my workplace multiple times, we get huge Claude generated documents, which people then ask Claude to summarize. I don't mind some of it in principle but people are too lazy now to input all the details so you get a bunch of fantasies that were explicitly ruled out already. It's tiring.
I went to a "town hall" at my company last friday, the slides were all obviously AI-generated, the text was too, without anybody proofreading it.
How do i know? Because i doubt "Access&Identity will help us push forward the productivity of all collaborators" was what the team responsible for the prompt wanted to say.
IIUC there are not 258 academic papers published in serious peer review journals [1], only 258 "preprints" in a free preprint server. It's like posting in WordPress but using a PDF instead of html [2]. I guess they start with "abstract" and end with "References" so they looks like papers.
For some weird reason in the last few years people forgot the differences and now any PDF is an "academic paper" [3]. (I'd be more annoyed, if the internal system of the university also forgot the difference. I'm afraid to ask.)
[1] Whatever "serious" means, that's a huge can of worms.
[2] They get an doi and a version number in case of a change.
Vadim Sokolov, the co-author of the paper mentioned in the post defends this insane practice and says they reviewed at least one of the 258 generated papers with their own eyes, they are thus not fake papers.
IIUC he claims that most of them are just internal notes. In the research group I am, we generate a lot of crap. I'm not sure we reach one per day, but probably 3 per week, but we have some "productive" weeks from time to time.
Sometimes I just write an email (in text) but sometimes it's a short LaTeX document (3 pages, title, date, my name, may have an introduction to ensure we use the same notation, usually no references because I'm lazy). Some of my coworkers are using AI that likes to make longer stuff and format it like a letter to the Nobel foundation.
Some are good, most are wrong or bad. Some are reproductions of others research work, some are early experiments, some are rediscovering the wheel, some are misguided ideas, and lucky a tiny part are good. Anyway, they are for internal use, we don't try to contribute to the thermal death of the universe uploading that crap to a free preprint server.
I don't understand:
a) Why they upload internal notes/reports to a preprint server?
This is similar to when Elon claims to be running what, 4-5 different companies as a ceo, and taking care of his children, and sleeping 6 hours a day, and then we observe him shitposting on Twitter constantly or showing off his gaming skills in a game that is literally a time gated grindfest.
You add up all the claimed activities and then realize it’s more hours than are in a day so someone’s lying.
I don’t see how this author can claim that these papers didn’t “…suddenly materialize from a magical “chatbot” in 2026” when it’s obvious that 1-2 people are not producing 258 papers that are tens of pages each in 9 months with any sort of academic rigor.
> showing off his gaming skills in a game that is literally a time gated grindfest.
What was even more hilarious was the fact that he had visibly no idea of what the game was even about (he was bragging about item level instead of stat and dismissing the best item of his gear because it was lower iLevel than the rest).
Why would someone ruin their academic credentials and reputation (at a reputable university) like this? Maybe there was nothing to begin with? Otherwise I can't understand this.
I think it might be taking a position in the use of AI, They don't appear to be concealing their use.
If the papers were of a uniformly high standard then I might not have a problem with it.
If they are hoping some of them are good and the review process will filter out the bad then it seems like they are not doing research, they are writing hypotheses and at pushing the work of testing them onto the reviewers.
I could also see that being done to make some sort of statement about the review process, but it's not clear if that is their goal.
He's wasting people's time, which is already the resource that many people lack any to spare. I'm not sure if he realises this part.
This is economics though, a discipline where you be like Kenneth Rogoff and Carmen Reinhart and commit the most impactful fraud of the century and face absolutely no consequence on your career.
When you compare that to the consequence of the hydroxycholoquine paper, which is comparable in public policy impact, where Didiee Raoult lost all its legitimacy and his job, it's easy to see that economics as an academic field doesn't value probity much.
It's incentivized by the system. You get grants, clout, importance, job opportunities based on your publishing in major conferences. As of last year, the variance of review scores have been higher between reviewers on the same paper than between the same reviewer on different paper. So it's effectively become a coinflip. The best strategy when it's random is just to spam stuff.
Oh also at least 40% generated peer-reviews are contributing to this statistic.
Because they won't. The various algorithms that create the "publish or perish" incentives actually reward this. Academic reputation isn't a term in those algorithms, so it isn't rewarded. Same for verifying other's papers and other parts of science and research that aren't actually publishing papers. This is the cause of the reproducibility crisis in academic.
"Show me the incentives and I will show you the outcome"
Reputation is a factor, because most processes involve a subjective evaluation by other experts in the field. And non-experts use service activities such as peer review as a proxy for reputation. If reputable entities use your services as an expert, you are probably a reputable expert in the field.
Not surprising. Many academics held in high-esteem in these behavioral offshoots are nothing more than frauds to begin with. Almost every X-studies paper is equally fluff these days (see the recent frontpage CHI post) - at least these have some semblance of theory given their equations, though are still largely garbage.
More and more evidence is surfacing that suggests AI addiction is real. It may be as simple as that Nicholas Polson has developed a severe case of AI addiction.
A ton of academia is based on putting out slop into not so well-read journals and crossing fingers it passes some dead-exhausted postdoc TA's peer review.
I'm not an academic, and I understand by osmosis that organized science has many problems related to publishing and connected mechanisms, but I have to say there's been quite a few moments in my life and career when I found the inspiration or solution I needed in a good paper.
For condensed knowledge on highly specific topics (i.e. more specific than is economical in book form) I often know nothing better.
Most recently I did sort of a self-taught crash course in ground-penetrating radar and applications for an archeological endeavour I write software for, and a number of papers have been really invaluable.
I think one of the end-game components for AI is to be able to read all the literature and generate a reliable list of which ones are not correct and a convincing reason why.
> generate a reliable list of which ones are not correct and a convincing reason why
That is not realistic, but I suppose where things are heading is that you have some indicator of the strength of evidence -- see Fig. 13 in the following insightful take:
Though, "strength" should probably be "reliability" and "validity", and I suppose those indicators are more for picking signals from the noise; i.e., what is even worth clicking and reading. That would be increasingly valuable already today due to the volume (and, yes, slop and other related stuff).
> That is not realistic, but I suppose where things are heading ...
And to expand on this, it's not realistic because science is not armchair philosophy. You have to go out and measure the world.
Sometimes, through force of will, a person can think deeply about a problem and come up with beautiful theories that explain our measurements. Many scientists had careers like this, probably most famously, Einstein. But it's worth noting that Einstein also got a lot wrong! [1]
Even if we somehow give an LLM the ability to go out and measure things, I seriously doubt that the role of humans in science is done. There's a big difference between "an explanation" and "a good explanation." Ask any physicist. There's a surprising amount of aesthetics involved. Good theories are consistent with the evidence, but it's more than that-- there's a great deal of "taste" involved. And there's a good reason for that. For any real problem, there are effectively an infinite number of alternative hypotheses. From a "theory of science" standpoint, this should cause scientists nightmares, but it doesn't. Because by the time you are a practicing scientist, you've developed a feel for what constitutes a satisfying explanation. If you spend time with scientists, especially in the "hallway track" at a conference, "taste" is a frequent topic of conversation!
I think you misunderstood. I'm not asking for an oracle that can determine whether a paper is correct, I want an oracle that can find real mistakes in papers (thus invalidating them).
I work full time on "lab in the loop" AI, so I'm pretty familiar with the need for real-world experiments. I am not proposing a fully autonomous scientist that could read an arbitrary paper and emit whether it's universally true without some verification method.
Also, to your statement: " Because by the time you are a practicing scientist, you've developed a feel for what constitutes a satisfying explanation."
I'm a practicing scientist (well, ex-scientist) and it seems like most "satisfying explanations" end up being wrong or incomplete simply because they seem so satisfying.
> I'm not asking for an oracle that can determine whether a paper is correct, I want an oracle that can find real mistakes in papers (thus invalidating them).
What's the difference? How do you find real mistakes without a model? Either you have a trusted mathematical model (in which case you already have a complete explanation) or you have to compare it against the ultimate oracle: the world. Or are you proposing something like "let's use an LLM to convert this hand-wavy English paper into a formal proof and then check it for logical fallacies?" In which case, fine, that would be useful, but that's not exactly the same thing (and also not as important) as saying that a paper advances a bad explanation. Just that the explanation is flawed in some way.
How do you determine that a mistake is serious enough that it actually invalidates the paper? How do you know that it's not possible to correct it and reach a similar conclusion with fundamentally the same approach? Especially when the fix would be complex enough to justify a follow-up paper.
This is realistic. We already do this today: it's called "journal club". A bunch of grad students read the same paper and then criticize it. I've read papers that I thought were amazing only to. have somebody else notice a key issue in a method, or a conclusion that didn't follow, or outright omission of an important detail, in a way that could be verified by both the students and the authors of the paper.
I've tried here and there, but even the top-end models always tend toward shallow generalist takes. I've not been able to get more than basic primers out of LLMs, and nothing close to the ability of a professional author to stay on course with an idea, detail level, specificity etc.
People in the early days used to often whine that LLMs just regurgitate text snippets (unfounded of course), but I think the way we currently train and RLHF them actually seems to largely make them unable to reproduce the knowledge they have been trained on, since they seem to just always want to please the mean with their output. I'm oversimplifying the mechanisms, but you get my drift.
Alan Sokal style hoax is what comes to mind first, of course. Wouldn't be the first computer-generated hoax either (see SCIgen). However see also the P.P.P.P.S. in the article, his co-author denies that in the comments.
Writing is a technology that served us well for over 5000 years, but now times are a changing. Written media will cease to provide any value in the next decade i think. In person chats/Public speaking will be revered again.
Edit - people are assuming that i want this to happen. no, i don't. i am reading more than i ever did before, maybe out of rebellion. i am just stating what I THINK is to come. it might not go this way at all.
One of the most memorable tech talks I ever attended was by a junior engineer who had just come back to the office from a major conference. Our workplace had an unusually oral culture -- decisions were made in meetings, and not always written down; there wasn't much in the way of bug tracking or work planning at the time; people generally preferred to talk through problems instead of working them out in email, chat, or forum posting.
He was supposed to present a topic he learned about at the conference. Instead, he gave a cheeky talk which introduced the very concept of written language to the audience, as if we had never heard of it. Starting from the history of writing on tablets and so on, and ending in practical advice like, "when words are written down, you don't have to remember what they were exactly; you can just read the words again." I loved the chutzpah and could barely hold myself from chortling out loudly at some of his slides.
I know that your post is an amusing take on communication, but I find the post you responded to be kind of a horrifying idea. Until recently, writing was like a time machine: somebody, from almost at any time in human history, wanted to tell you something. By writing it down in a common language, you could read what they wanted to say. Writing is flipping amazing.
There has always been bad writing, but historically it has been easy enough to filter the garbage out. Now we get garbage that looks so much like real thoughts that it's hard to tell. This is not a positive development. Separating the wheat from the chaff is extra work, and the overhead is a burden on humanity. I suspect that this is a burden that almost nobody actually wants, but we're getting it anyway.
We will be worse off for it. People just are losing the ability to read well. We are regressing as a species because of billionaire's thinking machines. A few companies are evolving while the rest of us become dependent.
Your intelligence and competencey are going to be 1:1 correlated to the quality and quantity of tokens a billionaire gives you access too (local, sure, they control the hardware markets as well).
Keep excercising your writing and reading muscles because it will definatley set you apart from the captive masses in the future.
I honestly think it's more likely that younger generations grow up seeing the ill effects of being AI-pilled for communication/writing, deciding by way of AI literacy that they want none of that, and develop better ideas on responsible use.
I really don't like AI for document writing as a whole, especially within organizations. Code can be a form of communication, but for the most part it's an engineered thing that can be re-run, tested, verified and so on. But documents drive decision-making, and decision making is often one-time. Decisions should be studied and understood, and depriving authors of much of both is not having any good effects.
As it always should have been. Even Plato/Socrates were intelligent enough to muse about the idea that writing has degraded our memory vs when we had to memorize entire epic poems. They had a lot of hostile stuff to say about writing in Timaeus and other works.
The fact that most teachers refuse to do in-person oral exams has always been due to laziness, incompetence, and the reality of the economics of dealing with classroom sizes where 30 is considered small.
I’m only half joking that someone will come up with a LLM text to speech synthesizer that pauses and plays depending on if your mouth is closed/jaw is moving. Analysizes your voice to synthesize as well.
Because the exact opposite will happen. The idea that allowing AI reviewers will increase accuracy is probably foolhardy. People being people will get lazy and review less by hand if at all. And the LLMs will miss plenty of things. Then those papers will get into the training sets for the next LLMs. I think you can see where this goes overtime.
The issue is generally not accuracy in my experience (humans are even lazier about this). The issue is that LLMs do not understand what is interesting and what is procedural.
> As someone who has known Nick in the past, I’m a bit surprised by this multitude of articles and books. However, I think you’ve missed a fifth possibility that would suggest gentle handling: a mental health issue. The papers and books on philosophy, religion, and issues well beyond anything econometric plus the videos on the Riemann Hypothesis suggest someone who is no longer bound by the intellectual boundaries or social mores of proper research.
I don't know whether the commenter is legit, and whether this actually happened, but I wonder if AI-psychosis (or at least, AI-amplified psychosis) might not become a real thing. People already on the edge having "productive" conversations with LLM chatbots and going down rabbit holes. It can't be a good thing, especially since LLMs are sycophants that indulge every crazy idea you have.
I'm not saying this didn't happen before, I just wonder whether LLMs enable a whole new scale of crazy.
I tried writing a simple 20 page paper with the latest Claude model over several weeks and that thing got ripped apart by legal and data science (for good reason), not going to try that again. And there I had all the numbers and analyses down and verified, just the writing itself by Claude was subpar.
Can’t imagine the utter garbage that Claude will produce when you give it a simple prompt and let it write thousand of pages.
Kind of makes me wonder about all that AGI talk if these models can’t even the work of a good PhD or postdoctoral student.
A benign but depressing interpretation is that even competent and respected academic researchers are not immune to AI-addiction and episodes of AI-psychosis cause by binging on AI.
The dopamine hit of having even their most speculative ideas expanded in a convincing paper format is self-reinforcing and soon the affected individual starts believing that the output is perfect and needs no review, because reviewing it would delay the next stimulation cycle.
Years ago I was an academic peer reviewer and hated it, you've probably seen published work thats low in quality - you wouldn't believe how low quality the unpublished crap is. I cannot even imagine the absolute bundle of garbage that must be handled by these unpaid reviewers today. We need a better system.
Another possible answer to the "What's in it for him?" question:
The Everest effect.
George Mallory: "The first question you will ask, and the one I have to answer is, 'What's the point of climbing Mount Everest?' And my answer should be, 'It's no use.' There is not the slightest prospect of any gain. Oh, maybe we can learn a little about the behavior of the human body at height, and medicine can change our observations for aviation purposes. But nothing else could come of it. We will not bring back a little gold or silver or gemstones or coal or iron. We will not find soil or earth that we can grow crops that produce food. There is no point. So if you can't understand that there's something in humans that responds to the challenges of this mountain and goes to meet those challenges, then the struggle that ensues is a struggle from life itself upwards, and you won't see why we're leaving. What we get from this adventure is just joy. We don't live to eat and make money. We eat and make money to enjoy life. That is the meaning of life and the purpose of living."
Why generate all of these papers? Because, for the first time, he can.
Doesn't have to be for the money or fame or potential for speaking engagements. It could simply be that he wants to test the limits of what's possible.
> Jesus Christ. I’ve heard that Cambridge University has an open position in their school of education . . . this kind of thing would fit in very well there, no?
my Z_spiritual(T) has been unusually high recently because all of this slop is supercharging the value I've been generating through ego-confrontation and progressive liberation
Squatting seems like the most plausible explanation. Explaining an idea to your friend at a bar doesn't make it "yours", but publishing a manuscript does.
There are obviously exceptions (e.g., the specification for the Java programming language is well over 100 pages [1] and is extremely low on filler), but most ideas don't work like this. In my area, 10 pages really is enough most of the time.
> “An Interdisciplinary Synthesis Across Economics, Psychology, Biology, Philosophy, Game Theory, and Spiritual Tradition.”
If one were inclined, they could shorten the title to "AI Synthesis Across Economics, Psychology, Biology, Philosophy, Game Theory, and Spiritual Tradition." Maybe that's an Easter egg.
AI submissions, soon AI reviewers, and shortly after only AI readers.
The guy's name is "nosloP" spelled backwards. Maybe spelled normally it's supposed to signify the opposite.
> The guy's name is "nosloP" spelled backwards. Maybe spelled normally it's supposed to signify the opposite.
Most insightful comment
Yeah, he should publish this exquisite finding in the Journal of Human Shit That AI Would Not Produce
He be Spool'n
> AI submissions, soon AI reviewers, and shortly after only AI readers
This is already happening. Write code with an AI agent, agent creates PR, other agents review, coding agent iterates, PR gets merged, and hopefully real humans use the application and provide feedback
The latter step is also now staring to be replaced by AI (user directs agent to access app, get data, perform actions)
Similar thing with email
https://www.pangram.com/history/51744f24-1896-4d5c-8bfc-786c...
It isn't even partially human written, it's just an LLM's output.
As much as I’m confident this reasercher is just a skinsuit for a slop machine I don’t exactly trust pangram. I’ve apparently been flagged for possible AI slop comments based on Pangram results before.
Oddly it seems to have happened on longer posts? Don’t know if that may play a part.
Isn't pangram more accurate with longer texts? Does it claim to be accurate for comment sized text?
It certainly isn’t perfect, especially when it gives a probability below 100%z .
That the name in no slop backwards is too wild to be real, but it's a real person.
I mean it's everywhere now. I complained already at my workplace multiple times, we get huge Claude generated documents, which people then ask Claude to summarize. I don't mind some of it in principle but people are too lazy now to input all the details so you get a bunch of fantasies that were explicitly ruled out already. It's tiring.
I went to a "town hall" at my company last friday, the slides were all obviously AI-generated, the text was too, without anybody proofreading it.
How do i know? Because i doubt "Access&Identity will help us push forward the productivity of all collaborators" was what the team responsible for the prompt wanted to say.
He is most definitely a real person, I know people who have written papers with him.
IIUC there are not 258 academic papers published in serious peer review journals [1], only 258 "preprints" in a free preprint server. It's like posting in WordPress but using a PDF instead of html [2]. I guess they start with "abstract" and end with "References" so they looks like papers.
For some weird reason in the last few years people forgot the differences and now any PDF is an "academic paper" [3]. (I'd be more annoyed, if the internal system of the university also forgot the difference. I'm afraid to ask.)
[1] Whatever "serious" means, that's a huge can of worms.
[2] They get an doi and a version number in case of a change.
[3] A few years ago they were call "whitepapers".
Vadim Sokolov, the co-author of the paper mentioned in the post defends this insane practice and says they reviewed at least one of the 258 generated papers with their own eyes, they are thus not fake papers.
Source: https://statmodeling.stat.columbia.edu/2026/08/27/258/#comme...
IIUC he claims that most of them are just internal notes. In the research group I am, we generate a lot of crap. I'm not sure we reach one per day, but probably 3 per week, but we have some "productive" weeks from time to time.
Sometimes I just write an email (in text) but sometimes it's a short LaTeX document (3 pages, title, date, my name, may have an introduction to ensure we use the same notation, usually no references because I'm lazy). Some of my coworkers are using AI that likes to make longer stuff and format it like a letter to the Nobel foundation.
Some are good, most are wrong or bad. Some are reproductions of others research work, some are early experiments, some are rediscovering the wheel, some are misguided ideas, and lucky a tiny part are good. Anyway, they are for internal use, we don't try to contribute to the thermal death of the universe uploading that crap to a free preprint server.
I don't understand:
a) Why they upload internal notes/reports to a preprint server?
b) Why are these reported as "academic papers"?
This is similar to when Elon claims to be running what, 4-5 different companies as a ceo, and taking care of his children, and sleeping 6 hours a day, and then we observe him shitposting on Twitter constantly or showing off his gaming skills in a game that is literally a time gated grindfest.
You add up all the claimed activities and then realize it’s more hours than are in a day so someone’s lying.
I don’t see how this author can claim that these papers didn’t “…suddenly materialize from a magical “chatbot” in 2026” when it’s obvious that 1-2 people are not producing 258 papers that are tens of pages each in 9 months with any sort of academic rigor.
> showing off his gaming skills in a game that is literally a time gated grindfest.
What was even more hilarious was the fact that he had visibly no idea of what the game was even about (he was bragging about item level instead of stat and dismissing the best item of his gear because it was lower iLevel than the rest).
Question is: what was Polson's typical output rate before ChatGPT appeared?
Why would someone ruin their academic credentials and reputation (at a reputable university) like this? Maybe there was nothing to begin with? Otherwise I can't understand this.
I think it might be taking a position in the use of AI, They don't appear to be concealing their use.
If the papers were of a uniformly high standard then I might not have a problem with it.
If they are hoping some of them are good and the review process will filter out the bad then it seems like they are not doing research, they are writing hypotheses and at pushing the work of testing them onto the reviewers.
I could also see that being done to make some sort of statement about the review process, but it's not clear if that is their goal.
He's wasting people's time, which is already the resource that many people lack any to spare. I'm not sure if he realises this part.
Could probably write a real research paper based on the acceptance outcomes of all of these LLM written papers.
This is economics though, a discipline where you be like Kenneth Rogoff and Carmen Reinhart and commit the most impactful fraud of the century and face absolutely no consequence on your career.
When you compare that to the consequence of the hydroxycholoquine paper, which is comparable in public policy impact, where Didiee Raoult lost all its legitimacy and his job, it's easy to see that economics as an academic field doesn't value probity much.
It's incentivized by the system. You get grants, clout, importance, job opportunities based on your publishing in major conferences. As of last year, the variance of review scores have been higher between reviewers on the same paper than between the same reviewer on different paper. So it's effectively become a coinflip. The best strategy when it's random is just to spam stuff.
Oh also at least 40% generated peer-reviews are contributing to this statistic.
Because they won't. The various algorithms that create the "publish or perish" incentives actually reward this. Academic reputation isn't a term in those algorithms, so it isn't rewarded. Same for verifying other's papers and other parts of science and research that aren't actually publishing papers. This is the cause of the reproducibility crisis in academic.
"Show me the incentives and I will show you the outcome"
Reputation is a factor, because most processes involve a subjective evaluation by other experts in the field. And non-experts use service activities such as peer review as a proxy for reputation. If reputable entities use your services as an expert, you are probably a reputable expert in the field.
Not surprising. Many academics held in high-esteem in these behavioral offshoots are nothing more than frauds to begin with. Almost every X-studies paper is equally fluff these days (see the recent frontpage CHI post) - at least these have some semblance of theory given their equations, though are still largely garbage.
More and more evidence is surfacing that suggests AI addiction is real. It may be as simple as that Nicholas Polson has developed a severe case of AI addiction.
Maybe his university is mandating or threatening to fire him like half of America’s corporate tech companies are.
AI psychosis? They think they're actually putting out high-quality papers. Nothing else makes sense.
A ton of academia is based on putting out slop into not so well-read journals and crossing fingers it passes some dead-exhausted postdoc TA's peer review.
Maybe to expose the utter uselessness of "papers".
I'm not an academic, and I understand by osmosis that organized science has many problems related to publishing and connected mechanisms, but I have to say there's been quite a few moments in my life and career when I found the inspiration or solution I needed in a good paper.
For condensed knowledge on highly specific topics (i.e. more specific than is economical in book form) I often know nothing better.
Most recently I did sort of a self-taught crash course in ground-penetrating radar and applications for an archeological endeavour I write software for, and a number of papers have been really invaluable.
I think one of the end-game components for AI is to be able to read all the literature and generate a reliable list of which ones are not correct and a convincing reason why.
> generate a reliable list of which ones are not correct and a convincing reason why
That is not realistic, but I suppose where things are heading is that you have some indicator of the strength of evidence -- see Fig. 13 in the following insightful take:
https://news.ycombinator.com/item?id=49407226
Though, "strength" should probably be "reliability" and "validity", and I suppose those indicators are more for picking signals from the noise; i.e., what is even worth clicking and reading. That would be increasingly valuable already today due to the volume (and, yes, slop and other related stuff).
> That is not realistic, but I suppose where things are heading ...
And to expand on this, it's not realistic because science is not armchair philosophy. You have to go out and measure the world.
Sometimes, through force of will, a person can think deeply about a problem and come up with beautiful theories that explain our measurements. Many scientists had careers like this, probably most famously, Einstein. But it's worth noting that Einstein also got a lot wrong! [1]
Even if we somehow give an LLM the ability to go out and measure things, I seriously doubt that the role of humans in science is done. There's a big difference between "an explanation" and "a good explanation." Ask any physicist. There's a surprising amount of aesthetics involved. Good theories are consistent with the evidence, but it's more than that-- there's a great deal of "taste" involved. And there's a good reason for that. For any real problem, there are effectively an infinite number of alternative hypotheses. From a "theory of science" standpoint, this should cause scientists nightmares, but it doesn't. Because by the time you are a practicing scientist, you've developed a feel for what constitutes a satisfying explanation. If you spend time with scientists, especially in the "hallway track" at a conference, "taste" is a frequent topic of conversation!
[1] https://en.wikipedia.org/wiki/Einstein%27s_unsuccessful_inve...
I think you misunderstood. I'm not asking for an oracle that can determine whether a paper is correct, I want an oracle that can find real mistakes in papers (thus invalidating them).
I work full time on "lab in the loop" AI, so I'm pretty familiar with the need for real-world experiments. I am not proposing a fully autonomous scientist that could read an arbitrary paper and emit whether it's universally true without some verification method.
Also, to your statement: " Because by the time you are a practicing scientist, you've developed a feel for what constitutes a satisfying explanation."
I'm a practicing scientist (well, ex-scientist) and it seems like most "satisfying explanations" end up being wrong or incomplete simply because they seem so satisfying.
> I'm not asking for an oracle that can determine whether a paper is correct, I want an oracle that can find real mistakes in papers (thus invalidating them).
What's the difference? How do you find real mistakes without a model? Either you have a trusted mathematical model (in which case you already have a complete explanation) or you have to compare it against the ultimate oracle: the world. Or are you proposing something like "let's use an LLM to convert this hand-wavy English paper into a formal proof and then check it for logical fallacies?" In which case, fine, that would be useful, but that's not exactly the same thing (and also not as important) as saying that a paper advances a bad explanation. Just that the explanation is flawed in some way.
How do you determine that a mistake is serious enough that it actually invalidates the paper? How do you know that it's not possible to correct it and reach a similar conclusion with fundamentally the same approach? Especially when the fix would be complex enough to justify a follow-up paper.
This is realistic. We already do this today: it's called "journal club". A bunch of grad students read the same paper and then criticize it. I've read papers that I thought were amazing only to. have somebody else notice a key issue in a method, or a conclusion that didn't follow, or outright omission of an important detail, in a way that could be verified by both the students and the authors of the paper.
You can be served this inspiration from LLMs
I've tried here and there, but even the top-end models always tend toward shallow generalist takes. I've not been able to get more than basic primers out of LLMs, and nothing close to the ability of a professional author to stay on course with an idea, detail level, specificity etc.
People in the early days used to often whine that LLMs just regurgitate text snippets (unfounded of course), but I think the way we currently train and RLHF them actually seems to largely make them unable to reproduce the knowledge they have been trained on, since they seem to just always want to please the mean with their output. I'm oversimplifying the mechanisms, but you get my drift.
Alan Sokal style hoax is what comes to mind first, of course. Wouldn't be the first computer-generated hoax either (see SCIgen). However see also the P.P.P.P.S. in the article, his co-author denies that in the comments.
If the past 30 years have shown us anything, it's that academic credentials and reputation are very difficult to blemish.
Polson is just pushing the knob to 11 because it's louder.
We've been in the dark ages of science for so many decades...
Writing is a technology that served us well for over 5000 years, but now times are a changing. Written media will cease to provide any value in the next decade i think. In person chats/Public speaking will be revered again.
Edit - people are assuming that i want this to happen. no, i don't. i am reading more than i ever did before, maybe out of rebellion. i am just stating what I THINK is to come. it might not go this way at all.
One of the most memorable tech talks I ever attended was by a junior engineer who had just come back to the office from a major conference. Our workplace had an unusually oral culture -- decisions were made in meetings, and not always written down; there wasn't much in the way of bug tracking or work planning at the time; people generally preferred to talk through problems instead of working them out in email, chat, or forum posting.
He was supposed to present a topic he learned about at the conference. Instead, he gave a cheeky talk which introduced the very concept of written language to the audience, as if we had never heard of it. Starting from the history of writing on tablets and so on, and ending in practical advice like, "when words are written down, you don't have to remember what they were exactly; you can just read the words again." I loved the chutzpah and could barely hold myself from chortling out loudly at some of his slides.
I know that your post is an amusing take on communication, but I find the post you responded to be kind of a horrifying idea. Until recently, writing was like a time machine: somebody, from almost at any time in human history, wanted to tell you something. By writing it down in a common language, you could read what they wanted to say. Writing is flipping amazing.
There has always been bad writing, but historically it has been easy enough to filter the garbage out. Now we get garbage that looks so much like real thoughts that it's hard to tell. This is not a positive development. Separating the wheat from the chaff is extra work, and the overhead is a burden on humanity. I suspect that this is a burden that almost nobody actually wants, but we're getting it anyway.
its not even writing, its the very concept of human thinking that is under question.
Why should anyone bother to think, if they can ask AI for a well researched, correct, and.. khm... "aligned" answer ?
We will be worse off for it. People just are losing the ability to read well. We are regressing as a species because of billionaire's thinking machines. A few companies are evolving while the rest of us become dependent.
Your intelligence and competencey are going to be 1:1 correlated to the quality and quantity of tokens a billionaire gives you access too (local, sure, they control the hardware markets as well).
Keep excercising your writing and reading muscles because it will definatley set you apart from the captive masses in the future.
> People just are losing the ability to read well.
I wouldn't blame "AI" for that, TV and Youtube and Tiktok did a good enough job.
Speaking is a culture that has served us well for 300 000 years or so. To note, it has been used for both good and bad..
right.
but i am making the case of in-person chat or public speaking, where in-person and public are the relevant parts.
I honestly think it's more likely that younger generations grow up seeing the ill effects of being AI-pilled for communication/writing, deciding by way of AI literacy that they want none of that, and develop better ideas on responsible use.
I really don't like AI for document writing as a whole, especially within organizations. Code can be a form of communication, but for the most part it's an engineered thing that can be re-run, tested, verified and so on. But documents drive decision-making, and decision making is often one-time. Decisions should be studied and understood, and depriving authors of much of both is not having any good effects.
Oh my god, Socrates was right all along!
Chilling.
try speaking from your own perspective instead of assuming a universal one. your opinions, however unhinged, will come off as more genuine.
As it always should have been. Even Plato/Socrates were intelligent enough to muse about the idea that writing has degraded our memory vs when we had to memorize entire epic poems. They had a lot of hostile stuff to say about writing in Timaeus and other works.
The fact that most teachers refuse to do in-person oral exams has always been due to laziness, incompetence, and the reality of the economics of dealing with classroom sizes where 30 is considered small.
I’m only half joking that someone will come up with a LLM text to speech synthesizer that pauses and plays depending on if your mouth is closed/jaw is moving. Analysizes your voice to synthesize as well.
AI voice synthesis and robots/projectors are a thing. Slop will arrive in the boardrooms and on podiums sooner than many suspect.
Yeah, the love doll companies already have voice synthesis and facial expressions. Full locomotion is the next obvious step
If AI submissions, then why not go a step further and have AI reviewers?
Perhaps this will help bring more scrutiny to the metric of publishing quantity not being as valuable as it has been treated.
In practice this already happens, unfortunately.
Why is it unfortunate? We should absolutely use AI reviewers to help catch errors and fact check papers
Because the exact opposite will happen. The idea that allowing AI reviewers will increase accuracy is probably foolhardy. People being people will get lazy and review less by hand if at all. And the LLMs will miss plenty of things. Then those papers will get into the training sets for the next LLMs. I think you can see where this goes overtime.
The issue is generally not accuracy in my experience (humans are even lazier about this). The issue is that LLMs do not understand what is interesting and what is procedural.
> Why is it unfortunate?
If you only knew.
Someone in the comments mentioned this:
> As someone who has known Nick in the past, I’m a bit surprised by this multitude of articles and books. However, I think you’ve missed a fifth possibility that would suggest gentle handling: a mental health issue. The papers and books on philosophy, religion, and issues well beyond anything econometric plus the videos on the Riemann Hypothesis suggest someone who is no longer bound by the intellectual boundaries or social mores of proper research.
I don't know whether the commenter is legit, and whether this actually happened, but I wonder if AI-psychosis (or at least, AI-amplified psychosis) might not become a real thing. People already on the edge having "productive" conversations with LLM chatbots and going down rabbit holes. It can't be a good thing, especially since LLMs are sycophants that indulge every crazy idea you have.
I'm not saying this didn't happen before, I just wonder whether LLMs enable a whole new scale of crazy.
https://en.wikipedia.org/wiki/AI-induced_psychosis
I tried writing a simple 20 page paper with the latest Claude model over several weeks and that thing got ripped apart by legal and data science (for good reason), not going to try that again. And there I had all the numbers and analyses down and verified, just the writing itself by Claude was subpar.
Can’t imagine the utter garbage that Claude will produce when you give it a simple prompt and let it write thousand of pages.
Kind of makes me wonder about all that AGI talk if these models can’t even the work of a good PhD or postdoctoral student.
What percent of these 258 papers was funded with taxpayer money?
A benign but depressing interpretation is that even competent and respected academic researchers are not immune to AI-addiction and episodes of AI-psychosis cause by binging on AI.
The dopamine hit of having even their most speculative ideas expanded in a convincing paper format is self-reinforcing and soon the affected individual starts believing that the output is perfect and needs no review, because reviewing it would delay the next stimulation cycle.
The SSRN ID of the page linked to by the article [0] is not the same as the SSRN ID [1] linked from a chicagobooth.edu scholar by that name [2]
0. https://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=170...
1. https://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=228...
2. https://www.chicagobooth.edu/faculty/directory/p/nicholas-po...
Years ago I was an academic peer reviewer and hated it, you've probably seen published work thats low in quality - you wouldn't believe how low quality the unpublished crap is. I cannot even imagine the absolute bundle of garbage that must be handled by these unpaid reviewers today. We need a better system.
Another possible answer to the "What's in it for him?" question:
The Everest effect.
George Mallory: "The first question you will ask, and the one I have to answer is, 'What's the point of climbing Mount Everest?' And my answer should be, 'It's no use.' There is not the slightest prospect of any gain. Oh, maybe we can learn a little about the behavior of the human body at height, and medicine can change our observations for aviation purposes. But nothing else could come of it. We will not bring back a little gold or silver or gemstones or coal or iron. We will not find soil or earth that we can grow crops that produce food. There is no point. So if you can't understand that there's something in humans that responds to the challenges of this mountain and goes to meet those challenges, then the struggle that ensues is a struggle from life itself upwards, and you won't see why we're leaving. What we get from this adventure is just joy. We don't live to eat and make money. We eat and make money to enjoy life. That is the meaning of life and the purpose of living."
Why generate all of these papers? Because, for the first time, he can.
Doesn't have to be for the money or fame or potential for speaking engagements. It could simply be that he wants to test the limits of what's possible.
> Jesus Christ. I’ve heard that Cambridge University has an open position in their school of education . . . this kind of thing would fit in very well there, no?
What a disgusting, tasteless joke.
Ah, the pearl-clutching woke grifters have arrived.
How many marathons did you run today?
my Z_spiritual(T) has been unusually high recently because all of this slop is supercharging the value I've been generating through ego-confrontation and progressive liberation
Squatting seems like the most plausible explanation. Explaining an idea to your friend at a bar doesn't make it "yours", but publishing a manuscript does.
Nicholas Polson is an anagram for Nonsocial Slop.
You lost the ‘h’.
I'll settle for NO SLOP, NICK! as an anagram of NICK POLSON.
"Pols" backwards is "Slop".
Polson - NoSlop
it was right in front of you :)
Nicolas Polson?
Sin: A Slop Colon!
Call me crazy, but if you can't explain the idea in 10 pages or less, you don't understand the idea.
(For instance, 58 pages, 80 pages ... I don't think of these as normal academic length if this is the main body content ...)
Wiles proof of Fermats last theorem was over 100 pages
There are obviously exceptions (e.g., the specification for the Java programming language is well over 100 pages [1] and is extremely low on filler), but most ideas don't work like this. In my area, 10 pages really is enough most of the time.
[1] https://docs.oracle.com/javase/specs/jls/se17/html/index.htm...