On the flip side, singularity or no singularity, we are not remotely close to a Lean proof that some treatment will work.
Pure math has conjectures and we know, a priori, a criterion that defines a proof or disproof. (We can quibble about the format. Do we want Lean? Rocq? A human readable thing?)
In medicine, the proof is in the trial data, period. Except that even that may not actually be good enough — problems may show up later. And a swarm of agents in an RL system cannot inspect transcripts to figure out WTF is going on in patients. If you want data, you do an an experiment. If you want 100 variants of that, you can’t just spawn 100 sandboxes.
Oh, and once the trial is done, it’s done. You can try to “distill” data from it, but you are not about to rerun the trial with rollouts from the next model in the pipeline. It may even be literally impossible: once the first credible drug for a previously untreatable fatal condition is available, you can’t do a another controlled trial against placebo.
Non-physicians would be shocked at how many medications we routinely use that have no agreed-upon mechanism of action. It’s not just cancer drugs—we really don’t know how many other drugs including anesthetics work despite their critical importance. Historically in medicine, you just treat these meds like a black box. If a reasonable dose produces a reliable outcome in a large number of people with no major side effects, you consider the drug reasonably safe to use.
Anyone who has trialed more than two psychiatric medications learned quickly that providers are basically throwing drugs at the wall to see what sticks for the patient. They know what a drug does, and for what symptoms, but they often have no clear idea why. The brain is wildly complex.
This is the crux of the difference between evidence-based medicine and science-based medicine. Is knowing that it works enough, or must you understand the scientific mechanism?
I think the distinction the commenter is going for is science which has been tested empirically, versus science which has been proven theoretically. It's like the difference between unit testing and formal verification of code, I guess. They both have their uses and are practical in different scenarios.
It’s an epistemic difference. In one case you have empirical evidence that A -> B even though the whole thing is a black box and really it only works sometimes. In the other, theory-based case, you have a model of how things work and that model has been predictive in some cases.
In medicine sometimes (more in the “old days”) someone would see that compound X caused Y and then they’d try to make a therapy that helped the patient without killing them. Other times, someone has a theory that pathway P seems to cause indication I, and so tries to develop a compound that relatively safely disrupts P in vitro, then tries it in vivo. Sometimes even after the drug works and is approved we later find the pathway P hypothesis was wrong, yet it somehow works anyway. And then people try to figure out what’s really going on.
And then, especially in psych and oncolo drugs, the meds mysteriously work on some people but not others. Teasing that out is tough.
I’m having trouble finding more background on the “science-based medicine” you refer to. Wikipedia refers to a website operated by two members of the “New England Skeptical Society.” The disambiguation link redirects to “evidence-based medicine.”
Misaligned to exploit the trial process is actually something to consider carefully. I would not put it past a drug company to design a phase 3 trial that has some subtle bias towards commercialization. And if we're just getting cocktails and trial protocols from the AI, it would be very wise to be skeptical.
For example, if the prompt is something like "give me a cocktail that is most likely to succeed in this specific phase three trial" there are any number of ways the AI might find ways to game specific features of the trial design.
Tao is obviously completely unfamiliar with the PED gray market, and what people are willing to put in their bodies that hasn’t even been verified by the likes of an LLM. If there’s a promise of changing or controlling their corporeality in some way, many people are shockingly cavalier. If it’s because you are going to die, but could live with this experimental treatment, the risk tolerance is that much higher. I agree he’s chosen a pretty tone deaf analogy.
> If AI conjures up a cocktail to cure cancer and we don't know how it works, but it passes a phase 3 trial, you can bet we will be on our knees thanking it, or more likely giving a standing ovation at our plenary session.
Ironic that his vaccine skepticism evaporates when there’s an opportunity to be contrarian. Or maybe he’s changed his mind now that the overwhelming evidence of the Covid vaccines proves he was full of shit
It may have seemed like fun and games to them at the time, but a consequence of having been on the wrong side of covid issues for cable news attention means people like Prasad won't be taken seriously on anything else anymore, even on issues where they might be right.
The current state of things is that AI has put us on the cusp of a future exactly as Kevin Flynn said in Tron: Legacy: "Disease? History! Science, philosophy... every idea man has ever had about the universe up for grabs!" We don't need to understand every bit of it to benefit from it; understanding will come with time, when our meager minds are ready for it.
We should be joyful and excited. Instead, those of us who gatekept knowledge, including us software engineers and even mathematicians, are viewing this future with fear and trepidation. Because we lose our power, prestige, and leverage over the rest of society.
On the flip side, singularity or no singularity, we are not remotely close to a Lean proof that some treatment will work.
Pure math has conjectures and we know, a priori, a criterion that defines a proof or disproof. (We can quibble about the format. Do we want Lean? Rocq? A human readable thing?)
In medicine, the proof is in the trial data, period. Except that even that may not actually be good enough — problems may show up later. And a swarm of agents in an RL system cannot inspect transcripts to figure out WTF is going on in patients. If you want data, you do an an experiment. If you want 100 variants of that, you can’t just spawn 100 sandboxes.
Oh, and once the trial is done, it’s done. You can try to “distill” data from it, but you are not about to rerun the trial with rollouts from the next model in the pipeline. It may even be literally impossible: once the first credible drug for a previously untreatable fatal condition is available, you can’t do a another controlled trial against placebo.
Know your unknown: that is highest.
Mistake your unknown for known: that is sickness.
Only be sick of the sickness, and you are not sick.
The sage is not sick: sick of the sickness, he is not sick.
— Laozi, Tao Te Ching, Chapter 71
Non-physicians would be shocked at how many medications we routinely use that have no agreed-upon mechanism of action. It’s not just cancer drugs—we really don’t know how many other drugs including anesthetics work despite their critical importance. Historically in medicine, you just treat these meds like a black box. If a reasonable dose produces a reliable outcome in a large number of people with no major side effects, you consider the drug reasonably safe to use.
Anyone who has trialed more than two psychiatric medications learned quickly that providers are basically throwing drugs at the wall to see what sticks for the patient. They know what a drug does, and for what symptoms, but they often have no clear idea why. The brain is wildly complex.
This is the crux of the difference between evidence-based medicine and science-based medicine. Is knowing that it works enough, or must you understand the scientific mechanism?
What a weird terminology given the fact that evidence is the core component of science
I think the distinction the commenter is going for is science which has been tested empirically, versus science which has been proven theoretically. It's like the difference between unit testing and formal verification of code, I guess. They both have their uses and are practical in different scenarios.
How can you prove satements about reality theoretically?
It’s an epistemic difference. In one case you have empirical evidence that A -> B even though the whole thing is a black box and really it only works sometimes. In the other, theory-based case, you have a model of how things work and that model has been predictive in some cases.
In medicine sometimes (more in the “old days”) someone would see that compound X caused Y and then they’d try to make a therapy that helped the patient without killing them. Other times, someone has a theory that pathway P seems to cause indication I, and so tries to develop a compound that relatively safely disrupts P in vitro, then tries it in vivo. Sometimes even after the drug works and is approved we later find the pathway P hypothesis was wrong, yet it somehow works anyway. And then people try to figure out what’s really going on.
And then, especially in psych and oncolo drugs, the meds mysteriously work on some people but not others. Teasing that out is tough.
I’m having trouble finding more background on the “science-based medicine” you refer to. Wikipedia refers to a website operated by two members of the “New England Skeptical Society.” The disambiguation link redirects to “evidence-based medicine.”
https://en.wikipedia.org/wiki/Science-Based_Medicine
Is this the school of thought that you mean?
First time I have come across this idea that not all is known about the chemicals we put into our bodies
I always assumed the research and trials are there just for that.
I understand from an engineering perspective.
Newton invented and used Calculus to great effect, long before it was formally proven to be consistent and logical.
Misaligned to exploit the trial process is actually something to consider carefully. I would not put it past a drug company to design a phase 3 trial that has some subtle bias towards commercialization. And if we're just getting cocktails and trial protocols from the AI, it would be very wise to be skeptical.
For example, if the prompt is something like "give me a cocktail that is most likely to succeed in this specific phase three trial" there are any number of ways the AI might find ways to game specific features of the trial design.
> Terrence Tao does not understand cancer medicine
As opposed to?
I have a feeling you didn't have time to open the link and read it.
Tao is obviously completely unfamiliar with the PED gray market, and what people are willing to put in their bodies that hasn’t even been verified by the likes of an LLM. If there’s a promise of changing or controlling their corporeality in some way, many people are shockingly cavalier. If it’s because you are going to die, but could live with this experimental treatment, the risk tolerance is that much higher. I agree he’s chosen a pretty tone deaf analogy.
>In response to the COVID-19 pandemic, Prasad used social media to promulgate misinformation and take a contrarian stance.
https://en.wikipedia.org/wiki/Vinay_Prasad#COVID_response
> If AI conjures up a cocktail to cure cancer and we don't know how it works, but it passes a phase 3 trial, you can bet we will be on our knees thanking it, or more likely giving a standing ovation at our plenary session.
Ironic that his vaccine skepticism evaporates when there’s an opportunity to be contrarian. Or maybe he’s changed his mind now that the overwhelming evidence of the Covid vaccines proves he was full of shit
It may have seemed like fun and games to them at the time, but a consequence of having been on the wrong side of covid issues for cable news attention means people like Prasad won't be taken seriously on anything else anymore, even on issues where they might be right.
The current state of things is that AI has put us on the cusp of a future exactly as Kevin Flynn said in Tron: Legacy: "Disease? History! Science, philosophy... every idea man has ever had about the universe up for grabs!" We don't need to understand every bit of it to benefit from it; understanding will come with time, when our meager minds are ready for it.
We should be joyful and excited. Instead, those of us who gatekept knowledge, including us software engineers and even mathematicians, are viewing this future with fear and trepidation. Because we lose our power, prestige, and leverage over the rest of society.