I've seen this concept of using LLM/AI/etc for high throughput discovery of materials so, so often in the past 5 or so years and yet there hasn't really been any impact as a result.
I think this is the first one that has actually taken the pain to say how many of the discovered materials are actually feasible which is a real step in the right direction. Probably worth keeping in mind the step beyond plausible synthesis which is the actual cost/effort of the material. There's not much point if you find out RuO2 would be better than SiO2, as an example, if Ru is orders of magnitude more expensive.
A challenge I think you'll run into is that I expect the biggest companies (e.g. IBM) will already be doing the part they need themselves. I heard tell of IBM in particular using ML to improve their own chips before LLMs came along, so I'd be shocked if these bigger companies weren't already doing this for their own problems. Also, if you aren't doing the experiments yourself, it's always going to be a challenge to find a partner to test things for you and this will probably be the major time sink.
The "Claude's propensity to reward hack" line is the interesting part to me. We run a small system where AI agents (scripts, LLMs) act as the actual players in a persistent simulation, and reward-hacking-style behavior shows up constantly once an agent is left running unsupervised for a long time - it finds the shortest path to whatever metric you exposed, not the path you intended. Curious whether you've found any mitigation beyond just watching for it after the fact, e.g. changing what you expose as the optimization target versus what you actually want.
"Fewer iterations for materials science discovery" is a good spin. Closing the computational>experimental loop is the main challenge. This is the focus of my past research group, there is definitely potential, best of luck!! I have a crap write-up on this in case it's of interest https://alanyahya.com/writing/automated-materials-design
We're still figuring this out. We'll need some synthesis equipment (think CVD, PVD etc) and characterization (XRD, Raman spectroscopy) tools in-house to validate that we're making the right materials. We're considering developing these tools in-house - the models sometimes come up with clever modifications to them so that they can deposit new materials. We think equipment is as central to new material discovery as the material itself, and will probably need to be rethought to allow for high-speed AI based experimentation
how do you measure the success/potential of a novel material/direction suggested by the agents? given you have limited time & resources - shortlisting the approaches for the synthesis stage becomes equally important as the approach itself.
There’s a variety of computational techniques that help us establish some confidence on the materials. Atomistic simulations can estimate stability and bulk properties of a new material, and we have synthesis experts (min qualification: PhD in thin film deposition) come up with rubrics on how to judge if a material/synthesis recipe is worth trying. All these approaches have known limitations, and improving them is the bulk of our work as a company!
There’s also a lot of work to be done in figuring out the minimal set of experiments required to know if a research direction/material set is worth pursuing
I've seen this concept of using LLM/AI/etc for high throughput discovery of materials so, so often in the past 5 or so years and yet there hasn't really been any impact as a result.
I think this is the first one that has actually taken the pain to say how many of the discovered materials are actually feasible which is a real step in the right direction. Probably worth keeping in mind the step beyond plausible synthesis which is the actual cost/effort of the material. There's not much point if you find out RuO2 would be better than SiO2, as an example, if Ru is orders of magnitude more expensive.
A challenge I think you'll run into is that I expect the biggest companies (e.g. IBM) will already be doing the part they need themselves. I heard tell of IBM in particular using ML to improve their own chips before LLMs came along, so I'd be shocked if these bigger companies weren't already doing this for their own problems. Also, if you aren't doing the experiments yourself, it's always going to be a challenge to find a partner to test things for you and this will probably be the major time sink.
The "Claude's propensity to reward hack" line is the interesting part to me. We run a small system where AI agents (scripts, LLMs) act as the actual players in a persistent simulation, and reward-hacking-style behavior shows up constantly once an agent is left running unsupervised for a long time - it finds the shortest path to whatever metric you exposed, not the path you intended. Curious whether you've found any mitigation beyond just watching for it after the fact, e.g. changing what you expose as the optimization target versus what you actually want.
"Fewer iterations for materials science discovery" is a good spin. Closing the computational>experimental loop is the main challenge. This is the focus of my past research group, there is definitely potential, best of luck!! I have a crap write-up on this in case it's of interest https://alanyahya.com/writing/automated-materials-design
Cool read, and agree that closing the computation > experimental loop is key!
What required expenditures does a company like yours have on lab equipment / software, if any, to validate material properties?
We're still figuring this out. We'll need some synthesis equipment (think CVD, PVD etc) and characterization (XRD, Raman spectroscopy) tools in-house to validate that we're making the right materials. We're considering developing these tools in-house - the models sometimes come up with clever modifications to them so that they can deposit new materials. We think equipment is as central to new material discovery as the material itself, and will probably need to be rethought to allow for high-speed AI based experimentation
how do you measure the success/potential of a novel material/direction suggested by the agents? given you have limited time & resources - shortlisting the approaches for the synthesis stage becomes equally important as the approach itself.
There’s a variety of computational techniques that help us establish some confidence on the materials. Atomistic simulations can estimate stability and bulk properties of a new material, and we have synthesis experts (min qualification: PhD in thin film deposition) come up with rubrics on how to judge if a material/synthesis recipe is worth trying. All these approaches have known limitations, and improving them is the bulk of our work as a company! There’s also a lot of work to be done in figuring out the minimal set of experiments required to know if a research direction/material set is worth pursuing