Introduction
In my first (real) podcast episode, I talk with Corin and Ari Wagen, two brothers who I met through my writing. They are building something super cool: a molecular simulation company called Rowan (which recently got into the Nat Friedman AI grant program). We discuss neural network potentials (NNP’s), whether dynamics are useful at all, the role of computational chemistry in drug design, what the future of the field looks like for molecular simulation, and a lot more. Also, for almost every paper/result discussed here, I attach the reference in the transcripts below!
If you work in molecular simulation, I recommend trying out their tool at rowansci.com. I’m not a chemist and cannot vouch for the tool personally, but I can vouch for how much I’d trust Corin and Ari to build something useful. Not a paid sponsorship, not anything I have an investment in, my opinion and no one elses, etc, etc, I just genuinely want their startup to succeed.
Finally, if you enjoyed this podcast and would want me to make more, please subscribe and consider upgrading to a paid subscription! Studio time is expensive, and, while Rowan generously agreed to cover the cost of filming, I don’t want to make that be a barrier to interviewing people. I have many ML-biology scientists I’d want to interview, specifically in the realm of cryo-EM, antibody engineering, and radiology, and it’s much easier to do that if filming is covered by this blog. I’ll probably still film stuff anyway, since I’m mentally bucketing this blog as ‘expensive hobby’, but money still helps!
Jargon explanation
This podcast is really meant to be consumed by people at least vaguely familiar with the molecular dynamics (MD) field. If you’re confused, I have written up a primer to MD here, which may be useful. Corin’s blog here is also incredible for understanding deeper nuances of the area. Rowan’s blog is also quite good, especially this one: the role of quantum chemistry in drug discovery.
Here is some breakdown of jargon used in the episode:
‘Molecular dynamics’ and ‘molecular simulation’: Technically, dynamics refers to time-dependent simulations, whereas simulation in general can also be time-independent. I use these interchangeably, partially because I’m not an expert, but also partially because even people in the field sometimes use them interchangeably.
‘Levels of theory’: This is equivalent to saying, ‘how close are we simulating the true nature of reality?’. In this case, the ‘truth’ is the solution of the Schrodinger equation, which is usually intractable to solve. Higher levels of theory means you’re closer to the equation (more accurate, including in quantum mechanics), lower levels means you’re further from it (potentially less accurate, usually only classical mechanics). We sometimes refer to ‘density functional theory’ or ‘coupled cluster theory’, both of these are at the higher end of the theory spectrum.
‘Tim’s tweet’: This is in reference to Timothy Duignan, a minor celebrity in the world of neural network potentials. He wrote what is likely the most famous thread in this niche field, where he showed a crystal nucleation event using a neural network potential that had never before seen nucleation events.
‘Periodic systems and molecular systems’: Rowan has a good article on this. But, just to recap:
Molecular [systems] are exactly what they sound like—isolated molecules or groups of molecules surrounded by a vacuum (or a dielectric field). This is good for studying small molecules, clusters, or even larger biomolecules….
However, most physically relevant materials are so large as to be effectively infinite relative to the molecular scale. Cutting out a chunk of these materials and modeling them with molecular calculation introduces significant edge effects. To solve this problem, we [model them as a periodic system].. Materials can be modeled using a single unit cell, where the molecule or group of molecules “sees” itself tiled infinitely in all dimensions.
Timestamps:
01:19 Divide between classical and quantum simulation
03:48 What are NNP's actually learning?
06:02 What will NNP's fail on?
08:08 Short range and long range interactions in NNP's
10:23 Emergent behavior in NNP's
18:16 Cultural distinctions in NNP's for life-sciences and material sciences
21:13 Gap between simulation and real-life
41:49 Is molecular dynamics actually useful?
55:17 Quantum effects in large biomolecules
57:03 The legacy of DESRES and Anton
01:02:27 Unique value add of simulation data
01:06:34 NNP's in material science
01:13:57 The road to building NNP's
01:21:13 Building the SolidWorks of molecular simulation
01:41:06 The role of computational chemistry
01:51:23 Selling to scientists
02:01:41 What would you spend 200 million on?
Transcript:
[00:00:00] Introduction
Abhi: Today I'll be talking to Corin and Ari Wagen, two brothers who are co founders of Rowan, a quantum chemistry simulation startup.
Of note, Rowan was recently accepted into the Nat Friedman AI grant program. Congratulations. Past that, I believe Corin is one of the most interesting thinkers in the intersection of molecular dynamics and machine learning today. He also runs an incredible scientific blog, which I'll attach in the description of this video.
Thank you both for being on the show today.
Corin: Thanks for having us.
Abhi: So first question, just to set the tone for the rest of this podcast, give me a high level overview of what molecular dynamics is and what neural network potentials are.
Corin: So molecular dynamics, is a way that we can study the dynamics, how they evolve over time of molecules.
So a lot of calculations focus on taking a static molecule and, asking some question about it. Molecular dynamics also lets us sort of time integrate equations of motion. Basically, we can make videos of molecules moving around and learn things from them. Neural network potentials are a way to accurately predict a lot of things, but most relevantly for this energies and forces of molecules.
So it lets us get accuracy that's much closer to the truth to quantum mechanics. at a fraction of the cost that usually takes. It's a more accurate replacement for traditional force fields.
[00:01:19] Divide between classical and quantum simulation
Abhi: One, immediate question I had when I was learning about this field a few months ago was this, divide between classical mechanics and quantum mechanics. You wrote this post a while back called the two cultures of atomistic simulations. I'm curious, could you just recapitulate that for the audience?
Corin: Just briefly, it's useful to have some historical context for the field.
So the field of computational chemistry is about 100 years old. and back in the old early days of Heisenberg, people, did calculations on two atom molecules on pen and paper, with the advent of computers, I'd say in the seventies, the field grew in two different directions. So some people wanted to scale up the very rigorous, very physics based approaches, the quantum mechanics, to larger and larger systems to the limits of the hardware.
And so this grew into quantum chemistry, where now you can model up to a couple hundred atoms, with, very good accuracy, derived from first principles with all these layered approximations. And then the other half of the field said, Let's simulate the stuff we really care about, like DNA, like proteins, like these complex biological systems.
And let's work backwards from how fast things need to be, and invent a theory that's fast enough to do this. And this is essentially molecular mechanics. So the early, CHARMM and AMBER work, you're essentially using polynomials to fit quantum mechanics, and it you know, it works astonishingly well.
Like you could model protein movement. You can model like antibody, like motion sort of solution structure around things. and this was just, I think blew the field open in the late seventies and early eighties. And so now what you have is you have these two opposing paradigms were what we'll call classical molecular dynamics, more of the biological side of simulation has results that you can model the things you care about, but you get the wrong answers because the theory is wrong and the quantum mechanics side of things.
You get very accurate results, but on things which are less immediately relevant. And so I think a huge open challenge is how do we now, 50 years later, start to try to bridge this gap and, bring accurate simulations to the things we care about, which it seems like for the first time we're maybe finally able to do.
Corin: Yeah, I think that's exactly right. And there's, this is a goal a lot of people have had. So this is, the quantum computing people talk in very similar terms, I think neural network potentials are there.
Definitely right now look like the closest and by far the most promising way to do that. that's what we, and I'd say a majority of the field is really excited about.
[00:03:48] What are NNP's actually learning?
Abhi: Just to give some background context, a lot of neural network potentials are based on this. or like train on this approximation of the Schrodinger equation, density functional theory.
What I've always found a little bit interesting is what these neural network potentials are actually learning about physics. Is it, do you believe it's learning some lower dimensional manifold of the results of the Schrodinger equation? Do you think it's something else entirely?
I guess some other contexts is for protein structure models. The prevailing theory is that they're doing some sort of fuzzy homology search and then local energy minimization on top of that for neural network potentials, which what's actually going on inside there?
Corin: I think we don't really know, like we're just seeing some of the interpretability work come out on ESM2, using sparse auto encoders to try to understand what actual like features in the feature space correspond to.
And I don't think anyone's done anything like that on neural network potentials, although hopefully they will. But I think in trying to understand how it's possible, like how is it even possible to speed things up so much? I think the scope of things we care about in the context of life sciences, even just molecules that can exist on earth, is so much more restricted than the scope of all possible molecules. So quantum mechanics is almost too good. Like people do benchmarks where you put random elements in random places in space, and then you score a quantum mechanics methods based on how well they do relative to very high level, like non approximated methods.
So you can score approximations this way. It's the mindless benchmarking. And you can say, we're okay being bad at, a beryllium here, a radon here, a technetium here, and, a krypton here. we don't need to be good at that. If we just learn that 15 elements that are in the human body, in ways that would not immediately explode in contact with our atmosphere, we have such a low dimensional slice of like chemistry that we need to learn that we can give our models an inductive bias in that direction.
Abhi: Like there is no free lunch here. You are going to be like failing on some weird out of distribution space, but you're fine with that.
Corin: Yeah. I think maybe what the field hasn't appreciated and what I didn't appreciate until I started, this is just how vast the starting distribution for like chemistry, the field is for like any combination of atoms that like, even taking like a tiny slice of that well encompasses like everything we could care about.
[00:06:02] What will NNP's fail on?
Abhi: Another question I had was after AlphaFold2 was released, you saw this rush of papers claiming that AlphaFold2 failed on this like weird branch of kinases or globular proteins. There's a lot of immediate pessimism after some really interesting result from the field pops up and whether that pessimism is like actually, like real or not.
it just, it happens consistently every time. What do you think that will be for machine learned force fields?
Ari: I think the thing people are pointing to is these long range interactions.
I have one charged particle and it's 20 angstroms away from another charge particle and the model has a cutoff radius of 10 angstroms. And so the particles don't see each other. And It's as if they were, infinity angstroms apart in the energy of the model returns. And you're like, look, these neural network potentials aren't good for anything.
And I think people are working on a lot of different, charge handling schemes. but I think a question to ask whenever people have this pessimism is is this case that you're pointing out is like a, known failure, something that we're going to be trying to model? Does it matter and do we need to remedy it?
And I think with charge handling, that's still like a very open question.
Corin: Yeah, I think that's exactly right. And I think there's, it's going to be really important to figure out like, to what extent is failure predictable or non predictable, right? Because something that works 80 percent of the time is very useful if you know which 80 percent of the time.
So if bad for globular proteins, that you can just not use it for globular proteins. If there's this stochastic hallucination problem, I think that will be a much bigger issue. And we're seeing this, Ari's done a lot more benchmarking than me on the state of the art models, but there's some we see are really good for confirmations, there's some we see are really bad at that, some for thermochemistry.
And I think it'll be, we're very used to the approximations we already have in computational chemistry. We have an intuitive sense for what would be good or what would be bad. And we'll need to build up the exact same intuition for neural network potentials. which will just take time and practice and hard work.
[00:08:08] Short range and long range interactions in NNP's
Abhi: A lot of these neural network potentials, like Ari mentioned, are largely based on modeling the short range interactions between atoms and long range interactions, like electrostatics, are left to purely physical equations that like go through the usual, classical mechanics process.
Is it clear when, when purely modeling short range interactions and like deferring everything, deferring physics to long range interactions will fail. Or is that also unknown?
Corin: I think it's a super huge open question. I think it's one of the biggest sort of architectural puzzles facing the field.
And you can get people with very strong opinions that are like directly in conflict with one another, all of whom seem very smart. So there's there's a body of work that says message passing is all you need, like scale fixes this problem, like just scale it up more. yeah, at the limit of low data, you can't learn the long range things because they require more data to learn, but like just 10x the scale and it will all be fine.
There's another field like body of work that says it's too much to throw all the physics out. We should mix the easy physics back in that will make things much more robust, much more stable. Then there's another body of work that says the architectures are all wrong, like chemistry is less local than we think.
We need to mix like descriptors across coarse grained link scales. And I think it's fundamentally really hard to answer this until we get just better. Like we, we need to figure this out experimentally. I don't think we can armchair solve this problem.
Abhi: Yeah, that makes sense. I like, do you personally have a bet that you're making?
Corin: I can say that with our current generation, we're trying the message passing is all you need thing, because that's, that's the same thing that Meta's FAIR-Chem team has done. There's a lot of like toy systems where you can show that it can have problems, but if you keep like it seems fine for everything that matters and there's cases you can find where people have tried to add in very fancier solutions and it's just worse.
So it seems like the default option right now is just try just building a regular graph, and then we'll see, I think we'll learn a lot from this either way once we're finished benchmarking our current model and maybe update for the next generation.
[00:10:23] Emergent behavior in NNP's
Abhi: One thing I've often seen, and I think many people have seen, you often see this emergent behavior in these large general models, like the zero shot linguistic capabilities in GPT3 and more relevant for the biology world, this, protein conformation generation ability of Alphafold2.
Is there some analog to that in the neural potential world?
Corin: One of the things that's different about these models is that they're not super generative in a sense. So the most basic use cases, you take a cloud of atoms and maybe some like metadata, like charge and spin, and then you return like an energy and the forces of which are the derivatives of the energy.
So it's very like you very restricted output schema. Like you're, doing a simulation and it's, you don't want to be surprised by the energy. Like you don't want an unexpected energy. Like you. The ideal thing is you get exactly the energy you would have gotten from running that like reference level thing that you're trained against.
I think what will be super interesting and where I do think we might be surprised is, like there, there's a lot of work on multi head outputs or what happens inside the model? And can we stitch this together? Can we combine all of the sort of weights and like representation that we get from these like really large, really accurate simulation methods and can we do unexpected things with that and that's like a vague, that's not like a specific proposal but there's you know people show with like language transformers that you take like a math model and a Japanese model and then it can do math in Japanese like what's the analog for that in chemistry?
Like what's that gonna look like? I'm not sure but it seems like if you can train a model to always predict structure to energy, you've learned something pretty fundamental. Like that in some sense is the most fundamental relationship in chemistry and it seems like that should be transferable. Like some amount of internal intuition should be transferable to other tasks.
Abhi: I'm not sure if I'm like reading this correctly, but there was like that paper by Unke and also like Tim's crystallization work.
I guess for like context for people who haven't seen like Tim's tweet, they observed, nucleation, crystal, like crystal nucleation events. using neural network potentials that had never seen nucleation events before. And I think fairly someone in the responses replied that the nucleation event itself, shouldn't have happened in real life, but it's cool that like, structure arose where there was no structure at all.
Do you, that is also potentially an example of emergent behavior?
Ari: Definitely really cool. And I think, one thing that's just I naively exciting about this is you could train a model and it could work on multiple phases, right? If you think if Tim trained that model on, liquid and solid crystal data, it seems like, okay, the model is able to make these phase transitions.
And so you can start to see a path towards you know, a foundation model for atomistic simulation, or something that can handle, like, charge and phases. And, hopefully eventually radicals and transition metals. And, this is one of the things people are thinking about is, how do we, expand the coverage of these models to work on all sorts of, chemistries.
Which is I think a very different research problem than like, how do we scale these models to work on really big systems? And they're both like very promising and interesting research questions.
Corin: I think it teaches you something really fundamental about how information flows too, which is Even if you might say that the event that, dissolved salt shouldn't form solid salt under those conditions, like the description of the event wasn't quite right, it shows that like training on liquids teaches you something about solids.
And that I think is like really cool because that's the premise of a large pre-trained model in the first place, right? Which is that like when we dump all this data in, like somehow the data from other domains is making my domain better. Because otherwise, like why not just train a separate model for every protein or for every task?
Why, like why train a large language model? Why not train a code model and a math model and a translation model? Like we have this idea that somehow in language space, like you get better at language as one unified thing and that training on math somehow makes my code better even if it's not like as direct and I think we're we're seeing this from this as well like training on one phase makes my other phase like there's information transfer there, which is really cool.
Abhi: Do chemists naively bake this information flow into their mental models of like liquids and solids or for them like these domains are separate.
Corin: Ooh. If you think in inspecting the internal states of models is hard, chemists are way harder. But I, I think, yes, I think there's some, there's a language of chemistry, like you think in terms of structures and drawings and like you have a sort of, I don't know, a specific like ontology or metaphysics as a field.
And I think that does transfer between phases pretty well.
I think this really gets the nature of what is interpolation and what is extrapolation?
Which is such like a fundamental question out like just not even in chemistry just like in general. Like I think Conway's Game of Life. There's the cellular automata and you can make, it's like a, each cell only understands the positions of its neighbors, but you can build these like massive emergent systems that display this like complex, like Turing Complete behavior.
And it's, is it that interpolation or extrapolation? Like you're interpolating in like rules space, but you're like extrapolating in like outcome space.
Abhi: Yeah.
Corin: And I think that's like the analogy that makes sense to me here because it, turns out like, it seems like crystallization in water should be like an extrapolation in rule space.
But it seems from the model, like it's an interpolation in like rule space. And then the rules are more fundamental than it seems. it seems like there should be a separate sort of like physical behavior governing like dissolved sodium chloride and like solid sodium chloride. But at least superficially, it looks like it, it learns the both of them just fine.
And so I think this is like a, yeah, one way to think about neural network potentials is conservatively, like just assume they can learn with enough data in a given area, like they can learn like the rules of that data very well. They can learn the chemistry that's contained within that domain.
And then you can run simulations, see like how much. How extensible are those? Like in some sense, like how, much ground does that actually cover in output space?
[00:16:58] Enhanced sampling
Abhi: On the topic of enhanced sampling, I feel like I see relatively few papers bringing ML into the picture.
And it feels like enhanced sampling is one of those very, magical areas where you're very much supposed to, know what you're supposed to be doing before you ever touch it. Is, do you see ML, poking its way into, unphysical, like, modifying the system in such a way that you get to the spots that are interesting.
Corin: I think, yeah, a hundred percent. Hannes and Bowen have done some work on this. I think a bit, like John Chodera has tweeted about this a few times. It's gonna happen. I think the reason why this is so like finicky to get right is that to generate steps, like you want to generate a distribution that's like consistent with the Boltzmann distribution.
Otherwise all your like, binding affinity integrals or whatever property integrals you get are wrong. If you just, you can imagine a lot of ways to enhance sampling where you just, shuffle things around, like a grid search, but those don't, reproduce the correct answer. you, you need it, you need, the right Boltzmann acceptance criterion in, the StatMech terms.
And getting the ML to, rigorously reproduce the, right physical limit is, It's tough, like you have it or you don't. So there, there has to be some way to bake in the like verifiable correctness that a lot of other schemes have.
[00:18:16] Cultural distinctions in NNP's for life-sciences and material sciences
Abhi: I noticed that there's a lot of distinction between neural network potentials for small molecules, neural network potentials for proteins, and neural network potentials for materials.
Why is there such a strong distinction between each of these fields? Is it just kind of the limitation of scaling up and like different fields, different areas need to make different inductive biases or is there, it's like some more cultural difference.
Ari: I think it's, a product of mostly the age of the field is like neural networks, deep neural networks are like relatively young.
And a lot of the focus originally was on language than images. Now, we're starting to see people working on these geometric libraries for graph neural networks. And so I think, what we're seeing is like the problem that people chose to focus on first. And there haven't been very many iteration cycles in these models.
People have gotten out at most, maybe five generations of models. maybe a few more, but there, if you pick up a problem space, you say I'm a materials researcher, I'm going to build a model for materials. You try to build one model for materials and then get it out in the world, you start testing it on things and you think about how do I make this better?
And probably you're going to, not expand your scope super aggressively until you figure out, the area that you really care about. And so I think, like an early computational chemistry and in the neural network potential space, we're seeing some people care about scaling quickly and they're trying to figure out how do I make these work on proteins?
There are people who care about maybe materials discovery and property prediction. and then there are some people who care about, really accurately reproducing DFT results in trying to replace quantum chemistry. And I think like our hope at Rowan is to, start with replacing quantum chemistry, like DFT methods where we can.
And once we've done that, and we're satisfied with it, we'll start working on these scale challenges. And so I think you'll see that, a lot of people do start to work on these, foundation models for atomistic simulation. I just think it's we're too early.
Corin: It's sort like the advent of television, however, in maps and new medium onto whatever they understand best in the old medium. People used to just basically take plays with no modification and then just film them. And then it took, it took a while for people to realize you could do like dramatic zooms, like you didn't need to emote as much, you could, add special effects, stuff like this. Yeah, I guess it's like a snapshot of whatever problem you thought was most relevant at the time the new technology is dropping. And then iterate from there.
[00:21:13] Gap between simulation and real-life
Abhi: Slightly returning to the idea of training these neural network potentials in the first place, the whole concept is you take, usually density functional theory trajectories, take the forces and energies from those and train a model to simulate those instead of relying on the physical equations themselves.
What I find interesting is that for most of the molecular dynamics field, ground truth datasets are basically impossible to actually gather at all. You're relying entirely on pure in silico measurements. What do people in the field think about this gap between simulation and real life behavior?
Is there a gap?
Corin: Yeah, there's always, there's a huge gap right now because the simulations don't work. So like even the best periodic DFT simulations of water, maybe not the best, but that once people often use like PBE water, probably the most common periodic functional is like a solid at like room temperature, like you, if you see a PBE simulation of water, it's usually heated to 80 degrees or so, just cause that way it's liquid.
Which is, it's just one of these things that you, I think once you're in the field for a while, you take that for granted and then you stare, you step back and you're like, hey, that's bothersome, isn't it? The foundational solvent for all life is not really modeled very well here, is it?
And, I think there's...
Abhi: So we can't model the boiling point of water?
Corin: You can show that if you layer on enough approximations, there have been a few papers about this in the last couple years, that you like, get better, but boiling point is actually pretty hard, like it's, I don't know, there's a lot of molecules involved, there's like solid gas, or liquid gas interfaces, like it's, a highly emergent property that results from like very, small inaccuracies in energy, and I, yeah, I think the intuition that a lot of people in simulation have, or at least that we have, I don't, need to speak for a field, is that like we can see microscopically the ways in which we're wrong.
So we can run our ultra high level quantum calculations, we can run the stuff we usually use for production, and we can see this is where we're wrong and by how much. We can compare, and then we can see macroscopically that our predictions are like inaccurate. We're pretty sure with neural network potentials we can fix the microscopic predictions.
The experiment that we're running, as with everyone else, is does that fix the macroscopic predictions? And it's very logical. The answer should be yes, but it's not. It by no means guaranteed.
Abhi: Are there like hypotheses as to why that may not be the case?
Corin: This somewhat ties to like the
There's so many ways you can imagine this not being true. So one thing that density functional theory and most quantum chemistry ignores is nuclear quantum effects.
So like hydrogen tunneling and such. Now I think most people's intuition is that this isn't, outside of like certain enzymatic processes that are like pretty circumscribed, this isn't a super big thing. You know it probably affects the kinetics of proton transfer in water a bit, but like we already know that like you can replace most of the hydrogens in your blood with deuterium or like a surprising amount and it doesn't really affect you that much.
So like it seems like biological models shouldn't be incredibly sensitive to the exact kinetics of like H atom transfer, but like maybe this is wrong, maybe these there's on the scale of a whole protein solvated in water, like even being a little bit off on these things matters a lot.
Yeah, I don't know.
Abhi: On the topic of boiling water, what do you think is like off there lately? We can't measure a system large enough or is there some like minor, do we need to go even deeper than density functional theory to actually model boiling?
Corin: So there's actually two ways we do DFT and this is like a not super well appreciated, I think. So there's molecular systems and there's periodic systems. So you're trying to describe the electron density and the electronic structure of a system. And for isolated molecules, that looks like putting basis functions, like describing the density in an atom centered way.
For periodic systems where your system is actually infinite, like a slab of metal or like a box of water molecules, that ends up not working super well. And so often people do these like plain wave, they use like a Fourier basis to describe electron density. And there's a lot of downstream things that you do.
So there's so many approximations that you do in density functional theory, and those approximations end up shaking down differently between molecular and periodic DFT. I think one of the consequences is that a lot of the most accurate methods from molecular DFT that we can super rigorously verify against ultra high level calculations don't exist in periodic DFT.
And a lot of the high level calculations don't exist in periodic DFT either. So the functionals, you can't do like electronic exchange, like quantum exchange, for instance, very well. and that ends up like being pretty important for a lot of things. And I don't know, we know when we run these functionals on molecular systems, we can benchmark, you're like, yeah, the water sticks together like 20 percent too much.
And so when you do periodic systems and you maybe it just imagine it's about the same, like you stick together 20 or 40 percent too much that throws the bulk boiling point off by a lot.
Abhi: If we like went even like more accurate, even more slower, like coupled cluster (theory) like do we, are we then able to model boiling?
Or even then, like there's like potential issues that start showing?
Corin: I think so. I think it is, So people have been able to dial in the accuracy. I can probably find the reference for this and you do approach the correct boiling point with a technique. So it doesn't seem like there's something like fundamentally massive that we don't understand here.
I think it just shows that, density functional theory in the sort of like life sciences is viewed as like an ab initio high accuracy method. But within the like world of high accuracy simulation, density functional theory is actually like the plebeian, like the dirty stuff for losers. those people are all working on these hyper orbital optimized like wave function methods that work on 12 atoms and you know from the theory point of view they're absolutely right. Like I think it just goes to show that solving the electronic structure problem is like Really hard, you know it the exact solution is O of n factorial which is terrible Yeah, like it's like the three body problem, but worse is all this quantum through space stuff there's hundreds of electrons and like it's it's, just hard.
Abhi: Are there, like papers showing that lately right now, everyone that uses DFT for, training data, but like potentially higher quality data is what you want to care about rather than the scale of data. Do you imagine like in 5, 10 years, people are going to go beyond DFT to something even more high accuracy or DFT genuinely is like sufficient for a lot of things.
Ari: I don't know. So one interesting model to talk about here is like the ANI model that was fine-tuned on a coupled cluster data set. It, performs surprisingly well on benchmarks against coupled clustered data like today. so I think like maybe we'll see people try to replicate that. but one thing that I would bet on first is that people are going to, ditch periodic DFT for generating training data.
Because the best methods or like the best DFT methods are only implemented for molecular systems. And so I think one challenge is figuring out, can I train a model, a neural network potential that will, work on periodic systems, but it's only been trained on molecular systems.
And I think that's a big question, but if you can get it to work, then in theory, you know, you'd be able to model periodic systems with higher accuracy than any DFT functional that's implemented for periodic systems would let you.
Abhi: Do you think you'll see this pattern of people like, like starting out really coarse scaled, and then bootstrapping their way up to have like smaller, like lower n data, but higher quality data. Do you think that's the future?
Corin: I think it is right. Like we're trying to solve such hard problems here. Like simulation is just really hard to do well.
I think every source of information you can get is valuable. so probably pre-training at a, ton of stuff at a lower level of theory to initialize your weights and biases is probably a good move. People do denoising as an auxiliary task. So FAIR-Chem has done this, that seems to be good.
Mixing in different like levels, like multi fidelity learning seems to be good where you can do it. even adding experimental data. So like crystal structures, we know the forces are zero, like that seems to be good. Yeah, I think it's, I, don't exactly know how much each of these things will contribute, but like any mix of like more tools to throw the problem, more sources of truth, I think is, super, super valuable.
And people even do you know, you can do backprop through a whole simulation. So if you have an experimental tautomer ratio, for instance, you can back prop through a simulation and like train to get the right answer, like over all of the molecular dynamics steps.
Abhi: Given a final end state.
Corin: Given a final end state. Yeah. Or like a different energy. The problem is that you have this sort of dimensionality problem because you have one thing and there's like one experimental result in so many states,. That seems like probably not sufficient to do it like a whole model from scratch.
Abhi: It feels like a very RL problem where you have this, like you have a final end reward and nothing else.
Corin: Yeah. And like the number of steps is more than chess. So it's tough.
Abhi: Yeah. That makes sense.
I think like like my initial suspect, like suspicion when I first entered the field is that, like obviously people are maybe using these in-silico DFT measurements for a lot of things, but there has to be like some sort of physical measurement being pushed through the window also.
And I assumed it was going to be like NMR, nuclear magnetic resonance imaging, because it feels like that's the only way you can actually measure dynamic movement of molecules. But it seems I have never seen that actually used in a paper. Is there a reason why?
Corin: The NMR timescale is so long, like you can get clever and try to view fast processes, but you're looking at about a microsecond just because the spin states are so long lived.
So like you can see two different species if they're like, and you can do the like pulse things to look at the kinetics, but they have to be pretty long lived species. So if you think what a microsecond is. So that's 10 to the negative 6 seconds, right? And the usual time scale for a simulation step is 10 to the minus 15 seconds.
So you've got nine orders of magnitude still, like there's just a lot of room at the bottom. Yeah. I think so the faster spectroscopy method you can do or something like multi dimensional infrared spectroscopy, which gets you down to about, yeah, 10 to the minus 11 I'd say seconds. So that's like much closer.
It's like the timescale of bond vibration are much closer to that. and so that helps, but again, it's difficult. Like you can't do 2D IR and make like a map of what a protein or biomolecule looks like and how it's moving. You can probe very specific things like a complex lifetime, but it like the, I think the value of the data and the difficulty to acquire each measurement just makes it, tough.
Like really crystallography for all we complain about it actually works really well. Like robots can look at crystal trays. You can grow a lot of crystal structures. And a lot of these like fancier spectroscopy methods require like grad student years. And that is an expensive currency.
Abhi: So like with x-ray crystallography, I'm surprised it gives you a sense of dynamics at all.
Do you mean that it gives you a sense of dynamics or it gives you a sense of something related to dynamics?
Corin: It doesn't give you a sense of dynamics at all. Obviously it's a static structure. I think what it does do is a crystal is something that's usually in a ground state, right?
Or modulo thermal and packing effects.
Abhi: So that's like a zero energy thing.
Corin: Yeah. And so zero forces, zero forces. And that lets you, that's like some piece of experimental truth that should be useful. It'd be like this thing, whatever it is, like a local minimum. And I don't think you can extrapolate the whole relationship from that.
But if I. It's clearly telling you something and then something is experimental and should be very useful.
Abhi: Is it, is like that fact ever used in these neural network potential papers? Like it's like a, as a possible like end state.
Corin: I, can you think of any, I haven't seen it be used, but it seems like it should be ultimately.
Ari: Yeah. I haven't seen many papers that are trying to like fit to experimental measurements. It seems like, you ought to be able to add them as tasks. They're in the benchmarks. Good benchmarks are on experimental sorts of data. And especially with benchmarking, these periodic neural network potentials, a lot of the benchmarks are, experimental properties.
I think my hope for the field is that sort of, as we work on building these models, we benchmark and train if we can on starting with small system properties and working our way up to these like bulk and larger molecule properties.
Abhi: Do you think like using these sorts of like real life data sets are like higher hanging fruit and it's not really worth engaging in it until we like really speed up the in-silico measurements?
Corin: It feels there's a lot of obvious things that we could do right now. And it like, I think it depends on how crucial like the experimental data ends up being. Cause it might be like, we've taken a pretty pessimistic view, I think, of the state of the art so far here. But, Schrodinger, binding affinity prediction, docking.
Yeah, they don't work perfectly, but, they work, they clearly provide value. It's a huge and great company. They, everybody uses it. And you can say, that seems to work pretty well. Yeah, exact numbers are off, r squared is not quite there, boiling point of water, yadda, yadda, some of the proteins need constraints.
But it's not like it totally doesn't work. You can say, look, we can make all of the forces way more accurate with training to DFT. We can do high quality DFT, this seems like verifiably very good data. Maybe that is enough to, maybe it doesn't get you R squared of 1.000, but it doesn't take that, maybe that's just pragmatically ends up getting you 90, 95 percent of the way there, and it doesn't take that much experimental data to fix this. I think that's we haven't tried the obvious strategy enough to know that it fails. And so I think it's putting the cart before the horse to start like, going for the higher hanging fruit.
Abhi: Yeah, that makes sense. Investing millions of dollars into better crystallization, better electron detectors.
Corin: Computational data is, yeah, it takes money to run the computers, but you just click run, you get your AWS credits and then you're, set.
[00:36:18] Benchmarking in NNP's
Abhi: I'm curious, how are neural network potentials benchmarked in practice?
Do you, is you, talked a little bit about this, you have this like potential end state you, we can back prop through that. And is one of the goals for neural network potentials is to recapitulate that end state, or is there some hope that it can also follow along the trajectory and match the trajectory that's like from DFT exactly?
Ari: A lot of the benchmarking work that I've done and I've seen is on, less exciting things than that. It's does this recreate energy and forces from DFT. Does this, sometimes like one of the benchmarks that people are using now is this like SRME, like thermodynamic stability benchmarks?
If I run an MD video, not does it recreate trajectories, but is it stable is what the benchmark tries to measure.
Abhi: By stability, does that mean like the atoms stay in the same place or like forces don't explode? What's the measure of stability here? I don't actually know how it's implemented.
Corin: At a qualitative level, early neural network potentials often looked good on things that looked like the training set. But if you run an MD simulation, they get out of distribution and then start returning random numbers. And, physically, the simulation, explodes. it's like a grenade went off in the computer.
And so people, one of the benchmarks people have developed is okay, make sure it doesn't do that. so that's pretty crude.
There's no like benchmark number that they give you. It's just two pictures side by side. And you're like, you got some of the wells and peaks, And others of them very wrong. And I think like more benchmarks like that, can we recreate potential energy surfaces? Maybe not exact trajectories. though that would be again, really cool.
I think like MD at least, at like room temperature or higher, like it's very chaotic. And so I think what I would be more interested in is hey, can we accurately reproduce potential energy surfaces? And if we could, benchmark that well for systems. I think that would be like a really interesting and useful benchmark.
Abhi: I remember I used to be really into computer vision when I was in college and there was this meme of people like getting 0.01% better on CIFAR-10, a common benchmark used in computer vision. Is there some like analog to that with benchmarking in molecular dynamics where people fit themselves really well on toy problems, but like the gain doesn't actually matter all that much?
Corin: There've been, I think, different generations of benchmarks.
So QM9 was a big one. There was like nine atom molecules with a bunch of different properties that saw a ton of use back in the day. I think people have realized, or maybe just people got too good at it or like it didn't end up being incredibly useful. You see it from time to time. I think it's early days.
I don't think there's like a uniform set of benchmarks yet. There's ML benchmarks. Maybe you're actually a better person to talk about this, because you've spent more time here.
Ari: I think for molecular neural network potentials, there's still a lot of alpha in benchmarking. It's a hole that I'm trying to work on filling in my free time.
Yeah. I think for like periodic models that are working on materials, the Materials Project has tried to do a good job building data sets and also benchmarking. And so the like sort of MatBench discovery leaderboard, people like post about OrbV2 beat MatBench Discovery, and then a few days later, Open Materials 2024 from FAIR-Chem, topped OrbV2 just slightly.
And they're like, our model is at the top of MatBench discovery. I think like one thing that I haven't seen yet is does a model that tops MatBench Discovery turn into shareholder value somehow? And I think, not yet the jury's out. We'll see. But I think trying to figure out, what are the benchmarks you need that really tell you, is this thing going to be useful or important to find?
And I think at least MatBench Discovery is an attempt to do that.
Abhi: Do you, can you explain what MatBench is?
Ari: Yeah, it's a collection of, benchmarks for periodic systems. And, so there's this nice table on the website that shows like the models in the rows and the benchmarks in the these questions like, can you predict energies of systems correctly?
Can you reproduce forces? They recently added the first, MD related benchmark, which is this thermodynamic stability one. And then they have a way to calculate a total score for the model at some, weighted average of those other benchmarks and you can sort them and they have some compliance criteria that the models have to meet.
I don't know exactly what those compliance criteria are. yeah, but it's just that it's a collection of pretty standard benchmarks, but it's a way to at least have some way of knowing, when someone publishes a new paper, is this better or worse than the last paper that was published in this field.
[00:41:49] Is molecular dynamics actually useful?
Abhi: On the topic of these models actually producing shareholder value. you may, you've made this point in the past about how a lot of these molecular systems are studying microscopic properties in hopes that they translate to macroscopic properties. Are there, it's surprising to me that at least I haven't seen any neural network potential paper try to poke at this problem of am I recapitulating the macroscopic property?
Or are there?
Corin: I think the papers exist. there's nice work from, some folks out of Cambridge essentially showing that you can get like a hydration free energy really well. So like how strongly is this molecule solvated by water that you can learn that really well with a good neural network potential that came out like maybe a year ago.
I thought that pretty great. If you get the details right, the outcome is right too. And I was really excited to see that. I think people are working on it with binding affinity with these big, like free energy perturbation, like protein ligand interaction questions. The problem ends up being that things are still too slow.
So you have to do various approximate methods. It's like end state correction. It's not a hundred percent clear that like existing models are capable of describing protein ligand interactions super accurately yet. So there's a lot of asterisks and the results I think overall are unclear.
There was a recent paper from Exscienta that argued basically it's not better. Like it, it's about the same as just refitting the torsions in the small molecule force field. I think one of the things that's, challenging about this field is, the questions you're asking, macroscopic benchmarking, like, how do we check that we're better at the things we care about, are like, they're exactly the right questions, right?
It's, I think the, very logical thing to ask. We're rebuilding, a legacy. drug and material science tech stack not from the ground up, but we're having to port pieces over. It's like porting things into CUDA. Like you need everything to work. And we're still like over the past couple of years, like frantically as a community building the infrastructure to like, how do we actually run FEP?
Like, how do we get like these protein predictions, like the melting temperatures, the helicity, all this stuff. Like, how do we do this with our own neural network potentials? How do we scale it? How do we, there's just a lot of sort of practical work that I think will and is, very actively being done. But it's, there have been, like, two papers this year which show you can use NNPs for full proteins, and they're, like, some of the first two that actually have done it in a useful way.
So it's, just very, early, I think.
Abhi: I think one interesting point I've come across as I like study this area is this disagreement as to whether dynamics at all are useful and you instead just want to like sample the distribution of possible dynamic states.
Corin: So dynamics is like clearly useful when you need timing information, which is perhaps like an obvious point, but one that should be said, if I want to, like if I have some kinase and it has like an open and closed loop confirmation, if I want to study how long it takes me to get from open to close, like what sort of the kinetics of that are you like, you really need dynamics there because that is like a dynamical question.
It's time bound. I think oftentimes people use molecular dynamics, not because they care about the time evolution of a system, but much more because they just want an efficient way to sample different states. So you're trying to take some statistical mechanic, like average, you want to get ergodicity, like some sort of like unbiased sampling and like MD is just a super robust way to do that.
But in that case, you might imagine that there's like much, much more efficient ways to sample than MD because your time steps are obviously very, correlated with one another. So the information per frame is pretty low.
Abhi: I think especially a lot of like pure computational ML people are very pessimistic about molecular dynamics as a field. Like from their point of view, they can do conformational sampling with AlphaFold. AlphaFold also seemingly has a sense of, flexibility on par with molecular dynamics.
You can do like in docking with DiffDock. I imagine there are some areas for which MD has been like genuinely outstripped by the capabilities in ML, but I'm also reasonably positive there has to be some areas for which like molecular dynamics is still going to be important and will continue to be important for the future.
I would love to hear what your thoughts are on the situation.
Ari: I think one thing that's really important to just say is that like MD today relies on force fields, which are these polynomial approximations of quantum chemistry. And so what people think of as MD is very different than what the hope for like neural network potential MD is going to be.
And so MD today is useful for some tasks, right? Like I think the one that comes to mind is like free energy perturbation, which generates, binding affinities, it seems a lot better than docking to me, though, also more expensive and not without it's failure cases. but taking a step back, like the most intuitive way to run a computation or to run a simulation is to simulate what's actually happening, right?
And, it's very small and it happens very quickly, but like the proteins and the little small molecule drugs in our bodies when we take pills, they do actually move around. They play out videos that happen over time and they interact and I think modeling that faithfully is always going to be useful.
Abhi: It feels like it's I do get the instinctive vibe to try and model things like faithfully to what's actually going on. I am curious, do you think the future of neural network, but we don't currently appreciate the value of neural network potentials because the current state of MD is just like really bad and just like intractable to do anything useful with.
Do you imagine there'll be like new use cases that spawn as a result of having NNP's are actually like fast, reliable, and able to be scaled up.
Corin: Yeah, I do. I think so there's like use cases in like model like. So here's an easy one is like covalent reactivity, right? So like modeling covalent docking and like the reactivity of covalent enzymes. Usually like force fields can't model reactions, quantum chemistry can't model large systems. Covalent inhibitors are reactive things that react with large systems, so that's like pretty intractable with state of the art methods. There's various sort of ways you can get around that, but they don't work super well. That's something where like you do, that's that's useful, like to actually model that, like covalent inhibitors are awesome.
They're widely used these days. you can look at the KRas work, And that's, to actually be able to model the covalent inhibition seems very important. And I think too, like echoing what Ari said earlier, there's some intuition that like if you want your like DiffDock or whatever to be accurate to actually get like super useful accuracy like you have to be learning chemistry implicitly somehow, cause we know that binding one molecule that has a hydroxyl versus one that has like a more strongly hydrogen bond donating group or like a less, hydrogen bond acceptor or like a different high stacking preference.
Like those all matter, like those demonstrably matter. And so maybe you can learn all of this in some implicit roundabout way with like tokenized language models. Like you put your ESM2 in, you put your tokenizer, like the recent QuickBind paper, you learn some sort of like interaction matrix there.
But it seems like you're just reinventing the things you want a neural network potential to do in a very like just a strange way a little bit like you to get the whatever accuracy you want like at some point like these are the sorts of modifications and like structure activity relationships that people really care about and like at least in small molecule space it seems like you need to know the chemistry and maybe the most parsimonious seeming way to do that is to teach a model chemistry and then model the process.
And, it might turn out that there's some DNA encoded library way to backdoor all of this, but that, it seems less likely to me.
Abhi: Yeah. I think that's actually a really interesting way of framing the whole problem that like neither side is implicitly discovering secret knowledge about the system.
The system is the system and you might as well like fit, like simulate what's actually going on inside of it rather than focusing only on static structures and hoping to learn accidentally what's going on.
Corin: Yeah. And I think this is problem specific too. So there's some cases where there is some secret knowledge you need to know.
It's like, you've written it before about toxicity prediction. That is a case where you're not simulating the liver, Rowan in 10 years, you won't have an atom per atom, like map of the liver. And then you like draw your molecule in and like just play and see what happens.
Like there are cases where you're learning some sort of groping around some large elephant of something and trying to like divine useful patterns there. And I think that's that's just a very different beast, but for the specific and like extremely important problem of binding two known things together in which we know that matters a lot.
We know we can't do it, and that's like a direct simulation problem in some sense.
Abhi: On this topic of like physically modeling the liver, do you not think like we'll ever get to the point with like incredibly hyper course grained models that like understand the many, many body problem that's going on.
And we can simulate like entire cells, entire organs. Is that at all like a genuine possibility in the next 10 years? Or it's just like in the realm of sci fi for the large part?
Corin: 10 years feels super ambitious there. I think chemistry is one like way to see the world. Med chemists like think in terms of atoms, like you can see the med chemists in a talk by the people who look bored for all of the slides with like histology on them. And then when they see any structure of a molecule, they perk up. That's a med chem phenotype. I don't know that it's the right way to tackle all problems. I think there's enough problems that it's like incredibly interesting, but maybe a sort of like phenotypic approach to some of these like high order problems is just more it's just better.
I think that's a, you, you've read about the Recursion play as well, this, almost coarse graining over cells is like a crude way to put that. And I think that's, even with like antibodies, like I think I'm certain there's like useful stuff we could do if we could do an atom per atom, like model of an antibody.
Like I'm sure we'd learn surprising things, but it might also be the case that we have so much inductive bias, we have so much like evolutionary information around antibodies that like it's not super crucial, to do like a rigorous atomistic simulation in the same way that like the, the models we have are going to be much more effective.
I think small molecules is such a such an unconstrained design space. Like you're literally positioning every atom that like needing to be closer to the metal of physics is just unavoidable. And I think the more atom per atom we go, the more true that will be, so in the like non canonical amino acid direction as well.
[00:53:14] Solvent effects
Abhi: Solvent effects is one of the things like a lot of pure ML models completely ignore. It pretends that, there's it doesn't exist at all.
It's not a factor to take into account. How important are like diverse solvent effects when you're dealing with these structures? Is it like the, universal solvent that's learned by like DiffDock, is that kind of sufficient for a lot of things? Or are there like a vast variety of solvents that actually exist within the body?
Corin: I think, to a first approximation, pH 7.4 water with some electrolyte background. a good proxy on the scale of proteins and ligands. I think obviously, with membranes, you, can, you, there's, going to be exceptions to that rule, on the scale of cells, it's not pH 7.4 water. We all know that, but on the scale of an individual protein as something right next to it, I think that's a pretty good model. I think where solvent effects become super, super important is in like the, reactivity, like crystallization process side of things. And then in material science as well, when you, just deal in much more diverse environments, That makes sense.
Abhi: Yeah. I think, imagine if you're going to like the thousands of Kelvin's, then like it, the, simulation actually becomes important. It's really hard to train a model for that.
Corin: Yeah. Or, like these, I know a super big problem is like predicting solubility under various conditions like for crystallizing out an active drug product.
And there you can easily have a blend of three different solvents, right? Or like even inside a battery you get the same thing. So you have some like ethanol, you have some carbonate, you have some water. and then this is like it, modeling solvent becomes like a factorial problem of complexity. But, the body, yeah, sure there's, cancer cells are slightly more acidic, but it's pretty similar.
Abhi: I didn't actually know that. That's interesting.
Corin: Yeah. There's some interesting work where you can design like acid released, like essentially payloads, that will, I think in theory is somewhat selectively be activated in the presence of cancer cells. I think it works less well than ADCs to do the same thing.
Like an antibody is more selective than a pH sensitive group, but it's a cool idea.
[00:55:17] Quantum effects in large biomolecules
Abhi: I'm, curious, do you think there's like interesting quantum effects going on inside of antibodies, like, large biomolecules in general that most people are just ignoring because it's too hard to study?
Ari: This is like really unexplored territory. I don't know how we would discover that those effects are happening, without new tools.
Abhi: Like, that Microsoft, A, AI2BMD or BMD2AI paper
The, context, it's a ab initio neural network potential paper that's specifically trained on fragmented proteins is able to scale up extraordinarily well, able to recapitulate the true dynamics of what's going on in proteins. As far as I can tell, correct me if I'm wrong, it's like the first way to actually study really large biomolecules using quantum, like quantum level accuracy. Do you think that paper is going to unlock a lot of interesting things?
Corin: I hope so. I really hope so. I think it's There's, there's like a divide here between there's known unknowns and unknown unknowns. We know we don't know how to do protein ligand binding affinity super well in a lot of cases. When we go to the realm of antibody dynamics, like maybe there's experimental data here that I don't know much about.
I'm not an antibody expert at all, but I think we just really don't know what we're gonna find. And that's I think, terrifically exciting as a basic research question. As like a startup person, I don't really see that as like a market that we're thinking about as like we're gonna solve antibody problems, but like I think there's just a humility you have to have no, we don't know and it like it could be anything.
[00:57:03] The legacy of DESRES and Anton
Abhi: This kind of leads well into my question on like the legacy of molecular dynamics. One of the, one of the big pushes in the field was D.E. Shaw's Research Anton, which is a, a supercomputer with thousands of custom built ASIC's, built by hardware engineers, paid extraordinary amounts of money.
The supercomputer led to papers with incredible titles, like 20 Microseconds of Molecular Dynamics Simulation Before Lunch. And for people who are not in the field, 20 microseconds is an immense amount of time for a dynamic simulation to be run. Yet, the company hasn't ever released a drug. It's largely been papers.
They have a therapeutic arm right now, no drugs has come out of this. What do you think is the legacy of DESRES (D.E. Shaw Research) and Anton?
Corin: I think, so it is worth maybe mentioning the partnership with Relay here that they have. Relay Therapeutics, a company in Cambridge, a lot of respect for them.
Pat Walters is there. Like a lot of great folks are at Relay. One of the original ideas is using DESRES and Anton to discover like allosteric sites for known targets that had proved like resistant, to previous treatment. It's very tough to know, I think, how much Anton actually helps.
To the extent that Relay gets drugs successfully approved on the market, what percent of the credit does Anton and MD and DESRES get? I think it's yeah, I think, there's probably a handful of people in the world who know the answer to that question, none of which are in this recording studio.
Yeah, I do, I do think, part of the issue, if you step back and look at the whole field, like, where would we expect molecular dynamics to be useful? We want simulations because ultimately they should be faster than experiments. Like we should be able to iterate quickly in the computer.
That's how other fields use simulation. If you look at aerodynamics, like you simulate a bunch of wings and flaps and then you don't have to make as many in the machine shop. that's, clearly useful. Like you have this R&D spend, you have this search problem, like a design and simulation problem, and you can quickly like funnel down the list of things you need to actually try in real life because the simulation is like of a sufficient fidelity.
I just don't like we're just not there like I think people don't really think about this because we take it for granted that you have to try everything in the lab if you want reliable data but that's where we want MD to be useful. Like that's where sort of the MD shaped hole is I think it's in like this, hit finding, hit to lead optimization part, at least in small molecule land of that drug design workflow.
And so we, we talk about prioritization. We talk about gaining insight. We talk about like some screening, but like the, at the end of the day, we're a Boston based company. If you wander around Kendall Square, buildings and buildings of people just manually doing the search that like abstractly you'd wish MD were able to do.
And I think that's, whether or not the impact of MD is like 0% or 2%, like it's not where it seems like abstractly like it should be. Like it's not doing the things that we'd want it to do and so trying to get there, like I think if MD were already like some fantastic workhorse in simulation, that'd be great for the field, but there wouldn't be a need for us for what we're doing like that.
Our company wouldn't exist.
Abhi: Like what's weird is that there clearly are successes of MD. Like Elliot Hershberg and Bruce Booth have written about Nimbus Therapuetics' partnership with Schrodinger. They supplied basically what became blockbuster drugs and they got zero money out of it, but like they produced the drugs. Do you think that was like a fluke that they were able to do those three drugs so well, but none of their therapeutic arms have like really worked out beyond that?
Corin: I think it's too early to tell what the Schrodinger therapeutic arms right because those are pretty recent.
Abhi: That's true.
Corin: So like the MALT1 one, there's the covid. There's something else I think jury's still out on those ones just because it's only a couple years old. I think you have to believe either like one of two things looking at. So I think that the Nimbus, the deal that I have in my head is the TYK2 inhibitor that they, sold to Takeda, right?
And I think the price was 6 billion. And do you remember how much?
Ari: I think Schrodinger got under 1 billion of that. I think, yeah, I think it was under 400.
Abhi: I thought they got like nothing. (I am wrong about this)
Corin: I think it was a one or 200 million.
Abhi: Okay.
Corin: And so there's there's two conclusions here. One is the value creation is low or the value capture is low.
And I guess, I think you, my, my hypothesis or like my gut, I don't know if it's even a hypothesis, it's just like that the value creation is lower than it seems that you're not bearing that much of the risk with the tools, like you're not, you don't just hand someone a drug on a platter, like you work with their experimental teams, but you still need, like Nimbus is a real company, like they have really smart people there who are like laboring in the trenches to build the drug and like some combination of the premium you carry for the risk and all the experimental work you still have to do even with the simulation makes it that, I imagine that split reflects value in some fair way.
[01:02:27] Unique value add of simulation data
Abhi: Yeah.
I guess also when I think of simulation, I also think of the potential to understand parts of your system that would be intractable to understand in real life. Is there such a thing like that in chemistry or like protein design, molecular design where you need simulation to understand something that is genuinely impossible to understand without it?
Ari: I think the easy example here is something like reaction mechanisms. They just happen way too fast to study with an electron microscope, you can't point an electron microscope at a reaction, but it's concerted. And yeah, I think that's where these tools get a lot of use right now and where they're really valued.
I think for, bigger problems, that there are these like predictive accuracy thresholds they have to pass to be really valuable. And like maybe MD was accurate enough for a few proteins, but not accurate enough for, any protein off the shelf. What else do you think?
Corin: Yeah. I think just the role of insight in general is it's very tough to quantify, like it's tough to put a dollar sign on like insight, how much value does insight provide your organization, but, fundamentally, atoms are really small, chemistry happens really fast, and it's pretty easy to live, weeks, months, years, as someone who works in like the world of atoms without really getting any like tangible like window into what's happening.
And I think this is why people like docking so much. You can read all these papers arguing about how docking is not, it's not useful, docking is information theory like negligible. As someone like told to me like, from an information theory perspective, docking is useless. But at the end of the day, like if you talk to med chemists and you ask what do you like? They're like I love being able to see how my compound might or might not fit into the pocket like to just get a sense of how big it is and how it might fit in 3D is really useful for me. Even if the numbers don't mean anything, I just derive a lot of satisfaction, and it helps me brainstorm just to see how it might fit.
It helps me generate ideas. And I think, that's, I think that's just, useful for people. And yeah, like, reaction modeling, covalent inhibitors, like dynamics, like watching how things move, like protein pockets. I think there's, this is it's not directly impacting the bottom line, but just building, really valuable, tools that help scientists think and build intuition better is underrated.
Abhi: I think, where my mind immediately leaps to, I know you guys interviewed a lot of scientists while like you were building up Rowan. How much of this like med chemists, intuition that oh, these tools are like really helping me understand what's going on. How much of that is like not real because they, like seeing that something might happen, even if it doesn't actually match up with what's actually happening.
How much of that, is just, I don't want to say the word cope, but somewhat cope?
Corin: Yeah, it might be cope. I think there is some sense, if your tool generates random pictures, I think it, people will suss that out. I think there is, med chemists actually are pretty quick to learn to distrust computational tools.
It doesn't take much. Many people you talk to have a well adapted immune system for not believing things on computers, which is I think probably rational from their point of view.
I do think like being conceptually useful and being correct are not always the same and if you're sufficiently correlated with being correct, people will perceive it as useful even if it's not. it's a good model for something, but that doesn't mean it's you're on the right path of searching for capital T truth in this case.
Yeah, I don't really, and people are just susceptible to pretty pictures too, so if you like, give them something nice for their slide deck, they'll probably like it, because it makes them feel like they're acting more, like they're more rational in their job, as opposed to just screening things randomly, which is, a very effective strategy and one that's widely employed.
[01:06:34] NNP's in material science
Abhi: I have very little background in the material science applications of this. I would love to like on the topic of catalyst design and areas within materials, material science, I would love to hear what, is the use case of NNPs there?
Corin: Yeah. In simulation people talk about material science as a monolith, but it's eight tiny fields hiding inside a trench coat, I think. So there's some things which are like, you can essentially use the same models you would for drug design. Like you're modeling organic drug like molecules, but in different contexts.
So some of these like redox flow batteries, battery electrolytes, maybe you have different salts and more phosphates floating around, fewer amino acids, but like it, it's very similar problems on some level, like it's solution modeling, it's molecular dynamics, polymer properties, like these thermostats, like you're, modeling distribution of systems, other things like solar, upconversion, like these, processes end up like being very, different.
I think material science writ large... Like for drug design, there's like a playbook. Maybe there's a few playbooks. There's an antibody playbook. There's a small molecule playbook. Maybe you guys are writing an AAV playbook. I don't think playbooks exist in the same way in material science, like it's much more like everyone has slightly different research problems and solves them in a slightly different way.
And so I think trying to port a bunch of like very specialized workflow tools hasn't worked quite so well in material science. like some people are designing OLEDs, some people are designing like new inks. Some people are designing like electrooptic materials. And you need to be versatile.
You need to be able to be generally useful. And I think it takes at a very base level though, if you have a model that understands chemistry, all of these are on some level chemical problems. And so you can be useful. It just, the solutions need to be a lot more adaptable.
Abhi: I think when I like the few stories I've read about, research in material science and specifically like the creation of blue LEDs and like ongoing work and, semiconductors, it feels very much like a field where it's like a, you try a billion things until something works is that, and it feels like chemistry, like drug design is a little bit more rational and how things are designed.
Is that like a fair distinction?
Corin: I think in both fields, there's moments of rationality and there's moments of sometimes the most rational thing to do is just screen a million things. Like I think that's part of where the rationality comes in is like figuring out when and how to screen a million things, because you can imagine blue LEDs, there's some like band gap you're tuning.
Like you can imagine how you change the molecule that will change it. But there's also there's packing effects, there's stability and you, want to get in close enough and then you're like, all right, now we screen. And I think it's the same with drug design. You don't, maybe there it's we often start with a high throughput screen. We start with a DNA encoded library, some fragment soaking something. And we, bake in the randomness up front. And then once we have a hit, the intuition is that we can use our medicinal chemists, computational tools, all of this sort of intuition and skill to rationally get somewhere.
But I think like one of the best and worst things about chemistry as a field is it sits between being able to be fully understood and being unintelligible. Like that there's at all levels, this mix of you need to understand things, but you can't understand everything.
And that's part of what I like about it.
Abhi: if you, go back to the Japanese salaryman, who created the blue LED in the first place and you gave him like material science neural network potentials, like would there be any real benefit? Could he like, can you do anything interesting with it?
Or even that's is a little bit challenging.
Corin: Can you, I don't actually know what the, I've read a tiny bit about like band gap effects and LEDs. Our coworker Jonathan put together a blog post on that, but I don't know, I don't actually know what the like challenge he was solving was.
Abhi: The challenge was almost like, in like deposition of one material over another, you need to get it like perfectly exactly right.
Otherwise certain things wouldn't work. There was also this there's the band gap problem. He needed to have, I forget the details, but he needed to have this weird structure to ensure that like the electrons actually flowed correctly. And it was just, he, I think the video I watched over, it says something like he worked like 15 hour days just working with the, like the deposition machine, just like trying a bunch of different things.
And because he built it himself, he could try different things. Yeah. it was like, oh, and that feels like a very macroscopic thing. He's like doing it and measuring what happens after.
Corin: Yeah.
Abhi: Could a neural network potential do anything in that vein?
Corin: To say no to an open ended question like that feels like rude, but that doesn't seem like something I would choose a neural network potential I don't want to be one of these tech people who wanders into science and is, proclaims that simulation will solve everything. If you look at fields like chip design, what's the breakdown of total R&D like expenditure versus like simulation expenditure. I think it's somewhere in the neighborhood of 5%, like you're 19:1 in favor of actually doing things in real life. And I think that's reality is complicated. Like we should have humility as people working in simulation that like, we're not going to get everything and where you need to touch grass.
Like you need to actually do experiments and find stuff. And if your simulation is not useful, then stop wasting money and just go do the experiments. You know I think there is that being said right, simulation is a lot cheaper. So whenever you can do simulation it's much nicer too, but there's a lot of problems that I think will remain experimental for some time
Abhi: That makes sense
Ari: With the blue LEDs.
I might have watched the same video.
Abhi: Was it the Veritasium one?
Ari: Yeah. I think that, there may be a way to use some of these materials models to, predict the, stability or relative energies of these different, crystal structures and figure out, at least what ratio of these atoms do you need, But I have no idea how that translates into if you get an answer from the computer, oh, like this crystal structure looks like it might be viable.
How do you like use the deposition machine to make that specific crystal structure? I think it's, still going to be the same long arduous process. And so it's really hard to say with any confidence that, oh yeah, there's real alpha there. But I, think that there's, a hope. Yeah.
Abhi: I imagine if like the Holy Grail is achieved.
You can scale up as much as you want, stuff like that does become possible, but it's also just so far away that it's hard to imagine.
Corin: And yeah, I think I'm happy thinking like one or two steps away from where we are today. I think, you get into like pop sci territory when you start trying to think like four steps away from where we are today.
Like in the future neural network potentials will like, design you a custom drug and then fold your sheets for you and stuff. like I don't, know, like it's, our company will have either succeeded or failed long before that comes to fruition.
[01:13:57] The road to building NNP's
Abhi: When I talked about Rowan at the very beginning, I described you guys as a quantum chemistry simulation startup, which is you guys are building a front end for actually doing quantum chemistry, but you guys have also pivoted to working on your own neural network potentials entirely. Is there a reason you guys went through that pivot?
Ari: Well, in the early days of Rowan, we were. exactly this. We wanted to build a web platform to help people run their computational chemistry workflows. We had no intention of training neural network potentials. We didn't even know what they were. at least I didn't know what they were. And, at some point last fall, the Isayev group at Carnegie Mellon released their like AIMNet2 model, which is like a successor to the ANI models.
And we put it on Rowan and it got a lot of use, from our users in industry and academia. And we were honestly really impressed by its performance. And I think, to this day it remains one of the like leading neural network potentials. And as we were thinking, what would the most useful version of this look like for our users?
We realized that there wasn't anyone who is building it. And I think that's, the moment where you sort of look around and you say, who's going to solve this problem? And you're the only one there standing ready to solve the problem. And we added a third co-founder to Rowan to lead the effort.
Abhi: Has there been like new challenges associated with trying to like, like previously, I imagine that Rowan, the main focus was not scientific. It was very UI UX based. And now you guys are moving into a more like pure science direction.
Corin: Actually, I think that's a fair assumption based on what we've shipped so far, but we, from the start, we're trying to do applied R& D.
I think to give a little motivation on what the sort of backstory of Rowan was. So I did an experimental chemistry PhD. I happened to idiosyncratically have some simulation experience and software development experience. So I was able to do like a lot of simulation in support of my own research.
Which was like, it was very powerful. It felt like the future of chemistry. It felt great. But it wasn't really scalable to anyone else in my research group or department. Like at the things I was doing, I couldn't really help other people to do. Because the solution of go learn to program for three years and then come back is not a practical one for most graduate students.
And so the core mission of Rowan was like, we should build like the tools that scientists should have. The future of chemistry of molecular design of working with molecules and materials should be like simulation seems like it should be a part of that. Like if you watch science fiction movies they're simulating things in computers like we simulate things in other fields. This should be a bigger part of the day to day workflow than it is and if you work back from like building tools not just for computational scientists but from every scientist, like it can't take a week to run.
And so we were doing quantum chemistry. It was too slow. We were like frantically trying all these ideas to make it faster. So we wrote our own code from scratch. We were trying out all these approximations. We were like fiddling with all these knobs and levers. All of like none of which are even worth speaking about because they all failed and it turns out like just trying to do the same thing, but find a way to make a two orders of magnitude faster in a hundred year old scientific field is like really hard.
There's a lot of 20 percent improvements. And so then when we saw the like neural network potentials and we put them on our site and saw that was the first thing we'd done that people actually liked, it was like a light bulb moment. This is what we've been waiting for. We were just totally on the wrong tech tree.
Abhi: I think a lot of people right now are like looking at tools they worked, with during their PhD and now they're trying to build companies that like help solve some of the problems of those tools. But it seems like they're running into this issue where the incumbents don't really care that much and they're happy to use the existing tool sets and Rowan feels like it's a bet on a brand new like way of working with these tools entirely. Do you think the types of people who are trying to modernize old existing or existing tools will succeed?
Ari: You have an adoption curve that you, I don't know, people think about this in startups, you have like your, early adopters and your innovators who are willing to try new things, who are willing to learn, maybe a new way of thinking about the place of simulation and design tools in their workflows.
And I think those people, are willing to try tools like ours and tools like these other companies are building. Over time people retire and new PhDs graduate and start working in industry. And I think that is going to be the biggest driver of like tooling shift at these companies is as people, start using new tools maybe for their research, maybe their early career, and they don't already have, a preferred tool, they're willing to try things and they don't have this like preexisting mindset and bias. And so I think that a lot of it is going to be very slow.
Abhi: Corin, you've talked many times about how terrible the tools you worked with during your PhD were. I feel like when scientists talk about how terrible their tools are, it's vague.
It's never specific, oh, this, specific functionality was impossible to do. What was specifically hard about these tools?
Corin: Like if you want to run just a sample calculation. So I have a molecule, I want to optimize it and figure out what shape it's going to be.
It's like a very simple computational chemistry task. To actually do that requires like you, you draw the molecule out. Okay. That seems like a sane step of the workflow. You write an input file with it where you like memorize all these like little cryptic, like acronyms to describe how you want it to be done.
You have to put in a lot of nonstandard ones to get it to work like robustly and with like state of the art things. You, transfer all these files you've created to a remote server, somewhere. You then, invoke this massive Fortran executable. It, runs, maybe it runs out of memory, maybe it doesn't, maybe it leaks memory, maybe it doesn't.
It then returns you, a hundred megabyte text file. And then you, grep through it and try to figure out where your answer lives in the text files. You memorize the phrases that you search for, and then you like cut and paste it out and try to make sense of the results. And so you can get really good at this.
So skilled computational chemists have an army of like awk scripts that they like selectively deploy it like the right moments. But it's, there's a lot of incantations and it's like a very steppy process. So like just the whole thing you described there, like even if the calculation itself takes, 15 seconds.
It's, you're, in for a 10 to 30 minute process. And that's like once you know what you're doing by the time you get everything sorted out and analyzed.
If you're trying to mentor a younger graduate student, you're trying to bring them under your wing.
You're like, all right, let's like, let's before you step into the lab, let's see if this molecule is even going to be the right shape at all. And then you give them like a 30 point stack of like instructions for how to do this. The adoption is just very, there's a lot of friction to use this sort of startup term.
And then trying to think about a lot of clever things like, okay, what if we want to scan through like a bunch of docked poses and evaluate the strain of all the docked poses where you do that automatically in a high throughput way for every different ligand and then extract the results and make a plot.
Like it just, becomes very burdensome.
[01:21:13] Building the SolidWorks of molecular simulation
Abhi: And actually on this whole idea of like the role of simulation in research workflows, you have this really amazing blog posts that discuss how molecular simulation software should try to emulate SolidWorks, specifically that it is easy to use, doesn't attempt to replace actually building things in the real world and assists human intuition instead of replacing it.
In many ways, Rowan is trying to build the SolidWorks in molecular simulation. What do you view as the biggest blocker to actually doing that?
Corin: One is that like our model of reality is not incredibly accurate.
So sometimes SolidWorks, you model a machine and there's not a lot of intuition, but if you cut a block of metal, it will look like the block of metal you cut. It's like deterministic. Like you can say this will fit together like this. And there's this uncertainty and there's this fuzziness around how we model things and geometries and properties.
And this is what we're trying to address with neural network potentials, like a scaling up accurate simulation to a useful pace and robustness and what we've been talking about. I think the other pieces on the like user exposure and the human computer interaction piece is very hard because there's I think generations of experimental chemists who have learned several things about simulation.
One of them is this is a thing that's for experts. It's very complicated. And if you do it wrong, people get mad at you. So there's like a learned helplessness. Another one is like this is a thing that other papers or other people in my organization do. I can tell it doesn't work. I don't know why they have jobs.
But it's not for me and I don't trust anything that comes out of it. There's maybe another couple of archetypes. there's some people are this seems like it should be cool, but it's like out of my pay grade. Like I can't do that. Like I, I chose a different, like character path in my like scientific journey.
And like I've, those fields are forever closed to me. Like maybe I could have done simulation, but like maybe if I switch jobs, I'll try to pick something up. I, think part of the, exciting thing about Rowan and also a challenging thing is that we're trying to build tools that like many scientists can use, people who haven't traditionally done simulation.
And we can solve, there's like a product, like an engineering, like a robustness, like a workflow, like packaging this all, like making it simple and understandable and robust, so it gives you high quality results. But then there's also just like a really basic education piece, like, you, you may have never done simulation before.
It's going to be good at these things. It's not going to be good at these things. here it maybe is fine, but you should like really double check it carefully. Here's about how long it will take. there's we're trying to inculcate like a new behavior. That's like before you just run into the lab to make something, maybe spend ten minutes checking if it's obviously going to be a stupid idea before you commit three weeks to it. And that's like a, it's asking for a change in behavior in a way that's I think harder than I expected because I'm an early adopter. Like I was always excited about simulation and we have people who use Rowan who are like that, but part of growth will look like us being able to woo people who are on the fence and were more naturally skeptical.
Abhi: And I imagine like your first user base are going to be PhDs in chemistry.
Corin: That's right.
Abhi: Who do you imagine like the second wave will be like pure machine learning people, will it be structural biologists? I imagine structural biologists are also like probably part of the first group Yeah, who's who is the second group?
Ari: I think it's folks who are working at small companies Whether they're in the like adjacent areas of material science where they're working, you know on biotech problems I think it's you know, maybe your fresh graduates. It's your company that you can't afford one of these legacy licenses and they're looking for a solution in their company.
Corin: Yeah. Sorry, just to clarify, you said something about the first group as being like chemists, people with PhDs in chemistry? Yeah. I think you actually are underselling the magnitude of the gap here. there's, if you look at an org, like a top 20 pharma org, and you ask like how many people in this organization really use Schrodinger or a comparable tool, I think I have it from a couple of sources in different ways that the number is usually about 40 or 50.
Abhi: Across the entire org?
Corin: Across the entire org, which is…
Abhi: It's astonishing, right?
Corin: It is astonishing. Like it's a power user tool. And I think that's because it's not a tool for somebody who's a chemist. It's a tool for somebody who has a PhD in using it, essentially. That's the thing. And so even like the, I don't want to call it a fight, but one of the challenges we face is you have a PhD in just a different area of chemistry that isn't simulation.
Like you're clearly very smart. You're clearly a motivated person. Like how do we get you to find utility and to be able to benefit from what we've built? Because we know that. for people who can use simulation. We know simulation is valuable, but there's there's just what about the other a thousand chemistry PhDs there?
Like that is like just a blue ocean, I think for us and for practically everybody, of people that need great tooling.
Abhi: I like, I imagine like, speaking as someone who mainly works in like machine learning.
Corin: Yeah.
Abhi: I would be terrified of using any molecular simulation tool exactly for the reasons that you suggested that there's so much going on there that I don't truly genuinely understand.
Do you think there are like, like in my head, everything in the world of molecular simulation is an art and none of it can be distilled down to like my level, like my like common denominator level. But do you think there is like this world in which there are actually a lot of workflows that I'm just not aware of that could be commoditized and fit into my non chemist head?
Corin: There's like a scientific maturity you need to have as a workflow to be push button, like where you run a couple of checks, you're like, yeah, this will be pretty good and then you just blindly trust the output answer. We're trying to get as much stuff there as we can.
A lot of our workflows at Rowan currently work like that. And we've avoided adding ones that don't work like that just because it feels cruel. But I, I think there is a challenge of for you to use a chemistry tool you need to understand chemistry and we're okay with that I think for now being an entry requirement, like Rowan can't, and probably for a long time, won't be able to teach people chemistry from scratch.
Maybe we're trying to work with classes, so maybe we can be involved in the education process. But if you don't understand anything about chemistry or like what a PKA is or what a reaction is like, this is going to be tough. Like we're, talking different languages. I think we feel like the people we, Oh, it too.
the people who we really should work for is the people who understand chemistry, but know nothing about simulation. that feels like that's the thing we need to be able to solve. If you're a great chemist, you've never touched anything about a computer, we should be able to meet you where you are and be like, here's how we can be useful.
Here's how we can save you time. You can trust us that we'll not lie to you. We won't talk down to you or try to sell you snake oil. Like here's what we can model and here's where this will be useful. And that, that I think is like something we're still. We're still iterating on, but I think we can succeed at.
Abhi: Yeah, it's instinctively a little bit surprising that, you need to know simulation, like you need to like have done your PhD with simulation software, even if you're a chemist, actually use it because it feels like it's not, it feels for example, almost like every biologist feels like they know how to code, like to some very minor degree.
Why is there such a, like a strong gap in simulation? Is it Like a cultural problem or the tools is just really that bad all of the above?
Corin: I think it's a mix of both. I think the cultural piece is I don't know, it's probably it's rated highly and I think it's properly rated. Like there, there's, I don't know, there's like fields have cultures.
I think organic chemistry in particular is my home field is I think the among the oldest of scientific fields and the modern practice of science, like the, foundation of the modern, research university comes from Friedrich Wohler, who was an organic chemist, in the 1830s. and there's, a rich tradition.
People have been doing this research that came out in the 1990s is considered like new in chemistry. Like it's such a vibe shift to come over to ML and be, Attention Is All You Need was 2017. And like functionals from density functionals from 2010, like still aren't in some software packages.
Cause they're too recent. there's, I don't know, like people. there's a it's like an art, like there's a craft to organic chemistry in some way and there's like this is what my advisor did who had his advisor before him and his advisor before him and then, you know, that guy was a Nazi and his advisor before him like it's, it's, it's suspicious of like tools based innovation and it's like very, very difficult to break into that I think.
I don't know and I think biology is just sort of more the wild west.
[01:30:05] Simulation workflows
Abhi: I'm curious, like what, what workflows do you imagine will forever remain in the realm of art?
Corin: So one, this is on the very, the very chemical side, but like reaction prediction and like figuring out how a reaction occurs, which is what mechanism will have.
So if you have some. reactant and you have some product and you're trying to understand how they transform, like actually understanding the mechanism by which they interconvert and like, how many molecules are involved? What is the geometry like? What's a reasonable possibility? What's not a reasonable possibility?
How do we do this search? Like it's a very open ended and creative problem. I did my PhD mainly in this, so I'm biased. I think it's a really fun sort of molecular Sherlock Holmes type thing. You have to mix in computation experiments and all the right ways. Like it's very hard. That to me is like an easy example of you can do automatic transition state finding things and they work if you have the right, if you know exactly what you're looking for, they can be effective, but the problem of like, how's this going to happen?
That is it's artistic and it's I don't think there's such a thing as a systematic solution there.
Abhi: You don't think like a search, like a search algorithm could figure it out?
Corin: Like a network search? Yeah, but there's so many it blows up really fast because you can involve multiple equivalents of things.
So you can have dimers, you can have higher order things, you can have solvent involvement, you can and maybe these are long tail things, like maybe I'm, being super nitpicky here. It's it's just one of these like network combination things that like feels like it's going to be really difficult to get right.
Yeah. And it's just, I think it's also something where you need to mix in inductive biases from experiment. Yeah. probably for a while.
Abhi: That makes sense.
Corin: And so that's not easy to black box. Like you need to, have that in. I think FEP is an example of something that's difficult depending on who you talk to impossible or very difficult to like black box today, like to make just like a push button workflow.
There's no, it really depends on who you talk to. I've got strong opinions on both sides here, but that feels like something that probably is solvable, like that you should be able to do that in a push button way. And it just. isn't quite there yet in most cases.
Abhi: My actually, I always assumed free energy differences, at least, or feel like they're pretty push button, right?
You just swap out molecules. Am I, missing some major nuance there?
Ari: So with FEP, to save on simulation costs, cause these are incredibly expensive simulations to run. You want them to happen overnight. There's all of this crazy. statistics looking work to me, where they're running, the protein once and they have molecule A and molecule B and FEP will like move on the slider of like how similar is like you, you model both at once and you model it at like different points along the spectrum between the model. So it's we're modeling 30 percent A 70 percent B here. And you will like often do this with a whole series of molecules and you have them ordered by how similar they are. you go from A to B and B to C and C to D. And so there's all of this like weird statistics work that goes into it.
And then often to the, proteins are too big to be modeled overnight. And so people just start chopping off random parts of the protein to get the system to run. And you chop off part of the protein, and then you start running it and make sure it's stable. And if you chopped off the wrong part, you have to try chopping off a different part.
And, you like the theory of oh, you, run a video of protein molecule solvent, compute free energy. It sounds really great, but to make it a tractable problem for the cost of compute today and the architectures, blah, blah, blah. There's all of this like work that's done by hand and there's a lot of guess and check that goes into it.
Abhi: Yeah. So it's like setting up the system is challenging, setting up the like state by which you transform one, the system to another state, when the system to another system is challenging and that's really challenging to automate.
Corin: Yeah. I think so. I think there's a lot of vibes based analysis, and there's just a paper out yesterday showing that like the input pose ends up mattering actually, like you hope it doesn't, but it does.
So you have to make sure that you get the pose right. You worry about proton transfer if you have potentially basic sites and acidic sites where you get proton transfer from the ligand to the protein. And it like, it's just like changing between charge states is hard. these are all things that you're like, yeah, you, could solve that.
One is you could just solve it if you could run it faster. If you could get to more time, we wouldn't care so much about the input pose. Otherwise if you run it for too long, sometimes the protein starts to fall apart and unfold or the ligand drifts out, so you have to make sure it doesn't do that.
I think these are all like, you should solve these, maybe neural network potentials will solve these, maybe other advances will, but it's just finicky.
Abhi: So like you've, discussed the things that will probably remain art for quite some time. What about the things that you think are lower hanging fruit than probably most chemists actually realize? And it's like fully within the realm of their capability.
If they were handed a good enough tool,
Corin: I think one of the most basic yet underrated sort of things to do is just to understand the confirmation, the shape of a molecule that drives so much of how it behaves. That's such a foundational thing. We take it for granted for everyday objects, but you can be working on a molecule and just not even know what shape it is.
Like rigorously, I don't know, for an entire project, I've definitely done that before. Yeah, I think this is. something that's like pretty easy to predict, pretty useful, like immediately intuitive. And especially as we get to these larger molecules, like macrocycles, these beyond rule of five things that can target PPIs, like interesting peptide systems, actually understanding the shape and the trends, how substitution affects like confirmation and properties are like, it's really hard.
Like it's not obvious at all. And it's like an outstanding challenge in the field, like understanding, like what shape is this even going to be? And if I switch this ancillary group here, will this totally remap like the overall, like confirmation of my macro cycle? And this is useful, like this, matters a lot and is like an easy computational problem more or less.
Abhi: This is going to betray how little I know about chemistry, but my initial impression of a lot of molecules is that they're incredibly flexible. So how much do you really gain from knowing like a few states?
Corin: Yeah. So some molecules truly are exceptionally flexible, like a massive fatty acid that is just like a snake of carbons.
That is intrinsically disordered small molecules, so to speak, and there I think it's not incredibly helpful because it's just gonna wiggle around no matter what. oftentimes there are, there's a lot of accessible states on the energy landscape, but if you're, say, 1 percent or .1 percent of the distribution, if you're a couple of kcals per mole up in energy, then if you bind in that confirmation, you're disfavoring the binding pose because you have to distort to get into the bound position. So it's like a really common way to gain some efficacy in a small molecule drug to make like to freeze something to lock it in the bound pose. so you can, using, this is one of the things people use quantum chemistry for now is you can create some map of all the different potential poses, like what the bound one is, where, how much higher in energy it is, and how much you're losing by not being in the ground state pose.
And then you just engineer a molecule that has the right pose. And that's like a, again, that's one of those things that like experts, people with PhDs do, but regular people don't do that they could and should be doing like it. It's and it's so intuitive, like it's so like directly maps onto things that are like intelligible.
Abhi: With small molecules, is there, a, like a strong suspicion that once you actually introduce it to the body, it's going to dramatically change?
Corin: I think there's so many fewer degrees of freedom that, I think it's much less likely, there's small molecules in some ways are just simpler. like you, you care about if it will be protonated or not, but you're not going to get like mass electrolyte effects or some sort of you're much less likely to get complexes.
Like you can always stick to albumin or you can binds to proteins. But I think it's the confirmations in water and in the body often are quite similar into a first approximation, pretty much the same.
Abhi: Over the topic of structure optimization, you've mentioned in a prior blog post that you could optimize the structure of azithromycin, a common antibiotic in five minutes using an open source neural network potential method.
Whereas it would take nine hours using a DFT based method, both of them and having similar accuracy at the very end. What's the ultimate payoff of this for a chemist? Is it like just much faster iteration time, less spent, less time spent in lead optimization. Something else I'm not thinking of.
Corin: I think this makes a ton of sense in the context of a, SolidWorks type vision, like structure optimization is like, just like ground zero for anything else you do.
If you're going to dock it, if you're going to try to figure out how it reacts or something, you need to start with the right structure in the first place. At the margin, like how fast something runs, I guess it affects your cloud spend a little bit, but it's not a huge deal. I think what matters is when you can hit this sort of like order of magnitude changes that engender a shift in behavior.
So a calculation that takes one month to run is of no use to anyone outside academia, a calculation that takes overnight to run like nine hours is different. That's a I'll check back on this tomorrow, and we can talk about this in group meeting next week. A calculation that like runs while you get a cup of coffee or use the bathroom is like a, oh, I can do this and understand it today.
And then the goal is like something that like just responds like intuitively, like a real time, like you draw your structure and you instantly get the right thing. I think the utility to the end user from like an insight and from a, design tools perspective, like increases exponentially as you like decrease the time it takes to run something.
And we do see this like with some of our users at Rowan, where they'll. They'll be able to just sit and like experiment, they do their design. Simulate, think, design, simulate, think, cycle, back and forth with the computer. When the, property they're optimizing over only takes, 30 seconds to run, I think people do this a decent amount with redox potential predictions.
That's more on the synthesis side of things, but trying to figure out how easy it will be to add or remove electrons to something. There, we like, we have a good solution that runs in virtually no time. And so you, if you want a molecule that has a specific redox, try it, try something else. Try something else.
Try something else. And this this starts to be like what you hope the future looks like in more and more areas. Like it'd be great if we could do that with, drug binding affinity as well, where you're like, oh, that didn't bind so well. Oh, what if we added something here?
Like that would be an amazing future for drug discovery, but it's, difficult to get there.
[01:41:06] The role of computational chemistry
Abhi: Yeah, that makes sense. Vaguely, you feel like this like culture shift of like, waiting for a month to calculate something so you just ignore it entirely versus like it takes a few hours to calculate something so you actually do seriously investigate it.
Do you think something like that happened with the rise of Schrodinger? and are there like lessons to be learned as to like how, drugs, were developed and like how they, that, that changed? I imagine it's like it's the answer to this question is known by like very few people but I'm curious whether you have a like an insider thoughts on that.
Corin: Don't know if I have like secret knowledge here. I think there, there was a shift, like you can look at the start of Vertex, you can look at the dawn of Schrodinger as like computers being useful in drug discovery for the first time that like, you can go back to the 50s and 60s and I don't, computers weren't useful for much in the simulation realm back then.
Then there started to be these like both internal tools and external tools sort of things becoming mass market, right? People built the computational teams or like new companies like Vertex built around computation. And then this, you start to have an expert who's like a scientist who like uses computation to support various things.
And I've heard the role of the computational chemist right now often described as like just a, like a helper. Like you support the med chemist. Sometimes it looks like do modeling. Sometimes it looks like doing data processing, like building little like ML models on the data. I had some guys say like, I'll do whatever is useful.
Sometimes that looks like getting coffee. Like it's a, it's very much a sort of role where you're trying a lot of things out. And I think we've, we have seen that this is now a part of like virtually every organization, like every top pharma company uses Schrodinger pretty much like almost every team involves a computational person, but then the like utility of that role has plateaued like a little bit, not maybe fully plateaued, but it's tailed off.
Like it's not like more and more people are becoming computational within these organizations.
Abhi: Does the average, like a computational chemist, think they are useful at like the average pharmaceutical company? Or are they more like, I hope someday I could be useful.
Corin: I think they are useful and I think they're useful, but I think there's a lot of…I don't have an axe to grind with these people at all.
I think the world of them and I think they're often very humble. It's like they understand that their models are flawed and that they, if they want to remain trusted and useful, which they, by and large do, like you, you're, honest about that. You're like, hey, like this is what the modeling predicts.
We think this will be good. There is often attention where perception of computation often lags reality. So like experimentalists are always maybe unjustifiably skeptical and computational scientists are always maybe a little bit too optimistic. So there's some sort of like dialectic there, I do think people have a clear sense of their role and are happy to be useful.
[01:44:06] The future of NNP's
Abhi: That makes sense.
Corin: We think of things, much more from the small molecule point of view. Cause that's, it's much more chemical. And a lot of the arguments I've, we've talked about have been around how this, what we're building will be useful for chemistry.
One bear case for Rowan would just be like, trivially, everything becomes gene editing and we never need to think about atoms ever again. Do you have a, do you have a take here?
Abhi: If you look at all of, like human biochemical processes as like a flowchart.
I think you're the one who said that small molecules are the equivalent of cutting out, one box to another box in terms of interactions. And the addition of proteins or genetic elements are like adding in a new box entirely. They do feel like they're playing in different areas. just like beyond, like the, flawed analogy itself.
There is also this like, small molecules are small. They can slip into places where like larger things just cannot reach. I do think there is this world where. small molecules do seem to be getting larger. A lot of, proteomics based drugs seem to be tending towards smaller. Like you're going from antibodies to nanobodies.
There may be a happy medium there somewhere where everything is like macrocycles. but who, who, knows for sure? I, think a lot of revolutions in therapeutics are very much like non iterative. They just appear out of nowhere. and that may very well be the case again here.
Corin: Yeah, it's interesting to think about. I think one of the things that I've been dwelling on a lot the past few weeks is like the, one of the big differences I think between proteins and biologics based approaches and like small molecules, it's just like the, almost like the information density per unit area is so much higher.
And I see a lot of the trends. With unnatural amino acids, with macro cycles, with, everything is getting more complicated. And it makes sense that as we, we want to go beyond 20 amino acids. We want to, access, we want to be able to turn more knobs essentially when we're optimizing something.
And this is I guess even the large molecules, the design starts to feel less evolutionary and more like a small molecule problem. But that doesn't necessarily mean that the small molecule tools are going to be the right answer for everything.
Abhi: That's an interesting way of phrasing it. I think traditionally I very much think of, I think a lot of like protein design is like trying to obey the laws of human physiology, a fair bit more than small molecules.
I think you've, described molecules as zero day exploits. I think proteins are very much like you're, we're trying to fit the same like binding pockets that like nature already has something that binds to it. I think the advent of non canonical amino acids does change the game a fair bit.
And I think I am probably the person least well suited to opine on like where that actually leads us. But it does feel like it's heading in interesting directions. There are like protein modeling papers that are trying to account for the, the existence of non canonical amino acids. I think it's still very much early days.
But I do feel like those are, that is one of the areas where I feel like, dynamics are really the only thing you have. Because you don't have this, decades long historical collection of, non canonical amino acids.
Corin: Yeah, I think that's right. I guess falling back on physics is like a decent, When nothing else works, hopefully physics will work.
Even it's maybe not the best tool for every job, it's at least it's reliable.
Abhi: When, like general AI companies are developing like their own foundation models, like you have Anthropic with Claude, OpenAI with ChatGPT, they're all hosted on their own platform.
There's very little interconnection between any one, one bot and another bot. Do you imagine a similar phenomenon will pop up with neural network potentials? You'll have this community of like open source neural network potentials created by some well meaning academic. And you'll also have these gamut of startups that are developing their own neural network potentials and no one will want to play nicely with one another.
Ari: I think there's going to be a lot of variance startup to startup. I think we're already starting to see this. Startups like Orbital Materials have been open sourcing their NNPs with really permissive licenses. And so I think like it, I could imagine a world where, a startup decides, we do train neural network potential sometimes, but we've decided that, it's not a core part of our strategy.
It doesn't help us build power as a business. And so we're gonna open source that work. And I think that this is what, a company like Meta FAIR Chem is doing too..
Abhi: I didn't know, Facebook had a, like AI, like neuro network potential research group.
Ari: They do, and I think it's because that there's a story, they're working on materials for their new glasses. They need to be able to model materials really well. Maybe somehow they're like, we should do basic research on materials in some part of the Meta organization. And, that's one myth for how the, FAIR-Chem people started training NNPs.
I don't know if it's a true myth. But, they're open sourcing their NNPs so far to they're saying, this is a thing that we want for our business, but it's not a core part of, helping connect people, which is, Meta's goal, the NNPs are completely tangential to that. And so they're happy to open source them.
I think there are also some startups who are happy to open source them. really going after the, we're going to be like a model building and architecture company and I would be so surprised if those people open source their models unless they think they can build some sort of great open source business around that Databricks or something.
I don't know. I think it would be hard. Yeah, so I would imagine a fractured future but still high quality open source models.
Abhi: I think there is it's like a genuine possibility that a lot of these protein foundation companies become like maybe winner take all situations where one protein model is genuinely good enough to model the full universe of all possible proteins.
Do you think that'll ever be the case in the world of simulation?
Corin: You always have some tradeoff between like specificity and generality, where you can, we've talked a bit about speeding up like inference here and you can imagine would you take a 500 billion parameter model that does all elements and all spin states and all like confirmations? Or, would you rather have a 20 million parameter model that's just really great at amino acids. There's definitely some applications for which like a much smaller, much faster model that's good at specific things would be advantageous.
I think at the limit, a lot of papers right now are like, just fine tuned on a single protein. And it seems like being able to quickly, not have to retrain a whole new model to modify your system. Like it seems like you want some amount of transferability, to say that the protein people get one model, the OLED people get a different model that doesn't seem ridiculous to me. And maybe you can distill the massive model down in some way. I don't know. AI people are great at their sneaky tricks.
[01:51:23] Selling to scientists
Abhi: Switching away from like the scientific discussion for a bit, I feel like I've heard multiple times that scientists are very often terrible customers because their needs are often so hyper specific and they also have the least money to actually give you to satisfy those needs.
if you agree with this, are you often trying to convince scientists that Rowan is worth it? Or do you try and target more executive level people first, or do you disagree with the concept entirely?
Corin: I think there's definitely some truth there. Like science is a, tough field. There's a lot of details to get right.
Like it's not, there's plenty of like horizontal SAS plays that are, I think are easier and like simpler. There's a bigger TAM, et cetera, et cetera. Like you, I think we really like what we do. I really like scientists. I like working with scientists. It makes it easy to get up in the morning and want to go to user meetings and scheme about how to make it better.
And I do think there is like it is a little bit cope that scientists have no money to spend because so much money is spent on science. Like this is like just the ballpark site, right? Like 200-250 billion spent on drug design per year. Oh, I can't make money in this. I'm not saying that it's easy to start a scientific software business, but it's not that nobody cares about this.
It's not that no money flows through this. there should be a way if you're doing good work that matters to people to make a great business here and I think it's, every business is a bad business for some reason or another, this one has its challenges, but there's, definitely a way to win.
Ari: Yeah, I think selling into a big organization, you really need to get a lot of people on your side to, to, close a big deal. And that. I don't think that we've really done this in a way that, I'm, dreaming of doing yet, but I think, you want the users to love your product, to be willing to use it.
And you also want, these, executive strategic decision makers to understand, how this is important, be on board with whatever, spend, they're going to commit to this tool. and I think that. With any company, there are all these communication challenges that are often underappreciated by really technical thinkers, where if you're an engineer, you might think I just need to make my product better and then I'll have a great business.
But your product is only as good as the people who use it. And probably the people who can pay for, those, guys to purchase tools.
Abhi: Have you guys seen the, like that Spring Discovery tweet thread about, Spring Discovery is like a high throughput screening platform for looking at brightfields images and the company's shutting down like after a decade.
Corin: Yes, I did see this.
Abhi: Yeah, I like, he had this like, really interesting tweet thread about how, like scientists love the software. It's deployed to five out of, I think 20 of the big pharma companies, a bunch of leading academic groups, Broad Institute, UCSF, University of Toronto.
But they just didn't make enough money to stay afloat. I think there's this, like the hard part of developing scientific software is like capturing the value that you're actually bringing the scientists. Do you think there's like a big failure mode that a lot of scientific software companies, especially in the simulation space run into with regards to this.
And are there like ways to alleviate it?
Corin: Yeah. I think so two failure modes that we've seen a lot, at least we've thought about a lot with Rowan is like, one is that you labor in ignominy and perish in obscurity, that like you can do the world's greatest work behind the closed walls of your company and in your platform, but if you're not, you need to be able to get other people excited about it, and you need to be able to communicate to the world what you've done, and you're like, you can't just do that when you're knocking on doors asking for checks, like it's too late at that point, like you need to be like legibly exciting to other people such that they like are trust you and want to work with you.
I think another issue is just like it's trying to connect what you do to like real outcomes, you know because the bottom line like what makes people money what creates shareholder value is, particularly in drug discovery so there's so many sort of RL steps away from what you do in like early stage R& D, like we make it faster for scientists to screen compounds or do brightfield imaging, which helps improve their capabilities, which will help increase our odds of not failing a clinical trials, which will increase our like, it's a, it's difficult to put numbers on that and to really...
Abhi: you're very far away from the money.
Corin: You're very far. That's a much better way to say it. Yeah. And so yeah. You have to be object level really good at doing the actual thing that you do and that is like what gets you in the door, right? That's how you get it. And then I think you have to be really good at being honest with yourself and like finding a way to justify the value that you're creating and to actually like to be able to say that in the negotiating table.
I think a lot of people fail at that and it's tough to really peer inside and figure out is it, who's wrong? Because if I think I'm providing you a million dollars of value and you're paying me 50k a year like, are you wrong about my value or am I wrong about my value because something's not right there. And I think that's super case by case. Something being nice something that scientists like to use is like necessary, but not sufficient to build a great business.
Abhi: How do you convince people that like what you've built is useful? I think like Rowan's an incredibly aesthetic piece of software, but how do you actually connect that to saying this will provide to you tangible value that's worth how much we're charging you?
Ari: I think you have to connect the simulation or whatever tool to a problem that someone is currently facing or at least cares about. Maybe it's not a current problem they're facing, but one thing that we've had a lot of success with is there are people who are trying to tune redox potentials of their molecules.
And they're saying, I really care about this, property of my molecules. How can I design molecules where this property is different? If we say look, you can draw the molecule, press this button. And it'll tell you that property, with decent accuracy, then that's really useful. People are like, Oh, like I'll start using this in my workflow.
If there are too many steps, you have to run this workflow and then you have to do some statistics and then you have to go to this other tool and do something. Then, the values not connecting all the way to the thing that the end users actually caring about, they don't know if the values in the software or the statistics or whatever other step you have them do.
And so I think it's a much harder thing to communicate about. And so I think we think about is like, okay, what are the problems people have and think about and how do we work to make simulation actually solve that problem all the way. And
Corin: It's something relatively few other companies are doing.
I think of trying to go the extra mile of just like at the instant in your life where you have this question that you could answer with simulation, we want there to be something in Rowan that's like answer blank question. You click a button and then it gives you the answer, like having a lot of empathy for the user there, which I think maybe comes from not being a computationalist by training, but there's, yeah, the best and worst thing about selling to scientists I think overall it's good.
It's they're very data driven. So we predict things and at least to the extent that we've done sales with success, it's like, you want to predict this thing? Let's try predicting it. The predictions are good. And they're like, Oh, the predictions are good. All right. Then, that's like a, you, can't really hide much in that.
Abhi: I guess my initial thought is, they look at the results, and they're like, oh, what if it's wrong in some other area? They're, like, they're so data driven, that they're unwilling to accept anything that falls under their normal distribution of, good, software to use.
Ari: I think a lot of people when they're testing some new tool, they resort to the standard known test cases that they've been trained on. So if this is like a new language model, maybe one of the test cases that people throw at it is they'll ask it how many R's are in the word strawberry.
And it's now like this informal benchmark that whenever a new model comes out, people go and immediately the first thing they ask it is how many R's are in the word strawberry. And then if the model says two, they're like, this is a horrible model. If it says three, they ask it another question. And I think when people are using our software for the first time, they do a similar thing.
Abhi: There's a set of sanity checks that they've informally created for themselves.
Ari: Exactly. And so maybe if they're computing redox potentials, they'll they remember from a textbook, I've memorized the redox potential of benzene, they draw benzene, and then they compare the, number, maybe they don't, memorize, exactly that.
But I think that they have these built in test cases, and if you fail on one of those test cases, they'll discount your software immediately and you have to pass those first, basic test cases. And then once you've done that, you have this baseline of trust that then you're building on and you can, work on actually okay, let's evaluate a data set.
And, go from there.
Corin: I think this is true for everything though. this is, there's some stat about Uber that like the first, one or two rides you have, like dictates how you think of the app, like whether you churn or not, like pretty quickly. If you open the app for the first time and it's no cars within 20 minutes, you're like, this is a piece of crap, I'm not using this.
And it's, I think it's the same with Facebook, like the number of, friends you have in your first week pretty strongly dictates whether or not you stick or not, I think it's just human nature, I don't want to waste time on something that's obviously bad. And that's very rational.
[02:01:41] What would you spend 200 million on?
Abhi: I think like the, like one of the, like the last questions I have is what do you think is the current bottleneck to making better neural network potentials outright? Is it data set quality, model size, data set diversity, or something else entirely?
And when you're answering this question, assume you've been given like 200 million by an anonymous donor to like push, push the state of the art as much as possible, like what could you get?
Ari: I think that there's some architecture questions that we would want to run experiments on. These are things like message passing. How much message passing do you do? Do you need to strictly enforce SO3 equivariance and do you enforce it at the beginning and end of your model? Do you enforce it at every layer or can you throw it out altogether? I think you want to ask those questions as you try to scale the models to, to figure out, if I'm going to start training this on more and more data, which one will actually scale and, more data means less error, better model. I think that there is a lot of data set generation work that remains to be done. I personally think that like we should generate a metadynamics data with range separated hybrids on all the systems we care about, and we should train on that.
Everyone's got their own opinions in this. I think, if we can find a way to do multi fidelity learning to use lower quality data sets, that'd be super valuable. And then I think as we try to scale to bigger and bigger systems, you walk into these questions about coarse graining. So can I coarse grain out hydrogens, residues, solvents?
Those are like the easy, coarse graining, beach heads maybe. And then I think, when you're thinking about using these neural network potentials for molecular dynamics. So when you start walking into these sampling questions, if I'm trying to actually now, run a whole video on a system, am I just running normal MD?
Am I going to run metadynamics? Can we do some sort of like Monte Carlo step generation and acceptance criteria that will recreate my potential energy surface? And I think that, there are going to be a number of great research teams that sort of start in the next decade and we'll work on answering all of these questions.
And if someone gave me a big pile of money, I would just start now, let's try to, get definitive answers to each of those questions.
Abhi: On that note, I'd like to ask you your opinion afterwards. Do you think the current resources like ATLAS and MD Repo are like they're tending towards a good direction or you think there's some fundamental, like failure point?
Corin: ATLAS and MD Repo, they're both on MD, right? Like just regular, like Amber or something like that. Yeah. I think, it's just debatable exactly what the quality of a ginormous MD repo is. If we have skepticism, which we do about like force field quality, like it's like a lot of these early QM datasets where they generate a ton of data at a pretty poor level of theory.
Like it's great as an ML exercise to see if you can fit the data. It doesn't move the needle that much in terms of ultimate quality because you can learn something on inaccurate data. It's still not going to match experiment very well. So yeah, I think there's definitely value, but I'd much rather just do the MD properly.
I think that is a self serving and easy to say answer it to not have to do it. But that's my feeling.
Abhi: As of right now, there's no real data set out there that you think approaches like the level of quality and quantity that you want?
Corin: Yeah, I think that's right. I think to underscore something Ari mentioned briefly , I wish our ML co founder were here because he could make this point much better, but there's not a GPT quality architecture for molecules or I think like other like 3D graph problems yet. We can't just dump in a ton of data and expect like the H100's to all just go brrr and it to like magically get better. Like I think it's possible, but there's still some sort of like scale architecture problems, which are not fully worked through yet.
And I think the only way to do that is to try scaling and to figure out what works. And I think it's very likely that things will function okay, but like...
Abhi: It hasn't been tested yet.
Corin: Yeah, we can't just like trivially spend 200 million dollars on CPU time, generate like a crazy amount of like terabytes of data, and then just like press play.
Abhi: They are like consortia that are trying to like brrr and try to develop huge amounts of data. Do you think there's a very genuine possibility that the datasets they end up with are not useful?
Corin: I, this is one of my big concerns with the materials science sort of NNP world is that, so Materials Project.
It's a really cool, so it's like a consortium type thing. Like they, they have a standard set of theory. They like have worked to put together like a big database. They have huge collections of data. And then there's also some papers showing that like actually the settings they chose were a little too cheap.
And you get these like sneaky errors here. Like here's all these issues. Some of the predictions are fine. Some aren't fine. And so then you're like... I guess you have like very highly correlated error. For better or worse in sort of molecular land, it's everyone for themselves. And some people do really good jobs and some people do really bad jobs, but there's not the same sort of single point of failure.
And so I don't know what the future should be. we generate our own data sets internally just cause so we can have tight control over that. I know some companies do that. Some don't do that. I think it's...
Abhi: It's the final results will dictate who is actually the winner.
Corin: Yeah, ultimately, we don't, we're not doing this because we think it's just nifty, and we're trying to waste investor money.
We want to make an impact on real things, and everyone has a different tactics of how to get there, but we're all trying to get to the same place. it'll be fun over the next year or two to see how it all shakes down.
Abhi: And, for you, Corin, the 200 million, would you also focus on just, generating high quality data, or do you think there's somewhere else?
Corin: Yeah, I think I agree with everything Ari said, and to frame it a little bit differently: there's this massive gap right now between how fast MD is and how fast neural network potentials are and how fast they need to be to do everything MD does.
So a lot of the things we do today are like QM, like DFT, but way faster. And that's super cool. But a lot of the really awesome things we want to do are like MD, but way more accurate. And you can say maybe we're four orders of magnitude off in speed. That might, I think that's probably about right.
And there's maybe eight different ways we could imagine getting an order of magnitude speed increase. And so there's some sort of problem of we just need to get like half of them to work. Okay. And that's I don't, it's not like blue sky research. It's not like figure out like a cure for cancer.
It's like an applied research, ML engineering, like an algorithm optimization problem, like some combination of figure out how to get our accuracy, like a bit higher with more data, figure out how to scale to really large graphs. And then it's just make it be faster. And I think that's, like just, I don't know, just spin up some little teams. Like a 2018 OpenAI. Right.
Abhi: Yeah, that's the dream.
Corin: That's what we're all working for.
Abhi: Yeah. Well, thank you so much for coming on the show and going through three hours of talking. Yeah, thank you so much.
Corin: Thanks for having us.
Share this post