title: The economics of discovery, with Ben Reinhardt
author: Complex Systems with Patrick McKenzie (patio11)
contenttype: podcast
publication: Complex Systems with Patrick McKenzie (patio11)
published: 2025-12-04T08:03:54
sourceurl: https://pscrb.fm/rss/p/prfx.byspotify.com/e/media.transistor.fm/8dad765a/019f6971.mp3
word_count: 8079
Welcome to Complex Systems, where we discuss the technical, organizational, and human factors underpinning why the world works the way it does. Hi, to whoever buddy my name is Patrick and Kenzie, better known as patio 11 on the internet. And I'm here with my buddy Ben Reinhardt, who's the founder of Speculative Technologies, a focused research organization. FROs, if you're not familiar, are a sort of innovation that is returning to tradition in terms of how we do science funding. But we've had some episodes recently about charitable giving and some episodes recently about sort of for-profit investing. And science funding sits uncomfortably in like the spiritual intersection of those two, where the gains from core science funding are often not directly captured in the way that the gains from funding a company would be captured. But they're not exactly charitable either. For one thing, those of us who have worked in tech have had long and prosperous careers due to someone in the past making a decision to fund some amount of research that we eventually built on top of. And with that super broad prompt, I just want to talk about what we currently do for the funding ecosystem in the United States and how we could improve it. I'm excited to be here. One quick correction is, Speculative Technologies is not itself a focused research organization. We help people start them and like aspire to run them internally. And I can get into the new degrees of like what a sort of the classic definition of a focused research organization as classic as a five year old definition can be. But that's that is the one little little asterix on that. Yeah, I bet. So from a super high level, I think the common narrative in tech spaces of the funding for basic research in the United States and elsewhere was that it was at one point in the past largely industrial game with places like Bell Labs that had large semi monopoly powers investing a large amount of economic rent into their internal laboratories which deployed huge amounts of capital by the standards of the time against basic research. And then that fell out of favor for a variety of reasons and in favor of the federal government funding almost all basic research through a couple of different funding authorities and organizations. And there has been something of a shift of that in the course of last few years with increasingly a lot of research in particular areas of interest funded through industry again. But the federal government remains the largest funder by quite a bit. Does that broadly capture the shape of the curve? That broadly captures the shape of the curve. I think it obviously depends on when you start when you start history. What is your baseline? So if we look at sort of the post world war two system, which is kind of what we live in today. The government has sort of always been funding a lot of basic research. And the industrial funding of basic research. I will put a star on this because I think the categorization of research into basic and advanced and development is a bad bucketing system. But companies do not do much less basic research than they used to. And now if you look at the breakdown, the vast majority of it is funded by the federal government. And I would love your take on why these three buckets are not a great tax on me for organizing the world of research. But just for the benefit of people who have never considered that question before. What is the like the quote unquote typical tax on me? And then we can go into reasons why that might not leave reality at the joints. So the the typical taxonomy of research. And this is like encoded into law. Is that there is basic research, which sort of in the sort of classic sense of your scientist who is like, Oh, I wonder like how snails meet, right? Like I think is is sort of like a prototypical. It is is research done into the nature of the universe with no mind towards how this might be useful for anybody. It's just like pure curiosity. And then there is applied research, which you are then still very much doing trying to try. They're still very open-ended questions, but you are now trying to do something useful. So now it is like, how can we make the structure of this material have this specific set of properties, right? So in a way that like we don't know how to yet. And then there's development, which is like for lack of a better term, like the the last piece before making a product. So to a larger extent, like the work to make starship that space X is doing is development in the sense that they are trying to do something that nobody has done before. And it is very hard, but there is a clear sort of product at the end of it. That's the quote unquote, just an engineering problem. No new sense required. Yeah, I hate it. Something that engineers in the add remember love hearing, particularly from from the CS field, like given the existence of terrain, complete languages, everything is an engineering problem. Yes, I think it is sometimes useful in terms of my mental taxonomy to think, okay, there is foundational research done into elect for magnetics. And then in the middle, we discovered that LEDs are a concept and then actually getting like a blue LED to exist in the physical universe at a price people can afford is more applied research. But the United States through the genius of its elected representatives has encoded this tripartite definition into law. Why might that not have been a great idea? Well, I think it's like semi conductors are a great example of why. So you think about a point at the history of the transistor and you know, it starts off with sort of this realization that there are these things called semiconductors where they're not quite conductors and they're not quite insulators and they have like weird properties about how electrons move through them. And like that, that is certainly basic research. And then people are like, oh, we might be able to use this to make better amplifiers. We might be able to replace these like vacuum tubes. And so the way that you actually do the work there is some like a mixture of applied research where you're like trying to get these semiconductors to be useful at the same time as like trying to figure out what the actual shape of the product is that you're going to make out of them. So this situation where you realize that actually like our understanding of the laws of physics is insufficient to explain how how electrons move through these semiconductors. You basically need to like do some updates on our understandings of quantum mechanics in order to actually like properly model how electrons move through this semiconductor, which and you're doing this sort of at the same time as thinking about the product. So it becomes this like wild and tangled mess with a whole bunch of loops. And so if you have a team and like one guy is trying to figure out new laws of quantum mechanics while talking to a guy who's like trying to like bond like hunks of semiconductor together to try to make this amplifier that is actually useful. What kind of researcher are you doing right it goes in no bucket and the reality is that if you zoom out like the vast majority of like useful research projects look like this. Yeah, I think it often gets a short shrift in discussions, even though it's been highlighted for decades and a number of places in Japan, the magic were just guys on which got appropriated slash homage to buy a US based consulting community and often with regards to semiconductors that the jargon thrown around is process knowledge there where there is something that hasn't been reduced to a paper yet. But you can't build a chip fab without having it where like just the designs and just the description of that, you know, all the specs for all the machines. You wired them all all up together and you will not get like a useful way for at the end of it because you lack the process knowledge and a dirty secret of science is that even when there is a paper process there's still process knowledge. Like there are many, many situations where the only lab that can actually do the thing that is reported in the paper is this this one lab that has the like the process knowledge that even if they are like trying to communicate it is very hard because you're like, oh, you have to like do like fiddle with this thing just this just the right way. And so I think process knowledge is shot through everything it's not just like even even when there are papers. This this sidebar is why the idea that we are going to teach AI to do all of the science just by feeding it a bunch of trying to get on a bunch of papers is one that I personally think is not going to happen. I think there's a number of reasons that one could doubt that broad hypothesis even though I'm going to we we hope that AI results in an acceleration science or else what the heck are we doing with it. Yeah, no, it certainly could. We will lack digital twins for a while to use another magic jargon word that people really like. And so to some amount of the science is going to be rate limited by people, you know, reading the AI from a screen and then using that to do something in the physical world and then iterate that loop a lot. At least until the point where we have a high fidelity simulation of the world that can be run into computer currently the high fidelity simulation of the world uses human language is to substrate or some derivative of human language in the model weights, which is a wild world to have ended up in. I don't think we appreciate how science fiction reality is at the moment anyhow worthwhile tension to explore some of the time, but we were talking about to a scientific funding. We can just talk about how science funding works in the United States right now let's let's just do like a broad overview of of the system and I will find that. How science is funded is deeply coupled to where the science is done so there's sort of these these two parallel tracks that are deeply interrelated right now using the the buckets the basic applied and development buckets as as they exist right now. I think it is roughly roughly order 900 billion dollars of it goes into science research funding in the United States and and sort of like of that we can we can slice it up in a couple of different ways the vast majority of it actually goes towards development so I think that it is don't quote me on these numbers but it's like order 700 of that 900 goes towards development. And then there's roughly two thirds of the remainder goes towards applied research and then of the and then the rest goes towards basic so basic research is the least expensive in this makes sense right like you don't need to be building like huge machines you don't need to be blowing up spaceships when you're doing basic research. Another way of chunking up that money is sort of like money like you sort of largely put into three buckets money that is from the government money that is from private corporations and money that is from private sort of foundations and other nonprofits and you'll see stats that again like a lot of the money actually does come from businesses. So I think it's roughly 600 of that 900 is coming from businesses and then another 200 is from the government and like less than less than 100 is from private organizations. The thing to keep in mind it so looking at this many people are like oh business funding for R&D is very high like what's the problem and the thing to note is that the vast majority of the R&D money that businesses put in is is D is the development and I'm going to go slightly deep right here which is a thing to keep in mind whenever you hear this number is what I'm going to do. But things can get coded as development spending by these businesses some of your audience may be aware of this but like for example like building a new feature in a piece of software can get coded as a development expense. And so when you hear this like that there's all this money going into to R&D and all this money going into to like R&D from businesses. It is like very legitimate research right like Microsoft and Google are building quantum computers and discovering elements and and all this stuff right so businesses are certainly doing good research but the numbers are a little bit skewed. So he's felt something like an impedance mismatch of how tax policy around software and specific has huge distortion area effects around one our understanding of just like the fundamental domain of science and around allocation of resources and the economy so you were mentioning that. Businesses can code this which basically means like the if the business identify as a particular line item whether that's an engineer's time or similar as a lettering up to research then they get the R&D tax credit and the companies are incentivized in many cases to like try to maximally claim for all the R&D which pushes up the claimed amount of that our research involvement worked on every year. However, as everyone who has ever worked in a software company before that knows like a certain amount of you know one has a software business. The project exists and a certain amount of the intellectual effort by engineers and product managers and similar on that software for every year is effectively op X. But doesn't look like op X in a certain view of your balance sheet flesh profit loss statement and isn't code as 100% op X when the accountants or consultants file the tax return because if you like if it's very flavored op X you get a substantial credit from the United States from it where if it's like simple normal garden variety op X you job. And as a result we have incentivized some of the largest version of capitalism to say well we we have an awful lot of cherry flavored topics every year where that doesn't change the fact of the physical universe doesn't change that the rate we are you know learning about reality all that much as you mentioned like the large firms in capitalism also do a whole lot of legitimately cutting our research on everything from them. Quantum computing systems to you know the attention is all you need paper is classic at basic quote unquote basic research where there was no application that could actually be made for it at the point it was written and then a number of firms interestingly not the ones that wrote the original paper chased after it with them and vigor and now we have you know magic answer boxes on our phones yes. A couple of other things to flag about sort of I think it's it's an addition to like I think we've sort of talked about like where the money is coming from and then there's the question of like where's the money going to and the vast vast majority of money that goes towards. What I call like this is my own term but I call it pre commercial research and so this is sort of work that. Sort of to your point maybe targeted at an application but is not yet I to some extent not yet a thing that makes sense as an investment because of the level of uncertainty timescales and. Public goodsiness of the research makes it so that a rational investor will look at it and say that I do not want to put my money into that if I want my money to make more money the vast majority of that right now happens in universities. And so when when you hear people complaining about the academic system that is that is sort of these universities and then there there is a good chunk of work that happens in in national labs and and a number sort of a long tail of of other organizations and so that the the sort of my hobby horse one of my hobby horses I have I have several I you could say I have like a hobby chariot. That that is pulled by my hobby horses one of them is that to to a large extent we we have developed a system in the United States where we sort of when you ask someone how does technology happen the the response will be well someone does pre commercial work in a university until it makes sense as a company and they will spin it off into a company and that company will raise a bunch of money and make products and low there will be technology. Similar to how I the the basic applied development model of research I think is is a bit flawed so to is this like how does how does technology magically happen is is also a bit flawed so I think there are all sorts of institutional incentives that. Through renters in the works here but one of them in particular is at the point where you are spending something out from a university. Into its own private company or selling the technology to someone are similar our friends at the technology transfer office at the university get involved and that you've had some choice words for them in the past that you want to as been out that thought on air for people sure so the explicit choice words are that. I'm generally a big believer in trust or consents in that most institutions that exist exist for a reason and need to be reformed but are by and large should exist that is not the case for tech transfer offices and and I think that the world would just be better if we bring them to the ground and. This is not against anybody any individual in a tech transfer office there are many many wonderful lovely people who work there. But that to a large extent the tech transfer offices are serving none of the purport their purported purposes right so if you think about what ideally what a tech transfer like you think about it it's like oh the purpose of a tech transfer office should be twofold one is to make sure that. Technology that is invented in the university gets out into the world and so the world can benefit from it and to make it so that the university can capture some of the value that they've created which will then hopefully be plot back into more research to create more technology and more science. I think the acknowledgment of an ad read sounds cooler in Japanese. This is the next sponsor of tech you'll be looking forward to. You might have heard of this podcast that cuts to PEPFAR and USAID were extremely disruptive to healthcare and some of the world's worst off communities. Private funders ended up picking up part of the slack. How would you decide whether that's the best opportunity for your charitable dollar particularly if you don't have a team of professionals working for you. Give well is working for you and everyone else give well is a nonprofit their team of researchers works in real time to track the impact of foreign aid cuts and they contribute their research to the comments or free. For example they've found one of the most effective interventions is paying caregivers in foreign nations directly in cash to take their children for routine childhood vaccinations. This decreases the disease burden on the kids and their families and reduces childhood mortality give well has spent 18 years researching global health and poverty alleviation. This work is funded by donors who think it's useful for directing their charitable giving. Give well also lets you donate to the causes they think are most effective and will pass 100% of your donation along to their recommended funds. To make a tax deductible donation today go to givewell.org and pick the podcast and enter a complex systems at checkout. Make sure they know you heard about give well from complex systems. Again, that's givewell.org to donate or find out more. I have an engineering degree and code my own websites. It's probably the most irrational choice I make in business. Low leverage, a spiky maintenance burden that always comes with the worst times and they don't even look good. Don't get advice about design for me instead take it from framer a sponsor of today's episode. Framer already built the fastest way to publish beautiful production ready websites and it's now redefining how we design for the web. With the recent launch of design pages a free canvas based design tool framer is more than a site builder. It's a true all in one design platform from social assets to campaign visuals to vectors and icons all the way to a live site. Framer is where ideas go live start to finish ready to design iterate and publish all in one tool. Start creating for free at framer.com slash design and use code complex systems all one word all capital letters. For a free month of framer pro framer.com slash design promo code complex systems rules and restrictions may apply. Both of these things seem like they're good things. But the reality on the ground is that textures of offices serve neither of these purposes. I looked looked it up beforehand and the fraction of tech transfer offices that are profitable of if you look at across all tech transfer offices is 16%. So the vast majority of tech transfer offices actively lose money of the ones that even do make money the total amount of money that they make sort of across the US is like single digit billions of dollars a year. And that's like every single every single spin out from every single university that's that is the RNA vaccine that is Google that is Gatorade which makes shockingly large amount of money. And so so yes, so the amount of money that tech transfer offices are generating is tiny compared to the amount of money that research actually needs most of them are not profitable. And I think the biggest thing is that like the amount of pain that people have to go through in order to try to get technology out of the university is is kind of mind boggling. So for your listeners, when someone signs an employment contract with a university part of that employment contract is that they do not own anything that they invent sort of using university resources or on the clock for the university university. So if you are a scientist and you invent something at the university and you're like this would make a great product you like must go to the tech transfer office and say hey may I license my own invention from the university in order to to spin out a startup. And then the the tech transfer office will like ham and haw and first they will feel no urgency on this and as your listeners know time is of the essence when you're starting to start up and then they very often will want like fairly onerous deals where they they demand monthly payments starting at day zero or a very large like unreasonable large amount of equity. There are questions about like counterfactually how much technology has not spent out of the university is because it's such a pain to go through the tech transfer office like how many companies have have died because people have tried to and it is either have been discouraged or the deals were such that it was then impossible to raise more money. They function as just one more in position on the PIs time doing paperwork instead of doing research a long time ago in a place far far away was an undergraduate research assistant and my understanding of someone who had been a student until like a hot minute ago was that you know i'm working for the principle some of $12 an hour doing this undergrad research. Obviously i'm not going to get any sort of upside in this research that's not the point of the exercise i'm learning things etc etc this will be a good way to use the summer i spent more than 90 minutes that summer myself doing paperwork from the university. Tech transfer office and the PI spent tens of hours of the course of this year and wild i don't think there's any PI in a university who only actually works 2000 hours but if it were you know a 1% tax on all research produced by the university you want to know like what are we getting for paying 1% of all the potential tax and if that answer is like it's literally negative and just cancel it without replacement and maybe. Interesting wants to do something instead they'll just say yeah we're going to put a hundred hundred K check into anyone who spends out something that seems like you know we'll we'll be in the things that are on the far right tale of outcomes and for the other one it's like cheaper than having an office staff full of full time employees yes that would be very that would be very wise and we've we've run this experiment. There are sort of natural experiments in this if you look at the university of Waterloo the university lets inventors own their own inventions and to a large like basically they say hey if you go off and you make a ton of money you know like give some of it back and i think people. I'm an optimist about human nature but I think people do like if it really is a thing where they're like oh I was unable to do this without the university like the university of Waterloo is doing quite well off of some of the things that are invented and I think the the riff on this is like your listeners know the dynamics of startups very well where it is like an extreme. Tale dominated thing so the vast majority the vast majority of things that spin out of university are not going to be incredibly lucrative and so trying to like squeeze every one the right amount does nothing besides sort of you fewer shots on goal yeah and. I think we saw over and over again at striping out list is that the and i'll say striped not necessarily endorse representations that I make but startups in their early days are just so fragile and particularly in like the proto startup form where i'm not exactly sure what i'm going to do with my life next year the p i says am i going to you know continue climbing the ladder i'm going to try to go after that and research projects that I just got a grant for. grant me again some full of wax or should I track commercializing the last research project and in those early days just the notion of like well 600 pages of forms to do like option B versus option A is like it still work I mean you know I work for a living but. It is work that feels less trainings and I won't do the one that has 600 pages forms in front of it and barely an exaggeration by the way. And it's not just like 600 is a magic number where people have an internal capacity for 150 forms and start up dies we were tracking down to like literally form field level on eventually play this out a few years it probably like his visible macro economic indicators but. We have intentionally and for often good reasons strangled innovation at the earliest stages due to the bureaucratic and administrative overhead on it. Absolutely there's a way I quote that the bureaucracy is expanding to fill the needs of the expanding bureaucracy but that haven't interfaced with some of these processes directly you definitely feel that you know indicative truth of that oh yes. I I both have gone through them myself and very close to many professors and and sort of here on a daily basis that looks like. And then the day that also bears this out that they they did some surveys and I think that bureaucracy occupies now roughly 40% of sort of like logged hours by by professors so it is a very real thing. If I if I can sort of double click quickly on on a term that you've been using that I'm I just want to make sure that the listeners all knows is you've been using the term PI a lot. And I don't know that we've defined it and so just that is is principle investigator which is sort of a term of art of that like our sort of modern system of research funding which is like the person who applies for and gets the money. And I'm I'm noting that noting this because PIs like in a university usually the PI is a professor sometimes it is a a postdoc or or sort of a research assistant but that's majority of the time it's a professor but are sort of like the entire. Grant making system is organized around this assumption that there there will be a PI and this person will both like write write the proposals for research be responsible for executing on the research and be leading the research. Which it's sort of like a very granular level and like that they are applying for research to do a very specific project that has very specific scopes and aims and I flag this only because I think that it is certainly a fine system right like it is is gotten us here. Like it is produced many amazing things but as as a way of doing everything it's it's kind of shocking because if you think about it that would be like it's sort of in in startup terms that would be like you're not allowed to have a CTO the person like you are only allowed to have one executive who must do all the fundraising run the team. And the the scale on which they can raise money is like for very specific projects like we need to develop this new feature or or like roll out this specific app and you know heaven forbid you find a different opportunity and want to use that money for something else because you certainly aren't. So that is a just a thing to flag about how how this all works and this goes back decades, but I sometimes wonder if we're not reacting to the fact of science that's conducted centuries ago where when you know looks in the history books as a number of the foundational mathematics and foundational science were done by essentially board novel one or you know pet clerks in flimously I mean one that that just add like a relatively tiny amount. Money a single practitioner that was advancing the state of the art and then just time to think about the problem and draw things on the blackboard have correspondence with pierce. Increasingly that's not how science is conducted these days you know we have plucked much of our low hanging fruit and now you sort of need a large team in a lab kind of environment to do much of the research that we're discussing. But the funding mechanism assumes yes we understand you will need a large lab however for whatever reason we can't really interface with large labs we have to just pretend that it is one scientist who has all the ideas in their own head and we will like essentially band of you a statute any notion of like you specializing in you know in private industry. Granted CEO almost always like owns the fundraising ball as one of their things but you're not also forced to be the person that is doing all the hiring and filing all the time cards and then doing server administration whereas under the grant contracts no it is literally on you like you have to be the chief higher the chief correspondent with your your funding agency and also like most of the brain trust. Which doesn't do wonderful things for those scientists who are good at science but not great that you know admin or managing people if they get essentially frozen out of the funding pipeline unless they have one of our quote unquote. supremely talented polymath PIs that can sort of like step in for them and write the proposal for them well like how many how many over you know a required line in the proposal is your work that you're applying for not somebody else's. Yeah yeah that is that is spot on another one of my hobby horses is trying to create more systems that are sort of outside of this where you can sort of have. You know funding at some some level above the level of like a lab and a PI like in the same way that you have a company and within that company there are you know several departments and they're doing. You know all these different things and not needing to you know go to someone outside for for and justifies sort of like on a project by project basis why they need by why they need that funding for the benefit of people who have only worked private industry so obviously we spend a lot of intellectual calories on fundraising and private industry whether through investment or through sales but the calories largely spent the company level. And then while budgeting is process teams have some flex in like how they spend their departments budget and often it only takes an email to get more allocated to you if. A PI at a university lab has their ambitious research project and needs an extra $200,000 how much additional difficulty does that add to their life. Like basically like I I pause only because it's like almost unquantifiable right like in the sense that there's one thing if that money is for like a entirely new project right like there's there are processes for that where it's like oh I have this idea of this project it needs $200,000 professors do when they when they start their jobs they get what is called the startup budget. So they have this like slush fund that the universities this sort of seeds a lap with and so. If they have that so some of that they could deploy but if they don't want to want to save it for a rainy day which is very reasonable and they want extra money for an existing project it is sort of there like there are no standard channels for that right so it's like the way that you would get that money is like if you happen to know an incredibly wealthy person. Or like you would secretly like basically apply for money for a different project and then like do some very sketchy like book cooking to be like oh well like this grad student who's on this other project is just going to be spending some other time on on this other project and like you know. manage the thought if you want to spend that money on equipment because something that we didn't mention is that like most of the sort of pre-commercial research funding that's the two grants specifies how that money must be spent like what what you are allowed to spend that money on and you need to say like yes this amount of money will be spent on like consumables like you know chemicals and reactants this amount of money will be spent on on personnel. And sidebar that personnel money is usually your mark specifically for for grad students or postdocs so you're not really allowed to hire professionals to to come it like specialized professionals to come in. And a very small amount of that will be your mark for for equipment which just in terms of of of going deep on things creates these very interesting incentives not to automate anything in science right so everybody's pointing at like you know there's there's the like everybody's ringing their hands about how science productivity has not increased as much as as many areas of the economy and I just some extent by the low-hanging fruit argument but I do not think that it is a complete explanation and that when it is relatively easy to get money for grad students and relatively hard to get money for equipment. You are very disincentivized for for like substituting for for getting more capex to substitute for opax even if it would make you much more productive. You know I've heard of very explicit conversations with with PIs being like so why don't you install some robots to do this incredibly repetitive task. That right now you have five different grad students doing like you know like basically like by petting like you know dropping one one drop of of chemical into another and they're like well it's easy for me to get money for grad students so why would I buy robots. This also has an impact on the the quality of the science done because we we reached for the grad student because it is the one option in the toolbox you mentioned. It is difficult to hire external professionals because the social system and the university makes that more difficult than just promoting a new person to grad student. Also the numbers thrown around might not sway industry wages for very long. But I remember there was a funded project conducted at my alma mater many years after I left that was doing urgent research into the effectiveness of a particular drug for treating COVID. And they had a 200 question intake questionnaire for patients that they were attempting to get into this drug trial. And we're realizing go figure that not many patients were getting through an online 200 question questionnaire to get approved into the trial. And through a random pathway I ended up discussing with the PI and I said so help me understand as the person who has not done medical research for his entire career. Why is the 200 questions is it that my hypothesis was it must be like institutional review board or something has asked these questions and they aren't you know comfortable for the actual success of the trial. It's like oh that's how many of the grad student coded like oh. So how many are required for like oh how about we delete the 196 and he said no one on the team knows how to do it because you know this is written in PHP only the grad student does PHP. Like it's a big internet out there you know you can find a lot of people who can press the delete key on a PHP. And it's like no like we have to do the grad student like and why is it feeling this meeting already and you know for like usual organization etc. reasons she wasn't in the meeting and wouldn't be available for days in this you know period of like intense criticality for the project and the project society that research is supposed to support and so they're quote unquote funding mechanism was having me asked the best man in my wedding who does code PHP. Hey can you look at this thing and press the delete key for us it'll be very effective and indeed it was oh that's that's wild. And you know grad students do a lot of very excellent work I've known some extremely talented people who were taking an incredible pick up relative to working in industry because they love the science and wanted to spend their life doing it but I think we are making poor use of their time and brains what as a society if we have in them pipetting chemicals into containers just because that is the easy thing to put on a grant application. Yes and and like that two two quick risks on that one is that that requirement to some extent like comes down all the way like from legislation because the United States government sees grad students doing research as fulfilling like two two different things. One is they're like we must train the work like the scientific workforce and two we must create research and you know this this sounds great at the level of like the Senate. But on the ground it increasingly has led to tensions where one we we are kind of like overproducing grad students if the vast majority of scientific research needs to be done with grad students if you're like okay every time we want more science we need more grad students. So we get this this overproduction of grad students and then to I would argue that we get lower quality research because like to your point many grad students are really amazing but at the end of the day like they are trainees. And I sort of like in this to if you were a software company and every time you wanted a new feature you're like okay cool let's have the interns do it and then you not only have the issues with people sort of like learning and doing at the same time. Which again can work but usually is helped by having many more senior people around but then also like continuity because the interns leave after a while and then people don't know how to like you know it's one thing if the grad student isn't in the room but often the grad student has who who coded the button has graduated and they're off doing their thing and you know it's like very hard you have to like do like code a lot of like code archaeology to do that and so that is that is sort of another way in which the system is. A little bit rickety well at least you can look in the source control for what they did which is a dark show how many grad programs that I've interacted with actually teach people to use source control I tried to introduce get hub to my lab during grad school like like this was literally literally. Everything was just files on computers and I was like hey there's this thing called get we could use it as source control and everybody was like man that sounds like work. Do you mind if I ask what your that conversation happened to that I believe that happened in 2013. Okay so the Joel test which came out in about like 2005 2006 or so it was a 10 questions test by Joel spillsky which was essentially helping people who had not yet joined the company evaluate where the that company had taste and so already by 2005ish in the private tech industry like do source control was a pass fail question yeah and it's roughly contemporaneous with my own research experience but that even 10 years later was still taking its time to percolate into the research community. You know we have a abbreviated time together today and I would hate to just well on on the problems and obviously one can't thank the research system in the United States enough for all the wonderful things in the built environment around us. But what are the you know reasons for optimism about this what has been working well recently. Yeah there's kind of like this emerging new world of people who have seen these problems realized that they're the system can be changed and we need to build new institutions and are sort of working on that and there's sort of like this the call us a small group of of missfits who are trying all sorts of institutional experiments and I think that sort of the the realization that we need to change this has started to percolate through more and more of the culture that's that's one reason for optimism. And then the other is like like as much as I think that it could be better like the reality is that the US research system is still sort of like the thing that everybody looks up to and emulates. So just like that is another thing to keep in mind is like despite all of this despite all of this there are still amazing people doing amazing work and and we should be very supportive of them. So just going a little deeper on the things that are going on we mentioned first right at the beginning a focus research organization. So so that is sort of this this idea of like oh what if we actually had organized research more like a startup where you have a core group of people working on a very specific problem. So that is that is one thing that's going on. There are a number of sort of attempts in various governments to to reorganize research funding. You have Arya in Britain. You have this sort of like ARPA model model after DARPA in the United States. You have organizations both on the the for profit and nonprofit side like speculative technologies that are trying to to sort of say like well what if we took all these these like things that I was I've been. Sort of critiquing and invert them and say like okay what if we do have like teams of professionals who are not burdened by bureaucracy working on problems that are not bucketed by whether they're sort of a basic or applied. Trying to do very useful things and there there are sort of many ways of implementing that and I'm very excited because I think that there's many people trying many different experiments with that. And so I think that that is a good reason to be optimistic. Well more experiments on how do you do experiments better and never hurt anybody. I regret that that's all the time we have to chat today but Ben where can people follow your work on the internet. You can find me on x at Ben Reinhardt at Ben underscore Reinhardt. I write both blog.spec.tech and my website BenjaminReinhardt.com. Thanks very much Ben for your time today and for the rest of you. Thanks very much and we'll see you next week on complex systems. Thank you Patrick. Thanks for tuning into this week's episode of complex systems. If you have comments drop me an email or hit me up at patio 11 on Twitter. Ratings and reviews are the lifeblood of new podcasts for SEO reasons and also because they let me know what you like.