title: Navigating a world in transition: Dario Amodei in conversation with Zanny Minton Beddoes
author: Economist Impact Events
publication: YouTube
published: 2025-01-27T00:00:00
sourceurl: https://www.youtube.com/watch?v=uvMolVW2v0
word_count: 8913
[Music] thank you for joining us I'm zy Minton BOS I'm the editor of The Economist and I'm thrilled that so many of you are here in this I must say very very nice surroundings very Swiss um we are going to do something we are going to find Clarity amid chaos as it says up there now I don't know how how long you have all be in Davos but I think finding Clarity amidst what's going on the world right now is a t order but it is what we try and do at The Economist we try and join the dots and we try and find Clarity and we are lucky in the next 90 minutes to be able to have the chance to hear from some of the smartest thinkers in the world on some of the most important issues but particularly in this first conversation to Dario amade who I I'm really not exaggerating when I say that you are one of a very small handful of people who is actually going to shape the world that we're all going to be living in um Dario as you know CEO and founder of anthropic uh one of the maybe one of the very small number of foundational models that will determine when we get to full AGI super intelligence how we get there what it looks like so the future that we are all going to live in is disproportionately going to be shaped by this man so that's why he's here that's why I'm really excited to have this conversation uh and I wanted to start but before I do that I must thank Zurich of course for making this possible uh you have um a fantastic surroundings thank you thank you for having us uh we're delighted to be here but Dario um two things have happened in the last few weeks and months that I think collectively might have a bigger impact on the development of this technology even than chat GPT and remember that moment two years ago when everyone thought chat GPT wow and I'm not sure that this is really sunk in but one of them is the development of what are called reasoning models these new front sort of topend models by some of your competitors not yet by you but by some of your competitors and the other is kind of seemingly from nowhere the development of really good cheap models from China deep seek if you haven't heard of it is an incredibly powerful model that has kind of come from nowhere and I think these two my my my thought is that these two events together are going to take this whole technology in a slightly different direction that we might have thought six months ago so I wanted to start by exploring those and firstly let's start with the US Cutting Edge models how big a deal is it that we now have these reasoning models which learn by thinking they're kind of like what you tell your kids spend a little time on it and you'll get a better answer isn't that sort of what they're doing and what can they do and what will we see in the next year or so from these models yes so first of all thanks for having me and uh very kind uh very kind words um uh so on the reasoning models in particular um we have a little bit of a different perspective on it um which is uh you know for the longest time most of the training that was done in models was what was called pre-training so this was you train a model on a bunch of data on the internet and that and that plus a tiny bit of reinforcement learning or reinforcement learning from Human feedback is what language models were until recently um there was another Paradigm if we think back to uh alphago in you know 2016 um you know done by by Deep Mind believe Demis aabis is is is uh is is here as as well was one of the Pioneers in that area um so this other Paradigm of train via reinforcement learning learn via trial and error um has also been developed and has been around for a long time what has happened is that we have started to marry these two paradigms together where you do the pre-training and then you do large scale RL and our perspective is that there aren't separate reasoning models and normal models that's an artifact of the early RL heavy models being trained on a narrow set of tasks what what is going to happen is that first we pre-train the model and it learns about the world in general and then we do large scale reinforcement learning where the model does very large scale tasks you know it learns to do do coding tasks it learns to do math it learns to uh solve scientific problems you know it learns to solve puzzles it you know it it even learns to do things like T comedy routines you know at some point at some point we'll get to that and so we are thinking about over the next year or two how to develop models to do that task much better than than than has been possible before by models we have seen with some of the early reasoning models and we will see similar things from from models you know starting to get to professional level performance at things like math and coding and that's the Striking thing right that they are now getting professional level performance is a nice way of putting it these models are getting you know as good as humans better than humans across a very wide range of thought tasks so that's the excitement about it but the interesting thing I think is also how this changes the economics of these models because and I'm going to forgive me I'm going to really simplify but the old style models you spent a ton of money training them ton of compute ton of capital to train them but then once they were trained it's pretty cheap to use them now because they're thinking as they're doing you're going to use compute while they're actually working that means they're more expensive to run what does that mean for you because it's going to cost you more for your models to be used yeah so I I actually on the on the cost side I actually don't think of it as different um I think of it as you're buying you're buying compute time on a CL cler and and what you're buying is thinking time in a mind so in all previous models it was just that the Mind wouldn't think for very long right you you would ask it one question and it would generate an answer in 10 seconds but the way I've always thought about the economics is you're basically you're you're you're renting someone's brain you're renting a virtual brain for a certain amount of time now what's happening is all that's changed is the questions we're asking the virtual brain are different and so now sometimes you ask the virtual a question instead of like you know what sports team you know run won you know the you know this this contest in some year um uh you know you're you're asking hey can you prove this mathematical theorem can you implement this coding feature and of course just like a human the model says okay it's going to take me 10 or 15 minutes to do that so you're paying for 10 or 15 minutes of its thinking time instead of one or two minutes now I do think that has implications on the output side on the price side relative to the to the cost side but there are implications we've always been thinking in terms of we have thought of these models and it relates to our picture of where things going as more more like labor more like effort than like software as these models get smarter they're going to do things like and you know I've talked about in 2025 we're going to you know release probably in pieces parts of a vision we call a virtual collaborator so it'll will be a model that you give it a task it works on that task for you know several hours maybe it checks in with you every once once in a while and you know that could easily generate thousands of dollars of value tens of thousands of dollars of value if you think of I give this model to a medical chemist um you know it it you know it finds a better drug candidate than it would have found otherwise or it reduces the number of you know animal trials it has to run or phase one trials it has to run it's quite possible the words that come from that model are worth you know thousands tens of thousands of dollars and so I do imagine a world in which and we're already starting to see it in some of the the the high tier pricing plans uh Enterprises and to some extent power user consumers which which Claude is actually very popular with may actually in some ways dominate um are the main source of Revenue because they're paying thousands tens of thousands of dollars a year because the quality of the model thinking is worth that the model is being paid like a coworker at least like an assistant but let me let me continue your analogy because it's a very interesting one that what you're paying for is your kind of you're you're paying for the labor of the of the of the of the AI right along comes right at the same time cheap Chinese version that's pretty much nearly as good frankly deep seek is is you know not far off Claude 3.5 son it um in fact as good in some ways um I they tell me don't don't like don't ask me but that's what they say uh you know the cheap Chinese version comes along why is anyone going to pay thousands of dollars for yours and why aren't they just going to go to the cheap Chinese version yeah so uh well I mean first of all it's uh it's it's not uh it's not currently as good um but uh um uh we're actually living in an interesting moment in time uh and it actually relates to the first question you asked which is uh this switch to reasoning models so if you imagine there's a lot of compute right we in increasing the amount of compute at all times in the past there the 99.9% of the compute went into one kind of training which is pre-training um we're now executing a switch over where uh uh we figured out how to put small amounts of compute into this second stage this reinforcement learning stage and because none of it was being done before there are big gains to that stage that the amount of compute in that stage is incre inreasing to the point where it will even become dominant right now we're in the switch over region where if with a little bit of RL training you can kind of catch up with the current situation whenever the Paradigm switches over the the kind of like the the landscape scrambles a bit but then it kind of reestablishes itself so you know later this year we and and probably others will have hundreds of thousands of chips I think you know probably uh there will be millions of chips from various companies in uh 2020 6 and More in 2027 um deep seek the Chinese company it's you know it's marketed as cheap but uh it's at least been reported that deep seek has 50,000 h100s which for reference is about half of what elon's uh Colossus cluster has I won't say exactly um how many of these anthropic has right now so so I actually think we have an opportunity and you know this this relates to China policy right now both sides are are you know roughly in the mid tens of thousands low hundreds of thousands of chips um that's why we're kind of close to parody now we're at the we're at the crossover point but as we go to hundreds of thousands and millions of chips there's two possible Futures in one of those Futures the US and its allies are able to provision that many chips fast enough and because of the export controls on on chips to China and because Chinese huawe chips are inferior China cannot get to that scale there's another world where both sides get to that scale let let's dwell on that a little bit more because I think given and this is not a a discussion about geopolitics but given the geopolitical backdrop I think this is really interesting because for for four years now I've heard from the from from American officials the single most important thing is that the US retains its lead in Ai and US policy particular towards China has been f focused on that and this entire export control regime has been built up and and for a long time you know Chinese were way behind we just used to write it Chinese are way behind they are not way behind maybe it's not quite as good as Claude son it 3 3.5 son it but it's it's not far off and were you expecting it to be that good honestly uh yeah so so I think it's a again I think it's an expected consequence of the switch over in Parise you think um I think deep seek in particular has uh higher quality than some of the other uh uh companies in China actually I you know we were aware of deep seek for quite a while and kind of identified it as more likely to succeed than some of the others like like Alibaba so it's it's a particularly uh it's a particularly talented team but the General Dynamic that we've seen uh uh is I think to be expected um particularly in the tens of thousands of chips smuggling is very possible I you know I I obviously can't speak for either the last Administration or the current one I think this is a bipartisan thing but I think the purpose of the export controls is to prevent China and other adversaries from getting to the hundreds of thousands of chips the millions of chips because that's really hard to smuggle and that's really hard to replace with Huawei chips so so there's there's this bifurcation in front of us and you know we can have we can have a big lead or you know as you're as you're suggesting one of these other routes um uh you know things things can be parody and those are two very different futures for the world I guess my argument is my question is can you really have such a big lead when maybe you expected them to be this good but I think a lot of people didn't a lot of people even in your industry have been pretty blindsided by how good these new Chinese models are and it leads me to wonder whether actually the half life of export controls is not a lot shorter than you think and even if you have export controls the Chinese innovate around them and they come up with other ways to fix things so the premise that you can through export controls ensure that the US is dominant in Cutting Edge chip provision do you want to base your entire policy on that so I I I actually believe this is possible I so first of all um it it you know this may or may not be something that works this may or may not be something that's well implemented um but I do think that if it is well implemented it it still has a fairly good chance of working and it is something that we should try uh because because because of this critical period right I I think I maybe didn't come earlier in this interview but in other interviews I've said 2026 2027 is the critical window and if you're ahead then uh the models start getting better than humans at everything including AI design including making making you know using AI to make better AI including making AI to you know using AI to make all kinds of intelligence and defense um Technologies so so uh you know I I I think this is I think this is pretty important and again I don't believe the export controls were were were designed to fundamentally change the chip allocation in 2024 or even in the first half of 2025 like again deep seeks cluster same order of magnitude as the cluster of the frontier companies in the US I don't think that could ever have been prevented interesting so 2026 2027 is when you effectively get to AGI across the board and it's the threshold moment whoever's ahead then is ahead forever is that what you're saying potentially I mean we don't know these things but but there there's there's a risk of this that's happening at a particularly uncertain time geopolitically but we'll get to that I've heard those same years you know floated as no I'm I'm I'm aware of how frightening this is so you've made the case for export controls we'll come to that in a minute you've made the case for export controls what else does the US need to do to ensure that it is ahead in 26 27 yes so energy provision within the US I think that's important um you know making sure that we're able to provide enough to the US and to its allies in Europe as well um uh you know making sure that we have a a coalition of democracies that are able to provide enough compute um there's another issue that's kind of the mirror image of this which is you know for all our concern about authoritarians and I'm I'm very worried about it because when when AI is so powerful there's the opportunity for authoritarians to abuse them um uh you know many limits constraints on the power of authoritarians comes from the fact that they need the consent of humans of followers to do what they want um if we start to automate this then very dark Things become possible uh so that's very important but at the same time if we have these models that are better than humans at everything that are like a country of geniuses in the data center we also need to worry about the dangers of the models that we ourselves are building um are they are they dangerous in an autonomous sense um Can people misuse them and so that's why we been Advocates of testing and measurement for AI systems so this was done by the US aisi the UK aisi it's all been voluntary so far which I think is the right approach because we don't know for sure how to run these tests and the test have focused on National Security risks like bioweapon risks cyber risks emergent autonomy of the AI systems and these tests have actually been helpful to us in some cases they've even helped us do a better job of deploying our commercial product because we understand better what the models are capable of for example in areas like cyber uh uh so we've been an advocate of keeping this kind of testing around uh and actually we found some bipartisan support for this um for example when we talk in Congress the idea of measuring models for National Security risks make a lot of sense to people I think people have the impression that this kind of testing maybe it's the name safety Institute is is kind of related to Dei issues or bias that's a misconception um that's not the kind of testing that's being done at the AIS is is this Administration key on it I mean the the the president and correct me if I'm wrong has rescinded the Biden administration's executive order on AI correct one what impact does that have and isn't this part isn't there some element of the the the aisi is separate from the executive order um the executive order focused on reporting requirements for models that were above a certain size I think those reporting requirements were were good but they were really small potatoes so there's a lot of discussion of the executive order it has this kind of totemic significance right as something Biden did and Trump is ripping up but actually there wasn't there wasn't that much to it there wasn't that much in it um I I I don't think it's actually the important thing and and you know generally I generally I I kind of advise that we focus on the important things and and kind of not the things that kind of gather this this polarized totemic significance so we're going to come to to safety a bit later because this is something you known for but just before we come to that can we talk a bit about where you are seeing the fastest adoption what kind of Industries what kind of companies because the impact that all of this will have for all of us depends on how quickly this is adopted how quickly it diffuses throughout economies and what are you seeing there yeah but uh uh without hands down the fastest area of adoption has been coding um uh multiple Hundred Year hundred million dollars of Revenue have materialized overnight over over the last three months um uh uh three months over the last three months or so um uh anthropics Revenue in general has grown from roughly 100 million to roughly a billion over the last 12 months uh uh but and you know Co coding is not the majority of it but it's the largest single area uh uh and uh there's many companies there's cursor there's codium GitHub co-pilot actually which is uh GitHub is owned by Microsoft they were actually willing to go through the step of of using am of using Amazon's cloud in order in order to Ser in in order to get access to our model um there are other others like versel augment um uh um uh cognition uh several several others and so this is happening very fast and you know coders across the industry are using it and the quality of coding is only going to get better so I think that explosion is only going to inflect upwards and with things like the virtual collaborator it's going to extend to other areas I think the growth we're seeing in coding that growth is going to happen in in every area and and and that's the way that that this industry is going to get relatively soon to you know 10 billion hundred billion dollar Revenue amounts um you have to imagine if you take Serious the you know country of geniuses in a data center better than humans at everything in two or three years well what are the commercial consequences of that somehow the the you know that that that doesn't make any sense unless the revenue amounts get up and I think they will for multiple companies within 2 or 3 years that sounds like a crazy prediction but I I don't know I everything else I've said is maybe even crazier presumably also for the nvidias of this world too uh yeah I mean you know I I I think there's multiple problems with me telling you whether to sell or buy a stock um but uh um uh but you know I I don't think it's a secret your hand is giving away the direction no no no intended to none um uh uh uh you know but but I it doesn't take a genius to see that probably Nvidia has a bright future um the only way in which uh in which they might not have a bright future is you know that you know of course there there there are other chips being being being developed by other players um and tell me about what kinds of companies are going to kind of reinvent themselves fast enough to be able to take advantage of all of this because I was in a really interesting conversation yesterday actually and and you know one argument is Big Enterprises have the capital the scale they can kind of you know afford to spend all of this money and then there's another argument which actually startups may come from nowhere because they'll reinvent the whole nature of what a firm is you may not in this world need an HR department you may not need an accounting department you may completely reinvent what the firm is and so startups will be much faster what do you think so we are seeing the following the Dynamics are very interesting um uh spendings from startups uh uh uh it has grown much faster than spendings from Enterprises there are startups that spend you know 50 50 million annualized or you know or or or or or or or even more of that we've we've seen that it's very hard for Enterprise and they and and they get to that level over a few months it's very hard for Enterprises to get to that level that quickly however the potential for Enterprises to spend given the value of the technology is much higher so you have this smaller pool that's spiking very quickly and this larger pool that's starting to spike but is is doing so slowly I think the Enterprises will ultimately overtake the startups um uh and will spend much more than they do but that's not the only possible outcome there's another world in which the Enterprises can't adapt fast enough and and we have a kind of massive industrial disruption of something we've never seen before where across Industries just just all the incumbents are knocked down one by one by these Innovative startups that can adopt AI much faster so um uh either either is either is possible they kind of both lead to the same world but they they look different while they're happening and what kind of time frame do you expect this in I mean if you have you know the Geniuses in a data center by 26 27 over what kind of time Horizon is every single industry completely sort of overhauled by this yeah so one of the things I dealt with you know I wrote this essay Machines of Loving Grace uh and one of the key questions it asked I mean it was focused on what are all the positive applications of AI right I've talked about a lot of you know risks and dangers here but what are the positive applications and the key question it asked is look the positive applications are going to be bounded by by The Human Side of things right I use this term called marginal returns to intelligence right economists talk about marginal returns to labor to land to Capital you don't usually think of intelligence as I mean it's connected to labor but uh you don't you don't think of it as as you know an asset all so what happens when you max out intelligence but but they're still the physical world they're still people there's still laws there's still Logistics what goes fast and what goes slow it's related to this Enterprise versus a startup question actually um uh cuz you know Enterprises are human institutions that do things a certain way have existing customer bases and and the general thing that that that that I concluded is kind of not surprisingly it sounds almost obvious but but things will go fast in in areas where markets function well um uh and in areas where things can be done digitally rather than physically I'm extremely optimistic like insanely optimistic about biology and health um this is an area that so so so I used to be I used to be a biologist I did my PhD in biophysics and computational Neuroscience um you know I worked on uh uh uh understanding real neural networks and brains as opposed to the artificial ones that that I build now um uh uh uh and uh then I did a postto in computational proteomics where I looked for cancer biomarkers uh uh and knowing what I know about these areas I think AI is just getting to the point where we're going to see some really transformative stuff are I mean you said sorry I didn't mean to B in but in the in your essay which I really recommend you read it's a really interesting essay you you said and this stuck with me that you thought AI could increase the rate of progress in biology by 10x so we could get 50 to 100 years of progress in 5 to 10 years yes that was my view and and just to be clear you know this 10x sounds like one of these kind of hypey things that like what does it even mean so so um uh I actually think that that intrinsically AI systems do things very very fast there's very few limits to it the question I'm asking is like like total factor of productivity as an economist would say like you know look at all the drugs we develop over the next you know F that we were going to develop over the next hundred years how quickly can we get them so include everything include the clinical trial system which will get faster in some ways because we can replace late stage trials with early stage trials and early St trials with animal trials and animal trials with you know cell you know with experiments on cells and cells with simulations um but you know it'll still it'll still take a while still take a number of number of iterations the AIS will as time goes on and there are more of them and they get smarter they'll find ways around it they'll say actually you don't need to run that experiment like I can figure it out by thinking in my head that other experiment nope too complex you still got to run it what is it what does it all add up to where where do we get to at the end of it um and and my view was that something like 10x is feasible something like 100x um is not out outside the realm of possibility but I'm skeptical um I think 2x is possible but would be in my opinion a pessimistic assessment uh of how fast things are going and then if I think about well what would we have so let's say 10x is right what would we have done in 100 years I think in a hundred years without AI you know uh humans humans have a lot of have a lot of Ingenuity I have a lot of respect for the basic biologists the folks who work in Pharma I think in a 100 years humans can you know live to maybe 140 instead of 70 or 80 um and so the prediction that makes is you know maybe that will be possible if we build our powerful AI in 2027 then in in you know 10 years by 2037 will humans be living to 140 150 um and you know I I I I i' bet i' bet someone even odds at that so I I want to open up to questions but just before I do I just want to kind of take stock of what you've heard here um because at some point thing I said was most crazy well no I I you know the first time I had these conversations with with um people like you I just thought like what planet are these people on but the last very self-conscious about very but actually the last couple of years the pace at which this technology has advanced has just been mind-blowing and so you know when you start saying you know we're going to have 10x biological Innovation by 2027 2027 we're going to have you know models that are better than humans at everything I'm kind of inclined to believe it actually now and so I just listen to this and and 2027 just is two three years away um this is what you've just heard here so with that in mind um and i' I've got more questions for you in a minute but I want to give a chance to other people to ask questions um I promised you that this was one of the most uh important and interesting people in this field Dario absolutely is so who has a question for him Martin go ahead okay um why the hell do we want humans to live to 140 it's unbearable already but anyway the serious question is this let's suppose you were standing in front of a of a group of parents in a school their children are 10 to 18 my daughter's in this situation now and they ask the question okay this is the world in which all the thinking is being done by m m machines what on Earth should we try to educate these pieces of wet where how and your first answer obviously is well you just made a big mistake you're making a colossal investment in a completely out moded piece of Kit scrap it and then they say well that's true I'm sure but we love our children and at that stage what do you say yes so I I I I talk a little bit about this in in the essay and and you know this is one of the risks I didn't uh uh you know uh didn't didn't uh touch on earlier but I think it's going to be an increasingly important one right we've often worked on these Safety and Security risks and this issue of competition with with China but I think just as these powerful AI bring power into the world um they also bring economic power into the world as compared to the economic power that that we all have as individuals um and so this issue of Labor displacement and ultimately meaning for humans is going to be really important um uh I think there's a short-term and a long-term component of that in the short term we're going to have Labor displacement that is large in magnitude but looks like the kind of Labor displacement we've had with previous Technologies like the internet like much of industrialization um you know I think that's that that always has some pain to it but uh you know we are thinking a lot about how to design our systems so that they're as complimentary as possible so that it's easiest for people to adapt to using to using AI again this is within the Paradigm of AI can do some things humans can do and humans need to adapt and do different things I think comparative advantage is very powerful even if the AI does 95% of what you do then the other 5% you know really expands to be useful and so I think the short term will be a little longer than people expect then there's like what is a little longer than I mean given that we're going to have the world right could be six months yeah yeah you know could be four years instead of two or you know I don't know short ter I guess shortterm and long term have taken on weird meanings in my in my world um but then there's a longterm which I think you were maybe a little more asking about and and there we I think we are facing a world where uh we're building systems that are as are more intelligent than we are uh along almost all axes right we still have a physical presence although you know I think I think in the long run um you know a AI will will will you know will propagate to to robotics as well so we have to confront this you know this kind of deep philosophical question of like our our intelligence is the thing that you know we value you know so much you know I think most of the people in this you know in the room you know put a lot of their selfworth and and value in in in their intelligence um and you know this is something that we're learning to create and copy freely uh and so we're just going to have to confront what that means right we've had our kind of socioeconomic contract for the last you know few hundred years that you know there's been problems with it but it's worked relatively well which is you know we all we all contribute value with with you know mentally and physically we're able to we're able to grow the economy we get compensated for it we feel a sense of worth for it all of those assumptions are going to break down uh and and we're it's almost like we're going to need a new social contract that uh uh you know almost a new culture that that kind of explains why why are humans why are humans valuable why is why what what meaning does our life can I just make one are we still I me you said you know sort of throw away we're going to have a physical presence I just want to make sure that is still true we are going I mean you know if you read science fiction novels there's all this discussion of like you know uploading people into the cloud and things like you know I that's a little bit far out even for me um uh uh I I I you know I don't think it's necessarily I don't think it's necessarily impossible but I do think probably will'll be in the physical world for the for the foreseeable future um but we'll be like the what the kind of you know pretty things around there for the intelligence I there's several there's several answers here right like one is that um we can you know we can we can use AI to kind of enhance ourselves and keep up that's one possible path another possible path is that human meaning is is you know it suddenly it's not about generating economic value or like making something unique um I actually think human meaning I mean you know you could talk about this question for you know hours days I think it's connected to two things I think it's connected to relationships to other humans and I think it's connected to you know uh uh a projects and goals over a long period of time so striving for something right um uh having some kind of long-term project where it's not just a game the stakes are real you're trying to do something important live for something important I think that's still possible in a world where the AIS are better at everything I think there's nothing about that that fact nothing about either of those two things that that AI detract from in fact I think they could even enhance it what that looks like I I I don't think any of us know and and my main anxiety is like I you know I'm probably not the best person to you know to give a full answer to that question like it has to come from all of us um maybe the only place that people like me can help is to prompt us to start thinking along these lines um because right now there are very few people thinking along these lines and the technology is coming at us like a speeding trade and that that makes me nervous because if we have zero time if when we react we have zero time to react our reaction is guaranteed not to be good it's it's guaranteed to be react just just knee-jerk crazy just the worst impulses you could possibly imagine and so we need we we we we we need to start thinking about this now so that we have a chance of rising to the occasion right of of reacting in a way that reflects our best selves um I understand with everything going on in the world you know that that may seem idealistic but we've got a couple of years it sounds like I I look I'm I'm just I'm I'm I'm I'm I'm just the one telling you what I think is going to happen like no you're you're blowing certainly me away um yes gentleman at the back and then lady in the front middle and then I think that probably be all we have time for thank you so much for this fascinating talk um the trajectory of progress in the physical sciences seems extremely clear in the social sciences or in complex human systems for example in policy design could you please give us your assessment of the trajectory of progress in those domains domains involving complex human choices in interaction and relatedly Demis has recently argued that it may be the case as a hypothesis that the so-called unique advantages of quantum Computing are not necessary for progress in such domains thanks um yes so I don't know if this is a question about how how much AI will drive the progress but I guess I'll I guess I'll answer it in that in that in that assumption so definitely I think the physical sciences will go will go faster right anything that is objective anything that there's a skill you you can do that you know if I think about um you know political or or geopolitical or social science expertise you know on which I on which I don't have any expertise um you know these these things seem like they often have long cycle times um they're also based on intuition I think AI will gain increasing intuition um and probably will surpass humans in the intuition uh but but also just gaining the experience to gather that intuition will take will take time and you know many of these decisions are like you know very kind of I don't want to say oneof but it's like you know if I if I think about you know go back to the last seven recessions right try to make a macroeconomic theory about what caused them so many things were different in those recessions you can't you can't make a um you know you you can't do a controlled experiment right it was like you know stagflation in the 70s versus the what the inflation that happened two years ago so many things were different about the situations and yet what humans do and what I think AIS will do as well they look at the commonalities they dive into the data they use their common sense to come up with ah here's a simple model right here's the you know here's the you know the the you know Keynesian theory of recessions or whatever here's the you know here's here's the other thing um and and so I think AI there's they will get as good and eventually better at us than doing that but there are limits to what knowledge is possible because because we're missing some data that's this is the marginal returns to intelligence at some point there's only so much you can predict I think that I suspect that ceiling is well above humans right our economic theories have gotten have gotten you know better for for you know for for for all that you know economists are often wrong about things the question is how high above humans is it um and and I don't know my my guess is that there will be a lot that's just fundamentally unpredictable and we look at it and we're like the economy did this crazy thing over the last year we don't understand it's some emergent phenomenon maybe will'll never understand like you know I think there there's going to be these these things that are lost to complexity and mystery of the universe even to the superintelligent AIS the lady there yes go for it thanks Danny for the opportunity and um um I agree with you Dario on the you know how the AI can revolutionize the Pharm you know the medicine field Amazon also is trying to create like a AI for his pharmaceutical company and they actually have collaborations right now do um but I want to Pivot to a different um question since I want to have a little question yeah so like um people used to go to trying to go to IV League schools like you went to Stanford and ctech but open Ai and also like Cloud AI is a really good teacher for students do you think that will break the polarizations for um the education in the future where people no longer has to go to Ivy League to become a good entrepreneur and be successful in their career yeah I mean I think education is one of these these interesting areas right and you know the model's getting again there's the short term and the long term for it you know the model's getting so much smarter we we maybe should start thinking of Education something as more as as a thing of you know self achievement right of you know try trying to become a human being who understands things better who has more capabilities who's able to do things that uh uh you know that that that you couldn't that you couldn't that you couldn't do before and you know for that uh you know I think AI will certainly help us a lot um I I do think learning is a social activity and and ultimately the social aspect of it and component of it will always be important so I think there will be an AI aspect of it and a social aspect of it um how those two come together whether it's at a univers or some other structure I don't know so I know there's tons more questions but Dar I know you have to go I'm going to end but there's just two specific questions I want to ask you and one is you're known as someone who has focused more than many of your fellow leaders in this field on the risks you've mentioned it a fair amount today um but in your essay Machines of Loving Grace as you say you focused on the upside have you become over the last year more confident in your certainty about the upside or about the risks uh I think I think I've increased my confidence in both because I've increased my assessment that the technology is going to happen um uh and and you know that that kind of push pushes on pushes on both um I don't think the balance between them for me in a in a factual objective sense has changed mostly I mostly I don't know I mean there's you know some world where none of these risks really really happen they're you know they're kind of they're kind of theoretical in theory someone could do X and someone could do why but maybe it doesn't work out that way there's another world where the benefits are slower than we think because you know these regulatory barriers like I think when I wrote my essay Tyler Cowen said I don't think it'll be 10 years I think it I I believe you that that then this timeline but I think you know regulatory barriers are so slow that it'll take a hundred years right so he has this libertarian perspective that like regulations just remind of him what you think the biggest risks are because I think we didn't Del well first of all first of all let me let me let me finish this thing which is which is factually my view not changed like like character logically and attitudinally it has changed a lot um I used to think that markets provide benefits and so it's important to think about risks um I now actually think that like we we we actually need to inspire people like truthfully you know we shouldn't make things up but we need to inspire people with the benefits that are possible because the benefits are so awesome and and if we can actually find a way to navigate all of the problems including you know the economics and the human meaning as well as the security and geopolitics risk if if we can somehow find our way through the obstacle course the thing on the other side is really beautiful it it it it truly it truly is and uh actually maintaining that Vision I think is is extremely important uh to finding our way through to getting to getting the good outcome um and so I think I think I've had a big change there not in terms of the facts on the ground but but but just just just in terms of the the the attitude with which you have to approach these things and lastly is this now inevitable is what you've laid out definitely going to happen can anything derail this yeah so um I I still think there's uncertainty less than there was before there's maybe two kinds one is a fundamental technical uncertainty so a thing I'll say is I would give a five maybe 10% chance that we you know that we get to 2100 uh uh and none of these things that I've I've I've described uh happen what's going on there well I don't know you can just be you can just be wrong about things you can just be seeing the situation completely differently from how it is I think that's still possible I have a list of how you know what what what could block it and and you know I've been Crossing items off that list it's getting shorter and shorter but it's not zero and you can you can always be wrong and you know maybe in you know maybe in you know 20 2050 when I'm you know when I'm old and gray you know I can laugh along with everyone else about all the crazy stuff what's still on the list what could block this um I think we could be wrong about the last part of scaling I think uh in reinforcement learning there could be a lot of things that are harder to get right than we think there could be something we're missing algorithmically particularly on tasks that are hard to evaluate um again not likely but I think it uh you know it's possible um the other thing and this wouldn't take us to 2100 but but if you're like what's the most likely reason this doesn't happen in 2026 2027 but gets moved out a few years uh some kind of geopolitical Crisis um uh the chips that power all this are are located in in maybe not the most stable region in the world um uh so that's one thing that could could really derail things and we may not be in the most geopolitically stable time right now uh yeah the the intersection it's not my world but the intersection between that and all that we're seeing here um uh uh gives me gives me great concern well we are going to go on to some it it you said it not me it is a perfect storm we have the fastest and most dramatic technological change I think in human history at the same time as we have a geopolitical shift that we haven't seen in at least a 100 years you said it was a perfect storm not me uh I mean I hope it's not but but this this is what we have to we have to fight against all all of these things converging in a bad way and somehow flip them around and make them converge in a good way well I hope you focus on that dar day thank you very very much thank you for having me