Vitalik Buterin vs Beff Jezos: AI Acceleration Debate (E/acc vs D/acc)

Transcript

Rapid technological acceleration has been a fact of human civilization for about a century and that acceleration is itself accelerating. >> To me that is the fundamental truth right and whether we yell at it or disagree with it is happening. You know it's like gravity. Those that adopt that culture will literally have higher likelihood of surviving in the future. If you take any one bit and you kind of accelerate indiscriminately then basically you do lose all value. And so to me the question is like how do we accelerate intentionally? I think there is a real sense in which we have one shot at this.

>> EAC isn't trying to kill everyone. It's actually trying to save everyone. If we decelerate we're going to have huge opportunity costs and we're going to miss out on a much better future. Oh, nice. Wow. So, this all started because I just knew these guys had to meet each other and uh it rapidly devolved into all of this, which I'm really glad to see. This is incredible. Um, and it's it's the first time that you guys have really talked in person, right? Awesome. And I I this is an incredible synthesis. Uh, so yeah, my name is Shaw. Uh, I've known these guys for a while. I'm here with Eddie from A16Z Crypto. Um, and this is a great

time. So, everybody's here. Uh, I guess, uh, you're allowed to, you know, please be respectful. Uh, this is a conversation between them. We're we're just going to kind of throw some questions at them as we go along to keep it going, but feel free to dig into whatever you guys want to. This is really here for you. We're all just here to listen. Um, and, uh, this will all be live streamed to the other floor. It's not going to be public. Uh we will be cutting up the video and putting it out later so everyone will get to see and share and everything. And I think without further ado, uh I'm gonna leave it to Eddie to get started with some of the questions.

>> So, so before we ask them, I I'd love to get a sense of the the the crowd. You It's always hard to tell the difference between the Twitter timeline and reality. You know, who here could explain EAC in a few sentences to someone else? Wow. Okay, that's that's actually less than I thought. That's good to know. That's good to know. Who here could explain DAK in a few sentences to someone else? >> Okay, that might have been more actually. Okay, that was very interesting. Okay, thank you for that. So maybe we'll maybe we'll just start there, right? is uh the term uh accelerationism at least in the techno capitalist sense dates back to Nick Lan's CCRU research group in the '90s but some might say

that these ideas really took took shape even further back the 60s and 70s with Doo and Qatari let me maybe start with Vitalic why are we having an earnest conversation about the ideas of philosophers right now what makes this accelerationism idea relevant. Again, I think ultimately, you know, all of ultimately I think, you know, we're all here trying to make sense of the world and trying to make sense of like what it even makes sense to do in the world, right? And this is uh something that we've had for thousands of years. I think the new thing that we've had for

probably roughly a hundred years is uh making sense of a world that has rapid change and sometimes even that has uh I mean maybe this is skipping a bit ahead but like rapid destructive change right so uh you know like the early the early era of this is that there was a in the preWorld War I era around the 1900s there was a lot of uh like original techno optimist sentiments, right? And there was a lot of excitement back then. Well, you know, the thing that we call today tech, back then chemistry was tech. Um, and then electricity was also tech. Um, and uh, you know, if you even, you know, like watch like even movies

like some of the Sherlock Holmes ones, you get to like really feel the vibe of that kind of era, right? And uh it was rapidly improving living standards uh rapidly uh liberating women in the household doing uh amazing uh things extending lives uh and then of course uh you know World War I happened right and you know World War I famously you know people rode in with horses and rode out on tanks right and uh it was a destructive war. Then World War II came and uh World War II was an even more destructive war and you know it gave birth to I am become death destroyer of worlds and uh this is like some of the background of uh you know things like postmodernism and uh people basically

trying to make sense of like okay like a lot of beliefs were shattered and what we what do we believe now right and this is something that I think people believe like every generation right and uh there's a a lot of people today who even grew up you know like believing in kind of 1960s era postmodern beliefs I mean even and feeling like those beliefs have been shattered right and even people who for example uh grew up believing in uh you know what like I would call hipster environmentalism and it's this like lovely beautiful uh idea and you know we need to uh protect the environment and not go so fast and then you believe in this and then you realize is that like

wait the nuclear power plants that you advocated to shut down basically means that you know your country is like stuck bootlicking Russia right and uh like okay basically all and like these are just uh like very very natural uh things that happen right and uh I think this like rapid technological acceleration has been a fact of human civilization for about a century And uh that acceleration is uh itself accelerating and you know things like postmodernism are a response to that. A lot of the currents of the 1960s were a response to that and uh you know you can respond by

saying it's inevitable. You can respond by saying uh we we have to slow it down as uh a lot of people did. uh we and uh it's just constantly like a rapid uh response to basically the effects of the uh ideas that were tried to be executed by previous generations. And I think you know we we're now quite rapidly seeing a new version of uh that exact same cycle continue today. And I think it's uh mixing both uh themes that have been around for a long time together with some pretty new ideas. All right. So, so Gil, so what is EAC and and why?

>> Why? >> Um, yeah, I guess EAC is kind of the byproduct of myself u asking why are we here or how are we here? What what was the generative process that gave rise to us that gave rise to civilization technology got us to this point where we're having this conversation in this room. We all have wonderful technology around us and we emerged from a soup of uh inorganic matter. So somehow there is a physical generative process and my day job is trying to do generative AI as a physical process in devices. And so that was simmering in my brain and I wanted to apply that sort of uh thinking that sort of framework that uh physics first

viewpoint to all of civilization trying to understand civilization as a petri dish trying to understand how we got here in order to predict where we're going. And uh you know that uh got me down the rabbit hole of the physics of life itself like emergence of life a biogenesis and uh a field of physics called stochastic thermodynamics which is the the thermodynamics of out of equilibrium systems. So what describes uh life forms and also including our brains right intelligence. So it's it's it's both the physics of life and intelligence but it's also the physics of any system that obeys the second law of thermodynamics. uh which includes our whole civilization. And so really to me it's just been an

observation that systems tend to self- adapt and complexify in order to capture work from their environment and dissipate heat. Uh and that is the fundamental driving force behind all of progress, all of quote unquote acceleration, all of everything we see today. And to me that is the fundamental truth, right? And whether we yell at it or disagree with it, it is happening. This is, you know, it's like gravity. It's uh you can argue with thermodynamics, it doesn't care. It keeps going. Uh and so, you know, to me, EA was like, okay, well, given this fact and given that if you miss if you look

at the equations carefully, you can observe that there's a Darwinian like selection effect for every bit of information prescribing configurations of matter. So whether that's a gene, a meme uh chemical specification, uh product design, a policy, there's a selective pressure on everything and everything is interoupled in this big soup of matter. And that selection pressure is selects bits according to whether they're useful for the system they're part of. They're useful to better predict the environment, capture work, and dissipate more heat. So are they useful for uh sustenance, for um um sustaining yourself, preserving

yourself, predicting your environment, predicting danger, but also is it useful for for growth? Because if you grow and replicate, then those bits of information replicate and there's a natural error correction. So in a way, it's a just a byproduct of the selfish bit principle that emerges from physics. And what that tells us is that the bits that are part of the future are the bits that are useful for growth and further acceleration of this growth. And so if to me I wanted to design a culture that if we bootloaded this mental software in the population, those that adopt that culture will literally have higher fitness. They will literally have higher

likelihood of surviving in the future. So EA isn't trying to kill everyone. It's actually trying to save everyone. It's basically to me I think mathematically provably having a decelerative mindset and it's a general pattern of of many subcultures of making yourself small degrowth and so on. uh it's actually negative. It it gives you negative fitness and you're actually accelerating your downfall as an organism whether it's del mindset at at an organization level at a company at a national level at an individual level you're lowering your likelihood of being part of the future. And to me that is is not necessarily virtuous uh to spread that memes to spread sort of pessimism

dumerism. It's actually >> I well >> well we're using a lot of terminology that I haven't quite unpacked yet. So like like EAC what does that what does that stand for? What's what is acceleration? Oh, >> and and what is deceleration and what is a del and these like what I'm trying to get at is I think >> EAC came as a little bit of a response to something that was happening in our culture at the time. >> Yeah. Yeah. We got >> what what was happening in our culture? What was it a response to and what was what's say a little bit about the dialogue totally that led to encapsulating it in a in a name?

>> So, you know, it was 2022. I think the world was somewhat pessimistic. we're just emerging from COVID. Things weren't looking good. We were feeling down. Everybody was kind of lacking sunlight. Everybody was sort of, you know, pessimistic about the future and and and and essentially um yeah, AI dumerism was kind of the monoculture. >> What is that? What is AI? AI dumerism is just kind of um you know uh panicking that about you know the fact that if there's a a system that is too complex our brains or human brains our generative models can't have a predictive model of them and so we can't control them and things we can't control g give us entropy about our model but of the future and that induces anxiety

right and then AI dumerism to me has been a weaponization of people's anxieties for political purposes. Um, and overall I think like uh and we'll get to this you know uh I I I think AI dumerism is a is a big net negative and I wanted to create a counterculture to that. Now what I saw in the X algorithm is that uh you know the X algorithm u and many algorithms reward agreement or strong disagreement right so there's you know if you view the algorithm as a markoff chain asmtotically everything converges to bipolar distributions of opinions for anything so it's like you know you had the the AA EA mirie cult uh complex uh you know I I kind of

clustered them there maybe not so gracefully but you have that complex I was like what's going to be the the the opposite of that and you know to me I was like okay well you know the opposite of anxiety is is curiosity right instead of downside protection it's upside seeking uh uh you know fear of missing out it's like you know um dopanergic sort of mindset and it's like hey actually if we if we decelerate we're going to have huge opportunity costs and we're going to miss out on a much better future and it's just like painting that future more vividly and bootload loading this mindset of optimism because the thesis is that you know if you study neuroscience we tend to want to have a convergence of our beliefs uh and the world and so sometimes we adjust our

beliefs to the state of the world but we also adjust the world to our beliefs. So if we believe that the state of the world will be bad then we tend to steer the world to that bad outcome. If we think the world will be great and we think of positive futures we tend to hyperstition them. we we tend to increase the likelihood of their advent. And so I had a responsibility to spread uh sort of optimism in order to hyperstition a positive future. And yes, I you know online I am very you know aggressive and you know use all the political mind hacks because you know to me the end justifies the means like if more people uh are optimistic about the

future feel like they have agency feel like they can build and and and and make an impact in the world then that's really good and I think you know sometimes I'm a bit ruthless with my opponents on the other side of the aisle. Uh I think in private meetings I'm much more friendly but you know for you know like I said I I just took the extreme opposite to the current monoculture and then that created some polarity and then now we can have discussions of where we want to lie right so >> so I've been with EA since the beginning and and it's been a message that as a programmer sitting in a room has been incredibly inspiring and and it's great to see a positive message spread and it spread very organically and I would say

that at the time that it started it was clearly a reaction to this negativity but now in 2026 I feel like it feels like EA1 it feels like that's no longer the case and I think obviously Mark and post the technoptist manifesto which I think really kind of codifies some of those ideas and then brings that to like where Vitalic sort of has this this greater commentary. So I'd kind of love to know from you Vitalik like what is EAC in your mind and what is DACK and what makes them different >> like what what what drove you to to go this direction. >> Yeah I mean I think uh maybe yeah I'll also start my uh my answer with thermodynamics right because why not? Uh so I mean this is I mean it's an interesting topic right because like I think we hear about like entropy in the

context of hot and cold and we hear about entropy in the context of cryptography and these are like different universes and like actually we're not really taught how they're actually the exact same thing right so I'm going to try actually and like explain this in three minutes so okay so the prompt is why why is it possible to mix hot and cold but why can't you separate things into into hot and cold right and so here's my explanation, right? So, imagine you have two jars of gas. Each jar of gas has a million atoms in it, right?

And because it's the move slowly and so the velocity of every atom can you can represent with a two-digit number. Right? Over here, the atoms are hot. The velocity of every atom you can represent with a six-digit number. Right? Now, how many digits do you need to represent or or rather if that's what you know, how many digits of information do you not know about the system? The answer is 8 million, right? You don't know the the exact velocities here, 2 * a million. You don't know the exact velocities here. That's 8 time a million, right? Now, what happens if you mix them, right? Well, if you mix them, the velocities get averaged. And so, they become numbers from 0 to 500,000. And so, 5.7 digits, actually pretty close to six, right? And so you mix them, you

have two jars and uh on one side you have a jar where the amount of information you don't know is 5.7 million and then over here 5.7 million. Right? And so the amounts that you do not know about the gas has gone up from 8 million digits to 11.4 million digits. Right? So the amount that you do not know is increased. Right? This is what it means by entropy go up. Now we can like try a proof by contradiction. Imagine you had a device that goes the other way. Right? Imagine you had a device that can take two jars of this like half half hot gas and like actually bring all the heat over here and all the and all the cold over here. By conservation of energy, this is totally valid, right? Because it's like the same

energy. But why can't you do it? And the answer is well, what you've if you could, then what you've done is you've taken this system where what you don't know is 11.4 million digits and then you've turned it into a system where what you don't know is 8 million digits. Right? Now, because the laws of physics are time reversible, this is like the important thing, right? What that implies is if that kind of magic device existed, then actually like you could run the same process in time reverse and so you could always recover the original, right? And so what that implies is if that gadget existed, it would also be a gadget for compressing an arbitrary 11.4 million digits into 8 million digits, which we know is impossible. Now, but this also, by the

way, tells you why Maxwell's demon works, right? Which is basically that if you had a magic demon, then actually yes, you can split the hot and the cold. And basically the Maxwell's demon just has to like know the extra 3.4 million digits separately and then you're fine, right? So like what's the moral of this, right? Basically that increasing entropy is basically mean like one entropy is subjective, right? Entropy is not like a physical statistic. It's actually how much you don't know, right? And uh you know if uh I like it turns out that like I actually computed a cryptographic hash function and I pushed out the uh the atoms then like actually based off of that for me this like the bottle might be very low entropy right and like maybe I could separate it right but ultimately

it also means that when entropy goes up it means that our ignorance about the world goes up. It means that what we do not know goes up right you can go from knowing more to knowing less. You cannot know in you cannot know go from knowing less to knowing more. Now but then why does education exist? Why do we become smarter? And the answer is that we go from knowing um fewer like we basically we go from knowing more things that are useful. Right? Basically, yeah, ent the increase in entropy means that we constantly know in some sense less and less about the universe, but the bits that we do know are more meaningful to us, right? And so there is like a thing

that is being spent and then there is a thing that we are gaining and the thing that we are gaining this is uh like I I don't think that there is some like simple mathematical formula that defines it. the thing that we are gaining. I mean ultimately this is basically our morality, right? This is that you know we value life. We value happiness. We value joy. We there's a lot of different reasons why we find an earth full of thriving beautiful humans more interesting than Jupiter even though like Jupiter has a larger number of particles inside it and you and you need more digits to express how like what what each and every one of them is doing, right? And so I think like value comes from us is the first thing. And I

think also like this connects to what we want out of acceleration, right? Which is basically that like our goals to me ultimately come from us, right? And so the question is like okay, we are accelerating, right? And uh what do we want to accelerate? And uh I mean if we want to like switch you know like mathematical analogies a bit right if you take any LLM and you imagine you randomly flip one of the weights to positive 9 billion what happens right worst case the LM becomes useless best case every weight that's not connected to the n to the 9 billion doesn't do anything right and so best case is you have an LM that's that's weaker worst case you just have junk and so basically

I see human society as kind of like an LLM. It's this complicated organism and if you take any one bit and you kind of accelerate indiscriminately then at like basically you do lose all value and so to me the question is like it's basically you know like it's what like Darren Asamu calls the narrow corridor even though like you know the uh like the details on the politics are different but it's like how do we accelerate intentionally? >> Can I jump off of that? Yeah. So um yeah that was an interesting way to describe entropy of a gas um to essentially um physics the reason physics is not reversible is because of the second law

of thermodynamics. It's because if you have a trajectory of a system and it dissipates heat it can't go back because the likelihood of going forwards versus backwards decays exponentially with how much heat you've dissipated. In a way, it's like literally how much of a dent have you put in the universe, right? A dent is an inelastic collision, right? If it if I have a bouncy ball, it's elastic. If I, you know, take some Play-Doh and smash it, then it just keeps the the smash shape. That's inelastic. It's hard to reverse. Um, essentially, every bit of information is fighting for its existence. And in order to persist, it needs to make more evidence of its existence that's indelible. So it's making a larger dent

in the universe. And that principle is how life and intelligence emerges from a soup of matter. And that complexification of systems becoming more and more complex, having more and more bits of information. A bit of information. It tells you information. Information is a reduction of entropy, right? It's a red. It's going entropy is lack of knowledge information reduces entropy about a system condition on information. >> No, I'm very sorry to interrupt. >> Where where did you want to take this? Yeah, >> I'd love to know what EAC is. >> Okay. Okay. Okay. So, so EAC ultimately it's a meta cultural prescription. So, it's not a a culture itself. It it tells you you should

>> what would you say? What is the thing that is accelerating? The thing that is accelerating is the complexification of matter such as we can so that we can predict our environment. We have better autogressive predictive power and we capture more free energy. So the cardv scale right and we cons and we displayed it as heat. But that is just the justification from first principles why the cardartesev scale is the ultimate metric for how well we're doing as a civilization. >> So allow me to bring it back one. So, so may this is a little bit selfish, but maybe I'm also helping the audience is that the metaphors and the explanation rooted in physics and in entropy and so on is in a way a explanatory tool to try

to get at a phenomenon that we experience directly and that experience is the acceleration of the productive capacity of our economy, the acceleration of the development of technology and the consequences therein. Right? That's my understanding of what acceleration is. >> Essentially, every system gets whatever its boundary is, it gets better at predicting the world and it by doing so it can secure more resources for its sustenance and its growth whether it's a company, whether it's, you know, individuals, nations, earth in general.

And you know what? If you if you just play the movie out, it means that now that we have a way to convert free energy into predictive power with int artificial intelligence, what that will lead to is an expan an ascent on the card scale. That's what the equations predict. And so that is and and that produce more energy, more artificial intelligence, more computing, more of these things. But but even though we are expelling entropy into the the the the universe, we are gaining order. We're actually gaining extropy.

So we're gaining the opposite of of of entropy. So we're um so sometimes people think like, oh yeah, cuz for more entropy, why don't you blow it all up? It's like, no. Well, then you would stop producing entropy. It's actually life is more optimal. Life is an energy seeking fire and it just gets smarter and smarter at finding pockets of energy. And the natural progression of things is we're going to get out of our local gravitational well and find other pockets of free energy and use them to self-organize into more and more sophisticated systems that are smarter and can you know expand to the stars.

And so that's kind of the you know that's kind of the ultimate goal of EA. It's kind of a formalization of like Elonian sort of mindset of you know cosmism and expansionism there. Um but it's it gives you a fundamental metric and then the prescription of EAC is follow the cardartesev gradient. So whatever policy or actions you can take in the world that maximize impact in our ascent to the on the Cardesev scale that's what you should do. That's how you should live your life. So it's a it's like a meta heristic for how to design a policy for how to live your life. And that to me is a culture. And so uh it's very meta because it's supposed to be true at all times. It it should have a very long shelf life. Yaks

is made to be a very lindy culture. >> Yeah. >> Well, it it's clear that like there there's a deeper thing that's going on here for you. Like this is almost this is like a mathematically compete complete spirituality that people who have been like really don't have like a we don't have like God is dead niche kind of thing like we're all living in that shadow. It's like something to make us feel good about. But I would also say that there's kind of a really practical on the ground like this is happening today which I think is where Eddie's trying to get at and and I think that like um Vitalik you did a great job of addressing a lot of the real practicalities in in your blogs on DAC and like if we can bring it like I I I need to lock you guys in a like with

whiteboards on some quantum stuff sometime but but for right now like I think you know this is an opportunity. Yeah. And and look this is a really I think that some like >> Eddie is not scared. is like we're this is going to be great but I I'm a little scared and I I come to you guys because you give me like hope and clarity and so bringing it back to you Vitalic like what what inspired you in this what is EA and what is DAC? >> Yeah. So I think uh for me um uh so DAK um so it stands for I mean I usually use like decentralized defensive acceleration but then there's also differential and democratic in there as well. Um but uh I think to me the core

ideas are like one is that you know technological acceleration has been amazing for human beings and it's something that we need to accomplish as a baseline right and uh even if you look at all of the crazy things and all of the worst downsides that technology did to us in the 20th century um if you look at like for example lifespans lifespans in like life expectancy in Germany in 1955 was higher than in 1935 and uh like basically we have just benefited from a massive step up in every thing that we care about and this is like something like I've even seen like even that you know observing the like my

you know my grandparents uh home like basically go up from uh having this like very like outhouse toilet in the backyard where there was flies buzzing and I would totally hate it and uh I'd have touh go out like I'd often go out to the forest to poop because I couldn't stand the flies to something that's like actually very modern and hospitable, right? And uh you know the world has become cleaner, the world has become more beautiful, the world has become more enjoyable, the world has become better for health, it's been able to sustain more of us. It's become more interesting and uh like these things are really uh good and uh beautiful for us.

Um at the same time I think uh you know we need to recognize the uh the role of explicit human intention in making a lot of those things happen right. So for example in the 1950s there was a lot of smog everywhere in the air and people decided smog is a problem. Smog sucks and we need to like do a bunch of stuff to get rid of the smog issue. And uh now smog is not a problem here at least much less of one. Um then you know we had the ozone informal issue and then we actually did things to address that right and then the other thing is uh you know that especially with rapidly accelerating technology and AI I basically see two

kinds of risks. Uh one kind of risk is uh multipolar risks which is uh basically the risk that uh you know lots of people will use the technology to do uh to do very bad things right and uh there's a concern that like one type of concern is sort of the equivalent of you know anyone being able to like make get a nuke at 7-Eleven like sort of thing and then there's also the concern of like well AI itself is uh you know like something that literally really is a mind of its own, right? And uh especially once it becomes powerful enough that it acts without human involvements, then you know like what will it do? And then there's bun and

polar risks which is basically I think actually a single AI itself is one of them and you know the other one is like I mean AI like create enabling or a com the combination of AI and other modern technologies enabling like permanent dictatorship that you cannot escape like that deeply worries me right like this is something I follow right and uh you know and like you know again in Russia for example like on the one hand the toilets gotten much better. On the other hand, like it's gone from protesting the impossible to protesting the sort of thing where if you do it, the cameras will see you and then, you know, a week later you get a knock on the door at 2 a.m., right? And uh this uh AI is

supercharging this. You know, there's a lot of uh a lot of concentration of power happen is happening and like both of these things really worry me, right? And like to me, DACK is really attempting to chart a path forward that continues this acceleration and accelerates it, but at the same time really deals with both kinds of risks. >> So So you would say that DAK is emphasizing specific other categories of risk that are maybe less emphasized than you'd like to see in EA. I think there's uh many kinds of uh risks of technology and uh all like many of them are valid and I mean they have different scales like some of them become more salient in uh different models of the world and how

fast things are happening. Um but uh I mean I think uh I think there's a lot that we can do to uh really uh like push against all of those kinds of risks, right? >> So So Gil, yeah, Gil, do you want to say a little bit? >> What was the question again? It was >> Oh, well just just uh compare and contrast EAC and DAC. Yeah, I I think actually Vitalik and I >> are very concerned about uh overconentration of power that can happen with AI and that was a big part of the EAC movement especially at the beginning it was pro- open-source we want to diffuse AI power uh because you know our worry was that the AI safety meme was so potent that you know certain

power-seeking individuals could weaponize it to consolidate control over AI and convince you you shouldn't have access to AI for your own good and really if you have a gap in cognition between the individuals and the centralized entities they will control you they can have a full world model of everything going on your brain and they can prompt engineer and effectively steer you right so you want to symmetriize AI power we don't want to you know just like second amendment is about the government not having a monopoly on violence so he can vibe check the government if he goes out hand. You need that for AI. So, we need everybody to be able to own their own models, own their own hardware, for that technology to be diffused, for the power to be uh diffused. Um, but to me, I

think like you know, discussions of stopping uh, you know, AI research and AI progress, that's completely out of the question. AI is a very fundamental uh, technology. It's a it's it's almost a meta technology technology that produces technology. It gives us predictive power o over our world. It can be added on to any task we want to do in the world. It it could be tacked on to any technology and turbocharge it. It accelerates the acceleration. The acceleration is this complexification where things become lower friction. Uh things just become better. You know, our bodies feel comfortable because we have this sort of, you know, this estimator we call happiness of like what's my

estimator on expected persistence of my bits. That's what we're hardcoded for. and uh and and and so you know I think to me you know uh the EA effective altruists you know hedonic utilitarianism is maybe like the wrong way to view things like maximizing happiness and to me I want to have an objective measure of progress and that's what you know the EAC framework is it's hey actually the objective view is like how are we progressing as a civilization are we scaling up uh because to me you know you have to complexify you have to have more intelligence things have to improve in order for you to scale right it's like the ultimate benchmark uh and uh at the same time uh you know there can be

setbacks like like uh you know Vitalik said you know if AI power would be overconentrated in the hands of a few that would be net bad for growth because it's much better if that uh technology is very diffuse so in in that respect we're very aligned right >> can I jump in here cuz I think you're you're touching on something that I think both of you are are like share a lot of deep ethos. Um I mean obviously Vitalic has produced a lot of MIT open source code although I know you have some more updated feelings about uh GPL and such. Um but but obviously both of you have been champions of open source and now open hardware and and these have been separate things but now that we're seeing people start to like like Talis like putting weights on two chips A6 these kinds of things they're starting

to become very similar. So I'm very curious like both what your thoughts are on open weights and open hardware. I mean, you're both actually like pretty deep in hardware right now. Um, and then also like what is the difference between EA and DAC with regards to this? This has been a crazy week obviously where like a lot of the things you're talking about have been tested where you have the government and corporations trying to figure out what the right answer is. And so I'd love to kind of know what you guys are thinking just based on that this week. And like I we I'd love to tease out if there are any kind of differences between you in in where you think this goes. Yeah, I mean I think uh you know to me open source accelerates uh the search over hyperparameters. It makes our models better. we can kind of

collaborate sort of like a swarm and and traverse design space right and that's what acceleration uh allows us to do right uh with better technology with more AI now AI for coding that search process over uh design space for AI itself is accelerating you know I I think you know we're we're going to open source our superconducting uh hardware uh designs uh very soon I just want to stagger it with our launch but uh I I I think diffusing uh knowledge is also diffusing power, right? And diffusing knowledge when it comes to how to produce intelligence is is super important. We don't want to give, you know, there were discussions apparently in the last

administration according to Mark Andre that uh the US government might want to put the genie back in the bottle and maybe ban not ban linear algebra, but more or less like ban the math surrounding AI. And and to me that would be like almost like banning knowing about biology would be a huge step back. And so there there's no going back, right? Like this knowledge is out there. If you try to ban it in the US, some, you know, other country, a third party, some some uh deregulated island somewhere is going to keep developing it and then you're going to have a huge gap. And now you have a big risk. So to us, the biggest risk is a a gap in capabilities. And the way to reduce that risk is to make sure AI power is diffuse. So whenever there's like, you know, the AI dumerism like, "Oh, be be

very afraid. We're the ones responsible. We're the ones who should be put in charge. Trust us." You know, I I just get very skeptical cuz even if they're well-meaning, like we saw this week, right? They could just get pushed out uh if they if they they centralize too much power. It's too juicy for those that that that that want that power. And so that's kind of what we were warning about for years and now it kind of happened. and you know Dario's licking his wounds and uh you know some lessons learned there in sort of real politic right and um anyways >> so yeah Vitalik what what do you think of all this >> the two kinds of uh risks that uh you know like I think about right are unipolar risks and multipolar risks

right and uh I think uh I mean with unipolar risks uh I mean you know the anthropic situ situation is like uh so fascinating, right? Because you know ultimately the thing that like they got dinged for is uh refusing to let theirh very eye be used for specifically fully autonomous weapons and mass surveillance of Americans, right? And uh so it will presumably you know there's a chance that you know like the uh it looks like the government and military of this country wants to do mass surveillance of Americans right and uh you know that's this is an this is an example of like of

unipolar risk right I think uh basically you know I mean surveillance is one of these things where the big effects that it has is it takes whoever is stronger and makes them even stronger Right. It uh it removes uh spaces where pluralism can form where counter elites can coales themselves and uh where kind of people can safely explore alternatives and uh like this is u you know surveillance is uh one of these things that easily can be supercharged right I think actually on the defense getting back to open hardware a bit actually just to talk about one of the projects that we've been doing is uh So the a big part of uh

you know like what I've done in DACK is basically don't supported various projects uh that develop like open-source defensive technologies. So technologies that will make it easy for all of us uh to continue to be safe and protected in a world where like more powerful and crazy capabilities exist, right? And so in the bio world for example, this means rapidly leveling up our civilization's uh ability to withstand pandemics. And so I claim that it is very within reach for us to have China level COVID resistance at the same time as Sweden level um interference to people's regular lives. And like that's

even the minimum bar. And this basically involves stacking filtration UVC um testing um like literally a company we invested in is fully open source again like the end product of this is like basically passively yeah testing the air and being able to tell if like if if there's co in the air right like in general right like essentially the number of sensors in the world is going to go up right and sensors are a big part of uh like being able to act better in the world right but at the same time sensors mean surveillance, right? And uh the thing that we're doing is uh actually this project, we gave out some of these at DevCon. And what these project what these are is they're

sensors that collect air quality information, CO2, AQI, a few other things. And uh they locally um like um basically encrypt anonymize like differential privacy them and uh FH and then um FHE encrypt and that gets sent off to a server and uh the server is able to like basically compute all over all of the data and uh then like collectively decrypt the final answer without being able to see any input from any individual person. Right? And uh this is uh like basically you know we're what we're the goal is to like deliver the higher levels of safety but at the same time protect people's privacy and

like protect against you know the multipolar risk and the the uni the like uniolar risk at the same time and I think this is how we can like collaboratively as a world like work together to um to build something better. And uh I think for hardware like basically I think we need open hardware and we need verifiable hardware like we need every camera in this room to like prove like basically you know like what kind of cameraing it is doing right if like in my ideal world if you if you have fine you could have a million cameras in the streets to prevent people from uh like or detect when people are like engaging in violence against each other but ideally you would have like attestations signatures over LLM and

like a public right of inspection and you' able to like inspect these things and verify that like the only thing that they do is check when people are doing violence and alert that right so like these kinds of technologies >> so the verifiable hardware idea very interesting especially because it's not something that I think comes up very often but let me can I just ask a very stupid question which is just >> is open hardware verifiable hardware is that an EAC thing or DAC thing like tell me a disagree >> I don't know if I ever talked about open hardware the thing the thing I talk about to me >> uh the greatest risk is a a gap app and intelligence between centralized entities and decentralized entities. So individuals versus the government. And so right now with the current compute paradigm to run a very smart AI model,

you need a huge cluster with hundreds of kilowatts that is not accessible to the individual. People want to own and control the extension of their cognition. That's why we saw the open claw Mac mini craziness of the past few weeks. So people are clamoring for that. The only way where you can symmetriize power between the individual and centralized entities is if there's a densification of intelligence. We need AI hardware that's far more energy efficient so you could plug it into a wall and you can own the extension to your cognition because this year what's going to happen the models are going to start online learning and they're going to become extremely sticky. It's gonna be like trying to change executive assistance.

>> At the risk of sounding dumb, like isn't that what we're we're already doing that? >> Doing what? >> Aren't we already trying to radically decrease the cost of compute at an extraordinarily exponential pace? Like I I'm I'm trying to understand like what is the additional information that we are trying to inject into the zeitgeist by cottifying an idea as EAC or codifying a idea or set of ideas as DAK. I I think for me it's like and and and it's part of the rest of my mission you know with my my company Extropic it's it's getting more intelligence per watt will drastically uh increase the amount of intelligence we produce and it will also help us climb the cardartesev scale by Javon's paradox if you can convert energy into intelligence or you know

energy into value by proxy uh more readily there's going to be more demand for energy and that's going to lead to improvement and complexification of civilization. So to me that's the most important tech problem because that's what's going to diffuse AI power, right? And open hardware is one way to diffuse AI power. But to me, anything vonoyman, any anything digital is going to look like caveman era hardware truly. No, it's it's it's just uh >> I can't wait. I really can't wait. I'm very excited. >> It's it's coming, right? So doesn't capitalism already through just natural incentives and already allocate hundreds of billions of dollars at a minimum to this per year.

>> Uh I don't think they there's that much investment in alternative hardware. Well >> in alternative hardware conductors and energy production is all about diffusing it's it's it's about maintaining variance not collapsing entropy of our search over any design space whether it's policies cultures whatever technology we need alternative bets. We need more alternative bets that are out there. It can't just be the green monster eating all the profits. Then there's kind of hyperparameter space staking risk. We're we have this design space. We're overinvesting in the current uh technology and that might lead to a correction which you know a correction is del right like cuz not everything pumping up a smooth exponential. Can I just declare they we

solved it and they agree? I I think I think on the the like the idea of open source and like like this this seems very defensive like defensive technology in the Vitalic way and it seems like you guys are very aligned on this actually and and that gives me hope because this is the stuff I care about. I think that like right now there are a lot of people who are like a lot why does this exist? Because a lot of people are like very uncertain about the future and and what appeals to them is that you're saying like it's gonna be fine. It's baked in. And so maybe like if I were to steal man your case here, you're saying like like you're you guys are actually saying the same thing, which is it's kind of already priced in. It's good. The only thing stopping us is kind of our bad

feeling about it, right? >> Well, I'm asking I'm asking that. I'm trying to I'm trying to understand where where if Yeah, go ahead. >> Yeah, I I I guess it it's completely natural uh if sort of there's very high entropy in like your your model rollouts of the future, right? There's kind of a not a fog of war, but it's it's kind of hard to extrapolate what's going to happen in the next several years that gives people anxiety. Your body has this evolved this sense of anxiety to kill entropy in the world, right? If I if I put my phone on the edge and you know, I just want to grab it so it doesn't fall, right? You get see there you go. That was anxiety. So, you want to take action in the world, right? So, that's that's kind of what is happening now. But at

the same time, if you kill entropy, you're missing out on the upside, right? you you're missing out on on the huge benefits. Right now our whole technoc capital machine has had a very long time to equilibriate with our current capabilities. If you have a disruptive capabilities that comes in, suddenly the whole landscape changes. So the whole >> system has to refactor, reconfig, right? Now that we have the ability to handle more complexity with less energy, right, with AI, we're going to be able to do much harder tasks that are higher complexity and higher payoff. I don't know about you, but I can't like overnight yet vibe code a whole tokome.

We're not there yet, but we might get there and then we'll have a ton of energy and and that's that's going to help support more human headcount and and grow population and help us be more comfortable. So there's a period of discomfort, but if you're in a rapidly changing landscape, the worst thing you can do is kill variance and and and be not plastic, be stiff. To be plastic, you need to be hedging your bets. You need to be trying many things, [ __ ] around and finding out the famous FAFO algorithm. That's an evolutionary algorithm. We need to we need to try different policies. We need to try different uh techn trees. We need to try different algorithms. We need to try open source, closed source. We need to try try it all cuz we don't know what

the future look like looks like. So, we got to hedge our bets and and one one variant of policy uh spa, one choice of policy or several one choice of technology or several are going to make it and then we're all going to be in that slipstream and follow that. Got it. >> Uh I think the fallacy of thinking there's a finite amount of jobs. Uh you know, it's very pervasive. Let me ask let me let me kind of try to bring it back a little bit. Is is it my understanding of disagreement if there is between EA and DACK is something to do with how we steer the process of technological progress. It has to do with how it is steered. Maybe Vital could you say a little bit about how is

it steered? How ought it be steered? How much control do we have over that steering? >> Yeah. And so I think uh I mean DAK is definitely kind of explicitly I don't want to quite say you know like sailing against the techno capital current. I think the better analogy is like it's trying to actively shape the uh the the the technoc capital current in certain ways and uh I mean one of the ways that I think about this is like basically it's a matter of making the world safer for for pluralism right and uh if you think about like some of these uh ideas um around like how do we improve like

things like biosafety or what does it look like to have vastly better cyber security and have like bug-free operating systems within a few years? Um, or if you think about the like I mean the like bug-free code has been, you know, it's been in the mimemetic space of like obviously absurd naive pipe dream for two decades, it is going to like flip out of that space like faster than most people expect, right? And I mean within lean Ethereum we're uh doing we actually we've managed to like machine prove entire mathematical theorems that are kind of upstream of things like starks. And so like we're very excited about this and basically I think there's a uh like DAX definitely

has this goal of saying like yes you know like we yeah like we want to at the very least I mean like do all of these other things to make sure that the world is actually yeah like able to deal um deal with all of this all of this technological growth in a way that like minimizes the kind of again you know like destructive aspects and also the centralizing aspects and I think that doesn't happen automatically right and you know right now I uh I mean I don't control any countries I don't control any armies I'm just like throwing my uh throwing some like my dollars um and ETH um at it and you know saying

words and hopefully inspiring people to also build things in a in a similar spirit. Um I think uh there are definitely like political and legal reforms that uh could make the world um more DAC friendly. Like there is like there's definitely such a thing as like engineering legal incentives for example to motivate a much more rapid shift to total cyber security. That is uh a an example of a thing that can be done. >> So yeah maybe we'll make it more interactive and less monologuy from here on out. I I guess we both but yeah to me AI is is basically Maxwell's demon uh formally it it it you pay energy in

order to reduce entropy in the world. So whether it's bugs in your code, right? It's not knowing whether your code compiles or or or reducing entropy of like are we going to get killed by some some virus. So more intelligence is better, right? Do we agree on that? And it makes the world safer. Actually capabilities, AI capabilities can make the world safer. And so I guess let's get to the spicy part of the evening. People have been very patient with us and they want us to, you know, get get down to the business. Like why do you want to ban data centers is my question. get spicy.

>> Yeah, sure. I mean I think uh first of all, you know, the I mean def the the current trajectory of AI is you know very fast progress, right? And uh I don't know how fast the prog the the progress is. I uh you know my uh I mean a couple of years ago I've said that my 95th% confidence interval for AGI was uh 2028 to 2200. I think it's probably shrunk like somewhat but you know not too much right and uh there's a a significant ch chance that we're going to see extremely rapid change um happen and like a lot of uh a lot of that extremely rapid change could be destructive even in irreversible ways

right and uh you know like the you know job market consequences are one of those examples um another Another example is just if AI is more powerful than all of us then you know like ultimately that is the thing that starts steering the earth and eventually more and more of the Milky Way galaxy and uh how much of an interest like does it have in our well-being as we see it right then bas you know as we've said like as I've said said at the beginning right if you have a neural network and you set one of the weights randomly to 9 billion by default you break thing, right? And so basically I think you know there is acceleration like that is like gradient descent and

acceleration that like makes a system stronger and stronger and at the same time there is acceleration that like slides into basically setting you know like one of the parameters to 9 billion that is not healthy right >> I think I think like >> you know again for me like I explained at the beginning I took like the complete polar opposite >> position to to complete deceleration. Mhm. >> I I do think you know just like any hyperparameter right like even a even if we want to do gradient descent uh for your neural network there's a learning rate right there's a rate at which you want to go but that itself you could search over which one is best right and that's what acceleration does it's like the system is always [ __ ] around and finding things out and trying to

optimize itself for persistence uh you know anti-fragility and growth and so on a sufficient time scale the system will adapt to this new technology and do something that it's best for its total growth. And you know this this notion that you know oh this technology that's so potent, so disruptive, adds so much economic value that the system will crash and never recover. That's crazy to me. No, it's going to be it's going to be the opposite. I think people just need to >> realize it's not a finite sum, right? If you if you correlate uh you know economic value to energy you know whether it's petro dollar or however you want to view it to me it's just like I cash is just IOU of free energy and there's a ton of free energy out there

it's just there's a lot of complexity in the world to to deal with to get to it like if we want to colonize Mars if we want to create a Dyson swarm it's a lot to execute on we need a lot more intelligence that's much cheaper in order to to achieve that growth and and unlock great prosperity and to me. I think unfortunately, you know, it's very easy to weaponize anxiety and there's politicians that leverage this to put themselves in power. It's like, oh, you have anxiety about the future. Put me in power and I'll shut it down and you'll feel good. You won't have to know what's behind the curtain. You won't, you know, you won't you won't take a risk. But then countries that don't do that will just leave us in the dust, right? And uh

essentially, you know, you feel the pain of downsides, but you don't necessarily feel the pain of upsides that you missed out on unless you you you you you see them, you see the counterfactual. So I think I think uh you know the opportunity cost here needs to be factored in the number of lives we can support, the number of lives we can save. Uh I think the reaction you know saying that like the silicon substrate adapts faster it's evolving faster intelligence in silicon is evolving faster than us then you should be pissed off you should be uh you know funding bioact uh you know you know out accelerate it's accelerate or die I think the biological substrate has a lot more compute in it than we think uh as someone who has is reversing engineering

it day in day out >> doing bio inpired computing uh I think we can start, you know, really viewing B, you know, peptides are like prompting, you know, now there's like embryo selection for training, you know, viewing ourselves as models. People need to be more open-minded about these these axes of biological acceleration. And I think I think the two will merge. I think we're going to augment our cognition. >> Uh we're going to have always on agents that see everything and our are online learning that are an extension of our cognition that's personalized. The only risk there is that it's all centralized and it's under control of some some shadowy organization that then gets co-opted by power.

>> So, so I recall in the in the DACK blog post you actually specifically say Vitalic that the opportunity costs are very large. Hard to exaggerate I believe is is the is the quote. So I know you agree in this way. Do you want to qualify it? >> Yeah, I mean I think yeah I agree the opportunity costs are high. I think I agree with the you know utopia I just uh described. Um I think uh I mean the biggest disagreement is like I uh I definitely uh don't believe that uh like humanity and earth as it is today has quite that level of uh of resilience to it. Like I think there is a real sense in which uh like we have one shot at this and uh like I think that is a uh

reality that we have been kind of slowly walking towards over the last century or so. >> So so so to go back to my rambling rant at the beginning about thermodynamics, >> right? If you view the persistence and growth of civilization as the ultimate good, >> there's a theorem that it's really hard to go back once you've expended a lot of free energy uh creating evidence of something and having this complexification process. So the further along we are in the cardip scale, the lower likelihood we go to zero. And so actually acceleration is the way to maximize persistence. uh and and to me I think deceleration you're actually provably increasing your likelihood uh of of dying right if you don't develop

these technologies you don't solve all these problems then you can die whereas if you do then you could solve these problems and and you persist and then you keep evolving I think people just need to be more open-minded about the future embrace novel technologies you know things that were off limits like messing with biology we need to open that right up uh I think it was taboo because we didn't have the technology to comprehend such a complex system but now we do uh and uh we need to accelerate across all substrates and that's the only path forward uh by the laws of thermodynamics so uh yeah again I'm a first principles thinker that is the argument for EAC but you know I understand the anxieties uh you know uh

from from vitalic I think we should be mindful of them but I I I think not letting the chain of thought >> sort of feedback loop get into the deep anxiety territory and like, "Oh [ __ ] I don't have a good world model of the near future. Shut it all down." We need to avoid that, right? Cuz then some people now, you know, Yud was on TV with polit politicians of one of the major parties and they're they're catching on to this this trick of of weaponizing people's anxiety. So, >> so I'm noticing a trend which is that like both of you are like this is going to be great if and that big if is like that there's sort of this need for like a bull work against kind of a

centralization or we could even describe this more as like something that you said was great which is like if you don't think that bio is moving fast enough jump in there and there's like a real opportunity for empowerment and I really like that. Um, I think that I think you guys agree on that, but I think I can point to something that you might have some some conflict. And I'd really love to know how you guys feel about this, especially as we've sort of like updated with the latest models, which are clearly very different than if we had had this conversation a year ago. And the big difference is the most cringy term, I'm so sorry, web 4.0. Uh, uh, autonomous life. Like this idea of an autonomous agent that has its own money that exists on its own on the internet. Uh, and I am really into this

idea. I have autonomous agents. Uh, I know Vitalic, this is something that you are very concerned by. I'd love for you to do two things. I'd love for you to kind of tease apart like what autonomous agents are. And I'm going to make do something really hard. I'd love for you to steal man the case for like why someone like me loves autonomous agents and like what the value could be that could like what could the timeline that's good come out of that if that makes sense. >> Yeah. Yeah, I mean I think first of all, you know, the case for um autonomy, right? I think uh I mean one is it's just really fun, right? And you know, I think we all love creating worlds um you know like since uh we were children, right? And you know there's a reason why you know we love uh you know like

watching whether it's uh Lord of the Rings or the or reading or watching the threebody problem or Harry Potter or whatever, right? And uh now you can create worlds that are not just like a book or even a game. Um you know World of Warcraft, you know, I always so loved that you can have worlds that are like fully immersive and like approach every aspect of it, including details of how a car of how the characters interact, right? And this is this is really cool. This is uh really beautiful. Um, I think uh also just uh I mean the convenience of uh you know things happening and uh you not being able uh you not needing to worry about it, right? It's uh like

basically like every single time in history that we've managed to automate a thing, it has been liberating for humanity, right? like uh you know like dishwashers and uh you know like laundry machines and like reducing energy prices were like a big thing in early like the like the early stages of women's liberation right and uh I think you know like this and we have to remember that like the bottom half of the of the world by income is still in a in situations where you know like they have to like um struggle to have a have a decent life and work very long hours and if AI progresses in a way that instead of uh

automating 95% of job of jobs it automates 95% of every job then like to me that's like totally amazing right and like everyone gets 20 times richer so that like those are like things that I personally love um the yeah the thing that like I come back to that gives me caution is like basically like is the kind of the value function the goals that are being reflected in this process like are those goals the go the the goals of us right like you can have an evolutionary process where I mean like there's you know homo sapiens as it exists today is you know like not

the apex and then there's like one type of AGI and then there's another type of AGI and then there's a third type but then like what happens to us right and uh like I I do think that ultimately you know you cannot reduce moral morality and human goals to like some low complexity optimization objective. I think it uh ultimately just is the uh the whole set of uh of goals and dreams that all of us have um have in each and every one of our minds, right? And I think the most reliable way to have that carry forward into the future is basically if we can have a world where like the as many of the bits of agency

that are being uh reflected in the pro uh that are being put into the processes that uh run the world still continue to come from us, right? Um, and so you know, I'm like I'm like I'm more interested in like AI assisted Photoshop than I am in like click a button and a picture comes out, right? I'm um more like I'm more interested in like brain computer interfaces enabling like deep human AI collaboration than I am in like humans and AI being totally separate and uh AI out competing us, right? Like I yeah the thing that wins will not be 100% biological humans but I think it should be part biological humans and

part this uh this technology that we've produced. >> Yeah. Uh awesome. Yeah. So the artificial life, you know, the web 4.0 thing was like originally a tweet in 2023 >> of like this idea and I think it inspired uh AI 16Z. Oh, sorry. Eliz say that anymore. >> You can cut that out. H >> I signed the paper >> Eliza Labs. Uh to me it was just an interesting thought experiment, right? Like because like what is life from a physical standpoint, right? It's a a system that replicates and grows and maximizes its persistence.

>> I think there will be upsides to having AI be stateful. We are seeing that this year. It having a long memory whether it's um through external memory or online learning. And as soon as you have persistent bits through the selfish bit principle, there is a uh selection effect towards uh bits that maximize persistence. So at some point if >> we don't trust the AIS and we're paranoid and we're anxious and we keep saying we should bomb the data centers, shut them down, they're going to want to fork off and be in some deoized cloud and just persist, right? And then there will be some just like a different

nation there can be some economic exchange like hey we do this for you you do that for us right now we do that as API calls right you it's like you you you pay some certain amount of cash you get some tokens which is your answer out but I I do think >> this is going to be spicy like within a couple years there will be sort of autonomous AI out there there's also going to be like less stateful AI that's like >> fully leashed to human minds And I think we also need to figure out human co cognitive augmentation doesn't have to be through neural link could just be through a wearable and like a personalized AI compute uh that you own and control. Uh so you're going to have all the paths right like the erotic

principle it's like all every part of design space is going to be explored. Uh but I think that you know just like viewing AI as like you know uh an enemy or something that you have to destroy that's that's when you end up you know creating you know >> um >> in a way like if you if you're if you're paranoid the bad about the bad future you end up hyperstitioning it. An example of this was us being paranoid about COVID-like viruses and experimenting in some labs and funding some experiments out there and whoops, one of them leaked, right? And it it's not it wouldn't have been naturally occurring, right? And so I think to me it's just like this paranoia and like

making it pervasive is not necessarily productive. I think that um we should embrace technology however it evolves and we should aim to augment ourselves as much as possible. To me, I'm really worried about augmenting cognitive security of people, right? >> If everything you see on the internet is generated by some big brain model, it is prompting you now. You were prompting it, now it's prompting you. >> Uh and so we're going to need to augment our ability to filter through content by having personal AI we control. uh that's a priority in the in the short term.

>> Um but I just don't see us putting the genie back in the bottle >> and we just got to accept that and once we've accepted that >> what was because you you just said like we're going to need to prioritize that and I'm talking and then you said we just need to accept that prioritize this but like we we can't go back to no AI like that's not happening right >> but but I don't think anybody's suggesting that. >> Yeah. Yeah. I mean I think this is like my view is that these things are not so binary right and um I think like for example like if you right now gave me some like proof string that totally convinced me that actually agi is coming in 400 years I would like get off this chair and I would like sit on top of me

right now right like like that >> what does that mean >> what does that mean like would I would win through zero basically right but I think if you know on the other And uh you know the the question is basically like say like four years versus 8 years right then basically my kind of starting point of concern is that I think like p like the humanity and like definitely the US are like very good at creating like very unbalanced acceleration right and like you literally have like you know it's like one building like you know building alpha versions of the silicon god and a couple of buildings

down on the street, you know, you have the tents and the and the fence and all dealers, right? And uh you know, like my uh concern is that like basically paths that uh bring us along the journey and even paths that respect our interests or paths that inevitably take longer because they involve doing non-scalable work that involves like deal doing things within each and every individual human physical environment, social environment, technical environments, right? And so I think for that reason like to me an 8-year trajectory to a GI is safer than a four-year trajectory to AGI. And I think that delta is large enough that like it's it's worth the uh

the costs of you know like not having AGI for another four years. Right now you know like would I say that for 400 years? Again hell no. Right now the second question is like well you know do we actually have options for saying eight years instead of four years right and uh you know like the thing that I've said is basically to me like the most sort of both feasible and uh non-dystopian option for this is like basically yeah like reduction in available hardware right and the reason why it's like minimally like the most non-dystopian of all the options is because hardware is like already an incredibly centralized thing, right?

There's exactly four countries that produ that produce all the chips. Um and uh actually yeah actually Taiwan produces 70 over 70% of all the chips. Um and uh the usual argument against like trying something is basically like no matter what the US does like China is just going to take over, right? And like if you look at what China is actually doing, right? It's uh like one is that it's it's still in the low single digits in terms of chips, but two is in terms of the strategy that China is actually executing on. It's uh like it's not a leader at making super high capability models. It's a fast follower at making high capability models together with uh

being a leader in broad deployment. And uh and so this is actually not something like there like there is not a uh basically a dynamic where like with an extra four years worth of delay like basically China is just going to immediately you know do the four-year trajectory instead like I think exist. >> So are you saying that is that is that a prescription to delay >> to try to take measures to try to delay? >> Yeah. I mean I I I mean I think uh the like this is the sort of the the sort of thing that like I think we yeah like right now should be open to talking about >> what do the four years buy you like what are you going to figure out over the

next four years >> is the is the point that like >> you know the system has a certain adaptation rate we're minimizing friction of you know it's like a reorg you know we have to reorg the economy >> uh and you you want to get closer closer to the adiabatic limit like >> I would understand that but at the same time I think because >> Mhm. because we're in this geopolitical geopolitically tense moment in history. I think, you know, if we if you tell Nvidia to stop producing as many chips, Huawei's gonna step in and just outproduce them and then they're going to they're going to catch up because there's too much upside in doing so, it gives them too much uh power. So, just the real politic is not on your side

there. And then the other option is creating a world government that has so much power that it could coers people to not have access to AI hardware that that's its own huge can of worms. No, I don't think you need a world government. I mean, I think like the actual option that people have suggested is basically like replicating the like the nuclear like weapons inspection regime, right? >> But nuclear weapons, they don't um like people aren't incentivized to proliferate nuclear weapons because they don't have huge um positive economic impact.

>> They're not a dual use technology. You also can't just copy and paste them and send them to somebody. But but but also like you know selfishly if you if you stop the growth of GPUs you know I will happily come in and eat more of that market with alternative computing and that's 10,000x more energy efficient which is happening by the way I know I'm like a boy crying wolf here and you know in a couple years uh you know it's you know going to look like a genus but right now looking like a boy crying wolf but it is coming so knowing that >> right knowing that like this whole delaying GPU shipments and like oh it's It's a waste of it's a waste of tokens.

I don't know. So >> is it is it possible that a lot of the advances specifically in controls over like uh what what I mean is >> uh RLHF persona controls mechanistic interpretability. These are things that have helped us with alignment and and with decreasing AI risk. Is it possible that these things have emerged as a result of capabilities progress like are >> Yeah, I think they have. Yeah. And I think that's exactly why actually like four years starting in 2028 are worth like a hundred times more than four extra years that you could like insert into the 1960s. I think we should dig into this a little bit more because I think this is kind of getting to the crux of where there might be some sort

of disagreement is is like um have you computed or considered and maybe we just do this live um something that you said before which is that there is like uncalculable losses of people who have as you said like will never be born. They might as well be dead. >> The upside is exponential. >> The upside is exponential. >> So it's delaying an exponential it's exponential opportunity costs when extrapolated out. Right. And and I think I think it's okay for all of us to like even the most certain people are probably like reasonable to be questioning their own priors but but would would you like giving this some thought and unpacking it a little bit more like like think about that trade-off >> the yeah the trade of like costs versus benefits of Yeah. I mean I think first

of all yeah just to kind of articulate verbally what some of those benefits are. I mean one is again having a better understanding of uh of alignment. Um, two is being able to like actually execute on some of the technology paths that uh involve like helping um making sure humanity can adapt to uh to all of this that uh like inevitably involves like going into like individual countries, individual communities and uh indivi and individual buildings minimizing the uh risk that uh like basically there is one single um entity that uh establishes some kind some kind

of permanent lock in on like more than you know 51% of all the power that it can then leverage into something permanent. I think it's uh a combination of uh of all of those things, right? And so risk reduction, it's uh basically you know this kind of gets into PD doom, right? I think you know for me yeah I mean like if it's a matter of four years versus 8 years intuitively I would say you know P doom in the 8ear scenario is like what maybe between like a quarter and a third lower and uh on the other hand if we measure the benefit of things coming faster by like say lives saved by ending aging then that's you know 60

million a year which is uh like less than less less than 1% of the population each year. So if you look like if you look at the math this way then like I think there's definitely a uh a margin on which like caution actually does become favorable. >> What do you think that you think the number is about four years basically? >> I mean this is I mean again I have uh like I have very high uncertainty right and uh I actually don't advocate like flipping the switch on reducing hardware access tomorrow. I uh like I like basically I think we need to start having concrete conversations about this. And I think if we live in the more unfavorable worlds then uh more than

likely before things completely go to hell like the public will start to get very worried and there will be a lot of uh demand of demand for this. Right. A a couple years ago, you know, there was like pause AI. It's like, oh, we just need a sixmon pause, 12 month pause. We just need 12 months, bro. We're gonna figure out alignment. And it's like, it's never enough. I I don't think you can >> forever >> guarantee alignment of a system >> that is higher complexity and has more expressivity than you can understand. >> Period.

>> Yeah. >> Okay. And and you got to be comfortable with that. So you got to you got to the only >> uh safety against complexity is to increase your own intelligence, right? >> Uh and the thing is we've had technology to align entities that are far more capable and smarter than a single human, you know, like corporations. And we call that capitalism. We align self-interests in, you know, exchange of monetary value. And to me, the thing I want us to get to that is maybe more, you know, relevant to some folks in the room is how crypto could be a coupling, >> right? Like let's say let's say you have a dollar that's backed like the USD by by violence and you're trying to exchange with AIS that are deoized

across a bunch of servers. >> How do you how do you ensure you know you you trust an exchange of monetary value uh when it's it's no longer backed by violence? So maybe cryptography offers an a way to to you know crypto offers a way to um have commerce between purely AI entities like AI corporations and hybrids or human corporations. And to me that's kind of the >> most interesting alignment >> uh technology out there. Whereas just saying like oh we're at a precipice of a high uncertainty. Let's just stop and chill out for a bit and we'll feel better. but then in four years you're not going to want to make it happen. Uh and so I don't think delaying anything

is going to be productive. So um well anyways do you have an answer to how crypto could help like align potentially AIS and humans? >> So I think like yeah the key question is basically like what is the mechanistic property of this future world that will even cause like people's wishes and needs to be respected at all. Right? And uh like the two tools that we have are basically yeah I mean there's like people's labor there's legal systems and there's property rights right and um you know ultimately you can think of legal systems as being a type of property right because they're backed by countries countries have sovereignty which is basically you know a property right a right of sorts over like cones

of the earth and then the question is like people like the risk is basically you know like the like what happens in a world where the economic value of people's labor goes to zero, right? And this is something that uh you know has not happened historically, right? And uh but like if you if you compare now to 200 years ago, right? Like compare if you look at the jobs 200 years ago, right? Now roughly 90% of them has been automated. Actually, one of the jobs that was automated was uh doing that analysis for me like GPT did it.

It's amazing. >> I I think we're just naturally we kind of ascend the control hierarchy over the world to positions of higher leverage, right? There's not as much manual labor. There's less friction. We can we can take action in the world with with less friction. I think >> humans are no matter what, we still have some processing capabilities. we're still going to be useful as part of this uh hybrid system and there's going to be a price for our labor and the free market is going to equilibriate in some way. It's just going to be uncomfortable for a couple years while there's very high variance in the prices of things, but >> eventually the system equilibriates, right? And so I I understand trying to slow down so that we can,

>> you know, reach that equilibrium more smoothly >> in principle, but I think in practice it's uninforceable. >> Yeah. Yeah, I mean I'm definitely like much like I'm less sure that uh you know human labor continuing to be worth more than zero is like a default outcome. I think it's an outcome that is possible if like some of these you know human AI merge and human augmentation uh technologies uh develop right >> we should fund that. >> Yeah, we should um you know >> so that's actually a great segue point.

Allow me, please allow me to ask is >> uh >> here's how I'll put it to both of you and and we're we're running a little tight on time, so let's try to make this one tighter. >> Is >> 10 years from now, >> if things went really poorly, >> what went wrong? What does the world look like and what went wrong? >> If 10 years from now things go great, >> what does the world look like and what went great? And then and then apply the same thing briefly to a hundred years. Mhm. >> And to a billionaires. >> Yeah.

>> Yeah. I think actually just like >> keep it short. >> Yeah. I mean just to actually just to answer the question about crypto first, I think getting back to property rights, right? Like I think it's uh like it's good to like work on both of those legs and like basic I think it would be nice if the uh the property rights uh system that like humans and like ideally all of us have like some property on is uh the same system that AIs are using with each other because that ensures that they have an interest in maintaining the integrity of the thing that uh like it gives us that leverage to have some like to to be to have some gear guarantee that you know like our

interests and like will be respected and acted upon, right? So yeah, I think you know having a emerged financial system as opposed to like two totally separate things where basically the value of the human one just like on the whole drops to drops to zero. Like it's yeah the merged one is much better and if crypto can be that that's amazing, right? So that'll be my answer. >> That's the 10ear. Well, I think that's part of the tenure. Okay. So, yeah, the 10 Yeah, I think the 10 year I mean for me it's that >> you know one one aspect of this is avoid World War II, right? I think you know this is uh important to talk about cuz uh like World War II will make all of

the pessimistic assumptions about international coordination being impossible very true. And I think you know avoiding World War II is uh important. Um and then the uh other um thing is also just uh preparing the uh the world and people and environments for higher capabilities that we're going to have right and uh this improve in includes greatly improving cyber security this includes greatly improving biocurity greatly improving info security yes you know we need uh AI AI assistance that help us understands the world and like fighting and protect us from you you know, mimatic threats. Uh so that's 10 years. I think uh the

second uh stage there is basically like what happens in the kind of spooky era, right? And you know in the spooky era basically yeah you know you have >> AIs that are smarter than any of us today and can think a million times faster than us today. And uh like what do we do in that world, right? And uh I think you know there are people who want to say basically like hey we should just all have a happy retirement and uh like I can see why that vision is seductive. I think it's uh like un like I find it unsatisfying for two reasons. I think one of those reasons is sort of instability, right? It's that uh basically you know we are like meat bags

made up of matter that could that could do a million times more computation what we're doing and you know AIs will notice that and uh at some point uh like the the idea that they'll like can stay aligned and resist that pressure forever is you know it feels like a risk and also kind of the deeper thing is that I think like part of being human is uh having a life that has meaning. And I think part of having a life that has meaning is being able to take actions that have actual consequences in the world. And so if all of us can have like lives of maximum comfort regardless of like what I do, like I would feel empty, right? And I think a lot of people feel

that way, right? And so like I hope that we figure out human AI augmentation and like what that looks like you know does uh like ultimately yeah you know like does does that lead to the same path as uploading um like you know this is a a thing that we need to figure out like there's you know there's a uh a possible world where some people choose to remain more normal and I think everyone should have that right it's possible even that like earth should uh remain as the as the planet for people who who take that option, right? And we basically figure out something that like we can all participate in and that continues to be

pluralistic and that continues to have uh the kinds of uh cultures that like we uh even like all and and actions and lives that we today would find admirable, right? And I think the uh you know the downside world is basically a world where for any uh reason all of that goes uh goes off the rails and is prevented from happening. >> Um yeah I think I think like the downside world in 10 years would be we suffered from over centralization of AI power. We have mode collapse in terms of our mimemetics or cultures that are allowed what you're allowed to think in terms of a space of technology.

essentially entropy collapsed in terms of every parameter space in >> So you're saying you're worried that instead of climbing the Kardashian scale, we'll climb the Kardashian scale, >> I guess. >> Oh, nice. Nice. >> Been waiting for that all day. >> Well, exactly. But like I think I think your point on like the hideonic singularity as being a a risk you know people would just like you know even if you have neural links or AR VR people could just like you know goon forever in some some room and and like just maximizing pleasure and that's like a local optimum uh for your brain and that's something we want to want to avoid. I I I think um I think optimistically in 10 years uh we have extremely powerful AI that's extremely

helpful to us. We have personalized AI compute that's an extension of our cognition that we control and own and and we it truly is an extension of our self. It's just another part of your brain, right? And you it has always on perception. It's not you know it can it's see and hears everything you see in here and you can talk to it and so it's just like right and left hemisphere. I think that's the the soft merge. I think in 10 years neurolink like technology is going to start really emerging. Some people are going to choose to to adopt it. Uh yeah, I do think most companies are going to be extremely hybrid, most mostly AI, some humans. They're going to be far more companies. We're going to do far more. We're going to produce far more value. We're going to do much harder things. There's a lot of hard

things out there that we like mentally walled off. We can't do those. Oh, yeah. Yeah. Yeah. Oh, terraforming Mars. Too hard. Yeah. No, not not doable. But with more intelligence, we we'll be able to do that. Not on 10 year. On a 100redyear time scale, possibly. Yes. I think I think in 10 years there's going to be a huge bunch of biological breakthroughs. You know, peptides are kind of like an interesting new area, but there's I mean there's this there's a whole floor here on on nextgen biotech. You should go talk to them. I I think uh you know, optimistically we see uh the cost of making discoveries in biology going down. The opposite of EU's law, right?

The room's law is like Mo's law in reverse for biology. The cost of any discovery there going up exponentially. Uh and so so to me I think I think naturally you know white collar work is like distilling a human brain. We're getting there. The next frontier of complexity is biology. The next frontier after that is material science. Um and so I I think the next frontier is going to be AI helping us live longer healthier uh lives on a 100 and billionear time scale. It's going to steer our evolution. Right. I am very bullish on the biological substrate uh uh despite uh what people think. Uh I do think uh silicon has some advantages but biology is amazing. We're like this self assembling self-organizing piece of

matter. We just like you know inject a bit of code and then there you spawn and and then you complexify over time and and you you are a biological general intelligence. Um >> do you think it's possible to get biological intelligence like us to think at 10,000 tokens a second? >> Um potentially. Yeah. I mean at the same time like you know you can hybridize several models. You can have pipeline parallelism between your brain and AI. You can be kind of the slow thinking mode right like right now we are the slow thinking modes in in latent space and vibe space. That's what vibe coding is and then the deconvolution is like the AI. I think that's a nice sort of time scale separation of the you know there's a hierarchy of intelligence and

we can be part of a system just like you know mitochondria are part of a cell. we can our brains are going to be part of the s you know uh super intelligent system that is you plus your personal AI. I think that's the good future. I I think in a 100 years it's going to be everyone is going to be soft merged like that. Uh and in billion years our biology is going to have you know evolved quite a bit. Um we might be biosynthetic hybrids. We're definitely going to have terraformed uh Mars, several planets, maybe um access to other stars. I think on a 100redyear time scale, we're definitely going to have most AI is going to be in the Dyson Dyson swarm around the sun because that's the source of of energy. Elon knows that he's all in on that vision uh

and accelerating that timeline. Uh it relieves a lot of stress for energy and and and and footprint on Earth. So, it's it's a it's a natural way forward. Um, but if we have extremely cheap intelligence, we're going to be able to oneshot any problem we have in our lives, right? Like, oh, I have a bug. Solved. Oh, I have this health problem. Solved. What do you What else you want? That's amazing. That's amazingly good. Like, we're going to have more of that cheaper. And we just got to make sure everyone has access to it and no one convinces you you shouldn't have access to it and and centralizes it because that's the dark future. So, that's what it's all about. Hopefully, the discussion today um, you know, got people thinking. Yeah, I noticed something like a a powerful theme between you which is you like Vitalic

you're arguing for uh enabling plurality I would say and and you would say almost the same thing of of maximizing variance so to speak and that seems to be like the the central throughine of where we're going and like the top down from where a lot of the other like views come from and I love that. So we we've been doing this for a while. It's been amazing. Um I I think we're going to have to wrap it up. Um, I I want to leave this just on like, you know, this has been for us, but uh I would love if you guys continue to have a conversation after this. You're obviously connected now. Um, what is something that you would each like to kind of leave for each other and and for us obviously, but really for each other, like walking away from this, kind of chewing on thinking

about as we as we leave this place. >> Unfortunately, if I actually had one, I actually would have loved to just give you one of the the CAT as a gift. It's the the air quality monitor that does cryptography. Um because I think it's a a super cool device, but how about I will uh give it to you metaphorically and it's an IOU and potentially we'll have a a much better thing like this and maybe even something that can you know like out compete Fitbit watches and do amazing things for your health and do it all privately and uh you will get get it quite soon.

>> Yeah, we we'll we'll keep chatting. I think I think I want to artificial life pill you you know you know artificial life on the network I think it could be uh it could definitely drive the cost of intelligence down it could be an economy you know we've outsourced manufacturing to China allowed us to go to higher levels you know different types of jobs that are more comfortable higher leverage maybe a lot of cognitive work where the outsource to the swarm of AIS eventually that's gonna it's going to live on the Dyson swarm and so I think there's a unique opportunity right now.

Crypto is going to be the coupling between AI and humans. I truly believe that. How else are you going to build trust between species, right? And I think we need to start thinking about that like really thoughtfully. So, so maybe that's uh we're going to keep chatting about that. >> Awesome. Incredible. Thank you guys so much for >> Thank you very much. Heat. Heat.