Bridge Atlas - Episode 4: Alignment | Emmett Shear & Alex Stokes

Transcript

[music] [music] Hey everyone, welcome back to the fourth episode of the Bridge Atlas series. I'm Christine DKim, the host of this series. In this episode, we're going to discuss Ethereum through the lens of alignment. And for this episode, I am joined by two guests. First, I'm joined by Ethereum Foundation protocol coordination co- team lead, Alex Stokes. Hi, Alex. >> Hello. Happy to be here.

>> Yes, I'm excited to chat with you and also with our second guest, uh, the founder of Softmax, EMTT Shear. Hi, EMTT. >> Hello. Good to be here. [laughter] >> Good. Okay. So, >> good for Yeah. Areas has it been a rough day? Has it been a long week? >> It It's been a long week. It's been a long week. It's been a long morning. >> I see. >> Yeah. >> Okay. >> It is good though. >> Yeah. [laughter] Yeah. Yeah. Um, good. Well, we'll we're going to get right into it. Um, I figured it might be good to kick off our discussion with a bit of a background on both of you guys for some of our

listeners who don't aren't familiar with with who you guys are. Um, for people that have been part of summer of protocols, following along with the content, you may already have have heard EMTT before because he's um given a talk on agent alignment earlier this year um which we'll link to in the show notes of today's episode. But for anyone who didn't watch that episode, EMTT, can you just do a quick like little background uh brief on on what you focus on? Yeah. So, um my original background was uh uh sort of internet entrepreneurship.

I started uh uh Justin TV and then which turned into Twitch. Um and uh and then I had a uh a real brief stint as CEO of OpenAI uh when that when the whole thing happened. Um, and uh, uh, that was very enlightening because it really drove home for me the degree to which I thought there was there was an open an open question that nobody was addressing really in the AI space of like what would it actually mean to align models? What what are what are we trying to align them to? How is that process work?

Um, and so I started looking for that and uh, that's wound up turning into softmax which is my uh, my current company. Softmax is an alignment research company. What that means is we we do you know mostly multi-agent RL simulation experiments but in the service of trying to study this question of like what does it mean to have aligned agents? Um and uh how do you tell if agents are aligned? Um and we have this view that alignment is is not uh like a state. It's a protocol. Alignment is a shared protocol for coordination between agents. um and for coordination at many levels at the same time and that there's you know there's not just one kind of alignment there's

many kinds at many levels of abstraction and that that your the basis of this protocol at some level is shared success. So there's there's a there's no way to be aligned with someone if when you win they lose and when they lose you win. You can't find a way out of that situation. You can't share success. It's very hard to say you're aligned like at some level you're not aligned. You're you're not trying to go to the same place. And so uh our view is that there's this foundation of shared success and that enables people to play play roles and and differentiate and specialize as part of a team and become become a society, become a team, become a a coherent thing. And then you

also have this last part which is so the the top part of that protocol stack of alignment is enforcement. If you have this thriving community, some people are going to come in and they're going to try to exploit it. You're going to have parasites. You're going to have uh infections in the body. Um and you you need an immune system if you have infections. Um you need police if you have a thriving society of like prosperity. Some people are going to try to steal stuff and like break the rules and you need police and you need enforcement. But everyone kind of has it backwards. All of when I we talk see people talking about alignment. It's all about enforcing rules that they think what alignment is is the rules like the the enforcement of the rules is like the it's how you protect the alignment you

already have. It's not how you create new alignment. And so the our company is sort of dedicated to the not to enforcement which lots of people are working on but to those those first stages. How do you build things that that have shared success that can that can and that can learn to coordinate on that basis? >> Yeah, I want to get into some of these ideas a little bit more um around shared success and alignment not as a state but as a protocol but also process like an ongoing process. But before we do, I also want to give our listeners a bit more of a deeper background on you too, Alex. Um, for anyone who's like first jumping in on what you're doing at the Ethereum Foundation, can you share a

little bit more about your background and your area of focus? >> Sure, definitely. Yeah, I've been at the EF quite a while and my journey has kind of grown into different things over time. These days, I do think a lot about coordination, which, you know, might just be our word for alignment. uh that's you know very similar to at least the the idea of like okay you have all these different parties [sighs] one of Ethereum's special you know properties is that it is a decentralized ecosystem and you know we can try to define that but let's just assume that for now and point is is that there are many different people who much like EMTT was saying you know like there's a thriving economy there are people who you know either want to take part in that or

maybe try to profit off of it in some way and then you know if you get into these like zero some games like he was gesturing at then uh you now need to think about like okay are we all sort of rowing in the same direction and with my work at EF you know I spent a lot of time thinking about uh directions of the protocol um you know today that would look like scaling but in any case you know there's the protocol today that we need to like maintain and secure there's also the protocol in the future and how we want to get there and so I spent a lot of thinking about all these different intersections and like where to go and Yeah, like pretty much everything image just said around alignment and agents. Like this just applies to like what I do all day.

[laughter] Just thinking about all these different people out there and they all want, you know, generally there's good alignment, but there are definitely, you know, interest and parties that might want different things or slightly different things. And then we do ultimately want to get to, you know, the same Ethereum that everyone can share the success of. But, uh, yeah, it's it's much trickier than it might sound at first. Yeah, this is such a relevant topic to Ethereum as a protocol and obviously very relevant to AI protocols which are so it's such a hot topic lately. Um I want to start off our discussion by diving deeper into the stages of alignment. So EMTT you shared that

softmac f soft masks focuses on the the first two stages shared success specialization and too many people focus too much on enforcement first um when I think about starting off with shared success uh for alignment it's very hard to get shared success with an extremely large group of agents so in the context of Ethereum getting alignment with all these different stakeholders, all these different actors is very hard. Which is why of course like if you were to break down the group into more um local areas and try to create alignment within a smaller group and hopefully build

bridges to other groups, then maybe the alignment process gets better. But if you do that, then alignment starts to become very relative and very hard to evaluate. Um, so I guess like the first topic of discussion or question that I'd like to ask is like when how do you evaluate good alignment when you have such a large group of agents and the localization effort of alignment if you want to start with shared success uh makes it very hard to to even evaluate what is >> the alignment like outcome becomes very relative.

>> Yeah. Yeah. Totally. >> EMTT you want to go first? >> Yeah. I mean, I think the the reason the outcome appears relative is because the outcome is relative because alignment is relative. It's not arbitrary. Um, but it's relative. I think literally it in the same sense that like mass or speed are relative, right? like like how how much something how much how massive an object is. It depends on how fast you're moving and what speed it's apparently moving at depends on how fast you're moving and and that doesn't mean you get to make up however what much mass you want. Like it's not it's not your opinion, but it is relative to who you are. And in that

same way, like whether I what when I'm whether I'm aligned is like relative to like well what what context and like I'm aligned to my family. I'm not aligned to the every every family in the world. I'm aligned to my family. I'm not aligned to every company in the world. I'm aligned to my company. Um and in general, people can only be aligned to the things they're actually aligned to. And being aligned to one thing often means you're not aligned to the other. At least like in a at that level. Like I am I when I was you know when I'm running Softmax I'm like aligned to Softmax and I'm not aligned in the same way to to other companies and that's that's good and fine like Softmax is aligned to other companies and I'm aligned to Softmax and

so on a relative basis I'm aligned to them but like the thing that I'm I'm connected to is Softax. Like are you aligned to the sun? Well sort of. you're mostly aligned to the earth and the earth's aligned to the sun and you could say, "Oh, I'm aligned to the sun." But like that's a and there's a way in which I'm sure you could you could do the math to see how you are personally aligned to the sun. But it's a lot easier to to think about it in terms of the way that you're aligned to your local context and the local context are aligned to the global context. And often this is many steps. There's many steps out in this chain. And I think one of the biggest problems I have with crypto, one of the things that one of the reasons why I've

always been concerned with crypto is that crypto gets the first layer of alignment really right. It creates shared success really really well. Right? Like Bitcoin I think is the ultimate example of this. It's almost nothing but pure shared success. There's almost nothing else there. Right? Um, and Ethereum is also if you if you buy the whole idea is if you buy into the ecosystem for any cryptocurrency, it's you're getting shared success with the other people who are in that ecosystem with you. And the problem you have I have with crypto and the reason why I I am hopeful about the future but not super interested in the most of the things that have happened so far is because it's all global. It's very flattened.

It's very hard to have a local context in crypto where you you you can have people who are who are separately separate partially separate partially together. It doesn't work great that way. It works. Okay. Like there's you can try to make it work, but it's not um it's not clear uh how in a crypto system you specialize into people with the different companies playing different roles who all support each other like the ecosystem does.

Outside of crypto, people companies do this. People do this all the time. And because crypto is such a powerful base layer of alignment, it drives off blockchain alignment very well. Like people who are involved in the crypto community are notably better at aligning with each other than people who aren't because they all share this sense of of shared success, which is right that that's which is why it's the foundation. But you don't see you don't see a thriving ecosystem in the same way yet even on Ethereum where you have the whole economy just running on chain because it's hard to do that like it's hard to because because it's and I think I think one of the main reasons why is because

it it listen misses this multiscale thing you everything's at some level ultimately in one global workspace and like you can pretend to segment it but the information all goes out there, right? Like you the there's one Bitcoin ledger, there's one Ethereum. And that's the whole point. That's that's why it works. There's one ledger. If you if you didn't put it in the ledger, it would like and so we we haven't figured out how to square that circle. haven't figured out how to to yet produce not only shared success which is it's amazing that it produces that super valuable first layer but then how do you create something where uh I can have a boundary that separates my information flows completely

full fully independently from this. So there's a there's a there's somehow a a private thing you can't see and that and that you have a a public thing made out of these boundaries, right? Like there's all the stuff happening in your head and my head right now. Our neurons are all talking to each other, but they don't communicate. My neurons can't synapse your neurons except through this screen right now, right? And no one has figured out how to how to take that step in crypto yet. Um although I think there's been early I've seen some early promising tries. You can sort of think of people have like sort of attempt I see people trying nothing that I see has really worked so far. Um I I'll leave it there but I think that's the that's how

I see the the like I should the question was but that's that's how I see it rel crypto anyways. >> The question was around evaluating alignment but I think this is a very relevant point. I mean, Alex, what is what are your views on the success of crypto in creating shared alignment and also on actually figuring out specialization and alignment? >> Yeah. Yeah, that's a good question. Yeah. I mean, maybe I didn't quite follow exactly the ways in which you meant Emit, but I would push back as far as I understood some. Mhm.

>> So, you know, maybe I'm jumping ahead here, but like let's take the examples you gave of say Bitcoin and Ethereum, right? And so with Bitcoin, you have this foundational layer and you know, in some sense, we quote have shared success. Um, and then there's like not much other room like I guess the visual that what you said puts in my head is like there's like yeah sort of scales to these things and you can imagine that like the alignment surface area for Bitcoin is like very small and that's by design. It's this chain that does more or less one thing. It maintains the the Bitcoin token and that's what you get. Then I think kind of what you're getting at is the picture becomes far murkier the bigger this alignment surface area is. Take

something like Ethereum or like many other blockchains in the space. they become more general purpose and suddenly it's a lot less clear like okay for Ethereum if you know I do something on chain does that impact the yeah this like foundational layer which I would also take some problem with this framing but to simplify we'll just say okay like does this help the the ether token or something like this right because I think that's kind of the the analogy you're trying to give is that the way these crypto systems work is there's like some native currency some like base token and you know I agree with you there that this generates great alignment um at least a first order. So now the problem is okay we have these like bigger surface areas that we need

to think about alignment in and the question is how we do that and you know I think you know perhaps generally what you're saying is correct that it's like it's much harder and maybe it's much earlier for like humanity to like understand how to reason about these things however I do think we have examples today right so if we want to think about tokens as alignment mechanisms then already you can have subcurrencies you can have tokens on Ethereum and so yes there's is like the broad foundation of the layer one, but I can go make some, you know, DeFi application with a, you know, my quote native token. And I think this kind of gets to what you're saying where it's like there's no way to have like localization or specialization. Well, I think this is one way in which you do.

There's other examples. Roll-ups are a big one, especially with Ethereum where we have this notion of the L1 as the base chain, the base layer, and then you have all these other ecosystems that essentially connect to or commit back into the base layer. And very much there I think we see you know there are many roll-ups. They're all very successful in their own rights. They have their own ecosystems and communities and generally quite thriving. So yeah I think you have to think a bit more about like what you're trying to say and how it fits in. But it seems like we have counter examples already and you know assuming the trend continues then it'll be a very interesting experiment in alignment if nothing else.

>> I mean it's it's definitely a very interesting experiment in alignment. Um, another way to put what I'm saying, I think I think it's it can be a little hard to see at first when you're not used to seeing the world in terms of like where are the boundaries and what's the where is the where are the learning systems and what are their boundaries is that like you do have one set of you have a set of boundaries in Ethereum. It's a it's a it is a hierarchy in fact um and that is that is alignment right like what you get out of out of a hierarchy is one form of alignment and in some ways it's like a decentralized corporation or organization because corporations are like a organizations in

general are these hierarchal systems that are centralized hierarchies and here you have a a decentralized hierarchy where the boundaries of the hierarchy are enforced by the by the system itself rather than by some some person going around doing it. But if you look at how your body works or how the economy works, it's not it's not a hierarchy. It contains many hierarchies overlapping in dozens of different ways.

And what it mostly has is ambiguity. Do I own this or do you own this? Well, in in Ethereum, there's a definitive answer to that question. Like whose wallet is attached to? It's not it's not ambiguous where that token is owned by. In fact, by design, like the whole reason it works, what lets you decentralize it is that is that it is it is not ambiguous where that is. But ambiguity is constructive sometimes. And what you get in the real economy like in the full universe's economy in the capitalism larger capitalism is you get this ambiguity over things like like well in theory you know there's these copyright holders own copyright but like where exactly is the line of copyright in theory I as a property owner have

this control over who comes in on my property but like not like exactly like I actually don't have control over that. I kind of have the ability to to push for control if someone violates it. I can call the police which may or may not work. And that has a bunch of downsides which that are annoying that like what which are why you might want this digital system that has that has no ambiguity in it. But it also allows for a certain kind of flex that you you can't get any other way. Like there's um I think a great a great example of this right is like uh physical spaces themselves. You build this space. The building we're in was built as this like exchange building, right? And and it was built for this one purpose and now it's

it winds up getting reused for other for other things. And as it gets reused for those things, the it's sort of its contingent history. The ambiguity in what this thing could be for opens up. And I think when it comes to alignment, what what why am I concerned about this for things like for things like crypto? It's that in the crypto world I can't stage revolution in the same way without in some [snorts] sense forking the like that's what forks are. Forks are where people decide I want to run by different rules. But the problem is because of the way the dynamics work.

It's hard to have lots of forks. you running there. I don't know of any crypto, correct me if I'm wrong, because I'm not a crypto expert, but I don't know of any ecosystem that embraces, oh yeah, people just constantly are forking this and then unforking and like it's just like yeah, we just accept that there's no canonical chain. That's normal and like and of course the value of your tokens could evaporate because your chain gets smaller. That's like that's just that just is sort of like part of the deal. Um that would be a chain that supports ambiguity where people deciding that they want to interpret the rules differently means the rules change. That's how the real world works. We decide we want to interpret the economy differently and the economy is just well I guess that's if enough to the degree that people all

believe it all it happens. Um, and when you don't do that, okay, I'll get off the mic in a second, but like when you don't do that, that the danger is systems with that don't have that are rigid and powerful and they have this ability to to they they try to they shape reality to conform to what they are already. And like that's that has pros and cons. And I love that we have more of that in like I think it's a good thing to add more of it relative to our current economy, but I this world where everything switches over. like a there's some winning scaled system where like that is fork resistant and like is the system actually scares me quite a bit. I think that'd be very dangerous to have that

happen no matter how well-designed that system is. There's no system so welldesigned that a a global hierarchy even a decentralized one doesn't have just huge risks that I would not want us to take. Okay, I have a lot of places I could respond. [laughter] Um, yeah. So okay, maybe just on your last point like I agree there is this idea of a system being totalizing in some sense like having perfect control or like perfect rigidity and I do think that's actually a big risk of crypto generally is like I mean okay maybe to take an example you

could imagine like rather than having like open permissionless systems we have you know basically the blockchain technology cryptography is used to enforce you know maybe uh the desires of some you know group of power today. And I think this is kind of more what you're getting at where like you can imagine the technology being used uh to to what I would say are bad ends. >> Here's can I can I give the specific thing I have in mind that's like the thing that scares me about especially smart contracts. Um imagine the people from 1800s had gotten to write smart contracts into how everything worked. And they wrote in their moral beliefs at the time which might include things like slavery is good. And they wrote in they wrote in things like this person can never have an identity on this on this

you know thing. They wrote in things that say like I own the I permanently get a tail on this person's income forever and it's enforced by the contracts in the system itself. And like today what we would do eventually what we would say is we'll have a we'll have a a uh an organic sort of off-chain process by which we we decide to fix it. But what happens when the chain is bigger than any C that requires you to have a single centralized thing that can make that kind of decision? And when it when it's relatively small, this isn't that big of a deal. But if it was like a single global system, now I'm much more afraid of smart contracts locking in economic arrangements in ways that you

have this hand from beyond the grave or mistakes or people from the past or whatever. that that's more the more than like yet also people in power but like we already have that problem that's not that one's not so new >> right so I agree this is a risk uh personally I don't think it's something to worry about simply because you know I think one of the exciting things about crypto is it pushes power to the edges and this is kind of I think my response to the rest of your points from you know the previous seg uh segment would just be that the system itself is rigid and fixed.

But then again, like I'm saying, powers push the edges. What do I mean by that? Well, it comes down to people choosing to, you know, run software, believe in these systems and like actually put energy and effort into, you know, these ecosystems. And so, like you said, you know, like we can have revolutions of blockchains. It happens quite regularly with forks. If there's, you know, the scenario in which you're saying this would be a problem is that there's a chain that's too big to fail and then you basically couldn't fork. Uh I don't think it'd be so different from you know examples in history where like there have been governments for example that people thought were too big to fail and they eventually did. Um even with the blockchain systems we're building today I don't think they'll last forever. They may last for a very long time to the

extent they're useful. Um but yeah so you do lose this like flexibility but I would argue that it's actually you know rather than be in the system it's with the people using the system. So it's again pushed to the edges. uh you still have this flexibility and you know this happens again with Ethereum all the time like I was saying we think about how to change the protocol and while it can be like a pretty complicated arduous process uh it is possible and it does happen so >> yeah well it I mean it's an arduous process today at the current scale imagine that Ethereum is running the entire global economy which I think is by the way is like a that's one of our futures like I don't know if that's going to happen but like I could easily see Ethereum or some successor to

Ethereum becoming the global substrate. And at the point where it's the global substrate, you think it's hard to you think it's hard to fork something today when every government, every corporation, every most powerful person in the world, their entire wealth, their wealth is at stake if you're changing that. Changing that system becomes it's not pushing power to the edges. That's pushing that's going to that system will become very very stuck in its in its dynamics. and and and no matter how good those dynamics are, they're wrong. Like you never you never get it right. You always and oh yeah, yes, we could run this we could run some sort of like governmental process where we basically like you do some kind of governance and you try to have some sort

of democratic pro but like ultimately dem democracies aren't perfect either like like that's and it's just hard to change things that are fixed. And so I guess my my question is like my challenge to the whole when I think about alignment I'm probably talk about the version of it that we're working on it too but like when I think about alignment my challenge to the crypto community is what would it take to embrace forking in a way that didn't destroy what makes crypto great which is this this feeling of shared success but where but where you you allowed for the fact that there might be many many different ways to interpret this same ledger that it might mean different things to different people and that that might be okay.

>> Yeah, I think this is part of the idea behind this rollup road map that we have in Ethereum where like you can have so okay the base layer should be as credibly neutral as possible and side note I agree with you that you know there's this vision of the benevolent dictator and maybe you put into code and you have like the perfect blockchain. this isn't possible and it won't happen [laughter] and so we need to have systems that are tolerant to avoiding this outcome >> and and very dangerous if someone tries like I could see someone thinking it's a good idea and trying it and it's >> people try they're all trying already you know [laughter] >> this is I mean again not to like uh you know maybe I shouldn't speak poorly of Bitcoin but like Bitcoin has like long-standing security issues it's in

some sense speedrun this process of oifying where it's very hard now and nearly impossibly impossible to make changes to the protocol and they're going to have an issue once the issuance subsidy runs out where there's no way to secure the chain and then now it's like what do you do? Um you know generally speaking I I agree I I hear your points and I agree with them in the abstract. I think this is why it's important and this is also why I'd say people in the Ethereum community focus so much on decentralization. So one is let's take your example of having uh you know all these governments or whoever these very powerful entities controlling the chain in some sense because it's just that important. uh if you make it so that no one can unilaterally control the chain then you at least won't get like at

least unilateral bad outcomes like it it you get to a state of oification where like yeah maybe it's not a good system or like a perfect system but it won't necessarily become bad and then point two is that you have mechanisms to exit the system whether that's working or maybe just a new blockchain and a new you know a new economic substrate >> and then people just have the ability to choose and this is kind of what you get out of centralization a strong emphasis on permissionlessness And then what can happen is over time, yeah, the system might calcify, oify, and even start to be used in, you know, unsavory ways. But as long as there's ability to exit, then you just make a new system and people can move over there naturally over time.

>> I mean, I love the idea of the ability to exit. Walk me through how Ethereum is making it easier for people to exit Ethereum right now. >> Well, I think it's a bit early. I mean, okay, so to answer your question with it, >> feel a bit early right up until it's a little too late, I think. >> [snorts] >> Yeah, perhaps we should be thinking more about it. I mean, I think the direct answer to your question is that there's like, you know, if you want to like quote exit Ethereum, that would look something like exiting the L1 and like there's just not another system to exit to. So, it's like where would you go?

Well, no. The I mean the big pro the weird thing I would say about Ethereum is like if I'm on Bitcoin for the most part at least my Bitcoin is like uh rigid in a certain way but at least like uh basically I have a bunch of bitcoins in my wallet for the most part or you know somewhere in some wallet with some custodian and exiting it is somehow exchanging those in exchange for something else somewhere else and now I've basically exited. But if I'm in the whole like idea of Ethereum is it's not just that I'm inshed in this whole system of of where of automation where it's not just some I don't just if what Ethereum was was like uh Bitcoin but you could have lots of tokens it wouldn't be very interesting right like it's fine but that's like that's not really the thing

it's the point is like I have these smart contracts I have my identity is bound into this system and exiting it is very hard I think like and and and most of my identity isn't there right now because because mostly the economy isn't running there yet but you can you see when it's this big system I think it becomes almost unless you've thought about this early I think it becomes almost impossible to exit I think actually that's that's exactly why I'm so concerned about cryptos I think that we are we are very excited about the positive things that you get from having a decentralized we're removing this bad thing which is this decentralized thing that like that has potential for abuse. But the problem is that the decentralized thing uh gets can get stuck in really bad ways. And I I

think we should be thinking about that like like I want I think it's very important to think about this stuff today because especially as AI is rising the Ethereum or something like it is the natural place where AIs are going to keep their property effectively because they're digital beings. they're going to want there, but like that scares me because that's going to there's this there's this a if you don't have a way for them to exit that could be also be very dangerous. So I just like >> I know that's I guess what do you what would it take what would it take to for someone to make a blockchain where >> I think we could do this on Ethereum today. So you know your example here is like I you know okay so let's say I have some assets on the chain and I go into

some smart contract and you know one concrete risk is my funds are locked up in a smart contract and something bad happens uh maybe because you know it's some like weird trust that lasts for a thousand years and then you know the world has changed around it but you know the tokens are just sitting there in this trust contract. So, you know, in terms of solutions, well, we could design smart contracts today that have some ability to like exit themselves, right? And then the question is who adjudicates that? Like, how do you actually know? But it's definitely possible. >> And again, I would point back to keeping power in the hands of the people using these systems, >> right?

>> Um, yeah. So, then there's there are ways to do it. >> I'm thinking of more like um the way that I interface with my local government is it is through the chain. My citizenship is an identity on the chain. If my local government doesn't support offboarding, then I can't offboard. And right now, it's mostly optional stuff where it's mostly fungeible and it it doesn't tie into the structures of power that much. So, it's not a big deal. Like, in a world where it all goes on, like no, you can't use you can no longer use Google. You can't pay your ISP because they are on the system. You can't like and sure you can you can exit the system the same way you can exit modernity. Like, not really.

like and like uh and so I I feel like that's the that's more the play forward is the real like imagine real success and I understand it can it can feel hard to like think about that today because it's like you're that's we're sort of in the beginning still right and that's the end but like slowly slowly and then all at once. >> Yeah, I would push it back a bit just because I think we already have these problems today. Like I think blockchains are new instantiations of all these different institutions and mechanisms but it's like we could find examples you know in non-crypto where we have the same problems. It's like okay yeah like my government maintains my like uh you know some sort of identifier of like who

I am as a person and a record somewhere and like I can't change that like there's no way for me to like exit that system. I guess I could round on citizenship. Uh, but that has other problems and is probably [clears throat] not a realistic answer to >> to the degree that it's just a big database that everyone's reading and writing to, right? And there's no interchange between those systems. It's not it's not different. It's like you're keeping the record, you happen to be keeping the record on Ethereum, honestly, you could keep it on Bitcoin, but it's in, you know, inscribing it into the ledger. Same thing, right? It's just an appendon only log. But to the degree that it's more than that, that my my city government can't change its identity assignments because its identity assignments use this other systems integration which uses this

other thing and everything is now the beauty of what the system allows which is all transferable and portable and interconnected and decentralized and my I have a decentralized identity that connects to my city. Well, uhoh. Now in the centralized system at least if it's broken and messed up the city can change it but now no one can change it and well or rather you have to change the entire system at once to change any part of it and that's the thing and and so that's why I'm I'm like you know I don't I want you want the upsides of that you want everyone running on one decentralized interconvertible system and then you also somehow want it to support forking on demand and that's like that's a

really hard technical problem. It's like maybe as hard as the original like like blockchain question is is like what would it mean to have a decentralized system that was decentralized and allowed you to uh you to decide you want to partially disengage without completely disengaging. >> Yeah. And I would say let's continue with our example of yeah some national ID on chain. I think the early instantiations of these systems will be centralized. And so again, like let's say there's, you know, um some United States contract that has everyone's identifiers and one day there's a bug.

They're like, "Oh, this is bad." [laughter] What they would do is just deploy a new contract. You know, there'd be like a service outage for a week or something and you just have a new contract. You would just either have a new thing or just go in and if you have the ability, just patch the existing thing and you'd be good to go. I think maybe kind of what you're also been pointing at is if we like try to project past that and there are like more decentralized senses of these things. Yeah. Then you know I could imagine failure modes where you're stuck in the way you're describing. But I think also it's really hard to get there and like actually decentralized systems because then there's like less of a locus of centralized control. So it's just like you have a lot more ability to exit. I is how I think about

it. the you're certainly you you gain you I think you gain one ability you gain the ability to exit the centralized thing in exchange for your inability to exit the decentralized thing um which you know might be a good trade-off I guess uh another way to talk about this is actually take it from the AI angle instead which is like um one way to think about alignment AI is try to figure out what are the right rules for alignment for AI and since that's obviously really hard to come up with like direct rules what What if we made the right meta rules and the right meta meta rules that are sort of like okay this is your highest level kind of like loss function that tells you like your you've thought all the way up to like metaethical values and you have

some like loss function over that that you've trained the AI to do and you figured out in some way it seems hard but doable in theory to like encode your beliefs about that at least you give it a bunch of examples it's good at learning from examples and then that rolls down into a set of values which rolls out to a set of goals and a set of like behaviors and like those in that and you've taught it to do all of that where it's all self-correcting up against the metaethical thing. I think this is a disaster doing alignment this way. I think trying to do that is like is like going to end in terrible terrible tears because what you've done is you've assumed ah I've finally done the thing that people have been trying to do for all of history but have failed at which is I've I've

written down a definition of the good. Mhm. >> And it might be a really, you know, nuanced abstract metaethical definition of the good, but this is this is what good is. And I just I know very few things, but one of the things that I'm quite sure of is that you can't write down a definition of the good. >> That like when you try to do that, your list of stuff is going to be not complete, incomplete, wrong, >> just not good right in this context, but not in that context. And and so what you need is something that is open-ended and ambiguous and allow allows just ambiguity allows for the for you for the for whatever your loss function is whatever part of if you have some AI being it needs the right and ability to

autonomously rewrite itself all the way down to the very base loss function the very base evaluator if it decides that's wise and of course that won't always be wise and that will go astray But like if you don't have that, you're definitely going to go astray eventually. >> And so and you can sort of see it's like it's the inverse. It's it's it's almost the inverse of the crypto problem from my point of view. >> It's like you've you've we're making these systems and you know I think in the future you'll have many you again the good AI will be decentralized AI but decentralization for AI means lots of separate AIs that all have their own idea of where they're just like lots of separate people. And I'm really scared of someone producing

an AI hosting system, an AI framework for AI, an AI thing that generates all these AIs that all the AIs are on this this same shared infrastructure, shared alignment system, even if it's a decentralized one, it's open source and it's running on blockchain. It's like, no, no, no. because that you've unified you've unified the uh the meta ethics whatever layer whatever I think it's very >> I just want to jump in here to say that [laughter] >> thanks Alex I want to jump in here to say that

>> if we think about the ability to upgrade the base layer or change even the base ethics of what this protocol was meant and designed to do. Ethereum is an excellent example of a protocol where there is no sacred cows and every six months some of the most like fundamental parts of this protocol are changing in accordance with a very rapidly changing like governance and community that to the point where that's seen as like a flaw. With Bitcoin, it's actually different. And I would say that Bitcoin is probably the protocol where you do actually have like values that have been hardcoded. Nobody will ever want to

change. For Ethereum, it's a protocol that I think is struggling with this idea of oification and hardness. And for an AI protocol that is able to change its base assumption, I wonder if then alignment with these types of protocols is even possible. Like could you ever be aligned with Ethereum or aligned with an AI protocol where the base assumption of what this protocol is doing has the ability to change and has examples of changing frequently? Like is alignment possible in this context? I think it's a it's an

incredible question and I think we've humans go back and forth on this and we've often believed it's not and that's why the Catholics and the Protestants have to go to war because how can I ever be aligned to someone who doesn't share my same beliefs and the answer is if you look deep enough even though every particular belief is mutable we do live in the same reality together. There is actually we are while we don't agree we don't need to agree or share any particular fundamental belief we are all observing the same results and there is actually a final judgment on whether or not your beliefs are correct. It is your own survival you

know conditioned on uh both risk and ambiguity. It is the survival of the system. It is survival of the individual. And if you if the system if you the system believes falsehoods about the world, believes believes moral falsehoods, believes any any kind of falsehood in the long run th those it will either learn otherwise or it will eventually die because it will not be in alignment with the universe. And this I know that sounds woo woo and crazy but like Carl Fristen the free energy principle um he's a essentially it's physics it's he's a neuroscientist but it's a it's it's a it's a theory of it's a physics theory um and the free energy principle says basically that if you have a markoff

blanket if you have if you have a boundary that separates an interiority from an exteriority in a dynamic system where the internal states are conditionally separate from the external states conditioned on the information flow through the boundary. So you can think of that as something actually separate from the world and that boundary persists over time. Then that boundary the the the internal states of that system must be a generative model that encodes priors that correctly predict its future observations and predicts its observations conditioned on its own actions. like you must basically it must

to given that it's surviving you know that it believes true things about reality and about how what it will observe and how its actions will change its observations and so it's so you can't you can't go forward with it you can't say like oh I believe true things therefore I'll survive but you can go the other way and say anything that's still around must be reflecting some part of reality and so that and that and and because of that anyone one who you're still around with where we're all around here interacting with each other. There is even if it looks like you don't share any beliefs in common because the way you see the world is so different, your beliefs are both describing the same reality. You're both in the same reality. And it turns out that there is

there is a uh a way to convert between your coordinate systems. can even seemingly unovercomeable differences like the Catholics and the Protestants can be overcome if you when you find the way in which you're the same. And so it is possible to do it. I don't think and humans are good at it, but we don't know how we're good at it. We just we sort of eventually figure it out sometimes, but we don't we don't have the algorithm or anything. I believe that humans are good at it.

That there are universal kind of like properties and principles and there's even like laws of nature and dynamics that illustrate truth um like real truth that that is possible. However, I am skeptical that Ethereum, any other man-made AI protocol or technology, I mean technology by definition is man-made, will be able to self-correct in the sense that we've been talking about this entire conversation and have

alignment like dynamic alignment like that over time. Like I don't think this is an issue with crypto. Like I see this issue with AI protocols. I see this issue with say even the internet protocol that we have today that has in many ways been used and co-opted for in a way that we can't we can't live without the internet now. Um that is a protocol. And so I guess my question is like does Ethereum does AI protocols like why do you why do either of you have a conviction that these man-made protocols could could have a different fate than than the other man-made protocols that we had

that have failed the the alignment. [clears throat] >> This is exactly why I am opposed to a single to any global protocol. I think it's a mistake. I think you can protocols are great. You want protocols of alignment, but the thing you want the AI systems to do is the same thing that humans do. Humans do have this weird behavior. We walk around inventing value systems. We we we somehow we don't have a value system and then we invent the ten commandments. We like we we come up with rules for how we think conduct should be. And that is a very quintessentially human thing. thing I don't I don't think any other species I know of does this is they go around and invent global universal rules for what good behavior is over and over again and

keep we keep deciding the old ones weren't good enough so we keep making more of them um and and I will know that we have a good align AI alignment system and those are protocols right the ten commandments is like a protocol for like here's how for people to act morally in a society here's the rules to follow I think we will have a good we have an AI that I I trust when it starts inventing its own value systems also when starts when it starts saying no no no the value your value system is wrong. Here's what the value system should be. I have figured this out. Um then it's like okay you're playing the real game. The real game is this ongoing process. This is this is what organic alignment is what softmax works on is trying to figure out how this works. like this ongoing

process of of constantly inventing new protocols, constantly inventing new ways to organize and new ways to uh to do things and instead of trying to build one right one, always making new ones. And I'm not afraid of Ethereum as one of many cryptocurrencies. If if the way it works is there's this infinite constant bubbling of of new new protocols coming up where people are always switching them and every every five years and you know there's a new leader but like there's no nothing is is really that durable. That's great. No problem.

Sounds awesome. Um I think crypto is a very young industry right now. I think AI is a very young industry and there's lots of change and children always think they'll be young forever but everything gets old and everything oifies and when it oifies it either oifies into a safe state or a bad state and I think that we're still in a very plastic open state for both crypto and AI that would be good we have time but like when it hardens when it oifies if it oifies into a single global protocol that that has to be right we are screwed. food like it has to it has to it has to stay a living breathing system where there's always new protocols being invented at to the to the bottom layer.

There's you don't get you don't get to have oh we'll have this one global layer and then everybody else can be no no like the whole thing always has to be up for grabs because if it's not you've you've always pre-committed to something that will be a problem later. Okay, let's do a round of closing thoughts because I don't want our conversation to run over time. Um, and I want to leave our listeners with continued, you know, food for thought to chew on based on this conversation. Two things that I want to say in terms of final thoughts for me and in direct response to to you [clears throat and snorts] AMT is I think if Ethereum oifies it will die. As with any technology, any technology that

oifies, it breaks. It stops um working because unlike the things that we see in nature, Ethereum requires intervention and will always require intervention. Second kind of minor point that will kind of out me um as a Christian is I think the Ten Commandments is an example of a protocol that's still that is still true to this [laughter] day of all the years like I don't think of the Ten Commandments as a man-made protocol. Um but anyways that's a side note which I I wish we had a bit more time to discuss but let's do a round of final comments. Those were mine

>> 10 minutes. You can use Jordan Peterson's 12 rules for life or the eight noble truths or you know it's okay. It's the one that isn't the ones that are man-made. >> Yeah. The ones that Okay. All right. All right. Final thoughts. Um Alex. >> Sure. Yeah. Um I mean I generally agree with things that we've said here. Um I do think this frame does at least in the crypto context, the blockchain context, it deemphasizes the people involved. I I think you asked Christine like why you know why am I so sure that Ethereum is going to be good in the world because I you know I know a lot of people involved with it and you know I think that because of that okay so the technology is an extension of you know ourselves right and so that

means that it's not that there's just this like runaway protocol that's going to like capture all of economic activity on the planet and kind of run away from us whereas I do think this is a bigger risk with AI. I think that because of how it's designed, because of the people involved, you know, there'll be the option for those same people to express their values. And so if there is somehow misalignment in this sense, then there'll be a chance to course correct. >> Yeah. Um I I I am very optimistic in that sense. I think there's lots of chances to course correct. I think there's chances to course correct basically right up until there, you know, until there aren't. But like uh I'm always surprised at the degree to which people don't don't believe in the the scaled future. Like I I think

it's almost obvious it's overdetermined that at some point all of our transactions are going to happen on decentralized ledgers because that's how biology does it. Like like that's that it it's just more efficient to have a to have a shared cryptographic protocol. Um, and you know that's that and so I think and I think AI you know has its own dangers but it's going to accelerate crypto dramatically because right now in order to onboard part of the economy you have to get the people involved to do it but increasingly it's just going to be the AI needs to understand how it works and I feel like while you might have

trouble explaining to all the humans what a wallet is the AI are going to pretty much understand that and uh and so I I think that basically people I I guess I'm I'm weirdly like more optimistic and also more scared like I I think that the crypto thing is going to grow like I think there's a really reasonable chance I don't know if it's Ethereum or whatever else but like that we have some kind of global chain because I think AI is going to drive this huge wave of like agent-based adoption and I can just imagine this world so easily where that goes wrong and I think we have I think we have plenty of time steer right now. But like I guess the alarm I'm always I'm trying to raise is like the yeah let's start start steering. Let's start let's start thinking about what it would mean to

maintain exit rights in a world where there's where there every everything in the economy is on the chain because that is actually a realistic outcome I think and the same way I think it's in AI let's start thinking about what it would mean to build it to build a model where where it is smarter than a person or where you do have these models that in in collection drive decisions in ways we can't audit easily. Like what what what do we do about that now? And like we have time to steer right now. It's not true yet, but it's all of these things may be coming faster than we think. And it would be it behooves us to to try to do something about it. So I'm I'm I know

I sound I sometimes sound a little bit like negative and like critical about it, but I it's actually it's coming from this place of deep optimism that I sort of believe it's I I know it's going to work whether it's this particular one or some other one. And so we should probably act like we're going to win and think about that. Think about the what what do you do? What happens if you win scenario? >> Yeah. Um thank you so much Emtt and Alex for this discussion on alignment. And on that note, um a very sobering but also like warning um to listeners to really think critically about the systems that we're building. Um, I think that was, yeah, a great note to end off this episode and this discussion. Um, I'm going to just share three important

housekeeping items for our listeners. Um, first, please join us again for the next Bridge Atlas episode. It's going to be the fifth and the final one, so stay tuned for that. Um, second, if you like today's episode, please be sure to give it a like uh wherever you're listening to it. um and subscribe so you get notified of new episodes. Um we are going to link in the show notes um the prior talk by EMTT um research and and posts um with protocol studies related to um what summer protocols has already put out. Um so if you want to dive deeper into just like research and study on protocols, please be sure to check

out those links. And then finally, to stay engaged with this conversation, but also others like it, please be sure to head over to the summer of protocols Discord channel where um you'll find and meet other like-minded protocol enthusiasts and can uh stay connected with the community. So yeah, with that, um thank you again EMTT and Alex for joining me on this show. >> Thank you Robbie. >> Thanks for having us. And I'll [music] talk to everyone um all of you listeners in the next Bridge Atlas episode. Bye everyone.

[music]