Stateless Ethereum

Transcript

thank you all for coming and giving me to some number of minutes of your lives never seen the thing of your whole life measured in weeks and like one chart it's awful all right today I'm going to talk to you all about stateless etherium this is the best thing happening that I'm working on and I that's not the right direction there we go woohoo I am Piper Miriam I work for the etherium Foundation I run a team of Python engineers we call ourselves the snake charmers this is snakey Mick snake face you can find me on the internet here or

everything I do is open-source everything my team does is open-source it's pretty cool all right high-level overview at the end of this talk you should all understand at a high level what we're doing why we're doing it all of these things and you should probably have some questions because I'm not going deep here I'm trying to give you the like thousand foot view while diving in here and there but to give you a very like good picture of why we're doing this these these questions I so when I talk about say lysse theorem we need to define what we mean by state I think a lot of people in this room know this but I'm gonna go through it okay so here's the state we have for every address account contract whatever

you want to call them in the etherion blockchain we have these values a balance how much ether do you have some code if you happen to be a contract are there any contracts in this room we've got storage things like that for your fancy token balances and we put these into a tree sometimes called a try and this is what we call the etherium state we have this fancy cryptography thing called a State Route that lets us like refer to this using a very small amount of data to refer to a very big amount of data in the blockchain we have blocks blocks are made up of transactions and transactions modify the state here we have a block with a transaction where Alice sends Bob one ether when we execute this block we

change some of the state Alice's account has one ether deducted Bob's account has one either added Vlaar law we have executed this transaction and done a state transition we've modified the state so we have this concept of a witness and a witness is sort of the subset of the state to the minimal subset of the state that we need to be able to do some sort of state transition we call it a witness instead of calling it data because it's sort of like a more official piece of data because it isn't data that you have to trust it's data that if you have the state route or if you know what the state route is supposed to be when somebody hands you this data you can you can prove for yourself you don't have to trust them

you can proof yourself that it is the right data that it's the appropriate data so Alice can't lie about her account balance in a proper witness when we talk about stateful clients and stateless clients we'll treat this sort of as a binary choice right now stateful clients have a state database that has the entire state tree they all of the the data in it and stateless clients really just use sort of like the this concept of a witness a minimal subset so they use the witness as their database we'll talk about a little bit about like where does the witness come from and how do they get this but like for now it's not super important to the idea that the high-level idea is important stateless means that they don't have a full database of the state but the cool

part is they can like there's no security trade-off here in the sense of both of these types of clients are still verifying the data in the same way which is a pretty powerful concept there's we'll get there so when I talk about stateless etherium what I am talking about is a broad effort that's happening right now with a large group of people across many different client teams and organizations to modify the etherium protocol as a whole to support this type of client in a very first-class way so we want to make it possible to run stateless clients on the theorem network in a very efficient way and we have some

good reasons for doing that and there's some really good extra things that we get by doing that and that's a lot of what we're going to talk about today who is working on this here's a slide with some teams and things on them this is not a complete lists and like slowly building this slide out but what I want to point out is that this is not like any theorem foundation effort this is not an effort by the NGO team or the Trinity team this is a very broad cross organizational effort that's happening involving a very significant slice of sort of like if you want to call it the core development community or whatever um it's cool to see this level of broad collaboration happen and this was

largely inspired by the collaboration that's happening in the eighth to ecosystem you have like what do we have anybody six seven eight to clients I don't know but like they all are getting like together and working together in this like really cool way to collaborate on a spec I said we should do that in eighth one and so we kind of are um okay so like why are we doing this um this is the business model of aetherium hey once store forever there are no economic incentives in the etherium network to limit the overall size of the state the logical kind of train you can follow

here is that the state growth is unbounded which means you will need bigger and bigger hardware to run a client if you have to have all of it which means people are lazy and they fewer people will run nodes only those that have a strong enough incentive to overcome that kind of Hill that you have to get over which means that we will have a further and increased dependency on centralized providers it's not so bad today we've got like a couple of thousands full nodes on the network but it could be way better statement that's like a year and a half ago I'd really like like this guy he's like really excited well and this girl like somebody may have farted near her or

something sir was this like thing to try to fix this fundamental problem of state growth a little while back some of you may have been part of that or remember it it ultimately didn't go anywhere and my opinion as to why is that it was a really hard pill to swallow it was addressing this and fixing it at a fundamental level by adding an economic bound to the overall size of the state by requiring people to pay pay ongoing rent in order to store things on the blockchain ultimately this fundamentally changes the paradigm of smart contract

development there did not appear to be any way to do this without making a like very non-trivial backwards incompatible change the EVM which as we've maybe seen before could lead to a contentious hard fork I set my magic clicker down so this didn't go anywhere now we're doing something different we thought say lysse theorem so how are we going to do this this is probably the lightest part of this talk I'm not going to get into details but this is the part that I would love to talk to you guys after and there are two talks after this one by Cena to talk about code moralization and one by Aleksey to talk about turbo goth and other sweet awesome magic stuff but

at a high level we are gonna make changes at the EVM level to account for the gas costs of witness generation and transmission we are going to include witnesses and sort of like the core consensus protocol which objectively means something like putting a hash of it in the header and we are going to be looking at the data network as a whole because the data network is going to have to serve kind of like a very different type of need and demand like today you just work under the assumption that everybody else is a full node and it works because that assumption generally holds true as we open the gates up the stateless clients the kind of like behavior of nodes on the network and the usage changes and so we have to really address that and I'll point out that I've included things like phones and browsers

and raspberry PI's and sort of the network level because one of the things that we can get out of this is like broader inclusion and I will talk about that some more here the why um I think is acts akin here Arius you walked in just in time see this is this was your question right why are you guys doing this why okay so you'll notice that I use the term mitigate state growth um we do not solve the unbounded states earth problem here we just kind of figure out a way to like come at it from a different angle and say well let's just make the state not your problem

we'll just make it somebody else's problem the idea here is that you kind of have the spectrum the more say flow client is like the beefier the hardware and the more complex the client needs to be the less stateful generally write or hardware and left's less complex piece of software and so the idea here is that let's fix this from a different angle we can still fix the fundamental problem at some point if we like can get there and figure out a good way to do it but for now this is a way to at least push the problem a decent way out if not actually make it not a problem anymore is that it gives us a path towards migrating to East two or eath to land we do not actually include the migration to eath 2 in our roadmap but we know that

this brings us a lot closer because eath 2 is stateless and we'd really need to get eath 1 there if it's gonna like really be part of that network so we get closer to e 2 and get a lot it's a lot more reasonable to expect the e 2 migration be easier in a stateless paradigm this is my favorite better client us today who who has a node that synced right now that's okay but it's like not half the room even um I don't running a node sucks it takes up ten resources on your machine it takes a big machine to run it

all of these things um and like if you turn it off for a day you have to wait a while for it to catch up and that's hard um if you like you're just getting started you might have to wait like a whole day or so for it's a totally sink in the stateless model you with a bunch of sort of asterisks here but you can download the client run it and as soon as you have your first pure connection have a fully functional node and that is a powerful ux change it opens up a whole different set of use cases rather than this like piece of software that you have to like maintain as infrastructure and it's hard and whatever um we see this as a way to open up the gates for more and different types of clients when

client complexity is high we don't see a lot of clients we've lived in the Geffen parody show for a long time and now we like live in sort of like the guests show um and that sucks I've been trying to put a client on the network for like three years now and we're like only just getting to the point where like we are like right it right at the like okay we were there we're like almost on the network it's hard death is full of optimizations that make it work and it takes a long time to build up that that software and by making it easier to write a client by making clients less complex to write we can get more of them written and on the

network and let people try things out um similarly if it is hard to run a client less people do it if it is easier to run a client more people do it so we can get bigger and better network participation under this model if it's if the hardware requirements for participating in the network are lower also we can get rid of the one-size-fits-all model uh right now you have one choice the full node and that is your choice however there's a lot of use cases and we can have clients that are really well tailored for these people and this is just drumming home that point of like opening the floodgates for more clients to join the

network we want more people to come in we want more clients participating we want more types of clients I want to open it up so that browsers can participate as first-class citizens in our network this is like a stretch goal we are not committing to doing this this is my like version of the world this weekend we're going to work on accumulating a shared version of the world but I believe that this is a very tangible and realistic goal for us to do imagine being able to put meta masks on the network as a first-class citizen that's a million notes that can come online and help out with this network it's like a really big deal all right when this is a like mediocre timeline

right now we have three clients who are actively working on beam sync beam sync is a version of a stateless client that eventually becomes a full client that second part isn't very important right now very soon we will have a definition of like how we can talk about witnesses and that will let us start passing Witnesses around to make in sync work better and start experimenting with witnesses in the wild as a whole the research effort should be done with the like kind of research and figuring things out problem by the end of this year and we expect the overall like implementation and rollout to take around a year these are rough road maps

this will be refined as we go there's a hard for hiding in here somewhere as we make some of these like fundamental consensus level protocol changes may be more than one we'll see ok I've covered the like who what when where why thing and when I gave this talk last it was very clear that I missed a giant point because all of the questions if you've been following along and you're really paying attention you already knew some of this stuff you might see or have the question how so where does the data come from how do we actually support this how can we possibly make a network in which all of these clients need data how does it not crush the full nodes under their load we are thinking about this this is

what it's like to be a full node you can imagine like piling a hundred more tiny piglets onto this full node and and if that's what every full node just has like you know a hundred pigment piglets like vying for a teat the the pigs go on strike the natural self-preserving behavior here is to be selfish with your data that that like if you are being crushed under the load of all of these people demanding things from you eventually you shut your doors and you like hide yourself from the the mob of demanding piglets um so I've been referring to this as the data retrieval

problem you have all of these you know new nodes on the network that needs state I mean I'm gonna kind of like forgive me if some of this seems pedantic but like there's important concepts here so we think we can think about the network in terms of capacity and demand the green nodes provide capacity the the syncing nodes are like ten of the source of the demand there's a bit of an oversimplification here but the overall point holds the demand the one important part is that the sinking nodes eventually sink and become part of the capacity this is the node we live in today this is the world we live in today which is great because even in a situation in which the demand outpaces capacity eventually these guys move over and while the

network may like bog down and slow down for a while in today's world eventually the capacity fills up and eventually everybody gets served and gets what they need if we introduce stateless clients to this which I will point out you can do on the network today you end up with the same sort of balance except for the pink nodes never cross over they never become full nodes so they never become part of the capacity which means that there's always a tipping point in which if you add enough stateless nodes the thing tips and it never untypical the stateless nodes can't get the data

that they need and they generally kind of stop working the sinking nodes can't get that that the data that they need to become full nodes and so they can't transition over into becoming capacity and the full nodes get crushed by baby piglets and that's just like it's bad for everybody it's not just bad for the stateless guys school so like we're in a good spot right now and we could end up in a very bad spot and maybe even never recover or something like that so I'll point out that the network has this problem today so we have ideas on how to fix this they're efficient there I have them

divided into two categories the first one here is the one that that like we we kind of know this if you can increase the efficiency of the protocol that these nodes are using to transmit the data you can efficient you can effectively move the fulcrum of the lever over which means that you need less full nodes to supply the overall demand this is like a thing that we can do on the network today this is an effect of the most concrete parts of our roadmap witnesses are a version of this right now if you make a stateless client on the network it is horribly inefficient it issues a ton of get node data requests and things like that and it and it's heavy on the on the full nodes but if we can like move that

fulcrum over and increase the efficiency then we can at least support more demand with fewer full nodes and that gets us some runway at least my preference is to eliminate the asymmetry in the network entirely and I think the best way to think about this is that today for you to be a valuable network participant you must be a full node or like very close but for simplification purposes you must become a full node in order to be a valuable network participant and to provide on the capacity side if we can lower the bar to entry for providing and helping with the network low enough then everybody can help out and we end up

with sort of a more BitTorrent style situation where even the people who don't have very much can still assist everybody else this is not an utterly I'm not I'm making no claims of the simplicity of this problem but in my opinion this is where we need to go to achieve a network that's that can support millions or billions of devices it's not to say that we have to go here now or immediately but in the other model it's still always tips eventually and because stateless clients are that much lighter and have such a better UX and all of these like really nice things it is not unreasonable to expect them to outnumber the full nodes by heaps and bounds and

so I would love to get to this solution where we address this problem had it's kind of like fundamental core layer how many of you in here work on this stuff full-time all right that's like a third of the room cool and by this stuff I mean like Ethier iam ecosystem stuff yes sir alright then my pitch is boring and useless because you're already doing this crap and maybe getting paid for it I love being part of core protocol because I love open source and doing the like I don't know it's like I I don't

like getting caught up in product land and like I'm glad that there are people doing product land stuff but like I don't have the stomach for it and like this is that's the most powerful lever I have to move the world and I like doing it a lot but this whole effort is not constrained to any specific organization or people or skillset so if you if you want to be part of the like largest overhaul to the etherium network that we will have done to date come join us talk to me trying to call they opened a pull request start just do I don't know that's what I got [Applause]

questions where do you have a question wait for the mic I guess like a natural question for for the acceptation that there will be lots of these light clients I guess similar questions were raised on the Bitcoin side a long time ago that do these light clients actually validate right I mean that underlying underlying reason for consensus is the double span protection and so on so if you only have these witnesses and you can see that okay so these transaction was okay that was interested in but if you don't if these white lines don't actually do the work of validating all the other

transactions as well our current model is that there is no difference in terms of validation between a stateless client and a full node that's holding the whole state there they're doing the same level of love block verification as a format so I was curious if you're familiar with the light client work that the madam asked em was doing like a year ago an ekala exactly that's like relevant or like the same thing yes and yes I had a talk with Dmitry yesterday I like had this like data this this sort of like remove the asymmetry idea and I've got a

thing building in my head and just been writing it down and when I talked to him he was like yep that's what we were working on so so yes I I've I've been talking to him and planning to pick his brain because I'm sure he's already gone down the rabbit hole sand cannon will help me avoid some of them so yes its I haven't but meta mask is on my like I need to talk to you guys list I just wanted to say that so my so you remember that picture with the piglets

and a pig well in my belief the situation is actually very different there imagine one piglet and ten pigs and in order to satiate one little piglet you have to provide them the access to ten pigs and this is actually what's happening so you need many more that welcome to full notes to satisfy one so basically if you even if you have half half it's already not working I don't know how to depict it but what is basically the picture implies that a single note can sync off of a single Pig and the idea here is that in order to sync today in the network you actually need to be distributing your requests across multiple nodes in order to

saturate your incoming bandwidth although each of those nodes may have their outgoing like state reads fully saturated so it so you actually have to distribute your request in order to like fully sync at the highest speed I can you just highlight the difference between a light client now and the stateless kind we're going towards yes okay so we have the Elias Network today um I didn't mention that I there's a that there's a bunch of complicated stuff we can get into about like why it

isn't reliable or if it's reliable or all of these things we're not using the term light to talk about this but in my understanding of where we're going and my version of where we're going this supplants the need for a light network by allowing extremely light full clients that are that are that are equally as functional but and also equally as light but just be a different mechanism and and in my opinion a better mechanism because they are still doing full block validation they could choose not to and at which point you have an

even even lighter client but that's the general idea so just following up on that question [Applause] does that mean that once you have a client that submits all the transactions together with the state Jeju emphases and you can download everything where the the witnesses that they belong to the blockchain actually the white elias is obsolete i believe that this has the capability of making elias like what's the point there's probably still a use case for that like it doesn't preclude the option

of elias but I don't see the motivation to continue there if you can do everything that you can do under this model on the same network without needing a benevolent provider of your data which is what Elias requires a benevolent provider of data you how will the transition from statefulness to stateless look like it'll probably begin sort of in the experimental background with clients like like right now Trinity Basu another

mind are all like working on a beam sync implementation beam sync is a stateless client and so will be like experimenting with this stuff but in a context we're like it's trivial if you want to knock us off the network by building a block that touches a ton of state that's like massive when we like make a witness so the work that we're doing over the next year and a half is to get those efficiency gains to drop the witness sizes down and so there'll be stuff rolling out along the way because we the beam sinking like clients who want to get on the network and do a better job of it will be experiment like will be the like fast thought first adopters of that stuff and like using it in a

production environment the things that we can't do on our own are the forks and things so like it'll be easy to lie to us about witnesses or to give us a bloated witness for a while until we get a witness hash in the header and that's something that's gonna take like formalizing and making sure that we have the right concept before we bake it into the protocol similarly even once we we probably can't put a witness in the header until we have a good well understood economic bounds to the witness size and so having a direct tie between gas limit and maximum witness size is sort of a requirement to know that attackers can't blast the network

with a giant witness because we don't want we want a stateless client to be able to know that they can get on the network and stay on the network and not get kicked off by like a rogue giant witness even if it costs that attack cost money you still have to fill a block with like broad state access but it's still not something that we like can accept as a like status quo whatever I have a question about in the world where it's very easy to spin validating thin-client is that once we have a stateless clients which are very easy to spin and they are

validating could be any problems with a big number of malicious clients doing something on the network um yes but it's in kind of no different than today like you can make a stateless client on the network today if somebody wants to take down the eath network like you can do it today with a decent bit of work it's not like trivial because you still have to write the code like get you on the network but like the network has this this problem today of too many leechers and the network falls over and no solid heuristic for being able to differentiate between sinking nodes and

somebody who's just like sucking up capacity to be a jerk that's already happened before before like the stateless nodes are just following along like the miner has all is already like made the block and there's nothing stateless happening there what is the computational overhead for generating such a witness and will miner have an incentive to create simple blocks in order just to love this create an easy witness I love this question okay so you pretty much already do 99% of the work

to make a witness when you run a block because the difference between that and making a witness is just keeping track of everything that was touched and so the overhead to generating a witness is small compared to to today and so we can make this compulsory by putting a hash of it in the header which means miners have to generate it and then the quit like then the next question is okay and they could generate it but why would they transmit it executing a block against your local state data verses executing a block against a witness the witness is going to be way faster because you're reading from memory rather than reading from disk and block are less likely to get uncle the faster they propagate around a network

and so by sending witnesses out with your block you make it easier to validate them and ease and thus in theory they should propagate around the network faster and even if a minor didn't and send the witness with it any other full node in the network who validates it also does the work needed to generate the witness and has to do it to verify it and they can just start that propagation as it gossips out and then the next part of that question was incentives for miners to make smaller witnesses and we're looking at including in the gasps incentive giving miners some part of the like in theory if you make a block that touches the same piece

of state over and over and over and over and over again it can use a lot of gas and make a tiny witness right that's like the best case scenario for a miner they get a tiny witness full gas usage and the idea is that if we can give the miner some of the overlap there for like you know in theory somebody might pay a little bit for each of those like witness inclusions except for it's already there so we are looking at giving the miner some of that extra as a financial incentive to making the witnesses with the largest overlap all righty anyone okay yes I guess I know the answer but I want to hear it

out loud so how would this affect the app developers there is going to be some form of change to the gas schedule I think it's very likely that it's a backwards incompatible change but only maybe in the same ways that we've made other backwards incompatible like minor tweaks to the gas schedule so but overall this makes no change to the smart contract coding paradigm and it really only affects like smart contractor to app developer if they're doing fancy gas things which is it's not that nobody does that but it isn't

utterly like ubiquitous so it likely has very a very very small effect on existing depths and app developers could there be any positive effects for dev developers I mean if you have to host infrastructure for your depth today you generally have to host like the full JSON RPC interface and run like full nodes but if all that your dad means is like specific state access in this model you can have a node that only has the state that your death means which lightens your infrastructure costs and things like that you can really that applies to individuals as well you can

run a note that just tracks your addresses and like contracts that you interact with so that your footprint on your hard drive is way smaller and in general it makes it easier just to hop onto the network and off and so if you're doing things like live testing or whatever you don't have to like maintain a development know that you can interact with you can do your coding and then turn your note on and then talk to the network and turn it back off and that's fine just to make sure the fork that would be required from what we know now would the only things I have heard so far are there's going to be a witness in the header and some gas schedule changes right or is there anything else that we

would have to include in that fork any body oh yeah we're gonna turn the tree from a hex area tree to a binary tree I don't think I ever mentioned that I did all right yeah and that's like a it's an like it's a non-trivial thing to do but it's a good thing and it's pretty objectively like I don't know I haven't heard anybody argue against it please feel free to if you not now we'll talk after hey Ben hmm that wasn't words I was just it'll help help ease - ah yeah like if we're doing like the bridge to approach

or anything proofs get smaller in the binary tree approach like there's a bunch of kind of like ancillary trickle-down things that just get nicer with with that change okay thank you very much [Applause]