Stateless Ethereum and The Portal Network

Transcript

check check check check there's a really cool talk happening in the main stage thanks for waking up thank all of you all nine of you for being here this morning uh my name is piper merriam i am here with angela liu and today we are going to tell you guys about stateless ethereum and the portal network so here's a rough roadmap for what we're going to talk about today uh we're going to talk about the status quo of ethereum clients what they look like why they look like that we're going to take a little side quest

down stateless ethereum which is an ongoing effort to bring this concept of statelessness to our protocol i'm going to hand things over at that point to angela who's going to tell you about the portal network and sort of how that came out of stateless and the general gist of this talk is that we can have better clients for ethereum and this is basically how we get there so when i say better ethereum clients this is what i'm talking about i'm talking about clients that work straight out of the box that you don't have to wait hours or even minutes for them to sync clients that can run on low resource devices your phone or raspberry pi your laptop things like that and we're talking about clients that still expose the same external facing apis that

everybody depends on and is used to the json rpc api and essentially clients that operate in this model in a scalable way which is not what we have today for lightweight protocol access here is the status quo for ethereum clients today they are heavy heavy pieces of software you're looking at hundreds of gigs of disk space your fans running at high speeds because of your cpu running and hours of time syncing and this is not a shortfall of the actual client teams or anything like that this is not a negative mark on their engineering ability this is something that is dictated by

our networking protocol the client teams don't have a choice clients have to be heavy because that is how our networking protocol was designed it's how it exists that's what we have today our networking protocol is called devp2p this is a peer-to-peer network and within it there are different protocols one of them is called eth that is what all ethereum clients exist on within that protocol there's a bunch of messages that get passed around between the clients that are part of the network things for gossiping new blocks or transactions that haven't been included yet um messages for retrieving history old blocks old receipts stuff like that and some things for

sinking the state although that is actually being replaced by the snap protocol nonetheless all of these things dictate functionality that has to exist inside of an ethereum client that is part of the devp2p eth network and the result is that you get this sort of you must be this tall to ride situation if you want to be part of this network if you want to be a client on this network you have to do these things you have to have a full copy of the state you need to be able to process the entire transaction pool you need to have all of the chain history stored and these are what make ethereum clients so heavy today this is something that we want to change and it's unfortunate because your your

options for interacting with the ethereum network fall on two very far ends of a spectrum running your own client which is very heavy or using something centralized like infuria alchemy pick your poison um but they both exist on opposite corners of this graph between centralized and decentralized heavy and lightweight and the only choices you have essentially are on very opposite ends of the spectrum and when i say better ethereum clients what i'm talking about is this adorable pink smart car in the upper left hand corner that you cannot have today because you can't build it and this is the limitation we are working on lifting anybody who is familiar with our

protocol might ask what about les les is the light ethereum sub protocol it has existed for many years and it has failed to deliver a reliable lightweight way to access the protocol and the reason for that is because it's built in this kind of fundamentally client server architecture where the capacity of the network comes from the servers who get nothing out of being part of that network other than work and the clients are pure leeches on that network that are unable to contribute back to the protocol that's sort of baked in and you cannot just add an arbitrary number of clients onto the network and have them perform well and that is the reality today so les has failed to give us this

um and there's good reasons why all right we're going to take a little side quest now because stateless ethereum is related to this but it's more of a catalyst than something that's actually a core part of this so stateless ethereum is a multi-team research research effort that has been going on for two years now or so um started off by alexi a number of years ago as he worked on trying to figure out the state rent problem and it led us down this path so when we say stateless when i say stateless ethereum this is what i'm talking about the model of block execution um

at a high level this is how the blockchain works you've got this previous state route which represents the state of the world before the block has all of your account balances contract storage token balances things like that you have a block made up of transactions that do things like moving tokens around and changing state when you execute the block those things happen and you end up with the post-state route and when we talk about stateless ethereum this is what we're talking about this dichotomy so all clients today exist on the left-hand side of this slide all clients they are stateful they maintain a large database of the state and and that is what they use for block execution as

as transactions are processed and accounts and things are looked up they look it up from this large state database which is a heavy thing to manage you have to keep it up to date and you have to stay online and you have to execute every block to keep it all the way up to date so that you can keep going when i talk about stateless ethereum what i'm talking about is modifications to the consensus protocol that that allow a different model of block execution a model that is based on witnesses which are these kind of magical things that give you exactly the state that you need to execute a specific block so in the stateless model instead of managing a database of state a client would actually have none of the

state it might have it might have no information about the state other than the knowing what the header chain looks like it pulls this witness thing down from the network and uses it instead of the state database to execute blocks this is what we mean when we say stateless ethereum modifications to the consensus protocol and the networking protocol to allow this different model of block processing and we need stateless ethereum this is a thing that is coming down the pipe and is actually like something that we fundamentally need for our consensus infrastructure and it has to do with validator infrastructure so uh take this with a grain of salt uh the specification for how sharding will work is not set in stone by any

means and so this is an approximation but the general gist is this as blocks progress and we move from block to block to block validators get shuffled around between different blocks and they have to perform certain duties on the blocks that they are assigned sorry the shards that they are assigned so they get shuffled around from shard to shard to shard and each and on each shard that they get shuffled to they have to do some things and one of those things is verifying the recent state transitions verifying the recent execution of blocks that move it from one state route to the next so the reason that we need stateless is that in the stateful model in order for a validator to do their job

they would have to actually track the state for every single shard to be able to do their job and that's too much work we can't make them do that it's actually potentially impossible for them to do that and so we have to move ourselves over onto this stateless model so that validators can have a zero overhead way of dropping into any shard that they need to and very quickly pulling down witnesses and verifying block execution this is why we need a stateless and now we get back towards this whole clients thing because stateless clients are cool in this model

that they don't need a state database but it turns out this doesn't actually get us to the world of our cute pink smart car lightweight clients that really work and the reason is subtle angela's going to go over that here in just a minute but the general gist is that stateless and this model of statelessness achieves these things the ability for low resource devices to verify block execution the ability for clients to drop straight into the network right at the front of the chain and start executing blocks but it turns out that we lose some things in the process of of of lightning the client and angela's going to tell you a bit more about that

cool thanks piper so okay i want everybody to imagine a full ethereum client as your favorite piece of pizza you have the crust and this represents the consensus infrastructure the foundation and then you have your cheese your sauce and your favorite toppings and these represent the user facing functionality and when i say user facing functionality i mean things such as submitting transactions constructing transactions the json rpc api and the problem is this piece of pizza it's too heavy so what we're going to do is we're just going to remove

the sauce and the cheese and your favorite toppings and we're just going to give you the consensus just the crust and this is what stainless clients are and you might be expecting when i say please run a stateless client you might be expecting a pizza but we're actually just giving you the crust and we don't want that so that's our problem so what we removed is actually the fact that stainless clients don't store state or chain history and these two things along with some others cause the high bandwidth requirements high cpu requirements high storage requirements and when we don't store these things we lose the user-facing functionality

the first thing being we are unable to be a part of the dev p2p network which means that we can no longer send transactions in the way that normal ethereum clients would and the thing is we actually can't even construct transactions anymore we can't read contract data we can't get the count balance get the nonce and this is just a little part of the bigger umbrella that we call the json rpc api but we really we really want the user facing functionality and piper and his team have been trying for three years to fit lightweight nodes into current ethereum network but the thing is they're just completely

different fundamentally and it just doesn't work when you try to do it but don't worry we came up with the portal network so we decided to go to design an entirely new peer-to-peer network from the ground up specifically for lightweight clients and this is how we're going to do it so the first thing that we're going to do is we're going to move all those heavy heavy operations from the client level down to the networking level so the portal network's going to take care of that and the second thing we had to do was break away from the client the current client server architecture that les has because we want as more clients to join we want the

network to become stronger we want the capacity to grow so we took inspiration from bittorrent where many small clients the net sum of their capacity can achieve the same functionality and capacity as several big clients so now that we have the portal network we have the foundation that we need to finally build fully functional lightweight clients these are low resource devices that require no syncing that can have to finally have the scalable infrastructure to deliver the json rpc api so there's a lot of devices that were previously unable to be a part of the

ethereum network your laptop which you sometimes close it can't be online 24 7 and when it's offline it and it goes back online it takes a long time to sync you have your phones which have low storage requirements and you have the raspberry pi which really doesn't have much of anything but the perl network allows these clients to be part of ethereum now and it was designed specifically for resource constrained devices such as these so there's actually something really elegant about the portal network and that's the fact that currently many ethereum projects have this point of centralization it's kind of hidden and basically these these projects have to are forced to run

their own centralized infrastructure in the back to talk to the ethereum network they can do it themselves or they can trust the centralized providers such as inferior or alchemy but the portal network is actually giving them a choice to remove this point of centralization because now they can actually natively embed portal clients and they can talk to the ethereum network so apps benefit by removing the point of centralization the user's benefit because you can now effortlessly help contribute to the ethereum network and the network benefits because as more apps mobile or desktop embed portal clients as we said we designed it so that as more clients join the network becomes has more capacity it's stronger it's more decentralized and

it's more secure kicking it off back to piper so this is an ongoing effort uh the portal network um only exists in a fledgling uh fledgling state right now um it is but it is actively being developed we have three independent client implementations underway um one by my team one by status who's building fluffy and their intention is to embed that into the status wallet and the ethereum js team has a fledgling start for their own portal client something that might end up being suitable for embedding in things like electron applications

things like that we're at the beginning of this process but we have spent a long time figuring out where we're going how we're getting there knowing which problems we need to solve we have a very thorough road map this is something where um we know where we're going we just have to actually do it now we spent years figuring out what are all the problems and and and knowing that at least at a high level that we that we have our head wrapped around them so um and it's a big effort this is a multi-team thing and what we are building here is larger in scope than the existing ethereum network it takes more to be able to actually do this stuff

at the networking level as opposed to at the client level so expect to see results sometime in the you know six nine 12-month timeline depending on how much you believe in software estimations thank you all for being here today uh happy to answer questions i think we have about four minutes here if anybody has a question now otherwise you can meet me afterwards and and we can answer anything you've got hi yeah so um is the general idea of the

uh stateless client that you pass the executor in this case like a light client a witness of the current uh tip of the chain and also the incoming transactions so in order to execute a block in this stateless model the block essentially gives you the transaction information for what needs to be executed and then the witness would need to essentially contain um you know it's a proof it's a miracle proof or if we're over in the post vertical transition a vertical proof um and it contains every account balance that was touched every storage slot that was touched along with any other sort of like

extra metadata that may be necessary there but all in some sort of you know provable proof style format um one more question here hey hey can you hear me yeah so so you um you basically explain like you you say that you solve the problem of like state access but you didn't say you didn't explain how

so maybe can you like develop a little bit about that on like how you probably like shard states across nodes or something like that uh yeah that was actually one of the hardest problems that we had to solve um the existing state layout makes distributing the state very difficult because the tree is actually unbalanced because of how contract storage works and ultimately we ended up going all the way back to a very simple dumb solution which is storing the data in the network addressed by tree node so there's nothing fancy going on this is very much how most original ethereum clients encoded their tree data in their database

most of them have since moved on to a slightly more advanced format but for our needs it was very difficult difficult to the point where we couldn't find an alternate solution and eventually went back to this very simple solution did a bunch of profiling and estimations to figure out whether or not it was going to be fast enough and the answer was probably yes for most things when we get past the vertical tree transition at some point here with uh state expiry and things like that um that will allow us to improve the state part of the portal network to the point where we we can move away from that kind of

sub-optimal storing things by treenode world um i didn't go into details on how we solve these problems because this is a 20-minute talk and i just didn't have time to actually get into those details but i'm more than happy to talk more about that with anybody who's really interested in the nuance thank you all for being here today you