The Path to the Ethereum Light Client

Transcript

[Music] yeah so what we're talking about now is is the is the ethereum white cin protocol so for just to give an introduction to why you like clients are needed I guess uh the important thing here is that we are dealing with distribut with distributed consensus and we're dealing with a system where every full no has to process every single transaction so you know right now with something like Bitcoin It's relatively fine one transaction in a second anyone can do it even although although you do need to download about 20 GB of stuff of stuff by now which is a which is a

problem now with something like ethereum you know in some ways our protocol is more efficient in some ways it's less efficient because we have to store a bunch more stuff but if scalability get if usage gets to any significant points if you imagine we have something like a th transactions a second then you know what the amount of resources that it will take to run a full Noe on the network is 100 to 200 megabytes per hour just processing transactions assuming 10% of those transactions increase the state which is about accurate for Bitcoin that's a gigabyte of hard disk space every 2 or 4 days thousand gas transaction that's a million gas a second I believe that's pretty much 100% of go ethereum can do right now if not more yeah we will have just en time

compilation of that point of course sure um so ultimately it's kind of important that for um decentralized applications uh we have a strategy for using these things on mobile devices the uh the general idea is to replace so many massively multi-user applications are things we tend to use the web for that uh these devices are going to be uh super important for actually providing people with what they want um so we have two problems the first is um given that a uh a like client um knows very little about the blockchain because it stores very little

probably only the headers um how does it find out about events that are going on in the blockchain that the decentralized application might be interested in the second is assuming that it now knows about these events perhaps it's a transaction perhaps it's a particular execution and a particular contract um how does it be how does it be certain that this information is actually true so just to give an introduction to the fundamental theory behind white clients everything is based on this wonderful construction invented by Ralph Merkel in the 1970s Merkel trees so the idea behind a Merkel tree is you arrange thata in a kind of tree structure where you have all the little little nodes you know keys and values at the bottom and then you take a few nodes to make a hash of them then you take and then that's

the next level and then you take a few hashes and make a hash of those and it all sort of bubbles up until we have a single rude hash at the top and you can think of the root hash as kind of representing the entire tree so there's no way to change a single any anything whatsoever about the tree without creating changes that eventually propagate up the tree and and and ultimately lead to a change in the in the root so if the root is inside the header then once you validated the header you've sort of implicitly validated the entire tree so idea with like clients is you only download you only download the block header Chain by default and uh if you need if you need data then you just then you just fetch whatever whatever data you want and and once you once you have that spe that

specific set of data you can just validate it by just making sure that the hashes hashes match all the way up so basic idea is that you can think of so you can think of a proof of some of of something as being the subset of tre nodes that are involved involved in accessing or processing it so if you want a proof that you know if you want a proof that some particular that you know some particular account has some particular State then you take the nodes that you the nodes in the tree that you need to sort of go all the way go all the way down to that that particular thing and that's and that set of nodes basically is the proof so the way you create a proof is literally this like this is this is how it's actually

implemented in the python client right now you just if you want to prove some particular part of an account you just grab it but you record all the tree noes that you accessed and the way you verify the proof is that you just sort of run you just run the same standard algorithm but using the proof as your as your database so like the nice thing is that you with the way the protocol is designed you said a lot of this stuff you just sort of get automatically because everything is based on these reverse hash lookups and you know if you if you assume that the hashes that you need that that you need to that you need actually already exist then you can just reuse the exact same code and it's just a matter of which you know which hashes do you need or which little nodes do you need to

download to make sure you have everything so thing you know in ethereum we have three Merkel trees we have a state tree state tree reords account balances stor storage a little merel tree for each account code in 1.1 maybe we'll consider we'll consider events so a wait for contracts to sort of do events that event that only get propagated processed in the future and then those events have to be stored in the state in the meantime transaction tree just stores all the transactions in the block so the receip tree that's pretty important and we'll talk more about that later but the idea is that it's sort of store specific results that are related to processing each individual transaction so what do we care about proving or what do we care about light

noes being able to securely access um account balance account Nots account code account storage data whether or not a transaction has been included um so one thing I know I was asked a couple a couple days ago is how does a node even or how does the JavaScript even know that a particular transaction was was processed so they can start so they can start sending off another transaction and you know this is one of you know this is something that the that the white client should be able to figure out from the blockchain also potentially if he wants to if your like clients wants to start actually probabilistically validating that a block was that a block was produced correctly they can they can do that and finally a thing that gav will probably

talk more about which is logs so account data we already talked about that prove existence so if you want to fetch account data you just to prove an existence of a note in the tree so transaction State transition is some more complicated thing but basically if he wants to prove that a particular transaction was processed or correctly then you take the then you take some of this data from uh you take some of this data from the from the receipt tree so the receipt tree contains an intermediate State rout after processing every transaction take the intermediate State Route you take the transaction and you make sure that if you just run the transaction on top of the previous intermediate State rout you get back the new intermediate state route and why is this stuff necessary

basically so one reason one reason is that if you want a somewhat stronger assurance that that things are being done correctly and if you don't necessarily trust the majority the majority hash power once proof of stake comes along and you can't just rely on difficulty as a as a sort of proxy for security might become even more important and finally Al so one of the things that we want to provide with with that is um the ability to um effectively like not think about the blockchain not have to think necessarily about transactions and not have to think particularly about State the idea is to provide an an abstraction mechanism for the blockchain for what the blockchain provides and this abstraction mechanism

that the sort of interface it provides becomes something that we're a bit more familiar with the notion of the uh notion of a sort of virtual machine um as the virtual machine executes um it's it's useful because it's not in an environment that you have direct access to you have to uh you have to do it in terms of transactions um it's useful to be able to put things like checkpoints in to see if a particular contract that you're interested in at a particular point in its execution such a point for example might be um that it has received a payment that it has a set a payment has been sent from uh from one of your accounts or um that one of the orders that you might have put on an exchange for instance has actually executed so this is kind of like the uh the io more

the O from the block chain and the I into the rest of the DAP but the io between blockchain and the uh the DAP as a whole the JavaScript portion that's actually executing locally in the browser now originally we had the uh this idea of of messages so in the like transactions messages are things that can um that actually go and execute contracts but messages can also be the things that um messages execute transactions of course always come from an external entity and that's they're cryptographically signed and we have the idea of message tracking so message watching whereby the the Javas the local portion of the uh of the D could um watch out for for changes on the blockchain that involved a particular message and it might also have filtered

it by what the input to the message was now this turned out to be um less than ideal and so we sort of evolved this notion of a log now the log is a uh like a breakpoint or a record it allows a message a contract to um to point out to to to an interface that a particular thing has happened logs can include a topic or many topics and these sort of identify the type of Entry logs also uh implicitly include the address of the contract that created the log and finally logs can also include data just arbitrary unformatted binary data and of course this data can mean whatever the contract wants it to mean um the difference between the topics and the

data is that the topics are indexed so topics can be watched out for by the JavaScript front end by the the rest of the DAT um the reason that we have logs at all is so that we can we can very efficiently um uh index them and filter them when daps required now because the light cine is an important um part of the the eventual protocol it was important to get this this interface in now so we can uh work out exactly how like clients will be able to work out efficiently whether any of these log entries in in on the blockchain have actually fired whether they're there whether new records have come along that we're interested in and the way that we do this as vitalic mentioned before is

transaction receipts so first of all just to describe how logs are implemented there is an OP code and the sort of Bitcoin each Bitcoin each term for what for what the log op code does is proof of publication so it just proves that something it sort of publishes an event in the blockchain by connecting by just connecting it with a hash tree to a to a state rout and to R blog header log op code has up to six arguments so first of all it has two arguments to grab a slice of memory that that it keeps as data then a log can have up to four topics and implicitly a log always has as a sort of argument the address of of the contract that that created the log and the data structure is that'll how a log

is stored is this rlp address topics and data so receipts in the transaction re so the way that the transaction so how the transaction receipt looks is that this med state is like the state route after processing that particular transaction gas used is a counter of how much gas was used after processing the transaction um bloom bloom filter we'll talk about that a bit later and logs so that's just a list a list you know the rlp encoded rlp list of each log where each log is en encoded like this and in the blog in the blog header we have you know the end state to match the intermediate States gas used at the end and and we have a sort of block wide balloom filter so each balloom filter is

is 64 bytes and the thing so the way Bloom filters work just if you if you're not clear it's a sort of mechanism where you can where you can put Keys into the bloom filter and the bloom filters this really compact structure where you can then check other values again once you've put a some set of keys into the balloom filter you can check some new value against the bloom filter and the bloom filter either returns no this key this key is is definitely not in the bloom filter or or maybe this maybe this key maybe this key maybe is in the bloom filter and the reason why it's maybe is because it's it's a sort of velocity compression where all the keys are like are in a sense hashed and packed and packed together so you're losing some data so the point is that so the way the

protocol works is you search the blockchain for Bloom for Bloom filters that are matching some particular topic that you're sort of watching for if you find the block that matches what you want you check every transaction you see which transaction or which transactions match that have a bloom filter that matches that particular topic and then once you determine once you find transactions you would actually ask for you would actually ask for know the are the the Merle proof going going down to the logs which would tell you whether or not logs were actually there and if logs were there what the actual content of the so uh the minor thing that has been uh missed from this particular list is that um we can search the header for for blooms that that tell us that a

particular block may contain a transaction that had a log that we're interested in following that we can actually request the transaction receipt block and search in that block for the bloom filters so the blooms are also for each transaction actually put in that block before we actually go ahead and search for the log entries themselves so it's a little it's a little more efficient before well search block search for search blockchain for blocks search block for transaction search transaction well I guess what not to clear but oh well so yeah so basic road map is is that the client that we're going to have for 1.0 is archive node simplest thing to implement it stores everything um then after that we wants to get into is he

wants to move from archive node so what we're calling full node so full node store the entire state but they only store recent history so you can imagine they store the state Roots going back maybe a week and then they just store some more block headers going going back maybe you know a month or a month or two and next store transactions going going back that far as well and then if they wants to grab ancient history then like they would just maintain they would just maintain some some block headers and they would use basically they would use the like light node protocol to ask for ancient history and so full nodes are actually much closer to light nodes than the archive nodes in some sense and then light nodes will just download Walkers by default and we are expecting it will take quite a while

for the full sort of leline technology to get rolled out just because it involves a it it involves a whole bunch of these various prerequisites yeah so from right so how does this work from a networking standpoint so um what's important is that when a light client wants to operate it does not have to request a proof um which are necessarily things that are customized for the for the requester from some of the node if that were the case then effectively like clients would be literally clients and the nodes that they come from the archive nodes or maybe the full nodes would be so service

and that's not an especially nice way of doing it it sort of reduces the decentral of the network so rather than so as an alternative to the ask for the proof mechanism which may still be necessary for some particular things depending on whether the state tree is uh is is well distributed and well stored by the network is um to store the various portions of things like uh headers blocks and transaction receipts on the DHT itself so these are necess these are naturally things that are identified by the hash they're actually identified by probably the same hash within the um within the the blockchain itself and the idea is that like clients themselves are a a

contributor to this DHT so they do their own um bit in terms of storing maybe some transaction receipts some blocks and some headers um that they can obviously should the chance happen that they need it they can use themselves but that they can in a in a bit torrent like tip Fort tap fashion um exchanged with other like clients in order to provide information across the network without having this Reliance upon archive or or maybe for modes so yeah so the idea is that once once again and actually sort of works nicely from an implementation standpoint is that the way that it's all you're doing is that you're grab instead of grabbing nodes from a database you're just grabbing nodes from this DHT so you're just sort of using that you can

think of it as like using the internet as a sort of extra hard drive and this can be used for you know reading the state it can be used for processing watches potentially probabilistic validation so as far as what we want what we want to have is we you know we want to store we want to store everything that light nodes are going to be storing by themselves in this kind of specialized DHD we don't necessarily want to mix it with swarm just because it's better to have first of all it's better to have the different protocols be separate I think people should be able to use etherum even if they don't care about whisper and swarm they should be able to use swarm if they don't care about whisper and so forth and there's an a possibility of making a sort of tit forat protocol for where light nodes can

sort of ask each other give each other proofs and that helps incentivize notes to actually store things so long-term like clients and proof of stake it's uh actually there are potential there are potential tricks that you know that could it's obviously going to have more infoo than than proof of work because with proof of stake you actually need to validate that people have stake there's this option of sort of artificially discretizing the protocol so you would have these sort of checkpoints that are far away from each other and each checkpoint and the is that each checkpoint would be signed by a very large number of nodes and then you would see that hey this very very large set of signers has validated this next checkpoint and then you sort of go from there um proof provision not

incentivized I mean said for Tad protocol sort of does sort of does it putting it on a DHD sort of solves the problem is you know is there some better solution will will we need to have some better solution hard to say like clients at ethereum 2.0 depends entirely on what ethereum 2.0 is going to look like um anything else you want to add no that's about it all right that's about it for