PoW, PoS, and PoA: Re-Recording

Transcript

okay hello and welcome everyone this is the etherium engineering group Meetup re-recording of the proof-of-work proof of stake and proof of authority presentation on Peter Robinson and please guys introduce yourselves I'm Rob Dodson I've been working with consensus in has a he days of October 2017 cool and I'm Adriene Sutton I'm also working with consensus in the Pegasus spoke I've been here month and a half now yeah cool at on Peter and I think consensus for almost a year

yeah and working in Pegasus - alright so I am now going to share my screen I'll just yep let's go back to here go share screen and desktop 3 I believe is the right desktop and now if I go into here I go file i select that isn't it you know let's not worry about this ok

so this is the presentation that we did yesterday and we'll present based on the PowerPoint slides that are they PFF got here okay so our consensus algorithms allow nodes in a decentralized net to agree on what trends actually should be included in the net lock and so the presentation that we're going to go through here talks about the consistent consensus algorithms that are in the etherium ecosystem there are other algorithms used st. Bitcoin ecosystems so we're going to start off I'm going to talk about proof of work and then Rob's going to talk a little bit about pls and

then Adrienne's going to talk about IVF T so if they're in to proof of work algorithm so this slide gives us a bit of an overview of what happened you know the miners come together and they pull together the transactions that they're going to put into a block and they also grab what are called uncle blocks they then have a candidate block that they use to they set up and then they all the miners are busily mining away trying to solve what's called a puzzle problem and then the one who solves at first they publish the proof and then the other nodes then are able to verify the proof okay so what I mean by assembling all the transactions so

what you're going to want to do if you're a miner is you'll have a hole see a whole stack of transactions but what you really want to be doing is you want to get the ones that are most valuable to you and so you'll find different transactions on the will have been submitted with difference and so you choose the to put the transactions which are paying the most so if you put want to have your transaction mind immediately you put a really high gas price and if you're happy for the transaction to be mined sometime in the next couple of minutes or maybe an hour then you put a really really low gas price and those gas prices can be say you know multiple orders of magnitude difference I mentioned on call blocks

before so you can imagine you you've got thirty thousand nodes in the network and it went into the network you might have a miner that would have mined a block with a certain set of transactions and it's perfectly valid block it's often block 100 100 there so it's the same height but it's no longer the longest chain and so rather than throwing all of that good work away what you do is you incorporate those transactions in by incorporating that 101a into 102 and so there's a reward structure set up to do that so as well the miners create a candidate block and

so that if you look to the right-hand side here we've got a snap a cut out of a diagram that actually I found searching about two years old I wish I had have come across that two years ago so I'm hats off to Lee Tom Thomas so this is got some essentially they can create all of the information in the block except for the mix hash and nonce just by processing the transaction so they have essentially where the state route is and they where you know what the transaction route is after all the transactions that have been put through what the receipts are for the tree of receipts what the parent hatch is so that's the hash of the previous block

the honors hash is related to the uncles beneficiary is the address of the minors node that is doing the mining so the idea is that they've got that together as essentially a part of something as the input to the overall hashing algorithm for the mining okay so then all miners on all of the mining nodes attempt to solve a puzzle problem simultaneously so each block period takes 15 seconds and they for proof of work you've got to find epochs and their 30,000 blocks long so that's about 125

hours so at each epoch you change the parameters that you're using for block generation so that you're essentially using that more memory to make it harder and this is allowing for the fact that over time computers and CPUs become more capable so for at each epoch you generate a deterministic sequence so you start off with a seed value so the initial seed value was all zeros and then you do a message digest of the seed value from the last epoch to get the current value until you can imagine and you can be generate all those seed values off out into the future just

based on you know values you've got now you generate a cash based on the same value so the cash algorithm is a time and memory hard algorithm and it's incredibly CPU intensive and the idea is that it's quicker you know to have the you can't generate individual parts of the cash quickly you've got to have it stored in memory and that's the idea of algorithm you can then and generate a data set and the data set the idea is to be able to quickly be able to generate individual entries of the data set but the data set is really large so if you

want to generate the whole data set it takes awhile but if you only need to generate selected entries it doesn't take too long because each entry in the data set only relies on 256 entries in the cache so as long as you've got the cash you could get parts of the data set on on an as needs basis so for the full nodes and the mining nodes they keep the whole data set in memory whereas the light nodes they'll generate the data set on an as-needed basis so all of the nodes of trying to solve this puzzle problem and but we've said that you know as time goes by the data size gets bigger and so when the window theorem

started the cache was 16 megabytes and the data set was 1 gig and it's growing at 128 kilobytes or 8 megabytes per Epoque so that's each you know four days or so and so we're currently we're at that block number which is epoch 195 so at the moment the case is about 40 megabytes and the data set is about 2.5 game so yeah so each epoch all of the nodes across the whole work need to change to the new cash and all the full nodes and miners will set change to the new data set and so obviously if you're a minor you're going to pre calculate all of

that data set so that doesn't slow down your mining operation so then an algorithm is used to which uses information in the data set to try and generate a value and so it takes as input you remember how we had the block header that was created you have a nonce value so this is essentially the value you try out so maybe you could choose random values or maybe just go sequentially and then you've got the size of the actual data set and then you have a callback you use to get values out of the data set so this is so that

you can either have a fixed data set or you could be getting the data values J and generating them on the fly and so the algorithm combines 64 values from the data set and which are each hundred 28 bytes wide and the output of the algorithm is the mix hash and the actual result so the mix hash as you can see from this little equation here is an input which combines the block header and the nonce value and generates the result and the reason why this is useful is that now just based on that mix hash you could check that the result is valid and so it means that you're tying the

block header and the nonce to that mix hash value so unless and so you could look in the block header to make sure that block header has these or if we go backwards a few slides notice how we've got that parent hash there so because the parent hashes there then you could check don't do we have the right parent hash if we do and it is the the immediate previous block then we know that this generated value here must be recent because it's based on that block header that's got the parent action so you know that it's not something that someone's pre-calculated a couple of days ago or you know weeks ago it's got to have been calculated in the last 15 seconds or so so this is a way of

stopping an attack because now you can just look at the mix hash and calculate that result just based on the mix hash and the block error and nonce so the Minor which solves the puzzle problem first they've you know they get to publish the block and by solving a puzzle problem what we mean is that that result is less than target value and so the target value is set so that the remaining period is 15 seconds so as more hashing power comes to the network you can imagine that the toe go up and if a some hashing power leaves the network then you can imagine the target

would decrease and so the idea is that you know you publish a result and then you push out your mind block which includes your nonce and your intermediate value 12 the gnomes okay so all of the nodes can verify the block but for the first thing they'll do is they'll mate the block header has got the correct parent hash in it and assuming it does then what it will do before it does all of the other checks it'll just quickly check does this block header and the nonce and the mix hash work out to the result so that's it very simple and you know quick check it can do and then it does the full a shamoto

like algorithm so I was going to talk about finality here and do lovely graphs and talk about statistics but I've run out of time and at a time as we were putting this together so what it turns out is that the finality the finality means when can you be sure that a block will never be reversed so you can be sure that no one's going to build a longer chain and so it turns out it's probabilistic so you can have a certain probability the one bloc's been built on top of the block you've got and then up to two after three or four mobility of there

being a reorganization which would invalidate you've LOC goes down but anyway I ran out of time to put the maths in here so if we redo this presentation in a few more weeks or months time I'll do this or if we create a specific proof of work presentation I'll put something in for this so in summary we've got a leaderless block creation algorithm and it's architectural II decentralized so if one of the nodes blows our four stops working we're all good but architectural decentralization is a bit of a different kettle of fish and so this diagram here which I've taken from

ether chain it shows as of about a week ago when this we were just quickly putting the presentation together which no were actually mining the best block and which ones were actually minting of creating blocks and you can see that you've only got a few of these miners are actually creating blocks and if you've got more than 51% of the total mining capacity then you could choose to say not mine transactions you can choose to you know censor transactions from some people for instance so yeah so though we've got good architectural centralization the political

decentralization is maybe not what it could be and there are a whole stack of links so the original PDF of this is on the slack workspace for the etherium engineering group made up so yep okay over to you Rob yep so I'm going to be talking to you about pretty good steak and kidney a probably at a much higher level to the level of detail that Peters given us primarily because the proof of stake implementations within a theory and still a work in progress these caspere is the way that people are talking about

it and the actual implementation details of Casper have been changing quite significantly and still are very much in a state of flux so um in the the high-level proof of stake and the value that we can do at the moment is just looking about talking about why you'd want Christmas stake how from a high level it would work and when we think it might be coming in let's move across to the next slide so the first first thing to think about is why and the I guess the driving reason

and the keeping behind proof of stake is the cost of proof of work and third if you look at in terms of energy usage the amount of work that goes into doing proof of works currently particularly on Bitcoin is actually the amount of energy that's used to power many nations and so that obviously is not a not agreeing thing and being able to have something that is better for the environment and does not cost that much compute power would be would be a really great solution and that's something that Brutus sake aims to doing the other thing that's listed there is the less

centralization so the the mining pools sort of list that we we saw earlier that Peter showed us in that pie chart and that is is something that is a cost with proof-of-work and unknown no problem with proof of work that even though architectural II there isn't centralization in in practice it does end up narrowing down to the to I guess the sort of three three pools can sort of work together and and it becomes a bit more centralized so by going to prove the stake we lose some of that centralization that is currently happening with proof of work and and the final reason is that there's

strong economic incentives to do the right thing so a part of proof of stake is that you need to take your ether or some sort of a financial stake with with things and if you break any one of a number of rules with with your stake your stake is is removed and so you lose lose the money that you put into the system and so there's I guess that strong or lose a part of that that money that you're putting in and so that has strong economic incentive to do the right thing but the whole reason that you're putting in the money for your

stakes in the first place is to get an economic reward and if you cheat the system there's also these punishments in place okay so if we look at the the next slide you'll start to talk a little bit about how proof of stake works and so the the idea is that instead of there being miners in the system there's the idea of validators and so the validators their job is to validate the set of transactions a set of transactions and propose a block-based out of them for a a node to be a validator they will need

to lock up some of their some of their coins as a stake after you've locked up and set aside these these coins so for example putting them into a smart contract they'll be able to start validating blocks and so what they can do is when they discover a block with a set of transactions that they think can be added to the Train they will validate and say this is the block that I think we should put into the chain by putting a small bet on the block if the block gets appended then the validators who were responsible is proposing that particular block will get a reward proportionate to their bets we move on to the next slide there's

more information on the staking invents so if they cheat so if some reason if it gets detected within the the chain that you've been placing bets on two different blocks at the same time you'll have the deposit that you put in for those bets deleted or slashed and so the critical take stake talks a lot about the conditions for when you when our deposits will be slashed and so that's essentially saying that whatever the coin is that you were putting into that transaction it it no longer exists as has any coin so the a proof of stake protocol and pile of the hard work in

getting a personal stake protocol right is specifying the slashing conditions so being the example that I mentioned before was having a slashing condition which is set up to stop people from betting on two different candidates Ibelin blocks but at the same time so there's yeah that that preventing that essentially stake of nothing touch situation so making sure that when you are staking you really are staking something and clean your your money I guess to see where your mouth is we look at the next next slide so this idea of these stakes and how much you need to

put in it depends and the original caspere papers and the original proposals have been talking about about a thousand eight and so in in my my world that Australian dollars and when I looked up a couple of days ago that was about about six hundred thousand to just under a million Australian dollars which did need to stake and then once you put that initial stake in you can start betting on blocks so that's something that's kind of a bit beyond the everyman you need to be prepared to make a big big kind of bet on this kind of thing and so be the world that's happening is that there

will be I guess that idea that they'll still be steak steak schools and sort of that idea that people will be able to pull together they're they're eager to have someone that does the actual staking work for them and you'll be able to get together into a pool and and do that and so there's a couple of the early proposals for pools sort of sitting there and companies that are setting up the the infrastructure so that once a theory and does have proof of state they'll be able to do it se the other thing to note with the staking and I guess we'll start to look on the next slide which starts to talk about when is

it's the 100 hundred million a fair question it's the the proof of stake has been three to six months away since at at least October last year and at the three to six month window keeps on getting it pushed out as time goes on um at the moment to - I guess somewhat complicate the problem crypto kitties for them exploded and really highlighted the need for sharding as well as proof of stake and the the two ideas have now been really for the coupled together and so that had the size of the design and the

architectural change has increased and so I think it's kind of three to six months from now but probably more like six months and three nuns would be the sort of my mental model currently the EEP which was having the the Casper a description so each double 1-0-1 has moved to a spot where I guess it's probably not going to be implemented so that this all said there's currently as I as I speak there shouting workshop going on in in Germany in Berlin at the moment and so it's quite possible that that quite possible

it's not impossible that they'll say Eureka we've discovered the way that we want to do shouting let's start moving now it could happen but it's probably not super likely that'll be the something that'll happen right away it's still definitely an active area of research and lots of smart people are working hard on how how sharding is going to work and how and how the casper and proof of prove mistakes going to work inside of ethereum so I think the the current target that I'm hearing the most about in terms of timeline is that it's targeted for late this year and it might slip to next year so that's our

bird's eye introduction to Christmas fake I'll now hand over to agent he's going to talk to us about proof of authority algorithms great thanks Rob so what is what is a proof of Authority algorithm it's proof Authority is basically a category of consensus mechanisms where the permission to create new blocks is granted entirely based on who you are it's not based on the fact that you've done work or the fact that you've staked money it's just we know who you are we trust you to create blocks typically that's done by having a pool of nodes that are authorized to to create those blocks so that if one of them down others can take

over that kind of thing and possibly have some fault reliability in different ways there and then the other key characteristic is that there's some way to verify that a block you've received synched via the earth protocol was actually signed and proposed by an authorized node that's that's typically true some form of cryptographic signature so we go to the next slide the other way of looking at this is to look at what the cost bad behavior is if you proposing rubbish blocks in proof-of-work it generally won't get accepted to the chain you've wasted your CPU cycles and you have to spend a lot of money in getting enough CPU cycles to be able to

find the right answer to the crypto challenge the proof of stake we burn money much more directly you literally put up money and if you do the wrong thing its slashed the provable authority what you're effectively burning if you behave badly as your reputation so you're allowed to to create blocks based on the fact that you're trusted and if you then demonstrate that or you're not trustworthy that can be taken away and usually results in things like people not wanting to do business with you anymore you see examples of this in a lot of different places so for example Airbnb it's basically a reputation based system if I advertise my house is somewhere you can stay and you come

along and it's not like I advertised it's really rubbish experience you can leave me a bad review people are going to know not to deal with me in future so I earned my reputation by doing the wrong thing there and Airbnb itself is acting as a central authority so that if they get enough complaints that my places isn't isn't suitable then they kick me off Airbnb altogether and I suddenly lose entire access to the system so proof Authority kind of depends on on both of those factors where we want to know who you are so if we kick you off you stay off we don't want to end up like comments on the Internet where you can create a new profile quickly and easily and so you don't care about reputation and we have to have some value attached to that

reputation so some way of of punishing you and having a negative effect if you have a bad reputation usually just we're not going to do business with you anymore so looking at when chains are most appropriate when provable authority is most appropriate the most common case by far is on private chains particularly within consortiums so a number of companies working together and know who they are well they know which company is operating which validator node and if it started misbehave to regulate and they can remove that company from the consortium and stop doing business with them but there's a couple of more interesting cases the rinky test network is a publicly

available test network which was created because the the previous proof work test network got swamped by people submitting a lot of bad transactions or a lot of transactions that took up a lot of time and they could do that and afford to do that because ether is free on a test network and has little or no value so Rin P uses proof Authority to give it a bit of centralization that people who are running the wrinkly network are all the validators and they can choose to censor bad transactions they see a user submitting transactions too fast they can delay mining those transactions into blocks or they can outright ignore just transactions that look like they'll harm

the network that are deliberately set up to be malicious and the third one is the provable Authority Network which is a side chain for aetherium and it takes an interesting approach in that it's a public network or stuff that's meant to have value it's not just a test network and they're validators are required to have a US government issued notary public license so what they've effectively done is outsource that that trust model to the US government if the US government says you're a good person and says this is who you are and that you're a good person then you're allowed to be a validator and you've got to publish that license in a way that that other nodes can verify how much trust you put in that is is up for debate and

will depend on what you're doing obviously if you don't trust the US government it's not going to be any use to you and there's kind of questions around how you revoke access if all you needed was a notary public license then it's unlikely that you could complain that they will you know censoring transactions when they're mining on on a side chain of aetherium and have the US government actually pay any attention and revoke their notary public license that's probably not going to escalate to that but if there's you know central authorities that we can all agree not to allow those two people to be validators just because we can identify them it can be quite work workable so you you regularly see these kind of trade-offs in proving Authority where you're really

just making a decision of how much trust you have and how much trust do you need we look at why the most common reason is just higher throughput and lower cost so compared to proof of work we don't have to burn all those CPU cycles much much lower cost and less resources required and because of that it's much quicker to my new blocks you can generally do them closer together most proven of authority algorithms are set up to try and minimize the number of network hops required and interactions required to get to consensus and there are different trade-offs involved in that but that

generally allows you to mind blocks quite fast and so you get higher throughput on your own your transactions and often load lower latency as well depending on the algorithm you can have either just faster or immediate block finality generally with proof of Authority algorithms the the finality isn't just probabilistic like this is proof of work but within a certain number of blocks and that's not a guarantee different algorithms to do different things obviously but with something like clique you generally getting consensus pretty quickly within two or three blocks you'd have a very high degree of confidence and especially once you've seen each validator so the number of validators you'd know that

they've all accepted those blocks but for something like IB of T you know that as soon as a block is is minted and on the chain then it has immediate finality it will not be changed again and finally unlike proof of stake you can use proven Authority even when your ether has little or no value it's not dependent on that it's purely based on the reputation so one simple example of proof authority is clique which is simple but effective in clique you have a set of validators that are agreed on any validator can create new blocks it's set up so that they're in tended to be round-robin and priority is

given to someone who is proposing in turn but the only actual restriction or hard-and-fast restriction is that that there is a limit on how often anyone proposed it can create new blocks the interesting thing in clique is how they add and remove validators from the pool they've co-opted a few block headers and so when a minor or when a validator proposes a block they can include the address of another validator either one they want to add or remove and they put that into the minor or the beneficiary header sometimes called coinbase as well and then they use the nonce field header to to indicate whether they want to add

or remove and as soon as you've gotten half of the validators voting one way or the other then you make a change to the validator list so you have the ability to update the validators as the train goes on which is obviously really useful if someone's acting badly you can vote them out if you expand a consortium you can add in nodes from the new company that kind of thing and there's a bit of an optimization in that at each epoch the outstanding boats are reset so that add in chibok you just have a list of validators that's the only state you need and that's in the block here so you can much more easily go back to the start of the epoch and work forward to work out the state you're up to in terms of boats but what clique doesn't do is have any kind of fault tolerance for

that you tend to talk about the Byzantine fault tolerant algorithms and that's based off of a paper from a while back which is practical Byzantine fault tolerance they used a particular this particular three phase commit algorithm to build an NFS client that that could handle faulty nodes in a distributed way and Istanbul BFT or IB ft has taken that research and applied it to the blockchain so with IB of T and DB of T as long as you've got strictly less than a third of validators that are faulty then the network can continue functioning normal now faulty includes offline it includes

buggy and and actively malicious and it's quite common that actually the bigger concern you isn't so much the actively Malou malicious which might happen if you get hacked but just that nodes go offline regularly things can be buggy eBay can be problems in in implementation you don't want that to take down your entire network just because one machine has gone offline and so if bft you have a lot of fault tolerance in there in addition to the the confidence that even if one one node or as long as it's less than a third of your nodes a hat then your system continues working so if we go to the next slide this is the this state diagram basically for i BFT it's taken

straight from the EIP 650 which and lays out how IDF key works well I'm going to full detail on to it but it's essentially a three phase commit so the proposal will send out a pre-prepared message and it goes into that pre-prepared state each validator that receives that sends out a prepare message with the same hash of that proposed block and that's basically saying yes I've received this block this is what I'm planning to insert on the chain once they receive to F plus one commit messages where F is the number of faulty nodes you can tolerate so essentially a super majority of more than two-thirds I have agreed that this is the node we're going to try they then

send commit messages to say I've received acknowledgement acknowledgment for every one that that's what we're going to try again you wait to receive to F plus one or greater than 66% of commit messages so you all know that you're working on that same block and then you finally actually insert the block there's a few ways that that could go wrong either the proposer isn't there usually the proposer is is selected on a round-robin basis out of the validators if they've gone offline then you'll reach a timeout and need to try again doing that and that's where it goes to round change the same thing if you don't receive pre-prepare messages or prepare

messages or commit messages anyway there you can reach a timeout and to keep the system moving you broadcast a round change message which is basically saying I'm trying to abort our attempt at consensus for this particular block and I'd like to try again start from the beginning and as long as you get more than two thirds the network agreeing that it's time to round change then you do that you go back to the start of the round new proposed ascends out a proposal for a new block the only other real detail there is if we go to the next slide is that for a particular corner case of safety ib ft locks blocks so once you are prepared on a block once you've got

agreement that that's the block you're going to insert next even if you go round change at that point used you refuse to accept the pre prepare for a different block it kind of involves a malicious act of sending different messages to two halves of the network and just eliminates the possibility of your fork in that case it's I'll leave you to look up the details in the papers themselves but I won't try and prove that but basically you only unlock on a block when it's when it's actually inserted into the chain because you're moving to the next height and obviously it's a different block or if you can't insert if you get to the point where you insert it and it turns out not to be valid the transactions fail to apply

then you round change and pick a new block and that's a that's an overview of IB ft and provable Authority well thank you hm so I hope you all all of you watching this video have enjoyed this quick summary of proof of web proof of stake and proof of authority algorithms so we tried to put together a slide which tried to summarize how they all worked together and so for proof of work and you know I'm looking at them well one way you can look at them is finality so when can you say the block is ready to go and you know you can reliably say

that the infant the transactions in that block and never going to be revoked and for both proof of work and proof of a stake it's probabilistic so you're never going to be 100% certain you might be 99.99 percent certain but you're never going to be a hundred percent certain because it's probabilistic project in raft though there's POA algorithms invariably they're instant another thing that's interesting is the ability to handle malfunctioning or missing or malicious nodes and so for P o W and POS it's generally less than 50% have to be in that category whereas for IB ft and raft or other IB

ft it's around you know just under a third and for raft it can't handle any used so raft is a distributed systems algorithm for gaining consensus that we haven't talked about but it's a good reference point though because it is another one of these Authority systems where which you know you could look at and is similar to idft in fact if you look at quorum by JPMorgan it originally came with raft as a possibility you now idft is the preferred algorithm and it's because of this feature about how many malicious or malfunctioning nodes can be handled and not handling any

malfunctioning nodes is obviously a massive problem because it's not the real world in the real world things do malfunction so there's also the trust level that's involved and so for proof of work it's trust plus so you don't have to trust anyone on the network and for proof of stake the goal and the design goal is it's got to be pro trustless because the whole game is to replace proof-of-work idft you need to have some level of trust but you know you know if in OB ft is good for I guess can consortiums where you know you can imagine as well you might have like a at what point do you not fully trust all

the nodes on the network or all the organizations on the network that have got no the network you know if you've got ten nodes on a network maybe you know everyone and you trust them but maybe ones you could once you've got a thousand nodes on a network from a thousand different companies you don't know them all well enough to fully trust them anyway so having an algorithm that works with some level of trust but not complete trust is good so raft it relies on all nodes being trusted and of course energy usage is a big one so obviously for proof of work is that you invariably I'm sure you've all seen those photographs of factories full of computers just burning electricity solving puzzle problem x' and so that energy usage is huge whereas

the other algorithms it's quite low so for those of you attending the meetup we're not going to have one in two weeks time but we will have one on July the 25th and that's going to look at advanced solidity and design patterns so it's how do you design your solidity code and your overall sets of solidity code and the advanced solidity is really going to look at how you have set it up so that your gas usage is either flat or linear based on the data that you're putting into your contract and also looking at advanced things like doing calls to the elliptic curve functions

and the modular exponentiation functions so essentially the pre compiles and how do you get that to work so I don't have to ask if you've got any questions but so thank you everyone for joining the call or watching this and yeah please up onto the etherium engineering group meet meet up slack work space and fire questions in there in the general channel if you do have questions about this video or you could I guess even put them on to YouTube as a comment as well and we'll get back to you about any questions you do have so thank you

everyone and have a great morning day evening thank you bye bye thank you