Managing Eth2 Validators at Scale

ETHCC 4 · July 2021 · 19:27

Laszlo Szabo discusses deploying and managing Ethereum proof-of-stake validators at scale.

Speakers

Topics

From ETHCC 4, hosted by Ethereum France.

Transcript

amazing cool should we start yeah so my name is las lucebo and um as my name may sound i'm more ethereum maximalist these days than than bitcoin um and i will talk today about managing these two validators at scale so i'm the ceo of skills and at skills we we manage um proof of stake values at scale on many protocols and obviously on ethereum so to this monu is going to be first of all i'll come back i'm trying to be brief but why staking is so much better than mining in so many

ways why did we choose ethereum 2 at the first place how do we manage validity at scale our cloud infrastructure feedback on the run running many validators pass to decentralization and then what will you earn when you stake on ether so yeah i i just want to you know do a heads up for justin drake and the ethereum foundation because today's staking is just more secure than bitcoin mining on ethereum which is crazy if you consider a cost of attack uh on a 51 attack on mining uh bitcoin is 160 million

ashes and one terra ash costs you 30 um with electricity and cpu it costs a little bit less than than 5 billion to do a 51 attack on bitcoin mining which is concrete for any states like china or the u.s and on a theorem um considering there is 5.3 of total heath that are staked the attack will cost a bit less than 6 billion so it's more than it's more secure than on bitcoin and the low to power ratio so the cost of attack divided by the market cap of the protocol is at least two to three times lower on ethereum so ethereum

is much more secure it's more much more efficient as well because it's two times less energy consumption from staking to mining on ethereum and in terms of infrastructure provider running mining rigs cost you 10 times more than running validators then this meme of ultrasound money right if um bitcoin is sound money now ethereum will be ultrasound money with ip ep 1559 70 of the gas will be burned and the burn might be greater than inflation itself

which pushes uh which put a therm as the most significant crypto the first most significant price to be deflationist crypto and you have to consider that um from a validator staking provider standpoint it might be a threat since we we get less fees after the merge but we see it as an opportunity in the sense that it will impact on the ethereum price and today our business model is mainly based on pure token model so we we take a commission on the rewards of our users so um it's good that ethereum is a better asset value than bitcoin why did we go on ethereum staking in the

first place so we are blockchain infrastructure providers since 2019 first started with a private network uh then realized uh it was private networks are completely useless uh we did then a therm tesos on full nodes and and we went on first taking protocol which is called pocket network and pocket network has this similarity to to ethereum which is a pure proof of stack network meaning every x number of tokens you spin up a new validator so it's 15 000 a pocket and 32 on ethereum and on the staking provider standpoint you have to

manage and deploy deploy and manage hundreds or thousand validators which is completely different than having one single validator on delegated proof of stake networks for example first secondly the company is super large and it's also the biggest stake in market it's already a nine billion annually staking market and it's we think it's going to be a more than 40 billion stake in market by 2025 and we actually agree with gp morgan here so and and last but not least the staking on ethereum kind of opens the door to defy right staking is the first defy layer

protocol product sorry it's internet bond and it's composable so we have the lidl guys just in front and lidl just shows you that making sticking liquid just opens the the first yield that you have the less risky because it's it's a protocol yield to the rest of the d5 um so once we decided to go on ethereum staking we had to choose validated the clients so we kind of tested all of them and we chose prism because the community was the largest one uh it was also in our standpoint the the most stable one

and we really loved the open source tooling that they were provided but we are looking at other clients the problem with prism is that everyone using prison today i think it's 80 percent the the ratio of people using prism um and and and then we we needed to to go on on different cloud providers so today we are using aws which is quite expensive but also european cloud providers and we'll add more and more uh cloud providers in the future so different valley data clients and more and more cloud providers to have a more decentralized infrastructure yeah so this is a typical eth2

infrastructure setup on the right you have clusters of three to four ethereum nodes um i will show you that we use kubernetes as a orchestration engine and these ethereum one nodes connect two bigger nodes right um one beacon nodes will connect to the validated clients so we think you can go up to 50 validator client per bit node not more because if you do more you take the risk of you know sending too many requests to the beacon node and um to just put it down so this being said you can mutualize keys on a validated client up to

50 keys so what we do is we use one validated client per user to make it more secure but in certain setup and to lower down the the cost of the the validation key so basically one validation key 32 if a user has 50 times 32 is you can you might use one validate the client um and then we use kubernetes clusters in in the cloud so everything is pods uh through kubernetes uh validators clients are are different ports um the beacon node and ethereum nodes are our ports and and and we have uh auto scaling group and persistence

caches uh volumes for the uh slashing prevention database and i will come back to it because it's really a critical piece in the ethereum two infrastructure that you don't want to mess up with secure the validation key as you would secure the transfer key this is really important guys because we don't own the transfer key right on ethereum two there is two keys um the the the fonts keys which uh retains the 32 e's and the validation key we own the validation key but securing the validation key is as important as securing the transfer key right because you can double sign you can

surround votes and this will lead to slashing so we use vault or hsm in the cloud but you know securing the validation key is as important as securing the the transfer key um yes feedback on the run so we use typical parameters graphene monitoring setup from prism open source it works pretty well but then you need to set up your own alerting setup right so um you want to make sure the beacon node and the ethereum nodes are up at least more than 99 of the time and you want to make sure you don't miss attestations

on valley data clients and that the value that the clients are up as well a good uptime would be from 99.3 99.5 percent um if you go below 99 you're already a middle i would say bad not performing provider and what you need to be careful on this is really important guys because i think most of the slashing events uh happen because of messing up with the slashing prevention database so you know the the validated client has a database with all the the the logs of

the signature um of different blocks that you did you signed and it tested and you don't want to move the validator client and the database from one region to another from one cloud provider to another or even from one machine to another you can do it but you have to be super careful and i would advise you work with the validity clients labs to do so and please please don't try to mutualize the the slashing prevention database i think one of the biggest slashing event on east 2 as of now was from a large u.s provider and he tried to mutualize through kafka

the the slashing prevention database because it obviously lowered downs the cost but he hasn't slashed up to 500 000 in east so better be offline than slashed please don't mess up with the slashing prevention database and usually use the one provided open source by the validated client prism or the others reputation is key like when you are a staking provider everything is on your reputation um so that's our asset i think uh alan mentioned it before uh ssv and and we try to to not having cowboy setups and and you know keep our reputation as high as possible

yeah this this is a this is a um a slide i i stole from from lido actually and it just shows the centralization problem to solve in ethereum's taking today it shows that exchanges are 30 percent of total stake other providers 30 13 sorry lido 10 and others approximately 50 so you know compared to bitcoin mining we still see centralization problem um exchanges are big uh big providers like us are big as well but what's different on ethereum which

is completely new and wasn't occurring on mining is first of all you can spin up today validated on a raspberry pi so it's much easier to to spin up a validator than a mining rig secondly you know dials like lido makes it more decentralized in two ways first of all they will make a frictionless onboarding of node operators so today there's only eight another operator nine node operator um pouring lido 10 of uh east to stake but tomorrow there can be a 50 hundred fifty just make it frictionless through the dial for any know the braided suit board and uh

also they make for users is taking liquid and staking uh from any um fraction of each you want to stake and last but not least ssv ssv is obviously you know signature validator validators um it's it's obviously like putting a decentralized weight of validating the um [Music] of the validation process you have 10 providers 10 providers won't have 10 validators on their own but they will get a quorum of 100 validators and each providers will have 10 uh of these validation power through 100 validators and and not 100 of their own 10 validators i don't know if it's clear but

but again you know this is a major advance into validation process what will you earn on east to staking so today you earn approximately six percent apy but i i like jester drake and foundation think uh it will go up to 25 after the merge because you're gonna take the fees obviously and this is not including emiv right um and obviously maybe so unpredictable that um it's super hard to forecast how much you will earn as a piyo validator but 25 is a lot guys especially on a protocol that is so

stable like ethereum and last but not least if you don't know what to do tonight you can come to our event on staking so there will be a liquid staking lido guys just in front of you mev flashbots our flashbots will integrate to prism and other validated clients on eth2 and secret share validators and alan that was presenting just before me you can follow me on twitter thank you very much and if you have any questions please ask [Applause] yep

uh so we have said that you have uh three uh if one nodes i mean clients uh so uh are there on a separate hardware uh are they separate in in some way from the from other machines and do you load balance between them or you assign just some big incline to fix it at one client and that's yeah so we have three to four nodes um on two different machines so there will be one node and one machine and and two another on the other one uh they are represented like uh pods on communities and yes we load balance in them um and and we try to spread the charge of the of the request right

um um to a certain number of requests will divide by uh two to three to four um to the to the nodes do you use some specific load balancer or it's built into the clients you know so the load balancing is um is like from the communities cluster and then we we have our own setup so we we know basically how many requests one node can handle you know we we can like deployed ethereum onenote since at least two years so you know we know that uh um passing certain number of requests uh better um better just spreading the the request um to to do to these notes thank you questions

one over there hi do you have any recommendations for a staking provider with the lease fees that's not lido oh come to us oh what would you say is liquid staking or uh or just a provider um i'm looking for like just a staking provider with just the leanest fees yeah that do not subsidize uh the rewards for miners that are i mean uh validators are still pending to come into the network so probably with liquid staking is there's

no really real competition so if you want to do liquid sticking just go to lido and if you want to uh get your centralized provider uh we we can uh stack you east so we're working with lido lydo we're working with with uh with clients as well uh institution mainly and and if you're interested just come to us okay cool um yeah i'll i'll follow with you out there after the in the follow-up event then perfect thanks any other questions thank you guys you