The Staking Endgame: Distributed Validator Technology (DVT) w/ Brett Li (Obol Labs)

Transcript

[Music] hello and welcome back to the strange waterer podcast thank you for tuning in for another episode here's what you need to run an ethereum node a dedicated computer capable of running this software that makes up ethereum an always on and unlimited power supply and internet connection and at least 32 eth which at today's prices is equivalent to roughly $75,000 today this is just barely within the realm of widely accessible and

that's really only if we stretch the bounds of widely and accessible you could could get a dedicated eth staking machine for about $1,000 or or maybe less and you probably have access to enough power and bandwidth by default but I'm a little stumped on $75,000 no matter where you are that is a lot of money and tomorrow well things are only going to get worse protocol upgrades like dank sharding and enshrine PBS are only going to raise the minimum requirements for nodes and they're also going to significantly increase the bandwidth requirements of running a

node but perhaps the most daunting part are the eth requirements if eth reclaims alltime High 32 eth will cost over $150,000 and we all hope it continues well past atth so look while it's true in a relative sense that running a node and staking from home is accessible from an absolute sense things are pretty dire and they're only going to get worse so is ethereum screwed well my friends the answer is an emphatic no and today we have the perfect guest to talk through the

technological Revolution that will save ethereum decentralization distributed validator technology or DVT Brett Lee is the Chief growth officer at Obel Labs if not the first then the most prominent company building distributed validator technology today's conversation is a fantastic Deep dive into all things DVT why it's needed how it works how it affects home stakers how it affects liquid staking protocols like Lio and so much more this episode is a fantastic listen for everyone with eyes on the ethereum road map and concerns with how we will enter a middle wear backed

modular blockchain Paradigm by the end of today's talk you should walk away realizing that in the ethereum endgame each point in the ethereum network Universe isn't just a star it will be an entire constellation one more thing before we begin please do not take Financial advice from this or any podcast ethereum will change the world one day but you can easily lose all of your money between now and then okay ladies and gentlemen Brett Lee Brett thank you so much for joining

us on the strange waterer podcast thanks for having me of course man so like I've said this to you so many times in private I'm so excited to talk um distributed validator technology and especially obal but before we get there I am a huge belever that the most important part of every conversation are the people in it so with that as a kind of frame can you tell us who you are um how you found crypto how you found yourself in this like weird Corner that is the start of hopefully the next bubble and I guess like um what about crypto like Drew you in as opposed to repelling you so I uh consider myself a web two and a half person I've spent most of my career in in web 2 and I was always fascinated by the potential of

Technology um to impact people's lives uh to make the world a better place um but for the most of my career I was in uh B2B web 2 which in my mind is the thing that is like you know maybe controversial say but furthest away in my mind from impacting people's lives right and uh I uh you know did a lot of uh I was a sales engineer at the start of my career did product you know halfway through um but I was kind of caught in this uh flywheel uh or or kind of this mouse wheel of constantly just chasing the next deal trying to sell software and I was pretty far removed from uh understanding kind of the impact

of technology and so uh my entry into uh into web 3 really started with the launch of the ethereum white paper uh and just learning more about uh the impact uh that a turing complete uh descent calized platform could have uh and I feel like the potentials were endless um and the impact it could have was was everywhere uh and so really got interested in the tech itself um and because of the tech started doing a little bit of you know personal investing and got the opportunity to jump full-time into web 3 with consensus uh in 2018 so I was there uh during the the Ico hype you know when everything

was really starting to pick off pick up uh consensus was fascinating to be a part of in still the relatively early days of the ethereum ecosystem developing uh all across the world seeing the early days of the wallet you know of metamask uh developing and of the various projects gitcoin Etc starting to take shape to think about how uh ethereum could be applied to impact different parts of the of the world uh got exposure to the Enterprise side uh this was back when all the Enterprises you we're talking about hyper lra Fabric and like using blockchain for supply chain for tracking you know I was I was really fascinated uh by the penil impact that I think blockchain could have on

businesses because that was the world I was familiar with right and so did some work there and then also uh traveled quite a bit to China actually uh I was part of you know I'm I'm I'm Chinese uh and uh had a lot of uh uh interest in how uh blockchain and crypto in general could could impact a a a country like China with its political structure and everything the hype there was very real uh there was a lot of people jumping in uh so anyway it's got a lot of exposure kind of the early days of of crypto um and uh was quite fascinated Again by the by the impact um after about a year though at consensus I actually decided

um as a marketer myself a growth person myself and one that wasn't interested in kind of hype tokens and you know make a quick kind of pump and dump cycle I thought it was too early for kind of true product grounded marketing uh to be done and growth to be done and so I actually left the space in 2019 uh and went back to web 2 uh joined an early stage startup uh grew a marketing team uh from just myself I was employee number seven at that startup uh and by the time I had left it was over 100 people kind of went through that Journey um and decided it was time to come back uh early last year uh and kind of found oble uh wanted to get into kind of infrastructure uh you know product an

infrastructure project that was um building the foundation of ethereum I still think uh there's a lot of Need For Better de developer tooling uh better infrastructure uh because that sets the foundation for all the Innovation that's happening um and all the that's going to happen in the future uh and thought that distributed validator had a really critical role to play in uh proof a stake ethereum and making the foundation of ethereum stronger in the long run and ensuring the decentralization of of the network so yeah uh now I'm leading growth for for Obel and kind of uh 2024 is going to be an interesting year for us uh so really excited to kind of dig into that as well yeah for sure so before we get into obal just while we're

still in your background I what I love about your story is you are like the person who entered crypto were here so early and then realized like that this space wasn't if it it it hadn't reached the maturity in which like your skill sets were needed and so you left crypto and then you came back and so like obviously you made the right choice but I wonder if you just have any Reflections on like why crypto let's say pre 2019 was not ready for um formalized marketing and and really like Market getting outside of like token prices and this kind of like pump and dump cycle and then um in that answer I I would be curious to hear

like what's going on now like where do you think the turning point is that like you have uh like felt that the industry has reached a point where like the things that you really believe in are um valuable I think back in 2018 2019 the way I would characterize um the E theem ecosystem and kind of web3 in general was that there was a lot of really smart really great Dev work and research being done you know we were talking about uh you know proof of work versus proof of stake and the different consensus mechanisms there was a number of chain launching on different consensus mechanism and trying to solve you know uh the the scalability trumma and and all of the kind of really technical work that is required to build an ecosystem

as vibrant as as web 3 is today and then there were a lot of people trying to make a quick buck and I think at the time the kind of uh was uh leading the actual maturity of the technology um and there was a large gap between what the public was thinking about uh with crypto and where the technology was and because of that there wasn't a lot of like true products all the use cases were still perect Concepts you know um there were early days of of designs of of things that are you know uh of how things could be applied in different things but nothing was mature enough to actually be ready to be taken to Market where there was a user base and there was demand and

there were needs to be addressed um and product mature enough to to be able to do that so I think that's why like for somebody like me who who my core is product marketing you know I I lead growth but like at at my core I'm a product person and I want to see technology being applied to actually impact people to change you know make it easy for somebody to do something or to make their lives a little bit better than it was uh before and none of that was quite ready I think back in 2018 since then uh I think a lot of things have happened uh you know defi introduced um we had nft hype which from a utility standpoint kind of draws into

people's creativity and into like you know digital ownership and all of those things which I think are quite exciting um there's uh a lot lot more uh tooling available I think from a hype cycle it's kind of died down a little bit uh which is why I went actually I left during a bearer market and came back during a barer market which was a little bit by by choice because I think the true uh development and and building happens during bear Cycles right um and that paves the way towards towards future Bulls uh but uh yeah so I think a lot has uh happened in the space um since 2019 to mature it um and I think there's a lot of true products now uh that need to be built and need to be

figured out how to apply it uh from a go to market sense so uh pretty excited about where where the space is um and you know I'm the reason I came back the the plan was always to come back after after about 3 years and kind of see where the space developed and I think now's the right time to like start to uh kind of build towards the future um I think a lot of exciting things are on the horizon yeah for sure and I think that um one of like the exciting parts of this like kind of pre- bll run phase is that the the narratives that are going to drive the next Bull Run are like starting to like become apparent and I think like so clear it's about like lsts and about the staking layer it's so clear that it's about igen layer

and reaking it's so clear about the modular blockchain and so um totally hear you on that I think I guess like last question before we talk uh modern day I would love to just hear do you have any um like thoughts or just any anything you can share about the experience of being outside of crypto during a full bull cycle and that's both like the exuberant highs and the crazy amounts of money and like Luna at 120 and also just like the cataclysmic falling and the like dinosaur extinction event and SPF and all of this like what was that like experiencing a space that you love so much but just not in it yeah it's a interesting question uh and I

think for me when I was out of it um I frankly didn't really pay too much attention to the headlines cuz that to me was still indicative of just a layer of height that sit on top of the true kind of maturity and continued steady growth of the space right I think that's where I was actually quite interested in what was happening right all the stuff with people trying to like capitalize on the hype to me is frankly the the part of of crypto that I hope goes out of the system over time you know I think you want to have good energies you want to good excitement uh you want to have excitement that feels progress and feels

growth in the space but you want to get rid of the the stuff that you know people are looking for opportunities to to kind of capitalize on on that hype um so you know I I don't feel like I necessarily missed out on the bull you know I you know I was still holding assets and kind of WR in through it and obviously was excited on the side but I think what truly um got me back into the space after and I I felt that it was time to get back into the space was all the the growth and development that was happening as part of the tech itself you know ethereum finally figuring out how to move to proof a stake you know after years and years of just research and debates and development Etc uh and now getting to a place where you now have

layer 2 uh scalability being built on top of it um you have uh you know reaking and kind of different use cases starting to be talked about right and you have a pretty mature now Foundation that can sustain and support an entire ecosystem of innovation to happen on top of it so I think it's really exciting where we're at now I'll never forget in 2022 I remember specifically where I was which for Americans it was super late when the merge actually happened and I just remember thinking like this is such a tragedy that this is happening like at this moment where it was it was so far after Luna that like the people who were going to leave have already left and there wasn't even like horror porn

stories on it anymore and then two months later any excitement over ethereum and the merge would be wiped out by SPF and so like I just main net is so interesting and so exciting and to like call anything that we've experienced a bare Market uh is to like I guess show your cards that you're where you're paying attention and so like I just totally feel you on that yeah and and I think you know mainstream understanding of web 3 is going to be is still we're still quite a a ways away from that you know and I think we're developing uh steps towards that you know I think the recent Bitcoin ETF approvals puts it back a little bit into that into that Limelight in a in a

positive way right and I think there is steady maturation from how mainstream sees crypto and how mainstream sees web 3 uh in general but I I feel like uh those those narratives will will have to Impressions will have to change over time I don't think we'll get there overnight yeah and part of that is developing like the capability that is actually applicable to real people's lives and not to internet Speculator exactly 100% 100% cool all right so let's uh pivot into obal and I would love if you could um kind of like give us an idea of what obal is but from the standpoint of yourself as you like looking for new uh opportunities in the web 3 face so can you like kind of give us the uh like the bretley view of like

what is going on at obal and why is it so exciting just from a super high level so obal's root comes from even back from my days at consensus uh in 2019 and as proof of stake was um the idea of moving ethere to proof of stake was being talked about right and this was preg Genesis uh there was uh still a good amount of people concerned about the potential centralization risks of a move to proof a stake you know the fact that now people are voting with their assets uh and it's very easy basically to trust your assets with professional parties that uh are experienced and then over

time that leads to basically pulled you know assets into a small group of professional entities which is mirroring you know where we're at today with web 2 right and so I think that fear uh was pretty prevalent back you know as the idea of the move of proof a stake was was being talked about and so uh the two co-founders of Obel uh Colin Meers and ocean Kine were both ex mesan they're they're both at consensus and basically started thinking about uh this problem in early days uh of how to make sure that ethereum stays trust minimize uh stays credibly neutral stays the

central ized you know and kind of resist again these these natural forces of centralization that happens with with proof aake and uh the idea of eventually matured into uh distributed validators and so the core concept of distributed validators in short is that today a validator in ethereum and with any proof of stake Network frankly is one node one validator so you basically have the entire 32 eth sitting on one machine if that machine goes down that note is offline right so there's no failover there's no redundancy now that has a ton of ramifications as a solo validator that

puts a lot of burden to make sure this machine stays up and you need to have devops experience you need to know what you're doing you need to know how to troubleshoot it right and that's a lot of Stak to risk on one machine so it's much easier much less risky for an individual who doesn't have that skill set to trust their stake to somebody else to a professional entity you know to a liquid staking protocol Etc right and that again creates that centralization impact that uh I was talking about earlier uh second is even professional entities they want to ensure that they minimize the amount of time a machine is offline because if that validator is offline they're not earning rewards and that lowers their potential right so what did they do the

only option today is for them to have a active passive setup May mean I have a machine that's live and running I have a failover that in case that active machine goes down that passive machine can be spun up and to basically be you know running that validator in Li the problem is if these two machines are ever online at the same time that same private key is on both machines and that's a double signing that's a slashing event and so that creates a ton of risk right even for professional validators who who want to try to keep that high availability so there's no easy way necessarily to like solve these kind of offline issues um and without without introducing slashing risk to basically improve the resiliency redundancy of of the network um and so

there's ramifications on the the solo operator side there's ramifications on the professional operator side and basically distributed validator allows this to be solved by taking a single machine and run or single validator I should say and running it across multiple different machines so now you're validating as a cluster of nodes with natural active active redundancy if one of the nodes goes down as long as three of the four in a cluster are up then that validator is up and running and then you can increase the number of notes to seven which increases the redundancy to two machines going down to 10 three machines going down etc etc and that lowers the barriers for uh solo validators and it improves the resiliency of larger uh operators as

well so it has a ton of ramifications across the board now why going back to your question of like me as Brett Lee why I was interested in this as I mentioned as I was getting back into the space I still think right now the things have that have the clearest uh need is still developer tooling and infrastructure projects and products right and I think the mission of what distributed validators has which is to ensure that ethereum stays decentralized that you know ethereum as a network stays trust minimized stays credibly neutral uh is an important one um and distributed validators is critical uh to

ensure that that happens right uh that's why it was part of metallic ethereum road map or merge road map um it's kind of one of the final steps on the merge itself uh and uh that's why I thought it was it was a critical project and I could lend my skill sets to it uh to help it grow uh to help it find product Market fit to help it gain adoption so that's uh why I decided to to join opal a year and a half ago I really liked the way you framed it as like pre- distributed validator technology we live in this world where it's one node is one validator and for the panics at home like yes I understand you can run multiple validators on one node we're talking about the other direction the one validator equals one node and and

that actually increases the risk which is if you have now multiple validators running on one node that's one machine still if that machine is offline now multiple validators are are offline right so it even just increas the risk there's no way to run one valid on multiple machines and that's what distributed validators uh distri Valor technology kind of uh solves for yeah and this might be kind of a basic Point um but I think there's something that gets a little bit lost in just the vocabulary in this space like the validator node and I think um you know like it's really helpful to think of the validator as something that is completely virtual it only exists with within the evm and like only has a purpose within ethereum whereas a node is an actual

computer like a real physical computer or part of like a server cluster yeah you're right but um like what what you're talking about with distributed validator technology is today without it like your validator within ethereum corresponds to a physical machine one physical machine and what DVT allows you to do is say okay we're going to say this validator is actually backed by usually an odd number somewhat divisible by 2/3 right but like five or seven or 10 machines and like the true key Insight here which you've already brought up is that what obal is doing is developing software that makes it safe

to run redundant backups of your machine that like today is not safe it is just not safe yeah 100% an analogy I like to use is actually um if you think of about a validator as an airplane right the body of the airplane with passengers in the airplane let's call it 32 passengers you know one for each eth you have there uh a node essentially is an engine and today we're flying on single engine planes and uh you know you can imagine you would probably not want to get on a single engine playe fly across the Atlantic Ocean like it's just probably not safe to do because there's no redundancy if that machine dies guess what that plane is going down right so now with J validator essentially is what

you have are multiple engines on a plane um and with a Boeing 747 you have four engines if one of those engines dies you know you're still okay because those other three engines will carry you to the other side of the ocean um and so it's another way to think about it is is you know a plane is the validator and the engines are the machines running it they're the nodes um and what you now enable is redundancy because you can have multiple engines powering this this validator and I guess this again might be a little bit basic but can you just talk through a little bit about why um it is I will never say risk-free but it's relatively safe to run an ethereum node until you start thinking about coming up with redundancy and I guess

like first like why might you care about having redundant setups but two like you you've briefly mentioned it like can you just talk through like what are the actual risks you're introducing by having this backup yeah so at the core of every valid is a is a private key right it's it's one ethereum key uh that uh you are using to sign attestations and to do block proposals essentially it's your key to having this valida be run and every private key is unique on purpose CU you want to prevent uh people trying to uh essentially Cil attack the network or hack the network and the the way to do that is by um you know for

example having the same private key multiple times and and you're trying to basically like mimic be a bad actor and mimic a true an actual good validator on the network and so inherent in ethereum as with all proof of stake uh networks there is slashing penalties which is uh it detects some bad behavior in the network or or incorrect behavior in the network and it's going to penalize you for for that and that prevents Bad actors from trying to hack the network a very common or the the the basic one for slashing is double signing if you are trying to do the same duties or attest to the same duties with two machines that have the same or two validators that have the exact same private key

that is a big no no you're going to get penalized in the network right and so because today again one validator one node one validator one private key right uh the only way to provide redundancy is to basically have a copy of that one validator with that one private key sitting in a different machine the reason you would want redundancy is if that machine goes offline you're as as you're performing duties uh proposing blocks you're getting rewards from the network you're getting rewards from from for from consensus right and if you're offline guess what you're messing out on rewards and so that decreases the amount of essentially yield that you're getting for staking ethereum um staking your e

so for more sophisticated operation ERS right um who wants to maximize the amount of rewards that you can get they're incentivized to try to keep up time as high as possible to keep every machine running 100% of the time ideally if they can right it's never exactly 100 but as close to 100 as possible and so the only way to do that is if you think about in the cloud sense like if a machine goes down another machine gets spun up no no nothing's lost right you're you're fine but in ethereum there's the only way to do that is as I mentioned to have a passive machine that can be activated when that active machine goes offline and that creates that double signing risk because if both

machines are on at the same time same private key uh the network detects it big no no you're going to get slashed and sorry just to like draw out an example here the idea is like so your primary like for whatever reason Falls offline and so your secondary is like no worries I got this and it turns out that your primary didn't actually fall offline it just like missed 20 minutes and so on the 21st minute it's like I'm back in let me sign something and the secondary doesn't realize that the primary is still in so it's signing stuff and now your redundant your redundant setup which is like done for this safety and like good nature operation of ethereum has now created first a security problem for

ethereum and in order to stop that security problem ethereum's response is to take your money that's exactly right yeah that's exactly right and and one actually uh Nuance here is that with off with offline uh missed rewards you're just not getting the rewards that you would have gotten if your machine was offline but a slashing is the E that you have staked we're going to take a piece of it and you're actually getting penal you're losing money in in that case or losing assets in those cases got it so makes so much sense um and I think I hope everyone realize like why this technology is so crucial because like in the modern world redundancy is how we create scalable like billion person

systems but the construction of ethereum does not allow us to have safe redundancy um and that's what DVT is so I think now is a perfect time can you walk us through how DVT or Obel with the Charon sorry for the pronunciation the client works and I Kant yeah I think it might be helpful to talk first through like as an e Staker what is the software that you're running and then now if you want to upgrade to become a distributed validator sorry a distributed validator like how does that change the dynamic of what you're doing on your machine yeah good question so there's basically two ways you can set up as distributed validator one is where you run all of

the let's call it say there's a four node cluster so four machines uh running one distributed validator cluster you can run all those four by yourself or you could have four people each running one node and operating as a multi-operator cluster right so those are two kind of distinct paths um for the first path basically what you would do is uh so we've developed what we call a Distributive validator launch pad actually our head of product uh was back at consensus and developed the original eth 2 Launchpad so when you you want to set up your ethereum validator you have now a Dap

You Go online it's a UI you walk through the steps and uh it'll walk you through exactly how to set up the validator we've developed something similar uh for setting up a distributed validator right so you go go online go to this web portal and'll walk you through the steps so the first step really uh is uh what we call a distributed key generation ceremony so you can think of now instead of having one private key each of those machines for machines within a cluster is running a key Shard or a key share those shares uh when they attest uh will through some you know fancy math be put back together and then create the

essentially the full signature to be able to sign as a single logical validator right so the distributed key generation ceremony creates the key shares and it's a one-time synchronous event you develop it or you you you run it and uh then you have the key shares then you set up the key shares onto those individual boxes as if it's a regular validator so really it just adds an additional step of that dkg and then you basically uh set up um the distributed validator as if as if each machine was its individual validator and it'll uh through Kon which is a middleware it'll wait for each of the it'll monitor each of the different machines wait for it to be online right and then basically once it's online uh it the Kon is what

Aggregates the signatures and then you know validates the entire uh sorry uh signs as if the entire cluster is a single validator now in a multi-operator setting the exact same steps the only difference is those key shares goes to four individual people and they would be setting up uh their uh individual machines uh as if it's a full validator again with Kon connecting the four so um basically the the difference is you just have this additional step of a dkg and then you're basically setting up multiple different nodes in order to activate this validator not just the single machine so just for the sake of Simplicity let's only talk about if our

clusters are operated by separate people and then you guys at home can figure out how to reduce that down to one person and it's just easier but um so okay so I think first like let's let's uh return back to this model that we have where Val ators are virtual entities that only exist within ethereum and nodes are physical machines that exist outside of ethereum so it sounds to me that like what's going on in this distributed key generation phase is that on the ethereum side on the validator side we're generating a private key that looks like every other you know validator private key from ethereum's perspective that's what your cluster will use to sign the

messages like all good totally normal what's interesting that's happening in obal and all DVT uh Technologies is that you're taking that private key that you generated and then splitting it into four parts and then I think the key magic is that you don't need all four parts in order to reconstruct the original key you just need sure it's configurable but you just need 2/3 of the keys in order to generate the full private key so first of all is that correct almost the only nuanced difference which is actually a pretty important one is that there's two ways to essentially create these key shares one is you have a full private key and

then you split it right but that means that at some moment in time that full private key is sitting somewhere on a machine which could potentially be a security vulnerability right um a dkg actually you create those key shares at the same time that full private key is never present ever even after you deploy it you can never actually get the full private key and that creates a lot of security uh value uh because as a hacker right if somebody tries to compromise that validator they can't just compromise one machine they can't basically you know have a man-

in-the-middle attack where like they can take that private key right when it's created and steal it uh you have to basically uh attack all four machines in order to get generate what is equivalent to that private key so that that's an important Nuance uh for for people that are you know a bit more educated on distributed validators it's an important one because there's a security Improvement that you gain from actually a true dkg which is you never have that single private key and you're not splitting it you're actually when you create the private key creating the key shards and the key shares yeah makes a lot of sense and I think like the big caveat that we'll have here is that we're not really going to get into like the cryptography of how any of this

works I think you just need to take it for granted that like when we say you can split a key then sign stuff with the split parts and then recombine it to so that it's like the original key signed the original thing like sorry guys just take it for granted take it for granted exactly if you want to read all about it there's plenty of uh articles on uh on threshold key signing and you know uh dkg and all of that stuff so there's a lot there's a rabbit hole you can go down for sure but yeah uh probably too advanced to talk about today very cool so I guess I kind of just gave away the punch line a little bit but can you talk a little bit about so we're post dkg and let's say you me vitalic and um who's another person tetron node are

all yeah are all decided to run a obal distributed validator cluster and so we've done their dkg ethereum understands us as being a single validator with a single private key we each have our shards of the private key which don't give us any information about the original private key but but can be used so can you talk a little bit about like from the perspective first of like just the Bret Lee machine in this cluster what's going on and then can you talk about the step where um the the I'm sorry the uh CH Charon the the you the Kon that your client um comes in and coordinates these four machines to like ultimately result in a single validator

signature I think we can simplify this in that um each individual node is just really to the person operating it it's just acting and behaving like a regular validator uh just doing regular attestation duties uh you know if it's a block proposal you're you're still doing that just regularly right the only thing is because when you're signing it you're signing with a key share that signature is going to is being sent to K on not directly to to uh the beacon chain right it's it's being sent to Kon which is a middleware and Kon is connected to all of the four machines within a four node cluster right and when that signature goes to Kon Kon all it's doing is waiting for the signature to come uh

it's saying okay three of the four four of the four have come let me try to aggregate these see if it's are are those all valid signatures and if so then once I have enough once I have more than 66% more than 2/3 right so in a fourn cluster it would be three out of the four I can now create the full signature for that entire validator and then send it to the beacon chain and send it to you know to to to the ethereum network essentially so really to the individual there's no there's no difference uh to operating a distributed validator node or distribut or operating the validator itself and so I think like the one piece of information that we may be dropped out of this was that from the perspective of cryptography a key Shard

and an actual private key correct me if I'm wrong but they're the same thing and so therefore you can sign a message with a key shard in the same way that you sign it with a private message right or sorry private key that's correct it just the key share would not be valid to if you sent it directly to the beacon chain which you can't but let's just say you could that wouldn't be a valid signature because it's not a full private key right and so you have to have at least three of the four in order to generate the actual valid signature for for the validator as a whole makes sense so let's jump to a 10 node cluster just to make the math a little bit easier but like let's maybe walk through um I don't know if this is the right framing but

like some of the failure modes so like totally Mak sense when all 10 nodes are like humming along attesting to the exact same thing sending the same thing to the car Caron um K Kon jeez um the Kon uh middleware client and then that's aggregated in sent to ethereum great so can you maybe talk through what happens like first let's say when four of the six sorry four of the 10 so less than 66% are offline and like kind of what happens there and then can you talk through what happens when we're not in an offline scenario but we're in a scenario where it's let's say nine of the nodes say one thing and the 10th says something conflicting like how

do how does I guess like the obal system work through some of these um like cons like tens tense consensus bits it's a good question so I think let's just uh start off with the scenario where four uh of the nodes are offline uh right and you have the other six uh Kon is basically acting as a signature aggregator uh that's that's all it does and it's trying to put these signatures together and to generate the full signature and so uh it because of the cryptography uh it requires at least two 2/3 of the signature signature to be valid in order to generate the full

signature uh and so in the case where the nodes are offline then it wouldn't have the adequate amount of signatures from the key shares from the individual nodes to generate that valid signature for the entire DB cluster uh and so it just basically doesn't send anything to the pacon chain because it says it's not valid we're just not going to send it right similarly uh in the case where you have you know Byzantine Behavior you have a malicious node within the uh the the uh the cluster um Caron will actually be able to detect uh you know again if that signature is valid or not if it tries to self slash Etc uh if it doesn't have again the amount

of valid signatures necessary to generate the valid signature for the validator as a whole or for the distributive validator as a whole then it just won't send anything to the beacon chain so it actually acts as a protection layer here right you basically have to have at least in this case seven notes compromised in order for any bad thing to happen with the validator in order for the valid to self slash or to do anything kind of malicious uh you have to at least have seven of the nodes compromis which creates a level of resiliency in the actual validator entity itself um to prevent against these malicious behaviors or offline uh scenarios part of the I hate to say trick here but part of the trick is that like with ethereum if you just don't participate when it's

your turn it's like not exactly but it's basically like net neutral right yeah you're just missing out on rewards exactly yeah where is if you do something wrong like the hammer comes down like the financial Hammer the ban Hammer like you're in trouble and so I think kind of what you're saying is that the uh K Chiron Kon Kon I don't know why that's we'll get this we'll get this right by the end of I know I can't believe we're recording this this is very embarrassing for me but anyway the Kon um client is leveraging that Dynamic to basically say like if there's any sort of question what's going on here just just don't show up right like we're

just not going to send anything and like yeah we might miss little attestation money we might miss like a huge me boost uh proposal but like if there's any question we're just not going to do anything because the worst case scenario is slashing and like that's not on the table when you use our product that's correct that's correct it's it's an extra layer of protection at the validator level that prevents against malicious behavior uh which today doesn't exist um because today if a validator is malicious it'll it'll try to send a malicious you know it try to sign on on the actual Beacon chain itself uh there's not that additional layer of protection at the validator level which is what Kon acts as and this is like a little bit of uh like just

detail but um like a cryptography detail but like let's say I have 10 signatures and I I hope that they're all valid um am is it pretty easy for me to check like every subset and figure out which ones can be combined and which ones can't so for example if I have uh 10 signatures eight are valid and two or not am I able to like quickly identify which two are invalid put them aside then aggregate the eight correct ones and then like present that to ethereum as a signature yeah essentially uh it's this is getting into a little bit into the cryptography but like it's it's based on elliptic curves and so it's not necessarily like like you're not adding things together you know you're actually

trying to like fit things into a math puzzle uh to find a DOT on this curve right and so yeah essentially if something takes you off the curve you can throw it out and take the next one right and so this is getting a little bit deep I probably would want my CTO sure to answer things more definitively if you will but uh yeah it's it's pretty uh you will be able to find the the valid signatures um and uh and be able to sign it so okay let's we'll jump back to our four-person cluster where it's you me um vitalic and super Fizz and like everything that we're talking about in DVT is so so awesome when you're thinking about like redundancy and like

your ability to uh just like create a more distributed it it works so well when you really trust your cluster right problem is we're in a trustless space and I the thing like I'm specifically thinking of is in our four-person cluster let's say um if I don't know if you can tell but I'm in Mexico right now let's say like there is a problem at my house and my computer is just down and then Brett has like really pissed off his ISP so they say like no internet to your house and so now like super fit in vitalic are like Rockstar e stakers they're only doing the right thing they've done everything right and now they've entered into a situation where like their trust in us has resulted in

like penalties for them and so I'd love for you to just talk through like the the trust implications of this like trustless solution to centralization yeah it's it's a good question and to answer that question fully I I do uh need to go a little bit into kind of how opal has designed uh our implementation of distributed validators so as a design philosophy we wanted to build opal in a way that basically allowed it to be modular allowed it to be used in all the different context of staking right and the most pure essence of doing that is

to build it as a middleware as essentially a new DV client that sits inside the validator text stack it sits between the validator client and the consensus client right now with that said we're not changing any of the trust assumptions actually in this case uh around the validator itself and so in our current version uh which we're calling V1 uh you essentially are treating The Logical DV cluster as a single validator and so all the people within that cluster we're not making any assumptions of that means the onus of trust is on The Operators themselves today meaning like I need to trust these three other people within my cluster

right to an extent right one of them can be offline Etc they have time to come back on but they all need to be good actors I need to know like that I'm in a cluster with them and um you know uh if uh a couple of people are offline and there's some permanent issues then we have some social consensus to be able to say let's exit the cluster and let's create a new one right so that onus today lies on The Operators of those clusters now with that said we know the future looks to be more of you know we want to develop something that's that's even more trustless than it is today today we're we're saying it's trust minimized right but it's not fully trustless you kind of have to know the other people within your cluster we are

doing uh some like we're doing a number of uh different areas of research to think about what that what we call V2 will look like uh where it's it's more fully trustless because then you're getting into penalties and you're getting into you know tokenomics and and those there's a lot of there's a lot of uh interesting areas that need to be developed um in order for that to happen uh so so that's the answer I guess to that question and and again the last thing the reason that we did this is to basically uh have a product that would be able to be used in various formats today right and so like as a for example a a professional operator you know I

trust myself I'm the only one operating it there's no trust assumptions here but there's still Val value to DBT uh to be able to like have redundancy within my architecture and Obel allows me to do that right for people that uh you know uh for liquid staking protocols that are running in many different operators you know they can uh have DVP implemented in a more kind of trust uh minimized sense and there's Community clusters being rolled out today uh that people know each other and they have friends that they've staked with right and they want to now have redundancy across that so that those are the use cases we tackled first and we will tackle the kind of more trust fully trustless uh type of structure as part of our V2 makes a lot

of sense so I think this is a good uh moment to Pivot into like kind of the implications of DVT and of obal and I think we we've touched on a lot of them throughout the last 45 minutes so uh don't want to be too fantic but um like yeah I would love to just hear like how you see this transforming what ethereum is and like let's let's just take the easy ones off the table it's like reduces the cost of U participating in staking from 32 e down to a more approachable number right um I think that's the big one like good thank you yeah for Community for Community validators yeah the the two that I am really curious to like hear um your thoughts on are number one the

implications for liquid staking providers and like of course the big elephant in the room Li though right love to hear how uh DVT is going to transform just the space in general and maybe any like specific like projects or or things that are happening right now and then the other thing that I'd like to talk about also super topical is uh we've seen so much communication about the importance of client diversity over the last I don't know 3 weeks whenever the Bess bug happened and then like two days later the nethermind bug happened and I would love to hear um the implications of DBT technology not necessarily for like creating these clusters of PE of different people to

create more decentralization and redundancy but like for me myself like what benefits could I like I guess I've already like given away the answer here right but like I myself could run a cluster that has all the different combinations of execution clients and then know that if one has a bug like that is not going to take down my whole stake so those are the two big ones I have on my head the implications for lsts and the implications for client diversity um and so love to hear your thoughts and love to hear if there's any other big categories that you see as DVT just changing the game on the liquid staking side um there is a big push in the community and I think there has been for a while uh to make sure that liquid

staking protocol stay decentralized uh with Lio obvious viously being the one kind of in the direct Target of that because they just they're the market leaders they have a lot of stake you know so they're running a good part uh portion of the network the challenge with liquid staking protocols is that they're putting a lot of trust into their node operators right in Lido's case they have 27 node operators each of them runs like 1% of network State it's a lot it's a significant amount of assets uh and and because of the trust assumptions that liquid staking protocols need to put onto the node operators they really can only trust the most professional node operators right

the ones that are battle tested the ones that have gone through it that know what they're doing that won't go offline they won't get slashed you know that can be trusted frankly with this amount of stake that creates a very natural centralization and effect on liquid staking protocols and you know Lio uh and others have realized that in order to decentralize they need to change some of those trust assumptions that's where distributed validators can come in is because now if one of the node operators goes down you have failover you have redundancy right if one of the node operators frankly go malicious or try to like you know do something bad on the network you now have three six nine other node operators

who are basically minimizing the amount of risk that that can posee on the protocol right and so distributive Val is a critical technology for liquid staking protocols to be able to decentralize their node operator set and for to get more medium-sized Community valid heck even solo stakers involved in being a node operator for liquid staking protocols right and so Lio uh we've been working with them for quite a while and recently uh we on the path to launching the first module additional module on liov V2 uh which is their next generation of how they're managing their note operations and uh this new module is called Simple DVT uh and it's the

first time that Su of validators is going to be implemented into the Lio protocol uh as part of their note operator set um and the reason for this is again to allow them to start now onboarding Community validators into their note operator Set uh and to have safety and be able to trust that they can safely do so right so simple DBT uh we just passed uh a critical Milestone which is our uh our test net with them uh we had minimum bars for Effectiveness uptime kind of performance metrics that we needed to pass as part of this uh test net in order to get approved or move towards approval for fora deployment uh we are happy to announce that uh all of the performance Bars were

passed uh and we uh shared a report recently about that uh and so we're on kind of the final step towards Mana deployment of simple DVT which is quite exciting it's it's going to onboard now 200 new node operators onto the Lido Network today there's 26 so essentially with a single module you've almost 10x the amount of node operators on on Lio uh and this is only going to grow as we uh you know put on to as we Implement onto mainnet uh we're going to run it for a while see how it performs and then over time that module is going to grow and I think it's a great test beted for the impact that DBT can have for liquid staking protocols uh running uh in a in a uh to decentralize

kind of the noo operator set for for liquid staking protocols I'll also mention real quick that uh we all we've also been working with ethery uh which is another liquid staking protocol for quite a while uh they uh actually started um with a more permissionless model they wanted to have a decentralized permissionless node operat almost from the start and so together with them uh we launched uh What uh we're calling operation solo Staker uh which is a program to basically sign on uh individual uh solo stakers individual solo operators uh to be able to be node operators for uh for ethery and um that has also uh used DBT as at its core uh so we now see multiple liquid staking

protocols leveraging distributed validators to basically create safety while still be able to decentralize their not operators yeah awesome man and I think like we just have to take a moment right here and just like congratulate you and thank you you Brett but you obal because like look like the the Lio problem is a real problem and it's one of those things that we were I mean it's tough to say sleepwalking into because of just the amount of noise about it but it's just something that was happening despite how loud the community got about it right and I think that we were gifted a gift from the ethereum Gods which is like say what you will about igen layer and this whole points thing but like it has

created the momentum needed to like create other viable lsts in size so like that's incredible but they're creating the exact same problem that Lio did and so that's why this like obal moment is even more important and even more special because not only are you like actually doing something that like takes us in real steps away from the Lio problem and like the Lio problem have taken to its fullest extent is like the end of ethereum like we've just created slower shittier AWS right so um I think I think not only are you doing big things for Lido but you're doing big things for ethery and soon I'm sure puffer finance and after that maybe fra uh eth and like all of these staking

providers that are doing interesting things like within the evm but like at a cost that is so high in terms of centralization that like it it draws into question like the whole Enterprise so first before I move on just congratulations on like passing all your tests on getting closer and like on on on doing big things for the Lio problem so thank you I appreciate that the one thing I will say though is that in in working with the Lio team I think um you know in general they also recognize the importance to maintain decentralization uh and I think you know that's why they have been so uh proactive in delivering

simple DVT and kind of you know they've also started building out a second module called their Community uh kind of uh validator module and so uh I I think I think in general like lier also recognizes the the potential risk of of of being too centralized but uh and I think as a whole the entire Community is just more aware of the the risk of centralization uh especially as um there is more excitement around staking there's more now with reaking that added excitement right it creates additional need and additional uh um importance on making sure that the node operator set of every liquid sticking

protocol is decentralized and truly decentralized uh because uh it creates uh risky forces in in the market if if there's too much calization in any one part of it all right unfortunately we're running long time so I'm going to have to say this is the last question but man this is I I really could go on for hours and hours so with the last question here um I fully feel you and I'm on board with like this is the right way to decentralize like the technology and the infrastructure behind lsts and um like this is all about the node operator set and like the physical machines and and like how to create decentralization in the real world my question for for you is as we enter this

new meta where we get more decentralized on like the machines contributing that almost kind of gives us the permission structure to centralize all of the actual value into like one or two single tokens like within the evm and so like I find that it's kind of funny that like the more we decentralize outside of the evm the more comfortable we are centralizing our assets within the evm and so I wonder if um like kind of where you fall on that observation like one tell me if you think that's just asinine but two like do you see that more as like that is the next problem we need to solve and we're just going to leave it to the people building within the virtual machine or

do you see that as like no no that's not a problem that's a good thing like what we're doing is creating like a centralized money Lego for defi that is backed by a decentralized trustless network it's really interesting question right um I think there is this balance that needs to be struck between um having standard ways of operating or transacting in the ethereum network versus creating single points of failure and I think that constant kind of balance and battle will continue for as long as web 3 is is around which is you

know going to be a long long time right uh yeah um so I I think the answer to that question is you know um just because that there's DBT doesn't mean there should only be one liquid staking protocol let's let's take that to its extreme right because there's other risks that are involved with having only one liquid staking protocol uh smart contract risk there's you know um uh I think just social risk right uh etc etc so I I I think there needs to to be this balance being struck uh between that um now you also don't want to get too far on the other direction where everything is too scattered right I think with

layer twos there's starting to be there's just a lot going on there right and so there's some standardization that will still and interoperability perhaps that still need to be uh principles that still need to be still need to be applied so um I don't know if I answer your question directly but I I think in short like there is this kind of Natural Balance and I don't have a clear answer about what that balance looks like right cuz I think the ecosystem will kind of evolve and figure itself out I think we have a good way as part of the ethereum ecosystem of self-governing and like you know if everything is going too far one way kind of having the community push back and push it the other direction so I trust that we'll find the right balance for all of this um in the future

but the goal is to continue to like remove and reduce single points of failure within the network as a whole and from a Distributive validator standpoint I think that's one critical area that we're we're focused on which is reducing that single point of failure at the consensus layer of ethereum and I know we didn't have yeah sorry real quick I I know we didn't have the time to cover it but the client diversity topic is actually pretty pretty critical for this as well let's spend five more minutes why don't we talk about it okay yeah so the Cent diversity problem is is quite interesting because going back to I think why some of the the the issue exists is because you want to be using the best inclass client when you only have one choice

right if you're running one machine you're running one validator like I want to make sure I'm using the one that everybody else is using cuz I'm the saf the safest I feel the safest because everybody else there's social consensus social support but that naturally creates a risk to the ecosystem as a whole because if there is an issue with that most used client gu right then a massive percentage of the network is is is impacted and so with the tributed validators you now don't need to make that choice because you can have with each node in your cluster a different client configuration different execution client different validator client different consensus client you know and you have redundancy now uh and so it it actually

makes it much easier to get client diversity because you're not making the choice do I go with a minority client that maybe is a little bit less proven or do I you know use the majority client that everybody else is using in increase ecosystem risk you now don't have to weigh those choices because you can have mix of clients within the nodes within your cluster so I think it is the actual solution to the client diversity problem uh is to be able to run a multi client type of setup in every single validator that that you have yeah and I think that this makes so much sense for the professional staking providers like why wouldn't you do it as a I I'm a rocket pool home Staker so like I'm I'm just trying to think through now like am I really going to go buy two more

computers and like maybe Co Lo at them maybe put them somewhere else maybe I'll get like an Amazon uh validator but then like what am I doing for and so like I do look most of the stake is with the big providers anyway so this actually might just straight up solve the client diversity problem and then I think too that as Solutions like DVT make the big players more comfortable with all the clients like that just trickles down into yeah you know like even as yeah but even as a solo Staker you can now have client diversity Within your cluster right so even if you're still running a single node yourself and you're maybe working with three other people to run a cluster each of them could have a different client configuration and so

like instead of running you know a single node with 32 eth maybe you're running four nodes on a single machine still right but with with four sets of different clusters of different people each with different clients so there's a lot of now more configurations you can and flexibility frankly that you can you can choose to to operate with that you didn't have before and so I think that's still a critical point even at the at the home state level yeah and so like are are you I won't make you take a position on behalf of obal but do you personally foresee like once uh DVT and I guess specifically obal's DVT is like permissionless and accessible to like everyone just with a few button clicks like do you foresee that um home stakers

like are going to be a major part of like your user base or like at least for like this like hyper growth like you only have limited number of resources face like is what you're doing today really about um like finding those like professional Staker opportunities and help them understand how much they gain by like building in their resilience and the trustless of DVT yeah so I I would say both are equally important for us uh we're we're spending just the amount of just the same amount of effort on home stakers as we are on professional operators uh and we didn't have the time to go into kind of the different things that are happening on the home Staker side but I'll just say that the first

set of community clusters are being started super Fizz actually ran uh the first couple uh you know we are now talking to uh you know hacker houses in Africa who want to like try to get more staking um into the continent uh you know we're uh educating uh we recently did a a staking block with shifi uh to try to get more more people staking um there as well so you know what I hope to see I don't know what I I won't make any predictions on the future but what I hope to see is that the barriers to entry for home staking gets lowered more and more over time uh and that we do see a higher amount of participation for me

at least medium small validators uh and even individual like solos like true kind of solo operators uh to have them have more opportunities uh to be able to participate in operating on the network um one last thing I'll say and I know we're probably over time is that uh one of the things we're hoping is to build an onramp for more uh home validator more solo sers to participate a big part of what's I think is missing in that is credentialing is um being having individuals be able to prove their experience cuz like with larger entities they have reputation right like I'm figment or I'm block Damon you know you know us we have credibility but if

you're an individual sitting at home even if you had 10 you know 5 years of experience uh validating uh and running uh uh validators there's no way to prove that other than your word uh and so what we're doing today uh we recently launched what we call the opal technique credential which is an SBT that we give out based on objective metrics of uh of of validation so you have to maintain a certain amount of a Time certain amount of Effectiveness run a DV for a certain amount of for certain amount of time and you get to earn this credential and what this credential gives then is uh we're working with Lio and ethery and other liquid saking protocols to gain credibility behind this credential where they can see this as okay this is a proof that this particular individual

has experience and has proved experience to be able to run a note at this level of performance that makes me more comfortable to allow them to operate in this in my community operator set for example right and so I think there are some of these that are also still missing in the ecosystem um to get more opportunities for home stakers that we're also trying to Hope or that we're hoping to be able to fill and at least contribute um to the development of having more home stakers participate in the network yeah and I and I while you were talking I just kind of had the realization that the point that I was making is like really really specific and yeah like I guess maybe there's going to be people like me that are interested in increasing resil resiliency by running like DVT techn

technology within like my own Empire right but like the much more important user from like the home Staker like standpoint isn't going to be the guy who wants to create resiliency within his own setup it's going to be the person who wants to participate with other people and like maybe that's with a group of their four friends or maybe that's like in a you know I I don't know how good the technology Works yet but like I could imagine a world where it's like a 5,000 person community cluster and you want to opt in to be the 5 ,000 in first person and like yes you don't trust these 500 5,000 people but we're at like the the number of people where you are like back into trusting statistics and so um like that is really

I'm sure what you mean when you say that they're that home stakers and and that ilk are just as uh important to you as like the professional operators and and and some of those dots are being connected right because if you think about with Lio with ether they're professional liquid staking protocols right the largest uh some of them in the network they're they see DVT as a way to onboard more Community stakers onto their Network right and so I think that now you see this interesting kind of opening up opportunities for home stakers even if they don't have their own e even if they're not solo staking to basically earn enough experience and credibility uh to be able to now be a validator for Lio be a validator for

ether and there's these on-ramps that are being built to allow them to do that and DVT is at the center of it because without DVT none of these Community uh modules or Community node operator would be would be possible yeah awesome man H frustrating like there's only so much time in the world and honestly like we didn't even get to like some of the more interesting Parts like you're going to have to come back like I really want to talk to you about the implications of DVT and then in latency and like we're already seeing so many games around like how the if you have enough Stak and you're willing to delay until the last possible second the implications on stake return and now we're adding in like a new step like what is that mean but sure just for the sake of your

schedule and our listeners uh ability to pay attention I'm going to have to cut us off here so Brett I just want to say first of all thank you again and also congratulations to like what Obel has done so far but really like the achievements that you're starting to like publicly list off with uh with Lio and with ethery and with all these other like massive pillars of ethereum so thank you and congratulations thank you so much Rex thanks for having uh me and yeah looking forward hopefully to have another opportunity to chat again soon of course well before I let you go can you share with the audience um how they can find you how they can find obal and if they're interested in DVT and like maybe um becoming an operator eventually like can you just talk a little bit

about where you are in the process and like where people should be looking for when you know it's open up to the public so we're we're already on uh main at open Beta meaning if you're uh somebody that wants to run a DV you can do so already uh you can find us uh on ob. Tech our website which will navigate you to the various places you can go uh join our Discord uh you can do it from there uh if you want to learn more about the technology you can do it from there as well uh me personally happy to you know talk to anybody that wants to connect uh you can find me on uh Twitter uh osus um and and also telegram on osus awesome man well uh I'm just going to say it again just thank you thank you thank you and congratulations and um

yeah man I hope to see you again on the podcast soon so uh Brett thank you and have a great day thanks BRX