Izzy - Upgrading Ethereum, Upgrading Lido

Transcript

Hey everyone. Um, okay. I think it's working. All right. Uh, upgrading Ethereum, upgrading Lido. When I wrote this title, it sounded a little bit better than it does right now. It sounds like a coming of age movie, but to a certain extent, it is. Uh, what happens when a protocol grows up and it has to face a lot of costs. Um, the point of this talk is not really to shill anything. It's kind of to give you guys like an educational introspection into how when the base layer evolves uh and upgrades, how like large actors or or participants on the network receive these changes uh how they need to think about them and what it means in terms of like how do you implement how do you stay at pace together with the evolving uh base layer and what it means for the users for the app layer uh and for the

ecosystem in general. So yeah, how network upgrades inform the trajectory of lighter specifically. Um, I'm Izzy, uh, like Ken said, Freeborn, uh, in in a bunch of like the Discords. Uh, and as of May of last year, I'm chief of staking at Lidol Labs. So, in general, we're going to cover like what are upgrades? Uh, it's pretty obvious, I think, from like an Ethereum perspective, but I'll try to distill a little bit what that means in terms of like upgrades from a protocol perspective, understanding how a protocol uh, like LTO is affected by Ethereum upgrades, in which cases it is, and which it isn't, and how we assess those upgrades, and what we do about them. um example uh upgrade that we'll take a look at like EIP7251 which is actually quite topical and then just some closing thoughts. So Ethereum is an

everchanging protocol and Lido must keep pace there is a tension uh I guess in general in governance circles around are we aiming for oification uh at some point in time like how great is trustlessness and immutability in terms of like an end goal and to what extent should this be an ultimate objective for everything from the base layer to the apps that are uh on it. I think in general like those are like pretty good goals but in a lot of cases it's very very likely that immutability isn't possible or at least it isn't possible until the base layer stops changing and at what point does the base layer change like we don't really know in order to continue to be a force for increasing the accessibility utility and participation of everyday users in securing the network like that's what

Lido does in my opinion um onchain staking protocols in general whether they're lighter or not have to maintain feature parity with the underlying network but there's a couple of key implications that result as as as a result of like this this premise. So one is it's practically infeasible to create a successful staking protocol that's immutable. And by practically infeasible, we mean that like maybe it's technically possible, but it doesn't make sense because nobody's really going to use it. Um from a technical perspective, since the consensus is constantly changing, there's way too many future unknowns to be able to say like we don't need to we don't need to make any more upgrades and now we can have an immutable protocol. Um, if you're an LST, if you were to have immutable protocols, basically every version would need to be a new protocol.

So, this means that multiple versions would lead to fracturing liquidity and complex dynamics with regards to integrations in CFI and D5. You would need to integrate new tokens every time. And then thirdly, you'd need to ask users to migrate from like the one version of the token to the new version of the token. And this is horrible UX. Like nobody's going to do that, and that's not a product that anybody wants to use. Uh, and there's a fourth point here which I forgot to add, which is like security implications. So immutability is great if the underlying system is relatively simple and you can be pretty sure the first time that you got it right. But as we can see in DeFi, uh there's a lot of bugs that like or exploits that rear rear their heads much much later. We saw with balancer like four years later a tried and tested

protocol um was like surprised by like an exploit that maybe we would have caught if it was audited now but we didn't catch back then. you see things like compiler bugs uh like in Viper you know that end up again two years after something has been pushed to main it exploiting protocols. So immutability is a great thing from like a values perspective but it also introduces complexity and potentially even risk from a security perspective. Um and secondly upgrades for onchain and decentralized protocols um are complex are expensive and they basically result in meaningful disadvantages via centralized alternatives and competitors. So when we think about what the market looks like five years from now from a staking perspective, we need to take into account what the costs are for each of the different actors in the

space to adapt to the changes that are introduced by um new changes in the protocol. So this is like a very very high level and basic way of how we consider potential upgrades um that are being rolled out in Ethereum. So the first thing that we consider is who is affected and or what uh and how. So for example, is it a consensus layer or an execution layer upgrade? Does it affect users in our case stakers, node operators or the protocol or some mix of all three? Uh and is the effect economic, technical or operational or again some mix of all three. Is the upgrade itself important uh and or urgent? So it might be urgent if it's security related. It might be urgent if it's breaking. It might be urgent if it adds u like a feature that we desperately want or something that's

really really important for the network. Um and so in some cases you might have things that are important but not urgent. Um urgent but not important. uh and important and urgent at the same time which is like most cases and finally what are the associated benefits and costs of both the changes itself uh its consequences and the work to effect like makes basically to make the change come about this is our upgrade process like the steps that we follow again actually a very simplified version if I were put the real thing on here it'd be like four different slides um and I'm going to go through it really quickly just to give you guys kind of like context of how we tie this together with those those guiding questions so first we identify and we scope changes. We actually have a team uh like our

research team that reviews EIPs and their job is to figure out how might this affect lighter? Does it does it affect it at all? Um if we decide that it affect us and that's something that we should upgrade, then we design technical and operational specifications around how we're going to implement this upgrade. We do this via aligning product requirements and technical requirements. Then we do like an early vetting of what we propose to do here. We do this both with internal teams and we do it with external stakeholders like node operators or perhaps researchers at the Ethereum Foundation. Um once we've done that then we approve the technical and operational design like a high level it's not super high level but uh it's more like an architecture design document um by the DAO and once that's approved and basically we we think this is how much it's going to cost us give

us this money to go about making this upgrade happen. We start actual like like designing the full technical details and preparing the implementation. So we do that in a manner that quite closely resembles what upgrades uh look like from a preparatory perspective for Ethereum. We have local devets, then we have epheirmal like shortlived devets. Then we have isolated test nets. Now the canonical lighter test net is on hoodie. Um and so that's like the main test net that we use for everything. It's almost like a QA environment for for our mainet instance. But we don't deploy most of the changes directly there. And we might have first like like a change in its own small isolated test net where we fully control it. And then when it's time uh to finally mimic like let's say mainet we would attach this to our canonical test net which is light on hoodie to enable

full endto-end testing with all of our different stakeholders. So that's users that's node operators like we have hundreds of node operators running on on on hoodie. We pay for no safe to exist there basically because like it doesn't exist as an application apart from that. So we need to mimic like real mainet infrastructure and make sure that everything works from Dow votes um to node operator onboarding and management and and of course like usage of the actual network and then we prepare the release for mainet. So this includes like deployment plans, testing scripts, uh a full test suite for upgrade functionality once it's actually landed on chain uh and the actual vote uh on chain. So all of this comes together and basically results in product uh sorry uh protocol upgrades being multiple stages and of an iterative nature of internal and external validation testing

alignment coordination and execution. So shipping an upgrade to a protocol like Lido uh which is decentralized and fully on chain resembles Ethereum governance like in a few ways but obviously not in some others and I want to establish those caveats first. So in so far as supporting new features is concerned, we are constrained by what the base layer actually ends up implementing. So what do we mean by that? A lot of times when we implement something from a technical perspective, we want to make it relatively simple. Simple things are harder to break. They're easier to reason about and they're easier to to do something or to fix if something breaks. From an Apple layer perspective, it actually constrains quite a bit what you can end up doing with this. Uh an example for example uh sorry an example on 7251 is we had this idea initially

that we might have different caps for what the max effective balance of a validator might be ranging starting from 32 all the way to 2048 uh you know in multiples of eight and and different things like that. Eventually, this was done away with from a simplicity perspective. Like, you will either have a a thingy, I call them the 32th validators or a fatty, the 20 48 validators. I can say it because I'm overweight. Um, and and and essentially like that's probably great from a protocol perspective. It means from our perspective that we're much more limited in the product decisions that we can make. So, for example, if you're thinking about about bonding requirements for a 2048 validator, it really constrains the design space. Um secondly, we have less time to execute.

So specs come together and get finalized for Ethereum. Actually like pretty late in the process. Huge props though like to the teams especially in the last two or three hard forks. This has gotten a lot better. But I think like the 7251 spec like finalized like a couple of months before uh it was rolled out. So this means that for certain protocol upgrades, especially if they're urgent, it's very very difficult for protocols to actually do this instep with a network like launch the upgrade day one or day zero. And sometimes this actually ends up with you needing to do like a band-aid uh upgrade to make sure that things don't break in the interim period between when you can fully support it and when it goes live on the network.

And third, um obviously like lighter is a DAO. There's an onchain governance perspective here. Ethereum does not have onchain governance for chain upgrades, but there are a bunch of networks that do. U and the difference here is that the coordination that Ethereum does uh via kind of like soft governance and diplomacy and politics to a certain extent uh we have to do via onchain mechanisms. Um and while that may not be the best solution, it's probably the only practical solution on an app layer basis. So let's kind of examine going back to EIP 7251. So the upgrade for max 2048 and consolidations to kind of like distill this process through what was the primary objective of 7251 to reduce the set of active validators in Ethereum consensus. So we're talking about number

of validators like technical units um not not entities themselves to make the network more efficient and to enable further scaling. So what are the pros? It improves the network footprint and efficiency. There's fewer validators to manage and the ratio of like actual validators as technical units to entities running them uh goes down. Staking rewards can compound. Uh consolidations allow for things such as validator key rotation without needing to write the full entry and exit Q. Uh and validator top-ups may be more efficient operationally than multiple validator deposits. Now, there's like a bunch of other stuff, but for us, these were the main pros. From a cons perspective, it means the Ethereum protocol has to support yet another type of withdrawal credentials. So, there's a serious amount of tech depth that you need to think about. And there's also intricacies in terms of how the

different withdrawal credentials like work in the system and the different things that they can do. Um there's a really interesting thing that came up recently where basically you can use consolidations as a way to like quickly withdraw and that that is like kind of like weakening weak subjectivity or I guess there's a debate about that or not but anyway um and another one was at least in the initial spec was there's an incre increase in slashing risk or rather slashing impact. So if a slashing were to happen due to what the minimum slashing um value was when the spec was first written uh it would actually be like really really dire but thankfully that was changed. Um and so huge props to like researchers and the EF and client teams for like hearing concerns around that and of course increased centralization pressures. So how do we

take those pros and cons like at a general level and interpret them through the lens of like lido? Um the primary objective for us is to basically help the network reach these goals of reducing network load like being good citizens, increasing network efficiency and adding new features to the protocol. For example, like uh key rotation, which is often asked for for by node operators because maybe their key management practices change over time. Um rewards compounding actually turns out not to be a big deal for Lido. And it's not a big deal because the protocol was already compounding. It was taking the skin rewards from the smaller validators and turning them into new validators. So there's only marginal increase here and only like a very very large amount of validators. If you're a small to midsized operator, it actually turns out

that it's better to run your validators in like 32E uh type of route credentials. Um and this has to do with like the what I call like the tick rate of effective balance in in Ethereum, which is basically how fast effective balance increases uh on a validator. Also props again to like research folks. they've acknowledged this and there's already some things ideas in the works about how to possibly improve this and make it more economically advantageous to consolidate. Um most staking protocols and this is really really big one were written with the assumption that a validator would have a maximum possible balance of 32 ETH. Uh and there's a lot of work that has to happen from a core accounting changes perspective to to change this to to allow protocols to either use both or

it'll only use 2048. Um, but when you think about what it takes to implement something like this, you eventually get to the realization that you have to allow for both because there's no way to transfer all of the stake from 32 based uh 32 Ebased validators to 20 48 based validators overnight. Um, in reality, both of these withdrawal credentials need to be supported. Um, and oxo1 max is also much more manageable for small homestakers uh due to the bonding requirements that we were talking about earlier. So it's very likely that the lighter protocol even if it's even in its future state will need to have a mix of oxo2 which would be the majority of stake and oxo1 for homestakers. Um and which means that we'd need to support modules like modules are kind of like proxies for specific sets of validators in lighter. So we have the curated

module which is permissioned and CSM which is permissionless for example. um to support probably either one or both of these withdrawal credentials from a simplicity perspective. You want to support only one and so that means having multiple versions of the modules if you want to be able to allow node operators to benefit from both. Uh the reduced initial slashing penalty is actually great. It means that we can reduce the bonding requirements which means that we can make staking more accessible to new node operators. Um from a consolidation perspective, blast radius management actually means that you end up running either the same or more amount of nodes. So what that means is that even though you can run more validators or like more ETH on a single validator client because like the the node ops requirements go down from a

bandwidth perspective, you actually don't want to do that from a security perspective because if something goes wrong with that specific validator client or that machine um then you have a higher risk of getting slashed. Uh and so what you want to do is spread out the amount of ETH that you have per like let's say instance or pod of stake uh and maintain that amount of ETH. So if you were running, I don't know, let's say like a 100 validators right now with 32 ETH on each of those validators, you want to run roughly the same amount of ETH. Again, if you're keeping the same security practices, even though the validators will have a larger balance, uh partial withdrawals and deposits may increase ops costs. This sounds a bit counterintuitive because it sounds like partial withdrawals are actually great, but what it turns out is if most of the

network is consolidated, they will be used quite a lot and they get very, very expensive the more that they're used. Um, and then there's very high implementation complexity. So cost and overall low direct value return. What I mean by this is that because large protocols like glad don't really affect from compounding and because there's such large changes that are needed in like the base infrastructure of how the protocol works, there's not a lot of direct value to the protocol here. Most of the value actually comes from just supporting the network. So what does an upgrade like EIP7251 cost? So there's over 6,000 manh hours from a tech design and implementation perspective. like this is just amount of people that are working spending like let's say a couple of weeks on this to design um the the technical implementation and including

the amount of time taken. So this is like a a bunch of months work for 10 people more or less. Um from $500,000 to $1 million in audit costs. Now this is assuming you would I audit this thing in isolation and we always have at least two audits for everything that goes onto mainet. Um the audit cost can come down if you're bundling this with like other audit scope. uh but it's still like very very important. Um over 15 endto-end test scenarios and like each scenario basically has maybe like 30 tests inside it in terms of what we test with external stakeholders. So every curated node operator when we go live with consolidations we'll end up doing somewhere between like 20 to 30 entend tests together with us and three months of testing um on an internal external basis once we get to like devets and and

test nets. Um so where does this leave us from like a closing thoughts perspective? Immutability is a really lofty goal and it's something that we try to work with but realistically from a lighter perspective there's only certain parts of the protocol that will probably become fully oified. Uh for the rest we're trying to get those trust as trust minimized as possible. So it moves it means moving away from trusted oracles to like trust minimized oracles using Zika technology for example um or offloading some of the implicit trust uh to to teams to more automated mechanisms. Um the industry generally under underestimates the complexity and cost of keeping up with Ethereum. Um we get a lot of questions like why has Lido not consolidated now? And the answer is it's pretty difficult. Uh and it's it's

very difficult if you want to do it right and make sure that like nothing gets messed up. Um this cost can and should be considered basically a decentralization premium and it applies to any fully onchain and decentralized protocol. Like Lido isn't the only one. Uh and the funny thing is that the cost is more or less static regardless of which protocol you are. So smaller protocols have a more difficult time in dealing with this. Um the alternative to this is basically like not having those costs and giving up kind of like competing in this industry. And this means that we end up with more centralized and tightly controlled service providers like essentially running the safety of the network. The cost to maintain feature parity with a core protocol generally increase over time because complexity and tech debt increases over time. Uh and they're generally much more substantial for

decentralized protocols than they are for centralized protocols. And lastly, there's quite a few similarities between the upgrade and governance processes um of Ethereum and like large onchain decentralized staking protocols. Thank you.