Lighthouse: Road to Eth2 mainnet

Transcript

hello everyone whoo let's get started my name is MIDI co-founder and director of Sigma Prime and I'm here to talk to you guys today about lighthouse and issue I'm super excited to be up here super excited super pumped for the latest developments that my team has achieved and I'm really really proud to be a pair sharing the latest progress with you folks so that's the agenda for this talk I might have to skip some of the content because I only have 20 minutes but feel free to ask questions afterwards very briefly for those of you who don't know who we are Sigma prime or CP for short isn't infirm in an information security consultancy

mostly based out of New South Wales in Australia we're basically exclusively working on blockchain technology and mostly on ethereum we're a group of security engineers academics and software developers who want to build a better and secure decentralized world so I've listed a few of our Information Security clients we're very very proud that a lot of those prominent projects in the space trust us with their security assessment requirements but I'm not here to talk about InfoSec I'm here to talk about lighthouse so lighthouse is a little baby we've started the project in July 2018 my friend and

co-founder Paul how now I started this as a side project and yeah so it led to a fully fledged client by now quite quite exciting we're building it in rust so rust is this amazing fast and secure language if you haven't sort of heard of it if you're a developer I highly encourage you to start playing with it it's quite it's got some amazing safety properties it's obviously like houses security focused it's got to do with the DNA of the company I guess and I'll be talking about the optimization work that my team has been busy working on over the last few weeks but first I'd like to

take some time to talk about the our key sector of lighthouse to make sure that some of these concepts are understood by everyone so very briefly not everyone knows this I've had some conversations with the community over the last few months I don't think this is clear for everyone but lighthouse and most likely all the youth to clients will actually be shipped in two separate binaries so you've got your beacon node which you can think of it as the Geth or parity for e2 so that's the piece of software responsible for state transition logics storing this type in the local database dealing with the whole sort of PAP connectivity connecting with other big

nodes sharing consensus objects and so on and it also exposes an RPC server and the value that a client or VC is basically the piece of software that interfaces you with your private keys or the validator private keys so it keeps track of when validators need to do stuff so all the validator duties and it will hopefully be shipped with built-in protection against slashing messages so the way it works as I said you've got your big node that's this piece of software that's exposed to the internet connected to a bunch of other beacon nodes and a value that a client that's connected through a REST API and basically the value that a client is

constantly polling the beacon node to get the duties of its validators that's quite important to to understand the the keys aren't actually managed by the beacon node they're managed by a valued at a client so yeah essentially the beacon node will be preparing blocks sending them to the validator client for signing the value that a client will send them back to the beacon mode and the beacon node will then propagate that object to the network and same same goes with with other stations so hopefully that makes sense and yeah so I'll start talking about I guess the most exciting part of the talk which is the the optimizations that's something the team has been really really busy with over

the last few months and I've got some really exciting numbers to share with you guys but first of all why do we optimize like why do we spend so much time optimizing our software there's a few different reasons and perhaps the first one is not something that comes naturally to mind the probably the main reason or the most interesting reason for optimizing blockchain client is security so think about denial of service attack vectors so if you if I know that your blockchain client takes seconds to process a certain message then chances are a malicious actor is just gonna dump a whole heap of those messages on that particular client which will effectively

lead to either a software crash or a panic on that particular client or effectively just slowing down the beacon node to the extent where it's never gonna be able to catch up with the head of the chain so by having a fast client we're basically mitigating some of those denial-of-service attack vectors another reason why we want to optimize the hell out of lighthouse is to ensure a large number of valid errors on the network so these serves the purpose of decentralization and something to keep in mind is the more valid errors we have on the network the more load on the beacon node the more resources a beacon node is gonna be consuming so effectively the resource consumption in

each two scales I guess cause he linearly with the validator count so the faster our client the more performant our client the more value there is we can support and the more participants we can have on the network another reason why we want to have a very fast and efficient client is for staking returns so say you're using your favorite cloud provider to host a beacon node if we manage to have the memory usage and the CPU cycles we're effectively having your AWS or GCP bill and what that means is that we're indirectly improving your stacking returns which you know

obviously makes you you know make more money by staking and attract potentially more validators on the network which I guess feeds back to the second point on the centralization and I guess the last part is probably the most obvious to everyone I think we all hate waiting hours and days sinking our blockchain so by having a fast client we're effectively able to have really fast syncing and a better overall user experience so what do we exactly mean by optimizing so there's two I guess families of optimizations in lighthouse or in each two in general the first one is your standard typical

software optimizations so trying to spot redundant routines and just get rid of them try to minimize your memory allocation of course because we're very very concerned or interested in minimizing our memory footprint and because if 2 has been or is being developed with this iterative sort of pattern we are able to learn from all the testing and were able to go back and update some of those data structures to be handled more efficiently same goes with the actual program structure we can learn a lot by you know running tests nuts and running simulations and then going back and updating how the whole software actually is architected so that's very standard that's I guess

stuff that you would do if you were to optimize any type of software but you've got very specific domain specific optimizations that were particularly interested in so if you were to go through the spec it's optimized for readability right this means that if you were implemented naively you'll barely get a working client especially if you were to try to handle a significant amount of nodes and validators you're probably not gonna you know get anywhere so every single team had to basically digest the spec interpret the spec and come up with an actual zuv real-world scenario of the spec that's that's going

to handle the load of the actual real real world I guess one example of such optimizations the we use we use a validator set right that that's like a do you see it as a list of valid errors and this list gets shuffled right we shuffle this list in a deterministic pseudo random way to basically assign validators to different duties so this shuffled list is consumed by all sorts of different state processing functions so if you were again to implement this naively you would shuffle the list every time you need the shuffled set of valid errors but we know that the shuffling only changes once per

epoch so a quick word optimized this is to shuffle at the start of the epoch to cache the shuffling and therefore just pass that shuffling to each function when they need it instead of obviously processing that and doing that every every time there's all sort of all sorts of other types of implementation optimisation sorry BLS is the cryptographic scheme that we're using extensively in eath to a lot of optimizations that can come out of that and also the way we handle miracle hashing can can definitely be optimized so I guess you'll share hopefully some of my excitement over the last three months we've managed to get our block

processing times to be 70 times faster that's within three months we've also managed to drop our attestation processing times and make it 20 times faster our RAM usage has been reduced by 75% and our I guess disk i/o has also been drastically dropped down to 90% of what it was in February only we managed to reduce block processing by another 45% we've halved our memory usage and Paul came up with this great new way of dealing with fork choice which was based

almost entirely on proto's proto's work and this yielded a 1,000 time improvement so these are amazing numbers which allow us to be probably most likely the fastest client out there it's really hard to tell we don't actually have benchmarks across the entire fleet of clients but I'd be really surprised if we weren't and there's gonna be a whole lot of cool stuff coming as well we'll be swapping to a new BLS library very soon which will also yield another 2x improvement and we're currently trying to think of clever ways of storing the state and dealing with I guess moralization and something Michael

has been busy with so yeah super exciting numbers he was looking great some of our achievements lighthouse or seem a prime was the actually it was actually the first team to produce a main net spec test net so the test Nets before that one were effectively all handing a relatively small number of validators we were the first one to actually prove that we can run with production settings quite comfortably so this was quite an amazing achievement we're quite proud of that we're also the only client that has a working discovery v5 implementation so

discovery 3/5 is the peer discovery protocol that we're using I was designed by Felix from the ETM foundation and it's actually running on our test net today I mentioned the new fork choice that Paul has been working on which was quite quite amazing yielded amazing improvements and obviously we're constantly feeding back to the specification making sure that if we catch any bugs we let the theorem foundation research team you know I know Paul has spent extensive time reviewing the ëthe one linking so linking f1 and f2 and that led to a few substantial changes to the spec we have hire two more full-time rust developers to work on light house

so Deva from Columbia and Adam from Poland if you're listening to this welcome to the crew and we also have some pretty cool advanced syncing strategies that allow us to again sink as fast as we can cool scrub some water alright so I mentioned earlier Sigma prime is a team of security nerds academics developers none of us has any product research design UX experience and throughout our conversations with the community over the last few months

we realized how critical usability is going to be for each issue and we basically decided to submit a request for proposal about a month and a half ago for a house user interface the amount of responses were received was really overwhelming thank you so much I'd like to take a few seconds to thank every single person who went through it and put together a proposal we couldn't obviously take everyone but we we selected Empire which is a product design team out of Toronto Canada and we're teaming them up with flex taps which is a blockchain development studio based out of Melbourne Australia it was

it was obviously sad having to go back to all the people who responded telling them that unfortunately there weren't the winning vendor and a couple of them were like we're gonna build it anyway so we're probably going to end up with a lot more than just one lighthouse GUI which is super exciting yeah so some of the objectives of this GUI is to a lot of technical users to do a lot more with a lot less overhead and also allow the broader community to participate in helping security if to network and in order for us to do that we need your help so I put up a QR code and a bitly link to participate in our issue UX

survey so we really need to understand what our users want to see in that UI and we're obviously you know deep down building this stuff and we may not really understand the usability requirements as well as we think so if you don't mind sparing a couple of minutes going through a couple of questions this would really make this exercise valuable for us this will lead to an open sourced report that the whole community will be able to consume and something I forgot to mention the front end will actually be open source as well so you can just take it change it potentially you know plug it on other clients and it should work quite well I've got five minutes I can talk about one of my favorite topics

fuzzing I think those of you who know me and know how much I like fuzzing my clients software and my team software as well so fuzzing is basically the the art of trying to find bugs by submitting a bunch a whole heap of randomized inputs to basically try to notice a panic or software crash and once that happens bingo you've got you selected an input that's actually triggered a bug differential fuzzing is slightly different you're not effectively looking for panics even though it will detect panics if there are any you're effectively sending this bunch of inputs to a

multitude of different implementations of the same specification so this is quite important context of each to with a whole bunch of different clients out there all implementing the same state transition functions meaning that we should all behave the same way given a certain input so what this does is basically hopefully prevents us from splitting the network if if we had one of these implementations behave differently so that something Sigma prime has been busy with as well we've received the grant from the e theorem Foundation a few months ago to support this effort and yeah so zr NT is the goal and executable spec maintained by proto pi spec is the python executable

specification maintained by the theorem foundation and we currently have on the fuzzer lighthouse Nimbus Trinity and very recently managed to integrate prism as well that's quite cool we're currently working on integrating Tikku or formerly known as autumn as' the consensus java client which should hopefully be integrated within the next few weeks we found some cool bugs already one affecting Trinity and a couple of affecting Nimbus one of them was essentially allowing yeah so the deposit object was not processed properly so it's quite a sneaky bug in the sense that you could effectively onboard users

on the beacon chain without having them deposit ether on the eighth one shine so effectively printing beacon shine ether out of thin air so that's something that was picked up by a by beacon fuzz so that's some of the stuff we'll be working on over the next few days weeks so hit me up after this if you have any questions or if you want help the road ahead everyone keeps asking us you know when main net there's a couple of things that still need to be sorted I've mentioned all these optimizations they live in different sort of branches so it's just a matter of merging everything down to masa which should not take long and be

done hopefully within a few days there's also more optimizations that we're thinking of like pruning the state pruning the database and a couple of more optimization strategies in terms of syncing and database storage we are kicking off an external security review to be performed by our friends at trail of bits so the initial kickoff is scheduled for about two weeks from now but we might push that back a couple of weeks just because we have again other optimizations that we are thinking of implementing before starting that process this probe sorry this probably the third point from the bottom probably

the last one but we're super excited to start working with the quilt team and it wasn't team to start prototyping some of those design and research efforts that have been going on for a few months years now and try to prototype some of them in lighthouse and really start integrating phase one and phase two should be should be super exciting I'm often asked what's production readiness for each - I think it's really we can't really launch without a slashing service right we need to understand that we need a way to polish the network so we need an efficient slashing service I haven't seen any yet I know I think prism has

been working on one but we'll be building our own slashing service very soon that's something Paul is going to tackle in a few in few weeks and obviously we want to value that a client to be protected against slashing so the value that a client is almost blindly trusting the beacon node so it needs a built-in mechanism to realize when it's about to sign a / herbal message and that's it for me hopefully that all makes sense thanks for listening and if you have any questions hopefully we've got some time so thank you we have time for one question if there is any

I original originally I think it was thought that slashing would happen if notes were offline when they were supposed to be validating recently we've been hearing that may be offline notes won't be slashed only malicious notes will be slashed could you could you talk about that yeah sure I think there's a confusion here a slash' Belleza j-- is a message that you can prove a message that hasn't effectively followed we've been following the protocol right so I think what you're referring to is being offline right so if you ought to be offline for a year I think the latest estimations basically gets you to bleed out of ether and you'll be losing ether

very slowly up to 10% of your stake so that's very important to keep in mind let's throughout a whole year a whole 12 months meaning that if you're offline if you're missing an attestation if you're missing a block the actual penalty is the reward that you were gaining you're gonna gain if you had submitted that consensus message if that makes sense you don't lose anything that you've staked you're gonna bleed out on the stake also say that again sorry the bleeding out happens to the stake or only to the potential gains it happens to the stager happens to the validator and the amount that they lose is the exact amount that they would have won if

they were to perform their duty okay I got it Thanks no worries thank you Maddy let's give him a nice clap for the antics [Applause]