Lighthouse: Towards Ethereum 2.0

Transcript

so I would like to welcome actually someone from Sydney his name is Paul Hana and he's co-founder and director of Sigma Prime thank you I'm and then after that shortly we'll have Adrienne Manning who will join him on stage thank you very much all right hello everyone welcome to Sydney as you know a lot of lighthouse development happens in Sydney my I myself have chosen to abide by the laws of mathematics so you don't have to worry when you're dealing with me just don't tell the Prime Minister so today myself and Adrienne going to be speaking about lighthouse I'll speak for a little bit then I'll handle over to Adrienne so if you don't know lighthouse is an implementation of a theorem to

maintained by Sigma prime to Sigma prime is a company that myself Adrian and others founded in 2016 since then we've been doing security work in the etherium space so lighthouse started in july shortly after the first kind of release of the s2 specification and vision it's written in rust which you think is a great language it's open source and it lives on github and of course we welcome contributors so our overarching goal with lighthouse is to produce and maintain a security focused aetherium to implementation so we want to provide a client that performs all of the duties that one might need to do on aetherium 2 we want to provide some some security guarantees along with it and we

get those security guarantees from our own internal security processes we will engage external security reviews and we're really looking forward to getting some hopefully getting some formal verification of some components so our immediate goal is to participate in a long-lived multi client test net so once all the implementations are conforming to the specification and ready we want to connect them all together have them sync together and then stay up for a matter of weeks or months so whilst we're doing this it's very important to us that we continue to provide feedback to the research team this is a you know kind of a first to the client project so there's a lot of untested things so we

really want to really want to try it sorry we really want to try and tackle the difficult problems first fail fast provide feedback and keep iterating to ensure that we have a really great Network so whilst we're doing that we also want to be very careful that we don't try and spend too much time focusing on small components whilst missing the larger picture which is also known as bike shedding so what we have right now which is very exciting is a client that can sync with itself and other nodes of its own type across the internet we manage this a few weeks ago which is very exciting so we've done this using a Russ native Lib PDP networking stack so you might know that Lib PDP is a project started by ipfs the protocol labs team and it's been adapted

by paradis they've written their own russ library so that they can use it in substrate and polka-dot so we've taken it we've worked with parody adapted it extended it so it works for aetherium too so this is a really really cool thing to see this kind of cross blockchain collaboration for a public good for all of us so our client we're a 1:1 patch version below the the the canonical specs the latest version of the spec we've optimized aspect functions I'll talk about this later we're maintaining a BLS cryptography library based on an Apache kind of call that some of the other clients are using - we've also produced a separate validated client binary that's capable of producing blocks and attestations and extending the chain so that's what we have but what we're working towards

obviously is this long-standing network you're syncing clients so there's obviously quite a few things we need to do to get there these are the major ones so we need to make sure that we're passing the test factors that the EF have recently produced we need to do some more large-scale network testing which adrian is working on now we need to do some more speed optimizations just to make sure that the user experience is nice and a bit of database work to minimize the disk usage because I don't think that a compact disk storages is a solved problem in f2 yet it's doable just we haven't done it so what I wanted to share now it's some benchmarking that we've done last month on a slightly older version of the spec so this is kind of we we believe it's the first of the kind benchmarking about through in

this way so what we will try to achieve here was to try and prove the design is sane so just to make sure that what the researchers have put forward will actually run you in in in an on a normal PC in reasonable time and of course while we're doing it we want to learn a little bit more about like the time the nature of our functions run in time optimize some low-hanging fruit and share share our optimizations with the community so we can get a better product for everyone so these are these were done on my laptop which is a like a dual core i5 not nothing fancy but also nothing too shabby so we did three scenarios with different numbers of validators the first one is 16000 validators so this is a fairly conservative number of validators we might see on main neck

it's probably the type of number will start to see on public test nets so in this case so just keep in mind that these are these are first first iteration numbers so so these aren't definitely not optimized to the maximum so we can see what we're trying to do here is produce like produce some process blocks in less than six seconds if we can't do that then the system starts to performances is very much degraded so here we can see we're well under the six seconds we've got this typical case block which is the block that we're expecting to see on the network and we've got this worst case block which is this block that could potentially happen so we should consider it so we can see we're well into six seconds in these scenarios which is

great so my laptop these all run much faster on my desktop the second one we've got is 300,000 validators it was actually 300,000 32 because we it's easier if we do powers of two divided by 32 actually so this one is also one in six seconds which is great we can see here that we've managed to get rid of a lot of the N squared time blow ups that we were finding originally so it was very positive to get to this number and then the final one that we have done is four million validators so this means that all F is staking so this is it's very unlikely that we're going to see this and it would be a pretty shabby system if this were the case so we can see here in the typical case we've got 1.5 seconds which is still reasonable

but then in the worst case we're running 12 seconds on my laptop so this is this is kind of where we found the limits of our implementation so this morning I found about a 20 times improvement in in one aspect of a pls library so I've done that and I was trying to run the benchmarks before I got up here but didn't get enough but I'm seeing about 30% improvement across all these functions and that was just something I woke up this morning and thought of so there's still lots lots to fix here so whilst doing it we did find a lot of a lot of ways to speed up the clients we've been sharing these the ones that we think we can do is we can improve our BLS add operation I think there's a still a bit of fat in there I think I've got about 20 times on

it this morning but I suspect we can go more also Vitalik as he does suggest it a really interesting strategy where we can verify the signatures of multiple attestations whilst only verifying one signature I promise it makes sense and I'm sure there's there's like endless ways we could go to continue to optimize and shave fractions of seconds off of our implementation so that's it for me I'm gonna pass word of Adriane be sure he follows the laws of mathematics - hello yeah hi everyone for the remainder of this talk I just want to give a quick overview of the

client architecture of light house in particular the beacon node and the validator client that we've we've implanted so inside light house there's two core binaries the validator client and the beacon node the beacon node is you can think of it as the f2 analogy to get our parity in f1 in essence that it maintains all the core logics to process beacon blocks it has its own internal chain storage so it's it has a database that maintains the peak and blocks and and all the state it has its own internal networking stack so it's it's the piece of software that connects to all the other nodes all the other clients and maintains the peers and necessary protocols it has its it has an internal RPC server so that third party

applications and external programs can interact with a beacon node and then we have the validator client so the validator client is the piece of software that you would want to use if you want to be a validator and participate inside the beacon block system so to become a validator you deposit your 32f in that process you will generate a key pair or if you have multiples of 32 earth and multiple pays the validator client can you'll give these keeper to the validator client the validator client will then perform all the necessary tasks or duties I think that's pronounced duties for Americans I found out recently that that you need to do in order to participate

in the beacon block and the beacon block system so one of the client one of the architecture architectural differences between earth to clients and f1 clients is that we've separated key management from the beacon node into the validator client this separation of concerns kind of allows us to keep focus more rigidly the security on on the validator client we can get the validated client to run in its own kind of execution system and really focus the security because that's the piece of software that's managing all of your keys the validated client software also for all of your key pairs will keep track of all of the tasks as a validate that you need to perform so it does all the signing and it knows when to do all the signing and all of that

kind of logic and we're in a process of in building an inbuilt safety mechanism so that if the beacon node or the many beacon knows you're connecting to for a redundancy is malfunctioning or malicious then and it tries to get you to do a slash herbal event the the validator client will pick that up prevent it and kind of warn the user so hopefully should we get slashed with a validated client so I just wanted to give a very high-level overview of what kind of goes inside a beacon no death to client specifically for lighthouse this is the rough general architecture of what we have it's mainly consists of three kind of sub core systems or processes the RPC process the networking process the message processing these

will generate a series of statistics and various information which we kind of propagate up to a a notifier thread that's that will then get propagated to the to the user so various statistics about the networking inside our networking stack we're using live p2p I'm sure you guys have heard about this in the last couple of talks so a little bit of PR might as well give the spiel again it's a it's an extensible modular networking stack it's not specific for f2 but out of the box it's supposed to it's designed to give you a number of transport protocols and like routing algorithms so you can use this for any kind of decentralized application we're using it for a higher to client in particular for lighthouse we have four different lib p2p protocols that we're using the first one at the

top is as a pub sub or a publish/subscribe system called gossip sub whether this is used in the maintenance is pending a large-scale network feasibility study to see whether this kind of routing algorithm is efficient and fast enough for us to propagate blocks across the network so we're essentially using this protocol for block propagation and attestation propagation so we have a six second block time if a validator client proposes a block we need to propagate that to all the nodes very quickly and we're hoping that this this Lipa Devere protocol will do that for us the second one is in f2 RPC this is a kind of a custom built protocol RPC protocol built for f2 it allows direct p2p or node to node communication this we use this for

essentially identifying nodes on the network when they appear we find out whether they're on the right chain is asked how many blocks they have and perform various syncing operations such as getting blocks and getting block bodies in lighthouse at the moment we're using CAD Emily o for peer discovery we imagine we're going to swap that out quite soon depending once everyone kind of agrees on what kind of peer discovery we want to use and finally we also have identify which is just a loop p2p protocol that identify other nodes on the network and what protocols they support also very briefly the validator client so again this is the the piece of software you'll be using if you want to be a validator and interact on the on the beacon chain so the validator client you can give it a an arbitrary number of public keys or public private key pairs

it will collect all those key pairs and then periodically poll the beacon node it does this every slot and when it pulls the beacon node it's essentially asking I have this set of validated keys for each of these validated keys when do these guys need to perform some particular task typically a validator should perform will get a set of duties or a set of tasks every once per epoch they may have to propose to a block or propose an attestation so we should know these Pirrip och the that we will the validated client actually polls per lot is because there is a chance that we get some kind of reorg in the chain and when that happens the tasks that a validator needs to perform can change so we check this

every slot so given a set of keys the validator client constantly finds out what tasks needs to be done when a task for a particular validator needs to needs to be processed for example validator five needs to process a block the validator client will spawn a separate thread that thread will ask the bigger node okay what block should I be signing the bigger node will send that block to the validator client divided a client because it manages the keys will sign that block send it back to the bigger node it verifies it and sends it through the network the same thing is done for enter stations so we won't game enough to do a live demo especially given the time frame so we've recorded a simple demonstration of a beacon node

starting up a validated client connecting to it solving solving or proposing blocks signing them setting to the beacon Odent and having them propagate through lib p2p so the top the top window here is the beacon node starting up essentially what happens is it starts up it connects to boot nodes it opens up the RPC which is what it's doing there it realizes that there's other peers on the network that have greater blocks than it does so it before it uses the f2 RPC handshake protocol and collects and receives blocks from the network so essentially syncing I think that's doing it at the moment now it's so it's requesting all the blocks that it knows about it's imported

13 blocks in this particular instance and it's sitting there up-to-date listening on the gossip sub network on the other protocol for new blocks which it's receiving so it's receiving blocks from the other peers on the network or the other validators so the second window which I'm running at the moment just now is the validator client so this is the valid a client so it knows it has four validator keys if you can see there it's called the beacon node it asks for what particular tasks are these four validates have to do it shows you that it has to produce a block on those particular slots there's four of them and propose attestation Xand and which char those agitations are so as time progresses for each slot it Bridget will develop that a client knows

exactly for each validator when to produce a block or protrusion attestation it does that it sends it to the beacon node and you can see that at the top it's it's importing it at the moment because it's not processing a block at the moment so it processes processes a block sensor the beacon oh the beacon node sends it via the pub/sub description in Lib p2p to the other nodes on the network I think you'll also see some Kodomo queries going on where it discovers other nodes yeah so that's the validator client and the beacon node running in a single client shot live test net and that's the end of our talk thank you [Applause] great thank you so I just wanted to

quickly ask what are the immediate next steps for you guys what's a note pipeline yeah okay so at the moment we have a short-lived single client test net we need to do some large-scale network testing for the protocols that we have once we have done that we also need to all get get started talking to with start talking with some of the other clients to agree on there on the network communications and see realizations and then I guess try and get multiple clients to talk to each other on to a test net and hopefully have that long-lived so Paul I want to direct a question to you given you're in Sydney and can you just comment on sort of the etherium space in Sydney and more widely Australia yeah

sure yeah we have Oh in the early days of etherion we had quite a large scene compared to say SF even given the number of tech people around and it's kind of maintained like that I mean after the dip you know people kind of leave but we have some really great core people here we've got like baki pooh-bah here who was part of the dao hack he's a local got other great people like Nick Addison who are doing really cool things here so there's definitely a scene here if if people are into it here there's definitely enough people and companies around for them to thrive great and and for those who don't know Bucky Poobah he is the guy with the puzzles just outside the speaker's room I saw a lot of people being involved so that's that's very cool he'll be back tomorrow again

thank you so much guys thank you [Applause]