All Core Devs - Consensus #57

Transcript

[Music] [Applause] [Music] so

[Music] [Music] [Music] [Music]

[Music] [Music] [Music] okay cool um i rearranged things

a bit this week so that we can um leave some room for follow-up and just kind of continue discussion on some of these uh far future upgrades and also this uh one coming up so we're gonna do um some quick client updates we're talking about this upgrade one which i think and i haven't even opened the issue because i i don't even wanna see the bike shutting dumpster fire but i think there's a debate as to how to name this thing um i'll leave that up to y'all and then we can leave some room to talk about uh merge and sharding i know there's been some after last week's workshop some different conversation especially going around on

the merge um and to that note i believe on the next awkward doubts call which is next friday i believe there's some room there on the schedule to also talk about merge and roadmap i will be there and i obviously i think you all are all invited if you feel so inspired then we'll leave some room for research updates back discussion um and any closing remarks so let us go ahead and get started and we can start with the client updates um and today we'll do teku hey everybody i am up uh so

Ben Edgington: we've implemented upnp for router configuration we are finding that basically our number one issue on the support forums was uh lack of peers due to misconfigured routers so hopefully this will help uh gossip scoring is now in though we haven't enabled it by default yet uh many thanks to lighthouse for all the work on getting that sorted out we've basically followed the same approach um the forthcoming upgrade it involves a lot of refactoring work i'm sure we're all going through the same thing we're mostly prototyping good ways to support uh versioned data structures at the moment

uh speaking of refactoring uh we've done a huge refactor of our ssz library to bring it into the 21st century and making it more maintainable for the future uh anton has done huge amount of heavy lifting on that the week's subjectivity calculations have been updated as per the latest version in the specs repo and we've done a lot of housekeeping work on getting the blast library integrated with our ci process so lots going on under the hood maybe not so much visible on the surface adrian wrote up an analysis of the work that would be involved in implementing the proposed lmd ghost balancing attack mitigation which has been circulated on

a couple of the forums and i've made a start on a kcg commitments library in c that we might use for um prototyping work in teku until something better comes along uh before production that's it from team

tekku

Danny Ryan: that's awesome very busy indeed um were there any learnings or things we're sharing on the the ssz uh fixes or is this mainly some incorporation of things that others are already doing

uh it's i

Ben Edgington: i'm not really familiar with the details but um the way the code was originally written was somewhat opaque so it's mostly um making it

much more transparent and maintainable and playing much more nicely with the backing tree structures and stuff like that cool

yeah might

Ben Edgington: might be worth a conversation with anton

sometime got it thank you let's move on to nimbus um mommy if you're speaking we cannot hear you i think we had this problem last week too okay um we'll let let me fix

the audio issues and move on to load start hey everybody uh so

Cayman Nava: we've been working on some refactoring um refactoring our sync and our peer management and just making that a lot more robust rather than just kind of get you get what you get we've landed so far like half of the sync refactoring just the initial up to finalized uh is landed in our latest release and work in progress is the peer management and some of the unfinalized sync other stuff that's a work in progress where we're also

working on refactoring the code base to handle uh hard fork and for different versions of the data structures um we should be near ability to actually just test out a hard fork i think we've implemented a few of the state transition changes and then the last thing that we're also working on is um the multi-proof creation and consumption and like linking that to our ssc library right so far we've got a like we can do single proofs on ssc objects um and now we're working on expanding that to multi-proofs

nice thank you

um prism hey guys terence here so we released version 1.2.0 on monday it included feature slashing protection that allows important export which satisfy eip 3076 we are continuing working on e2 beacon api we are about i would say 50 down there we fit some minor bug by reduce a lot of time in a testing and we also work on this uh trailing slot catch so we cache the next slot state immediately after process and verify the beacon block

so credit to power and lighthouse for the idea and this is getting behind a future flag and yeah that's it from team prism

Danny Ryan: great thank you lighthouse

Paul Hauner: hello um so we've also been working on optimizations for block propagation like prism mentioned we've also adopted a strategy that techy was using so i'm continuing the clients thanking all the other clients so i can thank thank you for their input there um we found that we can reduce our memory usage by about two-thirds by moderating modifying some gnu allocator params

so we're looking at um what exactly that means and how we can pass those settings onto users by default by researching attacks on the pdp network creating some real world exploits so we can test clients in a safe controlled environment um we're expanding the team by hiring hoping to have some new people starting next month but we'll see we had a pr merged and networking spec it's very minor it's just about shocking checking that the blocks slot is higher than its parent um you can find the pr on the specs repo with the number two one nine six um we've also changed the ips of our boot nodes and we made a pr to f2 clients slash f2 test nets repo that pr is

number 37 it's the only open one uh that's it for me

Danny Ryan: nice thank you and uh nimbus

audio fixed yeah does it work yep okay

Mamy Ratsimbazafy: so uh we released a 1.0.7 this past week it has a significant performance improvement on gossip traffic and scoring block validation and if one deposit issues so things that we talked about two weeks ago it also includes a doppelganger detection

that was coined by a superfice and this should prevent detect and prevent people from having two validators in different places running at the same time one of them will be shut down preventively also we had on nimbus dashboard challenge we will extend the deadline expect a new blog post about more details because even though we had a great interest we didn't have any submission yet so we figure out that uh some people didn't understand that they could use

uh pyrmont uh instead of being a full validator on mainnet uh to participate uh with the challenge um also not directly nimbus related but consensus diligence disclosed a vulnerability in new package manager and http module so we want to say that we don't use the name package manager or http module in nimbus and also that this was fixed yesterday by the name team and we'll be going into the new release which i expect over the weekend or next week we also want to thanks consensus for the interest and helping auditing the name language

and also thanks to a user called gcrtp who improves new json rpc over the weekend and integrated nimbus into rocket pool otherwise we have upcoming changes that are available in the unstable branch and will land for 1.0.8 related to i optimization so uh refactoring of or slashing protection to fit the eip 3076 as well and address ii high i o and improve the doppelganger detection

because both features were developed in parallel so they don't depend on each other are slightly inefficient right now and we are also adding new benchmarking tool to analyze bottleneck in our db storage and improve io favored on the line

Danny Ryan: great and on the doppelganger tackle ganger detection uh does that as it starts immediately scan the chain for recent uh signed messages that are not in the slashing database

uh so

Mamy Ratsimbazafy: right now no because um we didn't want it to depend on the

slashing database right away because it was uh under refactoring so right now what it does is wait for two epochs okay uh just making sure that there are no um attestations uh that are of validator but are not from us and then it starts validating so there is uh on restart and that can be skipped but there is uh always two at the station that will be uh lost and uh in the next release we will improve that so that uh you can look into the past two epochs instead of

wait for three parts yeah cool

thank you

Danny Ryan: um i think that is a very important feature

okay we will move on to uh the state of upgrade one the state of hard fork one the state of the unnamed thing as i mentioned in the e2o pm repo shall i open up an issue to discuss the naming of these things i haven't opened it yet if you dare please contribute

Danny Ryan: so there's still this vitalik stock from a few weeks ago this is um still primarily what's going on the sync committee stuff is merged the accounting stuff is merged

we are working on the adjustment to the penalty constants right now and there is another pr that encompasses a bit of these bonus features which does per validator inactivity leak accounting uh and at the same time uh reduces the overhead of processing empty epochs by uh a factor of 64. yes a factor of 64. um this recently we just finished getting some testing done um and i invite you to take a look at this uh feature to assess the uh both whether it does what it claims to do in reducing

the overhead in your uh empty epoch processing and uh the technical evaluation you know how how deep of a change is this to get into the code base um and if we get some general thumbs ups on that i suspect we merge

monday tuesday xiaowei

Danny Ryan: and i uh as well as aditya are spending a bunch of time trying to clean up the last bits on this um i'd really like to get you a pre-release mid next week that has the initial test vectors cut so that we can get some engineers running through this to feedback inevitably into the spec i'm sure there's something

wrong there so get some more eyes in there so that we can tighten this up for a full release um additionally as i said aditch is doing some work here there is this uh four choice block slot pair pr went up in the past 36 hours or so i'm doing some review and i expect some iteration there but also i'd love for client teams to um come in and chime in and the balancing attack fix i think is in progress um and i need to take a look at i think uh adjust you as well it's going to take a look at this adrian's p document which was just shared perfect so things are generally moving forward the

stuff that is in the uh vitalik's proposal that are under proposed consensus changes are happening um the proposed fork choice changes are also happening uh barring um some discussion i suppose on this balancing attack and the feasibility of implementation and additionally there's this bonus feature bonus that is slated to go in but i'd really really really like some engineering feedback on it merge it early next week i'm still trying to decide i know we talked about a little bit but decide how we want to do pre-releases and test factors or pre-releases but we will figure that out and get them out soon

um so all in all i think we're we're kind of at this point we're going to flip the switch from [Music] the specifications of these things um into kind of an engineering cycle initial engineering cycle to sanity check and get things kind of basically in place i'm really glad that a lot of the teams have been working on figuring out how to handle forking in terms of databases and conditional things and also with these test vectors on our big to-do list is to get these um actual like fork tests that kind of test these these fork boundaries so that we can all make sure that we're doing these fork transitions properly

additionally xiaowei is own is owning the fork choice test vectors and since i'm not owning it it will actually get done that so on our list of hard fork one things is to actually get us some consensus tests on four choice which especially since we're changing things and going in and fixing things that's definitely definitely definitely a good time to get

that done that was a lot um are there any questions thoughts concerns on anything around the current upgrade proposal okay uh

proto is

Danny Ryan: not here but proto has also been working a bunch on eth2 integrations and kind of modulation extension of hive to handle uh getting e2 clients in there which i think would be very good for this type of thing but will also be really useful uh with merge work because ethwon needs two clients to live in the same environment so guillaume can stop manually testing his uh merge code every time which is apparently a painful process but hopefully if proto stuff gets merged relatively soon we can experiment with um getting e2 clients in there and practicing and working on um fork tests around here okay we'll move on um the next up

is merge thank you everyone for coming to the research workshop last week thank you mikhail thank you for presenting on the merge specifically i know there's been a bit of conversation on discord and various channels since then um i wanted to just open it up in case people want to talk about merge theory practice questions concerns i know there's plenty of concerns you know and we can chat here or we can obviously continue to chat asynchronously mikhail um what are the state of

the kind of proto specifications that you have i know that there's a little bit of review going on right now

yeah like

Mikhail Kalinin: there is a draft of specification it needs to be a bit reformatted but actually um the draft of the beacon chain spec and validator and choice change um also yeah if one is two communication protocol needs some reformatting to be published i guess it will happen like in one or in one or two weeks uh so yeah i can just publish the draft if anybody interested in it

just yet yeah i drop it into chat

Danny Ryan: i've done a

little bit of review and i'll um finish that review in the next couple days i think yeah we can get something out the better

definitely yeah sure agree here also um as was suggested we're going to have like like the conversation between the core depths and probably two implementers as well like on more technical aspects and some breaking changes um breaking variance so um then you do have any sense when it can happen when what happened exactly um yeah i mean like

to continue the conversation between right right so um i've talked

Danny Ryan: to tim a bit i believe that on the next um awkward up call that we will at least talk about this more broadly in terms of roadmap and from there uh tim i and and others wanted to spend some time just enumerating based on those conversations in previous conversations all the things that need to get done um and then i think at that point it'd be a good time to maybe do at least like a a monthly call to just kind of think on the various things um some of us that have been working on

this for the past handful of months have been doing a more like private working group call from time to time but i think it's time to kind of open that up so i'd say let's do the awkward ev call in about a week and then from there get something on the schedule where we can regularly um iterate on this um and prior to that always the discord communications as well

sounds great to me cool if anybody has any other concerns thoughts and just

you may reach me out in a channel so i'm wrapping my head around like some feedback from the wall after the workshop and on the workshop um making some plans and yeah get back to work on the spec like since next week sorry okay um then we will move on and again i invite you um or you know at least maybe a member from each team uh people are interested to come to the awkward ev call in about a week time

where we're going to um at least kind of begin to talk about the things that need to get done on on that side of the stack okay next up i wanted to leave open up some time for questions and discussions around um charting r d thank you tonka thank you vitalik for digging deep into some of the components there last week in the in the workshop um that's it open up for questions comments thoughts yeah i had a question regarding a single leader

election a single of and the single secret leader election and uh is that going to be part of the shouting road map or you perceive that's going to be one of the hard work that's coming up um i think it's definitely a kind of part of the road map um so the one thing to point out is that like as justin mentioned in the like a couple of weeks ago just in order to kind of help avh short certain i got intentions for users we wants to focus on scalability or um oriented features earlier and security oriented features later um and so

like we we should still do and continue working on a single secret leader election um as one yeah uh together with you know proof of custody and that data availability sampling um but it's not a hard requirement for uh sharding to be able to be launched just

Danny Ryan: a quick update on on the state of the research i know there's been a couple of fairly viable paths there um justin do you want to give us a quick update on where we're at

yeah so we're

Justin Drake: very advanced um i'd say the research is the theoretical research is basically done

and um you know mary has some formal proofs which is great those are on github and we also have a proof of concept in in rust and and the performance is great and we think we can squeeze out even more performance with some optimizations so i'd say we're in a in a great position

to be in um from an r d standpoint um and are these utilize zero knowledge proofs

Justin Drake: yes so basically it's um a custom zero knowledge proof for the very specific shuffle statement that we have so you know we're basically doing this

permutation and we wanted to prove that this permutation was done correctly so it's not a generic uh proof system um and so one of the things that we gain from specializing is that it's um you know it's it's just it's simpler to uh there's less heavy heavy-duty infrastructure and so we can we can ship it out faster and maybe also fewer assumptions that we have to make on the on the cryptographic side of things um but yeah as vitalik mentioned i i think it makes sense to try and ship what i call kind of mvp shouting which

is basically just the shot data to start with and then all the security features of which therefore i know secret leader election proof of custody vdfs and data development sampling can come potentially after the mvp and the rationality the rationale here being that um we likely have you know plenty of security right now and getting to the point where we have world war three great security um can we can wait maybe maybe a year or two

got it yeah

Danny Ryan: thanks for the update um other items people want to discuss around shopping in general uh um what's the status of the validity proof for data availability sampling is it fully specified or is there r d still taking place in the topic

so

Vitalik Buterin: by validity proof you mean length proof um matt

i'm not exactly sure honestly what what the construction is at this point i was always in the assumption there was kind of like you know some sort of route that we know because it's a validity proof that we can prove that the merkel rule was constructed

correctly

that we're saying right okay so like

Vitalik Buterin: there is no mark over it right there it's just the cate commitment um and yeah i think commitments are kind of self-validating in that sense the only thing we needs to prove is the degree of the polynomial because that represents like how much data the commitment is actually storing um and for that we the protocol is basically finished it's uh extremely simple basically if you want to prove the polynomial is uh at most degree n um and you have uh uh t powers in your address set up then you just uh provide the same polynomial multiplied by x to the power of t minus m um and then you do one pairing check to verify that

okay great

Danny Ryan: thank you any other questions or general discussion points around

charting stuff before i move on okay um and we had a bit of kind of research updates scattered throughout there but are there any other research updates that people like to share today okay great

um i guess just a point of note for this group they're the stateless calls have uh begun again um and it seems like there's kind of a renewed energy in the stateless research um r d uh so you know if it's something you're interested in uh and you weren't aware of those calls and you want to get on that that email list let me know okay

Danny Ryan: uh general spec discussion i think we again have covered a lot of these points uh throughout especially when talking about upgrade one hard fork one the unnamed one but are there any other discussion points around spec

okay

Danny Ryan: open discussion and closing remarks

perfect well thank you everyone i will keep you all updated as we get these um kind of pre-release spec test vectors out um and otherwise continue to collaborate in all the asynchronous channels we all hang out in

thanks everyone thank you danny thanks bye-bye thanks for watching thanks thanks thanks bye thank you bye bye guys [Music] [Music]

[Music] so [Music] [Music] [Applause]

[Music] so

so [Music] [Music] you

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.