All Core Devs - Consensus #22

Transcript

[Music] [Music] the stream is transitioned people on the Internet please let me know when you can hear us and as always we have an agenda yeah we're live okay agenda is shirt the youtubers and agenda is chaired to s

first thing on the agenda as usual is testing update proto has been cataloging some of the issues that we've had and testing although the system is great and has a lot of stuff where you want to refine it and make everyone's lives this year so proto do you want to give us an update sure so yeah we started looking into the testing problems the other clients now starts to have I already knew present most I think some problems will be resolved eventually there are bigger problems nada or like there's memory

problems for some clients simply can't handle the bigger minute tasks so we're planning to split them up make them organize organize them better and so I posted this issue to describe the issues or like the smaller problems and then get a surfeit to see like hey can we improve this and now I proposed the solution to move forth and maybe you will get to implement it like sometime by the end of this week have to figure is that cool so take a look at that new issue I haven't had a chance

yeah thanks everyone for getting like responses to their surfing questions so quickly it's really useful to have this and thinking of the new solution cool okay any other testing things for in Devon great let's move on to client updates and today we could start with prism yeah hey guys so we're mostly just giving runtime to a lion with this bad

freeze version so we have been able to run single client with single beacon node with CC form elevators pretty smoothly and we're debugging a few sync related issues on top of that we're just optimizing SSE and our sparse McCrory implementation we realized that's quite slow us we started doing syncing with with multiple client and Raul also implemented a nice if one data a motor server to fit variability posting so without changing our prism code we can do the change star pretty easy and yeah that's about it

great thanks about 34 nodes that is able to son attach turns and propose blocks and we're right now we're just doing some other refactorings before designing for the next step so just for us it's hardness so a complete alert 0.8 update we're now patching to 0.8.1 and right

now we're working on some reference tests were worrying worth white block in prism to make sure they're kind of their eighth one mock RPC client is able to generate deposits that can output te amo and then we'll intake the female deposits in so we can generate a Genesis state that's the same as prismatic that's it thanks harmony we have completed one of the major milestones during this week was updating

to zero points back we have completed that the biggest problem bought with SSC especially with the bit list and between vectors and spar snorkel trees so cool request is still on the review but I guess it will be merged likes tomorrow anyway we pass all the tests now so that's it should be treated as completed I think the next we keep making progress on the p2p stuff Anton is about to start

implementing the subside protocol like since next week also we continued our research on attestation aggregation and a bit jumped off from this particular problem to the adjustment topic like attestation delivery and discover it some potentials issues there but want to double check our assumptions that we made given that research so it's still work in progress that the assurance

engineer position have a big dance regarding QA for next few months so that's it from us the this high so we implemented the most of 0.8.1 except SSC but we are just on the verge of updating or SSC with the latest scheme and also coming soon is a unique testing coverage of most of the state transition functions we found some issue with justification and finalization so

that's something that we are investigating we have a PR Oakland and currently being reviewed to be able to bootstrap or Genesis through web 3 and we have a lead p2p test and that has been now running for two weeks and we updated the build so you can just run make and what wasn't automated before was installation of the goalie p2p daemon and that's done now beyond that on the support library that we are developing we are focusing on metrics so that we will be able to have a nice metrics for affray m2 and or a sync

implementation formative reading note 4 features sorry and regarding in freedom 1 now we have promising speed because we can think about 400,000 blocks per hour until block 1.3 million when we have some issues and we also had unreal in to maintain in frame 1 he has been working with us as a very long time bounty contributor for about a year Sanji hi so we have a new Pisces he released for the version 0.8.1 and we're

close to passing the remote inspector of the that is released and thanks to proto for the survey of the head suit and yeah yeah that's very helpful and for passing the magnet test vector i i'm thinking now we need to do sunny sri performers touring for many tests that might be the thing that we need to do in the couple of weeks and our team is also working on pilot p2p implantation side for the functionalities that we need for

the trinity implementation and we also are also making progress of discovery version 5 implantation next week ok thank you [Music] so we're at 0.8 one for everything at this point we had a small bug that we're working through but should be fine in passing and we're in the process of splitting everything out it's kind of prepping for moving forward with more tooling stuff so hopefully i don't know

it's wednesday thursday so like tomorrow or this weekend at some point you'll be able to like npm install like ssz types you know config params everything like that and like individually import all of the early material so i'll be really helpful for dev tooling as we start moving towards that and we're just doing refactor is kind of on our code base to make it a little bit more modular so it's easier to import again because we're moving forward to like light clients endeavour Toulon but aside from that that's kind of where we're at okay thank you lighthouse hey everyone we're pretty much up to date

with the 0.8.1 tests and everything except for one one we'd Genesis that we don't have the runner for yet but what in playing with that and Michael found I think we may even found an issue in the spec today about get start shard so anyway we've just posted an issue one three one five about that today so that might be an interesting one we've introduced chaos monkey and we've seen some good results on the network stability like we've been working a lot on making sure that like a test Nets run for long times and with good block times and on networking our limb p2p RPC implementation is updated to the latest proposal as well and so we're just

working on lots of like sort of implementation details and getting ready for interrupt which is coming up and so we have a we're really excited to have a new resource working full-time tomorrow as well there helps out with lot of this stuff we're making efficiency gains in like the way we store data and reduce our size memory usage I was fleshed out our API spec to be sort of similar with other clients work towards interrupts and we've also been trying to make sure that our rust limpidity implementation works well with the gold implementation which has been not as straightforward and there's been some issues were working through with that and we've also about to review quilt teams as I said PR as well and that's it from us great

that's a esses parcels yeah quilt team submitted a PR to us that we're in the process of reviewing cool yeah I was starting to be in the other day and seems to be plugging away great oh let's do the k-league okay update the Cape you can chain farmville right now what's up I can't hear you if you're talking oh okay yeah so we're we're still working

on on doing this translation of the specification inkay and we yeah as we discussed before we are now actually looking into specifying the metallization operation in K as well because it turns out that we actually needed to do that so which is not going to be a big problem really it's just that will probably require some sample time to do but nothing fundamentally difficult there yeah once once we have

that I think because this is what you have been trying to do at least for the last two weeks which is to have something that we can actually run the tests on run the tests on without having to make any changes to the test so run the tests as they are and we are we were hoping that by doing that we will by doing this process we're going to be able to do that so initially on a subset of this specification and then eventually from its specification okay thanks for sir I believe that was all clients yes to research updates

so we have been working a lot on the phase one specification and there's a ton of little trade-offs we've been exploring metallic I guess you can talk about that and anything else yeah the things that we've been looking at is just trying to figure out some of the details in phase one one of them is nailing down and the persistent committee structure ends the approach that we seem to have settled on this because some this approach where we have a persistent committee the entire persistent committee signs every blog and the persistence committee has a size that's capped at 128 and that reshuffles

in a staggering pattern once every year basically one day so that's kind of both nice and simple and accomplishes a lot a lot of the objectives that at least I had with a persistent Community Design know that seems like it's kind of stable 0.0 stabilized the next thing that we looked at is cross linked data structure yeah the approach that it seems we like the most is one where you have the hell all of the headers first and then the headers are all fixed sized objects and then after that you have block bodies

and block bodies can be variable size object but we can compute the offsets for accessing any block from data in and that's inside of the headers and one of the really nice benefits of this model is that you can still access shard block data from inside of the shard blog header through because the header contains the data route and so we can still view as fixed a generalized indices to access shard but basically anything inside of a shard blog from across linked data route and basically pretend the whole thing is just a nest of that object which is really nice so that and there's a

er that's and doing those things that I worked on you made quite a lot of good comments on yesterday so continuing to iron that out the next thing I'm working on is updating the alight client protocol so the white client protocol was one of the lineup goals what we're making a like line of protocol I knew both efficient and secure and performance was one of the key goals of the persistence committee redesign so basically the approach now becomes much simpler because it basically says you have a blog header from the blog header

you download one one if you're a period committee which was more coal hashed into the header then you use that period committee hands the period committee from the previous period to authenticate blocks of a particular shard during the next period a decided box within that shard and then those blocks contain pointers to beacon blocks and when you points by doing that you basically can figure out what the updated state is of any particular shard or of any particular ORS the recent beacon chain state so that's something that I'm also

in the different in the process of reworking for ssz arc Scholes I made a kind of proposal for a guy canonical ordering for hashes in an SS ed partial object a key a and SSN multi proof and I have some kind of reasoning for why I guess that's a good idea those are probably the main things on the stay is of one shard chain inside I mean the other one is written is I'm also just how rewards and penalties work so that's the last thing that'll probably need to be written after this first BR gets in I mean on

the pass the questions we move on I know some of the shard chain terminology is maybe still a little bit foreign regarding the Marco partials would it be possible to make data structures so that we have the tree height root of the Margaux partial the same as the foods a complete Marko roots mmm good question I mean theoretically you could do that by me just making a kind of custom ssz type

thing so far I haven't really moved in that gone in that direction basically just because I mean first of all it would be more difficult but second it would you know basically a concerns that it actually would have add more couple of more complexity in a budget in a bunch of ways like the big the big one being dead a single object that says add partials would not kind of reduce to being exactly the same as a morkul proof anymore but it is something I haven't thought enough about and I could think think about it more I mean I think I think we can just I mean we don't have

the poverty that the the Marco Marco tree hash exactly deterministic we can have multiple tree result in the same tree hash or vice versa which has needs the ice age wait to be deterministic so I I just think this might be something worth thinking about and I actually have a data structure a Razzie minimization that can potentially do that yeah okay hmm think about it more yeah you want to

share that like - yeah I will I will finish Ireson today and share the name Thanks how are you currently storing general indices like what happens whenever they're larger than you 256 or something so the approach that I'm taking is basically saying that I don't think that we should be storing indices as part of the partial itself so the reason I have for this is that like one that's not what we do former Gophers former groups we passed the index separately - there's a lot of cases where like you should not need to be serializing the set of

indices because the set of indices can just be calculated in real time so the biggest example of this is if you have a partial that's going down to some object but then you wants to capture the entire object so like for example you want to capture an entire validator record at some index then really all you need is the validator index rather than the set of generalized indices so so it's more based on the past than it is the general indices are built in with a real time based on the past you know basically generalize need is a path right like you literally is a bit lists serialization of the path binary encoded

but I just want to mention as well there are variations in how indexes can be constructed based on what pack is being sought so in some cases you can have streaming index construction in some cases you can you need the entire data set that's for another time okay mmm full custody games what's that the next thing you're gonna bring up the talent well was that IPOC I mean I was going to suggested be incredibly inviting might be better for

talking about that great is Duncan here I don't think he is right well dr. nassif er up right now for integrating Legendre symbol into their custody game to make it more multi-party mainly multi-party computation friendly but you can tell us more about that maybe next time cool other recent updates well there's this

little gap in that face zero research a threads filled with surrounds folks I'm matching and so a loss leak I looked into it that's so that we post story with explanation and now we good if you just a few fix need to be polished but there's potentially at 30 times improvement there so we've reduced the size from like safety gigabytes 75 gigabytes to 25 a 2.5 gigabytes for eight months of data yeah I have a question actually

and what about cross links in phrase 0 I just remembered that we have an issue about finality and cross links the old case is the H case so what about this I mean do we have any solutions I think it likely that we simplify the mechanism but we haven't been focusing on there's no verification and potentially this and will probably address it there rather than making a change right now because this data is stubbed in phase zero it's not immediately super pressing but we'll

probably get a simplifying solution a nice governments I've got a couple quick updates from quilt great so John Adler just posted a document III search compiling some of the questions that we still have regarding East 2.0 in areas that we feel need to be more thoroughly researched I think okay yeah John dropped a link for that and then yeah like luke said we just completed initial implementation of SN c-- partials and rust it's a pull request and the white house repo that i linked to earlier if more if some more people could just take a look at that and ping

me with any questions or comments that would be really cool we have come up with a plan for implementing a harness single shard phase one and two engine on top of lighthouse and we'll be implementing that over the next few weeks in parallel with building a simple transfer execution environment similar to what vitalik proposed in the phase two proposal and will has been busy leading our efforts on that test net so we're bringing someone in to continue working on some of the questions that are still lingering about relays and b markets and he'll be collaborating closely with that person on those questions so hopefully we'll have more research around that next few weeks

that's it okay any other recent updates to work into the deposit side of things as to however the public effects gonna work we actual launch all events if people that are posit the proposal here being that if the EF does this in a more centralized manner that there's one set

of tools that everyone can use they make things a bit easier if there's a central resource so everyone to look to for their deposits and it also means that Revelator's aunt then tied to our users on time today should their client software because these clients hawkers will have to be able to ingest the validator these anywhere yes the basic idea here is I will add much to be working on trying to come up with my standards for that looks like for just role rather what the clients will need is to be able to cause the keystore files we may have will use what exists in each one right now that

seems to work fairly well the more complexities arise or how to do the deterministic arts for which which Aquis you get whatever the key tree but the current system is all based on yo curve and searching over to appear less or 31 has a few changes or other I have some new proposals around that that I've been working on yeah an epic to clarify it the intention

around building out some initial deposit tooling is for ease of use across the ecosystem but I think over time the auditor clients and/or implementations will probably take more of that responsibility on internally any other research updates great next thing on the list is networking any updates from various parties great there is that standard Asian PR

open I think the intention is to get probably the next five days a a lot of that lockdown so they can iterate at least have a stable surrogate I know lighthouse is already kind of incorporated the current we're still working on that so I should be updating that P I hopefully yet as you said within five days to say Raul anything of note on Furcal absolutely side nothing major I've been we've been running around helping a team since we've been pulled into conversations and participating as

well for that Jochen Road and responding to that as well taking action on our side to become even more helpful to the community would then we've approved a new ground for negativity and also if it's for changing beyond the peace from just been running around helping helping people as we've been called into action great thank you is there any movement on the team that might work on some conformance tests so not yet I didn't receive a few needs from my co not on Twitter and I need to follow up on them about Anton any anything going on luck

worth reporting I know you're in LA right now doing some inner out testing yeah way today we're artemius here also working closing was prism so threesome and Artemisa working on the cold start with a simple script that you can use to simulate the Genesis event and so we agreed on a kind of a simple set of tests for now and the Artemis team is hard at work on implementing the you know use go yep okay

jonbi one yes aspectus cushon any questions or comments or issues that have arisen I haven't I just saw that issue that was reported by Michael from lighthouse but I haven't actually post it I think that it's if that isn't actually an issue we can really similar release probably like next week that has a couple of minor things and it's bug fix any other respect related things that we want to address right now I have a few so I have a few things in the

Ethiopia Marie repo that people have looked at in there I think we will start just cataloging in like Interop slash standards or something the little things that will make our lives easier and like doing tests and tests so groups and things coordinating on called starts standardizing on key files that kind of stuff so that conversation and those mini specs will live there so keep your eye out for that and I'll make a note when there's stuff up for review people on following that Rico too closely and I asked as individually but as you do work on interrupts tough with say another

single client there are gonna be a handful of like things that pop up where you have to make decisions like oh well how we're going to do this how we do that if you push that those concerns and your potential solutions upstream into the e to OPM repo for conversation that will help a lot because I've tried it how to log the things that have come to mind so far but there is a lot of stuff that I'm not currently thinking abut it's only gonna really fall out when we have a couple people trying to connect okay it's back discussion over open discussion is there anything we'll want

to bring up address yes so we submitted a proposal for its proposal 81 on my walk for lodestar I'll drop a link here in the chat the it's basically going to give lodestar like three months of runway for a development and I think that would be great because it would be a really great if lodestar had the funding to make it to production and so Milwaukee ins who are here would appreciate the support on that well you

know now it is an overwhelmingly YES on proposal 81 you know you just never know what can happen with votes and so kind of like voting YES is going to ensure that lodestar gets there three months of runway but also kind of like creates a mandate because this is kind of a three-part thing they had broken up the grant into three parts rather than a single grant based on some recommendations so please vote YES for that if you have shares in Moloch okay any other things we're very early

today and I pushed a reply to that research questions on the research problem okay well seems like everyone's got a lot of work a lot to work on we I think two weeks from now is likely a good time I haven't taken a look at the calendar yet but well you know if that is not the case I know everyone's pretty heads down working getting ready for this Interop

thing in September super excited on the plane today okay thanks appreciate her we're coming talk to you soon thanks everybody thanks guys thanks bye [Music] [Music]

[Music] you

[Music] [Music]

[Music] you

[Music] [Music] [Music]