All Core Devs - Consensus #6

Transcript

[Music] you [Music] [Music]

[Music] [Music] you

[Music] [Music] [Music]

okay I believe this dream was transferred over YouTube can you hear us sometimes just like on YouTube but we're get started it'll tell us if they get here soon okay thank you everyone it's been okay they can hear us Ohio we had conned we had awesome gone there we have the key to a workshop day

which I think went really well and we can talk about that a little bit gonna get to it first you know clients these days but let's go through quickly what's been going on with the clients that are with ones and we'll go from there who wants to get started why not we can start with lighthouse so we've been regaining momentum at the Devlin we've been focusing on onboarding expanding the team as it was always got Alex Stokes who's joined it was a contributor

to Kasbah really smart guy really excited to have them around we've been working on Rustler p2p we're gonna start focusing on state transition what the biggest change we're gonna be for choice - and one of our security guys is being working to point the AFL fuzzing library at RSS said implementation so we can see if we can break it that's about it from us on our end cool thanks how about Nimbus members

okay anyone from Nimbus we can move on let's go to harmony we've been finished with other stations and now working on Els and finality regarding Bayliss we decided to start from Milagro to evaluate it and decide on whether it's not whether it's sufficient for us at the moment or not and maybe do some experiments and some tries to implement together custom implementation of VLS that's all from us know about Pegasus nothing to report on the client development front we've all

been away for a couple of weeks but I hope to bring more substantial news for the next meeting I don't know you taking research updates now Danny Nick I could probably give a brief update on that front oh yeah I would add some commands actually on github that was created a while ago I feel my town follow up on the conversation and will continue to work on the simulator as well and we plan to present it widely next week or in two weeks and that you

follow with jumps Thanks how about sharing with a question on the emulator yes does it wonder if it's relevant for the needs we have on beef 1.0 definitive so it could be used for 1.0 as well I can send you the link the resource is public so I can send you a link right now and we have no

documentation the next few weeks so awesome thank you very much and it's another I know people were really interested in my white block platform during con for a number of reasons and they did end up openings or there they either open sourced or have decided they're going to open source so that's another good simulation tool I think out there cool thanks Pegasus how about prismatic we're working towards transition between

Christmas Day and if stayed through a single stage so we're working towards that weirdo some work it's by this stupidity employee let's call her goomy and then we're implementing like no wrapper for POS 12 really one so I also want to make people aware of the hub repo he has been really great yes Python but he has Python bindings as well and where we have them somewhat under the p2p and we have implemented a good note so right now we're working on

testing on that and then we plan to work on the relay mode nets and that's pretty much it what was the Bayliss library that you've enjoyed using is it school salumi I'm posting the deed okay cool great thank you shall we do you want to give an update on pi M stuff so in the in coming weeks I will I will isolate the Trinity client

presentation with Danny and I can write a bounty call for the simple serialize in Python implementation which will made it to be a more production version and the third thing is that sorting out the version of the spec for the Trinity test net a for you hope that could be used in the end of this year yeah that's I don't say updates thank you cool yeah Charlie and I are gonna work through all the changes inspect and PI again pretty soon and expects a lot of like minor bug

fixes there on that let's see if you can hear me now yes yeah that's yeah yeah remember sorry about that so I'll just give a brief update there's a couple of things going on from the side one is that we're hoping to contribute a little bit more to the easy-to-understand explaining guides we started a repository where we'll be publishing a few tutorials one is already our on validators there's another one coming up on on the beacon chain and so on and we encourage everybody to join us in in those reviews there's a repo posted I'll put a link in chat later on other than

that we've been focusing on the light client working groups investigating how to approach that clients secure fashion while while gaining some of the while learning some of the lessons from East 1.0 so there's a working group for that going as well as the testing group which is making progress so likewise there's a channel for discussing testing yeah and then looking into small bits and pieces like progress on the little b2b team inside the tree hashing functions thank you how about chain thing so after

that's gone we've been refocusing and regrouping the project um so internally there's been more interest since people have been asking about it so this weekend we're gonna be holding another internal hackathon to have people work on it we're going to focus more on turning our implementation to more of a like client um and have it plug in with the current dev tools for the a theorem JavaScript ecosystem um and that's that's it in terms of an update cool thank you paraty no much of an update Sukhdev we're looking into the details of the

speck trying to dive in a little bit again and looking to add more people on it in the near-term future so we can start building stuff out again oh and y'all are thinking about using it's looking like you might use substrate that's still the plan and we don't see any reasons why you couldn't right now so hopes still hopeful but not going to promise that it will be but I'm hopeful we'll get to a point later that sort of yeah we can it's on the agenda so I'll pick up them as well and we have

some people from get here today anything I want to chime in on we're not we're mostly just listening in on trying to get up to speed no nothing nothing great work great thanks is there any team that I miss speak up so all right cool research updates we got a little bit of research updates from the Pegasus side anything else from anyone at this point cool I guess a lot of stuff has just

been refining the spec if you've been following we recently pulled out phase zero and phase one as two separate documents in which they zero now that we've unified the states and made a few other changes we expect it to be really solidifying in the neck in the coming weeks as we work through just more of the minor details bug fixes etc the phase 1 document is still definitely up for lots of changes we just kind of have the bones in place but if you want to check it out the phase one is the adding of the data change the shark tanks so if you want to check that out

and start internalizing it you're contributing please take a look I know a lot of focus and conversations gonna be on state execution now that the phase zero stuff from the base once I was getting that cool next thing on the agenda so working group follow-up we had a working group like workshop meet up the day before DEFCON it was generally pretty productive and there was decent amount of Education some decent progress and I think Justin Justin might have some updates for us soon but working

groups there is a link I shared in the that has some of the follow up notes I don't want to go through everything today mainly want to say some of these I know has some have I don't know where I got cut out Matt long you're giving us an update about testing yes so I'm adding a pull request on the chat and feel free to comment and collaborate on your next steps great Danny I've got a quick one you

know hey this is Lane so the sort of working session that I participated in at the eath 2.0 meetup was broadly about things like project management and coordination there was a pretty strong consensus in that conversation that's sort of a PMI coordinated using the coordinator type role would be really helpful for and it excuse me managing in particular the eath one X roadmap and so just one of the report as you know Danny there's been some progress there we've got a call lined up next week with Hudson and a number of people on the order of 7 or 8 people who have expressed interest in helping out with that role and we're hopeful that we'll

be able to recruit one or more of them in the near term and Danny I think you said that that's probably not gonna be a thing for the 2.0 roadmap for the time being but we are moving forward with it for the 1x roadmap yeah and to be clear I'm not opposed to it I think it will have a place here I just don't think we're quite there and it can play out just in the 1o before we figure out best practices and import it into this process yeah totally agree and hopefully people who participate in no one over at met will be able to help out with 2.0 stuff in the longer term as well nice quick update from Justin he's been Claudia had on his PDF ran in his research the estimate is down from

20 to 30 million to less than 20 million initial discussions with tezo's on collaborating it just seems like more and more teams are realizing that this is a pretty viable use case and so people are interested in collaborating on the research and the production you could go down that path I'm going to not dig into this security research but he has no taste of security research little and he's gonna do an heath research post on a deep dive on the various cryptographic Hardware assumptions oh he talked to me a little about this there's an effective a max where a max which is the assumed attacker advantage he actually realized as much the effective a max is

much lower and that it takes 1 1 million chance for attacker controlling two-thirds of the slots of bias one day progress and multi-party computation another PDF day in February and all right cool Thank You Justin I'm sorry that you might be some like any other follow-ups from workday when interesting thing like metallic and

Al from web 3 realized that just pork choice rules based upon justification have just kind of this inherent flip-flop issue and there's an e3 search post that I went to that proposes ways to make LMD ghosts in the context of justified epics or justified blocks stickier I'm having like multiple validators on one beacon of needing multiple Shaw's and then that you know OSI constraint so any comments on that I

think and we needed to do a little bit testing on this if the expectation is that multiple shards should be able to run in the context of a standard computer not hundreds but three five eight so presumably one validator client whose software could talk to one node that has a beacon node and multiple shards but because the validator piece of validator client software would be talking via our PC to a node it could talk to multiple nodes and it could one node could sync shards three five and

six another note sink shards ten and eleven depending on the architecture there so yes the shards live and the same client node software as the beacon node because it just inherently makes sense but if we designed the API appropriately the validator software could talk to multiple peek to our notes and tell each one which shards to sync she'll say that made some sense yeah sorry I'm just having some a little bit of mine I might have to come back to

this okay yeah and we can talk about it more that's something I want to focus on a lot is kind of defining that minimal interface between father your software in there node so we can talk about it more oh yeah the solution that kind of came to mind to me was that um the beacon node would be kind of like a manager either would connect to acquire another RPC to multiple shard clients so that like it kind of does the load balancing as opposed to the validated client having to kind of like load balance between these multiple validator between these my multiple beacon nodes where it doesn't really know that capacity and stuff like this yes so maybe we can

follow up one well yeah that might be a reasonable separation of concerns I had a question regarding the the architecture which is basically mm-hmm how much of this do we want to put in the spec or force as a as an implementation requirement I'm thinking here that for some clients in use cases it might be reasonable to use a different constellation of yeah I a hundred percent of granulars I think that one distinct goal that I have is to

define an API and such that a piece of validator software can be cleanly pulled out of a node or at least a cot a node that does implement this interface not that maybe nimbus would need to if Nimbus is targeting like embedded software and maybe isn't even targeting validators but I want imagine a node often my ship with a bout a built-in validation to suffer but the we want to keep the separations clean such that somebody else can write a different validation piece of software so we can have kind of diverse setups

and people can write pooling software easily and everything rather than having them super tightly coupled and this is kind of akin to the the RPC methods in eath eath one oh and not although it would be best if everyone did conform to those standards not everyone performs the exact standards you could definitely have a node that didn't necessarily open up those standards and still follow the protocol 100 but in terms of the more subtle components of like separating if you can node and a the shard nodes internally

that might that's definitely getting into like the client specific territory that doesn't really need to be defined inspect I don't think one of the outcome of the dos attacks group was having things in spec is good and maybe we should have like implementer breast practice so a kind of manual that says that this is implementation detail but this is what we recommend and if you deviate from that have a good reason like a MIDI devices or things like this yeah I think that's reasonable for a lot of it and in terms of internal

architecture I don't think I know so I want to go down the road but I understand in terms of DOS attack analysis like the order in which you process things and strategies you're throwing out blocks I think that those should definitely be in like a common repository and source of knowledge is how sure about later a managed private and public key I Dean it's something that we should probably write down within the working group as well so that people know what's the best security practiced right I kind of follow Piper's philosophy on that and

that I don't think that a node should really be managing keys and that the layer outside should should definitely but in terms of best practices of how the client should it's a great question there's all sorts of interesting things that you can do in terms of you know embedding keys and secure piece of hardware and things like that but I yeah I agree it's reasonable to cite best practices there are at least the the options there beacon roads are for the short data is the expectation in that they're

validated clients also still provided custody first of all so you cut out that validators do it is that the validators for a copy of the date to forge the signing or course they're providing custody but for good question Justin says yes in with yes you definitely need a store a copy of the data with respect to the separation of concerns I believe that the client the validator client rather than the node should be

responsible for this because the idea would be that if you turned your validator client off and swapped out a different node underneath the hood its evaluator client should know everything it needs to run with this new node soft memory so if it needed to know that extra storage then route then it needs to store with the validator rather than in the know Justin says they can rely on external storage but they that would also be taking on counterparty this but in general yachts anything that the validator needs to do to continue to do their job like remember what messages they've signed remember this storage the the storage that they've proven they have custody of

that would all live ideally in a sari now in all live ideally in the validator pieces normal rather than they embedded in the note cool any other follow-ups from that meeting workshop

I think um something that I wanted to mention too that we were in the I think it was a why protocol and no discovery um something that we were talking about a lot was the ability for a client or validated to be able to hide on the network to be able to and obscured it his a validator I know that that was work on the aggregation scheme and I'm not sure how that affects the ability for validated to hide I think would be ideal if it could but we have like there might be some cases where just makes it impossible so that's maybe something for to consider Nicola or Karl the about the the schemes that y'all were discussing

do any of them have issues with anonymity not hugely from what I'm aware of the always up the issue with the first person has to generate the data so you can always wear through the first person is before he's not a creating you have to aggregate but if you at that point right but even when you get even when you receive data you don't necessarily know who you received it from or you know who received it from from Network level but you don't know that they've produced it you just know that

you received it from them first right yes definitely but that's yeah right what about I know the assessment was generally that the naive aggregation strategies given the expected Network load will likely work where any of the more sophisticated aggregation techniques like the tree structures do those require the loss of anonymity Nichola it's at the limit it depends of a number of shards that you add on a

single computer so it's very at the limit in terms of and for if you use the tree based logic then you need to be identified as a participant in the aggregation protocol you can be participating without telling where the aggregation come from and where you can't you don't have to say wherever signature come from but you have to be identified as a participant I see that like you're like paid up here public is

identified as yes but you can have a specific key just for participating it can be different if you like from one that is going to sign okay just the idea is that you have a hole in which you have a place in between and this place depends on something that needs to be known so what's your kind of identity okay but your network level in that type of tree structure your network level I didn't see that can be distinct from your question is what incentive to participate in the protocol so if we say

that we have people who will be participating like this without any strong incentive when we can add any identity format right but it turns out you as a validator like I mean if you do it simply yes you can do things more complicated like people arguing but they would be participating without saying that the actual validator and for example we can aggregate for her shop without being valid at all for yourself right but like we without ring answer

some sort of alternative internalization I don't see just use your notes I think that your validator is likely to be quite difficult whatever protocol because for example if someone listen to the network you will discover who is going to validate on a shop because a note will appear at the certain points from the projection that we did we had to anot class including fraud validators

so it means that under sounds must of a note will be validated anyway so I believe a trying to ID but your validator is uh it's going to be difficult in practice good one thing I think potentially the best we can do is to ensure that your network level identifier z' aren't really who you are in the p2p network is not inherently tied to your validator as such that yes

you could probably be analyzed most but people can have at least a chance for more sophisticated setups running multiple nodes etc depending on their screens yeah at least we can try and make it difficult right like it's probably uh very in the present f you can pretty much pinpoint where transactions came from so we're always gonna be open to that kind of attack but really maybe it would be nice if we didn't have like some easy to find table of who is a validator because they're aggregating so I guess we'll just do the best we can this is also something that's definitely helped but a separation of deacon nodes

and validator clients because you can as metadata client just switch beacon nodes if you discover those to be a problem will have a backup even though yeah or run three at once and do different things for each one of them top your skate yourself okay anything anyway other follow-up from the workshop there great I just shared a link in the chat we don't need to go through this all today but I just wanted to point it out

that's Raul and Kevin defines really the the minimal lip Pizza P implementation as it came to my attention let p2p is a ton of stuff and you get to pick and choose what makes sense for you so this isn't an effort to help people who are beginning to work on the p2p implementations to narrow the scope of what they actually need to work on and I'm sure if you have questions about that Raul or Kat Ralph um protocol abs or Kevin can help you parse it understand it any questions on that before we move on

I just can't say I don't appreciate the effort really very helpful great okay cool so the next thing is a proposal to use simple serialize for consensus only this is something that is brought up by Aleksey in the past for a number of reasons and has recently been brought up again and so it seems like it's not dumb we're not done debating it I think one of the motivations especially was that now that we're going to move towards

this tree hashing algorithm for the state that some of the presumed benefits are no longer there does anybody want to champion this I know you taro wrote the issue so I think he'd probably feel so strongly about this but I know a number of you have taken a look at HUD I commented in the issue I'm not I can see like how it might make catching foster by getting rid of the length bite somebody made a point kale in there about that we don't actually use the length fights in in the

tree hashing I didn't implemented it yet so I think that's something I'm overlooking but I'm not personally I'm not I'm not convinced without seeing like some sort of benchmarks or data that it is actually a problem so I know the spec is we're trying to avoid like this crazy optimization and trying to lean towards it sorry not saying that was crazy but we're trying to avoid heavy optimization and we're trying to lean towards simplicity so I'd like to see like some indication that is this is actually going to give a significant benefits before we bring in the complexity of adding in like bringing in protobuf into the specs because another point I made was that I don't think that by shrinking like by shrinking down SS said and then using protobuf I don't

think we make the spec less complicated I think we make it more complicated because in my opinion we then basically do an import of all of the protobuf into aspect and we end up with what what is it like end up with two two encoders and one decoder so they're they're kind of my thoughts on it at the moment yeah I agree with that I guess I want to attack this question and twofold one does SSC not serve our not serve our purposes for the protocol serialization and should we be looking into an alternative and then to what should that alternative be I don't particularly want to take on the

complexity I know some people really like protobuf I don't particularly want to bring in a third party serialization algorithm my general feeling on that is that we do bring in a large amount of complexity by taking in somebody's seal diction algorithm that's gonna have versioning and updates and libraries and etc but since the first question does it now does sssaid not serve our purposes for the networking serialization I I'm not fully convinced of that does anybody think that it does not is elección

ethical no and I think he has probably the strongest arguments for it I'm I'm cool it seems like we don't have a ton to say but right now I think they would leave the issue we can have the conversation when the issue for the next couple of weeks and if it comes to a head we can talk about a next call yeah I spoke to Alexei like you know personal conversation and and he mentioned that he doesn't feel so strongly about it anymore I'm not sure whether yeah I'm not sure whether I'm misinterpreting or not but that might be just an interesting thing to note yeah I would add I would add that with

reassessing we've already separated out the consensus part which is a good thing I think elected to be developed separately and later on if we find that there's sufficient whatever we come up with is sufficiently similar then and they can be rejoined as well but attacking the problem and making the best like not not doing too many trade-offs to reach like a single pore solution is a better approach in my deal in my world the other thing is that I totally believe that it's appropriate with the customs realization algorithm for aetherium given that we're not prone

to change and software will be running for enough of a time with enough many nodes involved that that it's well worth the implementation efforts do you believe that we're our trajectory such that we have one solution that does not really solve the problems very well or do you just want to keep that in mind as we move forward well we've already taught from that with three SSD right and then there are like minor things to Tracy and minor things to SSD as well that we could debate I think I mentioned in the chat one of the most little-endian versus beginning and there there are these

little details where you could say optimize either of a and Andrea Suzy but yeah I don't want to take up too much time on the call with these things I think it's something like science the part that I think are good something that I wanted to add a note on that I was saying earlier was we had some internal conversation bonus is that and if it switched a little onion it's basically what we already have implemented for coconut and it seems to serve our purposes pretty well just as a point of reference

that's good to know are y'all using a tree version of the simple of that serialization algorithm for your state routes or have you taken over time 10 to really answer or super deep details but I don't think so Purcell rust credits and a repo for it it's called let me lick it Oh fine didn't like it oh I think maybe Alex's main critique was that for the

consensus it would be nice if we could hadn't if we could have an algorithm that can be streamed so that we avoid the need for a huge in memory buffer during this processing step and and that's maybe some I haven't really analyzed tree SSE that if whether whether it fits that criteria but that might be an interesting property to look at okay details it detailed all his argument on the beacon chain repo if I remember correctly so well there are like three issues open on that and as conversation is a bit everywhere but in

his position Israel okay maybe I'll digest what the original position was and put it into the new issue you can discuss there okay well then not ready to make any sweeping changes on that I think SS ed is a good place to continue forward on but open to optimizations or alternatives depending on what we uncover in conversation great uh apologies for my father what Philpott Oh having some internet issues

yeah I'm surprised my calls only dropped once manner that's terrible you okay big verse little Indian we don't need to have this debate but yosik just wanted to point out that there are potentially some benefits to move to a little Indian instead of paying it in and do you want to speak to that just briefly engage the temperature awesome yeah it's mostly it's again it's small ideas not too much time to talk about it but Hardware today basically all the commodity hardware uses little-endian modern serialization formats like Patterson and those tend to

favor little-endian for this reason because then you can you're just better mechanically aligned with the software and the hardware I guess there's a consistency argument for using beginning and on Network protocols for historical reasons and yeah just having to write the Indian this code by using big Indian forces you to write the end in this code on all those platforms that use Little Indians I guess that sound there's a correctness argument to be had there but it's pretty weak so I can write up a proposal if nobody's against it and then

like clearly against it and just by citroen I don't go one way or another but if most of the ash functions are using big endian as well so we have a lot of ash functions in ether to point also that's also one thing I'd play hash functions are really byte arrays and they're not Indian yeah it's true but for integers okay cool um right up to the proposal

and you can discuss it doesn't seem super controversial name all right can do okay general spec discussion like I said the game is call we we've been talking about these phases phase zero phase 1 phase 2 phase whatever and now we're gonna move we're moving towards having a clear delineation in the spec of sluice babies and the idea is that a face and just dependent upon all the phases less than them and that all the phases less than n can be built without having to think

about base right now let's just we can change kind of internally displaced 0 and then these char David chains on the outside of that from the next layer the state execution and other important things we've unified the state into a simple state route because we're using this tree hashing algorithm such that most when per block changes happen most of the state doesn't change and so most of the tree hash will not have to be hashed on an effort for simplicity and also allows us to serve components of the tree easily to the light clients

again I think I've made this pretty clear the phase zero stuff we're trying to get all of the big changes in now which most of them are and now such that people can really dig in and target that big zero well any questions comments thoughts regarding the specification at this plant great

as you see you can take that conversation to the issues on github we're pretty active these days okay any other comments questions concerns can someone just unmute me tell me I'm not speaking to myself just because I got cut out earlier speaking to yourself great thank you cool anything else before we close this meeting anything at all I'm considering organizing another workshop maybe in q1

maybe before after these Denver swag Gabe's the temperature on the value of doing that and if people are gonna be anything but already and that makes any sense I have something Danny yeah in the FIM magician forum so there is a first M which is biggest free and open source software conference it's on February 2 and they submitted a request for a dev room and we will have a blockchain and the sun's realization

they've room I guess so no decentralized internet and privacy so I guess we could have at least people at the Nimbus team will be there initially for name but we can add others from e 2.0 as well and I guess we can get out of the fre on or even blockchain echo chamber but still reach out to people with the same idioms as us perdy will be there as well I think it's generally a good God cool I don't know if anyone from the

research team is planning on going but we can check it out I suppose if enough people are gonna be at that event it might make sense whether I'm there or not to just get together maybe the day before and hang on some of these issues yeah and there is also a call for proposal if we want to do some kind of lightning talk the deadline is on the 1st of November I'll post the link to the frontier magician discution user chat if I can get the zoom to work because it's not alright you said the talks the applications the speaker do

you why so basically every room is organized by someone and you can do lightning talks like talk about something for 10 minutes like this is a big country this is a video for or something and the organizer I don't know who it is for the Broad Channel but but the deadline is in 15 days for various talks proposal right

anything else any comments questions concerns before we end this thing half an hour early today it's awesome No okay great thank you everyone for coming appreciate it being a little shorter I'm in Tokyo and it's good night we will I don't think there's anything conflicting with meeting in two weeks that will be finally after American Thanksgiving great be in the get er on the repos etc and sexual soon they say

why we go to weekend thank you thank you [Music] [Music] [Music]

[Music] [Music]

[Music] [Music]

[Music]