All Core Devs - Consensus #18

Transcript

[Music] [Music] [Music] [Music]

[Music] [Music] [Music]

[Music] [Music] [Music]

[Music] okay welcome everyone this dream is switch or switching over and they will let us know in the chat box short of a poll in the chat box up now hey hey Jose Rafael Gomez Gomez the only person in the chat okay cool thank you everyone for coming we skipped a week it's been

three weeks because we a lot of us met Kirsten New York last week discussing some Interop stuff and on the agenda today we will talk a little bit more about some of that and potentially on the yep in September okay so I just shared the agenda with you or even has it we will start today with testing a lot of stuff it's in the works to be released shortly proto do you want to give us an update sure yes great okay so like things slowed down a little bit lived in New

York City week doctormick although we do get like we got together with the implementers starts talking intro all these things and in the meanwhile meantime I got block operations in the dust repository if I booked transitions also in their separate frames and we got the sanity test rightists it's great the song yes we can the there's on mechanical keyboards in the backgrounds thank you so that's really like the status of

testing generally I'd like to work with clients now to create destra nurse or consume the tests and get some feedback on how to improve tests and how to go from there yep so those tests are currently on like an O 6x branch off of Master proto did you actually build the tests and deploy them to the test repo there are on that separate branch like a beta to merge them into the master branch before like they're actually merged into the master branch on the specs repository quotes

indent and reference well so they are there if you want to take a look at them otherwise this is something we plan on releasing favorably with like a version bump on the o6x stuff cool anything else on testing I have a questions Garden Destin yeah there is still Chuck 256 used in a TLS test factors do you need any help maybe here maybe where they can prepare a TR does it make sense or you won't have to release it to those tests

I don't know what is this my my guess is that that was unintentional okay but the art just has been business and maybe some oversight but I think that we should probably inform that in the BLS signature specification in each oh spec does it actually specify which hash we're using its using because I integrated both BLS and shuffling test last week and it's using sha-256 okay McHale which tests are you looking at

specifically okay maybe there is all information need to check it again and also just a wish we created an issue about booting the config to test vector repository is there any problem with that or what is the proper way of fetching a contractor to run tests so this is one of those feedback things I'd like to have from clients how they see their clients integrate relation sauce or noting constants I've seen but I really need like some clients move on to like load these constants into their like code

base somehow and then after available in compile time whereas others want to load configurations dynamically generally I support compile time constants then there's also these few constants that are really just configurable and shouldn't be called a constant at all and so there's just this like configuration could be much clearer in spec on how we do of them right I didn't ask how me like make them available I think we definitely agree they either should be dumped dynamically into the tests dr. levo or live maybe in a separate repo of their own configuration

file so either on either either of those potential paths they'll be available without having to pull them the entire spectacle that'll be the solution for the current situation generally I think you have to make things or improve things first and say we have these constants and we have these configurable and then standardized us then beside those are loaded sniff test yeah specifically like zero zero hash is not a configurable constant it's just a constant and so they're kind of mixed up right now for reference this is a conversation that we had three days ago

within the Nimbus team so I've linked the you in the the chat for now we'll use compiled time constants so we will be having votes directly in no code base and we will not lock them llamo so you'll find that are providing a frame to spec but this is a something that we are thinking of in the future especially once we have actual users because we don't want them to have to recompile nimbus every time like if they want to do a private chain and the main issue is with constants that are used for RS

sizes like sharp count others are not really an issue with there are loaded at compile time or a start time in ordinance we we have a config Tommo file you can swap out the network adapter even the type of timer switch ports you know specify your output parameters and then all of the constants to I just throw a link in there so you can literally just write about a script to transfer from you able to you do it that way just another option so elaborate on the current approach how to support

compile time constants very also like dynamic enough to support both configurations in my gopher chien a both a source file for each of the configuration files and then using a build constraints a compiler which constants to use and then also encode are unified and times thus that you don't see the difference yeah our approach has been to to make our types generic across the different sized array length that means for us that we we have to specify

all these different sets of different layouts array length but they can be chosen from dynamically at runtime so it kind of like a mix of both you know we're also can lower constants right in runtime and just switch from one convict to another so so something that also kind of accompanies this is the configurable constants for whether you can do some switch I have a branch where a new rating goes I'm kind of a file that takes a yellow file and says whether it's correct or not so that will help and some of these configuration things as well I guess regardless these

constants will likely live essentially I'm a test repo or cyber repo so that they can be easily pulled on by others other than that I think you can find move on today any other testing updates Antoine do you have anything about your test render yeah sure yeah I so a transition a Jenkins in states where every night we can run a build that builds docker files for right now lighthouse prison Artemus and then we're able to run this as a set of nodes

yeah is working without discovery for now and we're able to just destined for connection for for at this time to make steps to actually have them make sure to pier together and have them produce some output so that we can check that they are able to sync and then we can we can start having more and more sanity checks as part of that so that's what I'm working on at this time do you think about more complex scenarios like random peer disconnects on some other oh yes I'll bring

so the framework that I use is build by wedlock and it's actually pretty powerful you can do packet loss you can make you know can have sorry Charlie you can have network today's thing like that so it's easy for you to recreate kind of a real mob simulations I think we're a bit early seasons we don't have a everybody doesn't really have Network nailed down to to the point to porch for everyone so but it's good you have that that kind of example so you become easier over time so yeah that's maybe down the road that that's something we definitely want to tackle cool do you use a kind of simulation or is it a real

network connections I mean these connections Carol okay okay got it thank you look from CEO select you know okay I think there's another initial fuzzing effort targeted at the ghost back in PI stack initially is underway led by cuido it's not a lot of fuzzing on each one and that over time we'll be integrating clients and stuff there but without first we're gonna target just these like more minimal spec invitations but we'll update you all that on that as

it's moving on cool moving on let's do client updates we will start with parody so so we finished the block out there in six pack we have a pacing about a better implementation running one potential issue we have is PRS implementation we found that there are some like rare case is where we use the same library if your

son my stage you will be deemed invalid if your valid using the same lab so that's kind of a really bad issue and and we haven't thought of anything but we are working on still trying to figure out how to always a PRS implementation which library and we're using the same library yeah I am cell you should make an issue about and we can look into it

we found a few places few times where if you like get outside of the field or you have some weird behavior so we might have yeah yeah sure yeah I will I'm like so usually is probably like the biggest issue ever I have read no it's probably like it's very hard to reproduce but in do hyper sometimes so yeah we found it when we were using low-order little-endian case like using one in little-endian as a sacred case we would have problems okay yeah cool anything else Perry no this oh great thank you how about harmony we have finished our

update to six point one spec version including deposit contract and SOS styled offsets in SSE with we're passing all current tests need to check out the problem with the pls tests that I have just mentioned so what else we have almost finished our work on the networking stack have implemented wire protocol and regular scene if the sink has also online mode when it

just work with the new blocks and other stations that have been proposed by by the validator so it is disappear on a transport layer for now and we have introduced a couple of messages to notify about new block in the stations and and we have a command-line interface to run a node what is what is what flex here is propagation so there is no propagation of blocks and data stations

and yeah we have no persistence yet and who uses key value storage in memory in memory key value storage so what else here yeah and nodes can may fall apart after some few hours of running but this mostly related to consensus issues and how we process at the station at the stations in the pool so yeah but basically it works would like to connect to some other clients who use TCP stack

whose disappears and transfer protocol so what I'm not sure that right there are somebody who does it that way yeah mmm also I've created a short write-up about benchmarks goes to the link about an hour before the call a couple of takeaways from from the benchmark that there is a interesting thing that get a test in this is for a method from from this back boughs up ever

because it has a lot of little girls to eat you may just check the right app and see why it happens also I remember cool mansion I think that it it's too too hard it's too difficult to store public keys in a compressed format and each time to compress them and this is what every have found in our benchmarks to you as BLS public key aggregation function takes much time than verification that's less weird but this

is the reason is that we store compressed T and each time to compress it with verification so yeah that's it on outside great thanks yeah yes since your last confession you don't need playing the room dinner up New York we had a team-building exercise [Music]

exercises but we did work on an internal implementation of like watching validators register fire of just kick off gene crossing could be we even made it to where you could store that rerun it and start off straight from Genesis we also did some work on making the timer mechanism in Artemis swappable so we can experiment different times in Calgary thumbs and then we worked on some run scripts to some device and testing of different scenarios that's basically it cool thank you but Nimbus hi so it's a bit of slow weeks because

we had several people in holidays we worked on networking so we have implemented forward and backward sync which means basically forward sink give me all blocks that you know since that point and back wall sink give me all blocks in that range we still continue working on li p to p on the crypto side so we had the kind of a mix-up because we had someone implementing a compile-time key Jack 256 but at the same time we switched to sha 256 which is working fine on the test side we have

shuffling main net tests integrated and passing and same thing for the BLS maintenance tests so for reference we are using Milagro so if you have issues regarding Milagro feel free to contact us and we'll give you a link to our implementation and the minimal preset so to be able to switch constants it's in progress and we don't see any issue so it should be done this week now a bit of update on ephraim 1 so since a month ago we have no since two months ago we have a new joiner who's working on the

networking part of ephraim 1 and everything that can be reused for freedom too and he worked on fusing the discovery protocol so hopefully we'll be able to reuse that and freedom too as well and on the documentation side we have a doc generator for repo it should be able to be used for all language is not only name and on or next to the regarding documentation is the internationalization of for documentation so the first language will be Korean because it's we are very excited about a project and another to

do is automatically generating VIP reference and miscellaneous we are working on multi-threading and how to debug some issues in silicon which of reading library of NIM and that's it awesome have a prismatic yeah so we pretty much finished aligning core process into the version 0.61 office back and the next steps were taking is

how to properly use them in or a test at runtime and then figure out how to properly optimize them and in parallel we are also working towards updating SSC and the tree hashing also we're investigating alton BOS alternatives and also investigating how to incorporate test factor data in prison and also worth looking into collaboration with why block is covering five for pvp yeah and that's the update thank you yeah so we were working on getting compliant with

three point six point one and currently we have the SSC test passing and also our BLS tests we switched from Hermes implementation to the Milagro and that's working for us and in the coming weeks were going to be getting they're going to be implementing or bringing in the state transition tests and shuffling we also just landed a first version of gossip sub that was reviewed by the lib p2p team so we haven't integrated it into lodestar yet but it's kind of exciting so we're gonna be working on

kind of simulation testing so working and trying to get our train running into end like working through various stubs that we have as well as the kind of spec testing front and that's it also sorry also just want to make like a quick general announcement after working and chatting a lot with ye theme where you know about working together and kind of and to convert a bunch of lodestar components that are difficult for them to basically convert them into

webassembly so we're gonna be moving over a lot of our core components like SSC and we're gonna try to be us as well and native what webassembly I'm so that the Swift team can eventually go and you're the team could eventually use them and Swift so they're gonna be helping collab on lodestar to get that done so we can expedite some of their client how about my house hey everyone lighthouses been making some good city progress as a last couple of weeks Paul and Michael have been getting us up to date with zero point six point one they got a big PR that it's very nearly done

we're also implemented a pars of the at their own foundation tests and we're trying to make sure that any failure messages we get from them are also very useful and in doing that we've also put his in provisions for fuzzing in the future and it's we're glad to say that Laura is passing the vast majority of these tests as well on the networking side Adrian's were making steady progress with discovery version 5 and he's close to initial initial implementation with rust the p2p and I also published a proposed REST API between the beacon of alligator client which has largely been approved but is ongoing discussion for later in

this call in other news we've refactored our database rapper preparing for some optimizations with state storage in future and next week we're going to be starting to suppose some components of white house and that's us thank you about Trinity so those two weeks we are adding more functionality into our potential validator and adding a normal block syncing process in the dev p2p internally and in parallel we are making progress in discovery v5 implantation as

well Jannik has implemented is an S sorry you know our date high definition and the encryption decryption functions may sounds big plus the deposited countryside we've started to update the deposit contract with the review prepared from around Harvard vacation things so thank you mr. Park that's it thank you Greg I don't think they're on the line now yeah Dean did say he's

waiting for respect reasons and he'll probably benjin thanks weather cam cool and any any update on s side cool thanks everyone how about some research updates so from my side that there is a couple of major things that I've been working on for the last couple

of weeks so one of them is the version two of my ace two proposal which I have [Music] hold on I can I can find the document mo paste and I'll paste it into the telegram I'm one second basically the idea is that it builds on the proposal that I made a couple of weeks ago that I percent the last call except here instead of I unlike the earlier model where the there's the system still maintains this fairly big array of State for every execution

environment for every shard in this proposal the only state that you have per execution environment is a 32 byte hash and so the idea is that in the instead of the kind of full state things like people's account balances and so forth being something that does sort of get stored by this but by this the system in once in a standardized way it becomes something that can be abstracted across different execution environments as well and like even though it's more abstraction I would argue that this

proposal is actually simpler both at the consensus level and at the application level and there's and the reason basically is that it it because of how it lets you abstract some more different different things so one of them for example is because the big state isn't part of consensus anymore the so the big state basically being like people's actuals accounts contracts or what and whatever kind of larger pieces of data people wants to I keep track of we don't need to have one standardized the rent scheme and instead rents just becomes something

that individual execution environments need to figure figure out for themselves and that gives us a lot of freedom to come up with different schemes involving hibernation waking bit fields so like for example lets us make this major simplification that the big deals themselves aren't subjects to rent and we don't need to have like ante replay bid fields if they're if if for the bit fields they get hibernated and and things went like that so the proposal basically is worked says that inside of every shard you have this list of 32 byte hashes where each story to by dashes and it meant to be the state route for the execution environments and

then the execution environment is free to specify a function that takes as input a previous state and the block and it outputs a right now two things one of them is the output state and the second thing that it outputs is a list of withdrawals that it wants to make and that's which are basically receipts that can be used to I take if out of that execution environment and so any state transition logic that you wants to make can just be expressed inside of that function so that inside of that function you would include things like Merkel tree designs like Merkel tree branch verification and it would include

processing transactions and they all of the things that had verified before or that it included before so the advantage of this model of this model like aside from protocol simplicity has said it allows different execution environments to experiment with number one different work different types of Merkel trees so this could be X parsed binary trees all trees whatever but could even experiment with things like snorting and start seeing merkel branches we it could involve accumulators that don't involve more coal trees it could involve like different constructions between Merkle tree state

versus morkul tree receipt based and versus storing bit fields and it just allows us a lot more flexibility in kind of the ability to design things another really nice advantage of this is that the it creates a very natural place for the existing f1 environments to kind of salaat itself in which is basically that if we can write a eath one stateless client in webassembly then like we can just stick that into the execution environments like criminal of the ethan's of that and just what the system will uh the kind of live inside inside of there with all of the same rules that it had before him which is really nice

so i inside of the new proposal document it's definitely simpler but i basically did the same thing that I wrote up before which is just what the actual protocol level spec for it would be and a very simple example of how you would implement in shard eath transfers on top of it and that section turns out to be a you even simpler than it was before so if people are following along the document that's there's a few sections there is introduction out of scope standardized merkel groups we can change changes and then shard processing and then implementing in shard each transfers as a section that and it

describes how you implement just a simple execution environment for people sending each to each other on top so even like it's weird because even though this design is even more abstracted in some sense it's still like at least from my experience writing this it feels like the experience of actually like writing a concrete execution environment that people would use on top of this becomes a simpler experience in a lot of ways and so I guess people feel free to I take a look at that also talked about cross

showing execution the hell generic fee payments and markets are going to work and so forth so I know I already presented about this in the workshop a week ago but if anyone has any more questions I'm happy to answer so about that first our people still here yeah okay I mean if people don't have things I can also move on to yeah so the knowns are the like one is that

it's fundamentally possible to build like what people the but we've been advertising as the face to vision on top of this and my opinion is that the total complexity from having two layers instead of one doesn't increase by the by that much but that's more in opinion than a mathematical facts feel free to verify that for yourself one unknowns would be basically what so one unknown is that last week I was talking about one of the benefits of this being the

ability to go from having two types of committees to having one type of committee and the extent to which the simplicity and efficiency gains of of that direction actually are achievable is something that we still need to talk about more and there's some nuances and comp and complexity is there another unknown is basically that like what the challenge is in fragmentation and what if there's in many different environments and like can we make it does that mean that we have to make it easy to go to jump between different environments does that mean that we had

have to like that it would become harder to upgrade would it become easier to upgrade the main kind of proposal that I've that I've heard for mitigating it is we could do something like for example when we first launched phase 2 we base quake make sure we launch with one execution environment and possibly even only allow one for the short for some time or alternatively if we wants to be able to upgrade it then we could just kind of have a social contract where we agree that doing and mutability Forks on the code of that execution environments from for some period of

time is like you know caping for the go for protocol governance to do a third one yeah would be Sophie markets right so for this scheme to work you basically needs to have this kind of two-level few market fee market structure because the like we can't rely on and we definitely don't want to go down the path of requiring block proposers to actually know about what all of these different environments are because that would be a big centralization risk you know

basically just because pools would handle that responsibility way better than individuals would and so we need this two layer structure where there's one class of node called a real layer the real layer would grab by transactions from users it would it would add Merkel proofs it would package them up and then really or make purple would make proposals to proposers and proposers would choose one of the blocks from a real layer they were keys of the real layer the real work so the real layered piece of them the transaction senders okay it's a to the real layer and the unknown is basically just analyzing the economics of that design more deeply so in terms of risks I would say Trillian

fragmentation and like us not not having a good strategy for ensuring that upgrades can continue to happen in the context of this design is probably one a second new one is like what's would this I kind of America design end up like centralizing you know in unexpected ways another very specific issue is that like 51 percent censorship attacks potentially become easier especially after they can be if the thing they

wants to do is just sensors specific execution environments or within execution environments like sensor merkel branches that touch one one particular object and i mean for the second one i feel like there are ways to get around get a renowned big issue kind of inside of the execution environment so you can do fancy things like creating code that generates merkel branch as they get added to that get added to a blog but so the and kind of the inner so it should be possible to design execution environments that feel sort of all or nothing in terms of what you

accept but there is still this ability to kind like for 51% cartels to just agree to not accept blocks from entire environment environments that they don't like and there are mitigation so that but like the what extent are the mitigations going to be you're going to work something probably need to talk about more and of this topic another thing i've been working on recently is I mean this is totally not a the code is don't cigar so the idea behind an SS ed Merkel proofs for those parts of that is that object

but not the entire object and it comes in this Python wrapper that lets you Nick babies I mean the thing that I haven't written yet is the functionality for updating yet but that probably would would not even be that difficult so being sick we the idea here is that with and as I said with this object you would be able to like just like if you have one of these objects you would know that you would be able to just do whatever work with it that you are able to do on the original object and this me except very easy to write like client

protocols because probably the majority of hawaiians protocols are basically going to be alive coins that would run for if it was a phone note except it would Matt garnets questioner crippled a thing that you can do is one execution environments can generate a receipt and that receipt can be verified by the other execution environment so you could do across some execution environment transactions the same way you do cross card transactions and if those across execution environment transactions are within the same shard you can get them

done and if at base level and we could look into coming up with ways to add that functionality if we want to how would it be done in one blocks all right many two blocks oh sorry what I mean is with a one block expand in between gotcha since we talked last week a little bit about potentially letting execution environments in the same shards like mutate maybe another execution environment state I don't know if think that's still something you think might be possible or if you like thought of some reason why that's not a good idea in the meantime Sola spec doesn't have that yet but it's certainly theoretically doable so the idea would basically be that you

just have like there would be a foreign in the EEI for the just execution of in environments we just have code that says a jump over to this other execution environment and apply a block with this data inside of it that seems totally doable but then then a block would have like to have a post room state route for itself for the execution environment so if we wants to make it simple to run multiple execution environments inside of a block like even one after the other which seemed like a goal and it's not implemented in the proposed in though

proposal it's not written in the proposal yet but it's definitely like our goal that would be implemented if and it's not hard to implement then a block definitely wouldn't be multiple modifying different portions of the of the state but it's not that bad right but basically what that means is that in the blocks partials so first in the block in the yeah boo sorry I mean in the data the cross linker data for a block like you would just have to commit to some multiple more go branches instead of just one and like whatever if you're making a small contract language you'd have to write

the length the compiler in a way that allows you know you'd like smart contract authors to like pretend there is a synchronous interaction or going on but actually what happens is that when with the Vicker because having all that complexity from just you know having their like jump between execution environments and processing multiple execution environments per block that's kind of anything can get messy real fast so might actually raise it like say you know to do this in the compiler uh I have fun I guess fundamentally I definitely agree and I think the kind of the whole thing that's wonderful about this proposal is that it lets you make not have those things in consensus layer and just and

figure them out in compilers and layer two is later the only reason I could see it being a good idea to allow execution environments to dynamically call others is if we really really care about the school of making it very difficult for blog over fifty one percent cartels of what proposers to censor specific execution environments you know the finding like the one true meta execution environment that then basically executes dynamic execution environments like at random or something like you can just move that problem like one layer down

you don't have to have any consensus you can just say like this is the one true environment yeah yeah and I fund but there's actually a lot of things that you can do right like for example you could if we set it up so that there's one environments that has a lot of network effects and that environments could have code that leaves at least one quarter of this of the blog space to the other execution environment okay great so that's the second thing for me so as I said or any comments on sssaid partials anything people don't understand about us has had partials um

yeah I think I am right now or just wants to do some simplification of the code now one thing that and the one name actually issue about partials that that does annoy me a bit is that is that with our current approach for us realizing lists the kind of the path toward a value doesn't depends on the length of the value it isn't just the static thing and I have a proposal for changing how I'm hash tree routing a

dynamic list works that might come that would resolve that issue pretty nicely but I'll probably just write it up soon like it's I mean it has disadvantages to but in otherwise like it's the code is there the code needs to be simplified quite a bit needs to be simplified a bit on the Python side we also I think wants to rewrite it potentially rewrite the MSS oedipal the implementation to use types in a different way because right now it's the way types are written is them kind of ugly but otherwise like it's a thing that it exists and clients probably could start with the potential we could even start building test cases for it fairly soon yeah so the third thing for me is that I brought up the

kind of most basic PR for date availability proofs it's a just links it I mean the basic the basic idea just is data availability proofs as we've done them for a long time or as we've written in the paper with my stuff and the ideas not really changed there it does the link to implementations of binary am FFT is that you can use to compute the racial coding and that you interpolating fairly quickly I wrote up the and if complete design for the includes figuring out what es washing condition

would look like so it's still and I guess new if anyone wants to just take a look in the comments on it and or ask any questions on to get hub later I'd be happy to again for them that's it great thanks any other research updates I think the one for Pegasus R&D and so we've focused our efforts mainly on these last few weeks on finishing the handle paper so handle

if you recall is our scalable Byzantine fault tolerant network aggregation protocol we discovered and corrected for some attacks related to flooding we have a manuscript which we feel comfortable sharing which I'm posting in the chat right now which we submitted to USENIX and we're waiting for well this will probably take one or two months but we're waiting for feedback on that and outside of that this is already somewhat a few weeks out but we published a little blog post series on PCPs the

first one is already available in the second one is written but should be published shortly that's it from us great thanks any other research that they explore move on doing about half an hour left and a handful things cool networking stuff there's a lot going on Zak do you have anything to share with your research into gossip sub yeah so we've finished running our initial test series which was defined

within the PGP test document on our github come on Lobos I can't drop a link but I can add it as a comment to the the github so we're going to release the data tomorrow right now we're just working on presenting that data in a way that's you know human readable it makes sense so we'll share a raw data and we'll also share our charts and graphs that represent the results of the tests that we ran and we've also started working on discovery d5 in a similar way to like two tests and like writing a lightweight client for testing purposes

as well as working with prismatic on implementing it and yeah that's what I refer now because I don't want to go too far in a detail until we have our comprehensive data available which will be released tomorrow so I will I will share that one that time is appropriate so we've been wear it last week we worked with uh we worked credible with Felix we deconstructed discovery b5 identified the differences between discovery be four and five and identified like the minimum viable viable implementation for each did auto purposes and then we'll be writing a library for that implementation and go which is going to be used within prism

so that's no no I know we are using yours don't worry the one that you wrote the one that you're in in death which branch is it I don't remember which Bret is there a specific branch that we should look at in particular okay I thought I was looking at what

wherever you had shared with me before but if you have updated one just share that with me and I'll reference so if you wanna if you wanna help if you want we could help augment that work and

collaborate with you and flushing that out I mean love to do that so please let me know how we can be of assistance to those efforts and I can put some resources on it no no no we're not we're not doing that we need to deconstruct it and identify exactly all we need to write like a comprehensible library that can be implemented I'm not trying to do more work here so thank you mike is here from the community have any updates yeah

hi everyone I'm I'm filling in for Raul so I asked him before this call did he hang up days and he said no he actually he wasn't trying to do any major update this week though so I'll just shoot one thing into the discussion um we've put a lot more effort into specs for lit p2p because it's it's a big barrier for implementers so we've got a guy work on that full time now we've also devised a categorization system for how to differentiate between spec documents that are merely proposals versus things that are already you know ratified and in the in the specification proper so

other than specs I don't really have anything else to add to this group cool any questions for Mike got it are there any other networking Network stuff I know there's kind of an ongoing conversation around this outside of the call but anything that we want to address right now well one thing we'll probably want to do is after we've run all these tests is identify it spec out what the actual implementation for a would be TP is going to be that is

gonna be Jenny general general enough for everybody that hasn't implemented it to the reference because I think it's important that all of the clients implement things in the same manner to prevent any additional or unnecessary complexity and that I think you all seem up to to get some realistic numbers about the use case around networking so what's the block size any bytes right size and what what's Rookwood do you need should be valuable yeah so what are the average message sizes like what is gonna be the Matt the max size like what are we gonna we're testing like what what are those what are those metrics gonna look like

what are those values going to beat so because like our initial tests that we're running right now are good but they're not necessarily going to be indicative of what's actually going to occur within the production network because it's just the bare minimum protocol that we're testing so there's no we're not using any like SEC IO and there's no TLS that's and I've heard the bed adds a significant amount of computational overhead which has some performance effects so we'll probably want to revamp these tests and kind of take a more realistic approach after we get this initial data out the door based on those results and I think we should come to that decision as a group therefore I can put together with blocks

annotations okay great yeah so you can I'll share the link and you guys can check out what we're doing now what the sizes are how many messages per second we're sending and all that other good stuff I do have one thing to say we're kind of talking in a channel about this Interop a telegram channel if you don't have it ping me and I'll add you to the group and kind of that the discussion has been around why not lid p2p first why why go through and work on a minimum viable wire

protocol for inter up and the trade off I can't really speak to that but I just wanted to kind of bring that up that's been a kind of a heavy topic of discussion you had yeah that's that's a good point Joe and the proposal being - yeah so I have this this hack indie document that kind of describes what the you know this Interop lock-in is gonna look like and and I tried to just spell out the different stages you know that we would come and go through in order to

get the clients talking and I did it just in a way like so obviously like you know decision was made to be to use live p2p which is not that's not like what we're just we're not discussing whether or not we're gonna use of DDP just like how the clients are gonna initially talk so basically to simplify most things and establish a baseline you like I had I kind of spelled out this design we're basically you have a sea of eight clients to break them up to a bracket system clients pair up and then just and they come to you know they get to where they can talk to each other using basics you know basic TCP connections just you know just a static no topology no discovery no additional latency is added

and you keep pairing up until everyone all the clients can can actually can actually talk and I mean that shouldn't be that much code to to do that it basically gives us sort of to work from and then from there we can you know either decide to start to add additional nodes until we see it break or try you know different you know topologies or add bad actors or you could just go ahead and switch and say okay now that we've got this working let's let's try it with Libby now the reason the reason why I think it's um it's useful to do it this way it's just that it really just allows you to isolate the unknowns I feel super

confident that we could get to you know we could get phased stage woman done we could say that we're we have the nodes talking it eliminates just a lot of a lot of you know variance and bounding factors doing it that way so um I've heard mixed um you know sort of a mixed response to that some of it might have been just due to the fact that they might have think some people might have thought I was saying not to usually b2b that's not what I was saying so I'm just curious what the general sentiment that strategy is I can put a link to I think you should put a link to the document

and maybe open the RFC on the FIM to spec repos so that we can put everything so that it does not stay on jitter and it's Harper line later you'd go okay yeah so I think some of the pushback was with respect to if you're already have looking at the integrated deeply into your processes you know the extra overhead to add the simpler interface I can't speak directly to that because I'm not one of those teams age yeah yeah so

um from a perspective yeah there's we were trying to figure out that if we have lipid P as a as a base framework and that's something we're going to use in production lib p2p at if we strip out all the higher-level protocols we simply just have a transport which is TCP and we have a multi stream select or the or or the multi stream kind of functionality for the first level testing instead of in setting up raw TCP connections can't we just use like the basically p2p basic you did a simple TCP transport and the protocol on top could be some arbitrary just simple connection

does that does that still work I think I think that's why I think though it's most important is that everybody agrees on the message types and each client is able to interpret that message in the same way so that way they can execute the appropriate application logic yeah absolutely so - I'm sorry - I don't what Adrian said there's a document from several months ago that specifies what a like minimal Viable if the implementation would entail so if we showed that perhaps some implementation teams could start working on on that yes my understanding is that lipid apibe at

the moment is well for the ones that for the the parts that are already implemented interoperable so Russ they can go and JavaScript without the high level protocols are already interrupt so it's not something we need to test if we just have royal ability with multi stream select you should you should have multiple different clients being out or at least talk to each other and then a very simple wire protocol on a flat so the message formats already set up for us yeah yeah that's already set up it's very similar to the wiser a protocol already yes smell like a bare-bones live p2p is just a transport

and probably multi-stream select I think built on top of that and then other than that where we just have streams that we could just communicate across clients I don't imagine that to be difficult to to like that's not something we need to test I mean it's more this is more for people who are who actually have to bring up that part from scratch you know if you don't have at hand so you know they would have to you know then just

bind into the daemon and then and then they've also done the work of getting themselves a little bit closer to a prod implementation yeah that's what I was gonna suggest we built the the daemon version of Lib p2p specifically so that languages that didn't yet have a lipid B implementation which would be be able to interoperate I think that's the best that kind of work so there's the demon also wouldn't the demon result in some performance degradation zazz well I mean

depending on how you implements it I mean there should be a very specific way to implement it that isn't going to result in additional overhead and performance degradation effect that testing I wouldn't think so good I'm not too sure about this cuz I haven't got there but you know I'm not too sure how well the daemon will interact with stuff for in browser I'm related stuff that we're working on like our like client trying to get like the actual beacon chain itself in the browser so I don't know how that's gonna play out so that's why we've been just like heads down on the p2p yeah please have sorry good effort so for a Nimbus we do we do have an

innovation that box over the demon but it's a mess like from a practical Kundu it's just a mess so that's not somewhere we want to stay in no I'm not in the browser no no we have made a native binding to the demon but what's wrong well just just a necessity of bringing in you know if you'll go environment into your build process and then a lot of the stuff that you need to talk to the demon is lip p2p stuff so I don't know maybe you need some protobuf there you need to you know

serialize data into the demon there's like the advantages are in the meat and I'd say for for the complexity that that it brings in yeah something like that so we have to talk to the demon right and and and feed beta into it and control it from from from our rapper basically right you know launch it find the binary you know all these little annoying things Oh to actually like implement the p2p then you know trying to implement a wraparound indeed yeah yeah yeah that's

the point that's not a brown-bag you did what you would another thing that somebody brought up was benchmarking lid PGP and gainst of PGP so it may be controversial but a couple people brought that up sorry you say against deaf p2p yeah comparing the performance of each comparing the performance of that PGP and dental of PGP the performance of which components say uh yeah well it wouldn't just be it would be like consistency and

availability and the process of relaying messages throughout the network also implementation I got the gossip sub right yeah so right now we're testing gossip sub flood sub and then Plumtree in the looking to the implementation of gossip sub and flood sub and then we're working on the the plum tree implementation as well but that's what that's the one that Antoine implemented does anybody consider direct lately to be binding instead of using a demon that's only available in languages that

are actually supported that have library so it's it's just gonna be limited to go and rust I think he's talking about exposing the symbols you know from the ghost the go or rust library you know and then marshalling it through C and then using J and I or whatever let's say you're in Java to do that so sounds like a pain in the ass I started doing that I there's a branch of the daemon where you can you can literally I made a shared object and you can control it all um the only thing it's lacking is that it was um it still uses it still uses protobufs so I didn't rip that part out

yet and but that was back in January so yeah some regions want to bring out that um I think the most complicated part is not the findings itself because it just used the probe up to serializing and send a message to the demon through the TCP socket or UNIX socket but I think the the Troublesome part is when you need some feature like you want to you wanted this on you want to disconnect

appear who is trying to connect with you and currently the demon that's not support something like this so um you know we need extra support or modification to make these things to work so so I think the most troublesome party like a callback functionality so if we ripped out the protobuf later and bound it directly to the ghost symbols and didn't have that protobuf layer then we'd be able to expose events you know essentially function pointers and then you can do that right yeah from my point

of view I think the demon the functionalities in daemon is sufficient to do the internal testing for from for the beginning so we we can connect and disconnect and sin and broke-ass things from our understanding it is enough I mean I mean if we want to write the the naked the epitome or the bindings to to the libraries at they did it is a lot more work in the bindings so another thing some people

have been having issues with is like peers that are added to the routing table without any routable IP address and what PTP and that's their Kadeem lea implementation so I think that Felix is discovery v5 is going to be able to account for that because you can add arbitrary node metadata and then that will allow you to provide more going to yeah it's alright sorry if you have like

the the y0 particle which has essentially the multi stream kind of select in there and it or not because you and then you know even if it just means you know like right now it's I mean I mean it I mean right so can we not just

specify the components that we intend to use rather than right totally responsive but that's I mean that's also what we're intending to do with these tests like we start with the minimum protocol and then we can't begin to add other components and we test each one individually and we compare them to the baseline so that way we understand which ones are exactly going to result in some sort of performance degradation and then we understand what needs to be optimized well what Felix is talking about is a very immature as well because evolving and we might not evolve at the same pace so during work Felix says basically yeah

that's perfect hey Mike don't you all have a spec writer now you'll help us with that yeah no I mean we could actually I think we're trying to define what the what is a 1.0 implementation of lead PDP what does that actually mean in a separate thread and so this might be a good place for us to start yeah let's do it yeah I just called the document the McAra was referring to earlier there's an open issue in in one of the games I don't have it on me right now but I can post it I think she posted it yeah in the chat yeah yeah that one number four there's value and us working together

beside the networking second suggesting that we collaborate so that we clearly by now the share and we have a share channel quite block in protocol labs as we're proceeding with these tests so it's you know good to have feedback from them directly and also their input to just make sure we're not doing anything incorrectly so if like you know if we get like weird results or data it's not like it's you know it was a collaborative effort so it's not like any one particular person's you know mistake yeah I just want to clarify that we're not actually using that much lip here to pay things I think pretty much only two things we're using we have a

transport which is just a CP we have optional encryption so we can just take that out we have the multi string select and a multiplexing so we essentially just when we connect to to a single node we set up multiple streams based on the protocols they're the only things of Lippi to be essentially we're using we we have our own RPC we have our own discovery and we probably with depending on gossip subs whether we use that on that but the only live p2p fundamentals are using are just setting up the streams so there's not that much of it I also have a nun merged PR which which tries to explain clearly which parts of the beta P we're using which which isn't that much how big is the dependency to get some said well how big of a

component is that what was that we need we need a pub sub Manor gossip so via our initial testing many months ago was the best thing that we found if it needs to be modified or if we need to take a different approach we still need the pub sub capability so break like if that is broken in some way we don't necessarily need to break out of the p2p just solve it but we should we'll get the test results and things on

that work before we make this yeah any any sort of implementation should be modular so that way you can just be plugged in or removed without you know breaking the entire application okay there's a lot here and we will continue a conversation offline there is this Interop lock-in I think a lot of what we want to discuss related to it was this my opinion is to find that very minimal as Adrienne was suggesting components just talk to each other in p2p rather than something outside of it

I know that somewhere on the order of half the teams don't have the PDP right now if that is likely to change at some amount over the next few months and we can't forget that half of the teams do have lucky to be and already using it so that would be my preferred approach but let's merchants in these PRS and make it more clear what honestly p2p that we need in the stretchy boat as for this you can no valid API does anybody have any further comments for emergent think

we really need to talk about that stay on the call cool we're at an hour and a half let's probably close here in general on the spec side we're actively working a lot of hours right now we're echoing in these merged the intention is to release something at the end of May I guess in approximately weeks time and then release the frozen spec at the end of June and that is what we will do the tests that are being the roughly set

number of additional tests on oh six one and then probably stop releasing stuff 106 branch and the subsequently be honest I guess though so let's continue the conversation and all the channels any last little comments before we close what's up with Toronto is people are people going without stealing them okay so something cool thank you everyone appreciate it I'll talk to you soon bye

[Music] [Music]

[Music] [Music]

[Music] [Music]

[Music] [Music]