All Core Devs - Consensus #4

Transcript

[Music] [Music]

where is my chat I'm gonna rake on the hey David Alec in the YouTube chat box hear me very cool yes they can hear me great let us get

started I believe you all have the agenda that issue all right first thing on the agenda is a client update who want to get started rotala can you mute while your oh sorry no we'll do that right now all right cool client someone Prismatic go hey guys I can get started jirobo from prismatic a lot of updates we basically

are on our way towards creating some meaningful demo the contain workflow so having you know an initial Genesis chain starting and advancing through attestations of proposals we have merged a lot of stuff recently we expect our public we contain a PR a so we're able to stream to value to clients assignments chart IDs and and basically their validator index at every single cycle transistor and you're able to request a subset of public keys so say like you're some third-party application you want to like see the validator assignments for like n number of public keys you can you can touch those as well we stream those to like validator clients that are connected via our PC all of the rewards and penalties have been updated in our implementation with

the latest changes in the spec and I think Karen's here and I shan't can give more updates on our progress yeah sure we also kept up with the 2.1 spec we updated the FFG rewards and crusting rewards we also implemented the proposed such attestation check during during blood verification we also implemented this attestation service for the beacon note it's it's job is just to aggregate attestation and then save the aggregated at the attestation to the local DB and yeah that's pretty much it okay okay how about Pegasus team is now

set up to start working on implementation of the Iran because Iran Marilyn we learn insinuation of aspyn this web assists under who view next week and as when we started to work on the key and the business implementation like you will sink on Peter next week on first John Connell religion observe and

connotation some illegal and as one of the pianist as well competition and what you need to do first is to see what kind of performances you can get and is it something having a magician okay we need to spend more time on them Thank You Jaime from Nimbus so we focused on simple serialized and implemented it fully and also proposed EML test format that will be discussed I

think later and also started to focus on the processing timing that was proposed to two weeks ago and furthermore this is a plan and now we I think we can have common tests for simple serialized but also Blake to be and BLS signature and that would be helpful so that everyone is on the same page and I could start test feel like what is done with Alice for each one point O so that's something

I could do in the next two weeks I get for free topics great we can talk about a little bit later I have a new I made a new repo where we can put these common tests we can figure out a good structure for that cool McAra sure unloads arcane tape is not here but aiden is Aiden you can update yeah so we're also working on implementing simple serialize and PJs but we're expected to finish it within two weeks so by the next call and we'll have it available as an NPM module and we're still kind of working on R&D for gossip

sub pairings BLS and the vdf libraries that we're working on we've done a bit of we've created a bit of issues to get people more involved I suggested um and yeah that's about it thank you great and white help yeah it's using Milagro I made it so we took it and made it a bit more of a standard crate and read a bunch of tests for it it's passing them but we could use some professional cryptographers to have a look at it and

make sure it works on top of that we've been working on SS said we got our sterilized we got implementation of that working implemented some database fundamentals and just kind of building out the core of the program and then we did some benchmarking on block validation using the BLS and as I said we put it in that on issue that Danny's been running what's bad it from us great on the Python side we spent some time working through the rewards and found a few different bug that had been sick a few different bugs of the spec that have

been fixed via the PRS that I think you've probably seen at this point we also found a win benchmarking which we have some benchmarking results we found a bug and the shuffling algorithm that made the number of committees per per cycle unbounded whereas this should be the number of committees per cycle should be bounded by the char count so minor fix to that I know Xiao Wei is kind of getting ready to begin porting into PI VM to move towards a more production Python implementation so a lot of little things around that all from did I miss any team

harmony yeah no problem we have finished our work and bought proposers and work processing part in some places the our implementation is a bit not aligned with the spec especially in its database schema part but in general yeah it's it's like we have a high level design and some things that are and some details that are implemented from this back we were working on at the stations now and we have updated our roadmap the next things will be a cusper or with

finality and the POS signature aggregation so yeah that's it it's owned by mass great and next up is research update anybody want to go on third it sure under inion like i we move fixed another couple of bugs and this in this and then aside from that like i'm also i think in

the in one of the three search threads raised the suggestion of changing the fortress role from being immediate message driven to being like latest message driven and they think i gave some of the arguments in there actually let's see if I can just find it and paste it [Music] do you think that a parameterization between the two might be the like the way to go or it would be and I'm still thinking about this like one reason why is that like yes it makes that makes sense but another reason is I know it would be that we care about minimizing

complexities yeah so also another thing that we talked about yesterday was like if we're going to do kind of two layer beacon chain message a groggy at the station aggregation but basically because with the current spec I think in the average case we figured out that the

minimum peer-to-peer network load was something like 50 kilobytes a second and like in reality that's that's multiplied by also all of the various I peer-to-peer an efficiency isn't in the worst case there would be 500 kilobytes a second so if we wants to reduce that somewhat then we can basically have this structure where the we use the shard networks to kind of aggregate the other stations for a shard and then broadcast iographer gets into the main network right hmm another kind of idea I had is and this is not related to the other

things in terms of like an actual launch roadmap one possibility it would be it's that like we can create a version of the spec that basically says that we add an additional validity condition that says if the main chain actually kind of accepts some particular at the stations you were shard then that main change should only be valid if the if that I the station actually is valid and you could consist conceivably come up with a think of a version where like every client actually does download all of all of the full data and that might even be interesting as a kind of way to want sharding on me and not on training

wheels if we wants to and then the idea would be that we would of course like have they have all the shard yes limits be very low all right I missed something house how are we doing how is the how we're doing sharding on main net with training wheels so basically the idea is that you add a requirement that says every no it has to validate every piece of data and the main chain is not valid unless that happens or a rather a node would not consider a main chain valid if it links to an attestation that's invalid gotcha and bye-bye maintain we're talking about

the B contain the B key chain right okay sorry and when you say the gas limit start with a very low gas limit you mean the estimates or and what if there were given that there's not gonna be computation that would basically me in the byte limit okay so Dina says yeah yeah so like imagine you know like 4000 byte blocks in every shard and then multiply by 1024 that's like four maybes every 15 seconds or something like that possibly even less maybe now is a good time if I can ask

about phase zero which meaning added to the the wiki and I hadn't heard the term before but um can you all link the wiki page you're talking about yeah really say there was the beacon chain with validators hesitation but the cross links for the actual like char blockhouses is dubbed mm-hmm in KC I were discussing and realized that KC was

not aware of the potential term phase zero and that likely may be more of the community was also not aware of that so I added that to the wiki okay if you get a question regarding phase zero oh it's hold on let me go grab the succubus beacon chain without shards oh I see yeah um so I think the idea like what I just suggested basically I mean maybe we just came up with the idea independently but it basically is a way of implementing phase zero hmm yeah I didn't have a specific

question but oh good yeah liking a zero maintain that sounds like good yeah yeah like to me it intuitively makes sense that we should kind of that it would be a kind of safe launch strategy it's a start with launching the char didn't mean that with everyone downloading everything and then we can kind of sliding scale make more and more notes kind of quotes emulate over time hmm and like realistically right if we if we end up having large staging pools then those large staking pools are probably going to end up like running again if they have and I like 1%

of all of the ether then they're going to be getting called into ever into every shard anyway so they're going to have to have the data from all of the shards so they might as well just run a super full mode any other research updates or discussion closer to the mic there is the better yep thank you okay so yesterday I posted each research post

with a minimal vdf randomness beacon it's is minimal in the sense that it it goes right at the core the construction and it doesn't have complexities like difficulty adjustment and incentivization directing sensitization for the evaluators and then in the post I kind of argue that we don't need these complexities at least for you know the future I'm fairly happy with with this with this construction properties as well in terms of being decoupled the roadmap Rick wine they

told to know in protocol and sensitization over has all sorts of nice things I encourage you to kind of answer sorry to to ask questions and raise issues as you know is quite if we do go forward to admit it's quite an expensive project and so we wouldn't want to [Music]

[Music] having a lot of trouble here yeah so we've also confirmed the power usage ballpark around seven watts and hopefully we'll set up a grant with them shortly so that they can they can do some more work we've also got a quote from obelisk so the one of the companies

that is you know potentially going to help us design and manufacture the Dedes vdf Asics and it's a quote for initial viability study we also have a team in the UK working on on the proven aspect so the vdf has two parts it has the evaluator and the prover and the prover is not as latency sensitive actually it's more limited by throughput and so we're looking at the kind of hardware that would be most appropriate to implement approver there I've also had various calls with with Intel so three three guys from Intel are

very much interested in the PDF ASIC two of them are very experienced engineers you know they've been so close to 20 years each and you know they've been able to provide a lot of perspective and very interesting remarks so that that's been good in terms of where I'd like to be in the coming months I'd like to try and wrap up the viability study my my inviability estimate right now standards around 75% so it's been gradually

increasing over the weeks from one point and hopefully maybe only 19 we could have some sort of initial test net or CPU Justin when you say 75% feasibility what do you mean by feasibility here so what would the 25% to look like and the hardware relationships the finances

can we get something with very strong security at a reasonable budget so like what would a failure on the hardware side look like something like it's possible to get like 2 x SP speed gains by paying like 8 x more with pretty pretty much all the way up to infinity I mean I mean another thing is where I have some uncertainty is the design side of things so it's possible that you know there could be some new breakthroughs as

to how people do multiplications and things like that so I want to try and gauge you know what the researchers think there is in terms of fats formulation there I mean another thing is going to be the cost fabrication vdf rig in the latest proposal you know i'm suggesting that basically different foundation fully subsidizes all the all the hot collaboration with Falcone numbers and so if we're gonna you know ship 10,000 rigs we can't go more than a few thousand dollars per acre but wise

maybe just blow through the budgets right so to be clear you're interested in providing these commodity VDS basics at no cost - right exactly okay so none of having direct in protocol rewards and you know buying an ASIC rig would be investment because you'd get these rewards we scrap the rewards that are given internally to the protocol and we give these this Hardware for free and we need only one single person to run this hardware and to be online

I feel like adding a tiny reward would still being nice has just like a small incentive but running but yeah it doesn't need to be large yes the very nice thing is that we can have that actually indirectly incentivize finally overclocked you did a quick a few thousand dollars per day which equates to you know on the order

of one minute yeah yeah that's like one percent of the transaction fees or something like that yeah cool so the trick is have slightly slightly incentivize the inclusion but I mean the the winner of the race but not so much that it's worth spending infinite money to beaten to make create crazy hardware exactly okay cool let's move on I believe Casey has

something to share that he's been working on yeah actually before we get there on topic of media I did want to bring up this thread from the past week about where as kind of mentioned the vdf could come later and in later phases and the you know network could launch with just Randhawa initially I hear some more thoughts on that and I personally think launching was just ran to initially is totally fine I think it's it's gonna take quite some time

I'd say at least 18 months so the nice thing is that quite likely the protocol layer can survive on Randall it just means that the security analysis of the protocol will be will be harder or kind of more hand wavy and we might have to have these security margins and the various parentage that we choose so when we have this kind of this upgrade with the vdf we'd be able to even make the the whole protocol more performance by removing the margins or just making it more robust by by keeping the margins and having this margin I guess a lot of the value is in my opinion at the

application area where we exposed a code for strong randomness and you know if that's delayed if that you know that's only meaningful really for phase 2 plus so in terms of timing this there's no rush in my opinion great thanks do we have this back somewhere for the render not yet but it's something that's a very simple to include well there has been a spec for R and L in other contexts but like a spec for how R and L would be integrated specifically into the Casper 2.1 spec not yet but it is no it is very

easy to include right like the basic idea is just that you start off by do you require every validator at the positive time to provide a random seed and then every time they make a block they just kind of unwrap one layer of the hash onion but the mechanism by itself has already implemented years ago and block validity would be based upon the fact that so the block would I do basically I mean it would yeah it would check that the unraveling of the hash onion is correct so basically if you're if the previous saved round out seed for that validator is X then if you provide your end out preimage why it would check

the hash of y equals x and then it would change you're like pre-commitment from X to Y so to to areas where respect final speck from what was just described and I think it is worth thinking about is number one the staking pools we wanted to make it more friendly to staking pool onion the other thing which kind of I would I would like to see address somehow but it's unclear to me what the best solution is is what happens if as a

block proposer you propose a block and you reveal some piece of local entropy but your block gets doesn't get into the canonical chain for some reason is there a wave you know - I've multiple reveals next time round or to cancel this reveal and so an open question to me right quote any other bf4 and in this discussion before we move on okay well Casey has an update about some

visualizations he's been working on and maybe an update about them the simulation work you've been doing let me see if I cannot share my screen and the tower can you please yes yes okay can you see my screen yeah great so I'll try to keep this quick yeah the goal here was to visualize originally individualize phase two but you can't really do that without visualizing phase one but there's no details here of validators well you're shuffling or

random stuff none of that is in here it's just a beacon block appears then that points to a cross link and then after the beacon block then all the shard bucks appear at the same slot do you have simulations of like network latency impossible for king games here nope the only timings is in here there's no fork there's no Forks up in here let me research one more time it's like the simulation that I did that I did in Python that had that created the diagram that I think is part as like links to in

the spec has like the fortress rule and the possibility of forking built-in right and this is a visualization rather than a simulation of what I'd be like an optimal running of these make sense yes yeah yeah I just want to give credit to Danny for the idea of arranging it in 3d yep yep yeah no this is like adjustment of it and it gave me that tip on visualizing it a while ago so I'll give him credit you ya know this is a great way to visualize show it show it show the cross Center so you can see like smoke forming like yeah usually a good spiraling up so down here the blocks are to finalize and

I guess they're finalizing like one one cycle length back which is yeah speaking blocks because there's paint shards and then so then once this crossing is finalized then this this length of tear chain is fine lines not this one than this one and there's a bug where you'll see once it gets up to the shard length is pink is a eight shard blocks long finalized then then it sort of stops working there one two three four that's

it and I think yeah so the timing right now is exactly uh it's five seconds per for a beacon block and yeah I know that so there's the bug with the the finalization is not proceeding up correctly I'll fix that this is another twenty ornament that's about it yeah and so we call it the beacon chain Justin was proposing potentially calling it the spine chain as well and this kind of shows that as that as kind of the center of the entire sharding structure so anyway cool any comments questions

it's pretty cool if you want to add a feature you can just go to this observable notebook and then and then like pork it and then we're kind of itself you know live edit so cool I just show them blink great I guess hello yeah cool things up great cool any other research comment question discussion another longer-term question I had as it would be

interesting if we had some way of testing or doing an economic test of like how willing are people to actually lock up capital and though I mean I suppose we'll get there is not that much that we can do given those now given those numbers but it would study my dear still useful to have that information [Music] yeah exactly like you can ask people but then that's like not this like people don't really know like the other problem

yes people don't really know yet what the sneaking experience is like like they don't really have experience of like here it is what it what it actually feels like to run a node means in a node having eat up a bit of your bandwidth go offline some of the time and so forth so rocket pool recently published a pretty comprehensive blog post culture people saw that with a lot of numbers from there like test net kind of alpha or beta you know how many people stake how much if there was earned etcetera just interesting on if we might want to do something like that on investment I'll find the link and share it here you know in incentivizing a test that might be a totally legit thing to do so like providing even though it's test net

either providing them with extra rewards for participating yeah lets you 200 although the problem is like to having economic test you don't really it doesn't work unless you unless you are using like actually risking real money right okay I want to get I want to move on because Raoul from protocol lab is here today and I want to get him some time before we move on to that I know after

you just joined us do you have anything to update us from Paraty hey sorry for talking late we made some progress we have one developer researching options for 30 no and we don't have any details to share so I'm trying to make it pretty interesting but we oh thank you okay

we're gonna get the timing analysis we'll go to that graph to this but but um I like tissues Raoul from protocol labs the next so the point of the agenda is to discuss the lid p2p daemon which is begun to be work done but it's still kind of in the proto phases and lrl will take it from here yeah thanks everybody thanks for hosting me I'm happy to see this cross-pollination between communities this is Bravo I'm a core engineer the PT team here at protocol - and first week I also have a background and that beats a piano theory I know so I'm hoping that be useful to this Danny do we have like

five to ten minutes for me to do a quick introduction with a demon yeah awesome perfect okay so without further ado I wanted to talk a little bit about the demon in what we were working on so as you probably already know that b2b is a modular networking stack for the for building for you to be assistance and enhancing features like discovery pub/sub DHT is protocol in Street muxing transports and so on so far we have invitations and goal in JavaScript and rust and of course we're encouraging and supporting implementations in other languages but in parallel were also developing and apt and essentially this is conceived as a standalone process

that encapsulates essentially the universe of the b2b features in a single binary and allows local applications running on the same machine to interact with the peer-to-peer network no matter the language they're written so essentially with this we have a very quick solution for - applications for Java applications name applications and so on to be able to use that Oh essentially the array of limited features so what the daemon does with you is it takes care of connection management stream management multiplexing security negotiation and so on and essentially you get streams role streams back where each stream maps to a

baton stream and the key with a specific gear over specific protocol and also you're able to send control messages back and forth from from the team the newbie to be demon is written and go I think you all have a link to the code base in notes in the agenda for this call and it being developed and go it means that it has access to over 50 TVs originality so how do we actually maps and how does not just the mechanics work essentially here well we use inter process communication facilities like right now we are working over immediate priority and it's already implemented is yannick socket domains so essentially

each stream will appear over specific protocol maps over to a nice to a UNIX socket the main domain sockets sorry but in a roadmap we also have general transport so this will be developed a bit later on the demon itself exposes a control endpoints to quit through which you can essentially ask it to open connections and streams with tears and for each kept for each stream the demon gives you back a dedicated UNIX socket for that particular stream so it's actually all reads and writes from into that socket through as reads and writes on these treatments so I essentially your

application in this in this case each student annotations would attach protocol handlers on top of those streams so you'll be able to interact with peers essentially by doing Stata died right so we are aside from all this which is the core and the basics of the of the dynamics years from section you think neck chest appears opening streams negotiation negotiating protocol system and stashing protocol members we're also working on exposing different subsystems of the p2p like the DHD relays absorb and so on so essentially applications will be able to to get the values of the DHT to find those to subscribe to topics on pops up

to gossip and so on so this is just the demon itself but in essentially your application so particularly if Eve two implementations I'm particularly The Shining pork that Kevin has been working on and collaborating with em as well Janek simulations and so on could essentially use all these features from the programming language itself by using it by using the standard SDKs and most programming languages already includes facilities to attract but the elec sockets or you know the programming languages like like Java or include have very well-known libraries like letting

that have the support but what we are encouraging as people is for people to develop my twin bindings to the to the teeth in different languages like Python Java so on I know that kevin has already expressed interest and developing - binding Pegasus as well as express entrance for developing a Java a java binding so essentially what a binding does this is a very lightweight library that basically encapsulates all the all the exposes.the control api in an idiomatic and clean manner for that particular language and allows applications and particularly

implementations of youth to to attach rows of all hangers in a pneumatic matter for whatever whatever that making its mechanism is for for a particular program callbacks it could be called code routines as well we've already developed er a binding in gerbil which is a student direct and we are seen starting a bull binding which will likely become a word reference implementation of course we're also writing a spec so that people who want to build human bindings can start working as well as possible so just make sure that you watch our record everything so even in the ideation and the conceptualization and basically the

roadmap for the demon is being worked in the open I've just posted in the chat a pull request for the roadmap which has already been approved by several team members so I encourage you to take a look they're taking it through the road map it's divided into short term medium term and I think all features that each do needs and Charlie needs from from the demon are included I've made an effort to include that in the short term roadmap so we're actively developing and of course we are happy to have you go through the road map to you know point

out different features that you like in there - we accept all kind of contributions of course and of course firstly I'll be happy to support you in your development and tests of Libby to be for us a protocol labs we have made supporting the indian community a priority so I'll be acting as the point person for everything that you need from the the BJP team so if you've got any issues questions suggestions you can open them and github in any of our oppose the ones that is concerned for that particular issue and just mention me so that it comes to my attention and I can track it and of course I'm

collaborating with Kevin and Janet regarding the the sharding Paul we're trying to sort out as well what the network flows would look like together and they are also engaging with the ones will be appearing one point no teen it's also developing a lippy to be base proof of concept of whispered version six and yeah so just wanted to say there's a lot of things happening here I'm hanging out in your in your guitar you can find me there as well we'll take and just thing me whenever you have a question so I'm happy to take questions now as well I hope that was useful thanks that was very useful

um what uh are you looking at platform support is it Linux and Mac or something else as well oh yeah so yeah so when we when we talk about UNIX domain socket so that already has the operating system and inside in it but it has come to my to my attention that windows as well is including support for your next event sockets as in there I don't know if it's have a kernel level or some subsystem level but in order to make sure that we support all platforms were also looking at a memory transport so this means that that words basically be I think we are we are

targeting supported we basically target supporting a little bit system so it's gonna is gonna definitely be involved as we so that that answer your question yeah great awesome thanks thanks Kevin I don't know Kevin organic if you want to add something to to that I know you've been you've been participating in some of our discussions you've had the opportunity to review the roadmaps for the demon I don't have anything that right now just everything that we get that kind of collaboration started it's very cool

yeah thanks yeah we've made it a priority on our end as well so actually I do have one question it'sit's all it's regarding the the end points that you open up to the world later on um one issue we're having overhead statuses that a lot of people run like a tyranids only on weird ports and there's no masquerading support so you can't really if you have a hostile network it's kind of difficult yeah so all those things will be open and available yeah so we let me to be has support for different mechanisms for essentially hole punching through nuts is this this is what you're referring to

yeah it might be hole punching it might be you know that you're on a public Wi-Fi that has blockers that only has four four three and eight yeah yeah yeah I see I see what you mean yeah yeah of course so there's there's a variety of ways that we can make sure connectivity one of them is hole punching if from NASA another one in the circuit relays so let me just it has the concept of nodes being able to hand over to pipe through connections to other nodes so this is another possibility if you know Ferb's free you know in a particular context that certain ports are allowed by whoever whatever firewall you're behind then you can use those for it's to get you to some other peer somewhere

else that can then draw a connection to the final destination or it can go through a number of forms all right cool thanks question for Danny does that mean that lippy to pee is now is a blessed networking solution so if we want to implement it on neem now we're now sure that it would be useful everything so far is pointing to yes we will be using Lib PHP I don't we have not made a final decision as in we haven't and last time I think we were on the the research is

pointing us in a direction more and more and we were say 90 95 percent sure I am fairly confident that this is the solution for us but again we haven't I don't know taken a vote or whatever Yannick Kevin are you getting closer and closer to sit to giving your your blessing on this or what so I'm not quite sure how to answer such a question I think I didn't really make any progress since two weeks on that question I would say that even if the

higher-level protocols don't work out then we can still use the lower levels of the p2p stack so I think getting started on implementing upto some kind of vegetal ap to be should be yeah so in that that is saying say gossiped sub ends up not being the precise higher level protocol that you think that the bones of the p2p or such that may be a modification or a slightly different higher level protocol would work and so we're just exactly missing on scene okay yeah and I do I do want to stress transform a side that we are completely open to a gossip service is an evolution we have other algorithms

that we're exploring as well for gossip dissemination and so on or membership a few other things same thing is happening on the DHD level we've got a research group that's dedicated to different challenges on the DHD in terms of security foster lookups scoring and so on so we do have a lot of stuff going on and in our end so if you feel that at one point you know such a thing that you seeing at the present time is not maybe mature enough then feel free to to hit me up and we can see how we can support it and how we can make sure that is right for you right I'm sorry for me I am I hold

this in conclusion as yummy at least the lower the level of the database it's pretty it's pretty good to use and given and for the cossatot itself I think that we're still doing the testing for this part and sorry for late and progress but for a bronco yourself I'm pretty competent with it and but and maybe we if if we want specific features we can add something or do some modification to

it yep that's my ID at this point is that aligning and working with with p2p is probably a net gain for the EPOC versus women through the p2p protocols as well and so I think that's likely the right direction moving forward especially you know they've been very open and excited to collaborate with us so you know I got a boost for us yeah oh yeah big things to Rose

Brow's introduction to the gold a plea to the demon now I think we can give it a try I mean I mean for every ending for every language we can yeah I mean at least we can try and and for the language wish they'd be too late p2p the didn't support previously we can start from the goal epitome demon and boy yeah yeah so I'll be I'll be hanging out in

getter if you're intending so as I said we are going to be working on a spec for the demon itself so that it would be super easy for binding implementers for people who want to come in bindings to pick up the concepts and exactly how these to happen how the sake dynamics needs to play and so on so and of course I mean I'm gonna be hanging out in guitar and you can open issue so network we are we love get help so even for just open an issue coin I also wanted to mention that theorem and Falcone are looking to

collaborate potentially on the vdf ASIC you know it would be I think a massive win-win if we do collaborate just because of the the large amounts of expenditures which would be required you know even in terms of the early research at this stage I'm all for collaboration with with the foul coin cool great okay any more questions or comments Raul before we move on great Thank You Raoul and I believe Raul

and a couple of members from his team will be in Prague at the end of October so we can have some in-person collaboration then oh yeah forgot to mention that we're gonna be add we've been invited to the to the theorem 2.0 meter so would be there as well console you can have as many chats as you want right thank you so much okay the next thing is we have some preliminary results on block processing just the lighthouse and lighthouse and the Python and be contained implementation did some quick analysis to check sanity check our estimates on being able to process

signatures in that scale I just shared the link it looks the V in the results were very much within the bounds of reason and are solidified or lint credit to our initial estimates and it looks like as long as we can figure out the aggregation on network layer that these aggregate signatures are going to serve our purposes even in the extreme case where Olly is validating lighthouse contributed Paul you have any comments about the results really the only thing that I would like to know to make sure

that fully accurate is that the the Milagro library were using is is fit for purpose we don't have any test vectors for Baylor so it's just you know an early trust in so I know it but that I would be called and I made maybe I'm Justin Drake those you're looking for testing the question was if there's any test vectors for these libraries I don't think there are a unified test vectors for these bailouts implementations at this point but it's something that we need to standardize on yeah okay cool yeah but apart from that as we're pretty good

we're using our concurrency to - attestation validation I thought it was interesting that if we had like 10 million s miss point zero six seconds to evaluate a block then if we get to 100 million and - it's not it's not a ten-time to increase so I'm not exactly sure what that is but I think it might be overheads do two threads I think the way I'm doing it is using a rayon library in rust I'm pretty sure all my threads were getting spun up as I did the test so I think there might be some overhead there that's why the numbers don't match what I expected yeah I would be interested to see with no concurrency if you're getting the approximate 10x yeah that would be interesting we'll

check it out cool great yeah and I'll put a with like a standardized format kind of in this in the vein of what Paul and I have posted so if anybody any other teams as they get to that point want to just post some sanity check block crossing numbers that'll be useful but good results over there it's a hope hole cool so the next thing on the agenda is testing we have a llamo chain test format mami proposed a simple serialized test format that also had a

little bit more metadata that would allow us to use the same general test structure for our various tests if anybody has reviewed those and has some comments please speak up if you haven't please have someone from your team look these over because this coming week or two we have a new test we have a new test repo for the unified test we're going to start putting some tests in there under this format unless people have comments and questions or comments and changes they want to make to this format because we need to start targeting some unified

testing especially on things like our extra libraries like as I said cetera any comments on the testing formats at this point everybody cool homework take a look at it next thing up is Alex II wanted to talk about having two different serialization formats one for the wire I'm on / hashing he's gonna give us the intro on that and we can talk about it Alex II I'm high yes so basically I was

thinking about the I looked at the symptoms realization unfortunately a bit later just after the last meeting means that a lot of people started working honest but the thing I've noticed straight away is that it's essentially impossible to derive a sufficient sufficient structure from these serialize from serial extreme when you when you're just looking at whether you don't have a schema information essentially ready that might be okay but it's if you think about sort of the consequences that that precludes a lot of like a generic tooling and the things that you couldn't otherwise the traffic and and visualizers and stuff like this

and I also thought that you know one of the reasons why people kind of didn't like our P Oracle ROP is actually because there is a confusion between like the use cases of the wire format and the the format used for for hashing and I agree that ROP is not good to to produce the hashing inputs because basically you have to you have to pre allocate lots of big buffers before you can actually start hashing and then I had this problem with this when I was trying to optimize to a guest but actually is there is a wire format is pretty good because you get first of all you have this length prefixes which allow you to pre-allocate the buffers

and also it allows you to derive sufficient structure before you you without even looking at the schema so you know how many items there are you know like where they begin and end and so for the hashing itself I the basically having the length is actually a bad idea because it requires you to have that buffer before you start hashing so for that I would suggest just to have some format which doesn't have a prefixes so you can actually stream into the into the hash function like if it's a ket check then you can essentially use that property of the of the what's called the sponge yeah so imagine that if you need to hash a

huge huge hash tree then you can actually start streaming from the leaves and then as you go up you kind of have like one stream per level and then you can you can actually hash the whole tree very efficiently because at the moment you all need the buffers at each level and it's Revilla it pretty it's pretty memory intensive so my suggestion is to basically split up the the the serialization format and make them optimized for their respective uses and I would say that it's an unfortunately simple and serialize doesn't fit any of those requirements basically it it's it's not the optimal for any of the two categories so that's what I was gonna say

so what would a format that's optimal for hashing look like essentially it's like if you would try to serialize without any length prefixes just basically dump the the data is in this is essentially simple she realized but without link critics is that's that sounds risky because like how do you even guarantee you that the same thing couldn't like oh yeah okay so you can actually yeah you can essentially if you really want to do that you can actually add them as a suffix is not as a prefixes so that you can have a same same kind of known non-duplication thing but because you added as a suffix you can compute it without create pre of

occasional buffer yeah so you can have like a perfect yeah right right is here that would meet something like the zero byte that ends see strings sorry I don't get it what would mean all I get because you don't have the length basically when you read into the data you have to pre allocate a variable length buffer well another thing is that the thing is that when you when you use this as an input for hashing you never have to deserialize you only it's only one way so that's why you don't care you like the only requirement is that it's very easy to produce and it's also it's

also unique of italic tension so you don't have anybody to actually dis realizing that data so one other reason why I like it would be a nice to have the same format for both as based is that for like say the Python client and this would probably apply to clients and like other soul languages as well like converting from one serialization format to another is like a source of a huge amount of overhead and so it would be nice if you can like have something that you actually just treat us under I just treat us a blob of data well first of all the conversion will only happen in

one way as I said because nobody nobody nobody passes a hash format around exactly so the conversion will only be implemented from a wire format into the whatever it's called the zero the one but it could also be done very efficiently because the you essentially you take the length three sixes and push them into the end or something like this i we can actually we can look at this it's interesting now I'm also just looking at the crystallized state right now and trying to figure out like exactly what there even is which has variable lengths and as far as I can tell it's just the variable the

validator records and the Chardon Committee for slots right I really like there is also the cross link record which is like currently stored us about as a fixed size thing as a variable size thing but it's actually a fixed size thing because there's a fixed number of shards right hmm so they trying to say parade oh no and I guess I'm moving in actually is there any reason why moving the length prefixes to the end is not the right approach like bad even for a idea

for a wire format yes because the one of the things you really want from the wire format is that you want to peek at like a first couple of bites and you need to allocate the buffer to read to kind of safely read the data into and that's why the the prefix is in the beginning is quite because in ROP for example you first read the you treat the first byte and you decide like whether you need another four bytes and you reason another four bytes and then you can know how long the whole big data is going to be right you have the answer for the hashing format actually so one thing that confuses me about your claim is that like what are the cases where like that we have where you don't know what

the length of the thing is like until well you see so when I was implementing like I was reimplemented like the Patricia tree hashing in turbo gas one of the things was that when you have a string wouldn't have a string of bytes depending on whether they are like less than 56 bytes long or more than 66 bytes you've got a different size of actual prefix so I had to shift my buffer a little bit before I actually start hashing so I have to allocate the buffer do the computation and I have to shift the buffer by one byte to move the the memory around or to start from a different location so it sort of becomes a more tricky but with simple serialize the prefixes are all the length prefixes

are always three bytes long yeah that's correct however I would not have that prefix at all because in order to complete the prefix you have to look ahead into your like you have to actually before you complete the prefix oh I see because the previous is based on yeah I can hear you yes can you hear me can other people hear me hmm but regarding so look ahead I don't know

besides if you store in the linked list let's say most language the length is already saved so you don't need to look ahead so it's not really a cost I did sorry I just I because I did so okay what is using SSE look like in the

context of like shark tame transactions in the context of what change transactions charge chain transaction actually you know so SS Z it does not even like it's like it exists at this low level it does not even need to exist at the transaction level right because like with the abstraction but basically the way that blocks will be divided into transactions is something completely different which is the format where you basically have a bunch of shares and each share is like 256 bytes or whatever then the first byte a teacher tells you where the separators are and then the format of a transaction it really could

be anything right because like ultimately like all of this will be you know different transactions could even have different formats because of abstraction and like we don't even need to use our LP or SSE we could use in like a bi style format it doesn't well matter as much one quick note on performance of copying before hashing is that for aligned data I would say that it's very very fast for online data with some platforms it's a little bit problematic because you need to do it bite by bite instead of you know chunk by chunk mm-hmm so length prefixes are three bytes it's gonna cause some online

accesses and there are 4 bytes there 4 bytes now yeah all right so we're good what other by the way have we put any thought into like other actually I'm not even sure if it's the right idea like basically it's rehashing and instead of doing a tree hashing along the data structure lines instead of just hashing the whole thing is a blob the idea would be that this would just like make light

client access of the Crystal Eye state or whatever easier so issue if ashing as a blob is for example for c or c++ if language add some padding or something it's just this has to be detected right no no what I'm saying is that like basically instead the other alternative to all of this is that instead of having a hashing format we basically like hash the data structure as a Merkel tree so we would have a standard that says you go in like if you see an array then you first make a Merkel tree hash or if you

see a variable sized array you first make a myrtle tree hash from the variable sized array and then you just add higher levels you pretend that some bytes 32 so the goal of this is basically to make well first of all in some cases it'll make hash updating cheaper and in other cases it would just make like client access of any state variable simpler it sounds like an interesting idea because you can partition to that as well yeah and it's like you'll be able to paralyze hashing I'm not sure the unit the size of the data speak enough

to paralyze it but might be well like you imagine if you have like the validator record right and the validator record contains a million validators then if you tree hash it you'll start off by hashing a million things so you can basically split up the subtree or richer mm-hm I'm not sure about the cost I mean it needs some kind of knife analysis because if you are sure as a slot with which is just int16 it's very simple but transforming that into a metal washer will be quite costly and then you have to process a michael wash yeah so what I'm thinking of is like so first of all only applying it to

like specifically to our variable size lists so in this case it would just be the validator record and the like shortened Committee for slots and so forth I guess and the oh you also have the recent yeah and then the other thing that you that we can do as we can say something like oh if there's like we can have a rule that says oh if there's less than a thousand bytes or whatever we just hash it has one thing I can do a quick test right now what I would say that but like whenever you have these

little rules about like whether thousand bytes this way and the other like more than thousand byte this way it does create and validation which has to test around it so that's true if the things are not uniform then you just have a lot of Education right I would I would operate more under the assumption you know we will have more than a thousand bytes in the general case so let's use the optimized version and even if we're wasting no I guess what I mean is more like like at the bottom level at the bottom levels of the marquetry you would like hash it as like when the levels of the murder which

we get low enough to have 1000 bytes you would just hash it out an item and then above that something else okay well maybe flush the file sharing between now and the next couple weeks I need to think about Alexia's propose a little bit more i I might get as I would rather see one serialization format for simplicity but I I need to think a little bit more does anybody have any more comments or about this discussion currently and I definitely have the same the same instinct

well you see what I'm just gonna say another last comment is that the reason why I suggested that is because I'm looking at this at perspective like what have we done wrong in let's say in the theorem and the things that we know now that we can do differently so if we do this same thing as we did in a serum it might be one of those things because I can see now which which things are actually creating the inefficiencies on the foundation level and these are the serialization the unified serialization format is one of them one thing regarding is for examples Network I commented on the another of issue in the

container repo is that if we want a network analyzer like say a wine shark or something to be able to analyze a freon packet maybe we can just found some kind of plugins we've or own dis réaliser and this way we can use let's say SSC for serializing and it's work to love a specific Ephraim plugin for that hmm also I'm regarding the like hashing in serialization thing

another thing to keep in mind is that like a lot of the Oh like the exam the example i'm alexi mentioned about the patricia tree like that's probably not going to exist here because we're like if we're using this far smeargle tree than all the hashes are a nice clean 64 bytes so in this case the serialization is like at a like isn't a much smaller number of places and there is kind of like significantly less of it in an absolute sense yeah so I just wanted to say that for the in this regard so my main kind of argument is not that sort

of shifting the bytes but my main argument is that to have a civilizer for the hashing the most efficient you can actually create and that the event would be the one which doesn't really require require minimal memory we assumed the data from one place to another because then you can report much larger structures and with that necessarily in the memory although in this case the crystallized state is maximum four hundred megabytes again as I discovered like if you mean they would take what I did with the two but yes for example I

essentially say that we have a trade-off between keeping the the hash parts of the hash stream memory because we're actually very hard very difficult to read compute them brand actually like not you can fit in them but I'm saying in the hash tree much more efficiently then you don't or so much of it in the memory so it's like trade off and I want to ship made of in look if you have a very efficient computation of the tree then you be a memory for other things in your process sure I guess like at this point I'm so for the shard the be the beacon chains is specifically I think

they're basically isn't really any choice other than keeping the entire 400 megabytes of memory and the reason basically is that we have all these rewards and incentives that are going to be adjusting pretty much every single valuators balance every time there's a stereo there's a crystallized a recalculation so like yeah like although so to be basically like the beacon change state is a kind like he has a much more restricted thing that a false or incomplete VM I think we all wanted to say something about relived it should be

was it there was a mention to Wireshark plugins for decoding packets in the future at Limpy to be we have been discussing this I expect that we'll have a bit more discussions because the challenge there is the encryption so essentially getting is somehow privileged access the BTB stack to be able to dump you know symmetric keys and so on would be necessary for D consummating some of those packets and decrypting some of those packets just wanted to say that but probably move on today I think we should probably

continue this conversation offline and yeah Alexei has one another point about alternative tree structures do we want to get in that today a us can meet yeah yeah I'm happy to do it another time because what might might have a kind of better idea so it will take me a few minutes to explaining so I would rather live it to the next time so am I am I on some progress in POC as well yeah let's do that

thank you thank you cool so general z21 discussion questions again we're getting towards the end but if anybody has some comments or questions if you one now's a good time well I have one regarding the staking it's a one-way staking from Jimmy's one to East - would it be feasible to add a sort of an ice age in case the drinking contains some critic fault that the

funds are refunded unless there's an active decision to move on with it my gut would be to handle that as an exceptional case through a coordinated hard fork rather than get into it I'm sorry so you're proposing if we that people could get their money back out of that initial table contract if a certain time limit has passed and we haven't done

something yeah exactly as a way to encourage more people to be willing to stake the reefs mmm I see that I see the encouragement there because the otherwise it's kind of an unknown time limit I would rather the complexity go in the other direction but maybe we should we should consider a little bit more I don't know anybody have any thoughts on that just excuse yeah the idea right now is you can chain launches people can

deposit from East 102 become validators and you can change but at that point the beacon chain has they have no way to exit because the shard chains don't exist and you can only exit to one of the shard chains so early adoption it's really for enthusiasts that want to get their hands dirty because there's kind of this unknown time limit before they can get access their funds again that said they're also because a smaller amount of each will show up due to the risk profile they'll probably actually get much higher rewards at the time so I think that it kind of balances that out in terms of encouragement but definitely

we should consider how to encourage the audition a little more so worth discussing all right I can maybe write a post somewhere is yes nice didn't like it any other questions comment okay any other anything at all before we close the call cool thank you all for joining I believe

we'll have this call in two weeks we do have a meet-up in Prague on the 29th that I'm sorting the details out and I will be sending y'all further details via email etc and you know we we core to time okay appreciate all the stuff take a look at the testing formats keep working on bt1 let us know if you have any issues and we'll collaborate on the Gator thanks everyone thank you thank you thank you

[Music] [Music]

[Music] [Music]

[Music]