All Core Devs - Consensus #10

Transcript

[Music] [Music]

[Music] [Music] [Music]

[Music] [Music] [Music] [Music]

[Music] [Music]

[Music] [Music] [Music]

[Music] [Music] [Music] okay thank you welcome to the team 2.0

serenity call number ten people in the YouTube chat box please let me know if and when you can hear us and if any of y'all are monitoring youtube let me know what the transfer over 15 there and so let's go ahead and get started

we'll start with client updates I know there's some people on this call that haven't been here before let's do client updates and if you've even been on the call and your client comes up and you want to introduce yourself please do that and then afterward if you were not involved with client wanna treat yourself to do that I'm hearing a lot of thanking somebody oh great so let's get started who wants to go first plants Raul can you hear me how about prismatic goes first and Raul

needs his bike am I not I don't think I could hear anyone one second Tess can somebody I mute and talk to me

I can't hear anything yep hearing your letting clear oh you mean yeah did you did someone from prismatic begin did you ask did they yeah it did did anyone start the climate things sorry I'm not think my audio is that away no no one has started okay great Paul I can hear you let's have you start us off okay sure so we so we've been implementing GRP C API inspect of course we've started our validated service

which is pretty cool so that's something to come along pretty well focusing on the runtime and IPC at the moment and then doing safe should be it later one of our guys Mehdi has started adapting looking at that thing the H 1.0 C++ buzzer to try and point it at s 2.0 so that can hopefully benefit everyone we're kind of looking into as well a research thing as to how we can do hashing to G 2 in Milagro it's not as easy as we thought it would be yeah it's about us great thank you how about parity and we have way here maybe for the first time well if you want to do a quick intro can you get

close to your mic I'm sorry we're having trouble yes thank you okay and it's rather not really optimized and we calculate always recalculate the score on the fly and there are a lot of other optimizations we can do and right now we are working with factoring in subjects so that we can support the partner in both work twice first

infinity and I'm yeah because we also need these kids for the public to talk about thyroid health so we need to do that either way great thank you if you could point us to where LNG for choices and phonetic we'd love to take a look and if you have any metrics on kind of efficiency across the same work here is to thank you how about Nimbus hi so as usual we are keeping in secret

with the latest spec changes in the past two weeks we also focused on documentation continuous integration and reproducible builds regarding BLS we implemented the or scheme for and this time we copied the harmony approach but we still need to ensure that we are compatible with respect and so we'll be providing Python test vectors this week so that everyone can test where BLS implementation with test vectors and those will be built using pi EVM Trinity sorry as backing we also have a basic

fortress rule implemented with LMG ghost but without justified or finalized slots and the p2p demon wrapper is done for him it works for UNIX but we are waiting for the lipid to be team to improve Windows spots and for next steps we want to implement block storage because currently we only are using memory tables for the different blocks and we want to focus on the sink awesome thank you about harmony

yeah we've been starting to work to work on the implementation from scratch about a month ago this is because previously we work inside of TMJ code base which is licensed under GPL and we don't want a new new client and you library licensed under these spices and there is no feasible feasible way of extracting the code from TMJ code base and just moving it to some two separate repository and setting up a different license so we started from scratch and we're doing a

pretty good progress so far and we aim in to release standalone we contain client like in the first week of February so far we have implemented BLS simple serialize database infrastructure and all core we can change structures and the consensus is almost done we had not fear not the things that are left is our fork choice rule a validator service and roof work chain into creation and I

think that where we will manage to to get this implemented by the end of January I'll post a link to this repository now it's not there is no license set up for this repository yet I need to be careful here but we are choosing between a page two point O and and MIT or a the third option is to license it under both licenses on the clients choice so cool it is yeah thank you sorry that you got tied up

with licensing and had to redo a bunch of work but I'm glad to hear you are making progress yeah make maybe you could ask the people writing every mg if they are willing to allow you to realize inspired of their code to MIT or something we were thinking about it but it's about 70 contributors in material J ends I don't think it's it's feasible to to contact them all and to I mean yeah it's it's it might be feasible but it will require a lot of work on this point so I don't even I don't even know if all

of them are available at this time so that's this could be a big issue okay and we decided to start from scratch a month ago cuz that was the best time to start from scratch I mean at the time of the spec is still being an influx and there it still includes but yeah it probably was the best point to start a new repository okay thank you how about Pegasus I am so over the past two weeks we added two more

collaborators on the project we define the primitives from the spec we implemented the slot processing logic that's detailed in the state of the spec and then the other stuffs in this spec and then we started a POC processing we laid out scaffolding for our lid p2p interface and we shifted to micro-services architecture we think like long term that'll be beneficial for projects like in fira who kind of want a house like super nodes and we're kind of

continuing our comic sort of harmony about kind of combined in our efforts in java client cool thank you how about chain state lodestar hey we sort of been gaining a little bogged down was like expect changes so we've been spending a lot of time kind of architecting and doing the supplementary I work on my supplementary physic simple sterilize and BLS and they're coming along they should be done pretty soon and we can integrate them

actually like start using them in the beacon chain client and do to expect change we decided internally to kind of every two weeks will do an update unless something drastic happened in the spec that way we can actually get some progress going consistently for like two weeks straight and do a update and continue on that way outside of that we started looking into JSON p2p to kind of see if there's any holes that we can plug because there is a bit of a discrepancy between got some feature sets between that one and like the go client for instance are the go version for instance so we're looking to try to help them out and fix some of their

stuff up that's about it for great crust great thank you how about shall we with PI again hi so we moved our certain point all state transition called past to the Trinity a Satori and here's an overview document that and posting to the chat and most of modifications are said under kind of structures and beacon chain boxing and also the PI is the refactoring and

finished the tree having functions and Yannick proposed the test format and we can discuss that later after the regular updates that's the updates from outside thank you thank you how about prismatic passed since the last update we've had a lot of a lot of productivity so we finished up the state transition function with blog processing and processing up to a past few days in the spec there have been a few more changes in there that we're still working on we did a change start listening listener that monitors

approved work chain and kicks off the beacon chain after listening to a certain amount of deposits we have a simple deposits try also implemented and go that you know tracks and stuff and this listening to the latest contract we're gonna be using the VIPRE contract just to make sure that we're you know we're not we're not doing anything with solidity and we want to make sure that we're using what the spec is using we're implementing health checks for all of our services in our node just to make sure that when everyone's running in production things are smooth we started on LMDE goes for choice rule with the most naive implementation it's currently pretty slow around like 800 milliseconds to select a new head and I think that peleg mentioned that things should be around 50 milliseconds with a hundred thousand validators so we're aiming to optimize that over the next

update and thanks to Terrance from our team we removed shark committees from the beacon state which took a while because that was basically everywhere in our code base under the cool thing want to share with you guys is we have a llamo test for a full state transition without you puck processing so everything related to slot processing and also blood processing is done so I'll link with you I'll link you guys here in the chat kind of weather is Oh Terrence already sent - Thank You Terrance yeah so you guys can check that out play around we can basically simulate like hey we want sixty four sixty forty transitions I want you to skip these blocks at these slots simulate like a proposer slashing simulate like an exit so you can do a bunch of different simulations and see

what happens in the end we want to use this for like our end-to-end sweep so yeah yeah Moe has been really useful and building out the simulated back and has been really good for us so far yeah aside from that we're just you know we really want to wrap up for Chris rule and make it optimal so that's it oh great sounds like good progress and then we have Dean has joined us today Dean's been hacking for the past few days and most assure some updates anything spec in Swift I'm currently done with most of the helper functions the next step for me is

to implement simple serialize and to be LS stuff and then probably start working on the fork trace cool cool is that all of the clients I think I got everyone if I didn't cool when I said if you wanted an intro now's a good time maybe Stan I don't think we've seen you on one of our calls but I see new indicators what hey so I'm Stan I'm between jobs I work on random projects and one of those

random projects happen to be lighthouse I contributes for I've been contributing for about two weeks well are we trying to get up to date with this bag by attending the calls and rating it up this thing I'm looking at some issues I pushed some to tiny PRS into this bacon I intend to work more with you guys on get landing this thing Thank You anyone else when I get a couple words in and true I do a quick one because I also just like recently joined I'm Christoph I work on the Trinity team and as you may know Trinity of sunny theam one client as

well so my main work is not so much speculated but more on making sure that we integrate it too smoothly to work alongside everyone so Trinity will continue to serve both chains thank you anywhere else next up is research updates started I'll start on my side I spent some time trying working on improvements to or

coming up we still optimize the lmg ghost implementation which I get some of which I put into that file in the if you're young research repo it's in the ghost folder and they think prismatic has already been doing some updates based on those ideas so I'm really looking forward to is seeing more of the more client teams actually implemented goest and actually try running a chain with a very large number of validators and just seeing how long all the state transitions take is I think it's it would really help us in identifying whether ever everything is fine or whether we needs to and if try to work

harder on the for on on for sure its real algorithm stuck in love with some mechanism that's something that's more efficient also I've been thinking about my client stuff and yeah that one github issue that idea really I just published a couple hours ago today and we're basically what I am and uh suggest is that we commit the list the list of validator active validator industries into the country and state and we cheap Asia recent I history of those around the same way we keep erasing history of Iran down X's around and that that would

then meet and then what we do the shuffling for across winged committees and persistent committees around that and that calculate committee is even if or even for our light clients that only have via black headers and they can and they they would be able to do everything else was just Merkel branches so that's out there I think it's still the most recent github issue and it would be really nice to get more and if they thought and review on that for teaching on that otherwise there are some small things I mean it'd be ours to me make it so that these yeah up grew from work

shape I chain boating I said just on the deposit over it's over it's one black hashes but I think that already got merged great thank you does anyone else have research updates one of the main things has been my clients I've started thinking about those I guess historically are kind of ignore them somewhat and they're they're harder and more subtle than I when I thought and the design spaces is quite

large to an extent so I'll be reviewing you know the various proposals by the talaq and trying to improve them work and in terms of logistics around the github wish the github repo I'm going to try and move things a little bit faster so trying to close down all the issues that have been addressed that are still try and move faster on the PRS and I think one thing which might be a good idea is to avoid working directly on master so we have for example like

proof-of-concept releases that are spaced out by let's say six weeks and then we work on some sort of scratch pads that is out five abreast now I generally agree I think we can probably cut a version maybe at the end of this week of the next week and then begin to work on a dev branch I think that's definitely appropriate at this point and that'll help like chain safe was record discussing how they're gonna just do targeting like snapshots so we can we can begin to give those snapshots to people yeah and that will allow them to make change loves as well so that's you can wrap your head around the various

different snapshots faster another thing is vdf so a lot of progress has happened we're moving forward with with PowerPoint on on on some of the studies you know involving no very studies around PDF so a lot has happened and we will have a PDF day on February 3rd and there's other PDF activities on there before and there so quite a bit is happening behind the scenes and there will be even more stuff

that will happen on around these days and so I'm hoping to write an update communicating all the progress that have been made sometimes anyway great thank you I'll join you on that quest to move quickly and try to get all this stuff closed out still targeting something pretty clean and stable January 31st great how about anyone from the Pegasus research team outside with still working on the signature aggregation and we

because some time together Amazon notes we could because I'm just - there so we're starting to test and hopefully we'll have results in the next two weeks as well present thing in two exceptions and can you give us a quick reminder of you are testing yes so we wait on a protocol to aggregate signatures BLS in a Charles thousand of Els signatures in basically one or two seconds so on the simulation it works and we've done an implementation and go but we're going to test on 3000 nodes and we expect to have something in two seconds on something like this okay how about the ER you have any updates

for us yes can you see yes yeah okay so I have been working on a way to visualize the results of the simulations sorry if my voice is a bit broken so yeah the idea is to be try it because the simulations produce an awful amount of data it has five levels of perversity so I have been working on a way to visualize the data and so here is what I came up with this is the result of one

at small time that I did today one hour ago more or less and so it runs on on or NPA ranks and so it simulates 64 knots so it's a small network and here you can see the networker that is produced you can see the the light nodes here are nodes that have many peers and the dark blue nodes are nodes that are less beard so in the configuration file one can decide the minimum number of years that each node will have and then here you can see a figure that shows the number of peers for for all the nodes that are being simulated I don't know if

this is representative of a VR client at this point but I think for this run I said because it's a small network I set the minimum number of peers two or four nodes and then here you see the list of notes so when you click on one of those and you can see basically the main chain at this point none of this visualization part is it showing anything of the beacon shine or anything of the charts decide to start working on this but basically you can see all the blocks number hash the parents and minor and the time it has be mine and you can see

the entire chain you can also see the unco blocks in this case we have three for this simulation which is about 55 blocks long and then here on the bottom you can see the peers of these nodes which when you click on any of the peers you can see again the chain which should be the same chain you can also see the anchor blocks for this change will be CSS design and so on for all the nodes yeah so this is more or less what I have been working on for the last two weeks if you click on the bottom of the of the page if it guides it to the repository

so it's it's available online here you can see all the code and how to run it I just added the instructions on the wanted finish what I have been working on but I would probably add instructions for all the distributions I mean it doesn't change much and so this is again for a blockchain that is not charted at this point I have other versions that include the beacon shine and the charts those are private I haven't added those into this repository yet my plan is to lie slowly start adding those features inside this responsibility and also adding nice ways to visualize the the results of the of

this Malaysian yeah so I think that's pretty much it I will share the link if you want to see the the the visualization of the obvious Malaysian yourself and I can also add the link to the repository yeah and that would be pretty much all on my side cool great so in the system progress on that time thank you can I ask a question about this exit yes you know so what is the what are sort of hypothesis you want to test the simulations yeah so there

are there are several things that we have this question in the previous in the previous calls one was about you know what happens when the number of validators is it's not as far as one will be desired and how fast the cross-links will happen in that case another question is basically I think but it was posted in from one of these malaysian emails about simulating uncle rate versus number of transactions per block so that could ya know during well i think there are several other things

uncle rate of probably the beacon chain given Network latency might be interesting yeah that doesn't well indeed cool thank you can you unshare your screen I don't know how to give it back in yeah cool just to say what sir anything

great thank you any other research updates for move on I have a question regarding research it's a great time to ask yes absolutely I've seen a few posts on CBC vacation that can be kinky and those are very inside for thank you for that as a client implemented and what should we be aware of and do you think any of this structure changes will go into phase zero um I was so I think that definitely not phase zero at this point given that were trying to wine Z and like any age any further changes to the specter that

aren't critical to faith zero at this point that's at some time a leader I think it's still there's still there's still some details that are in that are in the research mode for T Tauri around finding ways to increase efficiency so I think it would be nice if we could that maybe after another like month another month or so of work of working on in continuing to improve this white for there to be a while at least the one team that would try kids to do what actually taken it take it and make an implementation of it just so we can see

how difficult it is and what the efficiency issues and what some of the other results are good but it's still like crack it's still like I like definitely further away than zero at this point great thanks Oh any other research anything for everyone another thing that we'll probably do is a reddit AMA for a film 2.0 research so if you have questions there will be a great opportunity to ask them ok research going once going twice sorry yeah sorry no mutant I'm just

wanted to ask patella giving it clarify a little bit more about fork choice optimization for people that might not be as familiar here with LMZ goes like you might I think what the big culprit for like the you know the inefficiency is counting the votes and figuring out the Glock ancestors but just wondering you know what you had in mind yeah so one of the largest optimizations that I had in is basically that instead of treating every single validator as a separate unit you would mean store a list of the most recent washed every validator voted for but when you

calculate the fortress rule you would treat every valid everyone who voted for a particular block hashes as a kind of single block and that makes produces a number of kind of units you have to worry about in calculate over potentially from like many thousands to a few hundred it sure makes things a lot easier also there's a team there's some small tweaks in terms of how you actually calculate the ghost fortress rule so there's this binary search mechanism for finding the largest hash that still or the most of you got a recent book that

still has over how over fifty fifty percent support and that's something that I've managed speed up well first from ofn if you do a really naive way then 200 again if you I do but or Oliver Lodge to give you a binary search based thing and now it's set down to a log squared n once and then there's a bunch of very very small scale tweaks to make it even more efficient regarding optimizations that I wanted to try on the fortress rule one thing is a data structure called the

Fenwick tree are also binary and x3 and it's used to keep track of prefixes and so like how do cells like that keeps being updated and everything is log N and another thing is that since we have everything coming in in committees instead of tracking every single validators just track at the committee level but I don't implement yet in name I just use a very naive one and but I

hope to speed up a lot as for choice okay moving on from research anything else anyone great the next thing is a posts an agenda from Alexi about primarily about how to deposit an alternative mechanism on depositing for validators from work to signal you can chain Lexy do you want to

give us a quick briefing and the FAFSA so yeah thank you very much so I listed the four in the post my post I listed four different points that I don't want them to be addressed now but I want people to start thinking and perhaps point me to a part of the specification which already answered those questions but I also present my initial thoughts about how to kind of improve the state of things and the number one some people already talked about it if finality gadget on the work chain which is which is going to be possible how soon is going to be possible after the introduction of the beacon chain so I've seen some post on the research research but I didn't get into the much

of the detail how like what are the details so second is the tapering of their wave of work rewards so essentially it's I don't think it has been figured out yet so how are we going to stop the the pre-work rewards inflating the the supply in the you know to point oh because then it will be sort of competition between between the miners in proof of work and then validators and proof of stake in terms of like over the supply ether and here my suggestion is kind of not very thought through is that if we implement the finality gadget as described in first point then it's possible to tie

the miner in reward with the remaining ether supply on the proof of work chain and since the the clients proof work lines will have to start watching the beacon change to implement the gadget they will also they could also consult the beacon chain about what is the remaining ether on the proto work chain and so the smaller remaining.if or the smaller is a reward I haven't thought it through but it's one of their ideas so number three is another thing that I started to think about is that the risk of the beacon chain validators essentially coming in getting their deposits in launching the beacon chain

and then not letting anybody else come in so essentially becoming the king of kings of the Queen's of the of this year 2.0 and at the moment I think when the current spec it's designed so that they have to vote on on let's say the proof of work block but what if they don't vote is there any penalty for them to basically just sealing the door once they come in and here one of the ideas I had is it's to basically tie the the validator reward in the proof of stake with the with the fact that they actually letting other people in so it means that if they stop letting anybody in then it they don't receive rewards of course it it has a problem that if the

proof of work chain legitimately completely holds then that means that the validators will stop getting their rewards but in this case I think the manual intervention intervention will be required anyway because you might need to hard spoon the hard spoon the proof of work into the proof of stake so because then a proof of work will be dead so the number three is the censorship resistance of deposit so the reason why I wanted to make it important is because there is an out talks about you've probably heard about the talks of change of proof of work in and some of the the arguments are predicated on the fact that you know if the miners are unfriendly in the future the name might prevent the proof of stake launch which

I trying to trying to dig into this argument and actually for I convinced myself personally that this is not in the problem because you can actually create pretty much perfect censorship resistance for the deposit and one of the ideas are described in detail there it needs some details to be figured out and the same at the same time this idea also allows the individual depositors to prevent the reorg attacks because they choose the you know they can wait efficient time before they reveal their deposits to the beacon chain so that you know they can prevent the attacks so I looked at also another question I looked into the specification around the

deposits truck data structure and I saw that there is the the in deposit data structure there is this branch which is the array of hashes and in index and deposit data so I'm wondering like we just I assumed this is the miracle proof but when the data changes the miracle proof also have to be changed so I just wonder how this is going to work anyway so that's it from me don't want to take so much time but any discussion is welcome but so the last question is basically that the because the proceed deposits or the Murli deposit merkel root in the approve

of or change change changes over time then how do you deal with the Merkel will Merkel witness is proving deposits changing yeah that's correct yeah so the reason why that's not an issue is that even if the proof of work chain itself might change walks of war the Merkel or first of all the Merkel route that exists within the state of the beacon chain does not change within it was in the process of executing a single beacon chain walk and so whoever a3 its to be in block it is our updated merkel

branches if they have to and then just include them because it's in it's all public date public data and so anyone can construct some easy i merkel branches based off of anymore what if they want to but also the merkel roots are constant for period or the for periods in 1024 blocks so in general i don't see it too much to to be worried about from that point of view okay so you're saying that either within this one thousand over the two thousand blocks there was no need to reconstruct the proof because you would just take snapshot every whatever 2000 block and then if you passed that period then you have to read reconstruct the

proof in and put a fresh date in there right yeah well it's the bigger thing is also that like even yeah the person basically whoever creates the block has the ability to edit to edit the bruisers they have to ok so then i think this is a this is a good which no it identities to my other observation about the incentive of the validators to essentially sends their new deposits so how are we gonna prevent this yeah so I thought your suggestion of making the

room validators rewards contingent on new validators joining what's interesting does that premise that does that require the new validators are like if we reach some sort of equilibrium we're given the risk reward profile of safe a zero or one that no new validators are joining does that mean that rewards slowed down at that point and does it require some sort of continuous flow I don't know I'm not sure if I understand the function yeah I think it might need to be and if thoughts of room or thoughts or more but in other points though is its is that this issue is not it is not great

especially greater for aureus I don't see why the issues especially greater for the proof of work support of state transition as opposed to or just regular operation of the group-stage chain because they if at some point in the future there is some set of active alligators and that set of active alligators decides that they wants to and of cord vegan chain your rewards for themselves and not accept anyone else joining then they will have the ability to do that I think Alexi is concerned about maybe a like the low security and low validator count environment early on the beginning when after the beacon changes started there obviously will be

smaller set of alligators and those say they they might be more of the problem than later on when there's like a larger pool of others one way know about unprocessed validators is like we are voting on this deposit roots we're voting on this piece of information from the contract and that could also include some sort of like index or counter and so you can chain could be aware of disparity and the number in cone deposits versus what is expected based on the Picasso group but that would still hinge upon people successfully voting on the deposit which would be the target I guess at that point that is interesting like if we require the

indices to keep on sick or or if we require them or the order in which the merkel proofs get included to be sequential which is something we were considering then and if we it could potentially make sense to just add a rule that says that a vegan chain block should be considered invalid if it if it includes a route that has like set that has a size larger than some value unless certain or a that has a route that getting that contains and Merkel branches unless at least n minus ten thousand or so or and minus whatever so

no were already included but then the the attack vector still becomes just not coming to consensus on the new well I'm not coming against I just on the proof awarding yeah because of Union if you can prevent deposit so you can probably just put out come into Texas area 14 yeah so I just wanted to throw this in so that people can start thinking about it and maybe we can come back to because I've listed out in the I listed out the the I think the problem that I see in the you know agenda so maybe we can come back to these questions if there's no immediate yeah home comments I do want to put a couple comments about a counterfactual one is one it it messes

with the initialization count so if we use that from the beginning you don't have that firm count of kind of chain start count then threshold if everyone's kind of actually doing it and I think it's harder to construct the initial validator set and it's also the mechanism is tougher to prevent double spins or like all deposits over some amount right now the idea to prevent double spins is to process deposits totally an order from the deposit contract whereas the counterfactual you lose that so you have to remember you do remember deposits because of the pub key so you'd

have to maybe like ensure that deposits can't be past a certain age or something once pug geezer cleaned out once the validators left the validator set so again I think the double spend is a little bit harder there we I'm going to think about this finger so I actually actually thought about this problem but I need to think about it more okay cool any other comments on these handful of ideas before we move on and yes I think it's probably good to bring it up I'll put on each other for two weeks from now you can say discuss these silly goodies yeah just comment on the small number of

validators issue this is something that I I want to study with you later on I it will be interesting to evaluate the case of a smooth transition in the number of shots so so we are right now working on a chain that has basically one shot and instead of jump into 1,000 charts from the night of the morning maybe would be interesting to see if it makes sense to do a gradual you know increase in the number of charts I come from the high performance computing and we know that it's getting is hard and we usually do the experience you know smoothly and proper this is something that I know that decrease in the number of charts is very very hard but maybe increasing it's not that hard

so this is something that this could be studied yeah I think the I think it can serve the role I can make something like the one complexity is it just that it adds consensus complexity but again it adds consensus complexity and it would be a probably very significant change especially given where phase zero already is at this point in terms of I don't think it would be a huge change

with respect to it would just be like slowly unstopping cross length duties right okay no hmm rather than it's such if the shard size was unbounded than it would then it would be really breaking but if the shard size was just bounded and then going up and going up to a maximum then you know wouldn't yeah I think I think that was okay thank you anything else on this kind of range of

Alexie's comments in the 340 mistake transition and alexei there is there was an explicit mechanism added to the spec the past couple of days that instead of just devoting on this deposit route and the contract it votes on a combo of de positive root a group of work hash and so the final of the finality gadget is explicitly in there to be used at this point okay okay right in muster yes okay so the mechanics from the beacon chain side are there and it would be kind of coming up with a plan and some social

consensus on what that looks like on the group report side over time probably like lowering the follow distance over time okay what can I think great we will move on to the next item on the agenda which is the test formats discussion Yanik has two new gamal test formats that have been posted hopefully you've been able to take a look Yanik do you want to just give us a quick on that and then if anybody has any comments now you can take them otherwise we'll try to get

these kind of like thumbs up or proves the next day or two so people can be done using them yeah like sonic here oh yes yeah I think that pretty simple we can start with a simple serialize test we have basically three kinds of tests and the first one is where everything is valid we specify the type of the thing we want to serialize then the value and then the still less value in bytes basically a hex string and then we also have a test for invalid values that have been sealed as invalidly for example with strings

that are not too long or too short that we do not specify the value but only the SSE string and a short description why that is wrong and a third kind of tests where the value to answer lies is wrong for example if you have an integer that it's out of out of range or an a byte string that's too too long again with the description and then we only specified value and not yeah not the SSE types if possible their specified strings for example bool un Speights addresses for lists we specify them as a Yemen list with a single element that

specifies the and the type of the element of the list and for containers we can use mappings yamir mappings which should be it should not be a problem because we and for some time ago we agreed that we don't have to care about order of fields yeah well use for most values we have ya know things which represent these values and I think all the rest is pretty straightforward and for the three hashing things we basically use the same type definitions the same value definitions and then we just add yeah the tree hash of the value yeah any

feedback on that to change what was the difference between number two and number three the corrupt zero as value versus the invalid value basically the direction so one is we have a value and we can't see realize it because it does not match the type basically for example an integer that's basically if you want to serialize 500s in you and eight that's not possible or if you want to yes you realize 50 bytes is about 32 it doesn't work and the other way around is if the if you get the serialized string and you can't deserialize it properly I see I see so it's 1 1 has the SSE

provider one has the value provided I see thank you exactly a quick question maybe the first case where everything is alright I assume that you're going to we're going to want to include some some sneaky edge cases in there maybe it would be valuable to include a description in there as well so that a developer can focus on what is Skaar to get it's in the description but very supposed to be a short string but I think one that one thing that could be valuable is so that the description is almost always the

same or offer is some kind of tag because this way you can recognize some types of age cases okay that makes sense and okay adding a description to four valid inputs maybe it also makes sense yeah and maybe you can make sure makes in them optional in all three cases because maybe for some it's very boring or obvious yeah I think in the general format the description is optional for all test cases yeah that's fair but mommy you were suggesting

having some sort of formal tag that calls for example in the if rm100 test suit you had a lot of overflow tests but no tags for overflow but so maybe it's easy to recognize okay so I try to run all the overflow tests and then may be invalid types and I don't know if I'm clear but that's what I would like for invalid things so basically some way to filter the test so that you only select

them those that make sense yeah well maybe it's too early to start that because we should start somewhere but if we get like 5 or 10 or 100 tests for because one edge cases seems to be quite tricky it might be good to to introduce tags but that can be done like that might be but essentially right now I want an overflow description and an overload description and might actually introduce some sort of tags construct separate and leave description as human readable yeah I think that would make

sense I have to think about this and also what kind of takes make it make sense ok oh I had one question and maybe someone from Paille bein us some pots I saw that the proposed bytes one that was to use the 0x prefix I know I remember a couple years ago there was kind of a pain and web 3pi with respect to parsing some of these Oh extremes I think if I just want to make sure that we're not we're not making something difficult here I think it might have been maybe trying to parse

the Oh X trains and in like ambiguous context and if that's the case this isn't really an issue Kristof do you know given the knowledge of issues of parsing X nothing been I'm aware of mom okay yeah I think I think it I think it might have been some ambiguous context and you couldn't really tell from the string and so if that's the case it should be good here okay these look generally good I'm gonna I'll think about them a little bit more and add some comments and if anybody has any red flags or whatever we please address it in the issue and we will I think generally just have the thumbs up on

this the next day or so so people can move forward cool thanks everyone thank you great so then we have the next item in the agenda is just general spec discussion does anybody have questions comments concerns about recent spec stuff each respect stuff etc miss Terrance here I saw like people shout out to Danny for working on the honours validate respect he has been really helpful i have been reading it

and he has been helping us a lot on the on the implementation side yeah absolutely but still working that's still a PR I generally have the bones in place and plan on getting it merged and then continuing to flush it out but I'm glad that it has been abused its it became more and more obvious that it was needed as people have asked me more and more questions about it thank you once we a little bit more mature may be a document in a similar vein could be a list of suggested optimizations for implementers yeah I mean you know maybe a little bit more of an implementation

guide or suggestions inspect stuff anything anybody [Music] the container and reformat the drown and how they encoded the fuel and encoded beef actually are for the rest of the

company like I'm just wondering in like like what our configuration for the GUI like the cult because it like like account of the perfect Maui County like between the dreamer containers track device but we need to wait for the food to fuel at 5:00 and we take that with of the prototype in the beginning for the third wonder like what of computer even here yeah right so just to recap I think the issue is when we prefixed the length

on the container in list types we've made it difficult to stream yeah I know we've discussed this here before does anybody have any thoughts on prefixing there's post fixing that length right main arguments in favor of prediction was that this serialization format is going to be used over the network and for the network stuff it's it's important to know how big the object that is gonna be decoded how big

it is that's why we need preferences you know that but for like networking stuff like like you wouldn't like like you wouldn't actually use Isis a format to just exist because like the graduated for medical consensus and you would prefix it in using some other data structures refer them into the network but one month in I think I think the perfect son called comfortable drawback

is for the the hashing there if we with just anything in the hands of chiropractors now that we must relocate faster than me we can't you at least read into act as I remember we had the discussion about simple serialize and maybe use wrote above for Network and they use simple serialize for consensus but as I remember the final decision was to use simple serialize for both

networking and the consensus stuff you know so I already pointed out the this problem that wage is described that if you try to use it as for hashing you have to pre allocate the memory that was exactly what I was talking about like probably a lot but if not like Lippe to P will produce some tools

Wireshark D sectors if I remember to be able to analyze whatever format we end up using for to peer networking in I show you a beat SSE or proto before captain proto or something but as of now we will use a Z unless I guess varies on module blocker and there is a discussion open or maybe closed on the tests also old if area becomes Enrico about that so you can ask wall history of the discussion with lots of inputs from my alexei well my current

opinion on this is that I agree with way that if there if this format is going to be wrapped into the other packaging like late p2p then that packaging would add necessary prefixes so you don't need to design as a Z for that purpose so in that that is it means that it you need to optimize for something else if it's gonna be used as an input for hashing then I wouldn't use prefixes as I said before well already have a hashing algorithm right I mean the hashing algorithm that is that is used in a consensus part of the spec it's already it's also described in in the simple

serialization spec in the three hash section so I mean do we even need any other hashing so far which require which would work with the streaming so the point is that we have a specified specifically hashing algorithm so right but is the idea that if we post fix the bite on s is e then we can more

efficiently hash the various objects in the tree yeah I'm not sure if that would matter much because for like the objects in which we either have a very short fixed length or if they're longer than generally they're more gold trees right so given that structure that we're using for the consensus hashing that we're not doing flat hashing for instance ashing does this pre or post tricks actually matter um did I get correctly that that SSD is not using named values for

serializing the elements of a structure what do you mean by using named values I mean like in Jason we have different fields with names and jason read it so give me a like to like do the variable the names of their key no like base like we want to know it once it just gets turned into a tuple with the and it is very kind of higher-level human-readable thing okay I I see so um for me that immediately shows a sign of a problem that might might be there

might be a problem of this because imagine you have a data structure that has many different fields of the same type and for whatever reason at some point for example you need to you need them to switch places in that case you might you might get into bugs where you have same filter of same type but in a different place within a within the data structure and for communications it might not might not be unnecessary to cut out the named fields but if you yes if you were to do that you might run into problems of of struck stack of two different structures that look the same

one normally is sorry nobody sec we should have some kind of spec version and so when you communicate with someone you also spent the SSD version you are speaking and so you agree on the field orders and the content that way so it's kind of a schema of of the the communication is written in the spec with a certain version okay well I just wanted to weigh in on that because I think it is it gives opportunity for bugs that are very

hard to trace if you for example forget to bomb that version it's very important we all agree on our data structures in the system anyway I don't think we're gonna be changing much for sending schemas around with these messages it's gonna be a huge overhead because we have to be really careful that how much data we're sending around yeah I did some benchmark initially like four months ago and I was sending more data with the scheme it was taking more data space than structural data we wanted to send yeah I can imagine it's not something we might

want to have I'm convinced do we need to open up an issue to discuss this again or are we happy in the current state my opinion is that we need some numbers I mean some benchmarks to see how current session algorithm works how its efficient and maybe compare it with the flat Hessian with

the flat hashed indents in streaming mode so just do some experiments and see which one is better all right second done yeah and I think on the network side I think we were going to wait until we were actually pushing packets around on a network to be able to make a more informed decision on whether as to see it's gonna be legalized there or not so let's continue forward as is for now and address this or an eye for information okay

shy away pointed out to me that we have this PR that we've all known and loved and then 139 that recently Justin approved generally the arguments laid out favored slightly in the direction of little-endian there's anybody and it also helps substrate because they only

support little-endian and I know does anybody feel very strongly about merging this poor request that has been sitting there graduate what are the arguments for a little endian again just enlisted consistent with was him consistent with commodity hardware consistent with parity codon pawns inconsistently ROP they call all the realizations ok wasn't all Indian okay I know Vitalik your argument Oh while back was

to not change the Indian Miss from one oh because then you have to remember and that's just cognitive overhead for pretty much not much game at all given the constant blossom is that still true yeah just fine okay this is a major major win for yosik he's held out for a long time and we can no jest at all cool I think how much people might be in

the Stanford area at the end of January I think a bunch at least some amount of us might be and eath Denver in the middle of February so you should get together communicate about that offline I don't think we're gonna throw do a specific event unless somebody else feels like organizing something cool any further sorry any further respects caption any further comments questions anything before we close this meeting great thank you all for coming really

exciting to see all this progress as always communicating the Gators we're gonna start versioning on the spec repo and punko and I think two weeks I've looked at my calendar but I believe two weeks okay thank you thanks everybody thanks thank you [Music] [Music]

[Music] [Music]

[Music] [Music] [Music]

[Music] [Music] [Music]

[Music]