PeerDAS Breakout #5

Transcript

e e hello

hello let's wait for one more minute everyone let's start the

Cod so uh we start with the client update anyone want to share think not every client uh attending today but anyone wants to start start so maybe before the update uh just didn't double confirm So Def One was shut down and then currently we don't have any active def is that correct that's

correct okay um CL update um Rec uh so during the last two weeks uh we work on a fix about our initialization also on a fix about our sampling about new feature we implemented the uh metadata vory with the custody subet count is the metadata so Bic basic if the pier is able to respond with the custody subnet counts in metadata vory we use it else we fall back on the Node record um we implemented this pool

request this feature um even if the P request is not yet merged in the specification um yes and what we are doing now uh we are working on the blob side CS uh um Bon API right now there is a is V1 Bon blob car Bon API is not able to respond with PES uh and also we are working on a better pipeline for data colum verification um yep basically this is it for prison thank you

and then Li house hello hello everyone um for Lighthouse um we have been debugging our networking issues um previously we had issues with um our Lighthouse node disconnect disconnecting other Lighthouse noes so um we've identified the issue and we've made a fix for it so there's some improvement in network stability and then we've also been doing a bit of debugging on the Russ kg library and um C has made a few fixes for us so I think our Des branch is getting a bit more stable now and currently we're focused on refactoring

and progress LLY kind of merge our D Branch into us into our main branch um we've made some good progress so we we're about halfway there um I think we'll continue to focus on finishing the merge into the main branch so we can kind of pick up the speed a bit um quicker after it's finished emerging uh another change that we've made is we've disabled um sampling by default for now because um I think sampling was causing us a good issue and they were trying to isolate some networking issues so it's um Curr disabled by default but we can also enable it um I think although there's also a proposal um from Franchesco and Alex stok about um not shipping s with P

so I think that could potentially be useful if we want to enable and disable um sampling in the future Deets so we've got a flag en flag added to to enable or disable P um P sampling um I think we're probably going to start preparing for devet 2 um soon um I don't think we have a spec soon but I don't think we have a spec yet but I think um we've talked about some of the things that we want to implement like the metadata B3 so we we might potentially look at that soon as well at the moment we're trying to focus on getting our D Branch merg into our main branch and then we'll start looking at the P um dep to specification changes um

that's open up from my house thanks thank you any other client low star hey guys uh so there isn't much update on LOA I'm working on getting the metadata p uh PR implemented and uh but otherwise I think without sampling we should be good for any uh devet that we can spin up but uh I have a question with regarding regarding to with regard

to uh to the data sampling so are we going to keep uh data availability sorry with regard to data availability are we going to keep data availability uh for the current slot or it is going for parent slot uh because on the basis of that there might be some refactoring or some Dev work that I might need to do for uh for the folk choice and I think this is sort of an important Point uh what is the availability that we are envisioning when we go live because yeah a lot of Works depend upon

that thank you um Brenan hi so is from gr team so uh we had some some fixes for the P so a few of folks are working on on this from the fellowship program so so essentially I think we are moving forward however I hear that there are sh there are discussions to

remove uh sampling and during this call I would like to hear what is the status on this and if if it's going to be accepted quite soon thank you and um do we have any take yeah uh I have found and fix it several issues with our default sampler and also we have implemented lost sampler but

it's difficult to test it in local environment uh so until devet launch I'm not sure in it uh both Samplers are not uh completely integrated in for choice so we just lock uh when we sampled we don't break Fork choice if we cannot sample uh we want more testing before enabling it and also we have edit all required by spec validation and gossip and the rec crasp we didn't have it before and uh everything is not merged yet uh we both me and an now on vacation so I hope we are not launching the net

to on this we uh the s for me thank you uh do we have NE bu here today yeah yeah hi hi yes oh yeah so last last week um we actually focused more on having a lesser diff um in terms of our Petra branch and the pias branch so we started off with upstreaming the ckcg to the latest and we fixed all issues that were um basically creating a diff in the pectra branch first uh like

having that extra pre-compute parameter uh to accelerate I think load PR setup um then um last last week we started implementing lossy sampling as well uh but we sort of halted it till Thursday's call like last last to last Thursday's call where I think uh we all agreed to drop sampling so there is not much progress uh in that region right now um after that we refactored um Alpha 3 specs and also like all the related crypto stuff as well and while passing the test vectors I actually came across that there was some inconsistency between the alpha 3 test vectors and the

alpha 3 um public functions that were exposed so I think that is fixed I think that can get fixed in the next Alpha 4 release we are still wiip on metadata V3 uh for custody subnet count and we are still uh working progress for the upcoming PRS like validator custody and the fork choice so yeah that is pretty much from nimus thank you [Music] so on the agenda it was about definely

one but think do Babas is here now and then do we want to talk about the restart of definite one or talk about definite 2 instead what's the Cod of the next def exactly I haven't been testing uh any per related branches lately on crosis I would be happy to begin this again uh I'm just curious about client ordinance basically

yep um can we set a date for the next devet if other clients uh feeling ready to and yeah and also like uh picks up on the features that we want for the next step I think before we sort of set the date for next I think yeah it's worth discussing what the spec is going to be and uh my question that I already raised uh in when I talked earlier was that uh so what how is the data sampling going to look like when this goes live because I think we would want uh we would want

the next version of devet to look as closely as how it should be in the life so if somebody has an input on data sampling point because I think I'm very curious about it um could you like clarify what issues with sampling do you have is it replace sampling by availability sorry uh I meant availability so data availability uh in the sense that are we going to do a data availability of the current current slot itself or are we going to do data availability of parent or let's say of an anth ancestor so uh what is the data availability going to

look like when we go live it's it's it's always for the current slot correct um currently with the neb if you want to validate the block you got to validate the blobs for it in the current slot not in the next slot and so on yeah that is okay but are we is that what we are going live with with what we will go live with for p yeah um I that's for us that would be how it works right now so if you get a a block you have to validate its da uh at its current

slot yeah for but in the spec for example it was mentioned that the da happen of the previous of the parent slot right so is this for sampling or cuz from what I understand for verifying any block for it uh you want to validate it da it's always done you know at its slot itself yeah this is not regarding sampling and I just want to sort of clarify with regard to availability I mean sampling is easy to do D of an ancestor because it does not affect the folk choice that you have and one of that is sort of my major major curiosity that uh that are

we going to stick with the current slot or the availability is going to change because if it's going to change then the fork choice will change and hence there is some I think that should be put in before we launch any other devs um so there's a recent um proposal by Alex and I think um reesco made a post about it like just today um yesterday for some time on um that um like how it look like with our sampling if we don't do sampling then we're just doing the um the custody column verification which means we we we're doing it for the current slot is that

correct if we're just doing the custody columns and no sampling we are only worried about so when we are importing the block and the availability check is only about the current slot of whatever custody that you want I think then yes then there is no major change that is required in the fork choice and sampling is basically to see whether the perer is relevant or not and it can happen at any point or not happen at all and obviously for example you are using uh availability of the current slot then you already can say that we have sampled for the particular custody

that we are interested in uh so I guess that is okay but uh yeah I just want to confirm that whether uh whether availability checks are going to be on the current slot or do is is there a possibility that availability of the parent or of of some ancestor will be checked because that will radically change uh the FK choice I mean you can't import a block if you haven't verified its parents da right so I'm guessing there's no change from how we do it in dinner right now

fors I think there was a version of spec where it said that availability checks will have have will uh will happen for the parent but of course if we are doing availability checks for the current slot while importing obviously we would have no block ined in the folk Choice which does not have ability so the parent does not come into the question uh so if it's going to stay that way then I guess it's okay yeah it should stay that way um I think you're referring to sampling then because that was when it was brought up okay I'll recheck because I think I'm confused I was confused and I was

thinking that even for availability that we'll import a block if it's parent is available so parent data columns are available so uh if that is not the case then yeah my confusion is sort of resolved yeah I I think you're referring to sampling in that case uh CU I remember that is when it was brought up okay cool thanks um with regards to sampling so I'm not an expert on this but uh because someone asked uh whether you know sampling is going to be the final form or it's not being it's not going to be in the final form I think it's still

under discussion and I think and by sampling I mean like sampling of not the current thing of the you know after the fact basically uh I think this is still under discussion and looking at franchesco's newest cost I do have the feeling that if um people think that it it greatly reduces implementation complexity I think it's quite like like you know making this clear will will be will make things easier to decide basically uh because it is mostly an implementation complexity reducing suggestion so you

know from this discussion I get the feeling that it really improves implementation complexity so if that's true I think it is worth mentioning uh and it might it will influence whether you know we end up doing something the end or not okay I'm just reading from the consensus specs that has been sort of linked from the EIP and it says are not on Fork Choice Fork Choice uh it will be a replacement of his data available

calling with column sampling instead of full download uh not that his data aailable will likely do a minus one follow distance so that you need to check the availability of slot n minus one for slot n so it's not about sampling it's about availability and if there is uh a change of availability in the sense that for in the current evet we are doing the availability of the current slot or block that we are importing but if if it's going to parent then it will sort of change the way the FK choice is implemented and it needs to that needs to be taken care of uh in the sense that why I mention why I mention this is that any Dev Nets that we spin off should be as close to the Deets that

we want to go live so if we want to do n minus one availability then we need to sort of Dev that code in and get it out and sort of make sure that next Dev Nets we are doing it on N minus one sampling instead of the current end sampling availability sorry so what are you referring to is trailing slot Fork choice so in the interrupt we kind of decided that it is not a good idea to do this because you don't you do not want to put sampling in the critical path for consensus which is in for choice so I don't think you know we have to like worry about checking for this that in that case can we sort of update uh the specs

and sort of Link the correct specs in the EIP because it can be confus yeah I agree um we should probably update there have been a lot of changes so on another note um I've I've gone through Francesco's post among all the other clients is is a contentus that you know we should uh go with subnet D and leave you know peer sampling as something to do in the future would it make implementation

easier for you guys yeah I think uh sampling is sort of what I see is that it's sort of an independent module that can be added and uh uh once we have a working devet we can sort of increase the complexity by adding the sampling so that the pairing is stronger and we can have a stable devet uh so as as of now I think I'm good with the adding sampling later on yeah yeah also agree as well um I think sampling um I think I think a few

clients already have sampling implemented but I think there a qu of edge cases that we need to think about and also handle so I think if we're exclus samp that's going to reduce the complexity quite a lot and make it a bit easier to to make progress so like when you you guys mention on removing sampling do you just mean for the devet or do you mean in general for pias as it is in my opinion I think we can decide about sampling in sort of later parts of

the deet where we think that things are stable enough that makes sense I sorry um I think franchesco's comments was also like saying that um like if we can tweak the network parameters of something do then sampling doesn't really add like more security I think that that was the argument that we that we could potentially exclude sampling I think there's some trade off there um

so yeah might be worth um having a look at the post I haven't actually finished reading it so can't really come into much about it but it seems like Fresco's pretty than that we done ship sampling the security will be okay sorry shall I no I just just want to wrap the conversation um so is that the rap conven that the client can keep working on the networking issues and the refactoring and that same time that battles will try to refine the simpling logic that what our next to do in the

next one or two weeks do you need give me a thumb up any objection thank you and then any other so we can talk about the sampling logic in the cat just before that I want to mention that we will have a a spec release uh today or tomorrow um the release notes is here so mainly the

regular release that I don't think it would be the def logic uh but uh this this re will include some kg updates and just want to sure make sure that it won't um influence that what you are working on lately yep and is there any like urgent PRS that we should included or it's okay and I suppose the next or I mean the next release or next next release will

will will try to focus on P definate I think it would be helpful I think it would be helpful if metad data V3 is uh like reviewed and merged um you mean the four D between the release give me one second um oh om okay this

one okay uh I will include it after this Cod thank you and any other update uh sorry any other request uh please let me know in like I gu is a Franchesco Fork Choice PR which is not get mered and should not get immediately merged but uh it's likely going to get in the next release and it does change a few things but it's not for this release but it

seems like uh the next spec release is also going to be important because it's going to include the sampling stuff and the forur Stu yep uh agree um I have a I have a query if we are planning to drop sampling in the consecutive Dev Nets then uh in the next release why would we have a sampling merged I don't I don't think it has been

fully decided whether we're going to Dr sampling in the next evets has it been decided I'm sorry if it has I have missed it okay okay okay uh okay I get it thanks but I I I do actually think that we should very soon decide whether the next devet should have sampling or not uh I think that is something we should do very soon yeah it sounds good I think we already decided that next devet will not have sampling in fact next few Deets will not have sampling uh but later on when we'll have stable Dev

Nets then we can decide to add on sampling uh but that was sort of my import of the discussion that at least for the next few Dev Nets we will not have sampling okay thanks I had missed so I I think clients you know who have already implemented sampling can continue doing so but it shouldn't be a requirement for any other client who's having issues to actually have sampling working so they they can with the with the only difference that they should not disconnect appear if sampling fails I guess because yeah so like for like like for us what we do is that uh we try to

sample something and if we can't sample it we just log log it up it doesn't really do anything but it it does test you know that uh our incremental D Parts they actually work and stuff like that just a question regarding the next um dep net launch uh is it work um is it worth creating a like hackmd pay um for the definite to spec and um I know I

know want metadata B3 implemented um I don't think we want to include any further changes but uh I think would be good to include things like making sure that every client serves the bu root and by range request before launch deet oh b b dropped one immediately nice um so think will include the

meta uh metad data and some kg updates I think we should aim for off for can the C please a is that's okay I think if we are targeting Alpha 4 I guess um we at least need some more time for the devet like if if we if we agree that the devet 2 is or the next DEET is

compliant with Alpha 4 yeah yeah okay okay yeah then that's that's okay yeah I'm not sure when off four could be merged in theory but I don't expect it to be happening right away mom I plan to mer 24 hours but of course without F choice so just current just and then the metad data apis okay

okay um do we have any other open discussion yeah just wondering if there's any progress with the custody group chain proposed by Pop um I think it's called decouple um the networking subnet with Dore um I think that's going to be a bit of cod change so I kind of want to know if that's something we're going to go ahead or not he's not in the call so you are sry pop if you are talking we can't hear

still know message in the chat that you can you hear me more reviews H yes yeah yeah I think I think I want to go ahead but uh

yeah but but but I think like uh I think I want more people to to look at it first I think like I think there are not many people to look at it curly but but but but I think I yeah I want to go ah it is that good uh okay a moment it feels like most people that comment

on the pr generally generally agree to it um but there's just not been like full more approval by actions yet yeah uh the pr looks good to me um just it'll take us some time to get the changes done but if no client against this I think maybe we can Target this for Alpha 5 think we got good supports today

so um yeah al Five we can Target on the Theo PR and also the Top Choice and um how to deal with simply that's the three main uh goals for the next next release okay and then any other open discussion um Franchesco is going to join the call in five minutes he says and he also told

me that the data availability checks should happen on the current block not the pre the parent so if the spec is ambiguous in this regard we should probably make it more clear that said I mainly joined the call to ask if anyone has questions or about the cas G libraries or um uh if people have I heard that lighthous is has has been using kev's rust Library which is nice

uh if people have questions or transition questions or anything um yeah no question from us at this stage um I've run into a few issues but C's fix it pretty quickly and um I think it's looking pretty stable for us now thanks nice any other open

discussion we're waiting for Franco I guess we don't need to wait for Francesco and we can sort of resolve

this on Discord or something right only two more minutes yeah um yeah you have any questions um looking at Francesco's um post um do we want to talk about the the complexity about sampling um has it does does any CL have any issues regarding um having samp robustly I guess the opposite would also

be interesting like uh having uh sampling implemented in not modular way suchar that disabling it is complicated if we end up with disabling it in another little open discussion

it was a long five minutes um should we move this conversation to Discord oh okay one f one open discuss so on the agenda I think someone mentioned that uh princess I I want we should make this by weekly Cod weekly that's the preference from you guys should we make it to a weekly so that we could probably isolate the the process any back

suggestions um what's the benefit of doing this weekly for Francis think mainly just um more frequent updates frequent Q&A frequent you know problem Sol I guess I I think uh francis's issue was that he's based out of uh Western us so he he can't attend these Cults at all even if we do this weekly um at this time he would still face the same issues right

so um so it's two questions one is to make it more frequent and then second question is to change the you move it to more us friendly time um from experience uh moving it to a US friendly time would make it like terrible for people in apad cuz um it would be easily I think 10 to 12 at night indeed if this man a little drum

TR voice Franchesco hi hello all right would you like to talk about your new document uh yeah sure I mean I guess George said there were some for choice questions um which didn't seem super related to the document so if people want to start with that you can talk about that otherwise okay then yeah um yeah basically I mean the document is just uh sort of uh a bit uh expanding on what Alex mentioned at

uh last ACDC or whatever it was um basically saying you know that there at least is U an Avenue to like further simplify um Pier d uh to where really just for the moment not worry about um implementing peer sampling and just leave that as something to to then do later and this can mean just that uh it can end up just being like a implementation and testing cycle kind of thing where you know maybe in like a few months um then we are in a place where we can put peer sampling on something that already exists and is working well or it can mean that in a first version um in the first roll out really there is no peer sampling and uh then is just

added later at some at some later point um and yeah I mean there's not that much more to that I guess more uh I mean the the document is basically just trying to convince people why why this is not a crazy idea and why it's actually kind of secure and uh um yeah like is sort of okay to to do um and I guess yeah some some little point that I I try to make both there and on Discord is that this is not some um you know last minute thing I mean at least some of us had been thinking about this from you know even at the end of last year um that as a potential option even though um it's not what uh kind of was uh pursued for for

the the beginning because it seemed like you know maybe we can make the uh let's say a fuller version of P dos work and and maybe that's still the case U but yeah this is like something that already kind of I mean the whole subnet us concept was basically this essentially like uh um essentially like an intermediate uh stepping stone where the sampling part is really just what is done through distribution like uh what basically custody like uh just um getting U columns through through subnets uh rather than having this a separate like peer sampling mechanism uh so it's it's really just about yeah avoiding the the complexities of uh of peer sampling which is uh I mean all of

the things around timing and uh you know figuring out when do you time out when do you send parallel queries do you send parallel queries how many do you send um just all of this you know dos protection like there's all these things that really are tricky about um sampling strategies and I feel like there's probably um room for that to be a bit more of a uh you know research um question or or at least like a something between like research and implementation as opposed to something that's just shipping um so that's basically it I guess I would uh yeah love to hear from from people if you actually think that this is helpful to you like if this you

think this is actually is a meaningful simplification um or if you don't think that it actually makes much of a difference and like getting sample to work um yeah just doesn't seem to be like much of an impediment at this point yeah I think not having sampling would U to simplify it for I think pretty much all clients but I'm worried about uh if we go into the fork with just you know essentially subnet Das um adding in pure sampling later you know after subnet Das

is live will actually be a lot more harder rather than you know we're thinking that we can just tack it on but from what I see um without you can't T it on without a hard Fork I'm curious why do you think that because it kind of seems to me that as long as you have um um you know all clients at some point being able to respond to sampling queries then actually enabling the sending of sampling queries can basically be rolled out in whatever way like even you know anyone can individually start doing that um without too much of a problem um but yeah I'm curious what what's your perspective

there yeah so like um you know we start something dust you would have a certain amount of uh you know C requirement and if you want to gradually introduce P sampling you would drop that cust requirement but you know your peers around you they do not know that you have actually dropped it right there's no way to indicate that Cu uh from this is like said you know in the config parameter that uh every pair would cust these amount subnets and therefore you know you would have all these columns right and you would participate in Gossip for these subnets but if you let's say drop the cust requirement and you sample for you know these other columns the PE around you do not know that you've actually

started doing that well like what we could do for example is just to introduce peer sampling at first but um still keep the you know just as a basically where it doesn't actually do anything it's just something you do kind of as as it is now in in clients as I understand and uh then at some point you you drop the cust requirement um and yeah I guess uh right that might need yeah they might need a coordination point I guess um but is I guess is that too much of a problem um I mean U if it needs a coordination need a for fork of some sort right

so right that's fair um I mean that also I guess right that's a bit of a complication but also doesn't seem the end of the world to me just because I mean at the end of the day it's uh like I think the way I think but at least most of the reason then to introduce Pier sampling is just to possibly get a bit more scale so it seems like the kind of thing that can just wait for for a time where we we can have such a coordination Point um like it's it's not something I me at least the way I see it it's not something where um you know that we need to like um how can I say it's it's there's no like emergency where like this need to

to be added for for the system to to to be complete and work I mean otherwise we we wouldn't you know we wouldn't ship a thing that that um doesn't work by itself um so at least the way I see it it would it be than to um wait for for another for coordination point or whatever but um yeah um happy to hear other opinions yeah I mean like I guess from implementation point of view it's it makes our job a lot easier but um it is kind of like Kick the Can down the road right you know U we got to think about the next Fork if we we were to you know

to add P sampling or not or would we just be okay with the end State being you know like subnet D um I think the the end end end State no um but um it's like it can stay like that for for I think a while like I don't think there's any you know problem with uh having to wait for another Fork to to then uh add Pier sampling um like you know I think basically I guess Anar has been like trying to beam into reality this thing of um a 3X U blob counting Crees every year I don't know that might be ambitious but like I think that this

kind of thing would be in line with that like you can probably get you know two three x whatever increase just by doing this initial initial version um and then adding F sampling later is just a way to get like another uh extra kind of throughput increase basically like do we have like numbers on like how much TR put increase that would be like you know let's say if we drop the Cy count by how much we're planning to increase right now with P sampling um so like for example if we right now set the the cust requirement let's say to um I let's say 12 just to yeah split the middle between eight and

16 which are sort of I guess to the reasonable numbers that that um people might want um then more or less you download uh I mean it's basically about like it's like about a 5x um kind of efficiency increase compared to 4 a44 um and yeah so that's basically the room you have to to to increase throughput and you know we might not want to increase it by 5x but um but we do have in principle uh like up to that much room um so I yeah I think it's not um it's it's not like a a huge constraint like I don't I don't think

that I don't think there's any world in which you know we 10x throughput or like 20 x throughput in a year or or something like that I think like we all want to be cautious um regardless of what you know these measures are like um in theory um so it feels to me that if we have a 5x you know ability to in principle increase it by 5x you know maybe we can increase it by two or three X um and then still have room to to kind of um move forward in in a later fork and then I mean yeah incre add in P sampling then uh it gives even a little bit a little bit more room um I mean I can tell the exact numbers but it's yeah I guess I don't know how much it's it's super important

but yeah that's fair enough makes sense any other question discussion on the simply um yeah s just one one last thing about this I guess um I I think maybe one other way to look at it really is to just say like we don't know at this point you know what what will be I mean

we don't know what the timelines look like we don't know um what we might be able to to to have done in uh however many months uh unspecified number of months until uh the fork um so one option is that we just start with this like just take it as a you know first step of uh implementation and testing and really just try to get that done and then only then focus on peer sampling and I mean it's possible that we end up um you know feeling like oh you know it's really um doesn't take too much more work to do peer sampling and um we can possibly like you know delay the the roll out and and just do the whole thing um or that we see that actually yeah P

sampling really does need more work um why not just go with the initial thing which already gives us a duple increase like that's kind of my thinking at least that like it feels um a lot less risky like it feels like that at least by proceeding in this order like first trying to really get um distribution and reconstruction done like basically doing the subnet dust part of the whole thing and then focusing on sampling then it's much more likely that something will like they will be able to ship something and get some throughput increase and uh yeah and possibly we can get the whole thing done but just we don't have to have that uncertainty essentially thank you

Franchesco any other open discussion so um back to the call meeting time just does anyone feel strong that we should change the time frequent uh the period or the time if not then then the next week will be

in two weeks on the 20th at the same okay any other closing notes before we say bye-bye to each other no okay thank you thank you for attending have a nice day thank you bye bye thank you bye bye