Post-Quantum Interop Call #22

Transcript

Hello. everyone.

>> Hi, Wood. >> All right. Um, yeah, I think we're ready to get started. Um, let's kick off with some client updates. Uh, Zam Team. >> Hi, Will. Hi, everyone. Yeah. So we have been working on uh uh on the parent uh syncing block by root syncing PR and uh now it works in our client and uh we have been fixing other issues with PR connections and disconnections and uh we have also been working on uh implementing a DevNet 2 spec and uh most

likely by Friday we'll be there and we also merged in a PR related to checkpoint sync because uh from the dev net it seems that uh we need the checkpoint syncing infra as well as we need the the checkpoint sync so that the when the clients go down they can come back up easily u and I think there is a checkpoint sync PR that parta has put up uh and it has been merged I believe and there was another spec PR that Kai did to uh to compress the justifications uh because we don't need to track the justifications vector in the state

before the finalized and I have to review that PR but I think Toma has already reviewed it. So uh uh if everything looks good that should be good to merge and I think this PR could be included in the DevNet 2 spec. Uh apart from that, I think we should be able to freeze the current DevNet to spec uh with including uh including uh this justifications uh state compression and uh uh uh so basically uh blocked by root syncing uh of the clients uh and uh and basically

the lean VM aggregation And on that front I I believe one PR regarding certilization decentralization uh has been merged and it was raised by Unchel uh from a side. So yeah this is what we have been up to. >> Excellent. Thanks very much Kender. Uh share dream team >> sweet. Uh so few minor things we uh we added some improvements to the lean road map. Uh so some lean sig uh lean sig specs were added as well as some of the uh benchmarks for leanig were added. uh

and uh Karta Karta uncovered some issues with storage space handling uh in re uh so uh we implemented pruning of our database to the past 128 states. Uh we added better logging for DevNet one uh which includes uh which includes like we uh we log our database uh storage stats as well as uh stat uh just like current head, current slot and all of that stuff for the DevNet. uh and and then we added uh metrics that were added to lean metrics like these are node info metrics such as what's the current uh current uh slot for the uh client and u uh like what time the client started and uh so

on. Uh as for 2 uh we have implemented the checkpoint sync API that was added to the spec yesterday and uh there's an open PR for that which I believe should be merged in uh today. Uh as for the aggregated addestation stuff, we have been following the spec. Uh I believe we're like one to day two days out to being on top of the spec and any other changes in the spec, we'll add accordingly. Uh on on the P2P side, we have added the ability to redial uh peers every 20 seconds. Uh and I I believe I believe that that's all from our end.

>> Thanks Sherik. Uh clean. >> Uh yeah. Hey everyone. Uh from our side, yeah, what was merged is yeah, we finally merged the enhanced better logs and uh connected pierce metric. Um yeah, also during our work on 2, uh yeah, we needed kind of we also needed the serialization in multisc u we actually also prepared the PR for that. It was merged and then reverted. I don't remember exact reasons, but anyway, yeah, the Zen team created the new PR, so we're good on this side. Um, yes, for what is currently in progress, yeah, we do have P request open for the

out of memory issue fix. Actually, the only thing that is missing there is that, uh, yeah, in this PR when the node restarts, uh, it crashes with some sync issue. But maybe if we going to introduce checkpoint sync um yeah we can just u yeah merge a shazis and then introduce checkpoint fix. So this is not an issue. Um yes then me and rouslan we were heads down for the past few days on the draft for the committee aggregation. Um yeah I was working on the spec side and he was trying to draft some super early version in clean so that we can make sure that the spec makes sense. Um yeah we're also working on introducing

some predicates to gossip in uh our libp library. Uh this is needed because uh yeah what we currently don't have in our spec and what we I think we'll need at to introduce at some point is kind of the predicates for gossip subtopics. basically the rules when we need to accept the message when we need to ignore them. Yeah, we do have this in beacon chain spec and we missing this currently and our BP implementation also was not able to kind of work with this predicates. Uh RP is good with this I think. So yeah uh yeah overall um yeah we need to catch up I think for the most recent spec uh but yeah we do have the main features for the devet 2 ready u so the aggregation should work um so yeah I

don't think it should take a long time to kind of be up to date with spec and start interropping with others u and start testing inter thanks yeah that's all thanks uh lantern >> yep hi everyone So basically added support for deva 2 in lantern. Um so basically all the tests in lantern are passing as it can consume the fixtures generated by the leanspec. Uh this was completed late last week. Uh since then I've mostly been testing the implementation uh by running a local setup for devet 2. Uh mainly just consisting of um multiple lantern nodes.

Um basically like during the testing I did notice some bugs in the quick protocol. Uh basically like after some runtime uh the nodes were facing uh handshake timeout errors. Uh but those have been fixed early this week. Um I've also been testing the syncing functionality via request uh in the local setup and I've also implemented fixes for it. Um so at the moment lantern uh only supports the latest protocol ID that is defined in the uh lean spec for request. Um and like for devet one it uh it supported some legacy protocol ids but for denet 2 I've uh

removed support for this. So any legacy protocol ids are not supported in lantern. Um in terms of next steps um I need to add uh support for uh aggregation metrics which are going to be relevant for DevNet 2. Uh and then I'll be releasing a new docker image for lantern which can be tested with other clients for interrupt. Um so yeah that's an update. Thanks. >> Thanks Mayor. Um I think Lighthouse typically goes now. I' not seen pop on the call of mouse. Anyone else from Lighthouse on the call to speak to client updates?

>> Uh if not let's jump to Grand Dean Sas. >> Hello. So yeah, so we just open sourced the client and uh two things that that we are working on. Oh, the missing things for the DevNet one interrupt and also some progress on Devet 2 uh implementation. That's all. >> Thanks L. Congrats on that. Um Keith Lambda.

>> Yeah. Um this last week we finished work on the on the gossip layer. We the nodes now receives and decodes both blocks and the stations and we tested this with lining quick start and sim for interoperability. We also added the state transition function and tested with linspec to ensure it was um spec compliant. We also added the naive LMD ghost implementation and we're now working on testing on integration.

>> Excellent. Thanks, Joan. Um, and then I just dropped a link. There's a a new client team, uh, GEN, a go based implementation. I don't know if anyone from the GEN team is here. This was just brought to my attention early this morning. Cool. Maybe next week. Um, we'll hear from them. Let's move on to spec and research updates.

Tamar. >> Yeah, sure. I can I can start. So this uh past week I have worked a lot on the on the lin um in order to try to transform what we currently have as a spec to like a minimal running client. So what I have done is that I haven't touched um any files of the spec. So not the state, not the block stuff, not the subspec, not the cryptography, but I have added some peer-to-peer stuff.

Um, some syncing stuff, some Genesis loader stuff. Um, yeah, trying to do that without any dependency. For example, I am not using lip P2P to to do that. So then I need to plug all of the things together. But I have tried to bring the different pieces together and had some tests uh so that afterwards if we need to experiment with some new peer-to-peer protocols or new stuff we don't need to modify the P2P as a core we can directly um experiment within the repo directly um so that this is something that I have

done I have also reviewed some PR um we have like an upcoming PR PR from O we discover that um the key generation is very long and we can find some optimization without degrading the the spec. So for example we have um this spec this PR that has been made public by O this morning about using numpy for um postadon permutation uh generating a huge speed up without degradating like the spec. So this will be something that will be merged. Also we have this spec uh this uh PR about committee aggregation from Camille

that's uh is currently in draft but should be ready to review and mer soon. Um and yeah we have the beer from the zim team also mentioned by gajinder that will be mer soon also. So we have a lot of uh work done about um this PPR stuff and all of the spec in the coming in the coming weeks and dates I will try to work more on the minimal client. So I have removed some part of the code that are unused and other stuff. I have tried to simplify it um a bit sometimes to review my code and other stuff. I have set up also some coding agent for those

who use cloud and I have put this uh P agent directly into the repo. So that for example we have like specific sub agent responsible for documentation writing or checking the Python code to verify that it is compliant regarding the latest uh Python stuff. So modern Python tooling and stuff. We have also like a code tester sub agent in order to verify the tests uh and check that no edge cases are forgotten or this kind of stuff. So yeah this is a bit of an AI setup so that for those who use cloud this can be easier to to interact with

the repo. Um so does it basically all um then we merge on the research side of things. uh we merged couple of PRs on blunky 3 on we uh to clean up a bit uh things and to al also accelerate the transposition of matrices but that is a research side so we are moving forward also on this cryptography um and I think that Emil can talk about that but Emil worked on a lot on the simplified branch of Lindm yeah Emil if you want to talk about this more. Yeah, feel free to do so.

>> Yeah, thank you. Um, thank you indeed. This week I'm quite happy because we finally have end to end recursion in the lin in the simple in the simple VM branch and currently the performance is about 1.3 seconds for 2 to1 recursion but as always um for this kind of first version uh I expect some big performance improvements to to come but that's pretty nice because it like everything works as expected. That's it for me.

Wow. Awesome update, guys. Um, oh, I wanted to call attention to >> the three. Sorry, go ahead. Uh yeah, ju just to close this parenthesis about like research and and lspec update. Um I have set up like the AI stuff for those who use clothes. Don't hesitate also like to modify my setups and like uh I have made it public to the repo so that everyone can use it and everyone can modify it or add new stuff. So please you have like the cloud MD and the dot cloud uh folders with sub

aents don't hesitate to modify anything at your convenience to be like more peer-to-peer performant or anything so feel free to modify this is not mine this is the one of all of us really cool effort to um in the world of specs I think that these highle documents are really important uh o thank you very much for championing this saw a lot of discussion I don't know if there's anything that you wanted to speak to with regards to uh this document

>> um not not really I guess um so so yeah so maybe over so definite 3 I basically summarized that from mainly discussion between Gajinder and and Camille. Um it's it's a very high level. So I I try to keep things um like not too much into the logic but mostly on more of the conventions like you know the the like new fields what's the field names what's the the gossip subtopics but without going into the implementation um there's one part that I'm not so sure about is the slot intervals um I com basically combined made that table up um and separated between like proposer tester and aggregator row um but for now like the proposer and the tester are are

the same um the the same entity while the aggregator would be separate indefinite tree. Um but this the interval in there I'm not too sure if that's actually accurate or optimized. So maybe if someone is more has more expertise on that um yeah reviewing that would be great. But apart from that I guess I I'll I'll mostly be just like monitoring what everyone is doing including um Katyas and and Partha's work as well. I'm trying to incorporate all the um setups from the devets uh devet one back into the document as well. So that's being reflected back into pqet 2. Um yeah, hopefully after we we start pinning down the the the

comments from each client um the Docker images, I I'll I'll update that into into this markdown uh but these markdown documents as well. Oh, could you repeat uh uh what your uh doubt was regarding >> Oh, so I think there's the well there's the part where I laid out the slot interval um from zero to three. So four four slot sorry four intervals in the slot and then each of the the separated row would be doing different stuff right now.

So like a quick review of of that is fine. Uh but otherwise like we can also like go into the implementation and once like we merge the PRs and lean spec I'm I'm happy to reflect that those things those PRs back into into this like highle doc as well. So it works both way. >> This is devet two or devet three. >> Uh that's three three. >> Oh okay. All right. Uh I think I don't know if you can see my screen but I think this table >> yeah I I think the yeah this table is basically like the the most like non-obvious stuff um in the entire

document. >> Got it. I will also look into it and uh if there is anything to add I'll raise the thing raise the PR. Uh but I think we should uh let Caramel first present uh the definite spec. >> Yeah, I think I I I think this table uh yeah was slightly based on my proposal. Um yeah, basically the idea was that yeah, we want to have um aggregators to be well some some validators they uh can opt in to become an aggregators. Um and then there was a question like when they

should aggregate and the idea was that during interval 2 uh if aggregators kind of have like more than certain percent of signatures from committee then they can aggregate. uh if not they kind of wait until they have enough signatures or uh if there is already interval if yeah we kind of get to interval three already then they kind of aggregate with signatures that they have so that they can contribute at least something uh yeah anyway that's just some verbal explanation but yeah we can discuss more and yeah if needed create more PRs into the Great. Um

any other comments on DevNet 3 planning? >> I think uh this is good. we can basically you know take it proposal by proposal of you know what Camille is proposing in the PRs and uh then accordingly update this particular table. >> Yeah, please note there is another PR u that slightly modifies the plan. It's basically just introduces a compute subnet ID function instead of just assigning validators to subnets in validators config. I think this is closer to what we have in beacon chain and in future it kind of would be easier to uh well currently we just can have a naive implementation so that we just do

modular val ID modular number of uh subnets to kind of calculate the uh their subnet ID but in future yeah it could be random kind of the same way as we do this in beacon chain so yeah >> and that PR is in lane spec repo. >> Uh this is the PR to the PM repo. PR 15. >> Yeah, I sent it in the Yeah, reply. >> Thank you.

>> Uh great. So I guess looking back for a second, uh Katya, I saw that you published this uh more extensive BNET one retro. Is there anything that you wanted to voice over with that? Uh so yeah hello everyone uh quick updates uh we are we are working on the devet one syncing so testing syncing um shut down and up clients and u so I raised an issue for uh devet 2 readiness uh regarding link quick start um so uh why why the chat is not available for

me Oh yeah, here it is. So uh so feel free to add your images there. So recommended is definit signatures PR. So I had to change um um the the prefix um Anna would please take a look and there will be some changes on the first set of PQ metrics just the naming uh not implementation but it gives more flexibility for naming

um so regarding the shared dashboards not ready yet will be will be shared soon and uh yeah we also have new uh client and uh if you have any questions how how to integrate into um into the link quick start. So please tag me in the chat. So I just shared some examples how to do this. If you're stuck with this uh please tag me. Um so retrospective u so is in a draft mode. Uh I had some new suggestions what what I can add there and will update it. Uh so it's still in progress. Um maybe I will update it with

the sync and uh it will be published in link mix start um repo. So we are going to aggregate them there. Uh so there the there is a proposal to u add some um additional metadata to docker docker images so I can easily um identify which commit uh which is the latest commit for these images. So this is important for testing. Um yeah I will I will raise the issue for for this and sharing in the chat.

Um that's it from my side. Thank you. And yeah, um you just mentioned something in the chat that worth discussing like this is the PR from Manass I think if I remember correctly the but about dynamic validator set. Um you raised the comment in this P that saying that for now maybe this is not uh worth uh worth doing this. Um but I I don't know like uh here this is just maybe some toolings added uh to the

repo. So maybe this worth just uh putting the tooling in place so that we already have this concept of like a validator exit exit request and and all of this. Um I don't know what is your opinion guys about implementing this in the clients um and stuff if if this will too much complexity for you if this is manageable if this is something that we don't care for now so yeah this is worth discussing because the PR if is open just very minor change to be merged in the spec so worth having your your opinion on Yeah. Um just to to quickly add um on

Toma um so that PR or my my assumption is it's trying to do yeah basically what you said like deficits and exits. Um but I think it's only the interface right now. Um so if we are going to do this this means like our valid data set is going to fluctuate and that means our um four choice and the yeah basically the two two3 calculation is going to going to going to be dynamic as well. So I I kind of sense personally that it it is quite complex. Um but yeah I I don't have a I yeah it's it's just a personal concern but I'm not so sure maybe other things that's as useful and then then

that's fine. Yeah, I sort of agree with O uh basically I mean as such uh we can implement this but uh the question is that uh how urgent or what is the use of this at this particular stage uh maybe you know when we basically are trying to formalize the spec for uh lean hard fork I think at that point uh it will definitely become useful but uh currently I'm not so uh my personal concerns about this so um so the devet is is quite unstable and um especially if the new clients arrive so

so we we can't reach a long long running devet uh because we lose too many clients uh so uh every time we need to restart but it's still doable so it really depends on the client side so how easy is it to do this? >> Uh for that basically what we can do is basically we can uh make lean quick start a little bit more flexible so that if there is a set of node that you want to run to change over to some particular client so that you know you have the definite stability if for example some client is not ready or has fallen off or uh whatever issue I think we can handle it through that.

>> Yeah. we already have it. Uh I mean when when it's the devet is already running. So if if we lose like one client it's still possible if we lose two clients which happens often then we we need to restart. So we lose finality and we need to restart. >> So you can even when the devet is running you can basically assign a particular node to some other client. I mean that is not really difficult. uh basic uh so so you can run multiple instances of the same client in two or more nodes. Uh this is for example how we use lean quick start to do zamto interop as well. So uh that is possible.

Uh so what uh so what I'm saying is that for example if some machine you want to shift to some other client rather than saying that okay all these validators will go to this particular instance of this client at this particular machine. So we can just say that okay you know we can switch out the client on this particular machine uh as an in as a second instant of instance of some other client that you want to assign it to. Uh so yeah with the checkpoint syncer and all the syncing ps that we have as I mentioned that all this switching would be easier uh we just probably need to make it a bit easier on the lean quickstar interface. Uh if that is a problem that we are trying to solve then

basically dynamic validator set. So there is an easier solution to having a dynamic validator set. uh if we want to do something with dynamic valid data set and show something on top of it then that is a different case. >> So um do we still have the um uh the set of clients like on the when we start the devet from from the very beginning even if we have checkpoints so we still need to configure it uh before we run devets.

So >> I mean the number of clients >> basically I mean it in lean quick start it's basically saying that ream zero or qin zero is for example now zam 1. So if you do that and you run rerun your envelope basically it should shift the client uh it should shift that particular node and the associated validators to a new client uh automatically and uh if we once we have the checkpoint sync and parent syncing working in all the clients basically this switching should just work out just like that. So with a small change in the configuration, you should be able to redeploy a different client to the same machine with the validators of that particular machine assigned to the new

client. >> Okay, I will try this. Uh but I think it it still u won't fix the issue when we lose uh like more client. But yeah, I can check it. Thank you. >> Why wouldn't it fix the issue? uh because the number of um the number of validators is set um at the very beginning and it it isn't changed. So, so >> once you shift >> you mean you mean on the go, right? You you mean to change the clients on the go?

>> So, yes. So, if you can shift the validators to a different client, so that solves your problem, right? This is what you want to solve. I got it. >> Mhm. >> Cool. So if that is not working out, just ask PA to get it working because it should be working as such in my opinion. But if it's not working, we can fix it. >> Okay. Good. Thank you. >> Okay. So just like um to summarize like about the the PR about dynamic validator sets. So like I am PR centrics but now that the the PR is done as proposed by

will in the chat I think that it could make sense not to do that in DevNet 3 but to report that to DevNet 4. Um and but so I think that as the PR is done right now and this is just the tooling we can merge the PR without apply without applying that to the spec so that this will have no effect for you guys so that we have the tooling in place and this will make our job easier for DevNet 4. What do you think about this about just merging the tooling right now? So just the class uh the structure the fields uh and and all of this but not touching for example like

the fork choice and all of this. >> So in that regard I have a question why but what is the haste to merge it? I mean the PR can stay as it and we can consider it when we actually want to have it in the spec. Um yes, but as the spec evolves very fast, I am a bit worried on the fact that this will in like one month we will have a ton of conflicts on this PR and so this will be very difficult to rebase. because that is the entire reason that if there are for example going to be tons of conflicts that means that a person who has to do work on some of the PR will continuously have to worry about this

particular set of changes uh and is basically an added tech debt for something that we are already not doing. So that is sort of my logic to say that okay you know if there are any conflicts they can be merged at a later point in stage. Okay, no problem. This is you guys that are implementing the client. So yeah, if this is okay for you, yeah, then okay for me. I think uh with regards to the dynamic val data set, there can certainly be some complexity around like activation exit time and like consider like figuring out the time parameters for it.

Um I would say like um by the time we reach devet three and four or even devet 2 um given that the clients will be implementing syncing via request rest uh like we should and checkpoint sync as well we should expect uh the dev to become a lot more stable uh to allow like uh in a longunning devet to allow testing of advanced features like you know activation exits um which are heavily time based Um >> yeah so I sort of agree and uh also to plan when we should actually have it uh just to say that it's definite four or

five or six. So basically what what we want is that I think uh we all want have want to have a view of what will be the end game of uh uh of lean Ethereum. uh so which which is basically you know when we are going to propose it as hard fork and how it's going to look and uh that is the basic point that is the point basically so that timeline will decide that okay this is the relevant point at which we need to start having all these features from the beacon spec into linthereum so that we are at the parity uh so basically we are building on top of what consensus spec is uh so

question is what what is that time uh and what is that stage when we'll be at we'll basically say that okay you know we want to have all these uh beacon features which include uh data availability sampling as well as uh execution and I think basically all those things all these questions will uh sort of become relevant at that particular point so so so I would like to think uh think that I would I would like to figure out that what is that point at which basically we'll say that these things have become relevant and we we want them and we need them.

Would it be uh poss possible and or helpful if we were to add readiness checklists to each of the highle docs so that week after week we are going I mean we're obviously going to be shipping parallel or multiple dev nets at a time um you know make sure that we're always in agreement in terms of like okay what are the blockers to devet to in terms of uh you know what's included there and what can be deferred, what's in DevNet 3, and even start to write up the DevNet 4 one now as we identify things that, you know, we want to push until later. Um, I

guess I just want to make sure that we're, you know, always driving these efforts to completion and, um, in agreement in terms of like, you know, when when a feature set or functionality slips from one DevNet to the next that we have a a coherent like shared source of truth for that. Great. Um, is there someone that would volunteer to take a pass at adding the readiness checklist to DevNet 2 and DevNet 3 for next week so that we can kind of use

that as a guiding part of the outlet outline. Mercy, thank you very much. All right. Um, and then Kata Arth, you know, I kind of keep one eye on the the DevNet channel. Um it seems like you're getting all the support that you need for additional servers for new clients as they uh complete in you know uh and are able to interact with the the DevNet. Um

yeah, sounds like you've got what you need there. No blockers in terms of services and uh and support. Okay. Um any other discuss open discussion items? I think um that's kind of the uh the gist of of the set program. All right. Um

another great call. looking forward to next week. We'll uh yeah, we'll have a chance to go through those readiness checklists for DevNet 2 and DevNet 3. Um and yeah, start start scoping DevNet 4 hopefully. All right, thank you all. >> Thank you again. Bye. Bye. Thank you.