Post-Quantum Interop Call #10

Transcript

How's everyone doing? Welcome in PQ interrupt. Uh we're at I think call number 10 is this week. Uh we've got a very short agenda. Um at least stated agenda. We'll see where the call goes. But uh Nepur, you're the um sole contributor to the agenda this week. you had asked uh or made a note about wanting to create some standardized dashboards for clients. Uh and let's see, is Kata on the call?

Yeah, Katya um also reached out to me about creating a a lean metrics repo which I created this morning uh got her access to and you know hopefully those efforts can can work together. uh Nupour maybe do you want to just speak to some of the things that you're hoping to get out of this? >> Uh so like uh there was a discussion on the last Zen call about uh um creation of a standard dashboard where each client expose a certain set of metrics so that uh we'll get like a consistency and a reliable way to track metrics and performance across clients.

So um like we actually wanted to continue that discussion over here with a larger set of audience. So and I'll let Gajenda take over from here. >> Yep. So uh I think uh in the last zoom call we had uh discussion that uh the metrics basically you know we not only define some standard metrics like how we discussed last time but we also have a standard dashboard so that uh the client team don't need to come up with their own dashboard and uh we have one standard I mean they they can have their own dashboards for further debugging and uh further insights into their own client but we have this standard

dashboard which basically every every client has to conform to and supply the metrics for and in that case basically you know it will be very easy to dig into how each node is behaving. Uh so that was the idea and we basically thought that uh we should just throw it out to a larger audience because we wanted to standardize metrics and in fact we should not only standardize metrics but we should also standardize the dashboard. >> Beautiful. Yeah, I have a feeling you're gonna make Katcha quite happy with this.

Katya, take it away. >> Yeah, hello everyone. So, uh yeah, sure. As soon as we have standardized metrics, uh the dashboard uh is like easy to build and uh use. So currently for uh for for example for consensus clients the standardized met uh dashboards are in the um Ethereum package repo. So there is the special folder um dashboards and there are different dashboards concentrated here. I don't know how it goes how it will go with the lin um so currently curtosis um is not ready but uh we can add the dashboard for example as soon as it's ready uh the

most important thing is the metrics themselves uh because uh to standardize them we need like um overlapping parts like for example state transition or multi etc. So this is the main part uh we should discuss probably and um yeah before we build metrics the dashboard is is not a problem it will be standardized at least uh for for the purposes of uh running devets uh we should have like the same dashboards um so there will be >> yeah I mean uh I think it's uh quite important that you know when we have the

view as same it basically enforces from top down that people follow the standardized metrics. So instead of going from bottom up approach I think if we have a top down approach to the UX it would be quite good. And with regard to that I think uh setting up the dashboards I think GM uh has an update for us. Yes. Sorry. Give me a minute. I uh I am not uh Yes, I will. Okay. Um well, I I have an update. Yes. Uh I couldn't work on it today. I was talking

Felix all day, but um we I was able to confirm that uh the intake works. Uh so I have Ze producing metrics uh VM agent grabbing those metrics and pushing them into the the ETH uh uh the E DevOps uh Rafana dashboard. Uh so now I just need to uh confirm everything like create maybe the the dashboard in question and then um I mean you guys should already be able to access the dashboard from what I understand. Whoever gave me their uh their GitHub account should have received an invite. Um but uh what we need to do is uh just make sure that uh

we can set uh set up the machines. Uh so uh I'm currently working on the on Zim one and we would also presumably uh run on that as well in Quadriium. Uh so if uh yeah if someone from each team could contact me I would I would try to make it work and this way uh also list the the metrics so that I can check the intake is working. Um yeah so that's um not quite I mean you guys like I said should be able to access the dashboards.

It's simply sorry graphana it's just the dashboards not running. Uh but yeah it's been confirmed to work. So let's uh let's try to get those uh those metrics to show up. >> Cool. Thank you. And with regard to again coming back to what the dashboard should be, I think uh will andless basically you guys can uh sort of sit together and figure out what what is the UX that you want to see with regard to how nodes are behaving and what is the information that you want to get out of the nodes in terms of uh the DevNet uh success metrics. Right? So I think that would sort of

figure out what what is the dashboard that we should be having because uh once we have that we can basically say that okay all the teams have to sort of collect these metrics and uh that's about it. >> Um yeah this could actually be a really excellent halfhour work session or something in Cambridge to be able to you know workshop. Okay, everyone give us your the top metrics and then I don't know I could sketch something out by hand. People can comment on it. We can iterate on it a few times and make sure that uh yeah, we come away from that with something that we feel like it's been socialized and has broad consensus.

I'll happily add that to the agenda. Great. Um, June, I feel like there was success criteria outdated possibly here. Okay. Oh, and the yeah, the DevNet markdown is something I've got on my list that I want to I'm planning to push a pull request today with the last several meeting worth of meeting minutes and then I'm going to do another one that's just the uh DevNet spec sheet so that we can have another round of uh all of the contributors commenting on that and get

that back up to speed. So, I'll drop a link to that into the telegram chat when that's ready and I think uh yeah we can also update the success criteria as we basically you know as we go on to the next week where we start uh integrating or we start at least try to do the interop or at least uh run the spec test uh the generated respect test. So I guess uh we will be continually updating what the success criteria is because we will be learning along as well.

>> Great. >> Um so there's nothing else on the written agenda. Um do people want to raise their hand if they have anything that they would like to discuss or drop it in the chat? KF >> yeah I just open the link and see the uh client interop readiness uh like bas basic signing attestation etc. So um these list can be like the initial list for for metrics like where we uh the clients are overlap and where we can have uh the same metrics. So I just wonder uh for example I quickly looked

into the clients and I I see on the seam on the Zim side I see like at a stations uh on the rim side I see more networking part. So um a question to the rim team do you always already have this attestation part? Uh so what could you elaborate more on the authentication part? So what kind of metrics uh you are referring to? >> Uh I mean you already have it implemented right?

>> Okay. Okay. That's okay. That's what I'm asking about. So you already have it. So yeah I will quickly look into it and yeah just to have an idea how we can standardize it on both sides. So um so I can just uh lean on this uh client interop readiness for maybe initial set of metrics. Um yeah and gon we can also discuss after it. You had mentioned at the beginning a link. Could you drop the link in the

chat that you're referencing? >> Yeah, I mean it's the same link that Jun shared. It's like um I I'll uh share it again. The client interop readiness this one. >> I see. Gotcha. >> So these parts from from the table uh could be kind of standardized if they're um overlap in the in the clients. So we can startize these parts. >> Uh Carson, welcome. How are you? >> Good. How are you guys? Well, I guess how are you?

>> Excellent. Thanks for joining us. Uh what's on your mind? >> Uh there was mention of working with the execution spec tests. So I was wondering if anyone had any pain points to raise um about that. Um, since we're in the middle of welding together the uh, execution specs and the execution spec tests, um, now is kind of a perfect opportunity if anybody has some stuff that's like, you know, it's always bothered me the way that either this works or the documentation isn't very clear about this or I always find this a pain, you know, to to work with. uh if anyone has anything they want to raise that we could like prioritize and improve during this process then just

toss it in the chat and I'll uh I'll pull it out of there. Thanks. >> Excellent. Thanks so much. Uh yeah, feel free to add it to the chat anyone. Um, and if it shows up in the Telegram chat or something, I'll make sure to mention you Carson or get you added. Um, Gender. >> Yep. So with regard to interoperiveness basically you know uh PK uh generated uh PK was able to get his tool ready which generated the files and uh on the Z side

we are able to consume it and uh start our node as well. So as such I think re guys should also have done that same because they were in the loop for that. in terms of so basically in terms of Genesis and config uh I think we we are sort of ready and uh most likely we our uh so we our local uh interop with z nodes that is also happening even though we there are few things that we are doing to make things robust and few things that we'll also need to accommodate in the spec for example we have not uh considered

clearing of justifications map in the state when basically there is a finalization and all that. So, so we need to basically uh do some housekeeping there as well because uh so that the state doesn't blow up and these are some of the changes uh that we'll be suggesting in the coming week uh on the PRs and uh we'll basically go from there but as such primarily we are ready to run spec test and we are ready to basically even uh do an interop barring few housekeeping tasks that are basically in the pipeline. So that is sort of an update uh with regard to Genesis generation as well as Zam update as well. And I guess uh I pass on to

uh June for the rem update. >> Uh yeah for sure. So I just want to provide uh the update for from the rim site. So uh yeah so we uh also consumes the outputs from the PK synsis generator and there's a uh so we already had some docker compos script that I posted on the link and it's consist of four rim node with uh n validators so uh I observed a finaliz finalization events in my local but uh also we need some housekeeping tasks and also we had we remain any core optimization tasks. So

uh I guess specific uh implementing a specification is uh done as in the rim side but we also have some kind of uh remaining tasks uh not relating to the spec but uh regarding engineering things. So uh I think yeah so we need several days uh no uh not not over 1.5 weeks but uh about 3 to 4 days to uh get it ready for uh those remaining tests. That's it.

Thanks J. Um Camille, did you have any client updates? >> Uh just the general one maybe. Yeah, we finishing with the fork choice and 3SF. Uh yeah, once ready. Yeah, we uh yeah, we don't we didn't currently test our implement. Well, we don't have support for the yeah, Genesis file format that Yeah, Jun just shared. Yeah, we'll try to uh yeah, introduce this support as soon as possible. Uh yes, and hopefully by the end of the week have 3SF and yeah, we'll be ready to start testing with other implementations as well.

Yeah. >> Great. Thank you. Um and not trying to Oh, ka. >> Yeah, I'm sharing the linetrics uh repo in the chat. So this where will be the repo where we will aggregate this the metrics. >> Thank you. >> And then anything on the specification front from I don't know Philippe. I saw you were dropping a number of PRs over the weekend.

Yeah, I can I can give an update. Um, basically what what I did over the weekend is that I um I took all of the markdown files that are currently there including the state transition function, the for choice and all of these including the basic types, SSD types and everything to have everything in Python. So that right now we we still have the markdown files inside the docs folder but we have everything translated into Python plus unit test in a separate folder that is called test. Um so that right now all of the upcoming peers

should be directly done into Python to avoid any confusion possibly wrong implementation or anything. And each time you do a PR you can just add some unit test also so that this is easier to review, easier to talk about and easier to make sure this is correct. Um and yeah with Felipe and Felipe especially currently is checking the basic types to have something really clean from the basis and start with very clean types and lean type. So yeah this is basically it.

Thanks um Marus. Yeah, for now also I Yeah, sorry just something to add. So for now I am while designing the SS type I am quite happy with the unsigned integer. They are quite smooth and well typed and well structured in Python but I am less happy with the way I designed the list and the vector stuff. So, Filipe is looking into this right now to to check how how we can do better.

>> Excellent. Thanks for the update, Tomas. Uh, Manus. >> Yeah. Uh, hi guys. I'm Manus. Um, I'm joining here uh from Lighthouse. Uh so I'm just like expressing my interest to um start you know um developing for DevNet. Um the only thing is that I'm still working on a few other stuff and then like I've started my uh you know initial boilerplate stuff. Uh I I'm not sure if I can make DevNet zero but uh I'm really interested in it. So yeah um I'll be joining the calls for now. Yeah. Thanks.

>> Excellent. Welcome. Uh >> yeah. >> Yeah. Uh I basically had a followup for Tom that when Tom when can we have basically you know first set of uh test cases so that we can basically start integrating and uh also do a CI integration of them so that we are always compliant. uh I mean even if the test cases are not full that is all right we b basically start need to start uh the process of running them and uh passing them.

>> I think that right now if you if you check inside the test folder you should have like a bunch of tests. Some are very very simple and basic just testing uh very simple stuff and others are more complex. So you can start integrating them and checking them. But again, as I mentioned in my message this weekend, I didn't ask for any review for these PRs because they were too big and too too much overload to review. So this would not be a productive stuff to to review these PR. So please if you use like the leanspec and if you check that, please double check the code and double check

the test that I didn't miss anything. special. Yeah. But you should have >> Yeah. >> Yeah. Yeah. So I will look into the test directory uh and basically see what the format is. But uh if you are aware of how the tests are in in uh you know spect beacon basically you know you have a bunch of uh SSG read and write test basically and then you basically have test for hasht roots of those objects.

Then you have test for straight transition function basically you know those kind of things test things at a basic level as well as then they test at uh straight transition level. And then we can also have test for fork choice as well. Basically give a series of blocks and then you have to basically say which is the uh which is your head which is your target and all that. So basically are we are so these are the test in the same format. I guess >> kind of I I didn't to be honest I didn't look that much into the consensus spec to to have my mind clear and make something from scratch and try to make the thing as smooth as possible and as simple as possible. But if you go like

to the linspec to the test linspec subspec for example you have types in the types I put a lot of tests of SSD in the same format as done in the remarkable library where you you can find like object and stuff that are tested exactly the same values as remarkable library and in the subspec you have also fork choice tested tests, networking test. Uh so this kind of stuff that you can try to integrate. Then if we need to organize test in a different way like you or Philip Filipe or any other should

be more aware of the best way let us say to integrate all those test properly. Don't hesitate to modify the structure to submit PRs or issues to to let me know how you prefer those test. But for now, I just constructed the most basic test case just to have something and try to check the implementation. That's all. That's all.

>> Um, any other items. All right, people are unblocked. That's good. Well, yeah, as I noted, I'll uh work to get all the the previous last couple of I think three or four calls uh meeting notes up and then I will push the uh the DevNet spec uh MD as a separate PR

so that we can start to update it and make it reflect kind of the the state of play that we're at right now. And I think other than that we should be good to go for this week. >> Excellent. >> Bye. >> Thank you all. >> Thanks a lot. Thank you. Well, byebye. Bye. Thanks. Bye.