Grandine Lean Client Development

Transcript

[music] is yours. Thank you so much. Another session by Cigar. Go for it man. >> Uh thank you Mario. So let me just start the presentation. So this pres uh project was on Grand Nine lean client for the main purpose of this project to just continue the development of the lean client and make the hive test cases pass like it was 47. This was the number of total number of cases on devet 5 that were passing when we first picked up the project. So the most number of failing cases were in the fork choice and the state transition

suites and apart from that we have some uh less cases in like verify signatures, client interrupt and request and response and also RPC compact which contributed to the failing cases and so let us see how the hive dashboard actually exercised the lean client. So there were actually three types of suites in the hive dashboard. The first of all was a single client. So what the single client cases did was that it just booted up a single node and it just ran the test like RPC compat request and response and just saw that if those cases were passing. Then next

we have the client interop where the uh environment booted up multiple nodes like uh all seven nodes together and [snorts] these multiple nodes like they tested how the clients were working together and how they performed and whether they are running a network as they supposed to. And next we have the spec switches which are actually similar to the single client test but this time we have like the client itself had some test drivers for these test to attach to and run and these were the fork choice test the state transition and the verify signature suit.

Yeah. So let us talk about the single client suits and what were the problems here and how we fix them. So first of all we have RPC compat. So the HTTP API had no route for the finalized block right. So we added the endpoints for fetching them and along with an API test for it. Then we have the request and response suite. So basically here the problem was that when the test case uh passed a invalid number of blocks for example a blocks by range with a count of zero or above the limit for what is allowed the client used to fail the call internally but because of the error it

does not it did not really return a error code but rather it returned an empty success response. So that is why the cases were actually incorrectly showing them to be passing when they are supposed to be like just return an error. So these cases which were supposed to receive an error got success messages. So these were these cases were failing because of that. And then we have the gossip suit. So there were two cases which briefly failed for every CL client at once that turned out to be a simulator bug which was fixed upstream in the hive runner itself.

Then we have the specs like the verify signatures and the straight transition. So what happened in the verify signatures was that all three cases returned HTTP 422 error. So here the problem was that in the test driver we have the fields for testing there was an mismatch in the name of the fields. So the grand lean client actually uses a survey case but the fixture field names were actually using a different format. So the solution was to just rename those cases and that see fixed all three of the cases. So after fixing these cases the renamed fields the fork uh not just

the fork the fork choice suit also rose to like 67 along with the verify signatures which got three out of three but the fork choice also rose to 67. So yeah this uh single error fixed a lot of tests. Then we have the state transition suite. So here there was a similar reason the rejection reason was being read as an exception reason in the test driver. And after fixing that we also like added cases for rejecting aggression bits and empty validator registries. So that fix the state transition suit from 13 to 100% that is 74 passing tests.

Yeah. So now about closing out the fork charge it had the most number of failing cases. So on the test driver we have skip we are skipping XMSS verify verification for mock proofs and we were honoring the tick to slot before advancing the clock and we were also reading the h has proposal boolean and signatures from the fixtures. So yeah and then on the client side like these were the test drivers till now but on the actual client logics client side logic we were rejecting unknown block and out of range votes and as for the other part the non-aggregators now verify gossip

signatures and the head votes also include latest known attestations. So next we have the another set of issues and I would like Sami to take over. Yeah. So even before we addressed the the test failures that we faced in the hive uh test suits uh the biggest issue that we found was that hive would usually build usually build on the latest tag but the latest tag on the grand lean client pointed to the DevNet 4 implementation. So essentially we were running DevNet 5 test uh test suits on

DevNet 4 implementation. Hence a lot of spec suits executed early and there were various different failures. So the process of finding this was just generally that we pulled the JSON from the hive results API and found that the grandine was building on the latest tag. Hence we raised this issue on the PQ interrupt group and raised a PR for this. So the deployment was moved to the DevNet 5 DevNet 5 branch the DevNet 5 tag. As a result uh test skyrocketed to around 240 tests passing which made grinding another joint up in client interop hive tests.

uh we uh implemented uh a CI/CD feature for all uh grinding uh PR granuline client PR that would be sent where essentially we would uh in each PR the entire Hive test suit would would run and in case there were any they would there was any component that would break or any test suits that would fail the CI/CD would revert and highlight that particular issue. So we would essentially build from the uh fetch hive from the master branch build it internally and then run all the hive tests.

All in all uh by the end only three suits were failing. When we start uh RPC compat and uh request response test suits when we uh in our initial implementation all the tests were passing but there were certain uh hive changes that were uh changes that were done to the hive repo where suddenly a couple of errors started popping up. But otherwise most errors were in the client interrupt and that was mainly because there were certain issues in the other clients while uh running the tests checking the test locks.

So then how the numbers moved throughout the fellowship was we started off at 47 then one fix moved 47 to 240 and then at wind down we reached to 305. There's a bit of a discrepancy in the number from where initially it was 321 then it by the end it became 317 but there were uh there were constant changes in the test suit itself. So few test suits were removed few test suits were added and so forth but why did we stop at 305?

Oh yeah before that hive 90% of the hive cases passed at the end and we ended at 305 simply because in clientline development was now gone down. So now post quantum work that exist existed only in client specifically would now move towards the existing mainet clients. Yeah. Thank you. comments, >> any hate, any trolls, anything goes. No comments, no questions. Yeah. Okay. Sal, go ahead. Yeah, of course.

>> So, I wanted to ask, did did you try to use AI to to solve the bugs? And if if yes, uh was how successful it was? >> Okay. So yeah, I have been trying to use AI not just for this project but my everyday task that I do on related to coding and it was pretty good at figuring out what the causes were but mostly I just when I couldn't understand why there the some issues were happening I used to ask AI but the fixes that were done were mostly by hand manually because EPF is like also a great learning opportunity so

rather and using AI, I would I like to do the actual things myself manually. >> Just one thing to add was what AI helped us quite immensely was to categorize the issues that we were facing. So like if it was something as as an extremely straightforward error, we would get it flag but if something complicated it would flag that something complicated that you need to inspect more into this and then accordingly we varied our approaches throughout. >> I see. So it's more like you used mainly AI to explain stuff uh yeah to get your work easier. And I'm just wondering is is there still lean dev nets uh running and did you try

to participate in that from the last update that I remember this was back in September there there was some changes I guess at that time done to the DevNet V5 lean VM site where uh I don't remember the maintainer name but essentially uh a DevNet 5 run was based on that particular updated implementation and not the PRs that we raised against. So there was some discrepancies there. >> Okay. Thanks.

>> Okay. Yes. Thanks. I very much appreciate the responsible usage of AI guys. That's it's uh very honorable. It's awesome. Um I think great approach there and yeah again thank you so much for all the great work and for the lovely presentation and for the um I would call it mandate compliance slides. >> [music]