Ream: BlackBox Interop Testing Post Quantam Lean Ethereum

Transcript

[music] >> Anyway, cool. We get Mohit. Man, it's yours. Go for it. >> Thank you so much. So, hey everyone. I'm Mohit and my EPF project was about extending Hive's test for lean clients. So, initially I started with validation and gossip, but then later I moved into the sign-up part. So, I'll walk you through the gaps I found out, why the existing test missed them, and how I turned those findings into useful tests.

So, this was one of the results from running a reem sign-up. So, this one uses XMSS. So, it's a post-quantum alternative to BLS. So, in this one validated key contains many one-time keys assigned by slot. So, the same one-time key cannot be used to sign different messages. So, here the key was active, but the tree material for it was not prepared for that slot. So, what happened was an RPC worker panicked and the connection closed. So, then the next valid request, but still signed. So, I want you to remember this image because today we're going to deconstruct this. So, notice here that being active doesn't

mean that it is ready to sign. So, we'll come back to that later. So, let's go 4 months back in time and my original idea was to send two conflicting blocks from the same proposal at the same slot and I wanted like I expected the receiver to reject the second. But, since both signatures can be verified and the receiver validator can store competing branches. So, this is not the invariant I wanted to test. So, the invariant is exactly one-time key assigned to slot shouldn't sign two different block roots. So, the test was not going to do it. So, I moved

the test to re-sign up, which is request signatures for two different block roots at the same slot, and then check what it actually produces. So, but I encountered a problem. Though, current architecture though hybrid had the infrastructure for what I needed, because the signing path by clients was not exposed. So, that was my next job. I traced the proposal path and extracted one proposal signer, which is shared by normal block production and the hive test driver. So, the designation of duties is that hive controls the root, validator, and slot. Those three things.

And this route is available like in only the test driver mode. And this lets us observe that just the production signing boundary. Because earlier what I had to do was I had to handle the block construction part, and after that the aggregate proof setup, which is like combining different XMSS signatures. But since I extracted that, so I could use the focus on the proposal signer and do testing with it. So, after that, I did the one of the experiments was that I was sending block root A, then B, then A. Remember, A, B, then A. And all with the same key and slot. So, but you'll be surprised that both A and B produced valid signatures. And

requesting A again returned the same signature as before. So, basically what happened is like signer allowed two different messages at the same signing position. But could we call that a bug? I did some digging and I checked that the original paper security model, it assumes that position won't be reused. And the lean sig, the library that is made on that paper, it makes it scholar responsible for preventing reuse. So, like we couldn't effectively establish that this test endpoint promised to enforce it.

And one verification there, like how do we know that those were the signatures we asked for? The client returning a public key with a signature isn't enough. A signature could verify under that key and still be the wrong validator's result. Maybe an internal bug that routes the validator to another one. So, our expected key came from the configured registry in the lean sig, and the root and the slot came from the request. So, I ran a separately pinned light Python lean sig verifier and checked that the response against that context. So, then I changed the root, the slot, the key, and the signature separately. I just basically did stress testing by variancing different things, and it it

uh passed every of these tests. So, now back to the active key thing I talked about in slide two. I had initially grouped three distinct things together. So, I want you to distinctify them. First is activation defines the slots where the key may sign. Preparation defines which slots have the tree material ready. And finally, signing history records what the key has already signed. And like, preparing more slots doesn't itself like prevent signing to different roots at the same slot.

So, in our fixture, activation ended at 262144. But preparation ended at 131072 open bracket. And so, the slot 131072 was active, but its tree material wasn't ready. And in this case, the particular fixture mattered, too. The earlier cached default keys in the hive had already matching activation and preparation ranges. So, there wasn't any selective testing done on the edges. So, the earlier cached keys couldn't reproduce this gap, so I made a new fixture for that. So, we check the actual intervals before running the test.

And like, then another experiment. So, with that distinction, we can read the sequence now. At 131070, we got a signature and verified it. So, that was the first part. Then at 131072, the key was active but unprepared. So, what happened is Reem reads the library assertion, panicked the worker, and closed the connection. That's what happened in slide two. And then 131072, again it signed and verified. So, this looks completely normal like it closed the connection and it it it eventually didn't die at the process. But, remember that like here 131071 is the belonging in the overlap of old and next prepared windows.

But, here is that the separated request is contained from continued signing. So, the process had survived, but it hadn't handled that failed request cleanly. The fix takes the prepared intervals before entering the library. So, the fix was essentially I had done a check before it, and the test allows the valid signature or a structured refusal at the boundary because automatic preparation wasn't an established endpoint promise. So, I was a bit early in implementing that. And with the locally fixed executable, both high sequences passed that, and we also ended a regression test through the production block building task.

And that somehow led me to check how LeanSpeak actually advances preparation. So, there's the two functions. There's a primitive, which is a low-level function. It already had a test showing that when it reaches the key reaches its limit, it returns the key unchanged. But, uh the validator service, which was the next test, it hadn't also, but they used a mock in which they always made progress. So, the gap appeared when I combined these tests together. For an unreachable route, which is outside the lifetime, the service kept asking the function to advance, and the key stayed unchanged. So, the loop has no way to stop, and it go somehow it could affect that. So, I added test

using the actual service and really small test keys. We limited the number of advancement attempts, so the original code would fail the test instead of hanging it. And this is like the simplified service loop which I used with the missing check. After each advancement, this service checks like whether the prepared window moved forward, and if it didn't, it returns a proper error instead of looping forever. So, it also rejects requests outside the keys activation range or behind the current prepared windows says advancing can't like magically make those slots available.

The low-level function stays unchanged, and the service can now handle its limits. So, after this, I also checked that handling a bad request didn't make a valid work afterwards. So, in Hive, we requested an inactive slot and a slot too large to represent. So, that was again stress testing. We'll just send bad requests and send good requests afterwards and check it stays alive. And I figured out like after each rejection, like I I requested a valid signature and verified it, and because ultimately what could also happen is client would reject everything after a bad request. So, that is not actually a good case. So, and in the lean spec part, I tested both

proposal and attestation keys. So, like where a failed preparation request had to leave the stored keys unchanged and still allow signing at an eligible slot. So, we also checked the preparation could advance to a reachable future slot, and it didn't just stop at all advancement. So, with this we I added eight new test cases, and all of them again passed. So, coming back to the failure we started from, uh where Reem could produce valid signatures and still mishandle an unprepared request. So, we needed to test the real signing path, verify against expected validator, root and slot, and check that valid work was still succeeded after a failure.

For a second client, I'd first check that a verifier matched its signature parameters if I ever implement this again. I'll check the parameters, hash functions, and the encoding because like otherwise a mismatch could look like a client bug. And that's what I learned in these 5 months, that you have to correctly identify what's a bug and what's a mistake in your actual test making. So, and I'll follow my mentors' advice, which is like you just observe a failure on a devnet, uh reproduce it in a smaller setup, and then turn into a hive test that catches if it happens again.

So, that's all from my side. Thanks to my mentors, thanks to Mario, and thanks to all the EPF fellows. Thank you. >> Thank you. Wow, thank you, Mohit, for bringing an incredible energy towards the end of the sessions here today. Awesome. Thank you so much, man. fire. Love it. Uh guys, any questions, any thoughts, comments hate trolls love anything? Anyone? Come on, one last question in this session today for Mohit. Anyway, how do you like Mumbai, bro? >> I love it. And I love the whole arrangement. I love the Devcon. I love the EPF. Like it's just too fire, man.

>> Awesome. So, thank you so much, Mohit, for let's give it one more time for him, please. Mohit, thanks for that. Thanks for closing the session today. And yeah, I will have just last couple of words before we wrap up here today and have for the lovely dinner. So, yeah, I hope you are enjoying Devcon. I hope you enjoy the rest of the conference here. We still have What is it? Two, three more days to go? Two more days, yeah. A lot of talks to see, a lot of people to meet, a lot of sponsor booths to steal merch from. Yeah, a lot of stuff. Uh but before we go, I just want to say one more time a huge thank you for all of you who contributed in this cohort.

Each and every one of you did an incredible job. That's why you're here. That's why uh we brought you all the way to the EPFD in Mumbai and I really do appreciate every single one of you guys. Um, it's very much possible it is going to be the last EPF cohort at least for foreseeable future. Uh, I don't know about that yet, but if it was the last one for some future, um, I'm really honored it has been with all of you guys. So, yeah, thank you so much for all the work. And, um, thank you so much for yeah, just like all the presentations today. It was really good like to see everything that's been done. Like it always seems kind of chaotic online, but then when it comes together like this, uh, it really felt so professional, so

well done. Like the presentations really incredible, guys. Uh, uh, each of you. And um yeah, tonight we have a we have a crazy 14-course dinner guys. Uh, it's going to be fancy. It's going to be nice. If you have a shirt, bring it. If you don't, it doesn't matter. Uh, I'm bringing a t-shirts and a shirt shirt hoodie and a says anarchy and a it's like long long socks and a and a short so I'm the organizer. I can I can do it. But you know, it's important how we how we uh look like a group, right? So, the more people from the group look fancy like this, it's going to be great. Um, so yeah.

I'll be there ideally around 6:30, but by latest by 7:00. I'm saying 6:30 so you arrive by 7:00. That's my hack there. So, please, uh, make it because it's a set dinner with the whole experience. So, I'm looking forward to experience it with you and we will hang out there and drink more. Thank you so much, everyone. Yeah. Thank you. I just want to add one last thing that a fee, uh, the cohort finally ends today, but the work will continue and so will be the friendships. Thank you all. Um, actually normally we used to close EPFD with a panel, but I'm too lazy for that this time. I'm sorry. I don't have Josh, so you know, it's my excuse for everything. Like, yeah. And one more time actually if he's watching or you

know, if it's in the recording like freaking Josh, like he was the he was the man and I I'm looking forward to see you on many conferences in the future. I hope you stay in the ecosystem. I hope you >> [music]