SSZ Engine API Call #4

Transcript

Tamaghna Choudhuri: Hi guys, why it's less 15 today, and it's…

Dustin Brody: Let's see, is there a… I just didn't check if there was an agenda, is there… but also, I don't see a lot of… The usual people here.

Tamaghna Choudhuri: Yeah, I got you. Like, the basic agenda would be basically, like, getting the PR merge and the reviews in, and basically, like, the… like, the idea is basically to get this PR merge, and then maybe a follow-up PR for the progressive types. Like, that's just the basic thing, and maybe the HTTP2 or 1.1 discussion, but, like, I don't see the people who generally discuss That's here today. That's a problem.

Dustin Brody: Right, right. Neither Geth nor Nethermind is represented, I think. I,

Tamaghna Choudhuri: Yeah, like, Morris is basically out of office, I had texted him, and no one from Nethermind, so, like, the thing is, like, would we, like, even want to keep the call, or, like. would we do it maybe next week with, me DMing everybody just to send somebody, because, like, it's the end parts of this call, so, like, I basically just want everyone to have an opinion, and then maybe we can get the PR in March, so, like, that's just the basic idea of what remains.

Dustin Brody: Right. Yeah, I, I don't think that with the people… oh, okay, well, there's another mine. I… I… I don't think… well, actually, it's a good one. So, that… I agree that it… the question is, does that help achieve this goal of… of getting, PR, merged? I… I understand also that Barnabas is kind of pushing on this, and I… I would like… it would be great to get a merged, all sequel. There's all… it is also so clearly kind of… how to put this? It's a draft that Marius, self-admittedly, like, hacked together with an LLM, prompted, whatever, probably more or less one-shot, and, I mean, to be fair, his… the dueling PR from Barnabas was the same, in that regard, and… and has not been touched very much. There's been a lot of feedback, to it in various forms, but the actual core PR has been…

not actually updated in response to some of this, so I'd be a little concerned about just, like, you know. push it in ASAP in that… in that sense.

Tamaghna Choudhuri: Yeah, that makes sense. Like, the basic idea would be, like, to ask every client team to have a look, but, like, because… Most of them somebody has, but, like, because ears off or something, like. Also, the progressive types remain, but, like, what I basically want to do is at least get, let's say, another new branch on execution API known as SSZ, merge, more years SPR in, then put another PR for the progressive types, get that merge, and maybe then have one final PR from that branch to the master browser, like, that's the basic idea, like, what I have for now. late.

Dustin Brody: Okay, I mean, I guess my question then would be, procedurally, in terms of how baked in would this be, and… right, I mean, I don't know if you were there, like, yesterday's… I mean. ACDECO, but in the… for example, in the context of the 8141, in terms of people wanted to SFI something, right? And my understanding is it ended up SFI'd. But the reason was, and the caveat was, and I promise this is directly salient here, is, like, there was a lot of uncertainty because, you know, the 8130, 8141, all that stuff. Fine. So people just wanted it in. And then figure out the details later. But the trade-off was, well, the thing being that there was an agreement among all, sort of publicly agreed to, that the thing being merged was not necessarily,

precisely what was going to be, gone with, in the end. Like, a couple of people mentioned that what they did not want as an outcome is it gets merged. like, and this is there, but I would say the same thing applies here, based on how you're framing this, as it were, no pun intended. Is that… that, okay, well, there was a lot of uncertainty, there was a lot of, like, open threads, but… but there was some pressure to merge, so we merged. And then… now, oops, it's a spec, you can't change it anymore, and Hegota's closed. I see a hand raised, so I'll stop talking.

Łukasz Rozmej: So, I would prefer having some momentum behind this, either merging the spec. and we can refine the spec later. It's like… it's an optional spec, so… It's, optional in this way that you can use, JSON RPC, if there's something you haven't implemented yet. You can potentially fall back to JSON RPC if there's any issue, but I would prefer… Either merging the spec, or having some kind of… DevNet, that, with the current spec, that there's some kind of testing. Going on. That we can, polish, the potential Bugs and, compatibility between clients.

Tamaghna Choudhuri: Alright, on the DevNet part, so, like, the basic idea was when, Barnabas was on the call, so, like, we not have a separate DevNet, like, towards the end of, let's say, this fork, we would maybe push… all of this into the DevNet for that one. Like, that sounds okay, but, like, then, let's say, I guess we need to coordinate that on ACDT, and I can maybe bring it up on Monday. And, like, yeah, like, I agree, like, having something merged in that momentum would be better than… keeping it as it is, because, like, it's just, like, it's just stale for a long period of time, because there hasn't been that kind of momentum. So, like, I agree that, yeah.

Łukasz Rozmej: Yeah, basically, this is, like, you know, backward compatible, right? Because we still have JSON RPC, it's still there, but I would start, you know, having this, any momentum, right? So… I agree that maybe we should have a… some kind of verification first before we merge the spec, but if we have a DevNet and it works for some client pairs, we can merge the spec, in my opinion, and… Go farther, and we can refine the spec later if needed. Again, this is… We can call it an experimental path for now. Until… you know… it works on, I don't know, DevNets, or whatever.

Dustin Brody: I mean, seems reasonable, to me. I still would, I, I would say, point to… and I, I wanna, as I'm saying this, I'm… I wanna kind of, double-check that I'm not, sort of, hallucinating these things, but I had thought that there was, Let's see… as I said, there had been some feedback, and it would be… I am… I am a little concerned with saying… I mean, I agree with the idea we… momentum is good, and getting the DevNets are good, although I'll say, honestly, I don't think Nimbus is… quite, quite ready for the DevNets, but I'm… I'm… I support them anyway, or, or access, or, or start… or starting to set up, use, existing DevNets for this purpose.

But regardless, that to the degree that there is this even existing feedback, even let's say we don't want to wait for unspecified further feedback, which may or may not come, or etc, to the degree there was existing feedback on this that has simply not been…

Łukasz Rozmej: So, looking at the… PR, last comment from Marius was on July 10th, and I don't see any feedback after, and it was about… That's… feedback was addressed, oh, even June 29th, really, and so I don't really see much feedback after. So, I think, like, this is mostly updated. Maybe there are some small details there, but I don't really see anything big. Maybe there's another place when there's some discussion that I'm not aware.

Dustin Brody: Here, where, where'd I? They're right, okay. Yeah, I'm… I'm actually looking at this again, I agree with you, I… I… I had… sort of misremembered a little about it on this. I thought that we're… But… but they… there… there is a lot of feedback, but it has been, as you say. addressed. Obviously, there's more stuff that Might be done, but sure, okay, fine.

Łukasz Rozmej: Shipped is better than perfect.

Dustin Brody: Yeah, yeah, indeed.

Tamaghna Choudhuri: Alright, because, like, like, like, you just said, like. A few features are basically waiting for, like, a few follow-up peers are basically waiting for this to get merged, so, like, they can add, let's say, whatever they want for the payload witness, etc, but, like, that would depend on this getting merged. Or also, like, the OpenAPI thing, so, like, that would also depend on this getting merged, that's what I'm waiting for. So, maybe here's the best part, like, let's bring it up on Monday, and let's, see… push for, the… reviews and the final, let's say, reviews getting in from every client, the basic idea would be one from every client, and then we'd also, let's say, talk with one of us and try to figure out the DevNet, or, like, the DevNet schedule, or the compatibility for that, and, like, that's all I can think of. Any other things I'm missing, maybe?

Also, this is apart from, Dustin's idea of the drop-off sync, like, there has been an EIP, like, that should be there, so, like, that's also, like, it's following its own procedure, but, like, it's just another, I think just another, method to the existing RPC that we're having with Materio, so, like. That should be covering also, like. Any other things, and then maybe I can… I guess we can concentrate to do.

Dustin Brody: Sure, I mean, ACDT… sorry, ACDT certainly sounds like an appropriate venue for this, the right… People, or enough of the right people, will be there, certainly including Barnabas, I guess, and people from the various… ELs and CLs. So Hmm… And it's, it's better than the ACDC slash E in terms of, How to phrase it, not formality, but… kind of topic choice, and I mean, this is a… probably too small a topic to… to discuss in the two big meetings, but… but the ACDT is good.

Łukasz Rozmej: And I think this… this… Yeah, ACDT is fine, because we also have ELs and CLs there, right? This is cross… Yeah, yeah, yeah.

Dustin Brody: Well, yeah, well, yes, although, obviously there's the breakout room set up, but the… At least lately, I guess, I… they've… the first half of the meeting has been everybody in the CL and EL devs in the same… Yeah, so that's… so as long as we make sure that it comes up in the… that part of the… and… and not as a breakout item, because this is obviously cross-layer, so… But I…

Łukasz Rozmej: One other thing, though, tests. We are missing tests for that, right?

Tamaghna Choudhuri: Yeah, yeah, yeah. Yeah, like, I have worked on the test, like, basically what's remaining is the PRs and, like, for the follow-up, like, the tests, it has been going slower than I'd expected, but, like, it's, it's, it's there, like, Maybe… give me another week, I'll try to… merge them in, because, like, what happens is, if I put a PR in, it takes a minimum of 3-4 days for Mario to have a look and then get it more, so, like, it's basically one PR a week, so that's the problem that has been there with the march. Like, it's landing in execution, specs, but, like, it's going to take a bit longer Like, maybe one a week or two.

Łukasz Rozmej: Yeah, yeah, sure, sure. Oh… Would you have a separate test suit for EL and CL, or how that would be?

Tamaghna Choudhuri: For now, it's just for the EL suit, and for CL, there's no other suit, because, like, there is no engine testing for CL as it exists, like, just, like, for now, no. the future plan is to have this testing, but, like, that would be also coming in a later date, and, like, for the entirety, the basic idea is to use the CL for all ELs and that thing, but, like, that's coming a bit later, and, like, that's a bit farther. So, for CL, nothing. For ELC, yes, it's coming.

Łukasz Rozmej: Okay. Yeah, can't wait.

Dustin Brody: I think if one big advantage, obviously, to merging is, I mean, from the perspective of, for example, the… Top-up sync aspect, although as, as Tamaghna mentioned, I mean, it's… it is sort of on its own track procedurally. It's sort of procedurally also awkward to build on something that isn't even… it's just a PR. It has no… formal status beyond that right now. And it is easier, for sure, to kind of refer back to a merged part of the Engine API. or execution API, then… Either way, than, than, sort of, some mid-PR.

Łukasz Rozmej: Yep. It's even PR draft. So not even a… PR, that is… Considered finished. We generally have this in master, so, I make… I'm making my agent review this… our implementation against the spec. It found some inconsistencies, so I will post it, both on the spec and some small… small update on our PR. To fix those inconsistencies, but… yeah. It's something where the test would help, definitely, and polishing and merging the spec will also help. Hence? Testing, and we're done, right? Let's… let's go. This is… This is an… I would call it a low-hanging fruit in terms of, In terms of… a win? Definitely also helping with scaling, helping with, Glamsterdam larger blocks.

So…

Tamaghna Choudhuri: Yeah, yeah, like you just said, like, it's a low-hanging fruit, and, like, everyone wants it, but, like. not enough, like, it's not that high of a priority, so that's another reason it's just lagging, so, like, having.

Łukasz Rozmej: Yeah, but the thing is optionality, right? In my opinion.

Tamaghna Choudhuri: Yeah, yeah, yeah.

Łukasz Rozmej: Each client should have a flag to disable SSZ if it implements it, so it won't announce it as a capability, and CL can then default to JSON RPC. If it's… then that's it, right? So even if there's… we can… we can even, like, kind of release it as experimental feature first, that is even opt-in, or we can release it, you know, whatever. Or if CL is coded But that's okay. With some kind of circuit breaker, it can… it can actually… do it… smartly, fall back to JSON RPC, but that's a bigger… a bigger… You know a bigger… problem to do on CL, so maybe let's not do that, but… We can erase it as opt-in first, we can, then make it default.

No problems there, in my opinion.

Dustin Brody: For sure, opt-in initially makes sense. I would generally stand by my view, and I recognize it's maybe niche, I don't know, and I know I've mentioned also in the ACDC slash E calls as well, once or twice, but the… the talk-to issue of… of something like Get Exchange Capabilities, or… or… or, sorry, Exchange, just not get, not, just exchange capabilities, or, or the like, the fact that there's no stable identity for ELs, or CLs, I suppose. But from a… the EL sort of knows what they're serving, and they're kind of… kind of, in theory, can respond to either one. They just have that open. But the CL has to sort of make a decision on which protocol to use, and…

And, and then from the s… the fact that a moment ago, an SSZ-capable sort of EL was there doesn't mean that now it's… you know, It's still there, or vice versa. And one could say this is kind of a pathological case, but this is exactly the kind of, I guess the word that the EF uses to refer to this part of this sort of process is hardening, I guess, you know, in terms of the hard forks and such. But here, the idea of saying, well, yeah, you can make a basic implementation that does some kind of fallback, but it's going to have a lot of edge cases, a lot of race conditions.

Łukasz Rozmej: Yeah, I would leave it to unload operators, but just having it disabled on EL in case of any issues, right? This is an experimental feature.

Dustin Brody: Yeah, initially. But I guess what I'm saying is, it should be… my… what my goal would be for… is for it, eventually, for it to become… required. I mean, not… Sure. Not in… not in… okay, okay, sure, sure. Okay, and that…

Łukasz Rozmej: Or at least default, yeah, the fault required.

Dustin Brody: Yeah, at least default. I mean, yeah, there's a… there's a little bit of a gray area where there's a… I mean, even… there are features even now in both the CL and EL that have been default for years, but technically you can still kind of disable them and see how that goes. And… You know, but, but… for the most part, it's not tested with or assumed. But yes, at least a fault. In that case, I think we agree, but…

Łukasz Rozmej: Yeah, even though… we can deprecate JSON RPC and even remove it, right?

Dustin Brody: Yeah.

Łukasz Rozmej: Underneath the… Yeah, that's… Yeah.

Dustin Brody: Well, exactly. I mean, that… I guess that's part of it, and the thing is, as long as we have the JSON RPC api, its constraints have to be designed around… and also the… and maintaining a parity between them. So, as one just general example of this, in terms of JSON versus SSZ, we've seen this in, like, Beacon API stuff. Anytime, which I guess is, you know, another REST API, people will have a temptation to… create if they're primarily focusing, just for example, on a JSON version of the API, things that are expressible relatively easily in JSON structurally, like, optional stuff, or just kind of variable structure, let's call it. Like, things which are… JSON is a little bit self-describing, not schema-wise, but, you know, the… at least syntactically.

SSZ obviously is not, in the same way, and… and so there are API, shapes that are… do not work in SSZ, and we've had to, like, when… point this out, we collectively, people who are involved in the, Beacon API, maintenance, that when somebody proposes one of these, like, how does this work for SSE? So… that my consens… so… And, of course, that can be handled and managed, but the more… but both there's a cost to managing… to keeping both at parity, at par… on par with each other. And also, if… if the… as long as the SSZ is considered kind of optional and secondary, there's absolutely going to be this temptation to do something that's convenient and expedient on the JSON side that is actually really hard.

to encode in SSZ. And so these are the kinds of concerns I have with treating it as purely some optional side project.

Łukasz Rozmej: Yeah, sure, but I would do something like treat it optional for, glamsterdam, make it mandatory for Hegota, something like that, because… This… doesn't affect… Fork schedule for any people? And, because in the middle between one and the other, they can… they can pick up that work. Yeah, I would do something like that.

Dustin Brody: Yeah, no, that sounds good to me. And, I mean, I… hopefully what that will allow us to do is, as you say, you know, in terms of, you know, shipping is better than perfect, to ship something that plausibly might be, even if it's experimental, we can claim that it's it exists for Question. At some level, and… and… and I think even as a… sort of publicity point, that's probably good. And… but… but from my perspective, the practical target is… is… probably Hegota for a number, like, just in terms of also the implementation support. I don't… yes, I partly mean Nimbus because that's what I'm familiar with, and I can say Nimbus is not gonna be able… I mean, we'll see what we can do for Glamsterdam, but… But I mean, I think in general, the CLs and the ELs do not…

Widely support in their shipping release branch versions. the SSZ Engine API, and I'm not sure how many will by… by Glamsterdam Shipping. So… I mean, so my… my assumption the whole time had been that Hegota, or at least, let's say, post-Glamsterdam.

Łukasz Rozmej: Hmm?

Dustin Brody: Would be… would be, when… Support would actually appear in the ecosystem.

Łukasz Rozmej: Yeah, like, proper one, yeah. But, again, we can, for example, ship it, in the next release. It's not a big deal for us. It's more of a big deal of testing, right? To be sure that it works with CLs, and that we are… Oh… To be sure that we are, Spec compliant, so we passed the test. That's the kind of big thing for us, and… I'm not sure if we will have… direct clients at the moment that I would actually benefit much from it, but… Maybe? I don't know That's always a cherry on top. Yeah, you can enable this, and… Your… communication cuts by half, and communication times cuts cut by, I don't know, half, or two, or more.

Dustin Brody: Sure.

Łukasz Rozmej: All that are performance, a performance, you know, critical might want to use that.

Dustin Brody: Right. And I think we've seen some motivation here, I don't know how much you followed this, but on the CL Beacon API side, in terms of that performance concern, the Produce BlockV4 API has a couple of… well, there's the with bid version. But there's also, in terms of, basically, does the… VC have to communicate the blobs to the BN or not? Like, through the… I mean, and… there, I mean, this exact kind of, and the same network objects, I guess, the blobs, concern has come up in terms of, like, if one assumes SSZ, then this whole… the performance aspect of that question is a lot less… critical, because obviously it is faster and more efficient in all of the usual things. And here, I would imagine that we might have a

a similar… Similar sort of outcome, where, and where the… Basically, my concern is that if we don't have this, that people will start trying to have workarounds. Because, you know, 10 megabytes or whatever of JSON-encoded blobs is just takes, you know, is, is, is, let's say, intense, in a time-constrained way to code and process. And so people try to find ways to work around it. And my hope is that one outcome of SSZ is that that temptation is significantly reduced.

Łukasz Rozmej: Bye. That's it? Anything else?

Dustin Brody: Hmm? Not for me.

Łukasz Rozmej: Not for me either.

Dustin Brody: Tamaghna? Yeah, I don't know.

Tamaghna Choudhuri: Not for me, like, like, for us, the best thing would be to bring it up in ACD, see the deduction there, I'll be doing that. Maybe I can even… I'll ask, whoever is hosting if we can maybe have a small presentation or something. I'll take over that, sure. Other than that, nothing.

Sameer Agarwal: Yeah, I'll just like to add that I'm implementing this for Lighthouse, and, like, just in general, just giving an update on the implementation is that, I've gotten working on Fulu with, and I've tested out with, Erigon on, Local Kurtosis devnet, so… Everything's working fine, so at this point, I'm just waiting for next steps on the spec, and… For the testing, as in when it's… Done in the next ACDT.

Łukasz Rozmej: So, one comment from me, if you're, like, picking with one EL, there might be two differences between ELs, and that's the problem, right? That's why we want the spec end, that's why we want the test end, because it doesn't mean that what you're implementing will actually be completely compatible. With the rest.

Sameer Agarwal: Oh, yes.

Łukasz Rozmej: Implementing its own Probably very small differences in flavor, but still. That we need to iron out.

Sameer Agarwal: Yeah, correct. I actually just started, testing with ethrex as well, since the last, it's been working. decent enough, essentially, but I still don't want to make any comments on that. I want to run it for some more time with ethrex. But it's looking good initially.

Tamaghna Choudhuri: Alright, and then that does it. See you guys on Monday. Have a nice weekly. Bye, guys!

Łukasz Rozmej: Bye.

Sameer Agarwal: Bye.

Dustin Brody: Yay.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.