All Core Devs - Testing #77 (ePBS Breakout)

Transcript

Uh, I think you have enough privileges. Um, I started recording. So, I will be will be recording. >> Perfect. Thanks. We'll get it sorted by next week. >> Thank you, Barnabas. Okay, just one more minute. >> I think everyone's been assigned to rooms now. >> Yep. >> Okay. Uh, hello everyone. Uh, this is the EPBL breakout room. Uh I guess let's jump right into it. Uh let's talk about the deferred payload processing PR. Uh Nico, would you be able can you give us a very brief overview?

>> Yeah, sure. So I'm hoping everyone is kind of aware of like the last week what has been discussed. So I'm not going to iterate over everything. Um I added the main pain points in the PR description which was mostly that the dependent route uh the checkpoint state and also the weak sub subjectivity spec as currently uh is underspecified and really not easily resolvable with the current glor.

Um so with the spec change I'm suggesting um like what it broadly does is it defers the execution payload processing to the next block. Um and the key property there is that we keep one state transition per slot. We also um remove the two states we have per slot per block route um from the spec and also uh no longer uh will that be a part of the clients basically the dual state architecture.

Um and this basically restores the checkpoint state semantics. It also restores the dependent route and um there are some trade-offs. Um I think POS already mentioned them. I have also document I graded last week and noted them. I think they are mostly minor. Uh I can go further into them if there's like interest. Um there are also a few more benefits um that are client specific or client implementation specific not really visible in the spec.

I can also go further into them if there's interest. Um yeah, I think that's that's broadly it um to get us started. >> Okay, thank you. Um I'm guessing I should ask if there are any clients that just are opposed to this PR in general. Okay. Uh maybe pis you could mention some of the open issues that you just uh commented like an hour ago.

>> Oh, I mean they're they're minor they're technical things. Um not sure if we want to go over those. Uh but my general comment would be that uh we we should review the PR. I I don't want to be in the situation like when we had a massive change which was making the builders not be validators and then after we merged that PR we found a ton of bugs on the spec. So I I would not want to be this late in the process in that situation that that'll be a horrible situation to be. uh this PR I I was adamantly against it because I suspect that it's going to be a change in the consensus where I don't believe this is necessary but on the

other hand the PR itself seems a very minor change to consensus even though it looks as a major change it's uh it's a minor change to consensus that brings major advantages to implementations uh so I I I I really strongly support this change it's it's it's going to make our life easier and I'm very sure that it's going to avoid bugs but I want people to review. So in particular the two things that I found is one of them it seems to me that were building uh the payload the next payload on top of just the the the payload being on top of full if the payload was synced and I think this is wrong. There are situations in which we may have a payload and still

want to build on empty. uh so that we should decide overhead instead of like on just the payload being present and uh it escapes in my mind what were the other uh issues uh I mentioned yeah there's some some hash that should be checked and a bunch of other things uh >> the builder payment you're saying that should be >> oh yeah there's one issue where the builder is u the builder the builder might actually get his payload to be canonical and then not pay anything because um the PR doesn't enforce that payment. So those kind of things I think we should really review with carefully before merging but I strongly support merging this.

>> Okay, great. Thank you. Yeah, I agree. These are relatively small just little spec issues. Uh thank you. >> Uh could we get opinions from other clients as well like uh maybe Teu or Loadar? >> Yeah. Um I can briefly give my opinion. So I I was I was we were definitely um struggling with this double state um um structure that we had to add into into our code our code base. It was not a measure thing but then triggers a lot of internal

um things that you have to think about and always making making sure that you're picking the right state. It was puzzling me. I was actually uh giving a lot of worries that well this could be very bugprone. Um, so I I was definitely advocating to find out uh an alternatives and I'm very happy that Nico spend time to find a workable solution to that and u so far uh seems seems good. So we definitely support uh the change

and uh yeah I want I want to highlight one thing that maybe Raul could double check. I was trying to estimate the amount of u um overhead in the bandwidth that we could have because one thing that this PR brings that is that we will exchange on on network the request execution request twice. Once when the payload arrives and and then when we the next proposer uh select uh builds the next block and the beacon block now will um contains the same

execution requests. So there is definitely an overhead on the networking. I was trying to um estimate that it seems like around half megabyte um at maximum for each for execution requests with 60 million gas was just checking it uh within an at the moment >> I saw I saw I saw that thread but I I didn't have time to research it to the level of detail that I wanted to uh before responding so I'm still on it. Um I just want to confirm basically like in the worst case scenario where we have uh the maximum amount of items per every request type uh what compression rates

we can expect uh on the wire before before responding. That's basically the analysis I was doing. I think that that should be enough right? >> Yeah. Uh I was not taking in account any any compression but it seems like we could have like kind of 2,500ish uh deposits that this this is the theoretical maximum that seems like brings us to like these like half a megabyte for for >> roughly but I have not probably taken into account compression so yeah.

>> Yeah. So, so that would just improve things. Um, there was this little site that we had put up a while ago that is just a window into what's uh flowing right now through the chain and how it would sorry through the network and how it would project onto the global split. Um, of course we could take a look at the and it does analyze the compression rate of every field. that just does some math to attribute the compression rate of every field. So, this could be also interesting for you guys to to look at while I'm um while I'm doing the analysis. But, yeah, it's kind of like just eyeballing it is around I think 200k

for the if we saturate deposit requests and then we've got consolidation requests to look at as well, withdrawal requests. I'm looking at an outdated uh calculator from yeah from the pandas that they put up I think for Electra. Um so yeah so this needs a little bit of a refresh. Uh but but yeah I think we should be able to get to the analysis between today and tomorrow. Is it is it really urgent? >> I feel like having a rough estimation not not really urgent but I think it could be good. But I feel like the the deposit is definitely the biggest because withdrawals are kind of kept to 16 and consolidation are just two. So

>> that's the most things are on the deposits. >> Yeah. Rather than focusing on the max and the absolute value, I mean I I believe that compression is rather relevant here. What we care is about the proportion of the compared to the beacon block because uh we can easily assume that the compression rate of the beacon block is going to be similar to request. Anyways, this is a good bound because requests are less compressible than the beacon block and uh um we just want to know what is the worst case request compared to the typical or the worst case beacon block because that's what's we're going

to lose. We're going to lose broadcast time on the beacon block and this is a problem. Can it become for example an attack vector on a builder sending a lot of requests so that it can anti-reorg the next proposer? >> Yeah. So the residual basically is what we're interested in. >> Yeah. So if if if requests are going to become like 60% of a beacon block, then I don't really like this. Um we want to like have a sidecar or something or enforce it on for choice have some sort of like fossil like feature. >> Uh and it becomes much harder.

uh if requests are just a tiny bit of the beacon block, then I don't care because it's not going to change our life. Mhm. Yeah, that's good context. Thank you. Thank you very much. I'll get back to you guys uh hopefully between today and tomorrow. >> Uh Eton, could you give us a a lighouses overview like how you guys feel? >> Yeah. Yeah, we really like this change. Um it simplifies things a lot for us uh implementation wise. Um, we had we're just about to merge the dual state architecture stuff to Lighthouse. So, we're really happy to revert all those changes.

>> Nice. Uh, and then SAS, how does Grand feel? >> Sorry guys. Uh uh so yeah we we kind of support this uh change and I think that uh for us it will be likely easier to implement it uh than the previous spec. >> Okay.

>> Um Nico do we have anyone from Nimbus? Oh, Sam, you're from Nimbus. >> It's kind of hard to hear you. >> I think not quite well. Let me check. It not working.

Uh it's a bit better now. Yeah, we can probably make it. >> Yeah. So I think the idea um for DPR is quite similar to how Nimbus work right now because we do not have um different state for um I mean snapshot for each parent. So we actually have um we play block and envelope starting from a particular state. So it's kind of similar how Nimbus work right now. So it's not uh I think it's quite um a good idea and it should have minimal changes to Nimbus I

would say. So yeah, it's uh great. We we we would agree with that. Hope that makes sense. >> Thank you. Uh and I can I can assume that Bootstar is also in favor of this. Nico, >> yeah, I mean we discussed it a lot last week. Uh there was only Tuen from our team which was initially skeptical but he has come around to really like it as well. I think um so yeah it solves a lot of issues for us we are bit in a similar case that like prism we have the dual state caches merged with mostly that work it's already a solved problem in

our case but I'm happy to delete a lot of code that is no longer needed so I definitely we support that change >> awesome uh for the PR Is it true that you have already worked on the test and you just haven't like pushed them? >> I'm not sure I have the tests in a spot where I can push them soon. Um, >> do you recall how complicated the changes are? I'm just curious like are we talking about >> No, I don't think it's complicated, but I'm also not I haven't really looked at what other tests we might want to add.

>> Make sure it works correctly. Um, >> yeah, I think it it would be good to add at least a few tests if possible. >> Yeah. So, who um I guess who should be working on that? I kind of plan to work on load implementation, but um I can also focus on the tests. >> Uh we can work on that together if you want and then if you just become too busy, uh Reky and I can take over. So, should we do that after all comments are addressed? Because I'm also still not fully sure the PR is in a final state.

>> Uh, yes, we should do that after. Yeah, >> because to be honest, last week I spent most of the time just vetting that the spec change is solid and just rushed the spec. Um, so we have something to look at. >> Yeah. So most of the problems we found uh on on that could be bugs uh on the current implementation on the current specs is uh are around a boundary block that is missing and then a full epoch or two or even five epochs of missing blocks. So that's the only place where actually

this the the current situation might uh show a difference between two clients. So I think we should add all those tests if we don't have them already. That is there is a block in 31, there is a payload in 31 and that payload includes a lot of requests like in particular this switch consolidation switching uh request and then you have after that block in 31 and that payload you have one two and five epoch missing of blocks and then suddenly there's a block that builds on top of that 31 either full or empty. We need those tests. Those are the tests that actually trigger most of the problems including the current problems in the current proposal.

Uh do we still plan on moving uh a lot of the spec changes to the onexecution payload function? I know you mentioned that in a comment somewhere, Nico. Yeah, I'm actually not sure about that. Uh, that was my initial idea when I looked at the PR because I think the only reason it's in the process execution payload is because it was modifying the state before, but I don't really have a strong opinion there.

I think it can be moved, right? It's only >> it's fine to just remove completely the process execution payload and just have it on as a for choice handler because that's what it is now. So yeah, I would probably do that change change then if everyone agrees or has no objection. >> I would like to see it but I don't have an objection right now. Yeah, the initial reason I kept it out was because of the diff in the PR because you can't really see what checks are there and what got removed once we do that refactoring.

>> Yeah, >> makes sense to have just one >> and since we are not changing the state anymore just to leave everything there. >> Could we also look at the independently to the problem of this validator switching to compounding? It looks like something that can create a problem that you have an immediate effective balance change on a fork caused by just the payload being there or not. Probably this PR already solves this on the nose. So, so but it'll it'll be nice to know. I mean for context for those that haven't read this issue, the problem is that if you happen to have a validator that has over 32 ETH and

requests to switch to compounding, this is the only request that immediately switches in the next epoch boundary. Uh it immediately applies the effective balance change. Any other request like with withdrawals or deposits or whatever, it takes at least five epochs. But this particular issue is insidious. You have a validator that has like say 35 E but it's uh its effective balance is 32 because it's a 0 point Z1 validator and it switches to compounding in the very last payload in the very last request. Then the very so you do the epoch transition immediately and then its effective balance has changed.

And I think this is just all this is already a bug in the spec. I think this is just an attack vector in the spec. No, >> but I think my change resolves that issue, right? >> I suspect it does because probably you uh you don't include it and then you just defer it at least for one epoch. So it it makes it instead of immediate, it takes an an entire epoch if I'm not wrong. >> Yeah. Yeah, exactly. Yeah, that's one I guess observation to point out uh in that change that if you are in the last slot of the epoch, you get delayed by a whole epoch. I don't think it's an issue. More of an observation though.

So, >> well, it's not necessarily one that's a lot only depends if the >> Oh, exactly. If there are miss blocks, right? >> Potentially a lot of slots. So I think the other issue is what I at least um in the current spec what I see is that we have a situation where like the dependent route can stay unchanged but the proposal look ahead and the PTC um window which is balanced based uh assignments they can change which is pretty strange.

>> Yeah. So that that is I think the biggest the biggest win of this change to my mind the biggest win is the fact that uh dependent route doesn't need to be changed at all. >> Yeah. But I think the checkpoint state issue is even bigger cuz that's one thing where we discussed on the API. I think there's an issue which is open for like 5 months and there's no conclusion yet how to solve that which would completely go away cuz I'm not sure how you want to solve what a finalized checkpoint stat is in the current CLA spec. I understand it's an API issue but also uh that that needs to be resolved.

I think >> I think it's overwhelming that people want to merge this. So probably it's a mute discussion at this point. >> Oh yeah, we I think we don't need to discuss it. Yeah. >> So we have only five minutes. Can can I propose that we decide what to do with devets? Uh how are we going to plant a target? The way I see this is it's very unlikely [clears throat] that we we can just have this merged in less than a week and make it a DevNet one relaunch or a DevNet 2.

So the the only I mean the most hopeful thing that I can say is let's try to target to have this by interrupt so that we get to interrupt and we already have a devet. Uh hopefully we can do this. I don't know if other clients agree with this and if you're really really trying to test I suspect that we can still test on a devet one with the current spec because consensus doesn't really change much with this. It's only the the epoch transition and uh and the first payload order.

So I don't know. So these are two different questions. Do we do a devet two? Do we try to target for interrupt? Do we want to continue testing on DevNet one and relaunch it with the current spec? What do we do? So, I propose that uh we could relaunch the net one. Uh it will have a very shortlived network anyway. Uh it's pretty useless now. So, we're going to be relaunching that and uh testing with the current spec. And then once DevNet 2 is ready uh where we change the beacon state and uh remove this double state thing uh we can launch that as soon as as possible and then during the interrupt uh what we can work on is the

merge devnet where we don't introduce any any more EIPs. We're just basically trying to get the ES and CS to merge uh to cler zero and then hopefully uh in a day uh we can get that sorted and have clerdamn net uh zero up in by Tuesday in interup. >> Should we even focus on devet one or should we just bolstein ahead on net 2? >> Yeah, I I didn't really see the reason to do that one. Uh but it's just I think maybe you just want something to be up.

>> Well, for me definite want [clears throat] we having a devet is useful. We we found a lot of bugs on consensus and on syncing and and and and the devet helps us to diagnose these things because most of the bugs are not really bugs. It's just things that we haven't implemented yet. [snorts] And uh I believe that most of consensus is not touched like for example all of for choice all of the proposing changes it's mostly not touched by this change. So I think it's fine to continue testing if if we can but if it's only one client then it's useless to me.

>> I do think it would slow things down for DevNet 2 though. >> Yeah I I do too. Uh I think we should what we should do is really get everybody to take a look at this uh PR, find as many bugs as possible, review it uh as much as possible and not really focus on the net one and then try to launch the net 2 as as soon as possible. >> That works for me. When when do you guys plan on on definite 2? So you want to try to have Genesis before interrupt or at interrupt?

>> Yes, I I would prefer to have it this week. This week is impossible. It's just plain impossible. Prism will not be ready this week. I mean, Terren would surely have a PR. It's probably already have the full implementation, but this week it's I claim it's going to be Terren's branch. It's not going to be present. >> No, but Terren's branch is good enough for a Genesis for net. Well, I mean, maybe. I mean, >> I'm not sure.

>> I'm curious what other other clients feel. Uh, also, >> I don't think this week is realistic. Um, I mean, the Spec PR hasn't even merged yet, and I think we still need more eyes to review edge cases and things like that. So yeah, I think interop was a realistic goal >> and also like we need to revert like a bunch of changes that we had previously.

So that might also take a little bit of time. >> Yeah, I'm sure prism is going to take a long time. >> Could we target uh maybe reviews then to be done by Wednesday and then target species by Friday? I think we'll have a psych release before Friday, but uh >> I think we've already made this mistake with your PR, Justin. Uh and then we had to have emergency releases because there were bugs in the spec. I think we really need to spend time reviewing and analyzing the spec for this PR. For example, Franches, this PR touches a bunch of things and Franchesco hasn't

seen it yet. Okay. However long it takes. Uh it's already 10, so we should probably wrap up. Uh for today and tomorrow, let's focus on reviewing this PR. Uh I'll work on or Nico and I can work on testing. I would like to have clients run the updated tests before merging the PR via like the nightly testing mechanism. uh when we merge it, I'll then make a spec release uh like within a day later. So sometime this week maybe.

>> Okay, thank you everyone for joining. Talk to you guys next week and in Discord. Bye everyone. Byebye. Yeah.