All Core Devs - Testing #84 (CL Breakout)

Transcript

Um >> [clears throat] >> I'm just I'm going to try to start recording. I don't have permission. >> Uh there's no need to >> I'm recording. No worries. >> Oh. Oh, great. Thank you. >> All righty, we have the space to ourselves. Uh what do you guys want to discuss? Maybe the issue that we're currently discussing in Discord, the thing that Enrico brought up earlier.

Maybe Protus would like to talk about it. >> What do you mean the PTC attestations in separate branches? >> Yes, that's right. >> Okay, so in order of importance, I believe we should not care about this right now, not in the next devnet, and not in the next one. Uh but anyways, so that's one. The second one is the issue is about uh whether or not PTC attestation should be available for branches that are not canonical.

Every client is currently sending PTC attestations only on the canonical branch. The problem is that if there is an actual split across different shock links, then any proposer, since you cannot achieve consensus, assuming that the split is uh considerable on every branch, uh then no branch can achieve consensus on the PTC. That means that any proposer can always reorg the previous payload. Therefore, rational proposers would keep the chain without a single payload.

To prevent this, clients should send PTC attestations on the branches that if that aren't even compatible with their head view, which is a problem for clients that work on the on the beacon API because the beacon API sets duties according to the dependent root. Now, what's special about PTC attestations? They are for the problem that I'm describing, it only makes sense to send them when you are attesting to be uh for an available payload, for early and available payload.

It doesn't really matter if you don't send an attestation for a late payload because anyways there's not going to be consensus in this situation. So, it's only important to send attestations for available payload and early payloads. This means that what I believe is what clients should be doing is sending such attestations as soon as they see a payload if they are only on one of the committees. That is, if you receive the payload that is canonical and is your head, you can just go ahead and send your payload attestation immediately. If you receive a payload for the current slot in which you are a PTC member and the payload is early, but the

payload is not canonical, then you need to check whether your head state uh if in your head state you are in a PTC committee. If you are in both committees at the same time, then you should not send a PTC attestation and wait until you see the payload arriving for your head. And that's it. That's essentially it. It requires a really architecture of the of the way that you request these duties. The validator client should request these duties every time on a payload event and every time on the payload deadline. And that's it.

>> Okay. Um my question is not about the mechanics of that, but about the situation let's just imagine in the mainnet um we have a two competing branches. And uh, these PTC voters matters where we have actually payloads built by builders, right? Uh, or or not maybe by builders, I don't know. So, I'm just my question is uh, will builders be building payloads, will keep building payloads in the situation on the main net where we have two competing branches and um, I don't know, one is not a

definitely the winner. Um, so any payload revealed in this case um, is it is there is a probability that it will be reorg or the whole branch >> situation. Rational builders should not build in this situation. But anyways, we're always good to self-build. >> All right, and the self-build as well is uh, does it require um, the PTC voters to enforce um, the full blocks?

Can't we just rely on honest proposers in this case that there will be an honest proposer that regardless, I don't know, there could be a few proposers dishonest before the honest one, but the honest one will not reorg a payload um, that was the proposed by the previous proposer. So, we kind of like need to honest proposers um, in a row, which is not unlikely this is like which is like quite probable, no?

>> Yes, I I don't expect this to be a major issue, that's why I think it's shouldn't be a priority. However, the problem is that we are relying on proposers being honest and irrational, which is not something that we do. So, we're only doing this now for uh for few um duties. Uh like aggregators is something that we rely on uh on altruism. Um there are other duties that we rely on altruism, but we don't rely on altruism on any duty that uh that involves getting money.

If it's rational for you to reorg a previous payload and try to steal whatever MEV is in the previous payload, proposers would do it. >> Yep. I I cannot dis- disagree with this. >> Nico. >> So, isn't that just a general thing if we have a overall participation of roughly 50% that you can do whatever you want with payloads? Like builders can reveal late as long as attestors process them in time and at the same time you can always

build an empty as a proposer. >> Yes. Uh so, if if the BTC will not achieve consensus, then both of those statements are true. Which which is why we really want to have uh nodes that are managed that manage to sync a payload. Since you're syncing already that payload, you're syncing already that block, then you have all the information you need to submit that attestation. You need to be really really unlucky to be in both committees at the same time.

So, if you have if we have such a long uh a long split and we are able to follow those chains because we're syncing those blocks, then you should be able to attest to them because well, you have the state. It just syncs a block. You have the state. You know you're tracking a validator. You know this validator is in the PPC committee. You just might as well just cache the PPC attestation and serve it when it's requested. >> Yeah, but I mean, even if we do that, if the participation is at 50%, the basically it's up to proposals to decide, right? And the testers just follow the

>> split. So, you're talking now about not only being a split, but also half of the chain being offline. >> Yeah, exactly. Yeah. >> Yeah, if the chain is half the chain is offline, then we rely on proposals not reorging payloads. >> Uh it actually might be a split, right? Say, almost 50 versus almost 50. in this case, builder may not reveal, right? And it will not pay.

Because the threshold did not meet. Or am I missing anything? >> Yeah, I mean, I was thinking the same. So, in those conditions, builder won't show up. So, they will simply disappear. Because it's too risky. And then uh self-building or non-MEV uh bid will flow. I- I- Is Is Is this correct? >> I think so. >> I I I definitely

I believe, even if I haven't tried to put any line of code on it, that it should be in theory not that complicated to to As Spore said, if we are following those chains, we must have some easy way of looking up uh validators and the committees. The only thing that probably is changing is to is actually how you do that. If you want to have the bigger node, it needs to start thinking and needs to attract the

validators not only for for for for blocks, but now for other things. You need to know about who the the connected validators and cache those stuff in advance, so you are prepared when the VCs to sign to send the proper PTC votes. So, it's technically feasible in theory. In practice, we need to understand how how big is the impact.

And also in the BNA if API changes. For sure. >> So, the core issue is the threshold, right? The that we need a PTC majority to vote on to reach any consensus for testers. So, I mean uh maybe I'm not sure if that opens up other scenarios, but one idea I had was to just um if we observe less than 257, I guess,

votes, we could always prefer the local view over PTC view. I'm not sure if that cuz that would solve all the issues we discussed but I'm not sure if that opens up other attacks I guess. >> Should we just continue discussing this on Discord?

I don't think that any decisions can be made right now. Anything else? Sorry any conclusion to that before moving on to the next topic if there's anything to say. Okay. Are you here to discuss those two beacon API changes?

>> Yeah, I'm here. I think the meaning these are two beacon APIs one is these APIs are mainly concerned with the builders like those two APIs contain like the proposal preferences SSE event and uh I like the the what is that the the API to publish the envelope like I understand basically like the stateless and the stateful like uh API like spec for publishing the envelope app. I raised it because I think it'd be nice to merge these changes so that it's kind of standard across clients so we don't run into issues where one client has done something that isn't working so

Yeah, so just wanted to raise it for that. I think Nico and James had pitched in in the APIs like Discord. James mentioned that it proposal preferences API spec just needs an improvement. So, uh yeah, and the and the I think the the other API, which is 580, produce block before with payload, that seems that uh James mentioned that it's it's outdated a bit, uh but and I think like Shane should ideally uh update that.

But apart from that, he he mentioned that it's fine. So, yeah, I I I think it it seems like more like a logistics and it doesn't seem anything like technical to be discussed, but yeah, I just wanted to raise it if any other clients has issues, so that uh it'd be nice if this API specs can be merged and like standardized. Yeah, that's >> Thank you. So, the ask is pretty much for clients to review those and uh leave their opinions.

Oh, yeah. Uh uh Barnabas mentioned that we should get the uh EIP 8282 merged into the IPs repo soon. It's a bit funny that the consensus specs PR was merged before the EIP was. Otis. >> I would like to reserve to have two reservations. One is a reservation to an allow list, and another one is a reservation to disallow list for uh for builders at creation time.

So, the disallow list is uh if your builder in if your version that you're sending is bigger than say 10, then we just disallow that deposit. So, this makes sure that we have a space of builders that we can use that we are guaranteed that no builder no no builder exists at the next fork. So, that's just a single line in processing deposits. And then another allow list, which is if your builder if your version is less than this allow list, let's say three, then we allow those bids to propagate and we allow those bids to be processed.

This allows us to in the event that we want to have a builder that we want it to be deposited during the current fork and be able to process bids during the current fork so that we can add new custom behavior in the next fork, we can onboard those builders during the current fork and they are active. So, this both of these features allow us to do things without forcing us to do things and they are both very simple. >> Um Yeah, I mean I've somewhat come around to the idea of only allowing certain builder versions like in the deposit flow.

Uh I would probably just go with allowing one extra, so like allowing version 0x01 or sorry, just version one instead of like 10, because I I don't expect that we would add multiple new versions at once. >> in the message the 10 was a disallowed list. That means that anything that is bigger than 10 we disallow. And then three three was in the allow list. That means anything that is smaller or equal than three we allow to propagate. So, these two numbers are different and separate purposes.

>> Yeah, I'm saying that we should be more restrictive and just disallow whatever the phrasing you said was, but disallow any version that's not zero or one when depositing and then I don't really see the I don't know why we'd want to res- allow propagation of bids for the through three or whatever. I mean, we can just easily change that at the fork. >> For the purpose of allowing those builders to be active, we have the option of making those builders active before the fork.

>> I mean, they could do that anyway. >> which otherwise we would lose this option for an unforeseen event that we might want to have this. >> I don't see why we would disallow. You go. Sorry. >> If we go with zero If the numbers 10 and three become one and one, that's absolutely fine. That means that the disallow list and the allow list would be the same. And still we would have the option of both.

Uh what I would not want is to have the allow list currently as zero. So, currently the numbers would be for the allow list zero and for the disallow list infinity. And I want to make those two numbers closer together. I want the disallow list to be less than infinity and I want the allow list to be larger than zero. I don't mind where those two numbers are, but I want those two conditions. That the disallow is less than infinity and the allow being more than zero. They both can be equal and they both can be one, that's fine by me.

>> I mean, I I don't think that builders that are version zero should be allowed to propagate bids. Like uh I'm I imagine that the next version of builder will be will do some will deal with a proofs instead of payloads. So, like why would a prover need to be able to propagate bids? Like payload bids. >> Because I have no clue what the next builder would need to be. This allows us to have behavior that that is unforeseen currently.

Like forbidding ourselves from using behavior that we just do not know what we're going to need in the future and for no reason doesn't make sense to me. Currently, we will not be able to onboard someone actively at the current fork just because we decided to not propagate any version that is non-zero when we don't need it. There's no usage of this field. So, there's no harm in propagating any version. So, what I'm asking is that we just allow more one more than zero at least so that we can actually use this if we need it. We just do not know if we're going to need it or not. If we don't need it, there's no harm done.

>> I mean, I totally agree that we should allow another version be onboarded before the fork. I'm just saying that I don't think we should allow it to propagate bids. We can just easily add that ability at the Like we can update the function to say if version zero or if version one allow. >> Not if for some reason we need these builders to be active and sending bids >> Okay, I mean, there would be one slot in which they could not send a bid probably, right?

>> Not if we need these bids to be active before the fork, which again is unforeseen behavior. I do not know. Perhaps this will never be needed. But the point is that there's absolutely no harm in allowing those lists now. It's just changing a number zero for a number one in the current PR. >> Can you propose this as a spec update or as like a spec PR? And I can give better feedback then.

I'm also curious what others and how they see this. >> Yeah, I'm not writing this. If someone wants to write it, it's fine. If if there's going to be like an eternal argument over this, I'd rather not do >> I mean, we don't have this allow list and this allow list for validator types, no? Like you could right now deposit a 0 x 0 5 validator into the >> Yes, this is different than the other what I'm trying to do is to correct the mistake that we did for validators.

>> I mean I I still don't see the problem. >> I don't want to say anything else on this topic, but I'd be very curious to hear other clients' opinions. Also, if we agree to do this, I can do the PR and make the changes or whatever.

Okay, uh Aton says that he agrees with Protus. Okay, let's discuss this more on Discord or something. Um Give others time to think. What else would you guys like to discuss?

Any implementation issues? Anything? >> I I do have a question actually about this assertion. So, so putting aside the let's say criticality of the the 20 minutes, that's a discussion that has been had. I don't I don't intend to repeat it. Um but but this idea of the bids being the bids only exist in the context of that very specific slot. I don't really understand even the notion of what does

it mean to have a bid before the fork? Why would it ever be useful or or desirable? >> I don't think we're talking about bids >> Okay, so maybe I just misunderstood. >> about Yeah, the what we were talking about is basically having bids as soon as possible right after the >> Oh, well, that's a very different point. Okay, I see. Portis is >> I think >> Yeah.

>> the thing you're you're I mean, there are two different things. The 20 minutes thing at this particular fork uh is about like onboarding builders quickly. And what we were discussing before, which I think it's completely correct, your your your statement, uh I was arguing that we should allow bids before the next fork, meaning at any given fork after EPBS, we might be in a situation in which we might want to onboard new versions of builders that we would treat differently and we might be in a So, that's one thing and then we might be in a situation in which in which we want those new onboarded builders to also be able to submit bids. Uh if you

ask me why this would happen, I would tell you I do not know. I just want to have the option because not having an option is worse for me if it's only one line check. Which we already do. So, it doesn't really change it doesn't really add any any code. It just changes a number in the code that we currently have. >> But a new builder version would be able to like propagate bids. Like if we say that they can. So, say at the fork we allow version one builders like they could send in a bid and like I believe at the fork, right?

And it could be accepted on chain. >> Yes, but if we currently ship the PR as it is now, then for the H fork uh version one builder will not be able to submit bids before the H fork. And this is just a prohibition that doesn't make any sense to me to force ourselves to be in that situation where we might find it useful to have version one builders for whatever reason be able to submit bids before the fork before the H fork. >> So, I think this is Justin's confusion and also my confusion why would you do that?

>> Again, I do not know. The question is why would you we forbid ourselves from doing that? >> Words semantics even of if if it's a new kind of builder, are they is it a new kind of bid? Is it I mean, I'm not saying these questions have answers to now but almost the point is they kind of can't have answers. And if they don't, then how is the protocol supposed to handle this? How how are they supposed to be What is even the What does it even mean to bid on a a The bids only apply to the slot that is happening immediately.

Like you can't prepare bids 10 slots in advance. >> Nothing. We can have arbitrary crazy execution. We might be in a situation, for example, that we want to have builders for the next fork. Um we are going to uh give them a price, give them issuance, or charge them issuance for bids that they submitted before the fork. So, this explicitly requires them to be able to have submitted bids before the fork. So, it doesn't matter what it is. The whole point I'm making is we should not put in the protocol things that forbid ourselves from doing things for no reason.

Why would we forbid ourselves from handling a situation that might be useful when not handling them is not a cost for us? >> So, you would use a whitelist basically to enable different type of builders. But then how would you trigger that whitelist without the hard fork? >> We already have a whitelist. The whitelist today is we only allow bids with version zero to propagate. So, what I'm asking is that that whitelist changes from zero to one. That means we allow bid versions zero and one to

propagate and to be included. And also, I want a blacklist. I want Today, we don't have a blacklist. We onboard any version on the builder deposit. And what I want is to add a blacklist that says you are not allowed to deposit a builder that has version 10 or more. Both things are separate and they serve different purposes. and I want both of them to be included. >> So, currently we also have a gossip rule which says the bid needs to be for the current or next slot. So, technically you cannot submit earlier.

Unless you mean exactly one slot before the next fork. >> No, I mean any builder from verse that sends souls now at the current spec but we are set my change. Then now builders that are version one would be able to send bids exactly the same as builders that are version zero. There's no difference between version zero and version one at the glorious fork. However, if we allow so the tooling is going to submit only builders of version zero.

But now we come up with some crazy feature that we want to have in the H fork that requires builders that are version one to be able to submit bids before before the H fork. Therefore, bids for their current slot at the G But we want to have those bids committed already on chain. So that at the H fork we can add this new logic that goes back and says if you have a proof for the beacon state during the G fork that was included by by a builder which was version one, then you get a price. Then or whatever it is. So then this is enabled by allowing those builders to actually be able to submit bids.

>> Yes, so basically there these bids are already included in blocks in glorious, right? >> Correct. >> Okay, yeah, then I misunderstood this initially what you meant. >> We should just continue discussing this maybe on Discord. Hey.

I don't have a strong opinion on it anymore. I'm happy to do it whatever clients make it best. I mean, we might want to then disallow them to propagate bids, payload bids at H* That might be a little bit of a complication.

If that makes sense. Is there anything else people want to talk about? >> When will Devnet 5 be up? >> It's a good question. Is anyone using Devnet like

are there issues on Devnet 5 that people are wanting to figure out before it's killed? >> I am on a couple of issues I hope to fix them. Uh do you >> Any any of how long that would take for you? >> I mean, we need to then approve and merge, so just maybe two tomorrow, two days. Like I don't I don't remember what what are the timelines for Devnet 6. Sorry, guys.

>> We can do Devnet 6 as soon as we have like two more ELs ready. Uh I would expect that to be maybe tomorrow. Uh most likely by Wednesday. But we can also have uh parallel Devnets. It's just a bit less optimal when people have to pay attention to multiple Devnets at the same time, so >> Yeah, sure. >> I would prefer if uh everyone just paid attention to Devnet 5 and then uh everyone would pay attention to the Devnet 6. >> That said, I don't have a strong opinion because I can anyway test uh these things probably even on Devnet 6, so do whatever whatever you think.

>> Yeah, I'm in no rush to shut down Devnet 5. Just doing a bad check on this. >> The only thing I'll kind of say, I don't have a strong view on this precisely, but it is unlikely that unless it's fairly critically breaking things, Nimbus is going to devote a lot of effort to fixing um the Devnet 5 specific issues when Devnet 6 launches. Um we we try to avoid maintaining kind of the divergent Devnet branches. Um but I mean Yeah.

>> Is there any news from Prism? Has Prism been struggling with uh the peering issues for a while? Uh is there any resolution in sight? >> What >> fixed a few bugs last week and I fixed the two more bugs like yesterday. I guess the TLDR was that we have this pending attestation queue and then like same bug as before that we verify pending attestation, but there are times that where we just want to use the head state which is cheap and fast to get to verify, but if we cannot verify it, which is not the peer's fault, we will

penalize the peer and cause the disconnect and we found one more instance of that yesterday where we were doing that from the pending queue level and this is not at the gossip level. So I'm starting to test um whether that fixes the disconnect issue. And yeah, so like you said, we do have some weird year error disconnect issue. That's why in Dora that like sometimes prism nodes are offline. So hopefully that fixes it. I will keep you updated.

>> Final calls and discussions. We have 8 minutes left. >> Does anyone could uh give us an update on the builder API implementation? I know that prism has a working or partially working implementation. And does any other client have anything? >> They told us that they're going to try

to get it this week. >> I have a tangent to a question related to this sort of maybe a question for breath is will map boost update? Like will there be an update for map boost or CPBS or is it going to be deprecated? >> Yeah, I mean so I think clients are likely going to implement the multiplexing feature. And with that like you technically like don't need map boost plus like you will for for for uh features like builder preferences and request off map boost would need uh to have access to validator keys. So I'm unsure like I think it likely be deprecated like flash bots is not set

much on it. Like Like I think if clients implement the multiplexing uh to me like I I don't see a strong argument to keep it around because like the only big features it currently implements outside the regular PBS flow is a timing games which I would definitely think clients will implement that. So yeah, map boost like I I I don't see a lot of value in like updating it as well.

If clients implement the multiplexing so So yeah, that's kind of like my take. I I won't comment about things like comment boost and stuff. But uh yeah, that's what I feel about I think comment boost and stuff they implement client signing arbitrary data so I guess they can just support the request off and all those things. But yeah, map boost doesn't have that and I don't know if there are plans but And I'm unsure if there's like as I mentioned again if clients implement the multiplexing I'm just unsure if it's necessary to update that.

>> Okay, thank you. Terence. >> Yeah, since we have five more minutes for Protossis it would be very nice to have the multiplexer feature set in Protossis. I'm not sure what the progress is, but yeah if there's something like that I'm I'll be very happy to test. >> Yeah, you you mean like having like multiple builders I I'm currently like working on that like hopefully I should get that done by tomorrow. >> Awesome, thanks. And then another one is that was missing from initial testing was having SSD support, right?

>> Yeah, I think we do have SSD support. >> Okay, you have it now. Okay, perfect. >> Yeah, you can just go for >> Thanks. >> But like we return that the response is currently we return in JSON. Uh but I can support like returning SSD in responses too. That's what we do in the launch. >> To Barnabas' question um the death net 7 with builder API support. Another thing I don't know if we actually tested thoroughly is that like having just builder on the PTP network like broadcasting bid. So I think we want to get that working first

before builder API, but I guess it doesn't hurt to have both. >> We were like broadcasting bids right on the the launch. >> So yeah, and all clients can support that? I know Prism can and I know maybe Lighthouse, Nimbus can, Teku can, I don't know. >> Yeah, I think Lighthouse, Lodestar, Grandine Teku can Nimbus also Nimbus also can. I think I think Dustin would you say anything to that? I'm I that was top of my head with the builder support active people?

>> Uh yes, Nimbus supports peer-to-peer beds. Uh >> Um yeah. >> Yeah, but what it's missing is building it for every single slot, right? >> Oh, um that's that's in progress. Right now, no. That's in progress, but but right right now, no. And um yeah, I don't know. I I We'll we'll see. >> Okay. Uh I guess we should end this call

now. Uh thank you, everyone. Um Yeah. I'm excited for Devnet 6. >> What's up? >> Just one more reminder to please make sure that you come with your opinionated opinions by ACD regarding the SSD decision. Whether we're going to be including it or not. Cuz I think this week should be the week when we either like as if I or the FI. >> Oops.

I was just on mute. >> Oh, uh sorry. Just a quick question. I don't need to do it here, but post on Discord if any client is actually actively uh reorging uh late or unavailable payloads. Um namely when there is a quorum for for no. Oh, okay. So, Nico is saying that uh that Okay, that Lodestar is already doing this. >> I believe so. Lighthouse as well. >> Lighthouse 2 and Oh, and then Nico go to okay, so we are the only ones that are not doing this because I wanted to run some tests on on cortosis and I think I'm seeing some very invalid attestation. So I'll post results later.

>> By the way, what image did you use? So for the I >> Whatever we're using on the I'm using the same as the gloucester devnet five whatever joker gives me from it panel on gloucester devnet five and default flags. >> Yeah, the loadstar one has broken PTC stuff. I put in there intentionally. So maybe you can use the devnet six. Cuz I suspect it's going to vote really wrongly. I did that to do some testing on the devnet.

>> Nico, are you still testing with loadstar teststar whatever it's called? >> Not on the devnet. We can put that up on devnet five. I haven't rebased the branch for devnet six. Not sure if Barnabas wants to test something. >> I think it's the devnet five is stable enough to try to break it now. Other than the one off that you bought that I can find with the invalid transition. >> Yeah, so you can just deploy to the same node again and see if clients are voting. I'm not sure if there's any update from grandino Nimbus on this.

>> Listen. >> I I haven't looked into it particularly. So update wise no. I think that'd be a an excellent way to kill DevNet 5 when they come sign for >> Yeah, while we wait on ears to get their branches up we try to destroy them as well. Okay, we are at time. Thank you everyone for joining. Talk to you again in a few days. Bye.

>> Ciao. Bye-bye. >> Thanks, see you.