Fast Confirmation Rule Breakout #8

Transcript

Mikhail Kalinin: Hey, everyone.

Justin Traglia: Good morning.

Mikhail Kalinin: Yeah, morning. So I would wait for… Maybe 3 to 4 minutes, and then start.

Alright, hello, everyone. Welcome to the… Last confirmation rule breakout number 8. I'll just drop an agenda to the chat. Yeah, so, have a few updates to Dave. From the spec side, and testing, and API, and so forth, and maybe research updates, and We will start with the client updates first. So… Are there any client updates so far? All right, so there is one update from Nazar, who is not able to attend the call today. he's basically saying that he's waiting for the review.

from his team on the PR. To get it merged and released, which is great. Once they're done, Nazara plans to, work on the bus confirmation API. event, the Beacon API event for fast confirmation. Yeah, that's… that's it. Marissa, maybe you wanna share any updates from your side?

Mercy Boma Naps-Nkari: Yeah, sorry, I did make, some adjustments to my code based on the feedback you gave me, and I also added some, like, up to 6 metrics, based on what we discussed, the last time. And also, I'm waiting for reviews from… Gradient team. That's all from my own end.

Mikhail Kalinin: Great, do you have a link to metrics that you have added? No, I just… New York link to the code, yeah.

Mercy Boma Naps-Nkari: Okay, I… I will push the code, like, in the next hour or so. I'm trying to, like, I'm running it locally, so I'm trying to see if anything pops up before I push.

Mikhail Kalinin: Alright, cool. Please feel free to share whenever you push this.

Mercy Boma Naps-Nkari: Okay.

Mikhail Kalinin: Okay, other clients' updates. Yeah, Nimbus, have added the event, to the event stream as well, so that's the update from Nimbus. And also, I've been checking with, Like, I've been checking the Nethermind releases, and apparently they have not yet released the pics for… were the safe block hash they had with Engine API. So, yeah, waiting for… for this to be… To be fixed, and put in one of… sorry, waiting for this to be actually… for this fix to be actually, put in one of the releases. Oh, right, I think that's all for the client updates today. Thanks, everyone.

And now, we'll move over to this pack of dates. Have a couple of things here. So the first… the first one to share is this PR. That… I recently opened. So in this PR, basically, It changes the… validation that we have, during reconfirmation. So today, the reconfirmation starts, with a check whether the confirmed block is a descendant of the, Of the block of the checkpoint. we run reconfirmation from. But, this PR does the change… does change this condition, and, It now, instead of, like, checking that the confirmed block is a descendant of the checkpoint's block, it, Checks the… the confirmed block, Has the checkpoint

That we reconfirm from has this checkpoint in the chain, in its chain. Because there can be an edge case where the, The confirmed block is a descendant of a block of that checkpoint, but that checkpoint is not in the chain of a confirmed block. And, There are two reasons for having this. First one is that if this happens, actually, this should not happen if all our assumptions hold. And if this happens, it's a signal for the case where assumptions are not holed anymore, so we want to fall back to find out in this case. And also, The proofs… the proofs use the, use the condition where the confirm block is a descendant of a checkpoint and not a descendant of a checkpoint block. So just…

Yeah, Justin, I think… Yeah, sorry. Yeah, yeah, I see.

Justin Traglia: Yeah, sorry.

Mikhail Kalinin: Yeah, GU is, like, a bit… yeah, no worries. GU is a bit, maybe, ambiguous in this case, but yeah. So, this PR has a test, so it's accompanied with the test, so you will not miss that, because, yeah, if your implementation does not have this check. And has just the check. On the block, not on the checkpoint, then this test will not pass. Yeah, that's, that's the, update to the algorithm. I think… It's… it should be relatively easy to implement. But if not, let us know. Okay, any questions for… Related to this one. All right, so the next, the next spec, update would be, it's not directly related to

to the fast confirmation rule, it's rather related to Gloas, but this, this PR, once it gets merged, this PR basically makes the fast confirmation rule compatible with Gloas. And, any other forks? And has a basically the one fork that is based on the Gloas. Yeah, so we're looking forward to get this merged. And I already, tried to run, like, tried to run, tests. for the Gloas fork, the fast confirmation test for the Gloas fork, and with this PR, they work, they just pass. So, yeah, waiting for… for the SPR to get merged. All right, this, this, these were the spec updates, and I'm gonna give a… Testing updates.

If any question you have for these two PRs, just ask it. And yeah, for the testing updates, as I mentioned, that, the Gloas. Tests will… will be… Yup. are fine, with this PR, and after this PR is, merged, I will open up a PR. To enable the fast confirmation rule test to run with Gloss and Haze, and yep. Also, on the testing thing, testing side of things, I think it makes sense to… disable BLS for fast confirmation rule tests. It will just make it, faster, for… the generation will make… yeah, generating those tests will be faster, I believe. Because I don't see any, case for BLS to actually… Be…

You know, important for… for these tests, because, yeah, we assume that, all that we see is, like. Comes with valid signatures, so I would just, yeah, cool. If anything, any problem with disabling this, the BOS for a fast confirmation test, just let me know, but I don't think there will be any problem with that. So… and there are some, some edge case, mostly edge case tests, that are still on the testing plan. And after all this work is done, I mean, like, this, octress refractor. And the follow-up, tests, update, for, for, fast confirmation rule. There is a plan to start working on those, edge cases as well. And, yeah, to have these, Fast confirmation rule test ensued, complete.

So that's the testing updates, The next is API and metrics. We have been discussing this a bit during the previous call. There is a very, very short, very small PR to begin API. It basically clarifies what is sent in the first confirmation. Event. We had a quick chat, about this, and, actually, To avoid any confusion, we… Should likely clarify that, these, The FAT confirmation event sends the… just the most recent confirmed block, so the tip of the confirmed chain. And, events for every confirmed block will not be, sent. Because we know that, yeah, the fast confirmation rule can confirm,

Several blocks at a time. And, in some cases, these, Confirmed chain can be advanced, By a lot of blocks, in case of restart, in case of falling back to finality, and when finality is getting updated. This all falls into, like, fast confirm and a block, and we don't want to spam, this, event stream with, Event generated for every, confirmed block. So we just passed the tip of the confirmed chain. Yeah, and… yeah, and the next PR is, to the Beacon metrics. It introduces, for… or metrics. That we want to have. So, take a look at them, please, and yeah, give your feedback.

I have just one, one… Yeah, there are. So let me quickly go over those metrics. The registered total reorgs of confirmed log, which is an important thing. Important security signal for us. There are fallbacks, there are restarts. Fallbacks, means that whenever we fall back to finalized, block, yeah, that's… that counter should be bumped. And, one is, like, the slot, the fast confirmed slot, the most recent one. From this slot, it could be possible to… to derive a… Pass confirmation, delay. metric, if client has a current slot metric, but it's not standardized, the current slot is not standardized, so I don't know how… How it is in different clients.

Whether they have this metric or not. But anyway, please take a look at these, set off. Minimal set of metrics, and yeah, provide your feedback if you have It would be just great to have these metrics standardized, because then we would basically… comparing the performance and different security signals from fast confirmation rule, would be really easy across different clients. Alright, any questions so far? Cool. Let's move over to the research updates. Are there any research updates so far?

Roberto Saltini: We are going through the last, Final round of reviews of the proofs, just fixing some minor issues. I know… No impact on the final results or on the algorithm. It's just some internal issues to make sure that the blocks they all fit together, but I think that's… mostly done, yep.

Mikhail Kalinin: Great, thanks, Roberto. Are there any topics, to discuss. March from what we had already.

Sam Calder-Mason: I've got two links to share. I'll post the first one in the chat. So this is a public dashboard that's connected to one of the Lighthouse nodes that we're running that has fast confirmation enabled. This link should be the last 30 days. It's… I think it might actually default to the last hour. You can zoom back out to the last 30 days, It's been pretty stable, around 97% just, like, immediately Confirming. yeah, it's just something to keep an eye on, I guess. And this is going to be more valuable as we step forward, and it's just continually scraping data. And then I also have another link, Which is off the back of some of the other discussions that we've had.

Around simulating the historical data, this is a much bigger… sort of discussion. I won't recite the the contents of the article. But to cut it short, I essentially created a simulator that wraps A couple of the consensus clients, And then sourced blocks from other infrastructure that we have. So we've got, yeah, we've got the era files, which are public for the canonical chain, and then we also have all the orphaned blocks. In our, like, tracer slash block archive infra. And then we ripped out the data from ZATU of what an actual node would see. I know there's… we've talked a couple times back and forth about, like, what data we should be feeding in here. The one that ended up

Being the most viable to run over a long period of time was just the… unique set of attestations that were seen in the first 12 seconds for each slot, across all the ZATU nodes, and completely discarding all the aggregates, because the computation was a little bit annoying to explode those back out into single attestations. So, I'm just looking at… Single attestations, on the assumption that aggregates are good enough on mainnet, like, The end result was, like, That we didn't fast confirm any, slots that got eventually re-ogged, or orphaned, which is, yeah, obviously a great thing. There's a bit of, like, discrepancy between the CLs, mostly just Teku, which I think is a little bit more optimistic, which I think we discussed previously. But yeah, you can really see it in the monthly one-slot, fast confirmation rate chart.

And then there's also a couple… in the distance distribution chart, like, there's a… a long tail there, where Lodestar and Lighthouse seem to have… some issue in there. It might also just be in the data or something. I haven't had a chance to, To actually dig into it, to be honest. Yeah, I've uploaded all of this, all of the raw data, to the Fast Confirmation simulation repo that we've created. So it should be recreatable. Everything's, like, pinned to… the commit versions and stuff, so if anyone has any questions, or wants a hand, like, stepping through it, or is interested. Feel free to just send me a message.

Mikhail Kalinin: This is huge. Thank you, Sam. Yeah, that's just awesome work. On all this analysis. And the dashboard looks great. I just have one question I want to ask you to clarify what's not confirmed means on the dashboard, because it might be confusing.

Sam Calder-Mason: Yeah, so that's just essentially, like, it didn't get confirmed in the same slot. Because we don't get those consecutive Beacon API events, and we… it's not easy in Grafana to sort of reconstruct the parent state, I guess, or the chain, so we just… Yeah, flag that. Slot is not immediately confirmed. I'm sure if you, like, if you went back through it, though, it was eventually confirmed by the parent, or… Parent, parent, or something like that. It's just a limitation of the… Yeah, the way we're displaying this data at the moment.

Mikhail Kalinin: Thank you. Alright, yeah, that's great. Okay, any questions to Sam? Right, I have one organizational thing. Sorry, yeah, go ahead, Roberto.

Roberto Saltini: I had a question about the dashboard. Is the data coming directly? from Lighthouse, or say, like, if we wanted to, say, fix this issue of the non-confirmed blocks. What… is it something that can be… Done externally by… Yeah, we're checking the child-parent relationship, or… Or is the fix more complex? Because this data comes directly from Lighthouse?

Sam Calder-Mason: Yeah, so… We already have enough data in our ClickHouse cluster to, like. fix this properly. It's just really gross. And, like, if we were ever gonna do deep data analysis, we have it all there. The, like, proper fix, and all the nice fix, would be getting the Beacon API to event… to emit an event for all the blocks in between the tip and what was previously confirmed, but then… This is what the Beacon API PR before that was posted, is doing the opposite of, and, like, that makes sense, because it's another, like, thing for the clients to track, like, I think. I'm not sure how painful it would be. I'm also not sure… Yeah, like…

if it's that useful for other… for consumers of this event stream, if that makes sense, Roboto.

Roberto Saltini: Yeah, yeah. I was thinking, like, externally. In user, we need to check that. Oh, cool. That the parent… Like, the block is the parent of… what has been confirmed later on. So, like… And to fix, say, to, to… Fixed is non-confirmed. visualization issue. Who you need to… Basically… ugh. Speak out the data that's showing on the dashboard, not directly from the CL side of things, but we need to have Another layer in between the checks that… The particular goes back along the chain and confirms. Block that has not been confirmed before, right?

Sam Calder-Mason: Yeah, that's correct. I can probably, like, hack something together if this dashboard is going to get some serious visibility, and it's going to be, like, shown to, potential users of the fast confirmation rule. We can get something going, but, like, as a… outside of this sort of Grafana setup, and the long-term I guess, recommendation would be that we say to… implementers or users, like, when you get an event come through, you should be, like, progressing up the chain and hitting the Beacon API, getting those headers, and, like, actually walking. Back to the previously fast-confirmed checkpoint, so that you get the definitive list. Yeah, I'll hack something together tomorrow, and I'll post it in here. I don't think it'll be too bad, but it's… yeah, this is something, I guess, to keep an eye on for consumers.

Roberto Saltini: Thanks a lot. Yeah, I think it would be great, like, ideally… We wanted to have something like this embedded in the… In the website, because, you know, it's just a showcase of Live performers, so, yeah, it'll be great.

Mikhail Kalinin: Yeah, and I have a related question. So, if we introduce this metric. Let me just drop it into the chat. To speak at first confirmation rule slot, which is the most recent… the slot of the most recent confirmed block. And it helps, anyhow? to actually… Measure the delay between the current slots and the confirm block. I mean, like, if we have this metric, we could measure that delay. If we have a… something like current slot, metric as well, so… I, yeah, I'm not sure if clients have, if all clients have the current stuff. I know that.

Sam Calder-Mason: folder.

Mikhail Kalinin: head… yeah, sorry.

Sam Calder-Mason: Sorry, sorry to cut you off. I think for current slot, you can actually just sort of hack it together and throw a timestamp of Genesis in there, or something, and then, yeah, divide by… Slots, or epochs, or whatever the arithmetic works out. So that's, like, a little… that'll sort of work. The problem with, using the metric is most Prometheus Scrapers, like, have a 15-second interval or 30-second intervals for scraping, so… We're gonna get a bit of noise there, given that our slots are 12 seconds, and it'll be hard to, like. Yeah, sort of correlate it. I think… I think this metric is going to be useful if we, like, zoom out at a month level or something. Once… once this R2 method has got enough data, we can then go back and compare side by side, and it'll give us, like, a rough and…

Rough enough. Yeah, second data point. If that makes sense.

Mikhail Kalinin: Yeah, I see. Yeah, thank you.

Roberto Saltini: Personally, this event doesn't… It doesn't provide information about the child-parent relationship, so… 850, you know that this lock. It's been confirmed. It doesn't mean that the… What was proposed and is not before was being confirmed as well, because it might not be the parent of… The block paused in the current slot.

Mikhail Kalinin: Yeah.

Roberto Saltini: Rosie.

Mikhail Kalinin: comp.

Roberto Saltini: This wouldn't, I don't think, without serving this non-confirmed… strategy.

Mikhail Kalinin: Yeah, I see, I see what I mean. Right, and Mercy, thanks for providing a list of metrics that you… you have. Yeah, okay. If we are done with all this stuff, I have one organizational, Question, or… Remark? I'm planning to be on vacation in two weeks, so I will not be able to run this call in two weeks, and I propose to have an asynchronous updates in the chat, in the Telegram, instead of the call. For, for the, next time, and, meet in, in four weeks, 4 weeks. Unless there is a strong case for having a call, and there is a… somebody who is willing to actually host the call.

Roberto Saltini: I mean, I'll be happy to ask the call if there is a need for the call. But if there is no need, I might be… Go escape.

Mikhail Kalinin: Okay, let's, let's then have a skip as a default option. and, decide on this, Yeah, let's just have a skip the default option and decide on this, maybe, this and next week. Alright. Thanks, everyone. Let's rip up. Yep.

Sam Calder-Mason: Thank you.

Justin Traglia: Thank you. Bye, everyone.

Jihoon Song: Thank you.

Mikhail Kalinin: Mobile.

Jihoon Song: Bye-bye.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.