Fast Confirmation Rule Breakout #7

Transcript

Mikhail Kalinin: Hey, Justin.

Justin Traglia: Good morning.

Mikhail Kalinin: Yeah, good to see you. I was like, wondering if anyone will come.

Justin Traglia: I suspect a few people will come, but who knows? I was thinking, like, the Nimbus people. Thank you, Tom.

Mikhail Kalinin: LLC.

Justin Traglia: I just saw your comment on the, sorry about moving git safe execution block hash to fast confirmation.

Mikhail Kalinin: Oh, yeah, that's fine to move to this topic, yeah. I mean, like, yeah, okay.

Justin Traglia: I think that makes perfect sense. I'm gonna close my PR, and .

Luca Zanolini: governance.

Justin Traglia: some of that. Good luck. It looks fine.

Mikhail Kalinin: I don't know, I can, if you want me to handle it.

Justin Traglia: Sure. I'll just close it and let you handle it. I'll leave a comment here in a minute.

Mikhail Kalinin: Cool Yeah, basically, we didn't have the right place for this function before. Like, I mean, the proper one. It's why we had… Separate, spec document. But now it means… it seems like it's quite… At least, with the recent changes, it's quite attached to Do the false confirmation anyway. So… Yeah, makes sense to move it there.

Justin Traglia: I agree.

Mikhail Kalinin: So we'll just wait. For a few more minutes. And… End story. Hey, Mercy.

Mercy Boma Naps-Nkari: Hello? Good afternoon.

Mikhail Kalinin: In Azar?

Nazar Hussain: Good morning, good evening, afternoon.

Mikhail Kalinin: Yeah, good time of… Every Wednesday. We'll start in a minute.

Nazar Hussain: Sure.

Mikhail Kalinin: Right, let's get started So… Yeah, here is the agenda. So, I believe that. Hey, Jihoon. I believe that this, this call is gonna be quite, Sure, and we will start with the client… client updates. So, who wants to go first?

Nazar Hussain: I can start from my side.

Mikhail Kalinin: Please, go ahead.

Nazar Hussain: Yeah, after, merging the… PRR plus X. there were few tests that were failing, in my implementation, probably last two commits of the PR, which was expecting it, but… Those are some minor, rounding bugs for the integers, weight calculation, so… I fixed those, everything is ready. We have deployed them on our infrastructure for testing. It's been there for more than a week, and things are smooth so far. And, I'm now getting… A review from my team members, hopefully, once that is done, within a week more. we will merge, our PR, then it will be available for everyone to run under the feature flag. The only observation that I got from Some code reviews from my team members are…

That we don't have the specs test for the mainnet reset for it. Which… I understood, like, by design or by limitation of the performance. But do we have any plan in the future? To have… Mainnet. Spectache?

Mikhail Kalinin: Yeah, that's a good question. I haven't thought about it. We have these for… basically for performance reasons, mainly. Yeah, is… is it… Kind of, like, highly desired to have those tests.

Nazar Hussain: It's not highly desired, but it was one of the… observation that Kind of giving the… improved trust from the team that, okay, we have some specs for the mainnet and which are passing.

Mikhail Kalinin: Yeah, I see.

Nazar Hussain: So maybe if we… for… we can add, let's say. 20% or 10% of the specs. Or even less for the mainnet, for some edge cases, which are very vital for the fast transformation. Maybe that will be enough.

Justin Traglia: I mean, are there.

Mikhail Kalinin: That's true.

Justin Traglia: Does that affect mainnet and not minimal? I mean, are they not the same code paths with different… Configs, presets.

Nazar Hussain: There are but… of… Yup. We had been testing everything when it comes to the, like, come towards the, like, a… spec test, we always had something for the minimal and for the minute. So this is the first time we observed a case where we have Feature spec test, which are only for the minimum. And my team observation was that, okay, why is it like this, and is there a chance that we can have some maintenance vectors?

Justin Traglia: I wouldn't say this is, like, the first time. I think this is also how we do light client spec tests. Like, many of the… Like, client tests are minimal only. For performance reasons.

Nazar Hussain: Okay, I was not aware of it, then I will check it.

Justin Traglia: Mikael, how slow are they mainnet tests? Do you know if they would be able to complete within 6 hours? Because I… minimal ones take 1 hour.

Mikhail Kalinin: I haven't even tried, honestly. So… and, we also have some folk choice tests that are, For minimal preset only? Yeah, for the same reason. So… Since these are basically fork choice tests, like, quite similar to…

Justin Traglia: Yeah, my biggest concern is just that it would take too long to generate, because we have a hard limit of 6 hours.

Mikhail Kalinin: Yeah, I see. Yeah, I'll try, yeah, that's a good, that's a good call. I'll try to… To the place to generate some of those, you know, some of the, A subset of those tests to make them work with mainnet Reset as well.

Justin Traglia: One other thing to keep in mind is that we're already pretty close to our limit, for, like, our 90 tests. Like, we're at, like, 5 hours, so if… If doing this pushes it over the limit, that would be a no-go. Unless they were, like, a separate, action, which it could be.

Mikhail Kalinin: Yes. Yeah, that's… that's… that's to be… Yeah, let's… let's… let's go… let's finish with the client updates. We have a testing stuff and agenda, so we can discuss this more, because I was… yeah, I have some test, inputs as well.

Nazar Hussain: Yep, all content from my side.

Mikhail Kalinin: Cool. Nazar, you mentioned that you spin up a node with this code running? Or did I miss this too?

Nazar Hussain: Yeah, we have more than 5 moors which are running this thing.

Mikhail Kalinin: Oh, that's great. Did you… do you have any metrics? Do you have any metrics or, you know, any measurements that you have? Are doing with your nodes.

Nazar Hussain: Yeah, yeah, we have a couple of metrics, so I will then… extract everything out. We have different nodes in different configurations. One is running in the mainnet, one is running on something different configurations, so I will put them all together on my PR, so… and then I will share with you guys to look at it.

Mikhail Kalinin: Awesome. That's… that would be, really helpful. Thank you, and congratulations on, yeah, on… Getting quite close to… to releasing this feature. That's great.

Nazar Hussain: Thank you.

Mikhail Kalinin: Any other… yes, sorry? Okay, any other client updates?

Mercy Boma Naps-Nkari: Yeah, on the grinding side, we have, like, after the specs PR was merged, while running the FCRM factors, I confirmed one of two pre-existing issues, and then… The first one is, our relative epoch helper rejects lookups more than one epoch from the state, so while the spec needs to epoch back for committees, the… Then the second one was, related to unrealized, justified drips on fork chains, so it's still, like, a hypothesis for me, so I'll need to add, reapply the head state fix, and then add some tracing to figure out how, the previous epoch, to confirm which one was actually, like. driving the over-confirmation before touching any code, so once I get that done, I will update my PR and then ask for team reviews. Although, there's kind of, like, delay in terms of reviewing FCR code for grinding, that's… there's some other things on priority list, so…

But if someone else, from the specs side can also help review, that would be great.

Mikhail Kalinin: Can you please drop the PR, link to the PR, to the, probably, first confirmation group on Telegram?

Mercy Boma Naps-Nkari: I don't think I'm on the group.

Mikhail Kalinin: Oh, really? I'll add you… let me know your Telegram handle, just pop it in the message.

Mercy Boma Naps-Nkari: Okay.

Mikhail Kalinin: I'll add you there. Right, so, you mentioned the case where, the committees are requested from Not from the previous epoch, but from the epoch before the previous one, right?

Mercy Boma Naps-Nkari: Yeah.

Mikhail Kalinin: Yeah, and this has some difficulties on your end.

Mercy Boma Naps-Nkari: Yes. I'm trying to figure out how to tackle it, so I… It's… by Friday, I'll probably have something.

Mikhail Kalinin: Yeah, I see. Yeah, it's an edge case, And we have a test for this hatch case, so… Cool, thank you so much for… for your update. Yeah. So… The other… the other updates, from… from the client side that I'm aware of, he's, okay Yeah, I'll send you an invite link. to the first confirmation group. So the other updates are, like, the Nimbus has, released this feature… Not… yeah, it's not… it's enabled by default in the release they made, so it's not, like, behind the feature flag. And the issues that they have faced with is basically

The issue related to Nethermind, I'll just… Thank you. link you to the PR. So in other mind, there is a problem with the safe head. So the implementation, their implementation is not compatible with the engine API spec. What I dropped in the chat is basically a fix for this issue, Nethermind, but it's not yet released. It has been merged, but there were releases after this PR was merged, but yeah, it's not… this fix is not yet. Released by Nethermind team. So… And then Nimbus basically rolled out another release, where they have basically disabled the FCR. So they have a version with FCR, and they have a version without FCR. And once the… this feature will be, once this bug will be fixed by Nethermind team.

Nimbus will… bring the, FCR. Enabled by default, back. Yeah, so that's… that's the story. Of these bugs, so just, you know, it's… yeah, be aware of that issue exists. When you probably run your implementation with Nethermind as EL. Alright, and yeah, hopefully, hopefully, other clients, I haven't heard from… Prysm and lighthouse. Lately, so we'll see. Hope they will just provide some fine updates in the group. Yep. Cool. Anything else on the client side? Otherwise, we will move on to the spec. And testing. All right.

Thank you so much, guys, for the client updates. So, this pack… We have this PR merged, the main PR. It's not Gloas compatible. That's one of the things to fix. In the… Short term? And, we also have, Have tests, like the, major… you know, say, core, test ensued, has been also merged, within the spec PR. But there are new tests to come. So my, my current, focus is on the Gloas. So what's, to make the, fast confirmation rule spec. Work with Gloas. I'd like to at least try to do some adjustments in the fork choice. Abstraction layer. To make it easier to work with, like.

Basically, that the fast confirmation rule can be can just use the folk choice abstractions, regardless of whether they are from GLOSSS or from Phase 0. So this will be abstracted away by the fork choice specification. I'm currently the work in progress on that, and that's gonna be a big change, and I'm not sure how this change will… play out, because we know that everyone is focused with GLOSS and the ePBS, and having this kind of change is much easier to have it earlier than later, but now we are quite deep into the process. But that's a different stuff. If that will not work, like, these kind of changes will not work, then we will have to make the fast confirmation rule.

spec for, basically, fork-aware, which is also doable, but… You know, less pretty. Yeah, also on the testing side. Justin mentioned we have a limit, but we still have some tests to write, some edge case tests, mostly. So we'll see. I'm planning to work on those tests after this pack is completely done. So that's the spec and testing updates. That, we have for today. Any questions?

Justin Traglia: Yeah, I have a few questions. The changes to fork choice, like in Gloas and previous forks, wouldn't actually change behavior, correct? Just, the way things are organized and helpers, and etc.

Mikhail Kalinin: Yeah.

Justin Traglia: Okay. Yep, that sounds fine to me. If, for whatever reason, it is, like, a really big change, I would suggest splitting it up into multiple PRs. That just makes it a bit faster to get merged.

Mikhail Kalinin: Right.

Justin Traglia: doesn't…

Mikhail Kalinin: Dude.

Justin Traglia: If it doesn't make sense, I understand, so… If only if possible.

Mikhail Kalinin: Yeah, yeah, I see. Yeah, it's… it's gonna be a change, like… We're changing, like, multiple parts of this pack at the same time. It cannot be done. Separate, unfortunately. I mean, like, broken down into separate PRs. And, yeah, there will be updates to tests as well, Like, it's not… Strictly speaking, there are no semantic changes, but there are some Strictly speaking, there are some semantic changes to clause, but they do not affect the behavior, the intended behavior. So, it's just… yeah. I'll try to highlight all those changes in the PR and, you know, do some equivalence comparison. Of what was before and what is after these changes, so to make it easier to reason about.

Okay.

Justin Traglia: This sounds good. Looking forward to reviewing it. And as for testing, on the minimal side of things, I think we have a lot of headroom, so… Previously, before FCR tests, it took maybe about 10 to 20 minutes to run everything, and then with the FCR test, it takes about an hour, so you would have an additional 5 hours of testing headroom. But… We do use that for, Like, quick turnaround tests, so it does kind of hurt to have them really long.

Mikhail Kalinin: Yeah, I see. Okay, cool. So… For mainnet tests, we have a little bit of a room, right? So… so we can include some core… more important tests into that. I mean, like, the fast confirmation rule tests.

Justin Traglia: Possibly. Yeah.

Mikhail Kalinin: Okay. Alright, if there are nothing about spec and testing any more today, so… Let's move on to the research updates. Bert, Luca, do you want to… Do you have any updates for today, maybe about the paper, or anything else?

Luca Zanolini: about the paper, not really, in the sense that I think it's, ready, I suspect. I need to see if there are a couple of feedbacks that are not yet addressed, but other than that, I believe that it can also be published very, very soon, like in… I don't know if… what Roberto thinks, but… Should be ready to be published, probably. I haven't looked at it for next… for the past couple of weeks, because I was focused on something else, but the last time I checked was kind of ready. But maybe I missed something, if Roberto has something to add?

Roberto Saltini: No, we just need to go through the latest feedback. From a guide, actually, so… Yep.

Luca Zanolini: Perfect.

Roberto Saltini: I mean, that is a much, bit of… But not substanti, substantial changes. So… Yep. Won't take long.

Mikhail Kalinin: Cool. Thank you, guys. And, kind of… Are there any open questions? Or discussion? Stuff. Alright, so I have one thing, yeah, Mercy, please go ahead.

Mercy Boma Naps-Nkari: Sorry, I have a question. Is there, like, a plan to have, all the clients I have implemented so far to, like, run them together and then, sort of. Collect data on it to see how it actually performs. Collectively.

Mikhail Kalinin: That's… yeah, that's a good point. We would like to have these, such measurements, and it would be great to standardize metrics, basically. So… We can… Easily compare, you know, the measurement, the performance of the clients. This probably will help us to find some, you know, obvious, bugs, in the implementation, or just, you know. Yep. But, yeah, there is a question with the standardizing metrics. And, we need to do some work in that, but Yeah, what I want… what else I wa… Yeah, Marissa, do you have anything else to add on that?

Mercy Boma Naps-Nkari: I don't know if we already have metrics, something we want to, like, instrument, like. specific things that would help, so that I can also, like, include it on my own,

Mikhail Kalinin: communication. Yeah, so what, I, I probably… yeah, that's a good call, probably it's time for me to create… to at least open up a PR to do the metrics repository with, like, most meaningful FCR metrics. Thanks for bringing this up. What we basically want to measure is, the main metric would be the Delay between the current slots, so the distance between the current slot and the most recent confirmed slots. like, the slot of the most recent confirmed block. That's the main metric. We would like to see. Also, the other ones are kind of… Re… reworks of confirmed block. Maybe restarts… You know? And, yeah. restores, like, I mean. The restart, when we start from the greatest unrealistic point, and, also, maybe…

Measure where a confirmation failed or not, something like that. And also, like, how much time, basically, the runtime of the algorithm in milliseconds, something like that. I think all should fit into Grafana metrics. Yeah, and… Right, so… I'll… do this. I'll do this in the next couple of weeks, open the PR with the metrics, so there is an idea of what we want to measure. And share it. on everyone.

Nazar Hussain: The, yeah, the one, the metrics that you just mentioned. I think these are all covered already by Lodestar. So you're tracking the fast confound, like, confirmed block slot? And we already have the finalized slot. So, we can calculate the distance, in the Grafana dashboard.

Mikhail Kalinin: Awesome. Yeah, that's… that's really great to hear. All right, so… I wanted to share that there is a small Pick an API change, which adds the fast confirmation events to the event stream. it's pretty straightforward, I mean… She just basically has to… Two fields, block and slot, which is… The root and the slot of the most recent confirmed block. Yeah, so this PR has been merged, and, keep this in mind. Maybe… It makes sense to add these from the very beginning, and yeah. Include this into implementation as well. This, this kind of thing is basically, We know that the consumer can basically go to EL and request the most recent confirmed block hash, but

Potentially, some consumers, may need… may require the actual beacon block route that is confirmed, not the, execution payload hash. And this will be, like, the source where they could get the… that, we can block route. By basically listening to the event stream. All right, that's… that's it about updates from my side, and one last thing that I have, just to think about how… how… when… when would make sense to have a next call? Is it, like… I think we'll probably have one in a couple of weeks, like, as the… According to the cadence that we had before the interop. But then, let's just, you know, think about how we want to proceed with this breakout room.

Okay, and discuss these next time. Alright. Yeah, that's… that'd be great, Jeff. It could be on a dashboard. Right?

Jihoon Song: You mean, like.

Mikhail Kalinin: Like, you…

Jihoon Song: carafe band.

Mikhail Kalinin: nops.

Jihoon Song: Err Yeah, I mean, like, Dora, the Explorer, that we use. When you run some DevNets, so we can modify that Explorer to show the most confirmed block.

Mikhail Kalinin: Yeah, that'd be great, but, And you will need a client that… the door, I guess, for information from to basically run the fast confirmation rule. Yeah, but that's great.

Jihoon Song: Yes, yes, exactly, yeah.

Mikhail Kalinin: Alright, cool. Okay, thanks, everyone. And see you in a couple of weeks.

Nazar Hussain: Thank you so much.

Mercy Boma Naps-Nkari: Thank you.

Jihoon Song: Thank you.

Roberto Saltini: But… but thanks.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.