All Core Devs - Execution #219

Transcript

Tim Beiko: Good morning, Matt.

Justin Florentine: Good morning, y'all.

Tim Beiko: Okay, I think we're probably good to get started. Josh, do you want to move us over to the livestream?

Akash | ECH: Hey team, I'm helping with this team today.

Tim Beiko: Oh, thanks. Hey, Akash.

Akash | ECH: Hello.

Tim Beiko: Let me know when we're live on YouTube.

Akash | ECH: Yeah, I'm ready to go live. Give me a final…

Tim Beiko: Yep, we can do it. Okay. Are we good, Akash?

Akash | ECH: Yeah, we are live.

Tim Beiko: Okay, awesome, thanks. Welcome, everyone, to ACDE number 219. I have a bunch of things on the agenda, as always today. We'll chat about Fusaka, how the nuts are going, especially the non-finality testing we've done this week. Then, there's some configuration PRs, for, yeah, that the PentOps wanted to review, and we can chat about what the next steps should be from here. After that, I wanted to check in on, Holeshky, so we discussed shutting down the testnet. at the end of September, so I want to make sure that this still works for everyone. We can chat a bit, about some EL gas limit bottlenecks, continuing the conversation for Monday, and then discuss some, updates around Glamsterdams and how we should approach, EIP discussions for the fork.

But to kick us off this PandaOps, I want to give an update on, the current DevNets, where we're at, and how the testing has been going this week.

Barnabas Busa: Sure. Dentistry has been going through a non-frenalty period for a little bit longer than we anticipated. We wanted to do approximately 2 days first, and now we're hitting day 5. We've been trying to recover the chain, and we have encountered a bunch of syncing issues that a different client had. There are lots of fixes coming in, and we've been rolling those out as they come. We are approximately at 50% to 55% participation now. I have triggered a few exits from Nimbusia to try to gain back the final tea. Those are being processed slowly. Nimbustia have been struggling to keep up with the non-frenalty period, and there has been quite some memory issues there. That's why, we decided to…

Kick some, ventilators off. Regarding notes that are, struggling to sync up. Lighthouse, seemed to be stuck syncing, made many different ELs, Lodestar seems to be stuck syncing as well. Brandon seems to have an issue, and a few technical nodes as well. Most of the Prism nodes are, pine, and, Most of the technicals are also fine, but a few of them seem to have some, No more issues. Regarding the EA, there also seems to be a few bugs reported by Eragon and Bezu. That they are investigating.

Tim Beiko: Thank you. Any of the client teams want to chime in or share more context?

Barnabas Busa: there would be another update, so we did a non-finity test on NFT demo.9, which was basically, pre-FULU, with using all the latest, stable releases from each EL and CL, and we tried to replicate the same, non-funately, we were offline for, I think, a day and a half, and we brought the notes back online, and the chain has recovered within, I think 2 hours.

Tim Beiko: Nice. Yeah, any client teams, or anyone else have… context I want to share on the cognuts?

Barnabas Busa: One of the main differences that could be, I just want to make sure that this is set up. I used the… on the… Photoko.net 3, the initial way, how we try to recover the chain is just, removing the EL database and resync using the CL database, without checkpoint syncing, and hoping that all the CR clients would just reorg, back to the Back to the inbound block. And on NFT.NET9, we have actually wiped the database of the EL and the CL, and the recovery was very fast there. And on NFT DevNet9, we seem to have, about 80% participation. Yeah, I think we also had 80% participation before, and there seemed to be a few Ergon nodes,

I'll find her.

Parithosh Jayanthi: Yeah, the main reason we made the change for NFT DevNet 9 was that there was none left on the network to provide the invalid state, because we did the resync in kind of all at once. So there was just no one… no peers to get the invalid chain to then claim it was invalid.

Tim Beiko: Got it. So in terms of next steps for the DevNets, obviously we want to fix, the issues that we found, and the CL teams are working on that. Do we… do we think we want to have another DevNet after this? I assume we at least want to get this, non-finalized DevNet back to finalizing, but yes. After that, would we… would we move forward and, launch… launch DevNet 5? Do we want to do that in parallel? .

Barnabas Busa: So, ideally, DevNet3 should recover to close to 100%, and then we would want to do another non-finity test, and see if we can actually recover within a reasonable period of time. Just to make sure that we have actually ironed out all the bugs. Launching DevNet 5 with known syncing issues, Probably not worth it.

Tim Beiko: Yeah, that makes sense. And do we think, most thinking issues should be, like, at least have tentative fixes this week? Is that something we think we can, like, yeah, get live on the DevNets today or tomorrow and see it progress over the weekend, or do teams feel like they're going to need more time to address those?

Emmanuel Nalepa: I can speak for PRISAM, it should be okay for this week.

Tim Beiko: Okay, Lighthouse is saying goodbye mid-next week. Any other teams?

Parithosh Jayanthi: Maybe one thing also worth mentioning is, now that we have our… we have, like, a malicious getth image, as well as another manned image, that we can trigger this fork very easily, and cause non-finality and everything on DevNet 3. We should be able to reproduce this as often as clients want, so if we have one client with a fix, we can already deploy it and already try out their fix, and we don't have to wait for all clients to be ready before we do. Before we do another test. And same thing goes for, DevNet 5. We've sped up, we've sped up our Ansible stack significantly, so it should reduce the amount of time we take to spin up, like, larger networks now.

Tim Beiko: Awesome, thanks. And yeah, so Lifehouse is saying they should probably have a fix by mid-next week. I'm curious how the other CL teams are looking, like, So I guess this would mean we probably… don't launch DevNet 5 until, like, at least a week from now, once we've had time to actually deploy these fixes and potentially see the network recover. Is that relatively the timeline that themes think makes sense, and also, can we handle the DevNet 3 being non-finalized for another week? And will that… Compound problems somewhere or another.

Okay, so Taeku… Okay, so it seems like Tegu, Lodestar as well, are gonna have fixes for next week, so, like, realistically, we're not gonna have all the CL fixes to get DevNet 3 finalizing by this week. Yeah, the only concern I could see is if things get significantly worse on the DevNet, because it's not finalizing, for what will be, yeah, probably, like, 10 days by the time we have the fix, then we may want to think about other ways of testing these… these fixes, or, like, yeah, whether or not it's worth, recovering DevNet 3 if it's… if it's, if it's fully degraded. But we can discuss this further, I think, on the testing call Monday and, on Thursday, if,

Yeah, if we need. Barbara, let's say we can finalize the 3 today, how would that work? Would you just make all the, or, like, two-thirds of the steak PRISM?

Barnabas Busa: So maybe, like, we have a bunch of nodes that are syncing, but the sync times are just so long, especially during non-finance period. Forward syncing takes a bunch of time. So, we can see that some nodes are still making progress, and the participation is growing. So, hopefully by the end of today, we're gonna have a lot more nodes that are synced ahead, and we're gonna be able to reach 66%. And I think that we could possibly get back to 100% participation by End of day tomorrow, hopefully, or very close to 100%, with just, checkpoint syncing and, removing the databases and forcing them that way. And ideally, yes, the thinking, the thinking fixes should be tested again, and if everybody says that they can be done by next week, by mid-next week, then maybe we can trigger another non-finality, next week, and in the end of, next week, we can then, schedule DevNet 5, if everything goes smoothly on DevNet 3.

Tim Beiko: Yeah, that sounds like a plan. Alex has a question asking, do we have any theories on why sync is slow?

Alex Stokes: Yeah, this might be a bit of a parallel thread to the present topic, but I am curious.

Tim Beiko: Okay, guess not. We can maybe discuss this when we, have had more time to look into it on Monday or next Thursday. Anything else on the non-finality testing? Otherwise, Barnabas, you had the four PRs that you flagged, Yeah, to, to update the blob schedules on, a bunch of, the testnets. Yeah, do you want to give… Reach out to those?

Barnabas Busa: So, that, mainly Nethermind, is the one that have a big change in how the blob scheduling is handled. And ideally, we want to merge all these in by end of day tomorrow, and this is required for us to be able to do shallow forks on OSCI, Sepolion, Mainnut, and UDI.

Tim Beiko: Anyone from Nethermine had a chance to look? Can potentially give a quick… 1 or minus one?

Barnabas Busa: Nevermind already approved it, but I would just like to have a few more approvals from different EL teams, because I know that some clients might rely on these files. So I just want to make sure that they are aware of this change.

Tim Beiko: Okay, so yeah, people can have a look, and I think if in 24 hours you haven't heard any complaints, we can go ahead and merge them.

Barnabas Busa: Sounds good.

Tim Beiko: Okay. Bye. Okay, next up, so Alex had posted an updated timeline for Fusaka, obviously, you know, based on the current state of DevNets, this might be a bit delayed, but yeah, Alex, do you want to give a bit more context on that? Curse, also…

Alex Stokes: Thursday, sometime.

Tim Beiko: type to review, yeah.

Alex Stokes: Yeah, I mean, I can walk through it, So, essentially, just trying to track everything so that we can hit a pre-dev Connect launch date for Fusaka. Basically, what we had discussed kind of before the past few days with the sinking issues, we would have, trunk branches together by the end of this week. Demet 5 next week, do some analysis, use that to figure out how to set up Pulaski and Sepolia. We'd do those forks mid-September, or we'd have releases mid-September, do the forks later in the month. That would then set us up to do releases beginning of October for Hoodie and Mainnet. do hoodie, do mainnet. You can look at the dates here, and yeah, I mean, the thing is,

this is all kind of on hold with the present issues with DevNet3, so yeah. I think what we want to do is focus on DevNet 3, If we get DevNet5 up next week, that'd be amazing, because then these dates don't change too much, and we can kind of stick to the schedule. But, yeah, it's… Slightly premature, just given… given the current status, so… I think we all know what we need to do to keep moving forward.

Tim Beiko: Yeah. Thanks. Anyone else have comments on that? this.

Emmanuel Nalepa: Yeah, so basically on Prism, we have… Yes, almost everything in the developer branch. Which is our untappable branch, but we continue to run the DevNet tree on the PRDAS branch, because it's quite simpler. When you have a new bug detected, we just… push the fix on the PureDust branch, which does not require any review, and when it works, we do a real code review when we push it in develop branch. So basically, develop branch and PureDAS branch differs only on the new bug fix seen in the DevNet. But, yes, We have the backfield, which is not yet merged in develop. But, so else is… Every… everything else is… is matching.

Tim Beiko: Nice fix. Yeah, so there's a comment saying we don't have one month between the first release and the test sets, and Yeah, so we would have… the plan is to have one month between the mainnet releases and mainnet forking, but then we would have shorter, shorter timeframes for the testnet forks, and notably, Haleszhki would be kind of the shortest, because we also plan to deprecate it shortly after the fork.

Lightclient (Matt Garnett): I mean, isn't that what we agreed, though? That we would have one month before the DevNet forks?

Tim Beiko: After a picture? You mean testnet?

Lightclient (Matt Garnett): Or sorry, testnets. Yeah. Like, the complaint before was that we were making the Sepolia release, and then Sepolia happened, like, a week or two after, and same thing with Koleski. And we kind of told the… The application companies, that we would give them extra time.

Tim Beiko: I thought this was about maintenance?

Lightclient (Matt Garnett): I mean, it meant that we always have one month.

Tim Beiko: Yeah.

Lightclient (Matt Garnett): Like, the complaint last time was not about Mainnet, it was about Sepolia. I mean, I specifically remember Optimism being frustrated that they need to do a lot of work now, based on… they're waiting for the client releases to update their software, and then we fork the Sepoliad network one or two weeks after we make those releases.

Tim Beiko: Yeah. And yeah, Frederick has this, linked it, and, like, yeah, this did say 30 days before the first test set.

Fredrik Svantes: Yeah, yeah, it sets in the process there that we should have 30 days between the first testnet and when the client's release is ready.

Tim Beiko: Okay, so I guess if we wanted to do that, then yes, we would obviously have to modify, the schedule quite a bit. It is something, yeah, we can do. We should probably get input from all of these L2s specifically as well, because, they're also the main consumer of the blobs, so… Yeah. They may…

Lightclient (Matt Garnett): It's just like, how many times can we ask them, you know? It's like, we just keep asking them until they don't respond, or they are tired of us asking. Because they kind of came, and they told us what they wanted, and we haven't even had a hard fork since they told us what they wanted.

Alex Stokes: I will point out one difference is Pectra had many things that, like, impacted them directly, and that they needed to, like, actually change, whereas Fusaka doesn't have things like this.

Barnabas Busa: I mean, they have to… Calculate the proofs, so that is a… That's gonna fall on them.

Alex Stokes: roofs.

Barnabas Busa: the blot proofs?

Alex Stokes: Doesn't the EO handle this when you submit the blobs?

Lightclient (Matt Garnett): Do ELs handle this when people submit the blobs?

Alex Stokes: Yeah, like, if I give you a blob over RPC, then the ELs handle this stuff, right? So my point is just that the things that we change at Fusaka are, like, fairly transparent to all the people we were just talking about, whereas with Pectra, that was not the case, so I could see how, you know, I think it makes sense for them to be like, okay, with something like Pectra, this makes sense. We need time to update, you know, our code. Whereas, there's not that much here. So there's less To have so much time.

Lightclient (Matt Garnett): I just don't think that because we decide that there's less things for them to do, that we get to change what the agreement was before.

Tim Beiko: grab pullers…

Potuz: Rollups typically base their main node on the… as a fork of Geth, or as a fork of whatever L1 client is. And this is the main reason why they required a lot of time between having a release and they themselves having a release that supports the same features of the fork. I think this is the… one of the reasons why they requested this extra time, and this is going to be independent of what kind of features they want to ship or not.

Tim Beiko: Thanks, Oscar.

Ansgar Dietrichs: Yeah, I just wanted to say that, like, I think we have to make… be careful, though, to not make this mistake of, like. weighing the cost imposed onto a few very visible, entities higher than, basically, the cost distributed more across, like, a larger set of people. I think there's costs both to a two-accelerated rollout, like. we just are discussing right now, for L2s, mostly. And then there's, of course, also a cost, for example, if, say. the outcome here would be, can we still ship the fork by the end of the year, or slip into next year? That's, like, an extra month of delay, that also delays kind of, like, Lamsterdam and the follow-on work. I think the problem is just that the people affected there is a much more diffuse set, it's basically, like, everyone in Ethereum, but only a little bit. And it's hard to weigh, like, a few people, like, strongly affected, whereas, like, a lot of people a little bit affected, but I think it's important that we don't

Only focus on one side of that, of the picture.

Lightclient (Matt Garnett): I mean, we made a very unambiguous agreement with Rollups and other participants at the time with Pectra that we were going to give them 30 days from a client release to a testnet. I don't understand why. This is about the third time for Fusaka we've had to come back and, like, re-agree with this.

Ansgar Dietrichs: Can you…

Lightclient (Matt Garnett): And a week ago.

Ansgar Dietrichs: Maybe in the governance process where we did that? Like, the meta-EIP for that, that was on Universal.

Lightclient (Matt Garnett): spreadsheet.

Ansgar Dietrichs: I think it was mostly…

Tim Beiko: Yeah, Frederick's…

Lightclient (Matt Garnett): lifted it.

Tim Beiko: Doug, yeah.

Ansgar Dietrichs: Yeah, like, you talk, I understand, but… but, like, it was never, there was no…

Lightclient (Matt Garnett): It's in a folder called Processes.

Ansgar Dietrichs: I understand, I understand, but what I'm saying is, like, this is a best-effort type of commitment, and, like, if there are costs, the circumstances arising, like, I don't think it's a guarantee, that's… I agree with you, like, we should try to stick to this as much as possible, but we should also try to, like, not let the Ethereum community down. Like, it's… there's a trade-off here.

Lightclient (Matt Garnett): The Ethereum community wants all these systems to work, and they don't want all the applications and the Layer 2s to roll out half-baked updates. And they… L2s and apps asked for 30 days to give them the appropriate amount of time to securely upgrade their software.

Tim Beiko: I'm Zoop Podus.

Potuz: I'm just stating the obvious here, but if we do have fixed 30 days for testnets, fixed 30 days at least for mainnet, and at least 30 days for DevNets, it's really not realistic to have a 6-month work cadence.

Tim Beiko: I don't think the definite thing is correct, but yes, like it… like, yes, if we have even 30 days for those two, and obviously they have to fork and go and, you know, go smoothly, that's probably 3 months you add to the process, between, like, putting out the first release and, And, and having Mainnets go live. Death.

Lightclient (Matt Garnett): This says… it does say 30 days before the first testnet, so, like, we can pipeline the testnets a bit. It doesn't have to be one month between each testnet.

Tim Beiko: Oh, yeah, yeah, for sure. And then… but then I think we also want at least a week between each of these testnets, and even that, you know, if there is an issue, a week is… not… like, you know, it's doable to fix the issue within a week, but it's kind of a rushed timeline. Yep.

Fredrik Svantes: I think we… we decided that it should be 14 days. Apart from each upgrade.

Lightclient (Matt Garnett): Alright. So, fastest possible is two and a half months? Or I guess the two 14 days add together.

Tim Beiko: Before mainland?

Fredrik Svantes: I think it's gone 37 days. Don't matter if the process is followed by the book.

Parithosh Jayanthi: I think one of the differences between earlier folks is also we're requiring almost every client, if not all clients, to have all of their changes in their trunk branches before we start talking about testnet releases. I don't necessarily think this was always the case. That's also another location where we're adding, like, a few weeks, which in the past we wouldn't have.

Lightclient (Matt Garnett): Sorry, I don't know if I might follow that. You're saying we didn't require in… for a testnet release?

Parithosh Jayanthi: Exactly, for a test methodologies. In the past, it was very often that you'd get an RC release from a branch.

Lightclient (Matt Garnett): Okay. I feel like for, at least for us, we've always… for a testnet release, we've made an actual GoEthereum release, and so at that point, the… New code is available for people to begin integrating with? I guess some other clients did that differently?

Tim Beiko: Okay, so I think… there's… I guess… two paths forward. One is we kind of base the, release schedule just on this process doc, and, you know, based on the previous discussion around DevNet 5, if we assume DevNet 5 goes well, we fix these finality bugs, we could set the dates for the first testnet, and kind of, you know, just schedule it out from there. I think if we want to have a shorter, I think if we want to have a shorter schedule, then, like, yeah, maybe it's worth trying to get explicit feedback from, these L2s and other info providers again, potentially opening up a PR to Frederick's doc and trying to get, Yeah, trying to get, like, some…

I guess, input from the community there. realistically, we don't have to make this call today, like, we… we still have time, because we're still fixing DevNet 5, but I think over the next week, if, If we… if we could actually get, If we could actually get, like, some roll-ups to share their preferences, even though we've asked them in the past. I think it's probably valuable, and they are kind of the main stakeholder, being, affected here. Yeah.

Alex Stokes: Yeah, so I can make a version following the stock to the letter. I will also reach out to some roll-ups and try to understand if they care. Let's see…

Tim Beiko: And I think if we can get, like.

Alex Stokes: into other parties beyond roll-ups?

Potuz: Staking pools would be affected by the KCG proofs on the… on the sidecar, perhaps? I don't know.

Alex Stokes: Meh…

Potuz: I don't know. Like, restaking tools, perhaps, I don't know.

Alex Stokes: Yeah, I'll reach out to some people.

Tim Beiko: Okay, yeah, I think if we do change it, we should just open a… we should, like, actually open a PR to this process doc and make it clear what the change is. Yeah, and that's also, like, a concrete place we can send folks to, like, voice their… Preferences.

Lightclient (Matt Garnett): I just think it's a really bad precedent to… keep letting decisions change. Like, we've talked about this in many different contexts. We even talked about this exact situation about a month ago, when we started putting out the timelines for Fusaka, because we had forgotten and then remembered about the agreement we made. And we said, okay, we really need to think about this, and, you know, we need to get the releases out mid-August as the only possible way to get a pre-DevConnect Lusaka. And now it seems like we just keep kind of changing what our agreement is to fit to this arbitrary deadline that we've imposed upon ourselves. And I just don't think that's fair for the people who are not full-time people part of this process, to have to again and again come and say what their preferences are.

Tim Beiko: I think that's true, but I also think that there is, like, a… a big part of the community that does want Fusaka at the ship, and, you know, they don't come and Say, like, oh, we want the fork to happen in, you know. one week after the testnet release, or something like that. So… Yeah.

Lightclient (Matt Garnett): I don't think we should choose timelines based on what the community necessarily wants. Like, the people who are shipping the software said they want 30 days to deliver high-quality software that the community is going to use. So… Until that… until we, like, revisit that decision with all of the people involved, there's really not that much point of arguing about it.

Tim Beiko: Yeah, okay, that makes sense, but I think we should at least… Double-check with these, like, roll-ups and infra providers that This is still their preference, because the same way we had forgotten about it until it came time to do the releases, it's not clear to me that they all have this expectation, and Yeah, like, I think if this is what everyone wants, then great, we can stick with it, but we… yeah, we should make sure that this is actually what they want, and I'm not 100% convinced that they… they would still stand by this. Yeah, so I don't think it's, like, a super high cost to check this in the next week or so. And we can follow up with, like, yeah, who was involved in the original discussion.

Justin Florentine: But just to be clear, the decision is to stick to what we have already agreed upon, meaning the 30 days outlined in the document, right? We assume that what we currently know is still true until we hear otherwise. Is that right?

Tim Beiko: Yeah, I mean, we can prep the schedule, like, we haven't chosen either way yet, but, like, yes, we should prep the schedule with what's in the document, and then in parallel, check with, like, the stakeholders that are affected. If this is, like, still their preference, and… If so, then we would stick there, but I think if we get other signals, we should also propose, like, some alternate schedule, and do that also by, like, changing the process doc to kind of reflect these updated preferences. Okay, I think, yeah, we can follow this up async, Anything else on the timelines people wanted to… Discuss.

Okay, related to this, so we did say we were gonna deprecate Poleski, after going live with this fork, and… that would mean, you know, probably shut it down sometime, at the end of September. We want to put an announcement up for this, but before doing so, just wanted to check in here if anyone has any reasons why they would need the testnets longer, or think that we should yeah, somehow push that out. Otherwise, in the next week or so, we'll just put up an announcement saying that, Holeshky will just be shut down a couple weeks after, Fusaka goes live on it. Okay, no one fighting to keep Holeshky longer. So yeah, expect a blog post in the next, week or so with, an update on that.

Anything else around Fusaka people wanted to discuss before we move on? Okay, next up, so on Monday, we discussed, some of the gas limit work, we were… we were, trying to see whether we could raise to $60 million before Fusaka would go live. Some of the EL clients were looking into, some performance bottlenecks, and… and trying to see what potential paths were to addressing those. So I just wanted to check in and see if anyone had updates, Yeah, on that work generally, in terms of just… Yeah, potentially moving to 60 million before, before Fuseka. Or at least, yeah. Getting somewhere there.

Luis Pinto: I can add a few words for Basu. So we were a bit hammered with, the mods and division, and so we are looking at those, opcodes, and we have a working implementation for mods and DIF. That we will most likely want to test in a few days. Which is much more performant than the previous one. So yeah, we're making progress on that. On that front.

Tim Beiko: Awesome. Thanks for sharing. Anyone else have updates on this?

Ben Adams: Question around signaling, because it does take a while between signaling and the gas going up. What would be the, I suppose, the process or the timeline? We… Do we wait until everything's happy and confirmed, or do we… Signal a bit sooner. Everything inside.

Tim Beiko: I think we should wait until no client has, like, major concerns, because going from 45 to 60 like, in terms of actual block updates, happens pretty quickly. I think… if we… what helps also with the signaling and just the adoption is if it's actually included in a release. So, one thing that could be nice is if, you know, if we're confident in going to 60 before Fusaka goes live on Mainnet, then, we could potentially just have the clients default to 60 in, say, the first testnet release, so that the first testnet release, you know, if people run that on mainnet, it defaults to 60 million gas, So I think that's an option, but I wouldn't want to just… I don't know, if Besu is… if Besu still hasn't, tested, like, their improvements, I don't think we should, like, soft signal and kind of force Basu to catch up that way. Yeah.

Yeah, on to go?

Ansgar Dietrichs: Yeah, I think, maybe, concretely, I think it's good if our process basically, as you said, right, that, like, the threshold should be that every client has a release out that we are comfortable can handle, the kind of the new… gas limit that we are signaling, but I think there's no need for us to wait for any specific adoption metric on that, on those new client releases, because I think… If there ever were any hiccups, basically the important thing is that people can quickly upgrade to the version that can handle it. As Ben was saying, it will take some while for this to be rolled out anyway. In the meantime, people are upgrading, so I think, basically, like, that seems like to strike a good balance. Client releases out, but no specific threshold for adoption.

Tim Beiko: Oh yeah, yeah, exactly. And this is why I think if we can couple it to releases that more people are gonna be using anyways, then that's good. Like, at the very latest, if we say we want this before Fusaka. all the Fusaka mainnet releases should default to 60 million, right? Like, but yeah, I think the testnet releases… On the way there are just, like. Yeah, also a good opportunity to put this as a default, and as long as every client team has a release that could handle this, then we're generally in a good spot. And this, yeah, Terrence has a comment, like, this also includes CL releases, because the CL releases kind of set the values for, for boost, which is where most of the blocks get built.

Lewis has a question about Zen. Does anyone have more topics around, like, yeah, the specific bottlenecks around Zen? And how that's blocking for $60 million. Sorry, Ben, you were gonna say something?

Ben Adams: Yeah, just more related to the, ELs and CLs that… the ELs sort of determine whether they can handle it, but we have to wait for the CLs to Update there. defaults. I was just wondering if it'd be better to have a mechanism where the CL could query the EL, but the gas limit. They should be… Registering with us.

Tim Beiko: And to be clear, we don't… we don't need to wait, though, right? Like, operators can update themselves, it's just not the default, but… yeah. Terrence?

Terence Tsao: No, I was gonna say it's possible to do that, but that's, like, an engine API change, and Engine API is typically, like, we only changed that during, like, the network upgrades, so for Fusaka, it's probably too late.

Tim Beiko: Yeah. Yeah, we probably shouldn't be changing the engine API design this far late in the process, yeah. Felix?

Felix Lange: Just as a quick comment, so I don't think that is technically true, that we can only change the engine API on a fork boundary, especially for this kind of thing, which would be a new feature anyways. But also, I think it is totally possible, even now, with the existing RPC API, for the CL to query the EL and then adjust itself based on that. Like, I don't think this is technically, like, an engine API change that requires a fork or an EIP, we can just kind of do this.

Tim Beiko: Yeah, I guess it requires, yeah, it requires the CLs changing how they pass that value through. And yeah, Perry on the Zen comment said, that, it wasn't their focus yet, but they have a benchmarking, and they can gather data on any issues with that. Any other updates or thoughts on the 60 million? Okay. Then I guess we can move on to Glamsterdam. So, first off, we had a, block access list breakout this week, the first, the first one. Yeah, is there anything from there that people wanted to bring up or discuss?

Toni Wahrstätter: Yeah, I can give, a quick summary. Basically, the… EAP7928. Breakout call is now happening every two weeks, so if you want to join, feel free to do so. Clients are quite busy at the moment in implementing the EAP. Right now, every client team has started implementing, and I think, got the feeling that we're getting closer to a first step now.

Tim Beiko: Thanks, thanks. Anything else on block exorcists? Okay, then I had a couple things for just the rest of the EIPs, being proposed for Amsterdam. So first off, there's a lot of gas price or repricing EIPs that are being proposed, so, Maria and Ansgar put together a meta-EIP to try and keep track of them, but it's… kind of a unique thing, to have, like, a nested nita EIP, so they had questions about, yeah. how do people think we should, approach this? Anskar, yeah, you want to give some context on it?

Ansgar Dietrichs: Yeah, perfect. So basically, Maria and I, were talking with people, and in general, the need came up, or, like, the idea came up that it would be nice to have a document in general to basically collect. And outline, kind of, all the peace and flight around repricing, because we also expect that that will still, kind of, quite actively, be a topic of iteration for the next few months, or next two months, at least, or something. The… we were basically… we chose to do it as a meta-EIP, at least. as of now, and then wanted to get a little bit of feedback on whether MetaEP is the right format, and whether people think that's useful. The idea specifically would be that the MetaEP does not have any specific role in the governance process, so include, like, being listed on the MetaEP would not directly correlate to

Inclusion decisions in… glam stamp, we would, in the MetaAP, kind of basically just list the status of this is kind of where it's at in the governance process as well, just for convenience. But the MetaAP would be more like a document that then could be useful also, for example, in, like, early DevNets, and basically just, like, as some sort of specifying some sort of just, like, recommended set of EIPs. we… after some talking with people, we think that Meta EIP might be the most useful kind of version of that kind of document, but we'd also be happy to kind of change it to just some other thing, I think. Pari had the point that it would be nice to be version controlled, so we can just, like, target specific versions for definites or something like this, so…

Yeah, but basically just looking for feedback, and maybe one more point there, the reason why it's kind of Maria and I is that, at David and I used to kind of lead the research on this topic from the EF side, in the past. David is now moving to other topics. And Maria is now taking over from him, so going forward, it will be kind of Maria from our side as the domain experts, and then kind of me in collaboration, so any kind of more research side topics you have around repricing, I think would be best discussed with Maria going forward.

Tim Beiko: Thanks, Eddie. Comments? Okay. Like, it's like, think we can give this a try? And yeah, it is also… like, yeah, version control on the IPs is something we've… Talked about in a bunch of contexts, and never really… adopted, so this might actually be a good place, yeah, to justice. Roman, are you asking about, in general, on the gas limit testing efforts, or, Just for the CIP, like, which repricings we want to do.

Roman: Yeah, in general, but I'm asking this now, babe, because both are kind of intertwined.

Tim Beiko: Yeah, so we have this, Yeah, so I guess we have these two Telegram channels that are, like, quite high activity, and then the Nethermine team has just been keeping, like, the high-level, yeah, the high-level, bottlenecks, in this sparkdown that I shared. I don't know that we have any kind of intermediate resolutions that you either have, like, the, you know, 10 bullet points skim, or the two groups with hundreds of messages.

Parithosh Jayanthi: Yeah, we do some summaries on ACDT, so I guess that might be the second best approach, but yeah, I mean, TLDR, the limits for 60 million gas are listed there. Most of the work right now is on state bottlenecks, and we're focusing on sync-related state bottlenecks, and we have 2x mainnet state already to test with.

Tim Beiko: Okay, and yes, Maria shared. Actually, yeah, it might actually be a good idea to just share the… if we could link the Grafana dashboards. I don't know if these are actually public.

Parithosh Jayanthi: Yeah, so Docs has been a bit lacking, and we wanted to wait to finish rebasing the benchmarking tool on top of East, and I think Camille has… basically achieved that over the last day, two days this week. So I would assume soon we would focus on documentation, we'll make sure all the links are in one place, and yeah, make it a bit easier, I guess.

Tim Beiko: Awesome, thanks. And yeah, on the state side, Carlos, said… shared, the, BlockNet page with also… which also has a ton of, of, status updates. Yeah, hopefully we can start to, To align all these different resources, but, yeah. For now, it's a bit of a mess. Okay, so, yeah, back to repricing, so we can, keep the EIP, experiment with it, and, and, yeah, see how useful it is through the process. The other thing, I want to discuss, so now that we have set the headliners for the fork, we should eventually review and at some point decide which EIPs to include.

we said that the deadline to propose the IPs for the fork would be when the Fusaka mainnet releases go live, which is still, you know, at least a month or a couple months away. there's already 18 proposed EIPs, and I think there's a couple more that just haven't been merged in yet, so I can expect we'll get 20 or 30 of them. I wanted to check in with teams about, like, how should we go about reviewing these? Not even, like, making decisions about them, but just, you know, when and how do we want to discuss or ask questions about them. Besu shared their preferences, and I think historically what we've done is kind of have people semi-randomly show up on this call and pitch their EIPs and not be super systematic about it.

And it just gets harder as the number of EIPs scale. Like, if we have 30 EIPs and we give them each 10 minutes, that's 300 minutes of call time. So, yeah, curious how people think we can do this. Yeah, Justin?

Justin Florentine: Yeah, I just wanted to clarify real quick. I wanted to be very explicit that these are the EIPs that we listed that we would like to hear more about, so that might be a mechanic that we can leverage. You know, we're gonna get this list, like you said, it's gonna be 30 long, it might be the point where we have to schedule people and say, hey, you made it to, you know, you have this much interest from the core devs, please show up on X date and present this one, and we might We might just have to actually, coordinate this. I think that the important part of this is that the core devs reach out and say, hey, the following, you know, EIPs have questions that need to be answered, these are the questions, and proactively set an agenda for them to come in and discuss.

Tim Beiko: And, yeah, Tony, sorry, I didn't mean, like, a full-on presentation, but I think, like, yeah, each of them itch that we actually want to consider probably takes some time on the call to discuss the trade-offs, and it's not clear that we… there's probably 70 IPs we don't even want to bring up on the call, because no one thinks they're, like, a serious candidate. Yeah, so I… I think it would be good to have some sort of, like, pre-filtering, and I think a requirement for this is, like, that teams actually have the time to look through the list, and I know that there's already a lot of things going on, and, like, we're asking people to focus on, like,

Yeah, on many things in parallel, so, Like, if teams feel like they have the bandwidth to actually at least look through the list and, like, flag the, you know. 5-ish, or whatever most interesting ones they want to discuss, like base U. That seems pretty straightforward, like, you know, starting on the next call, we could… we could try to have, like, yeah, more conversations about the ones Basie listed, or if there's ones other teams want to discuss, too. But is that realistic? Like, do people think in the next week or two weeks they can even open the list of 20 IPs and figure out which 5 or 6 are most, most important? And yes, obviously we have beat the magicians, like Greg says, so we can do this async. I think we… we should try to do as much of that as we can, but realistically.

Yeah, we do end up discussing a bunch on the call, and Yeah, we can potentially improve the curation there. if… I mean, if no one else have… have… has thoughts, then I think I'm happy to, reach out to the… EIPs that BASU listed, or potentially, I don't give people a couple more days to share their preferences, and then for the next ACDE, we can have a set of them present. There were a couple people who wanted to present today, and we have some time left, so if there's nothing else people want to discuss, we can also do that, and if folks don't want to sift through the presentations, you can obviously hop off a bit early, but…

Before we do that, Benny, Other thoughts on just how we approach this? Okay, so… yeah, let's do, let's see if client teams share more EIPs. I'll follow up with the one space you mentioned, and then, yeah, we can spend the rest of the call trying to go over the ones, I think there were five that people wanted to discuss. First one being, EIP2926 by Guillaume.

Guillaume Ballet: Yep. It's a short presentation, so you don't have to worry. I just need to find how I can share my screen, here we go. Can you guys see it?

Tim Beiko: Yes?

Potuz: Yes?

Guillaume Ballet: Wow, that was the fastest I ever… was able to do it. Yeah, so EIP2926 is actually a very old EIP. It's starting to be a tradition at this point, but it's an EIP that was resurrected. It's an EIP that dates from 2020, and introduced code, trunking in the MPT. And the reason for this to be resurrected, was a discussion we had at the interop with, well, first that was a suggestion by Tomas, and then we talked about it with Ignacio and Jocham, and I think that's it. To introduce code chunking, because coaching brings some advantages, immediately, and we don't have to wait for, binary trees, and… and or vertical trees, but in all likelihood, binary trees, to be implemented and tested, in mainnet.

So what does that… what does that do? Like I said, it's, miraculization in the MPT. What we changed from, from the initial EIP is that we bring in some of the learnings that we did, while working on vertical and binary trees. So we have 31 byte chunks with one byte of push data. We have… EIP4762, or at least the code access part of it. This is still an open question, but I don't think it's gonna be a very complex one. The question is just how much, how much do we pay for code chunk accesses? There's a very well thought out, gas price for, for Verco that we could reuse. There seemed to be, some, thinking when, during the interrupt, we were considering 7903 or 7909, I don't remember the…

the number just now, that the code access should be much cheaper, so that's why it's still an open question, but I don't think it's a very complex one. And, yes, it requires a transition, like an overlay… actually, not an overlay tree transition, but something a bit similar, something that is, that is, defined in EIP7612. But because we're only changing, we're only trunking the code and inserting it in the tree, it will take less than one day, probably an hour, to be honest, but, yeah, it needs to be implemented and tested to know for sure. Okay, I don't know why I can't go to the next slide. Yeah, so… the TLDR? Nothing changes in the EOAs. You still have the nonce, the balance, the code hash, and the storage route.

But for contracts, you add two extra fields, the code size, so that you can just access X, like, you can serve X code size immediately. And then, the code root, which is another NPT, just like the storage tree, that is where chunks are just, added sequentially, except for the last, field, which is the last, addressable. integer, which gives you the version. And the version is probably going to be RLP, so if we encode it as, 0x, as 0 and not 0x80, we could, we could just not have the version until we start Using different versions for accounts. The transition… This is just a quick explainer. You have an iterator, Ed, that goes over all the…

all the accounts in the state, so, like, the black squares are just the nons, the balance, everything. And then, the green squares are the code hash, so if the code hash is the non-empty hash, represented by the empty set symbol here. Then there's a contract, and the contract is represented by a scroll, and so at every block, the iterator moves by n values. and just sees if the contract is a, sorry, if the account is a contract, and if it's a contract, they will go get the code, chunk it, insert it, and add the extra fields. This is much simpler than the overlay tree, because you can check if the conversion has happened for a given account, by the presence of this storage root, sorry, this code root,

field, but if you know that this is a… you can easily know whether an account is a contract or not. So, for example, here, if it doesn't have If it has a non-empty hash, but doesn't have a code root and a code size, you know this hasn't been converted, so you can fall back to the old behavior. Yeah, so just a quick summary, the difference with binary trees and vertical, well, it's… we are using the MPT, we're not using a unified tree, we're just, using, like, we're just doing the same thing as we do with the, with the storage, storage tree. And, sorry, this is wrong, I meant, well, we could potentially get rid of the account version, so this is, my slide is wrong here.

Yeah, why do we need this now? Because it solves some, or at least it helps mitigate some attacks against the KVMs that we know of. And it's a simple fix for increasing the code size, because If, you make the code, as large as you want, each chunk access will, will cost money, or cost gas, so, you can, you can disincentivize executing too much code this way. And, yeah, you get the code size field. In the account, so you can serve that, without, without having to load the entire code and, and computed size. And yeah, that's pretty much it. If there are questions, I'm happy to answer them.

Tim Beiko: Thanks. I guess, yeah, I had a question around what the, you know, end-user benefits are, which, then sort the cheaper proving times for… or cheaper proving for certain worst cases and smaller proofs. Dragon is asking what the downsides of the CIP are.

Guillaume Ballet: I mean, I can't think of any, to be honest, except that the conversion might scare people off. And, but it's, like I said, it's a much simpler one. The other downside is that it feels like this is settling down, like we could have the better model with a unified tree and everything, and people might say, well, look, this is good enough, let's stop there. So, The, the other downsides… well, okay, actually, I should have, I should have, answered that better. The other downside, of course, is that, people… like, there will be an increase in gas costs, but I don't think you can, you can go around that. And, that, if you need to pass proofs, you will find yourself with extra, extra chunks in your proofs, so some tooling will need to be updated, which is still an open question. I don't know who will be affected by this in the short term.

Tim Beiko: Thank you. Any other questions? Otherwise, we can move on to the other EIPs? Okay, Kevin's asking, could we benchmark the performance increase for ZK Proofers?

Guillaume Ballet: Sorry, I didn't quite get the question. Kev. I know it's Kev, but that was too fast for me to read. Could we benchmark the performance increase for ZK approvers? Yeah, we can. I mean, someone needs a proof of concept first. I don't really want to make the mistake of writing a ton of code for, for something that's gonna end up in the garbage. But if, yeah, if it's, if it gets, included, I will gladly, re… you know, I would gladly take the code that already exists, adapt it, and, you could have that in, in a week, in less than a week.

Tim Beiko: Okay. Thank you. Okay, then, anything else on these? Sorry, on this one, otherwise we can move on, Mark had two EIPs that he wanted to bring you up, 7793 and 7843. Mark, are you on the call?

Marc Harvey-Hill: Yeah. Hello. Yeah, I don't have a presentation, but I just wanted to kind of give a quick pitch so that, maybe people can review async, or maybe I can kind of speak a bit more about it if there's interest. So the first one is EIP7843, and this is the slot num op code. So this is, like, a very simple IP that just, returns a slot number on-chain, and the motivation for this is, kind of making it easier to change the slot length in future. So currently, you can calculate the current slot by doing some, like, basic maths with a timestamp, but this would break if we change the timestamp in future. And there are other ways that you could potentially do it, like,

kind of proving against the beacon block route and stuff, but then these other ways are quite, kind of, gas expensive, so I think that in reality, people would likely fall back to the kind of seek way that would break. So the way I see it, this EIP is, like, very simple, kind of low-hanging fruit, that will just kind of give us one less thing to worry about if we're gonna, change the slot length, you know, in maybe in a few hard forks' time. So yeah, that's quite a small one. And then, so the other one is EIP7793, which is called, Conditional Transactions. And the motivation behind this is improving support for encrypted mempools. So these are kind of like a cryptographic mechanism, if you don't know, that can give users protection against front-running. So to kind of

quickly most, like, explain how this would work. So, slot X, user submits an encrypted transaction. All of these, encrypted transactions, along with other users, they're all ordered. And then in slot X plus 1, then the unencrypted transactions are put at the top of the block, and the order should match, the ordering that was kind of determined before. And so the motivation here is, currently, there's not really a good way of actually enforcing this ordering, so we… we unencrypt it, we have the ordering, but there's not really a good way of actually, you know, knowing for sure that these transactions actually were placed in the correct order. So what this EIP does is add a new transaction type called a conditional transaction, and essentially this just adds a field where you can specify the index of a transaction where you want it to land in the block.

So I can send my transaction and say, I want this to be at index 1, and it will only be valid, it will only execute if it's actually included at that point. And so, between that, and also doing some kind of smart contract logic on-chain, then you can enforce the logic of the encrypted mempool. So the one other part of this EIP is there's the new transaction type, conditional transactions, and then there's one new opcode to go along with it, which allows you to actually if this is a conditional transaction that's being executed, then you can check what the actual index is on-chain in order to enforce it. And so, as I say, this kind of allows encrypted mempools to actually have a lot stronger guarantees against front-running, and allows this logic to actually be enforced.

But yeah, that's what I wanted to talk about, but with this second one, if there's interest, then I can prepare something a little bit more in detail.

Tim Beiko: Yeah, Marius has a comment about the amount of work that the conditional transactions would add to the mempool. Do you have a sense for the Yeah, implementation complexity there.

Marc Harvey-Hill: I mean, I'll have a think more about that, and kind of… maybe just discuss with Maris, kind of what he thinks the problem with is there.

Tim Beiko: Substant. Any other questions or comments? Okay, then, last step, POTUS also had two EIPs, which I think are still, waiting to be merged. I'll put the PRs in the chat, but yeah, POTUS, do you want to…

Potuz: Yeah, this is gonna be really fast, faster than Mark. Hopefully, I… I also won't have a presentation, and I just wanted to keep the ball running, to have the ball rolling. The… the idea would be to try to separate a little bit more execution from consensus. Today, we have this mechanism to send requests from the execution layer to the consensus layer, and every time that we need to add a new request type. because we just want to have some new feature being provided on the consensus layer, we need a hard fork that would involve both layers, the EL and the CL. We would need to have a new contract deployed on-chain, we'll need to, like, have a bunch of things, like the EL would need to do different things, and add a new list

In the payload, or in the payload envelope in the future? When we actually… today, we have this consolidations contract that is already quite generic. The consolidations request, the EL sends us a signature verified from the withdrawal credential of a public key for a validator, and then a second public key, which is 48 bytes that can be actually interpreted anyways on the CL. So what this idea of DCIP is, is to just reuse that contract to pass on the second public key from the same contract, to pass on the last 48 bytes, arbitrary data to the CL to interpret however the CL wants to. This way, for example, we can just add a feature. I mean, this came from discussions into adding a feature to convert your validator into a builder after EPBS, and to overload already the consolidation request contract. There's tons of trade-offs.

I mean, the good one is that it allows the CL to do a hard work without ever involving the EL. On the other hand, it would reuse a contract that is expected to be for something with a particular cadence, with a particular number of requests. per block, and then we might add some new request type that might require many more, for example. There's issues about pricing, there's… there's a bunch of reasons why not to do this, and a bunch of reasons why we should do this, and I just wanted to have the right people involved in the discussions.

Tim Beiko: Thanks. Felix?

Felix Lange: Yeah, just as a quick note, so definitely I'm interested in this stuff. I was very interested also in the execution layer requests, and, I think from my end, yeah, it sounds pretty great to make it more generic. At the same time, the, the mechanism is already very generic, so it's not a… we actually made sure that it's not a lot of work to add in the ELs, like new request types, and also I want to mention that the consolidation contract is not extensible in any way, and neither are any of the other system contracts that we use right now, so… It will have to be a new contract anyways.

Potuz: No, no, no, I want to reuse the same contract, actually. So, that's the whole point. Like, if you look at the second PR to actually use it for builder changes, it uses the exact same construct. And on the CL, we process them in the same place, in process consolidation requests. It's just that we interpret the second 48 bytes in an arbitrary way.

Felix Lange: Oh, well, I mean, as long as it's 48 bytes, it's totally fine. I mean, you guys can interpret it however you like, but definitely, like, if you ever want to store a different number of bytes, then it will have to be another contract. That's really my whole point.

Potuz: Yeah, more than 48 bytes of data would be pretty bad. For that contract.

Tim Beiko: Okay, anything else on, POTUS's VIPs? Okay, thanks, Boris. I think this covers everything. we… Had on the agenda. Anything else people wanted to discuss before we wrap up? Okay, great. Well, we can wrap up here, Jen. Thanks, everyone, and we'll talk to you all soon. Have a good… have a good day.

Potuz: Bye-bye.

Marius van der Wijden: Thank you, bye.

Alex Stokes: Thanks.

Fredrik Svantes: Thanks, buddy.

Carson | STEEL: Thanks, bud.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.