All Core Devs - Testing #48

Transcript

Parithosh Jayanthi: Hello, everyone. Let's give people another minute to join, and then we start off. Okay, let's get started off then, and people can still join in, as time passes.

So, the… Yeah, thank you. Perfect. So, … The first point of update is, Fusaka updates. We have DevNet 3 that's continuing to run from the….

Danceratopz: From about 10 days now.

Parithosh Jayanthi: Last week, we did two non-finality experiments. In the first one, we took out just the full nodes, and in the other one, we took out a subset of the supernodes. In both of these scenarios, we had non-finality for… in the first case, for about an hour, and the second case for about half a day. And the network was able to, regain finality after a non-crazy amount of time. I think the first scenario, we have finally about an epoch after, which is actually great to see the speed. And in the SuperNote scenario, we regained finality about 20 to 30 minutes after, which is still quite a fast, regaining. We still have to dig a bit more into the results, but nothing's really stood out. Bandwidth limits seem to be fine, so the network's largely okay.

The next exercise we're planning on doing is, limit a subset of nodes to just have about 1 Mbps or 2 Mbps, especially full nodes, sorry, especially super nodes. So they're still participating on the network, but they're very unreliable supernodes, and we want to see what that does to the network. Besides this, I don't see Bharat on the call, but, we have been… oh yeah, Bharat, do you want to talk about Get Blog V2 and MEV workflow on DevNet 3?

Bharath Vedartham: Yeah, I don't… Get Blog, you too, … that I'm not sure.

Parithosh Jayanthi: As in the private mempool stuff.

Bharath Vedartham: So, actually, with respect to the private mempool. updates. There were some… I had… I faced some issue… I found an issue during, like. fork transition in the Get Payload API, like, it's a small issue, like, I was mainly, like, working on that. So that is something I was doing, and outside that, I haven't been monitoring the DevNet 3 for a… since last Friday or something, but outside of the private, mempool stuff. I do believe, like, … we can… we have a way to directly send transactions to the builder. Like, I think PK has been pretty great in, like, modifying spammer to directly send transactions to the builder. I think we should probably, like, stick to that, and, like, I think that should probably work for…

For private mempool testing for, like, 2 or 3 days, or 1 or 2 days. We'll probably have to monitor and monitor the spammer jobs, but I believe, like, that's a path we can take right now. So, that's the way I'm seeing it, like….

Parithosh Jayanthi: Okay, that sounds good. Then, let me just get this setup, working again for just spamming on the private member, and then we can, have a look at the peer-to-peer data and make sure that Get Blurv2 is properly disabled and there's no other bugs. And we can release a report or some data on how that goes.

Bharath Vedartham: Yeah, I think, like, once I fix the fork transition issue, I'll… I was planning on spending more time on the private member stuff, yeah.

Parithosh Jayanthi: Perfect.

Bharath Vedartham: Compute.

Parithosh Jayanthi: Yeah, so after that, there's, there's also some more information. We've had a couple of, community tools starting to join in DevNet3 as well, which is a great sign. So we have Vouch, Who's joined on as a validator on the network, and they seem to be producing blocks as expected and testing as expected. The other bits of, testing on DevNet3 is we have a fork of Prism, and our Prism fork essentially adds hooks, and this hook allows us to change what it does at runtime, and one of the things we're doing is mutating data columns. And there's a probability on what data columns we're mutating, either we're doing byteflips, or we're changing the column indexes and then repropagating them on the network. And since it's a… it's a proper prism fork, every other node would see it as a prism node on the network that's just malfunctioning.

This has been live on the network for, like, I wanna say 2 hours-ish now, and hasn't seemed to cause any chaos. We also validated everything locally, and everyone's rejecting the malicious payloads as expected. The good thing is, with the hooks, we can extend this to more malicious scenarios, so if anyone has ideas, then please engage. It's a thread on the Interop chat, and I've posted a link there. Anything you want to talk about with regards to DevNet 3 right now? Okay. Moving on, we've, set up DevNet 4 as well. This is in, in coordination with, Sunnyside Labs as well as BASE, and it's a really, really large network. We have about 1,500-ish nodes, touching almost 1,600 now, and more are coming online.

You can find the distribution as well as a plan as to what all we intend on testing in the network in the message I've posted in chat. In case anyone has any deviation from that testing plan, then please reach out, and we can accommodate that over the next couple of days, but in general, the network's doing fine. We have the next BPO scheduled, for the next few hours. And we're going to be going all the way up to 48 blobs, and the idea of doing a very high blob load on this large network is we can see how peer-to-peer would actually function. This is still mostly the perfect scenario. For example, Sam's done, like, a alpha… sorry, yeah, Max is 72. …

Yeah, Sam's done, like, an alpha analysis of the data we saw today, and we already see attestations reaching everyone in the 500-ish milliseconds range, which is unrealistic, because mainnet, it… the P99 value is roughly 2 seconds. But I don't really see a way of inducing that in a DevNet. Everything is, just as a fact, pureed better than mainnet ever will be, but it should give us, at least a hint of how the network would perform higher blob loads. And in case someone wants a graph, then the Sunnyside Lab team has made a graph for us, so please have a look there. Yeah, the only client that's been having issues on the network is Nimbus, and Dustin's been looking into it. I don't see him on the call.

But one of the things that he mentioned is that, … they're getting information a bit later and deeming it as invalid and causing some issues, so he's looking into why this is triggering specifically on DevNet4 and not on other networks. Yeah, okay. That's, I think, it on the DevNet front. Does anyone have questions on Dev3 or DevNet 4? Okay. The next point was that we spoke about, host key releases on ACD last week, and one of the concerns that all the clients might not have releases in time for, For the earlier proposed timeline for Holsky, and there was a question on if we would like to do releases from Fox, or to specify a branch, or have an RC release, or whatever for Holsky, such that we can still hit the September timeline.

We, checked with the community chat we have with Holsky validators in case this is something that they can support, and the only request that some of them had is that we give them a… either a branch that they can build a binary out of, or a Docker container that contains the binary, such that they can extract it to use in their own tooling, and they're happy to accommodate that. So, at least from a community perspective, we shouldn't have any blockers for having ad hoc releases for Holsky. But I wanted to ask if any client teams have put more thought into it. Is there any update on how we want to do Holsky, and if we have to communicate anything beforehand?

Okay, I guess we, bring that up as a topic on ACD and async. Yeah, I think Prism… has a message, so James has a message from Preston, releasing Holski off of Develop isn't difficult, but it breaks their normal flow for releases. Yeah, that's… that's a fair point. Okay, I assume that we're talking more about this on ECD than if no one has strong points to bring up today. In general for Osaka, the next step for us is finishing up DevNet 4 testing. Devnet 3 will continue running, and we're gonna just get more malicious with it, and, … Yeah, I think that's… we have a Shadow Fog planned, but we'd probably wait for some final releases before we do the Shadow Fog.

Does that sound okay as a testing plan for Fusaka? Is there any concern, anything that we should accommodate this week on the large network? Because we will be taking it down on Friday, so please reach out sooner so we can accommodate any testing, and it's quite time-consuming to spin up these networks, so we want to get the most out of it. Definite 4 doesn't have any non-finality planned on it. We can have a short non-finality period, but I don't know, do you have a particular reason you want to try non-finality on DevNet 4, Kavana?

Pawan Dhananjay: … I didn't have, like, any particular plan in mind, but, like, if we had, like, a longer non-finality period, then… Like, some attacks… Could become a little more… I don't know, like, potent in some ways, so… If we were planning to have a longer finality, then we could potentially look into those as well.

Parithosh Jayanthi: Okay, would you be fine with us spinning up a smaller network to do this? instead of DevNet 4.

Pawan Dhananjay: Yeah, I think that would be fine as well.

Parithosh Jayanthi: Okay, cool. Then we can organize that once we've sunsetted DevNet 4. Okay, anything else for Fusaka? I have this one EIP from Dano that still needs, still needs merging. If anyone… has the time, then please review it and leave your approval. If not, I guess we'd much it this week. Okay, moving on to the next topic, we have loadNet and PuffNet, so more about gas limit scaling. The sync tests are going on, and there was a miss in the… there was a fix missing in the NetherMind image that led to a failed sync test, and the fix was already known, so we just have to re-trigger the sync test for NetherMind.

And Raphael has ordered a few more machines so that we don't have to always rely on the same machine for sick tests. In general… we have this page up, so you'll be able to track historic sync times, disk usage, etc. per client, and this is for Fusaka, of course, but we would have the exact same page for PuffNet, and ETA should be, I guess, mid this week or late this week. I think that's all the updates for gas limit this week. I don't see Camille on the chat, and Carlos is on weekend this week, so I don't think we'd get any other updates. The next topic on the… yeah, Mario, you wanna go?

Mario Vega: Yeah, just a small update, we are capable, I think this week, of making another benchmark release, but I… I think about, … Camille, I can share it, the release once it's done, also with the CK team, so you can run the test with that. Yep. That's, that's it.

Parithosh Jayanthi: time. Perfect. Actually, maybe, do you want to mention the update with EASE, the gas benchmarking tool, now working together? I don't think we've brought that up on ACDT yet.

Mario Vega: I think we… yeah, basically the idea is that we're making benchmark releases. And they contain this, this new fixture format for the test that is called, NGINX, which basically is, normally for our tests previously, we had a Genesis. per test, and it was different per each of the tests. So now with this mode, with this new mode, we have a consolidated Genesis, which allows the Netherland team to basically just pull one Genesis file and run every single benchmarking test that we have. This is very good for a lot of reasons, not only for benchmarking, but for benchmarking, it allows you for just send the new payload, and just benchmark that, and then you report that. We're gonna extend this mode to other,

to not only benchmarking, but also consensus tests, so we can run the thousands and thousands of tests we have much more faster in the engine API mode. But yeah, in general, for benchmarking, it is that, the new consolidated format, and we're gonna make another release this week, and yeah, we'll keep you updated, basically.

Parithosh Jayanthi: Thank you. The next point was, relating to Glamsterdam. So, last week at ACDC, one of the headliners that, at least as of now, has been chosen is EPBS. So the question is, how do we want to organize Glamsterdam-related testing? I am not sure if we have block-level access list DevNets or support on DevNets yet. I see Tony on the call. Do we have any images or any specs or anything we want to have people start implementing?

Toni Wahrstätter: Yeah, we do have specs, so the execution specs exist, and Director has already started an implementation in Geth. But we don't have a test definite yet.

Parithosh Jayanthi: Got it. Is there a Genesis fork? field that we need to enable for enabling block-level access lists, or does it activate on Glamsterdam time?

Toni Wahrstätter: I think we can activate it on Glimpsadum.

Parithosh Jayanthi: Okay, cool. In that case, then let's get in touch, and I guess, get you a Cordosis config, so that other clients can also start implementing parallelly and have something to test locally. Yeah, sounds good.

Toni Wahrstätter: I wanted to bring it up on ACDE on Thursday anyway, because there's still some open questions, so if someone on the call is listening and wants to talk about SSC versus RLP, and if we should keep the storage locations. in the ball or not, so we should talk about this on Thursday.

Parithosh Jayanthi: Thank you. We already have local testing port for EPBS, so we can pin a config on the EPBS chat, so that anyone having that local branch can also test EPPS. I think the question I had there, also for block-level access lists, is do we want to spend time rebasing on top of Fusaka now, or do we want to focus on first getting a DevNet where everything is working together, and then focus on repricing? Do we have, I don't know, … any EL or CL teams with opinions on when to spend time rebasing? Okay, I don't see anyone on the PBS side here, so I'm not sure if, …

it's worth discussing that bit here, but Tony mentioned that block-level access list is already spec'd on top of Fusaka, so maybe that's a good approach for it. I'm assuming the repricing is also going to be able to spec it on top of Fusaka, so let's choose that as the option. And then we can have individual devnets, as long as they're all activated by Glamsterdam or the related timestamps, and we can merge them into the saved DevNet as soon as possible. Okay, any other Planstitam-related testing questions? I think we're still in the phase where we're figuring out how the testing stream should be, so… Please speak up if you have opinions on that. What is the… okay, cool. So Justin mentions that he's going to merge in the intervals per slot, PR soon.

That is good to know, thank you. It's posted in the chat in case someone… In case someone wants to leave any last-minute feedback. And I guess, I think we need to modify some things on kurtosis in order to support this, so we'll work on that later this week once the PR's been launched. And just surfacing one of the talking points on chat, the repricing we're talking about is gas repricing. Okay, the Sunnyside Lab team, do you guys have an update you want to share today?

J Sunnyside Labs: No, not much from our side today. We expect to have another report by tomorrow. for, the ones that were running before the DevNetfall. That's pretty much it.

Parithosh Jayanthi: Okay. Thank you. Any other topics? We have an open floor. I think that was most, or all of the agenda. Yeah, Mari?

Mario Vega: Yeah, sorry, just going back to the question of repricing, do we have an EIP for that, or…? Or is this, like, a general….

Toni Wahrstätter: there are… there are some EIPs, so there is a fixed 7904, There's also 7778 that might fall into that category. Then there is an EIP in the making around multi-dimensional metering, so I think there are some smaller EIPs floating around.

Parithosh Jayanthi: Yeah, that's correct.

Ansgar Dietrichs: Yeah, the one I was… referring to, primarily, that's already in any IP search is indeed 7904, I think. 7778 is interesting, but it's not… that's more like a kind of… a pretty self-contained topic, so I think there's less value in, like, early definite inclusion. Multidimensional metering is still more in the exploratory phase, so I would not… I think that would be premature. So for now, out of the ERPs, it's primarily 79104, but also there's a lot of ongoing repricing work that isn't yet, hasn't yet made it to the ERP stage, so it was also a bit forward-looking. So more like, as we have a few more of these repricing ERPs, I really think that from the EL side, Lamsadam.

at least, you know, in my vision, if it comes to them, a lot of the focus will be on repricing, so as we have, like, 2, 3, 4 of these bigger repricing ERPs, I think… basically trying to have them be prioritized for, like, a shared dev net together with EPBS and BLs would be… would be very valuable, because then you can really test running the kind of… the EVM and the EL in general at basically the speed that we think we might be getting to after Glemsada.

Parithosh Jayanthi: Yeah, is there maybe a plan to have a meta-EIP, or somewhere we can track it? Because that would help with, DevNet spec work, if we don't have to always go through a process to add a new EIP in.

Ansgar Dietrichs: You mean Meta EIP specifically for the DevNet, or for, like, this kind of this more, more topical for, for.

Parithosh Jayanthi: I would have said for the topic of gas repricing for Amsterdam.

Ansgar Dietrichs: Yeah, we can… it doesn't yet exist, but we can make that happen, absolutely.

Parithosh Jayanthi: Awesome, because then we can always reference that, and it would be easier to update the meta EIP rather than add each EIP individually.

Ansgar Dietrichs: And maybe, if I may, one more comment from my side, just maybe to align intuitions here, and again, this is just my opinion, it could be completely wrong, but my feeling is, if we really end up with EPBS and… popular access lists and repricings in Glamsterdam, then I think that will probably mean, and that will probably make sense to be aware of earlier on, that really Glamsterdam is very severely impacting, kind of, the throughput levels of Ethereum, at least potentially. Like, I think, in principle. in reality, things always play out differently, but in principle, kind of with the efficiency gains, we should get there on paper. We could really, like, see the network go to, like, I don't know.

Four times, or something, the kind of throughput level from before. or something like that, maybe even more, maybe less, I don't know, but basically, like, really, like, a very significant amount, and of course, we wouldn't actually raise the limits at the time of the fork immediately so much, but that means that I think if we do the testing. too much anchored at today's throughput levels. I think we're basically, like, missing out on really stress testing where the network will end up. So, and given that it's so unusual, I don't think that will ever happen again at any fork, maybe except for ZKVM at some point in the very far future, but other than that, I think it's really, like, a one-time thing, so I do think for testing going forward, having in mind that Glamstam will be very unusual with the potential scaling impact, I think

Would be a good direction to lean into.

Parithosh Jayanthi: Yeah, agreed. I think that's also the approach we're taking with Fusaka. So, for example, DevNet4 is already testing 100 million. So, even the blob numbers we're getting in all the data assumes that Mainnet is at 100 million rather than at 45 million. So I would also approach Klamsterdam with the same. Thought. I think we have to still find what that upper limit number would be, but I'm sure that'll get clearer as time passes on. Yeah, Marius had a point about consensus issues found on Fusaka. Does either Marius or Mario want to talk about it?

Mario Vega: I'm not sure which one, but 7223, if that's the case, there was a consistency in Geth. Basically, there was an edge case where one of the, … constraints of 72.23 was reached before, 7883, if I remember correctly. We have already a task for this. We're gonna put it on the next Ys release. But yeah, is that the one that you were referring to, Marius, or do you have something else in mind?

Marius van der Wijden: Yeah, I think… There was one, and there was something with breath. If I remember correctly. I was out of hospice, so I only… Nope. Saw some mention of it at some point.

Dragan Rakita: Again, save… Saty about that. Calculation of append the modex base and mod is 0. We just returned minimal gas. But with the changes with Osaka. This is… this is not the case, and… That chaise depends on exponent. So this is… this was the bug. I think Besu and us basically returned the minimum gas, while others returned That gas that got calculated.

Mario Vega: I… yeah, I think we already have a PR merge. I'm gonna check with the team to see if we… if this test that we merged actually, his issue. Yeah.

Dragan Rakita: I tested it with state tests, and it should be fine.

Marius van der Wijden: So basically, the two issues were both with, … the gas cost calculation in MODX, but they were unrelated. Is that correct?

Dragan Rakita: Yep.

Marius van der Wijden: Okay, thanks.

Parithosh Jayanthi: Do we feel like we have enough coverage of this field, then? Or are we spending more time now to see if we have enough coverage?

Mario Vega: I feel we merged a bunch of tests last week. I think it wouldn't hurt to double-check. The… the… the coverage in this. This week, yeah.

Parithosh Jayanthi: And this is already running on Hive, right?

Mario Vega: We need to make another DevNet release, unless I'm mistaken.

Parithosh Jayanthi: Perfect.

Mario Vega: Yeah, yeah, that, that….

Parithosh Jayanthi: will do.

Justin Traglia: I just want to mention that the EF protocol security team has begun reviewing clients. Like this… The PR I shared is from one of our team members.

Parithosh Jayanthi: Awesome, thank you. There was one point, Philip noticed while testing about each call as well. Do you want to mention it and get feedback from clients, Philip?

pk910: Yep. Yeah, just a small question. We have that TX gas limit in EIP7825. And we've noted at least one client implemented this limit for ETH calls, too. Which causes some issues in some downstream tunings. So, the question is if the limit should apply to Eve calls too, or if it's solely for normal transactions, which I would have expected. We unfortunately don't know which client has implemented that limit on ETHCOL yet, so there's no one to blame yet. If it's a buck.

Lightclient (Matt Garnett): Yeah, I don't think that they should be limited on East Coal.

pk910: I actually think it shouldn't apply to ETHCOIL, too, because that's… highly used by UIs to fetch data from the chain. If we limit that, that will cause some problems there, especially as there was no limit before.

Lightclient (Matt Garnett): Yeah, and we also haven't limited based on the gas limit for the block, either.

Parithosh Jayanthi: I guess we can manually make an ETH call against all the client IPCs and figure out who has it mistakenly configured, right?

pk910: Yep, that's relatively easy. You just put the limit in there, and it directly rejects.

Parithosh Jayanthi: Okay, bye. Yeah, Alexi, do you wanna come?

Alexey: Yeah, I just wanted to confirm, so, below limit is, still, is applied or not, because I saw some… Mentioned some of that in the chat, that probably we, … Still adhere to block limit, but not, … Not like a transaction limit. Or we, do not impose any limits at all. And, if it can be described somewhere? This book.

Ben Adams: I don't… I don't think, block limit, but it's, like. 10x block limit is sort of a usual thing, and then some RPCs go higher. I don't think Unbounded is… Entirely what you want, because otherwise you're infinitely… But, yeah, much higher than the block limit. For instance, there's NFTs that do, you know. Reconstruction of on-chain images that are much larger than The block number.

Marius van der Wijden: Also, RPC providers want to limit the amount of computation that you can do per call, so having it entirely unlimited is… … Isn't good, but also having it limited to the… to either the block size or the… Contract, or the transaction size means that Yeah, it's harder to call these view functions on the contracts.

Parithosh Jayanthi: Do we have to update the EIP for wording, or are we happy as is a dispatched the bug? Yeah, I guess we leave as is for now, and then first figure out where the bug is, and then take it from there.

Marius van der Wijden: Yes. Also, I wouldn't really say it's a… it's a bug. It's kind of like a design decision of that. No. To do it this way.

Parithosh Jayanthi: Yeah, and….

Marius van der Wijden: we can say that we think that the design decision is wrong, but I think it's… it doesn't need to be specified by us what they do with that client. In this case.

Parithosh Jayanthi: Yep, fair enough. Okay, anything else for the call today? Thank you all for coming, and have a nice week.

Lightclient (Matt Garnett): Thanks.

Ansgar Dietrichs: Thanks.

Marius van der Wijden: you can, but….

Fredrik Svantes: Thank you, bye!

Mario Vega: Thank you.

Justin Traglia: Thank you, everyone.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.