All Core Devs - Testing #56

Transcript

Parithosh Jayanthi: Hey, sorry about that, I was on the wrong color link.

Pooja Ranjan: No problem, just let us know when you're ready.

Parithosh Jayanthi: Yeah, I think we should be good to start.

Pooja Ranjan: We are live.

Parithosh Jayanthi: Thank you. Yeah, welcome everyone to ACDT. We're gonna be starting off with, Holsky updates today. So, we had the Hosky fork last week, Wednesday. Other than one hiccup that we noticed, on the first day, where it seems very likely that it was related to, operator error, or having too many validator keys on one node, we haven't necessarily noticed anything else on hold scheme. We should have BP01 go live tomorrow, but in the meantime, there were still two points I wanted to sort of dig a bit deeper into. We did notice a slightly higher reorg rate on Lighthouse nodes. Has the team looked into this already? Is there anything to report?

Or a pre-release we can try?

Pawan Dhananjay: I think we have looked into it, and in the subsequent release, we will be releasing a fix for that as well. We had, like, a bunch of Issues where we were… processing… more, attestation slashings than we needed to. So we have fixed that in… And that latest unstable code, and we should be cutting another release. Sometime, this week or the next?

Parithosh Jayanthi: Awesome, that sounds good. Thank you for the update. There was another, I guess, unrelated incident, but still a testnet incident, about a nethermine bug. Does someone want to go through What happened, as well as… If the patch is ready to be tested by more people. I see Lucas on the call, do you want to talk? Okay, I guess… not. If that's the case, then the information can be found on the Testnets channel that I'm linking here, it's on Discord. Both the bug as well as the potential patch and the plan forward are all listed in the chat tab.

Anything else related to Fusaka and Polsky that client teams want to talk about? Is everyone feeling ready with BP001? It should be roughly a day from today, right? I had to… Go ahead.

Pawan Dhananjay: I had a question about, something that the OP stack team brought up regarding the get blob side… the blob sidecar endpoint. So, with Fusaka, we have… basically the KZG proof that we used to give along with the blob sidecarb pre-Fusaka. post-Fusaka, we just dub out the KZG proof field, and… That… the OP stack team, I think they created an issue on our repo saying that The proof wasn't verifying. And I think there was a Discord thread about it. where I think, like. Yeah, like, if I check the thread now, the OP stack team said that they don't need the KZG proof. To be there, On the… on that specific endpoint anymore, but I'm not sure if other, roll-up teams agree as well, so maybe we can check…

If other teams need it, or if we need to update the… HTTP endpoint as well. I can link the Discord.

Parithosh Jayanthi: Yeah, could you link the thread? And just to clarify, is this for super nodes, or is this for a full node? Because the get blob behavior for full nodes kind of fundamentally changes, right?

Pawan Dhananjay: Yeah, I think this is for SuperNotes, so… We basically… when anybody calls that HTTP endpoint, we reconstruct the full blob sidecar and return the blob sidecar, but we did not also compute the KZG proof, because we thought it wasn't needed. But, since… Pierre, like, since the roll-up teams brought it up, maybe we need to just check with other teams as well? Oh yeah, sorry, Jay just.

Parithosh Jayanthi: Mano, do you want to go?

Emmanuel Nalepa: Yeah, so I briefly talked with Nitro team and Arbitron. And they are going to switch from, get blob size car to get blobs. To the new endpoint.

Parithosh Jayanthi: Got it. But then, there was also a comment about updating the attestant library, right, to support the new endpoint. Has this already been taken care of, or is this something we should do soon? Okay, I think we'll check up… With the testing team to add support for this, so that, yeah, everyone should be ready. Are there any other air tools we should be reaching out to? Okay, I guess the least we should do is put out a statement saying that this is the change that's going to happen. And then see if anyone pushes back against the change. Since it is an API change, we can still, Yeah, fix it if… if someone's depending on it.

Okay, anything else related to the API topic, or for Holsky? Blackline did ask about, had target vote rates dropping. Yeah, most of the thesis is just that the amount of work that supernodes have to do is significantly more than before. And once we have the Lighthouse patch as well, it should be better. Especially because, yeah, everyone's processing a lot of slashing right now. Okay, the last call for Rapolsky. Any other topics?

Pawan Dhananjay: I just wanted to confirm another point regarding the MEV flow, so… I think we… the GET payload API, like, we were supposed to, like, at least we were… Using the, the Get Payload V2 version, even before Fusaka was activated. And we were doing that just to ensure that, like, if there was anything wrong, with the API, then, like, we can figure it out pre-fusaka instead of… switching to the V2 endpoint post-Fusaka, and then finding out just then. I was just curious, like, I, know that Nimbus replied, saying that… like, I don't know, some clients replied saying that they switched during the fork. I'm not sure if the behavior is consistent across all CL clients, or…

how other CL clients handle it. Do they have switched to the V2 version right at the fourth boundary, or do they… use the V2 version before as well.

Parithosh Jayanthi: Prism thinks that a switch at the fork, but, they will cross-check this. Do we have anyone else? Lodestar? Yeah, fork boundary? Anyone from TECU?

Pawan Dhananjay: Lighthouse basically uses the V2 endpoint now, like, even before Fusaka. Because, like, we… Yeah, like, that's the sort of… the… The relays don't actually return the blobs today as well, so we just don't… expect the blobs at all, we just expect the relays to… Propagate the blobs themselves.

Parithosh Jayanthi: Should we make a thread for this on EtherMD?

Pawan Dhananjay: I think there was a thread, I was trying to find it on the… Chat. I'll paste it when I find it, somewhere.

Parithosh Jayanthi: Sounds good. I also don't see Bharat on the call, but I think he's mostly taking care of the MEV workflow, so he might have a better overview. But we can ask them to chime in on the chat then. Anything else? Oh yeah, just to raise Barnabas' point, most relays or… and or builders have stopped support for Holsky. So, we don't have, any… or many, map blocks flowing on Holsky, but this shouldn't be the case on Sepolia and Hudi. There was a brief, issue with the builders on Sepolia and Hoodie that has now been fixed, and we should have steady MEV blocks again. Yeah, and another reminder that, OSCI will officially be end of life at the end of the month, unless announced otherwise, or if we notice any issues we want to keep it around for.

Okay, anything else regarding… Regarding Hosky? Okay. Panelists, do you want to give an update for Fusepa DevNet 3 and the non-finality testing last week? Okay, he's unable to talk, but yeah, we had non-finality testing for two, I think between two or three days. And since then, we have recovered, so if for some reason, your client is not, up to head, then please reach out to us and we can debug it, but I think we should be mostly fine. We did have plans to do another round of a split network tests this week, with the chaos testing framework that the Sunnyside Lab team has. And we'll let you know how that goes, but yeah, I don't expect that we find anything because it is a similar approach.

Okay, anything else for Full Circa? And just to have it logged, we have a lot of data, being showcased also for Holsky on the PandaOps website. So, if you want to have a look at how the network is performing once BPO1 goes live, then please have a look there, it should be working through the BPO. Okay, next topic is, gas limits and scaling L1 in general. Do we have any updates about 60 million gas from Bezu?

Ameziane Hamlat: Yes, so the, the implementation is, is almost done. We expect to, To freeze the implementation by tomorrow. And then we can start, fazin, and, and, and testing. But, yeah, big progress. We have seen, like, around twice performance improvement. on almost all the, arithmetic opcodes on top of mod. Yeah. That's it for now.

Parithosh Jayanthi: Awesome. Thank you. Is there anything you need from any other plan team or security team for the fuzzing? Or are you guys all set?

Ameziane Hamlat: Yes, so we already reached out to them. So we have, we have… Prevade discussion on… on Telegram, Yeah, so we should start working on it on Wednesday.

Parithosh Jayanthi: Perfect. Thank you. Yeah, Camille is off today, but he did leave a message about Getting the first version of Zantas on top of BloatNet working at the end of last week. I see Carlos and Joachim on the call. Perhaps you guys have a bit more context, and you want to talk about these tests?

Marcin Sobczak: Yes, we have, progressed into generally in, stateful test on top of the Blownet, and there are first sent tests, like, Camille will have a great cooperation with Johim on this. And, in parallel, we are working on extended analysis, of Modex repricing, on the network, and especially on the application layer, we collected a lot of data, and right now analyzing it, And, yeah, that's it.

Parithosh Jayanthi: Okay, perfect. Is there any active item you guys need help on, or is the next step just continuing with the test, trying to break things, and then see if we need to fix something that's broken?

Marcin Sobczak: Yes, that's, like, like you said, we need to continue what we are doing right now.

Parithosh Jayanthi: Okay, perfect. Is that… do you have anything to add to that, Carlos, for status, or is that mostly the… Meat of the work over the last week.

CPerezz: So we have done quite some progress. The work of this week is to agree with East, in order of which is the best way to orchestrate all of that, because we need to adapt, let's say. how the framework from NetherMind works, and how it executes stateful tests. So that will probably be the work of this week, and let's see what we can get at.

Parithosh Jayanthi: Yep, I know we already have a scheduled call for that, so we should be able to sort that out this week. Cool. And… is there anything else for, the scaling topic that we have to talk about? gas scaling. Okay, the next topic is in, Amsterdam. Could we get an update from block-level access lists? How's… how's everything going? Local testing?

Toni Wahrstätter: I can… I can provide a quick update, around specs stuff. So, today we just merged the execution API specs. Thanks to Mikhail for reviewing that. Let me… Post a link into the chat. From the execution, spec side, everything is done, so we have been ironing out some bugs, last week. Mainly around complicated edge cases such as 7702 or reverted transactions. But so far, it looks quite good. So, from the spec side, we're done. Maybe someone… who is more involved in testing, maybe Rahul or Felipe can provide an update on testing later. There is the block of Access list up, breakout call happening on Wednesday this week. Yeah, feel free to show up. We will discuss client updates, and also finalizing a decision on block-level access list HUD versus block-level access list route.

Parithosh Jayanthi: Thank you. Rahul, are you on the call? Do you want to speak? I don't see you right now.

Felipe Selmo: I can… I can add an update here as well. The testing side's looking good. We… we had a… One test that was reported, to have been Not correct that we fixed. And then we have had a few more that were added last week. And we have an open PR. To add a few more, That should be ready, early this week as well, and we plan to get another release out. And I wanted to say, Reth was added to the tracker as well, and it looks like Breath and Geth are passing most of the tests that we have currently, so it's looking good.

Parithosh Jayanthi: Awesome, thank you. I do notice that… so we have a Cordosis config also, on the spec sheet. For the DevNet, has anyone already tried interop between Geth and Wrath, or is that the next milestone? I'm gonna assume that's then the next milestone, and then… Yeah, we can… we can go from there. And just to also say that out loud, once we have 3, ELs working, then we're ready to launch a block-level access, let's start that, so… hopefully that's incentive for the next EL to join in.

Felipe Selmo: Yeah, I… I also think Nethermind is pretty close, as far as I know. Locally, they were running tests and passing a lot of them. But we haven't been able to get the right config for another bind on Hive yet, and… I do believe… Somebody's looking into it. But I need a little bit more context. I don't… I haven't had the right config to be able to run it.

Marc Harvey-Hill: Yeah, I'm having a look into this now, so yeah, it might be, like, a Hive config issue or something with our… Hive test runner, potentially, but yeah, hopefully it'll be resolved soon.

Felipe Selmo: Awesome, sounds great.

Parithosh Jayanthi: There was also an ask from Barnabas that would really help with, with our automation tools, on all of your client repositories on your main repository, so ideally not a fork, could you guys push the code onto a branch called BAL DevNet0, so that we can set up all of our auto builders, and Hype will pick up the changes, and… Kurtosis will pick up the changes and so on. It becomes a lot easier for us to automate testing if we have a standardized naming scheme and a standard place on where to look for changes. Okay, cool, I guess we'll follow up on that one later, and… yeah, one of the… Easiest things for us to also do would be to have Asserta run the execution spec tests against the running local block-level access list DevNet, and that should already knock out a few bugs.

That sounds all vague. Nice work, everyone. The… anything else for block-level access lists? Anything that we should discuss today that might help? Are there any immediate blockers? Okay, the next topic we have is, EPBS. Can we get an update from anyone on EPBS?

Justin Traglia: I think it would be better if client teams gave an update. I'm not sure what the implementation status for DevNet Zero is, but I just know that they're working on that. And we will have a breakout call this Friday. Please comment if there's anything you'd like to discuss. There were some things at Awkward of… That we talked about. What was it? whether or not to allow, like, trusted payments and all that kind of stuff. It's a bit undecided, but I feel like we're sort of working… or we're sort of on the same page now. But we do need to hash out details.

Parithosh Jayanthi: Perfect. Is this Friday, this week?

Justin Traglia: Yes.

Parithosh Jayanthi: Okay, and is this call meant just for client teams, or also external teams that we should make sure to invite?

Justin Traglia: Whoever wants to join, external is fine.

Parithosh Jayanthi: Okay, cool. Then, if anyone's listening in and has an opinion on trusted payments, then please chime in. I'm not sure if any client teams have implementation updates, though. Or is the idea that we work on the spec first, and then there'll be alpha implementations?

Justin Traglia: The spec is ready, I think. They're just… Yeah, just working on the implementations. Or are you talking about for trusted payments?

Parithosh Jayanthi: No, just for, in general, EPBS.

Justin Traglia: Okay. Yep, spec for DevNet 0 is ready.

Parithosh Jayanthi: Okay. Then we probably have a spec sheet for… okay, cool. We have a spec sheet for EPBS. I would assume we just have to update the values there. Cool, and, ideally, same rules, so, please have a branch on your main repository called EPBS2.0, so that we can automate some building and image pulling. There was one other topic on the agenda relating to, RPCs. It's more of an info task, so web posting, Hive RPC, pass fail rates on Discord now. So there's a weekly post. We're gonna update it to also showcase if your client has done better or worse with RPC tests over the last week.

In case you want to be notified when an update is pushed here, then please let us know. We have a new role that we can add people, and you'd get a bit, yeah, you'd get a notification when there's something. And there's, also an ETH Research post about, having a lot more tests and a lot more focus on application-focused tests for the RPC, so please engage with, with both of the topics. Okay, anything else for the call today? If not, then…

Justin Traglia: We could bring up the math questions again. Brad is in the call now.

Parithosh Jayanthi: Yeah, cool. Yeah, Pawan, do you wanna bring up the topic again, so that Bharat has context?

Pawan Dhananjay: I think I've linked the… linked Bharat's reply there, for, basically. for the get payload V2 endpoint, Lighthouse starts using it before the Fusaka fork, but some other clients also, basically switch at the fork boundary, so… I was wondering if, like, the spec… clearly specifies what is the right thing to do there. We do… we do it before the fork, because we wanted to see if there was anything… if there was an issue with the endpoint, then we figured out before Fusaka, instead of just at the fork boundary. So, like, if there is a right way to do it, then, like, we'd rather switch to it, but we just wanted to check if there is a spec-mandated way of doing it. Should we switch at the fork boundary, or…

Before is fine as well.

Bharath Vedartham: Yeah, so I can chip into this. So, ideally, it is not fork-dependent at all, this API, like… You can use it in Electra, you can use it in… any four, it doesn't matter. The only issue here is that we do… when we hit mainnet, we need to ensure that all relays, like, have implemented it. And that is generally, like, because the thing is, like, the EF, like, we don't control… the relays are closed source, we don't… we have informed them, like, multiple times on the criticality of implementing, like, Get Payload API, GetPayloadv2. So, that's probably the only thing, right? It is… yeah, it's in their best interest to implement it. But again, so, I personally would recommend to, like.

use it, like, after Fusaka. It… and it's not Ford-dependent, but I would recommend to use it out of Fusaka, so that… because we have informed them that, okay, you have to get it implemented by Fusaka. So, right, the worst thing that'll happen is, like, yeah, the API will… Fail, but, yeah, like… I would recommend to, like, use it, like, after Fusaka. It'll probably be safer, like… Because I'm unsure, right, if you… let's say you do it on Electra, like, It should be fine, but maybe if you, like, release that quote, like, before, if you have a release before this thing, what is that? Fusaka and, and, like, and the relay has not implemented it, it might impact that. So that's probably my only concern, like, outside that, that API is not really fork-dependent. My only concern is, like.

whether the relays have implemented it or not, and we have given them, like, a… like, information that you have to at least implement this by Fusaka. So, that is, like, So, just like using that post-Whusaka, it just seems like the safest way, like…

Parithosh Jayanthi: Yeah, okay, and yeah, Pawan's argument also makes sense, just to keep it easier to debug. Could we just have a list on Discord for how every client's implementing it, so that we can just do it consistently? Could you maybe make a list, Parath, and then we can do from there?

Bharath Vedartham: I can do that. I can do that, yeah, sounds good.

Parithosh Jayanthi: I would… Anything else we want to discuss today? Okay, then thank you everyone for coming, and see you next week.

Pawan Dhananjay: Thank you.

Jochem Brouwer: Thank you.

Jen: Thank you!

Justin Traglia: Thanks. Bye, Ron.

Toni Wahrstätter: Bye-bye.

Ameziane Hamlat: Thank you.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.