Block-Level Access Lists Breakout #7

Transcript

Toni Wahrstätter: Awesome. Then, I would say let's get started. Welcome, everyone. Today is the November 19th, 2025, the 7th EAP7928 breaker call. We have quite a packed agenda today, I see, because some of us are in… at DevConnect, as you see on the… on the video. And we've been discussing for the past hour everything related to Book Club Access List? And we put all those things on the agenda, so… We agree to have a look at the agenda? I would say we can directly start with the first topic, which is… The first topic is actually just a summary of what was decided last year… last week. So last week, we discussed if we should add the gas used values into the block access list.

And we decided against it, basically because, things stay way simpler if we just assume clients, order transactions by the transaction index in the block. But I just wanted to put it on the agenda so that we flag this again, and yeah. Does anyone have, different opinions this week that… that you want to share? If that's not the case, I think we can just assume we will not put the gas used values into the block access list and keep it as is. Perfect. Then let's keep it as is.

The second agenda item is syncing and pulse. So I would be interesting to hear from client teams what their, strategies are when it comes to syncing, especially which block level access list might be able… will we be able to just drop, and how many should we keep around, at least? And I'm curious what client teams think about this.

Jared Wasinger: I can say for Geth that we haven't discussed this very much internally, outside of the fact that we do see, bowels being useful for removing the healing phase of the SnapSync. I guess my thinking right now is that… Probably we will just keep as many bells as we have blocks for, but that… it's probably a good idea to… and also that I… I do think it should be optional to serve the BALs for historical blocks, because, like, it's optional to, to serve, like, the SNAP protocol, And then, Yeah, I guess, my thing is that we just have to decide what The window… what, like, a suggested window would be to maintain them?

Yep.

Karim Taam: Why it will not be necessary to… To serve a certain block, a certain amount of block, And this amount will be mandatory. Just like that, we'll be sure that, for SnapSync. We'll have the reson block, so it should help every time.

Jared Wasinger: Yeah, I mean, it could be mandatory, right? But if you think about, like, the SNAP protocol as it currently is, it's not… at least to my understanding, it's not mandatory that a node serves the SNAP protocol if it's… Like, that's… Nuh. It's also optional.

Karim Taam: I mean, if you sell Snap, you should also sell block access, at least I was thinking maybe about something like that, but…

Jared Wasinger: Yeah, I think… I think the SNAP protocol would become obsolete. Like, this would be the… serving the BAL would be kind of like a replacement for SNAP, but…

Karim Taam: I'm not sure, because we have different steps in SNAP, so we have the first step, where we are downloading the leaf, and we are reconstructing the tree, so I think this step is still mandatory. And after we have the hill, when we are trying to heal the tree, and maybe the FlatDB, if you have FlatDB.

Jared Wasinger: Yeah, yeah, yeah.

Karim Taam: There's a halibut, yeah.

Jared Wasinger: Yeah, yeah, you're right, I… yeah, I… I… I agree.

Karim Taam: I, I also had.

Jared Wasinger: I think I agree with you that it would be mandatory to serve… to serve the SNAP and, the PALs, yeah.

Karim Taam: I had also a discussion with Guillaume, he said that Maybe if we have a reorg during the snap sync, We will still maybe need a hill, because, block access list is just, It's not bidirectional, we don't have the… previous data, we only have the post data, so we cannot roll back a block access list. So if you have a reorg, maybe you'll have to do a heal, but in the happy pass, normally. Block accesses will be in us.

Toni Wahrstätter: Yeah, Dragon?

Dragan Rakita: I talked with Red Team about persisting balls, and the feedback that I got is the best case for that would be to not persist it at all, and not even serve it on over a P2P. That would be the simplest solution to us. If ball is kinda needed for the snap sync, maybe adding, I think Jared, proposed this to have SnapSync endpoint that will basically fetch ballpark the blockage. Would probably work.

Karim Taam: But… Block access list, in the block header, we have the block access list hash, so all we can verify the hash if we don't have the block access list, which… the only possibility will be to replace the block and generate it.

Dragan Rakita: In general, you receive the ball from the engine. I would assume that's the point where would we check if this hash is correct or not. But if we dropped balls from the history, I… They think that cash is… is going to be there, unchecked. I think that could be fine. Because you're basically… Syncing from the top tip to the bottom, and verify all the hashes there.

Toni Wahrstätter: Okay. So just to summarize, so, Jeff and… Yeah, so get… Jared was saying you were… Planning to keep the bars around, and you make it optional for serving for historical blocks. And Dragon was just saying he would prefer to drop them and not serve them at all, or at least keep a few of them around, am I correct? With a few, I mean, enough for SnapSync.

Dragan Rakita: Yeah, I think the team said that they're fine with dropping them all. But I'm not that familiar with that part of the code, so I'm just, like, transmitting the message that I received.

Toni Wahrstätter: Okay. Yeah, this sounds like we would want to keep at least enough block lab access lists around for, like, the week's objectivity period, so, like. Am I… am I correct here? Like, 2 weeks of block lab access lists? Is that what is, like, needed in the worst case?

Jared Wasinger: Can you… can you expand on… On the rationale for that?

Toni Wahrstätter: Like, what is the… what is the, point in the past when you would start Snapsing?

Jared Wasinger: Oh, yeah, I don't know, I mean, I guess it kind of depends on what the… After you download the entire, initial state. Like, how long… the amount of time that you would keep the balls around for would depend on, like. How long it would take to download the initial, like, state snapshot. And how far you would be behind, so it's kind of, yeah, like, I mean, it's dependent on… the… However fast the given node can sync.

Karim Taam: I think, in general, it's, less than a day. That's true.

Jared Wasinger: Yeah, 2 weeks would be a long time.

Karim Taam: Hmm.

Toni Wahrstätter: Okay, perfect. Any other client has an… with an opinion on that?

Ben Adams: Isn't that normally, sort of, 64 blocks, or around there? Because otherwise you stop getting responses from, You need to be quite close to the head to discuss it. So, so a two-week period would be fair enough.

Toni Wahrstätter: Okay, this is, this is good, because this is, then way less, data than I initially expected. Yeah, I think it's good that we, have discussed it already. Maybe we can also… Bring it up in two weeks for the next breaker call, just to, confirm that again. And yeah, use the time in between to look into this, more carefully. Perfect. Then let's just move on with the next agenda item. Which is block lab access list metrics and Tracer. So basically, it would be great if we had the metrics that Katya proposed a few weeks ago, if we can get them

as early into the protocol as possible, so best case that we have them in definite zero already. careers will… clients think about this, so how far are we on those metrics? Have you already looked into this? They assume… This means no. Yeah, I'm not… I'm not sure, in terms of priorities, what we should do here, if we should… get the metrics already in Definite Zero, or if we want to focus on having, clients running smoothly on Definite Zero without the metrics set, then we're fine with that.

And on a related note, I think Felipe has a proposal for including the block accesses in the logs, just as Raph does right now. Felipe, do you want to expand on that?

devconnect: Yeah, I can… I can expand a little bit. Right now, debugging is… is quite tough, and so I think that if in debug logs we, for… Block-level access list mismatches. If we can get the, the bells, logged, it would help significantly on the testing side. And so this is… this is more just of an ask to… to try to get those as debug logs, because right now, for a lot of clients, we just have the… The hash mismatch when it's invalid? Yeah, and that makes things significantly difficult. to, to find the issues, so…

Toni Wahrstätter: Yeah, just as Penny says here in the chat, I'm relaying that. Putting the whole diff into the log, and not just the hash mismatch, that would be great. Perfect, and just to summarize those two points, I guess we are… Sorry, Mario?

devconnect: So, when you, when you say, diffs only, it's… you still full, print the full block-level access list only when diffs happen? Or do you mean that we should only print the diff? Of the block-level access list against the expectation, or could you expand on that?

Łukasz Rozmej: You only print the diff, so print the things that are missing in the access list found, and print the things that are excessive in the access list constructed, right? So don't print everything. Because access lists can be huge, and diffs can be, like, one or two addresses.

devconnect: Sure, sure. I think then the log should be very clear, like, which one has the diff, right? Whether it's the test vector or the client. Because sometimes that can't… that's not clear, so as long as we agree on that, then… yeah, I think the diffs make the most sense, for sure.

Toni Wahrstätter: Perfect, thank you very much. Then, let me quickly summarize that. So, we will likely not have the metrics and the definite zero. But we will definitely try to put the diffs into the logs, as soon as possible, just to make testing and stuff easier. Great. There… okay, what… sorry, let me quickly read the chat. What… what do we do with the orders? Yeah, this is a good point. Do you have an idea, Lukash?

Łukasz Rozmej: I would have to think about it. Is the order important?

Toni Wahrstätter: Yeah, it is important because we, of course, enforce some ordering. But, yeah, I'm thinking, is it… is it that important to have that in the logs? Because we… if we have the diff anyway. the problem might not be in the ordering, right? So I assume there will not be Back in the order.

Ben Adams: And overly? Generate a different option. We just have a generic…

Jared Wasinger: If the ordering's different, can't we just have a generic error message in the logs, and it, like… You don't really need the… the BAL diff won't… Even be necessary if that is the source of the error.

Toni Wahrstätter: Right, yeah. Yeah, that's my feeling, too. That if the ordering is wrong, we might be fine with just having a special message there. And for all other issues with the bar, we will include the… In the law.

raxhvl: The ordering itself could be part of it, let's say. We expect a certain order, and this is the given order. It can be formatted like a bit.

Toni Wahrstätter: Perfect. Is there anything else we should, discuss regarding making life for testers easier? Is there anything else? Mario or Philippe that you could think of that we might need? Stefan was just saying, any updates on including generated block access lists? In the debug that block call, Does anyone have insights there?

Jared Wasinger: I made some progress for implementing it in Geth, And I mean, I can definitely see why it's useful. But it also… it also kind of makes the code path a bit convoluted, because now, when we are executing. And we receive some error from a transaction, we can't just immediately stop executing the block and terminate, we have to, like… execute every single transaction and reconstruct the block access list, and it just creates, it makes an already complex code path just, like, a bit more… difficult to reason about, so I didn't finish implementing it yet. I mean, it can definitely be done, but it's… Like, we haven't… on the get side, we haven't… There hasn't been that much, like, review on the code yet, so… it's, like, adding another piece of complexity on an already…

Significant change to the, To the execution and, state route calculation path, but… no. But that being said, I mean, I only spent, like, a day on it, so there's a… there's… Maybe I'm… Overstating how… Invasive it would be, but… That was just… this is just my impression from having done a bit of, like, cursory… investigation into it.

Toni Wahrstätter: Has someone else from other client teams already worked on that?

Karim Taam: On the zoo side, we have it, so… should be, should be okay.

Toni Wahrstätter: Awesome. Anyone else?

Dragan Rakita: From red side, we're executing all the sections, and recreating the ball, and just comparing it at the end. So we will not fail if… No, we'll fail if we fetch something that's not existing in the ball. But we will not fail if the change is different than expected. So, there is some cases where we will We maybe could know beforehand. But we would check it at the end of the block.

Stefan Starflinger: My question is, are you providing an endpoint for the get bad blocks, and including the generated block access list that you generated in this endpoint as well, so that we can use it for debugging?

Dragan Rakita: It's… it would be easy to add, it's just executing and recreating the ball. But I'm not sure if we have the endpoint for the bad block.

Stefan Starflinger: Would be great if you could, get back to me on that.

Toni Wahrstätter: Yeah, this would be great. Maybe, maybe, would be great if, Stefan, if you could drop a message in the blog access list, Discord. Just that we have it, yeah, somewhere written there. So that we can refer clients to that message, then. someone from MetaMind, who can give an update on… on that topic?

Ben Adams: I don't expect this, request.

Łukasz Rozmej: Oh, sorry, I don't know. Mark is the only one that has the newest knowledge. He's working on it, but I don't see him on the call.

Toni Wahrstätter: Okay, yeah, no worry. Anything else regarding, things we might wanna have for testing? That we should discuss today?

Dragan Rakita: Just to say that the last testing feature is really great. It fixes all the bugs. So, it's perfect.

Toni Wahrstätter: Yeah, it looks like Ref is quite far when it comes to testing. We can… we can directly go into the client updates, so how are all our clients doing when it comes to testing with the latest release?

Jared Wasinger: Yeah, I've been mostly working on the performance side, so I haven't, been working on the, getting conformance with the tests in the latest release yet, but… It's… on my list.

Karim Taam: So, on Bezos' side, we fixed several issues, thanks to the new test, but there are still some issues in the test suite, so we'll see with the new version. We are also… we have a first version of the stateful computation background that is working, but we are trying to do some improvement on it. And we are starting to work on the perfect parallelization using BAL. It's all…

Toni Wahrstätter: Thank you very much.

Dragan Rakita: From red side, here on the tests, I think one is failing, but 40… 40 faulty ones were fixed. Yep, that's… mostly it.

Toni Wahrstätter: Someone from Aragon here? I don't think so. Let's continue on with NetherMind, or should we… or do you want to wait until Marchin will… or Mark will provide the update? Or do you, Ben, or Lukash, do you have insights into… okay, it just so pumps up, perfect. Yeah, there is… there is this one topic that has been a recurring theme over the last months, basically, about the REIT locations, or the state locations in Nepal. And… The latest I've heard so far was client teams very much were for including the relocations. Because it enables parallel and I.O.

And this is the question, like, how would we best, test it? I know that Lukash has suggested that we just test it in actual client implementation, so, for example, on ketosis or something. Is this… Something that other clients think is… might be the best thing to do, to actually determine if it's worth adding the state locations, because, of course, they add a lot of data to the block. It's, like, 50% of the… Ball size comes from the read locations, the state locations that are not changed. And we have been discussing it in a bunch of breakout calls, but We haven't really gotten to… benchmarks that were, like, super clear when it came to adding the state locations. And especially, we have to think about higher gas limits, like.

beyond 100 million, and if it makes no sense, when we are at those high limits, to have the state locations, or if the value of having the state locations in the balls is kind of declining over time with an increasing gas limit.

Łukasz Rozmej: So, that's a very good question, and exactly we can do different things. We can… Blindly at state locations, we can blindly exclude them. We could try to… Implement kind of both and benchmark it on different… on different versions and different… different kind of blocks. We can all… then the last thing, we can always try to be smart, and by smart, I mean… Do something like… include them conditionally, right? So, by conditionally, I mean, we could have some heuristic, and that heuristic is that if the transaction does at least N reads, right, or something. Then you include them, otherwise you don't, right? So, there is way, way… different strategies to target different cases. I think we should be aware about the size increase, so generally, I'm a bit hesitant about including them blindly.

But on the other hand, if there is an edge case transaction that does a huge amount of reads. And okay, with 16 million gas limit, it's still… it's still bounded, so it's not that bad as without it. But even then, maybe there's some threshold where it's worth including, so… We can do multiple things, and, depends how… How much precise in-tuggling bottlenecks we want to be.

Toni Wahrstätter: Yeah, my… my concern with, having, like, heuristics for the reads is the following. it's like, imagine disk I.O. becomes a bottleneck, then the worst-case block would not only be worst case in terms of having a lot of S loads, but also having additional data in the block. So it's basically, like, for blocks that are already, like. very bad to execute. We would be like, okay, let's add additional data to exactly those blocks, and the blocks that Our lightweight will probably not have the read values in them. You know, it's like… It feels almost, like, counterproductive to be like, okay, our heaviest block will now also include additional data.

Łukasz Rozmej: By heaviest, by network transmission size? That's what you mean, or…

Toni Wahrstätter: No, by just workload for I.O, for example, assuming, for example, a block that does only S loads.

Łukasz Rozmej: Okay, but that block now is, can… those S-loads can be parallelized, right? Like, completely. At least for…

Toni Wahrstätter: Yeah.

Łukasz Rozmej: worse transactions, so actually, we kind of alleviate the I.O, bottleneck here, by including the reads for those. So, my expectation would be the opposite, that we actually have the worst blocks, that Let's say we have, I don't know what gas limit are we targeting? 300? Okay, 300 gas limit, so 300 divided by 16, that's around 19 transactions, worst, and if they are all doing loads, so… we could… you know, parallelize that. The networking is the bad thing, right? That we get hit by this huge increase in balls. So… I don't know, it's hard to say without the numbers, it's hard to say without benchmarking, it's hard to say without… Additional complexity of clients that will hit us here and there.

But… Yeah, I think, like, in most cases, we don't need readables?

Ben Adams: So they would be just an overview.

Łukasz Rozmej: overhead for now.

Ben Adams: working in the proposal.

Łukasz Rozmej: But here and there might be a transaction that we will say, oh, now this transaction, all transactions are finished in few milliseconds, and this one transaction now tails for next 100 milliseconds. And if we had balls for it, read balls for it, it would just… Be 10 times less. So that's my guess. But, without measuring it, it's hard to say.

Ben Adams: One, kind of objection I have to the parent. Read slots is they don't… sorry, the read locations is they don't have what transaction they're ready. So they're all just, lexographically ordered, which… is the equivalent of, like, a random lift, so you… you can't overlap the running of transactions with the I.O. You can't… you can't say, alright, I've loaded the data for the first 10 transactions, so I can start running them now, while I'm still loading the data for the following, you know, 100 transactions. You don't know. what date you're reading? Obviously, there's a size buffoon going back into the.

Toni Wahrstätter: Yeah, that's true. I mean, this is in general something we can discuss, also alongside having the state locations in the house, if it would make sense to add transaction indices. Of course, the problem here is, as soon as we add transaction indices, we cannot aggregate them as well as we can today, because today, if we have a right to any storage slot, no matter in which transaction index, we don't include that right as a read anymore. Of course, if we would now add transaction indices to REITs. Which are not needed for parallel execution. then we would have to kind of deduplicate those entries, and you would have the same storage slot as a write and a read, with potentially different transaction indices. So, my expectation would be that the block lab access list would definitely grow in size by…

Ben Adams: If… if we assume, that you parallelize transactions in order, so you… the first transaction you run is… The first transaction, and the second transac- you know, the second transaction, even if it's happening at the same time. It's the second one. Then you could… achieve the same effect by just marking the first transaction that that slot was read in. Not, not every transaction that's read. So you just need to say, oh, right, this is read in transaction 1, this is the first time it's read. Transaction 1, transaction 50, or whatever. And then you can You could shed… you'd have enough to start scheduling. You wouldn't need to record, you know, this is read by transaction 1, 5, 10, 11.

Łukasz Rozmej: Yeah, so… exactly, so if we had write first and then read, we don't need to include the read one, and if we had read first and write later, then yeah, we would have duplication. But we, again, wouldn't have to… Only once, right? So only one duplication. We don't have to mark reads for subsequent transactions, like, only the first occurrence of a read. In the block.

Toni Wahrstätter: Yeah, this makes sense. Then I guess the question is, is this worth the complexity? I guess this might also depend on the numbers we are looking for, right? So, with numbers, I mean the benchmarks that we might still need to get, yeah. Because, of course, this would be, like, a worst case that, imagine, we have, A transaction that is at the very end of the block, and also scheduled to be executed at the very end of the block. But… The prefetcher starts prefetching the store slot of that transaction first, because they are lexicographically first. And then we would end up with execution not having the cache ready to be consumed.

Ben Adams: I think it'll become more of, Leaving a lot on the table as the… as the blocks get bigger. That's the slots get shorter. Because whether, you know, where you want to do a balustr. Or, lovely better.

Toni Wahrstätter: Yeah, I totally agree, but this is definitely… My only concern would be complexity here. like, will we find… like, this rule that, you and Nucha suggested, that… If there was a write with a lower transaction index, then the read would just not be included at all, like we do today. We don't include it if there is a write. And then, like, if there is a read with a higher transaction index, with a lower transaction index, then we would actually include the read with the index, kind of telling the client this is something that needs to be prefetched right at the beginning. So what do clients think about complexity here?

Ben Adams: I also think, if they're in the same transaction, so you do, do an S load first, and then you do an S store later in that same transaction, that would only need to be the lead, because you don't… sorry, that would only need to be the right. Because… The reason implied. So it would only be if it was… I read it. in a… Earlier transaction that had an array. Hopefully that shouldn't… necessarily add too much superposition. I don't know how much overlapping there would be an exact job.

raxhvl: We could also wait for some data to…

Toni Wahrstätter: Yeah, go ahead.

raxhvl: We could also wait for some data to come in. I think, every client could have their own niche parallelization strategy, and we could see, like, what part of execution is idling on I.O. And once we have more data. This would also vary a lot based on investor advice, so we could see, like, where are the rare, limits for education idling. And then maybe… For lower draftsmen, maybe under 100 million, we're not even need it now, so we could just add the complexity as we go later.

Toni Wahrstätter: Yeah, I can see that point, but I think… I think, like, we should definitely design a system to work, like. to be, like, future-proof. For example. Ready to support, like, 300, 500 million guests, and so on. So, this is, like, why it's so difficult right now to tell those things, because we have no experience with such high gas limits, and as Ben says. the REIT indices, so having transaction indices mapped to the REITs, might be becoming more important with increasing gas limits, because Of course, the worst case would then just be a transaction at the very end, and the prefetcher starts prefetching everything that is read in that transaction, although it's only needed at the very end of the block.

raxhvl: I agree. Yeah, we should, we should target, Like, getting numbers, or at least hearing them alone. And we could also see if we can, like, optimize our parallelization strategy. To work around this. And only the larger output would be to add more complex material.

Toni Wahrstätter: Right, yeah. Yeah, I guess at least from the spec side, if I think about the specs, this shouldn't be too complex, because this would be a change to this quite isolated in the block access list builder. Basically, the part of the code that takes the… everything that was tracked in the state, and then builds the aggregated block access list. So from a spec side, this should be Okay-ish to do, and not, like, super complex. But of course, we were just talking about removing the read values, and now we are like, okay, let's add some more inputs to the reads. I get the argument that, okay, reads make more sense if we Have, like, this additional, 8 bytes, which is, like, the transaction index attached.

But yeah, I would say we just, wait for numbers, and then we decide upon those things.

Łukasz Rozmej: One comment about, having some data about big blogs. So Nevermind, Camille from Nevermind, with Carlos are doing those, some experiments where they have a tool that combines new payloads from mainnet, they're using from mainnet, that combines new payloads into very big new payloads. So, they try to aim around 1 giga for new payload, for example, but this is more or less configurable. And then they replay those extremely big blocks on the, on the notes, to get some performance metrics from it. So, this could be adapted to get even more better, maybe real-life numbers, or if you want to check anything there. That sounds like a good idea, to, to get some experience with.

Real-like, blocks that are around 500 or 1 giga gas.

Toni Wahrstätter: Okay, yeah, this is interesting. So this is… this is our… this is separate from Bloatnet, right? Because this might also be something we want to test, like, how will everything behave on a bigger…

Łukasz Rozmej: So, currently, they were working on mainnet, because they wanted to get, I think, like, more mainnet-like numbers. But this could be used on BloatNet, too, with no problem. But there is a lot of setup, because each client has to have a correct snapshot, etc, so it can be rewinded to that point of time, so it's… it's, It's quite a manual process, or at least the… maybe not manual, but there is a lot of manual, like, preparation in order to automate this process, and it's not really very, like, Easy to just adjust it on the fly, right?

Toni Wahrstätter: Right, yeah. Okay, but this is good to know, thanks.

Łukasz Rozmej: I think they will present it somewhere tomorrow about it, on some Nevermind-related presentation, if you want to. No more.

Toni Wahrstätter: Yeah, maybe you can drop the link, if possible, in the botlab Access List breakouts.

Łukasz Rozmej: Fuck.

Toni Wahrstätter: channel, or… or you send us a DM, yeah.

Łukasz Rozmej: Yeah, I can, I can ask them, For, for the link, too. Okay, thank… no worries. Dragon?

Dragan Rakita: Yeah, I'm just saying, commenting on my gut feeling. the block journal and execution journal is not just loading the data, it is some CPU. So, having what needs to be read from the database makes sense, because we want, when that CPU happens on… 4, 5, 8 threads, whatever we have. We want to have something done in the background. So, having what needs to be re-read It seems that it's going to be impactful. Knowing order, but needs to be read I don't think it's going to be that important. Because even if some thread needs to wait for something to be read. We still… we are not going to be… Bottlenecked by that much. It's not something that's going to take, 100 millisecond.

So, there will be slightly… there will be a need for mechanisms that will prioritize. The… the reads that are happening now, in comparison that older it. But I'm not sure if the benefits, like, this is just a gut feeling, like my comment. I'm not sure if adding the indices to the reads are going to matter a lot. At least that's… this is how I'm thinking now. Without data, this is just, like, hypothesis.

Toni Wahrstätter: Yeah, this makes a lot of sense, and this was also, like, where… what I was thinking about, that maybe as long as you can do parallel I.O, There's one process busy anyway, and you are just, like, executing in the meantime, and then you would at least have everything in cash already for the last transactions, and for those first transactions, where it would be handy if you have them ready in the cache. you would just do some prioritized I.O. for exactly that transaction, so you can continue executing. But this, yeah, I mean, we need the numbers for that, we desperately need them, but I guess… It's… Yeah. not so… not so easy to tell, because I also have hard times, telling what is the actual, efficiency gains we get from parallel I.O. versus splitting, distributing transactions over threads, and then having this implicit parallel I.O. to

Or I should better frame it as batch I.O. versus parallel I.O, because if batch I.O. is really, like, super powerful. then it might… and, for example, I could… you could imagine, like. I.O. would take a fraction of the time that the actual execution takes. Then, having the indices might not make much sense.

Dragan Rakita: Just additional comment, reading in… when you know the data is ordered, at least MDMEX have, like, the cursor that can… it's faster to read, like, when you have ordered data. So, having a batch of all the reads that needs to be read with the sorted order is going to be faster than reading it by randomly. I'm not sure by how much. That's, like, big information here, but I would say 1.5 to 2 times faster.

Toni Wahrstätter: Awesome, yeah. So, what do we think about, those numbers that we actually need? So, what would be the next steps? that, what are you guys planning on doing so that we might get those numbers from actual implementations? Would you need anything from… Would you need any specific tests, or… Like you said, with the increased block sizes and that kind of stuff. Like, how would we actually get to those numbers that we would want to have, to have, like, certainty over adding The state locations, or even adding, like, transaction indices to the state locations. Yep, Ben?

Ben Adams: I think in the guests… testing, or the… or maybe the BlakeNet channel. So Joachim came up with… he came up with this little… Tests that were performing quite badly. With S-loads in their stores, and it seems… They're exactly the access initiative address, so we may already have seen that. But I'll look into our family.

Toni Wahrstätter: Have other clients looked into… Yeah, go ahead.

Karim Taam: I just wanted to say, maybe we need multiple devnets with different configuration at the same time, and And we can see the different performance.

Toni Wahrstätter: Yeah, of course. This comes, of course, with some overhead that would be nice if we can avoid, like, having multiple defnets with different configurations. I totally get the argument that it's, like, difficult to simulate it in a vacuum without having actual client implementations running. Yeah, I'm not sure. What do other buy-in teams think about this? Should we actually… go for different definets, or would it be enough to have, like, a DEFNET that has it implemented, and we measure everything from there?

Karim Taam: Maybe not by configuration, but for example, one DevNet with a big state, one DevNet with a… as it may be a shadow fork of mainnet to see the current performance. just some kind of configuration like that, and in one, for example, in the blood devnet, we can have multiple BISU with different flags, but .

Ben Adams: Yeah, would it be useful to be able to pass in a flag to turn off any block-level access lists, Optimizations, like, taking advantage, not doing any preloads, etc. And then, not doing any parallelization. And then literally run the same chain. one with the flag on, saying, don't do any block. I mean, obviously, you still have to, perhaps, verify it. And then you'll… you'll see this is the effects from the optimizations, you know, parallelization, doing… Loading state, etc, and here's the effect without the wheel. We'll see what the appliance does look like.

Toni Wahrstätter: Yeah, I would agree. just a little bit hesitant because of the complexity there. I don't want, like, clients to have, like. overly complex, implementations on the DevNet. Not sure, if this is actually super complex for clients. To have, like, a flag that turns on and off the state locations. Basically, the state locations are always there, but that turns on and off if we actually use the state locations.

Ben Adams: Sort of have that for, I think both Netherland and REC, and maybe some others, so we do cache and pre-compile results, and in order measure that for, HERFNAP. And for the, the benchmarking, we have a flag that literally switches off parts of Switches off the optimization, so we can get access to the raw underlying, you know, what is the performance of this reconfile. So, I think it's… I think it's possible, if it would produce, worthwhile results.

Toni Wahrstätter: Maybe a direct question to Jared and Dragan and Kareem, what do you guys think about this?

Karim Taam: I don't think it will be difficult to have some flags, We have already some flags, we already started to have different flags with different configuration for block access list, in order to be ready for this kind of test, so I don't think it will be difficult. To add more flags, or two. To add more configuration.

Dragan Rakita: I'm not exactly sure. What flags are we talking about? Flags that would remove prefetching of the reads? Just that one.

Toni Wahrstätter: Yeah, exactly.

Ben Adams: And, and then the… Paralyzation, analyzing, if we've done it.

Dragan Rakita: Can you repeat that, Ben?

Ben Adams: And any parallelization, or, sort of using the… SCOR changes to then use that to parallelize all the transactions. I assume. We'll hit this up.

Dragan Rakita: Okay, removing parallelization of the sections. Okay. Those two flags.

Ben Adams: So you, so you, so you can do a, This is without block-level access, this is with block-level access.

Dragan Rakita: Makes sense.

Ben Adams: And then, you know, the next day, you put gas to 500.

Toni Wahrstätter: Have you been looking into this, or thinking about this already, Jared? Is there any concerns that you would have regarding adding a flag that turns off prefetching the state location?

Jared Wasinger: Yeah, we can do it, And it should be possible for the transaction parallelization as well. I'm… I guess I'm just not… fully understanding how the insights gained from the… so, sorry, I kinda… I… my attention kind of trailed off for a bit, but, I… I'm not really understanding how the… the insights gained from this Will… Help us determine whether the… current read format is sufficient, or if we need to add the transaction indices. I mean, yeah, I can… I can add flags to disable these various, the… the… these… these flags are possible, but it just doesn't… I'm struggling to connect the dots in my head.

Toni Wahrstätter: Yeah, I think there are two different things. One, we want to know the difference between having the state locations versus having… don't having the state locations, and for that, a flag that just, decides if we use the state locations from the ball in the prefetcher or not. Would be enough. regarding the transaction indices, this will be more complex, and I think we should more… first focus more on the first, determining if the state locations are worth it, because if, for example, Jagan's feeling is, right, that We might not need the transaction indices for the state locations. Then we will find that out with only that flag, with only the flag that turns on and off the state locations.

If we find out that at a higher gas limit. State locations might be helpful, but only if they actually included the transaction indices. then this is different, but I would assume, we can just continue with having this flag that only targets, basically, what the prefetcher does, Either it runs with the state locations, or you would just run the prefetcher as it runs today.

Jared Wasinger: Yeah, okay. So, for us, it would be easy to… to enable or disable The asynchronous loading of the read locations, but… The prefetcher as it runs today is not… a thing, post BAL, forgetth, but yeah, otherwise… Okay. Yeah.

Toni Wahrstätter: Okay, yeah, then, my fault. I was just assuming that the loading from disk of the state locations would be prefetching, but yeah. Sorry for that confusion.

Jared Wasinger: No, it's okay.

Toni Wahrstätter: Awesome.

Jared Wasinger: Wanted to clarify.

Toni Wahrstätter: Yeah, then let me… let me quickly summarize. So, we decided today that we do the… we talked about, SnapSync, that we might keep some buzz around, at least to enable SnapSync, and then we also serve them for SnapSync. Regarding the logs, we decided that we all put… so all client teams put the divs into the logs, so that, this makes life of testing team easier. And finally, we also decided that we want to do, a defread with potentially adding, or with adding a flag that turns using the state locations on and off in order to give us numbers on how important the state locations actually are. Great. We are already at the end of the call. Is there anything else we should discuss?

Karim Taam: I just had a question, are we talking to add this flag for the next DevNet, or another one? Because, for example, in Bezu, we didn't implement it yet, the prefetching from the block access list, so… just to see if we need to pre-authorize that or not.

Toni Wahrstätter: Yeah, that's a good question. My feeling would be we would probably want to have that on DevNet1, just because we… the earlier we get those numbers, the better. Or for DevNet 0, sorry. Yeah, I would say we should edit for that next year.

Stefan Starflinger: I also would like to say that right now, we still don't have 3 clients that are consistently working and not forking off. So I still managed to get Nethermine and Geth to fork off, and kind of the con… Or the prerequisite for us to, launch the DevNet properly, so that clients can work with a little bit of a transaction load, especially I would like to remind everyone to try out the EVM buzz. scenario, the spammer, and see that the client, is stable with it. Red, seems to be working well, and Diesel seems to be working well so far with it. But otherwise, the other clients are still struggling.

Toni Wahrstätter: Perfect. Thanks a lot for that update, Stefan. Then I would say, let's… yeah, is there anything else someone wants to bring up? Otherwise, we can take the discussion offline. Awesome. Then, thanks everyone, and see you at the next Waco Calling 2 weeks.

Jared Wasinger: Yup, bye-bye.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.