Block-Level Access Lists Breakout #9

Transcript

Toni Wahrstätter: Hello, everyone. Let's give people a few more minutes to join.

Perfect. Is the stream ready?

Akash | ECH: Yeah, be alive.

Toni Wahrstätter: Great. One second… Just pulling up the… Agenda of today. Awesome. Is my audio okay?

Barnabas Busa: Yep.

Toni Wahrstätter: Okay, perfect. Welcome, everyone. I post the agenda into the chat. Welcome, everyone. Today is December 17th. It's the 9th EAP7928 breakout call. And we still have something on the agenda that, basically was the topic of last week. Namely, Namely, how long do we store block lab access lists, and how do we make them, accessible over the engine API? If you look into the agenda. There are 3 PRs, one of them, the second one is against the execution specs, and the other two against the EAP. And what is proposed in there is basically That we make sure that we keep at least enough block access lists Until the weak subjectivity period, and the rational behind this is we want to make sure that, if a node goes offline.

for less than the week's subjectivity period and wants to catch up by re-execution, then there must be enough block of access lists around, such that, Catching up with the head is actually feasible, because, of course, you might need parallelization in the context of re-execution. sync. So, yeah. Have people had time to look into those? Especially, do client teams have any opinions? By the way, I almost forgot, this also includes the topic of removing the block web access list from the block body.

Yeah, I guess there's, no one against that. It seems like this was the broad consensus that we reached. Also Karim is saying that in the chat, he's okay with removing it from the block body. I remember Gary and Jared were also advocating for removing it from the block body last week. The question that remained was only. How does it… how does the ball get from the CL to the EL? So far, it has been passed from the CL via the engine API using the execution payload. And Yeah, Justin Karim and me had a call with POTUS yesterday, And… POTUS mentioned that the execution payload header will not be stored on the CL site anymore, so we don't have to care about the execution payload header, but we would then just put the block-level access list in the execution payload.

It goes to the execution layer. And the executioner would then need to take the block-level access list, hash it. And then start the state transition. And afterwards, the CO would use the… Hash from the provided block of access list and put it into the block header. And in the end, after execution, the EL would compare the generated block level access list that was generated during execution with the block hash. And if this check, matches. it means that the block level access list was actually exactly the same that was derived during generation. So this would make sense. This also means that we have to store, at least 15.7 days of block lab access lists. This is around

Yeah, you can think of it as a max of 10GB. Doesn't fit into memory anymore. Great. then we can, I guess, if there's, no opinion on that, I would, interpreted as consensus here. Because it feels like we already reached that point in the Discord discussion, so… Just wanted to bring it up again, that everyone is on the same page. What does this actually mean for our tests? Maybe, Rahul or Felipe, do we have to change a lot there now?

Felipe Selmo: Yeah, that's a good question. Since the block lab, block access list is no longer in the, block body. I think we need to… Including them in the vector. Somewhere else. Just, to have a reference to it. And we'll have to figure out how to arrange that.

Toni Wahrstätter: Yeah, it's interesting. What is, like, the starting point? from the tests, because we start basically at the EL block, and not at the execution payload, right? Because the execution payload would basically be everything passed from the CL, which is a lot of header fields, plus the withdrawals, if I remember correctly, execution requests, and… The transactions itself. Oh, maybe… Maybe, do you know, Jochen, how we are treating this, or what would be the cleanest way to… To, to implement that in the testing framework.

Jochem Brouwer: So, what's specifically implementing?

Toni Wahrstätter: The fact that the block type access list is not part of the block body anymore. So, yeah.

Jochem Brouwer: So, where's this currently, then, a part of? .

Toni Wahrstätter: So it comes from the CL as part of the execution payload.

Jochem Brouwer: Right, to be honest, I don't really know. I think we have to… I will check, how the… Blobs are basically handled, because this is somewhat similar. At this point, I would not know.

Toni Wahrstätter: Yeah, I'm wondering if we even need it, if we even need to change something at the tests there. For example, this is a question to you, Felipe. Are the test cases, are each individual test case also checking that the bul is correct? Like, doing a hash check at the end?

Felipe Selmo: Yeah, yeah, so… Obviously, so we all… we also have, like, the engine version of the tests, right? And these don't actually have… the full bell. This will take… Just the block access list, hashed. And so, honestly, I think for debugging purposes, it's really nice to have them in these test vectors. But if they're going to be… If we're gonna have the hash, that's proposed. then I think this may be the… all we need. It's just not gonna be ideally Very debuggable, which is not… Yeah, it's not ideal while we're in, like, such a large testing phase.

Toni Wahrstätter: Right, I can see that, yeah. Because the workflow would then look like the following. The CL would pass the execution payload to the EL, In the execution payload, there is the full bar. and you will then hash the bul. And this is basically where we are currently starting, right? Except that we have both the hash and the bul. And now we would basically only have, like, the hash. We would then run execution, would derive a generated bul, would hash it, and compare it with the hash and the header. Yeah, this is… this is interesting, because it's… it's like the bowel provided by the seal. doesn't… does never need to enter the state transition function on the EL.

Because the EL, like, if you execute transactions, you will just generate the ball for validation. But you never need… to… Have the provided bul enter the system in that sense, because it's… it will just be used to… to parallelize. And in the end, we need a hash of it, at least in the header, to make sure we can actually validate it during execution. So the power from the CL should also never enter the test cases, but I can see the argument why it makes sense for debugging.

Jochem Brouwer: So, I just checked, like, it seems that for the blobs, So just to… just to compare that, because in the blobs, it seems that in the test format, indeed, the blobs themselves are not provided, but the blob version hashes are provided, of course, in the block. So, yeah, for… in this case, The debugging would be harder. But of course, like, this is, like, the end result, like, if we were to remove it from the block body, then it's indeed, as you say, like, not part of the state and such information. So this is, like, a logical result, but for the debugging purposes, if we think this is, wanted, then I think we should feedback this in general to the testing framework.

That we somehow need, like, this data. To also help de-booking this. But this is the part of how we test it, yeah.

Toni Wahrstätter: So then we could either have, like, an optional field in the state transition function or something in the specs? Are you thinking along those lines?

Jochem Brouwer: So it will be more part of, like, the testing framework itself. how this will be handled in the testing format. So it needs, as you say, like, these optional fields, like, maybe like this metadata or something. That, includes the fill block access list. Which is optional. Because in the end, the clients will… the EL clients will only verify that the hash of this thing is correct. And for debugging, we would, of course, need the expected block access list. So this will be more part of, like, the testing framework, so we need, like, some kind of… how we would fill these tests. You would need some kind of endpoint at the EL, or debug endpoint to get this… the,

The expected book access list in order to include this in the testing format, so in the fixture which the clients would consume when testing this.

Toni Wahrstätter: Okay, yeah, this sounds better than what I was thinking of. Awesome, thank you. Yeah, this seems like this is something we need to figure out, like, because we can't… We can't lose, this nice feature of having, like, good debug logs that compare them.

Jochem Brouwer: Yeah, I think what we should do here is to also look a little bit back in how the blobs were tested. Because I think this is somewhat similar. Because I also don't see, like, the, blobs In the, fixtures themselves. So I'm not really sure if that was a problem back then, how this was actually tested.

Toni Wahrstätter: Yeah, I think this will be…

Jochem Brouwer: Yeah.

Toni Wahrstätter: Blobs are different, I think, because they don't really interact with execution, so there was… we probably never had, like, a script that compared two blobs with one another, right? You would probably create… have many tests for how you create a blob. But then, on a testnet, I… I don't know, actually, but I would assume we did not compare Blobs with one another in the tests.

Jochem Brouwer: Yeah, that would actually make sense, yeah. Okay, let me think about this a little bit, how we should approach this, if it changes, yeah.

Toni Wahrstätter: Perfect, thank you. Is there anything else we should discuss under this topic of specs and And, testing updates.

Dragan Rakita: Maybe I can… About this discussion, are we talking about changing the JSON format for the test? I feel that current format is fine, I'm not sure why we need to change it. I'm not understanding that point.

Felipe Selmo: Yeah, So if the block access list gets removed from the block body itself, the blocks part of the vectors Would no longer have this, like. black access list field, right? And I think what we're just trying to discuss is It's super useful for debugging,

Dragan Rakita: Yeah.

Felipe Selmo: And maybe it can perhaps be, like, an underscored field or something that says, like. This is not part of the actual block body, but this is, like, useful information for testing. And I think, yeah, I agree with Jakam that, like. Somewhere, like, if we decide we want to keep this for debugging, somewhere in some metadata section, it might be really useful to just keep it around.

Dragan Rakita: I would even just leave it like that, because it… you have additional fields, LLP or something like that, that is basically… we have a need to have special type to decode the JSON block that's inside the JSON, so we don't care if it's underscored, if this is named something else, or if this name… Block, access, list, ball, or whatever. Okay. Thanks.

Felipe Selmo: Yes, yeah, that makes sense. I was looking at the RLP just now as well, and…

Dragan Rakita: Yeah, it's special format, it's special, like, it's not… you cannot use block to… to deserialize that kind of JSO, you need special type that's going to do that. But, yeah, I think that's, like, okay, it's a small decision.

Felipe Selmo: Yeah, it's a small decision, yeah.

Toni Wahrstätter: Perfect, yeah, we can also resolve that topic, async afterwards. After looking deep into it, like, what is the… Most efficient and clean way. To still get, like, the logging, but on the other hand, we also don't want, the specs to deviate from what client should actually do. Any other opinion on that? Also on the other topic of the retention period? So there are two topics now. The bowel being removed from the block body. Stored separately, and then… Secondly, the retention period of… 3,533 epochs, which is, like. A little more than 2 weeks.

Perfect, then I guess we can… Move on to the next topic, let me quickly check. But there's nothing in the chat neither. Looks good. Perfect. 10… the… Last update on the agenda that we have today is client updates on optimizations. We can combine it with, in general, client updates. Kind updates on optimizations is very specific, because As discussed a few weeks ago. we need certain optimizations also for the repricing work that is going on for Glamstadam. So, for example, in order to get reliable benchmarks for the repricing, we at least need a client Or, or better, multiple clients that have parallel execution implemented, and parallel I.O. Because those… Would then inform us, especially on the state-related pricing, and people are… Blocked by that.

for doing benchmarks, for the repricing. Is there any work going on? I remember, like, there were clients that were already working on parallel I.O. Maybe someone could give an update.

Karim Taam: For example, on Bezoo, so, we have the parallel execution implemented, so it's, we are doing some tests currently. For the parallel I.O, as I said, it's not yet implemented, we didn't have time for the moment, but, It depends on when we will start the DevNet, but I think we'll be able to have something really, Those are next divinets, in my opinion.

Toni Wahrstätter: Awesome, thank you And when we say parallel I.O, we mean, this parallel batch I.O. Where you load everything in one… in one go, basically, right?

Karim Taam: Yes, yes.

Toni Wahrstätter: Okay, yeah, because I also referred to it as parallel I.O, but I think parallel batch I.O. or batch I.O. Might be even more accurate, just to not, mix things up. Any other… Yeah, go ahead.

Karim Taam: I just wanted to say, it depends when we want to start this DevNet, because there is some vacation, etc, so… Yeah, I don't know when it would be ready, but…

Toni Wahrstätter: Yeah, what's your plan, Stefan? Maybe you can give a small update on…

Stefan Starflinger: On how we're doing here. So, it seems, that clients are much more stable now, which is really, really nice. So, the critical path most clients, seem to agree on now, which is great. We can start the dead at one. I think… Today or tomorrow, even. But unfortunately, there are, like, holidays coming up, and most people won't be doing much. But I mean, if that's not a problem, we can… and if some people are working through the holidays, I can go ahead and start that at one. But if most people will say, like, no, probably not really worth it, we can also do it, beginning of January, straight after. I'm open to… to that. Both is fine with me.

Mark Holt: Yeah, so, from an Aragon perspective, we're in a bit of a strange place, in that… we're quite well advanced with optimizations, but we're not running… we're not passing Hive tests, basically. So… I… so… we've got two optimization tasks going on at the moment. One is… Execution optimization. The other is parallelizing our, our, commitment calculation. I think the execution parallelization and the kind of parallel I.O. We're just testing at the moment, and probably by the end of this week, we'll have a first version of it. Now… when we're ready from a BALS DevNet perspective, is dependent on getting the, the hive tests that are failing at the moment, passing, which I'm also shown

basically, I spoke to him before the call, and he's expecting that to get fixed by the end of this week. If he doesn't, because I'm working at least part of next week. I'll switch from doing the optimization work that I'm working on to actually just getting us through the, Through the, the hive testing. So I think probably by… kind of… by… mid. by the week between Christmas and New Year, I think we should have… we should have both of them completed. If that makes sense.

Toni Wahrstätter: Awesome, yeah. With both of them, you now meant parallel execution and parallel post-state route calculation, right? Or was the…

Mark Holt: Well, we're basically… so, parallel execution, parallel… so, parallel… what we're doing is parallel execution, parallel state calculation, right, and parallel I.O. There are kind of three pieces. And… we are exp… The… our latest iteration of the code runs all three of those things in parallel with each other. And also, execution and… Commitment calculations, R. parallel themselves, if you see what I mean.

Toni Wahrstätter: Yeah, okay, I think I gotcha, yeah. And I can think of the parallel batch I.O. as basically a process that just, goes through the ball, fetches everything into cache while the execution is already, like, starting.

Mark Holt: Yeah, and the commitment calculation is also starting. And effectively, the way that it works is whichever gets to read… whichever of those three processes gets to read the data first will read it, with the expectation that the… the… the data fetcher will get there before the other two, if you see what I mean.

Toni Wahrstätter: Yeah, this makes sense.

Mark Holt: Yeah, yeah, because given we start fetching data when, you know, as soon as we get the block, by the time we get to the other parallel processing, the assumption is that we'll have everything, everything cached by then. If for any reason we haven't, we'll have, you know, we'll go and get it during that parallel process. And then we'll see how fast all that goes.

Toni Wahrstätter: Perfect.

Mark Holt: Yeah. I mean, I think the slight problem that we've got is we really need to be on the BALS testnet to get any decent data out of those benchmarks, because we need the bowels in order to actually have, like, perfect parallel execution. See what I mean?

Toni Wahrstätter: Yeah, I can see that. Okay, yeah, thanks a lot for the update.

Jochem Brouwer: Yeah, maybe just to chime in on the benchmarks and the testing, we are currently also working on benchmarks, which are specifically targeted for the block-level access lists, and for these, what we do is we create benchmarks. Which, have transactions, which you cannot execute, Parallelized, because these transactions interact with each other. So, With these benchmarks, you should see that if you use the block-level access list correctly, then you can actually parallelize those, and if you don't use block-level access list, then you have to execute these transactions sequentially. So this should also be part of, like, the benchmarking effort that you can really see, okay, does it work or does it not work? Because also for these DevNets, like, if you have a very optimistic, or not very optimistic, but, like, well, a somewhat optimistic prefetcher.

where you just start out transactions, where you start on the pre-state of the current block, then in most cases, I would say, and I think Tony also has this nice site. Where you can see, like, the interactions of these transactions. Like, most transactions don't actually interact with each other. So, if we would, well, I would say somewhat naively, look at the block of access to this result on mainnet blocks, then we can already somewhat parallelize those. But of course, we want to see, like, the actual effort if we want to create… if we create some blocks where you have to execute these transactions sequentially if there is no block-level exit list, and when there is a block-level exit list, you can actually execute this parallelized.

And then you can see if you have implemented this correctly, because this should, in the end, lead up to, like, well, close to the amount of threads. Which you run, as a kind of speedup factor. So I just wanted to share that this is an effort for the benchmarking part of this.

Mark Holt: Yeah, so, just on that, it's kind of… on mainnet at the moment, we get, like, a 2.5 parallelization factor by just running parallel execution with no bounds, so I'm not sure whether what you're talking about is the difference between serial execution And, And bowels, or the difference between running, let's call it non-bowel parallelization, where you get, interactions. Now, obviously, if you create a block where Everything's dependent on everything else, it will become serial. Basically.

Jochem Brouwer: Yes, but the key point of the block-level access list is that you can now have the pre-state of the transaction, and read this from the block access list. So that is when you can actually use the block-level access list to know what the pre-state is before, like, any transaction of the block. And then you can use that to parallelize the transaction. And if you would not have the book-level access list, then this is not possible, because you would need to execute the previous transactions in order to execute the current transaction, because you need to.

Mark Holt: Yeah, only, only, only if there's a dependency.

Jochem Brouwer: Yes, exactly. So these benchmarks, they create these tests where, without a block-level access list, you cannot Execute them, well, how you would execute them now, like, parallelized, because then you would end up with a different statehood than if you would execute them, sequentially.

Toni Wahrstätter: Yeah, I understand it as essentially as a worst-case for today scenario. Basically, it would force you to sequential execution today. And with Block Live Access List, you should get to parallel execution, and then you should, like, see a steep drop in execution time. Would be interesting to think about, okay, could we have, similar scenarios, for I.O. and the post-stateload calculation? I would assume, like, I.O. is just doing as many S loads as possible. We have benchmarks on that, and parallel state route, probably doing as many S stores as possible, which we should also have. Because those will be interesting to test, even with block web access lists afterwards, like, what is the… Worst case, is it, like, a block that is well-parallelizable? Because every block is parallelizable, but has a lot of overhead when it comes to post-state route, or is it I.O?

This is still something we have to figure out, and also depends a lot on how we do the repricing in the end. Awesome. Any other client team that has an update on… How is it going, and how is it going, especially with regards to the optimizations?

Marc Harvey-Hill: I, for another mind, made some good progress on the parallel transaction execution and state route computation. Haven't started on the, batteries yet. Also just, fixing issues, so it seems like we're passing all of the RLP tests for the new, release. I'm just, checking out the engine tests as well. Yeah, that's it.

Toni Wahrstätter: Thank you very much. Any other client team with an update that… We forgot that we didn't cover so far?

Dragan Rakita: From Redside, parallel state route is in progress. And, parallel execution is… I think Matt started working on it. Yeah, we had one bug with self-destruct with the last, ball. 2-0 test that I fixed. Yeah, that's it.

Toni Wahrstätter: Great, thank you very much. Did we forget any client team now? Is someone from GAF here? Otherwise we can also check async. Doesn't look like it. Yeah, nevermind. Right, is there any other topic we should discuss today? I would say we definitely need… would be great if we decide, should we launch the DevNet, pre the holidays? I saw Stefan ask that question in the chat. It feels like it's… it might be worth to do so. What's your feeling, Stefan?

Stefan Starflinger: I think, I can go ahead with the clients that are running smoothly, and it's not a problem to add clients later, as well, as soon as you're ready. I can deploy that. It's just not good to start with two little clients that are stable.

Toni Wahrstätter: Right, which clients, are stable right now? I saw it's Raph and NimbleCL, right?

Stefan Starflinger: Nethermine, and Noombus are running pretty smoothly at the moment, and Bezos just has one issue with the call. That they're fixing so far. And Aragon, I'm still waiting on the BAL Definite One branch to… to try that out.

Toni Wahrstätter: Awesome. Yeah, this sounds like it might be worth it, huh?

Stefan Starflinger: to phrase it differently, is there any client that, will benefit from having the DevNet up, over the holidays?

Toni Wahrstätter: Yeah, seems like we're still very early and we have time, so… I'm also fine with delaying the deflate, and then making sure we have the… The bowel block body separation already implemented? And same for the retention period, and that… kind of things. I'm wondering, for client teams, like, how much work is this in addition to Store files on disk, and… Keep some of them in memory, which might be needed for syncing.

Stefan Starflinger: Yeah, I also kind of agree with Barnabas, we shouldn't change too much now.

Toni Wahrstätter: No, no, I wouldn't change it now.

Stefan Starflinger: So…

Toni Wahrstätter: Yeah, I wouldn't change it now, it was more like we could… we could ship a defnet with the version as is. And so, with exactly what you mentioned earlier, where clients are already, like, interoping. And then, because in the end, it's a small change, to move the… the ball out from the block body. It's just, not something we have dealt with in the past, so it's more like, I guess, complexity on the testing side and so on. And definitely should not go in a dev lab that we would want to launch before the holidays.

Stefan Starflinger: Yeah, that sounds good.

Toni Wahrstätter: So, in this case, it feels like we… will launch at DevNet before the holidays, or what… what's the… what's the outcome now?

Stefan Starflinger: Yeah, I think we can do it. Another 30 cup. we should go ahead. Otherwise, we can just then, with the changes, launch another definite as well.

Toni Wahrstätter: Exactly, yeah. We can then… we can then launch another DevNet, later. It won't… also, those changes won't affect, like, the smaller… Or… Difficult details of the EAP, like, what content about should. should have. Great. Yeah, this is… this was basically the last agenda point we had for this week. It seems like, to summarize what we covered today, we have broad agreement on both topics of the retention period being the week subjectivity period. Of around 3,500 epochs, a little more. and that we separate the block of access list from the Yelp block. It travels through the execution payload via the engine API.

And… Yep. This is basically, what we covered today. Anything else someone wants to bring up or discuss before we wrap up? Jorgen, yeah.

Jochem Brouwer: Yeah, I just wanted to, say, to repeat, Karim from chat. He said… Because I'm also not sure if the chat is in the stream or not, so it's nice for the… And… the thing afterwards, but, he says, could, we are also in the testnet having the two nodes configuration with and without, the block driver access list? I think this is a very good point. Because then we can see, like, the impact with the block-level access list, which I have on the performance. Because I think at some point, like, we have the implementation, so the spec of the block-level access list, but this also implicitly means that clients have to implement this parallelization. And, I think, like, as a good measurement, it's also nice to have, like, these two kinds of nodes on the testnet.

And also as a verification that these block-level access lists are implemented correctly in terms of the optimizations and the parallelization of it. So just wanted to mention that. Nice point from Corrine, yeah.

Toni Wahrstätter: Yeah, it's a good point. I think we discussed it at some point on… The 7th or… the sixth or 7th breakout call, where clients basically agreed that every client will have, like, a flag implemented. for… are the… do we actually use the read values for patch I.O, or don't we make use of them? And this will then later help us to Determine the impact of the reads or of the state locations, onto the performance. Perfect, yeah, thanks for… For mentioning that. Awesome. Any more… Topics before we wrap up. Seems like it's not the case.

Then, I would say we wrap up earlier today. And see each other on the next breakout call in two weeks. Thanks, everyone.

Jochem Brouwer: See you! Bye-bye.

Marc Harvey-Hill: Bye, thanks.

Felipe Selmo: Bye.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.