EIP-7594: PeerDAS - Peer Data Availability Sampling

Transcript

Ethereum being like its own kind of standalone blockchain but eventually having this goal of enabling scalability through layer 2s the basic idea of kind of what pier does is going to end up doing column sampling but this is again very abstract the gossip protocol is not built for gossiping a lot of data and yeah I think it's fine when you're not like pushing the limit of basically bandwidth that nodes can use uh but once you start pushing against that you really kind of hit a barrier there and I think we're kind of moving away from that like I think pure does really is like a pretty big step forward. If there's anything that looks like a security problem, then well, that's like priority number one for Ethereum. Like there's never.

Welcome back to Panipe, the one and only EIP series where we explore Ethereum proposals for upgrades and those making an impact across the Ethereum ecosystem. This is episode 156 brought to you by EC Institute. I'm your host Puja Ranjan joined today by my colleague Akash from Multimedia team. Today we are diving into something that could support how layer 2 operates. EIP7594 peer DAS also known as peer data availability sampling is a very wonderful proposal to talk about this documented in January 2024. This is a

standard track core EIP in last call status and scheduled for intuition stage for busaka upgrade. If you ever have used an layer through project and wondered why fees aren't even lower yet or why Ethereum still struggles through scale data, perhaps peer das might be an answer or at least one of the answers that we are looking for. So to unpack all this, we are joined by someone who is deep into the design and research and also one of the co-authors of EIP7594, Fresnasco Damato. Welcome, Francisco.

>> Thank you very much and thanks for having me. >> Francisco, your ex profile suggests that you are a researcher dealing with wacky protocols and weird acronyms. We'd love to know you more. Maybe a brief introduction please. >> Yeah, actually the the Twitter bio is pretty accurate. I mean, pure dust is is one of these weird protocols, but yeah, I joined the EF four years ago and I've been in the research team since and I've kind of hopped between I want to say most of the consensus layer research problems at this point and some also like beyond the consensus layer. I think more or less I've been involved in like protocol design like broadly. But yeah, over time I've sort of drifted a bit more into being a bit closer to the protocol like not so much in the pure

research stage but also a bit in the maybe specification stage and uh yeah and also just generally like paying more attention to the actually you know getting an EIP to mainet part of the R&D pipeline. So that's kind of how I got to peer dust like I at first I was just you know interested in what the other people in the research team that were a bit more closer to the actual protocol what what they were doing with this and uh yeah kind of got interested with some colleagues started thinking about it and eventually it yeah ended up kind of working on it in in a much more like hands-on sort of capacity and yeah that's where I'm at now like a lot of I guess my time is spent thinking about like where do we go from here we have this like upcoming upgrade but yeah

what's next essentially for the DA layer but then also just more generally kind of yeah still like protocol design and and thinking just in general like how do we want to evolve Ethereum in the future >> that's pretty interesting background and the work that you are doing around I understand this is not your first proposal for listeners who may not have followed it before I'm happy to share that Francisco had been on the show before and had discussed about a proposal which unfortunately could not make it to the parade but good that we are having him again to talk about a proposal which is going to be in Fusaka upgrade. Now the proposal in last call so we are hoping it's going to make it.

Thank you so much for sharing about yourself Francisco. Without further ado, let's uh peep into the proposal. >> Sure. Uh yeah. So as I said pieras it's EIP7594 but actually before we get into that it's worth taking a step back and maybe just having a refresher of EIP 4844 which is kind of how we we got here in in the first place. Most of you probably are familiar with this, but yeah, more or less we have this overall story of Ethereum being like its own kind of standalone blockchain, but eventually having this goal of enabling scalability through layer 2s and this sort of needing a lot of data to enable basically these layer 2s to post transactions on Ethereum. So in this case, you know, here these like red and

blue transactions are meant to be could be arbitum, could be optimism, could be starknet, some layer 2 that's essentially just putting it's uh transactions on Ethereum and and not having ethereum actually execute them to achieve like basically greater scalability than if the chain had to execute everything. So yeah, this data needs to sort of be on Ethereum to get the maximum security that you can get from this kind of construction. And as of last year, now I guess a year and a half ago or something, uh, with the IP for 444, we've had this uh quite like big unlock in in scalability of this construction because we've added this sort of like what's looks to be here like external data that's sort of outside of blocks. This is what we call blobs and they've basically expanded the capacity of the Ethereum data layer by

by quite a lot. Initially about 3x and then now a further like 2x in the in the last upgrade in in pectra. But they've also kind of set the stage for what's to come. They've kind of really in a way created the Ethereum data layer as a separate product. Before roll-ups used call data, but it was more of like a hack in a way. So yeah, specifically what uh two main things that 444 has done it's well it has created blobs as a resource and in particular as a resource which has its own independent market. So it's somewhat separated from L1 execution in particular which call data lake wasn't really and uh so yeah this is like quite important. it it's somehow these are not competing with L1 transactions essentially they have their

own kind of capacity and price and then they're also like stored and served only for fixed amount of time like 18 days so in particular they don't kind of keep bloating nodes like nodes only have to keep a certain amount of of disk storage for for these blobs and it doesn't grow forever essentially and so this is on the good side like the the sort of good things that 444 has has given us plus it has set as I said set the stage in particular in terms of the cryptography for for what's to come. But there's also the bad part which is nodes are still downloading all of the data. So you know this like red blob, blue blob and so on.

Every node has to download all of it. Then sure eventually they can forget about them but in the meantime they have to download all of it. So it uses like as much bandwidth as as sort of the whole amount of data requires and so that's the part that kind of uh we want to still address and that's where data availability sampling comes in. So the availability sampling again right now we're not talking about pure dust purely but more just as a general technique but it's worth I think going through it in this way. It's just this like yeah general idea that you can take some amount of data in this case it's uh this blue part this could be any data you can think of it as whatever kind of arbitrary bites and it's split up in some way and then you can basically introduce redundancy in it. So this uh

orange part which basically makes it so any half of the data allows you to reconstruct the whole data. So it's just introducing this kind of extra information that lets the data be constructed from any sufficiently large part of it and this turns out to be like a quite powerful thing for for the problem that we have that which is we don't again we don't want nodes download the whole data but we still want to keep the same security guarantees that we have before and uh yeah what it gives us is that basically to check that whether a blob is available you only need to make sure that 50% of it is available not the whole blob and now this is where kind of sampling comes in once you have this 50% property It's something that you can essentially just sample like you can just keep trying and it gives you

like a better and better the more samples you get the kind of lower of a failure probability you have. So yeah basically what we ideally want is this that a node doesn't have to download everything. It doesn't have to download the whole blue part or the whole blue or the whole orange part or even any 50% of the data. It just has to download like a few chosen kind of pieces that it it randomly chooses. H and the number of these can be constant. It doesn't have to be scale with the with the size of the data or something like that. And so yeah, basically it really should be like a small portion of the data. And what does this look like if we have so I mean yeah this you can think of as as an actual blob like an Ethereum blob. You know maybe the blue part is a blob and

the orange part is this extra information and so the whole thing is this like extended blob. But then what do we do if we have a lot of blobs? There's multiple things you can do, but one thing you can do that to sort of trivially extend the base construction to having multiple blobs is that you just stack them up on top of each other and then you do the same sampling for each. So right now instead of sampling one piece for each blob, you basically sample like a whole column from this whole kind of matrix that you obtain by putting all the blobs on top of each other. And this is the same thing that if you were to sample each blob individually like here like here you just you know for each blob you're just choosing some pieces to get but you just always choose the same indices for so

for each row you're choosing the same index and so you end up with sampling columns and so yeah that's the basic idea of kind of what pier does is going to end up doing column sampling but this is again very abstract protocol is much more like concrete than that. question is like okay how do you take this abstract idea and actually put in the protocol in a way that guarantees all the security properties that we have and you know how do you think about how do you actually get the data networking and and all these things and there's more than this but like these are maybe the three most important highle parts that that we can think of that we get when when we actually concretely think of trying to put sampling into the the protocol and to make use of it. The first one is distribution. So we have

some proposer or builder like whoever is actually making a block and it's going to contain L1 transactions and L2 transactions and then all of this and so these blobs kind of on the side how does it actually get the data to the network in the first place like somehow this data has to go from this source to people that basically eventually want to sample it. Then once the data kind of exists in the network some nodes actually want to sample like specific parts of it. They don't again they don't want to download like the the whole data they just want to get you maybe these columns here and each node is going to have their own kind of choice and so how does this process actually happen and then reconstruction which is so we have this idea that okay in principle 50% of the data is enough because it can be

reconstructed but it's not enough that it can be reconstructed it needs to be reconstructed if the the case arises that it's actually not fully available so if you have you know 51% of the data you actually eventually want to go to having 100% of the data so you want to go you know, from this sick kind of network where these columns are missing. And sure, like there's not that many that are missing. So overall, everything is still reconstructible, but you still want to actually do the process of reconstructing to end up here in this like nice healthy network with everything being available. So these are at a high level like the main phases that we want to try to somehow construct in the actual protocol. And that's where EIP 7594 appear comes in. It actually says, you know, how exactly you should

do all these things. And the first simplification is that it's doing distribution sampling as one process. So it's not thinking about okay somehow disseminating the data to the network like in whatever way and then nodes are going to do sampling once the data already exists from the in the network. Distribution and sampling are essentially one and the same. And how does this work? Well, nodes want columns ultimately. This is the thing that they are trying to sample. And so this is exactly how they distributed on a lower level. There's basically these like column subnets. This like sub network that you only participate in if you want a specific column. So you know maybe this network you only participate in if you want the first column and this is

for the last column and so on. And so this is only going to contain some nodes, the ones that actually care about the column because that's the one that they're sampling. And so essentially yeah we kind of split up node in these sort of subn networks and then they columns get pushed into these networks. So the proposer will like will send this one to the first one, this one to the last one and so on. Uh and notes gossip and around as they usually do in the in these kind of gossip topics. Meanwhile, the block is being sent to everyone in the same way exactly the same kind of gossip process which is you send the block to like a few of your peers, then they send it to a few of their peers and so on. But this happens with all the nodes and this happens just with the

nodes that are interested in those specific things. Um, and so yeah, eventually testers, but also full nodes will end up basically getting this block and then the columns that they're actually sampling. So whichever things they're interested in, they only get those. So that's distribution and sampling basically as one process. And then how does reconstruction work at this stage? It works with this one of honesty assumption, which is essentially that there has to exist at least one node in the network that's actually participating in all of these instead of just some. So there has to be some node in network that's not actually doing just sampling but that's fully kind of interested in the in the whole data.

This could be for example an optimism node that is interested in being able to always recover the data so that it can participate in the fraud proof game or it could be I don't know someone that's running a note. It could be Danny Ryan running a node in his basement. It could be anyone really. uh it could be altruistic and actually there's mechanisms in the protocol to make this not just come in from altruism but as a baseline assumption it's still like a pretty weak assumption like that we don't really necessarily need to incentivize to be quite sure that there's someone out there that that's doing this because also it's not particularly expensive to to do but yeah so there's this assumption that that someone would do this and if they do then how do we go from the sick network to the healthy network quite simply

there is this node again it could be Danny's node in in his basement it gets all the columns. It does the reconstruction locally and then it just receds everything to the all the topics. So, you know, maybe this was a sick topic that was missing a piece of data and then this node would kind of recede the the data in the topic where it's missing. And concretely, how much data are nodes getting like what what the sampling like concretely look like kind of numerically? Well, nodes basically have to for security reasons that we don't really have time to like go into too deeply have to participate in at least eight of the 128 column subnets.

So there's like 128 of these vertical subdivisions like columns and uh you have to sample at least eight of them at this state. I mean in in this first version of of appears and so what does that mean is uh is that you're downloading 1/16th of all columns and one that's 1/8 of the original data because we have kind of the original data is this blue part but then it's doubled into this with this orange kind of redundancy part. And so if we were to just think about this we basically could support 8x more blobs essentially because the at least the weaker nodes they only need to do 1/8 of the work in terms of the bandwidth and storage. Now, of course, this is not the only things that that matters. There's more aspects than than just sampling. Like the whole

system is more complicated than just this one kind of parameter, but the first approximation that that still tells us like something useful. But yeah, speaking of the the whole system, it's actually worth like zooming out a bit and and not thinking just about so we've kind of been thinking about just this final part of having a proposer and sending out the data. But the whole system, as I said, it's it's more complex and it's worth like taking a look at it. So as usual we have Ethereum we have these like two big components of of the protocol the execution layer and the consensus layer and on the execution layer. So so far we haven't talked about it at all but the main contribution of the execution layer to the main relevance of the execution layer to the the ZIP and in general to the to the

data layer is the mempool. So we have this blob meool where basically someone that wants to make a transaction like for example this could be like a optimism get node that wants to send its transaction to to the network. It might choose to do it through the mempool and so yeah it would send it and it would come with a with a blob with it and then eventually all of these blobs get taken by some proposer some Y node through the engine API which is the this basically API that links the Y to the CL. So it sort of allows us to cross this barrier. they would end up at a CL node of the same kind of entity and then we go back to the process that we had before like at this point okay these blobs have been turned into this whole matrix with this

extension and like this you know breaking them into columns and then we basically publish them in the in the same way that we discussed before and eventually they end up to a tester they vote we finalize and all the kind of things that we we know on the on the CL and uh yeah I mean basically what is happening here but I mean some somehow block building is uh touches everything so how you get transactions from the memp pool. What exactly are you doing here? How are you doing this publishing? Like block building is somehow like really the key pipeline in all of this. Um and so it's worth like basically zooming into this and what are the the challenges there. So yeah to build a block you need to be able to do a few things in a in a timely way. You need to

be able to run a mele first of all somehow you have to get these transactions from somewhere and uh you have to be able to compute KZG proofs. We haven't touched on this and like what even this means. It's part of this kind of cryptographic magic that that happens in the protocol but I'll mention basically what's happening with this and then you have to be able to do the this initial distribution of columns. So when you are kind of pushing the columns to the network you have to be able to do that fast enough. So first for the meool part something that you might notice is at this point we've basically discussed let's maybe go back to the picture with you know this is the memp pool this is this kind of cl distribution part this is where we've mentioned columns where we mentioned sampling so we have this

like red kind of samples here there's nothing like this here we just see blobs there's no columns there's no sampling and so the immediate question is doesn't this kind of defeat the whole point of the construction like we introduce sampling on the right but all the blobs still exist on the left and there's no sampling. What is the point of that? And so actually it seems counterintuitive but there is still a point and basically the the reason is that the CL in particular this kind of critical path of actually pushing the blob data to the network is much more of a bottleneck basically than this kind of mempool part that's happening on the on the for essentially two reason. One reason is that there is no strict deadline for propagating things on the on in the in

the mele. So propagation you send a transaction to the memp pool. Of course it's great if it propagates fast it's makes your transaction end up on chain faster but nothing breaks if the transaction takes 10 seconds to propagate even 20 seconds to prop or really however long in principle sure it's it's maybe bad user experience if it takes too long but that's about it. The protocol is not going to break. We're not going to be missing slots. There's not really going to be like any evidence that something has gone wrong other than maybe like some rule about there being a bit a bit upset that the transaction took a bit longer than it should which is important but much less important than mis votes and all these other consequences. So here we can use basically 12 seconds of every slot like the whole time in principle is open for

propagation and here we cannot there's only a very kind of brief part of the slot which is at most 4 seconds which can be used for this propagation thing. So everything is basically much tighter. The other thing is that on the eel, blobs are propagated by what might call announce and pull. So you basically tell your neighbor, here's a blob. Do you want it? And then they maybe say yes or no. And what this implies is that notes never download things twice because once you have it, you always say no. And so you're never going to waste bandwidth downloading something that you already have. And I mean this implies that there's this this kind of announcing part which adds a bit of latency because you need to like announce and then get a response before you send anything. But it also means that there's no

duplication like you never waste bandwidth basically on the CL and partially because we have less time. So everything is a bit more kind of in this tighter window. We don't do this on the CL you propagate by push. So you don't say I have a blob or I have a column. You just send the column directly. And this actually ends up in a lot of duplicates and so a lot of wasted bandwidth. And I mean this could be improved in the future and I think should but that's not how it is today and that's how it works for everything in in the in the CL in general. It's not like a feature specifically of columns. So this means that with a combination of these two things, the L is essentially much more efficient in propagating blobs than the CL. So the CL is kind of it's

much more important to first start having sampling on the CL than it is on the LL. Basically the L is has a lot of slack for now. Then there's the second component that I mentioned which is cellg proof computation. And yeah, maybe we have to take a step back and first say what even is a KZG proof? What is a cell KZG proof? So we have a blob this in this case sorry a column column is split up in in cells each one is coming from a specific blob but then it has associated to it a kijg commitment which is this like little thing next to it it's a cryptographic commitment to the whole blob and a proof and so yeah this basically we have kg commitments we have kg proofs for each cell and what this proof does is it allows you to prove

that this cell is actually the right cell for this commitment at this index so given we without having the whole blob. So given this commitment, the proof and the cell, you can be sure that the cell is actually the right thing, the right piece of data that's supposed to be there, which is very important when you're sampling because let's say you sample this cell and this cell. Since you don't have the rest, it's very important that you can tell that okay, still this is the thing that I was supposed to get. This does correspond to the commitment to this piece of data. So this in a way like it's what you might do with a Merkel proof in another context that you still you don't have the whole data but you still can figure out that this piece of data is part of this hole that I have a commitment for

like a Merkel route in that case but this is kind of using different kinds of cryptography that's that's more efficient for this case but still basically same idea and uh yeah basically a column comes with all of these proofs and all of these commitments so that you can verify that the column that you're supposed to get essentially that you're not being tricked into taking a thing that looks like a sample but isn't actually the right sample and yeah you to when you build a block you need to compute all of the KCG proofs for every cell which is a lot of cells that's like 128 cells per blob so it adds up and proof computation is is quite slow so for a single blob it takes this much like about 200 millconds let's say whereas proof verification I mean as usual there's this kind of

asymmetry between computation and verification verification is always faster and in this case it's maybe I think like 10x faster or something like that. It's much faster. It can be batched across the column. And so it's it's somehow much better if this computation is not being done. Well, for one thing is it's not being done over and over again, but also it's not done in the the wrong parts of the slot. It's not done in the critical path in particular. So this is what a design might have looked like where proof computation happens right here. So these blobs come through the meool. Then you decide that you actually want to kind of include them in your in your block. you create this matrix and this matrix has to come with all the proofs. So it's actually here there's one proof but it really means one proof here one proof

here one proof here like every cell is going to have its own proof and so yeah you have to basically do this computation for every blob of computing all the cells worked like this and the problem would have been that now this is in the critical path so as you're trying to produce a block you need to do this kind of heavy computation and then yeah eventually I mean we didn't have this before but actually what it looks like is that each column is going to come with proofs here and this is how you actually check that you're getting the right sample and instead what what it actually looks like in 7594 pieras is that the basically the mempool structure is changed a bit from 4844. So what happens is instead of gossiping just blobs you're now gossiping blobs with all of the proofs already and you're

like verifying these proofs like already in the mele. So somehow we're like moved verification from just here to like the whole system essentially and this means that proof computation is now pushed to the transaction senders. So I'm again maybe this is a ZK sync node that's that's trying to send a transaction block transaction to the network. It needs to compute this cell when it's doing it and so it is the only node that's actually doing this computation. So if you're want to send a transaction that kind of requires this somewhat heavy cryptographic operation you're the one that does it. Everyone else from the me pool to everywhere else in the system only has to verify it. And so in particular it doesn't have to happen here. Also the proposer only needs to verify it. it doesn't need to do it anything really heavy in the in the

critical path. And then the the last bit uh that we mentioned is uh you want to be able to do the initial distribution of columns fast enough. So this is um this part like in the initial distribution by like in the in the CL once you're actually proposing a block. So again this is the the critical path of block building. So you want you want to be able to do this fast enough because again we have this tight timeline of a few seconds for for all of this to happen. And more or less what it looks like is that each column should be sent to about eight peers. It's like how the parameter of of gossip in Ethereum. And so what you end up doing is that you end up basically sending about 16 times the data because there is this 2x eraser coding thing. And then besides that,

you're also sending to kind of more than the one period of time. And so sampling doesn't help here because other nodes are going to be sampling. But you as the proposer, you're originally the only one that has all of this data. And so you have to send all of it. you can just send like a sample of it. And yeah, this what this mean in practice is that if you wanted to have 50 blobs, sending it all out in 1 second would require this much like 800 megabits per second upload, which is not in feasible, but it's also not something that we normally think like a a solo staker should have for example. And so there's a worry there of like okay, what does this exclude certain actors from from participating in block building? I think there's like a broader conversation about block building but specifically

right now in in pure dust the main solution for this is that so first of all yeah we're mainly worried about local block building not me boost kind of relays professional builders because as we said the numbers before is something that if you really do this professionally you can get a connection that that's okay with uh with being able to do this but yeah for a local block builder which might have 50 megabits or even less we would then assume that all the blobs that you're using to build are from the memp pool like if you're building a block with your own node you're not going to get private order flow or somehow the transaction that you're going to use to build your block are just going to be from your from your local meool. So this assumption simplifies things a lot and I mean it's

quite realistic for what we are actually worried about and yeah the way we use this is that basically again we can go back to our usual picture the part that we're worried about is sending the data here and we're saying okay all the data needs to be sent by this node because it's the node that has all the data but actually before here we have this whole part on the left where the data is already gossipped in the yell meool. So actually this node is not even though it's proposing this block with this kind of matrix associated to it. It's not the only node that has all the data like when it's getting the data here from it somehow this is this is where the the entry point of this data get into to this kind of block building unit. The data already exists it's something that

has been obtained from the local mele. So it already somehow exists in the local mele and then presumably in other people's memples as well. And so that's the thing that we essentially exploit. you uh basically send out the block but you outsource the task of propagating the columns to other people that have these blobs in their meoof. So again same kind of same picture as before um except the cells have been taken out but they still exist in principle and yeah basically what happens is now okay you send out the block people would receive the block like for example this node will receive the block and then it kind of asks its yell node so it again crosses the this barrier of CL to yell it basically does this call that's called get blobs which is literally just asking this other node do you have the

blobs already like do you happen to have like received them from the mele and well in this case this node does have all the blobs. So then this no they kind of cross again the barrier of you know EL to CL they get turned into this matrix and they get to the the CL node which then can do exactly the same thing that this this proposer would have wanted to do but maybe doesn't have enough bandwidth to do like successfully and so it sends out basically these columns and this is showing one node but then it really could be like a bunch of nodes. So it could be that you know some node is propagating this column. Some some node is propagating this column.

This is why it's called distributed because in principle the data already kind of exist in the network. There's a bunch of nodes all of the kind of CL counterparts of these nodes already maybe have the data and they can contribute to this process. So it becomes a bit more of like a collaborative process in publishing the data. And even just this node doesn't need anyone's help. It already has all the data. So it can just immediately say okay I have it before receiving it from from anyone else. And yeah, that's so I mean this is kind of the last like challenge on the kind of current protocol and the the block building side which I think is is the one that's really where sampling doesn't necessarily have all these benefits and so you kind of really need to think a bit more carefully about all these like

various problems. But I think for the kind of throughput that we're going to have with pure then all of these are somewhat like okay solutions to to all of these problems. And I mean even even for like future levels of throughput for for the most part but there are some limitations and I mean in one way you can think about them as limitations in one way you can think about them as the next steps like what are basically how do we evolve the protocol next to actually keep increasing the the throughput and uh yeah one step I actually kind of mentioned already we have this inefficient propagation on the CL so we have this like redundancy of sending basically too much data around so that you end up having a lot of duplicates and this is okay if it's things like attestations that are very

small, but it's not okay when you're sending around blob data because it's a lot of data. And so you really want to be using bandwidth close to as efficiently as possible. And so there's I mean there's a lot of possible things you can do. Could be more complex like eraser coding, but it could also be as simple as just doing a bit more pull like a bit more of this like announce and pull thing that we do in the the me pool and less of like pure just pushing the the data. So basically biasing towards like bandwidth efficiency a bit since like bandwidth is a limitation here and or at least it would be a limitation at higher levels of throughput and uh yeah get blobs this kind of process that we said kind of allows uh helps with local block building and I mean with the system in

general as a bit of a problem that it needs all the blobs to be in the meool. So for this to be useful when you ask do you have the blobs you're supposed to get all of the blobs not just some subset of them and this is like for technical reason we don't have like too much time to explain this has a few like negative consequences but basically the solution is that it would be nice to have cell level messaging. So being able to really send messages at the granularity of a single cell instead of a full column so that essentially even like a subset of things could still be useful and like useful piece of information that can be exchanged. I mean this has a lot of benefits in general this like cell level messaging thing and it is really something that it's maybe like I want to say the main

thing that hopefully we can do next after for you know the next iteration of uh this protocol. Then the meleool I already said as well is not sharded. So we we yes we do have this as a symmetry where it's less important to shard the mele than it is important to to shard the CL. So do sampling in the in the CL but still eventually it is important as well to do it on the EL. And again this together with level messaging is probably the second most important thing that's kind of on the plate right now for what to do next essentially. And then there's something that's not really super related to pure dust specifically, but more broadly about scaling um and and just the way we use resources in the protocol today, which is that just the slot does not use resource resources

very effectively. It's divided in this like three equal parts with like block propagation at thenistation aggregation or wellistation aggregation and then propagation of the aggregates. And uh yeah, these parts don't really make too much sense anymore. And if you really want to try to use uh the time in the slot efficiently, this is not how you should do things and somewhat I mean EPBS hopefully in in the next fork is like a big step in in trying to to correct this. And so yeah, again this is not unique to Peras, but I think it's a general boost that's going to come in the future in to Ethereum scaling in all kind of parts. And uh yeah, I mean this as I said kind of two and three are really things to look out for in the next year. I think there would be like a

lot to see like concretely about these two things and yeah hopefully all these limitations really point us towards what's the next step to to increase the throughput. So I think there's really even immediately after the fusaka there is supposed to be scheduled some BPOS some blob parameter only forks to increase the throughput. I think if I remember correctly the last one is supposed to be for a target of 14 and a max of 21 which is a bit more than 2x from from here but after that I think there's a lot of room in principle to observe what happens in the network see that hopefully everything is fine actually I think what we're going to see is that at least for full nodes this is even going to make things much better than now because we're not increasing the throughput nearly close to this like

8x kind of theoretical number that I said before so actually it should reduce the bandwidth consumption a lot for full nodes specifically. So yeah, hopefully we can observe that everything is fine. Maybe increase the throughput at some point a bit more with scheduling more of these like BPO forks and then eventually we we hopefully can have some of these like concrete improvements to push things like basically forward from from there. And yeah, I think that that's all I have. >> Thanks for the presentation and for bringing clarity to what and why behind this proposal 7594. I love the graphical walkthrough of the process. that was really effective and I also appreciate you having the recap of 4844 and KZ app proof. I'm sure these proposals are

important to understand before we go ahead with proposal 7594. Thank you for sharing that. >> My pleasure. >> For anyone who are looking to maybe dive deeper, I would recommend checking out episode 104 with Terrence and Po on EIP484 as well as episode 96 with Dan Grad on KZ Proof. We will try to add the links to description so people can follow that. Uh Francisco, it was simply ingenious to apply eraser coding to cut data download requirement by 50%. I believe the change of mele structure that you mentioned in the presentation was nice to have and you also shared some challenges of the current process and possible improvements. I do hope to have some more questions around that to

just to get more clarity about where we are and what we are looking at in the future. All right, I would like now go into the next round of question and answer, but I see some audience here. We do have a question from a guest. Kev is online and Kev, if you would like to relay your question directly to Francisco. >> Yep. Hello, Franchesco. Nice to meet you. >> Hello, stranger. >> Out of those limitations that you mentioned, uh, which one of those do you think would help Perus the most? >> Good question. I think probably either one or two. So I think one would make a huge difference just kind of taking out this I mean basically the gossipy protocol is not built for gossiping a

lot of data and yeah I think it's fine when you're not like pushing the limit of basically bandwidth that dead nodes can use but once you start pushing against that you really kind of hit a barrier there so I think that would really make a big difference and I think cell level messaging would also make a really big difference because I think it really would make the integration between the mempool and the CL part of things just work much better. It would like I mean yeah it would really mean that we can rely on it much more. You can kind of see it from the comparison between the EL and CL side of things in in that slide. It's somehow much better if you can just pull things from the meool. Uh there's there's a lot more throughput that you can push through the mempool even without sharding even

without really tweaking the meool too much then you can push through the CL. Of course that differential will go down if you like improve how the CL works. But yeah, so and I mean in general I think besides even this current higher level of inefficiency on the CL I think like more generally you will always kind of want to do as much as possible in advance like out of the critical path with pre-propagation rather than kind of having propagation be something that happens after block proposal. So yeah, I think cell level messaging is still like a really kind of key thing that unlocks a lot. I would say those two. I'm less I think eventually we do want mele sharding. I at least personally don't think it's as crucial as these other two

things, but like I I mean I think all three it's probably something that it would be great to make uh progress on in in in the next year. >> Right. That makes sense. I guess on the mele part is there a limit to the number of blobs that can be put in the mele because the get blobs functions was assuming that all the eels had all of the blobs right the relevant blobs. >> Yeah. I mean do you mean is there like a def facto limit or is there like an actual limit? >> Yeah. I guess is there a practical limit like can I just spam the eel with like >> Yeah. I mean it's well like there isn't I guess in like I mean there isn't like an in protocol limit of of any kind and there isn't um an allocation mechanism.

I think personally I think eventually would be good to have such a thing like somehow allocate like for regular transactions is not as important but for blob transactions that consume like so much so many resources and like where we really have like quite finite resources to to allocate I think it would be good eventually to have like an explicit allocation mechanism even for mele throughput but yeah for the time being there isn't and really just relies more on like local rules of clients and I mean just like rate limiting and of course like there is like you know you check for payments and like these kinds of things. But yeah, in principle, I think like sending a bunch of blobs through the mele wouldn't be like the you know crazy expensive thing or something like that. So yeah, I I do think there is a bit that's one part of

why it's a bit hard to like fully just go all in on this like pre-propagation and kind of relying on the meool. I think we would all feel more comfortable doing so if it was more clear exactly like what is the behavior of the memp pool and if there were like clear kind of like guarantees like properties around okay this is as much throughput as can go through the meool for example because we explicitly allocate such throughput and we don't just leave it to sort of yeah what whatever gets sent and then kind of clients just locally deciding what to do with it but I think that's farther away probably >> right that makes sense I guess my last question is like a couple years ago we did the KZG trusted setup before and technically speaking there was like four

of them for different blob sizes. Do you think that will ever change like the blob size? I mean, I think yeah, eventually like if you project far enough in the future, I think it it most likely makes sense, but yeah, for like as long as a blob still remains a sort of reasonable unit to be used by roll-ups, I don't know that there's like super clear benefits to switch into a higher size. I I'm I'm not entirely sure. I have to think about it a bit more.

>> That makes sense. Cool. Thank you very much. >> Thank you, Kev. Francisco, my first question for you is about the real problem. Why is data availability the real bottleneck for rollups and why can't they focus on computation? Yeah, I think we're actually in a way maybe at a switching point at least for the kinds of rollups that are on Ethereum in the sense that I think for some of them it might be that computation is going forward going to be the the main bottleneck and I think we're already like a bit seen that like I think I mean for one thing rollups that are on Ethereum for since 4844 have essentially not had any time of congestion like there's been like brief times of the DA layer being congested but for the most

part whatever barriers they've had on their scaling have been more like their own kind of bear is like related to their chain and on on the DA layer. But yeah, until I mean until today still like if you look on L2 beats now the the total throughput of the chains that are using the Ethereum DA layer is something like 250 transactions per second. ensure transaction per second is not the greatest metric, but it still did some it tells you something. And yeah, so I think until this point I mean it's still something like 15 times more than the throughput of just Ethereum mainet. So already like a lot has been achieved in terms of scaling in that sense. But until this point still kind of working with like it cannot really it's not a

scale that can support even a single chain that's hugely you that's uh uh with like thousands of transaction per second or or something like that. So I think that's why it has been like at least perceived even though the delay has not been congested but still like I think the perception is until this point that you cannot really scale a chain too much by just relying on Ethereum DA. So in that sense I think it has been the the bottleneck at least in like planning for the future like can I actually take my chain to be like this scale and so on can I go and try to acquire like all these new users or will I then hit eventually saturate the whole DA layer by myself and so on and I think we're kind of moving away from that like I think pure dust really is like a pretty

big step forward I mean it's not going to happen in one day as I said the initial scheduling is that the throughput will like a bit more than double for the maybe I in January is is the last PO. I mean, as the the kind of tentative schedule that has been agreed on, I think very very recently last week. And uh yeah, so at first it's not going to be a huge step all in one go, but it really sets the stage for then doing this consistently. So it's not meant to be like 2x and then let's talk about in two years. It's meant to be like this 2x is like the first step and then there's kind of like meant to be some of these steps pretty regularly for at least some time. Of course, eventually there there's no, you know, exponential growth forever, but yeah. So

once you think about this in the context of already for it for four was a 3x then pectra has done another 2x now we have another 2x and we have an a plan for how to keep doing this for some time I think you really start to see that okay actually da at least for the chains that have been on Ethereum that are using Ethereum DA will not really be the main bottlenecks and yeah they'll have to turn to kind of their own like state management computation and so on to actually be able to make use of the throughput that the DA layer gives them. Yeah, you indeed mentioned in your presentation that this could be the first version of Pas and there are CL critical path that can be improved in future. So we are just looking forward to what are other upcoming changes with

respect to this particular proposal. Talking about uh engineering pain points from a networking perspective what was the hardest part of designing pas was it peer discovery gossip routing or proof verification? >> Um I think depends who you ask. I mean maybe if you ask Kev or or someone that has actually worked on the cryptography they might have a different answer. I actually sort of started working on this when most of the cryptography in a way had been done because like the forer4 already existed so a lot of the infrastructure was there. Of course there was then like more to do for pier specifically but like a lot of the foundations basically were actually made for for 4. So yeah, maybe not that part.

But yeah, from purely networking, I think like generally things don't actually look in a way that different from 44 for4 if you squint enough. There's still these subnets. You still kind of send these like relatively big pieces of data in subnets, but introducing sampling. And so the fact that now all of a sudden not every node in the network is someone that you can actually get the data you care that you can sort of that's an interesting peer for you in terms of the the sampling needs that you have. I think this is something that really was quite challenging. So yeah basically peer management I think more broadly like figuring out okay I have this set of nodes and I really I needed to be able to cover my syncing needs and my assembly needs and so on and I need to

manage what they tell me. like basically each node is telling you okay I care about this column this column that column and you somehow have to manage a pier set that allows you to also like satisfy your needs essentially so that I think that part is something that like was a bit different from what came before and and and challenging >> that makes sense well I'm trying to understand this if pas is the answer for data availability scaling that we are doing with Ethereum and you also mentioned that there could be some followup upgrades or follow-up EIP so what are the follow-up designs look like? Could that be multi-dimensional DAS or any other research that you would like people to start looking into?

>> Yeah. So, I mean, historically the somehow end goal of DAS was was meant to be this like 2D construction which does have a lot of nice benefits, but I think at least the way I see it, I think Peras kind of gives us a good foundation to keep improving on for quite some time. Like I think there's a lot of relatively low angs kind of improvements that that we can make on the foundation of pure dust and they're not really like deep redesigns or like huge protocol changes or something. It's more like tweaking certain parts of the protocol, building on certain things and improving them. So like like some of the things that I mentioned, they're not necessarily I mean they're not in principle even necessarily things that you have to do a hard fork for. So I think for at least some time it probably makes sense to try

to just like push this to basically the maximum throughput that it can support like really try to like squeeze as much as possible from this foundation that we have instead of trying to like fully rethink like the whole design. And then yeah I mean I think there's definitely space for constructions like 2D or or other things farther in the future eventually. There's also the postquantum question. So like at some point there's definitely different kinds of considerations but I think for quite some times like I I wouldn't be surprised if this is what it looks like in 3 years still more or less with with some like bells and whistles. >> That is good to know. I have a question which is more based on the documentation part of this proposal. I understand when the proposal was initially documented it was marked as a networking proposal but

now the category has shifted. I think it shifted recently on September 5th. So it's less than a month. So why was Pas moved from networking to core? What was the trigger and why do you think that it is important to keep it in core category? I have to say I'm not really an EIP historian so I don't know even like yeah I I don't know exactly like what is the the reason for I think I remember Alex made this PR so you probably better person to ask but I I assuming it's because yes in principle you can think about it as is just some swapping a bit of the networking back end but then it actually from like a user's perspective it looks the same as 44 for4 there's just blobs and there's

KG commitments so like you can't Think about it as just there's some backend thing that's mostly networking that you're swapping, but really it's more complicated than that. Like it's touching the meool, it's changing the cryptography, it's definitely changing the networking, but it's changing lots of kind of objects everywhere. So I think core probably because it more accurately reflects the fact that it is a huge change is like yeah I think it's it's as big of a a change as for for itself somehow like it is an evolution of it but like it's really kind of an IP that touches a lot of things and that really sets the stage for in a way like the next phase of uh this whole part of Ethereum the the data layer. I I don't know exactly what are the I mean uh why something should like what are the rules

for classifying something core or networking or so on but core does feel more appropriate given the scale of what the IP is >> well whatever the reason may be I'm very happy to see this proposal finally moving into core proposals because I feel like this is the essence of the Fusaka upgrade and having a proposal that has the most impact in others category didn't feel right to me so I'm sure that there must be very technical reason to switch the categories but I'm happy about it and uh happy to see it in course section now well I have a few more questions for you those are kind of juicy questions you can perhaps choose to answer in one word or maybe a few sentences whatever feels okay to respond to those question so if Pas makes

Ethereum data availability layer dramatically cheaper does that eliminate the economic incentive for alternative DA providers like Catacia or Ion DA >> I I think there might always be some room for alternate DA providers like even in the farther future. I mean I think there definitely is some room now because even with pure dust and even projecting you know maybe one or two years in the future that's it's still not going to um be able to satisfy like the demand for DA across like every possible chain. And I mean yeah this might just be a goal that isn't actually possible to satisfy whatever global demand for DA exists. Then whether or not Ethereum should care about this is a

different question. It's unclear. I think it's very possible that you can have a DA layer that's large enough to satisfy most of the things that you might care about satisfying. And I'm sure there is some tail out there, but that you don't care about it. Or it might be that that's not even necessary. Yeah, it's it's not kind of important to somehow fulfill like all the even important DA or like relevant DA needs, let's say. So I think that's just yeah, not not a super clear thing yet. But I think whether Ethereum cares or not is my next question. Just imagine that in future Ethereum DA becomes extremely efficient. Do you see these projects competing less and coordinating more or will the DA pricing war get even more intense?

>> So I hope regardless of for a future where whichever or at least like some amount of chains that are using Ethereum DA settling on Ethereum just generally being like in the kind of greater Ethereum as L2s or app chains or things like that. I hope that there is like at least some core of them that are more cooperating cooperating that have at least like lots of amount of of shared activity and and so on like through mainet it would be even better. I don't think this is gonna come automatically just by like even if you somehow made the free I mean which it almost is today but yeah even if you made it free for you know somehow guaranteed that it was going to be free for for a few years I think there still is incentives I mean the cooperator or or compete incentives

I feel like come from uh something else like there may be competing for the same users and yeah depending I mean they might sometimes have incentives to yeah cooperate or or for example try to have to build like interrupt infrastructure together or things like that for other reasons but I do think some element of competition is sort of it's impossible to to erase >> and maybe it is healthy as well some level of competition is generally healthy all right uh my next question is your honest answer or honest thought on is Ethereum becoming too dependent on rollup what if uh layer 2 stops playing nice and optimized only for their own benefits >> yeah I mean I think that's definitely a fair concern to have And yeah, I

personally think part of the answer to this is just making mainet activity itself be kind of or making mainet itself attractive to to certain kinds of activity. Making mainet itself be like um more attractive like hub for liquidity. So that somehow I mean in this it t kind of ties back to this like cooperate versus compete that we were mentioning before. I think it is possible for mainet to be a more interesting cooperation partner and this kind of depends also on like the activity that that happens on mainet whether or not there is liquidity there is activity there are users that are that are kind of based on mainet there are assets that are based on mainet and things like that. So yeah, I think this really depends on like how healthy does this look like. If there are reasons for

for rollup to kind of still be rollups to be close to Ethereum to talk to Ethereum to interoper I mean in this I mean in a not to people building on Ethereum but just to the actual like DAPs and so on to use bridges and basically to interact with what's happening on Ethereum then I think they will like I think the incentives will naturally be that if there's a lot going on on Ethereum then you're not going to ignore that and they're not just going to use DA and be their own kind of island out there. So yeah, I think this really depends on basically what does mainet activity look like and can we kind of build this like network effect around it.

>> Yeah, and I definitely see a mutual growth of all the projects around like cooperating with each other and we can take advantage of the tech explanation or text exploration that is going on in different projects to make the ecosystem better unlikely to happen. But what if pier does unperform? What's the plan B for scaling block throughput if there is any? >> I mean I I think the the the general answer is that if there's anything that looks like a security problem then well that's like priority number one for Ethereum. There's never any kind of scaling or any feature that comes before security before aliveness before like if for example you mentioned reconstruction if it for whatever reason it turned out

that there's too much data in order for reconstruction to happen timely or and then this caused the chain to to have problems like skipping slots or somehow like yeah losing livveness for some time or something then yeah I think the clear answer would be like taking a step back and undoing that which might just look like reducing the blob throughput or something like that which actually bo kind of blob parameter only forks kind of help doing in a bit easier way than if you had to do like a like a whole fork from scratch but yeah I personally don't think that's going to happen because for two reasons like one is just there's a huge amount of testing that that goes on like all these kinds of scenarios reconstruction and I mean a lot of other possible edge cases or just general system load that might happen

have been tested are being tested and and so on and will be tested a lot more until Maynet and partially because also the way that throughut will be increased is somewhat conservative in in two ways. Both that the actual throughput numbers I think are fairly conservative that the ones that are that are chosen but also that it's going to be gradual that it's going to be the mainet launch which is not going to have any throughput increase and there's going to be a BO with a small increase and then a second BO within another small increase. So I think this like kind of graduality should enable us to kind of see if there's any degradation in in behavior and then react if needed. And then yeah the other I mean the final reason is also just I think of course there's theory and there's code but like on a

theoretical level I don't think we're putting some you know huge amount of new load on the protocol like I think the weaker participants are some of the ones that you would be mostly worried about nodes that are running in data center or something like that. I really wouldn't worry like this is not some crazy amount of of load neither in terms of computation or in terms of bandwidth and when it comes to the weaker nodes that's exactly who the protocol in a way like why this whole sampling thing exists it's actually making I mean and given the throughput numbers that we're going to have at first it's actually going to make things better for them like they're going to download less data than before so yeah this is more or less why like I mean of course it's a huge change with every change comes yeah there can always

be bugs like we're still talking about 5CL clients five clients, massive network. So like yeah, everything can happen but I think there's a lot of reason to be optimistic about things going well >> right and I think having BO folks giving this ability to adjust the parameters independent of like upgrade network upgrade that happens already gives us a lot of comfort. It was just an hypothetical questions. >> Yeah. Yeah. No, I mean it's a very fair question. >> Well, my last question for you is uh mostly inspired from the internet. People suggest that D may be the major step before Ethereum becomes the global data layer. What do you think? Is it real or is it a hype?

>> I don't really know what like what would global data layer mean to be honest. Like I mean I guess one interpretation would be just the data layer for all blockchain data. Yeah. I mean I personally think it's it's unlikely to be the case. Again I I think depends how you weight things. If you're just talking about pure data amount then I think that's just impossible. there's always going to be all kinds of activity that that just cannot exist on Ethereum. Like we do have constraints of well we want to have uh you know normal nodes be able to to participate in this like sure sampling helps but like it helps only up to a certain point. So I think there's always going to be activity that cannot exist on Ethereum activity in a different way with kind of some economic waiting and I think there there's maybe

a bit more hope like I think the optimistic case is that well any you know maybe like 90% if you weigh by actual value so it could be fees or something like that of activity might be on Ethereum that might also be optimistic I'm not sure but yeah I definitely don't think that purely looking at data then this is a realistic thing to say like all data is is going to be on Ethereum like someone playing you know online chess on the blockchain is probably not going to be a herum for the foreseeable future, I want to say. >> Yeah, that makes sense. But thank you so much for providing an honest answer to this question. Well, this was pretty interesting deep dive. Thank you so much, Francisco, for sharing about Pier Das. I believe this is one of the most discussed proposal for the Fusaka

upgrade. Thanks for your time. >> Thank you. Dear listeners, if you enjoyed this episode, please like, share, subscribe, and tag us on Twitter. If you want us to cover any other specific EIP, please send us your recommendation and we will try to reach out authors to make it happen. Until next time, make the upgrade safe, make the chain safe, make users transaction safe. Let's keep scaling Ethereum the right way. This is Puja Ranjan signing off. Cheers everyone.