Ethereum Peer Data Availability Sampling (Peer DAS)

Transcript

Hello everyone, my name is Peter Robinson and today I have got Franchesco from Ethereum Foundation who is going to tell us all about peer daz so data availability actually daz anyway look I'll throw it over to Franchesco who's going to not only introduce himself but remind me what DAS stands for um and um go from there. Hi everyone. Um, thanks for having me for school. Uh, I'm I'm Franchesco. I'm a EF researcher. I've been at DF for for a few years now and kind of jumped around um different topics and yeah, for the last um maybe a year and a half, two years I've been involved with this u

blob scaling effort and and DAS, which is data availability sampling. Um, and yeah, I've especially been quite involved in the in the peer effort for for the last year. Um, so yeah, basically my plan would be to uh kind of tell you where things are um in uh in DAS land in Ethereum um today and yeah, kind of where we're going uh with with the next fork um and really kind of try to dig in into some of the details of of what's going on. Brilliant. That sounds awesome. And um yeah, please present your slides.

All right. Um can you see well? Yes, they are very clear. All good. Nice. Uh, okay. So, yeah, as I said, Ethereum DS in 2025. Um, let's get into it. So, okay. Yeah, everything works. Um, yeah. So, first before uh getting actually to the let's a bit recall like where we're coming from like what's what's what's happening now uh with blobs. Um, and yeah, that is EIP 4844 which came in the in the last fork. Um essentially as of lust fork we have these uh blob sidecars that that go outside of blocks uh they're essentially sent separately um and uh that just contain data that rollups post. So this

like red blob might be like an optimism uh blob this might be like an arbitrum blob and so on. Um and uh yeah I mean I'm not going to go into the details you know of the rollup ccentric road map. I'm kind of assuming people know what the blobs are for. Uh but yeah, basically given the fact that rollups do want to post data to Ethereum, right now the the way to do it is through through these blobs. Um and uh the kind of main two things that actually uh Yep.44 did for uh for scaling is uh one that blobs now have this independent fee market that's separate from the L1 Flea market uh of of gas essentially and then that blobs are only stored and served for about 18 days. So you don't store them forever. um and is just two simple

consequences like the because of the independent fee market they're not competing with L1 so L1 could be quite expensive but blobs can still stay relatively cheap uh and also that there's no storage growth like there's kind of a fixed storage associated to blobs uh so it's much more okay to to scale them up in in that narrow sense um and this is a bit uh overwhelming but like we'll see this a lot so I think eventually you'll get used to to the picture um this is kind of one way to visual ualize what the whole architecture of of 4844 looks like. Um, so there's the execution layer on the side uh on the on the left side and the consensus layer on the right side. Uh, we have these like gap nodes, but like could be different clients. I I I just,

you know, didn't didn't want to uh bother u copying a bunch of different ones for different clients. Um, we have the the blob mele on the essentially, you know, this might be like a rollup node or or like a node run by a rollup. uh that's submitting a transaction to the mempool, a blob transaction to the mempool. This blob transaction go around, they propagate, eventually they get to the proposer. In this case, the e node of the proposer, the execution layer node of the proposer. Um through the engine API, we kind of cross this barrier between the execution layer and the consensus layer. So the proposer runs like an execution node, a consensus node. U the blobs get sent together with the block to the CL node. There's this lighthouse, but again could be prism,

could be anything. Um, and now we kind of are in the in the consensus layer side of things. The proposer needs to basically push out these blobs through network as fast as possible. Um, and the way it specifically does so is kind of similar to how it works with a block. There is a certain uh gossip topic. We we call them like a way to organize nodes uh so that they can propagate information. Um, and so there's one for the blob and there's actually different ones for the blobs and a different one for each blob. So not not a single gossip topic for all of the blobs, but each one for each blob. Um, so yeah, this red blob is going to be sent to like different beers than than the the blue blob. Um, and then eventually, you

know, things propagate. They're supposed to get to a testers, these other uh beacon nodes. uh and uh yeah that's kind of the critical path of consensus like they test and we sort of establish consensus on uh the block and the availability of the data in particular um and yeah so let's zoom in on the right side uh specifically on this consensus layer networking architecture that that exists in 444 um so I mean this is the basically the picture you already saw the proposer and all these subnets um so one more advantage that it's not as obvious from from for44 is that that we kind of see from the uh this more specific look at the networking architecture is that as I said blobs don't don't travel in uh

don't travel together I mean for one thing they they don't travel with the block so it's not this you know block plus blobs bundle which would be the naive kind of way of of uh going about uh sending out the block they travel separately and in parallel like quote unquote in parallel u but yeah on on these different kind of subnets Um, and this actually has like quite a lot of advantages. Like one way to to think about it is on the right would be this naive way of doing it where you just send this full kind of bundle of blobs altogether. You would send it to eight peers. This is just how gossipy tub works. The the way we propagate things. You kind of have um a mesh of eight peers uh and you just send stuff

to them. And uh yeah on the left side is how it actually works with these kind of parallel uh gossip topics for each blob. Um so you would send each blob to eight peers like the and the total amount of data that you send is the same. You're sending each blob to a peers in both cases but in the left case which is what actually happens. There's this kind of much wider funnel where you you basically get a lot more people to contribute with their upload bandwidth earlier in the process. like very early you get all of these nodes to do a little bit of work for you essentially in helping you propagate things and yeah in practice you can see that it it works out better or also in simulations you can see that it works out much better um and yeah so we have these advantages

that we talked about but there is one kind of glaring you know issue that that's not really been resolved which is that nodes are still downloading all of the data like we have these I mean we're calling them subnets which kind of stands for subn network but in practice actually u everyone one is in all of the subnets. They're just the same as this kind of beacon uh block topic. So all of the nodes actually participate in all of them and so they all get all of the data which is ultimately not not the goal. And uh yeah so the question is how do we move beyond that and as you know you might expect if you come to this talk it's through data availability sampling. Um so quick recap um what basically the where dability sampling

comes from is u or like what's the core idea is erasure coding. So we take uh essentially um let's look at like this blue part is this is the original data and we u split up now look at it not as one big chunk of data but as like split up into like finer chunks essentially. Um then we we do some eraser coding um and um which I'm I'm not going to go into like exactly how it works but it um I mean it's it's it's not you know some something super specific to Ethereum um and uh essentially increase its size by 50% by adding uh redundancy and now any 50% of the of the um this new data uh can reconstruct the the full data in in

particular the original part and uh what this does is is basically makes the problem of checking availability of a blob uh become uh so which would be like checking 100% of availability like making sure that all of the pieces are are there um it becomes making sure that only 50% of it is actually there which is a much simpler problem and the and the reason it is a much simpler problem is that you can do sampling. So once you know that what you're looking for is just 50% of the data have been available um you're actually able to just maybe try downloading like a few pieces and get like some higher confidence that or like high confidence that the the whole thing is actually available. Now there's some subtleties

here. It's it's not so easy to individually get really high confidence because someone could always just give you exactly the data that you're asking for and and uh not uh actually reveal the whole thing. But when you have a lot of nodes that are doing this then kind of collectively you can get like really good guarantees. Um but yeah then based on this idea essentially the goal would be that full nodes should only download like a few chunks of the data or like some uh you know smaller than whole portion of the data eventually. And uh how do we plan to do this in practice? Like that's u you know that's just one blob. In practice you have a bunch of blobs. The first step would be to uh basically just create this kind of matrix from blobs just

stacking them on top of each other. Um each one would be like horizontally extended with this ratio coding that I mentioned and then sampling would be essentially columns. So um you know kind of the same as this but extended vertically to to the whole matrix. Um and so one way to think about this is that um it's the same as if you were to individually sample each blob. So you kind of take each blob and and you do some some sampling. You randomly choose some indices to get. Um but in this case you use the same choice of indices. So you just choose the randomness once. You say okay I want like you know two uh six and whatever this is and then you you use the same choice for all of the rows.

And so ultimately what it means is that you you get you you want to get a column uh for as as a sample. And uh yeah basically if uh 50% we have the same thing that we had for for individual blobs. If 50% of the columns are available then the whole matrix is is available. This is just because if 50% of the columns are available then for each blob 50% of the cells are available. Uh yeah and these will we'll call cells the the individual kind of um pieces of a blob. And uh so yeah this is kind of you know very high level like how what's one way that one might go about doing sampling for for the whole set of blobs that we have. Then the question is um how do we actually

incorporate this in the in the protocol like where does this actually fit you know before we add this like more detailed picture with all the networking things and so on. Um, and so one way to to think about it is that there's more or less three kind of phases that we need to worry about and then we'll like actually make these more specific and like put them in the right places in the protocol. Uh, the first phase is distribution. So at some point there's going to be a builder or proposer that has all of these data and needs to send it out to the network. Um, so this before in 4844 um is just this like subnet system that we have. Um so yeah that's one question like how do you push out the data to the network actually um then how do um full nodes validators

like whoever is is actually uh doing some sampling how do they get the samples that they need like okay I want this column where do I get this column from like maybe they did exist somewhere but then yeah how do I actually get the piece that I want and then the final piece is reconstruction so I mean the whole point is that the data might actually be less than 100% available it might be that not uh all of the columns are available. There's some they're missing. And in that case, the the whole point is that we should be able to reconstruct. But not only we should be able to reconstruct, we actually need to do it and kind of heal the network. Like you can see, you know, here this red kind of sick network where some columns are missing. Uh eventually we want to

actually get to this healthy network where like everything is is reconstructed. And then later we'll talk about more specifically about the the kind of timelines of uh you know how fast this needs to happen. Um and yeah that's kind of an introduction but now let's more dig in the details of uh what's happening specifically in Fusaka with EIS. Um so the first thing is u the first phase that I talked about distribution and the second phase sampling. They're essentially one and the same. They're they're merged. Um and actually it looks very much like in 4844. So um we have still this like kind of subnet structure with these gossip topics that are um kind of again in parallel more or less. Um but uh in this

case so there's two big differences. One is that before we were sending blobs. So we were sending these like basically blue uh rows like the the blue part of the of the rows is what we were sending in these subnets. Um now things are basically instead of uh being horizontal they're vertical. So we're sending columns. So we kind of you know stack these blobs on top of each other. We do this razor coding and then we send out the the vertical pieces. Um so that's one that's one difference. And then the other difference is that we're actually doing sampling uh in the sense that you only participate in a few of these subnets. Um and so you know for example in this case there's a bunch of these but I'm only getting this piece and this

piece which is what I would have kind of wanted to sample. And then the other uh piece is reconstruction. And u so the way that it it's meant to work in Fusaka is with this one of an honesty assumption. Um and specifically the set of people that we're making or actors that we're making this assumption about is um nodes that are willing to participate in all of the subnets not not just sample but like really participate in all of the subnets. Uh download all of the data, reconstruct it and recede it if needed. So if the data is missing uh basically there has to be this node that actually is in all of the subnets. So gets all of the columns that are actually there does the reconstruction and then recedes and kind of helps uh heal the network. Now

of course it doesn't have to be a single node that like carries this whole load but the and and yeah if there were multiple of them this would be better. Some of them can send some columns some that can can send other columns. um they can all help out with the receding but then the point is that the assumption is just that there is a single node that that does this um and uh yeah so let's look at this uh in comparison to what we had before with with 4844 um as I already said like yeah blobs are horizontally extended so we have this like orange part and this kind of 2x like size increase that comes from this um we we don't have this horizontal blob blob subnets we have the uh the vertical column subnets and in specifically we have actually 128 of

them like this is how kind of how the the blobs are chunked. So one other small difference that it's not just like you know it's not just this vertical horizontal thing it's really uh this quantitative difference that we only have a few blob subnets like today we have six of them because there's almost six blobs whereas here we start having like a much higher number of subnets we have now 128 uh and yeah and then then the sampling you only participate in a subset of them um and so specifically the number of subnets that you participate in which is essentially your sampling count is eight. So this is the minimum like you at least sample eight columns and this is basically for security reasons. It's something where we can say okay if everyone is doing this um there's kind of high collective

asurances that about the data being available or or rather like another way to put it is that you can and most trick a small percentage of nodes into making a mistake about availability basically. Um and yeah basically what does this mean? 8 out of 12 128. So this is 1/16th of all of the columns but because of the 2x extension this is 1/8 of the original data. So with the current parameters basically the what you get in terms of um basically factor of bandwidth improvement like higher kind of bandwidth efficiency for sampling nodes is 8x essentially. Um and and I mean same thing for for storage that you only need to store it like u 1/8 of the of the original data

essentially. Um but so that's for nodes that are sampling. Not all nodes are sampling. Um in specifically I mean full nodes are definitely sampling. Nodes with a few validators are also sampling. So they only do this minimum amount of work. Uh there is this concept of validator custody where uh nodes can do more than the minimum or like they're they're basically default behavior um through that the clients will actually have is that they will do more than the minimum proportional to their stake. So if you have a lot of state, you'll basically download more columns. Uh and specifically if you have I mean the this is the kind of current uh formula you you download um um oh sorry this should be max but yeah like uh max of eight and

validator count. So um if you if you have more than eight validators you you download like a bit more than a minimum and once you have uh hundreds of validators like for example 128 uh you actually download the whole data. Um so this also helps create this like default uh set of nodes that are actually uh fulfilling that one of reconstruction assumption that that we talked about because if you do participate in all the subnets then by default you're actually helping with with reconstruction um and like not only um and yeah from the so we we mentioned this like 8x improvement factor for nodes that are sampling for nodes that are not sampling of course you don't get this improvement factor

but these nodes are kind meant, you know, the idea is that they would be on the higher end of of um basically bandwidth requirements compared to to like a random full node. Um and you still actually do get some benefits for even at this higher end of nodes that are not sampling. Um the reason is basically that when you think about it uh in terms of the these nodes that download everything, you can think about it as there's this giant message that they're trying to so they're really interested in the whole message in this case because they're they're they're participating in all of the subnets and you can think of it as like there's this giant message that they're trying to um get essentially to like propagate around. Um and this message has been eraser coded and split into um 128

pieces. Um, and this is a much better way to do propagation that what 4844 does. Um, I'm not going to go into detail of this, but like essentially from the perspective of these nodes like we've kind of you could think of it as like we implemented like a more efficient propagation system um for for them. Um just yeah basically just generally like doing uh smaller pieces in in coding like just works better as a way to to spread data around and like from the perspective of the nodes that want all of the data then this is what it looks like essentially. Um and then yeah let's finally get to uh a bit more the details of of reconstruction. So um let's think about a case where we have a proposer

which maybe you know is malicious or a builder which is malicious. it proposes some data that's uh not fully available. So these these black columns again are not available but it's more than 50% available. So this is a case where the data will might actually be accepted by some nodes. So here there's some beacon nodes um and a bunch of them are actually thinking that the data is available. The reason is because they're doing sampling but they might be you know just avoiding these columns like or these columns might be like a small percentage of the total columns. So you might still end up with a lot of people thinking that it's available, but there's someone that that thinks that doesn't think so. Um, still this block is going to get a bunch of votes again because there's a lot of people that think that it's available. And so we

kind of it's going to stick in the chain probably we want to to kind of heal the network. Um, and so the the idea is that this kind of availability state is what exists at the attestation deadline. Um at this point basically once it is greater than 50% available there there's this node that actually gets all of the uh all of the available columns does the reconstructing and receding and as all of this time like between the basically attestation deadline and the next slot like the the next time that a proposer needs to come in and do their job uh to do reconstruction and or I mean to do the the full reconstruction uh pipeline of like reconstruction and receding Um and yeah, so we eventually get to this

kind of yield network and everyone actually passes the availability check. And importantly, the proposer also passes the availability check because if there had been no reconstruction and let's imagine that this guy was the proposer here, then this guy would have actually made a mistake in their fourth choice because it would have thought that the data is not available though most other people think that it's available. So you know here basically you can see why it's important to have um reconstruction because you cannot have people disagreeing on what the correct view of the world is specifically with respect to availability. So if most people accept that something is available, you want everyone to accept that something is available. And uh the first instance where some where you know this is

actually important is the next proposer like you it's it's kind of the first case where some where someone might make a decision uh that's based on their local view of availability which really should be correct and agreeing with everyone else's. So this is why like this is the the reconstruction time essentially. So in practice this is 8 seconds and I I think we we think that it's it's plenty of time for for this to happen. And uh I mean and you know one one reason to think this is u the data in the first place is only supposed to take 4 seconds to to propagate uh and most I mean ideally less and reconstruction itself does not take that long. Um so yeah that that's why 8 seconds in principle should be should be planning for for this to happen. Um and

uh yeah again let's uh look at our high level picture. Um so we've kind of dug in into this consensus layer side. Um now maybe let's look at what's happening on the execution layer side. And so in particular the memp pool like what what essentially is for for the you know the purpose of DAS um or pieras um the the only part of the execution layer that's actually relevant that that changes is u possibly something in the memo. Um so one thing that you might have noticed is that well we actually only change what's happening in the um consensus layer side once with this column kind of structure and sampling. there's no sampling uh in in

the picture in the in the Yel and so yeah in the yellow meool blobs are still being fully downloaded which is kind of the thing that originally like we didn't want in the the consensus layer one question is like is is that does that not like nullify the point of doing sampling are we just like doing sampling on the CL while the bottleneck is actually on the EL at this point and so there's a few reasons why this actually not as straightforward as as this it really sounds that straightforward you know here you're downloading all of the things and here you're not what's the point of the the latter but it's not quite as simple um and so the reason is because on the yellow me pool first of all there's no strict deadline for things to propagate so you eventually do want the transaction to propagate but um

it it doesn't have this kind of uh deadline given by the testation um attestation deadline um so you don't have this like a strict kind of 4 seconds that things need to happen in on the in the critical path to the CL I mean you could already see it from the critical path wording. There's no such thing as critical path in the me pool. Things just eventually need to get around. Um and yeah, because of one like because of the fact that there's no strict deadline. Um we also can kind of be more uh lax like or relaxed about propagation in the in the meool. So you don't um you basically um announce that you have a blob transaction and then let the receiver say if they want it or not.

like you send them the transaction and then they say do I actually want the blob uh that's associated with it. Um and uh yeah you you you don't people don't just like send you immediately everything that they have. Um and so this means that there's no duplication essentially. So you only if you request a transaction uh from someone you get it that that they're the only person that actually sends it to you. Um on this Y instead because things are much more compressed in terms of timelines. So this issue of being in the critical path um we have this push um kind of model propagation where once you get something you immediately forward it to all of your mash peers like these like um kind of peers that you have in in your topic this like eight um nodes that I

mentioned before. Um and so this leads to a lot of duplicate messages in principle up to to um to eight uh essentially. Um and yeah this kind of uh means that a lot of bandwidth can be wasted because a lot of you you might be downloading a copies of the same thing. People might be sending sending out a a copies of the same thing even when actually a lot of their their peers don't don't need them and so on. Um and so there's like these things compound together the these two effects um to mean that the yellow meool can actually in principle tolerate like a much higher number of blobs. Um so thinking about the yellow meool there is this number that's I mean or or just generally for for DS there's this number that's gone

around of 48 blobs um I probably won't explain I mean yeah I can actually explain where it comes from the it comes from the spectra target being six and then this 8x factor that I mentioned before um that comes from sampling and so 6 * 8 48 it doesn't actually mean that 48 is the right number it's just from that narrow perspective this would be like the number that gives you the same bandwidth four sampling nodes. But yeah, this is a number that's gone around. So, let's just use this number. Um, even if you add 48 blobs, which I think would be like a pretty ambitious target for for Fusaka, maybe we'll get there at some point. I don't think immediately on on release, but um even then like in the me pool, this would only corresponds to 4

megabits per second. It's it's not nothing, but you know, it's clearly also not like something that like blows up the the whole system. Um and this just because we have like 12 seconds uh to kind of um spread out this load over and there's no duplication on the CL. The same thing would mean uh first of all it has to propagate in 4 seconds. So now we're already talking about 12 megabits. Um and you also have the duplication factor. So duplication factors might be something between like 3 to 8 um something along these lines. So you might get like 10 to 20 or even more uh kind of less efficiency on the CL than than in the than in the mele. And u so yeah where um I mean

still if we did want to um kind of um try to limit as much as possible any problem coming from the pool. One thing that we could do that that's could be quite simple is to uh basically allow uh some rate limit like local rate limiting where you just set that you only want to give this much bandwidth to your to your uh blob blob meool um and you basically don't pull blobs at a higher rate than that. So you know maybe uh you only want to use uh 2 megabits per per second for for the memp pool then you will uh only pull at most 48 blobs per slot or you can I mean you can set the kind of per time uh rate of how many blobs you actually try

to pull. Um and in the medium term there's kind of other ideas. I mean one thing is that it's it's unclear exactly what roles will the mele play in the future. Um I think it will always play at least the role of censorship resistant resistance fallback but it's not clear if it's going to have a bigger role beyond that. I think likely it will because because of what I mentioned that actually sending things in a meool can be very efficient as kind of a pre preagation step. Um and so yeah, I think in principle it would be it would be nice if we could make use of this. And so there's some ideas around uh what might be called mele sampling. Uh so yeah, basically introducing sampling concepts also to a memple uh meshing

kind of what's happening on the on the CL but out of the critical path. So basically get the benefits of sampling also get the benefits of doing things out of the critical path essentially. So basically allow um blob propagation broadly speaking to happen kind of in this more relaxed way in advance of of the critical path. Um and uh yeah so now we've kind of covered the consensus layer um at broadstrokes the the execution layer at least as you know as to what's happening with respect to to the meool and which is most of what's relevant to u to pieras and kind of we have now this this green box of uh basically building um and uh yeah let's

let's kind of dig into that um so there is a few things that you need to to build a block one. We've already talked about it, so we'll skip it. You you need to run a me pool. So, as long as that's a feasible thing to do, that that's not something that you have to worry about too much. Um, in fact, we also maybe need to worry about it even less in some sense. Um, in the sense that you you don't actually even need to run a meool all the time in order to to build blocks at high blob counts. uh you could even for example just say okay I'm most of the time I actually am implementing these rate limits uh if if you have you know reasons to to do so um but when it's my turn to build then I'm

going to like fully try to get as many blobs as I can um and again as I said it's this doesn't require like a crazy amount of peak bandwidth for example it could be something that makes sense if you're concerned about um kind of sustained bandwidth consumption like you you you really have you know you have like data limits or or something like uh like monthly limits u then yeah you could say okay I I can actually only when I'm proposing like fully just try to participate in the meu like pull anything that I see g as many blocks as I can and then try to build kind of the best block that I can if that's your your limitation um then there's this other part that uh it's basically that we haven't talked about at all and so

you know this now maybe doesn't make sense uh what even it's saying case deep proofs for cells and like needing to compute them. Um, and we'll we'll get into this. And yeah, the last part is that you have to be able to push out the data fast enough. So this kind of what we talked about on the CL in the critical path, you have to quickly send out data so that it propagates in the network uh by the attestation deadline. So these are kind of the key components of block building, at least the the DAS part of block building. Um and yeah for um let's let's get into the the selfp proof uh stuff. So this is something that so far we've just omitted um when talking about sampling. So okay you you were um on the CL you're you're

subscribed to these subnetss you want to get these columns you get them. Um now how do you actually verify that these columns are really the right data that actually matches the KCG commitments uh in the block? Um and yeah basically I mean uh I should I guess a brief word about KCG commitments. These are basically just the uh little pieces of data that we have in blocks that uh basically um yeah that commit to which blobs the block has like the block doesn't contain the blobs it contains only the commitments. You can think of this as like Merkel roots or hashes but like with deeper cryptographic powers. Uh essentially yeah it basically just lets

the block um contain only like uh tiny amounts of data but still commit to a certain set of blobs. Um and so given a certain um kg commitment for so you know here we would have a kijg commitment for this blob a kg commitment for this blob for for all of them for the whole list and this is already the case in in in the current system for 844 um you would want to be able to uh get this column and know that it actually corresponds to the the right commitments even without having the whole blob. Like of course if you had the whole blob you could check that a blob responds to the commitment but uh yeah now you have only like a piece and kind of have this this issue of how do you do this um and so in

practice how you do this is with proofs um the yeah the cryptographic box that I'm not going to like fully uh or almost at all open uh around KZG but uh what this looks like on the on the network level and like what a what a a node is doing is that uh when you're subscribed to one of these subnets, you know, you you get something like this column. You actually don't just get the column, you get the column, but in this bundle with commitments. So, this is like this little thing that commits to the whole row. So, not or to the whole blog, not just to to to this part, but to the the whole thing. Uh, and then a proof. And this proof is basically uh what we were calling cell KZG proof. So, it's a KG

proof for specifically this cell. And so what the what this does essentially it lets you prove um an opening of a commitment at a certain specific index. So it lets you say okay uh there's this blob extended blob uh it corresponds to this commitment. Um I supposedly I have a cell for index in this case three or two. Um so so yeah supposedly someone's giving me this cell they're claiming that it is uh the actual value of this blob at this uh at this index. So you know blob of I equals cell essentially um and it gives me this extra piece of information this proof that I I can use it to verify that. So

given the commitment, the proof, the blob and the index, I can check that actually without needing all of these other piece of data like the rest of the blob, I can check that yes, that claim is true. This thing is really the value that this blob has as this index and then you can apply this to uh to the whole column. So you have, you know, one proof per column as here and you you check that the whole column is what it says that it is essentially. And uh yeah one thing to to know about uh selfgi proofs is that I mean there proofs the whole point is there's an asymmetry uh to some extent to uh between computational verification um and so proof verification is quite

fast like even an individual one um I think it's like one to three milliseconds depending on on which hardware um and also uh importantly because there's a lot of these like each cell has one proof so Um yeah you have like for each blob you have 128 and then many many blobs. Um so batching is very effective. So if you have 64 blobs it takes about this much let's say half a second to verify all of the cells and this without even parallelizing the the computation. So in principle verification is it's not so bad. And I mean even you don't actually need to verify all of the uh cells. You only need to verify a certain set of columns that you're sampling if you're a

sampling node. Um but uh yeah proof computation on the other hand is quite slow. It is perfectly parallelizable because you have a bunch of blobs and each blob is completely independent. So you can do the proof computation for each blob completely in parallel but still it's quite slow like on a single blob it's uh about this much like 150 to 200 milliseconds um to compute this like 128 proofs. So this you can see that you know if you add 20 blobs this is 4 seconds. If you have 100 blobs, uh these 20 seconds like it's it's it adds up even even with parallelization. And so this is a bit of an issue because um the you know by default where this would happen is here in the critical path. So you know we've left uh everything here

kind of basically unchanged on the meool. So you get this this blobs like nor as normally as it happens today. They end up in the in the uh proposer and get sent through the ng API and so on. At some point you have to take these blobs and convert them into this matrix with proofs. So you know doing the extension this uh turning the blue thing into also the orange thing like having the redundancy it's quite fast but then computing the proofs I said it's quite slow. So the the issue is yeah that this is really happening um or I mean by default it would be happening um in in part of these like critical 4 seconds where things are really supposed to be like uh you know fast and and kind of making sure that you meet the deadline

and so that's the the worst place to put like a very um computationally expensive operation. Uh and then yeah I mean just as a note like you know here uh this is meant to be one proof per cell. So this would be like you know one here, one here and so on. And same here, this will be one proof per cell for the whole column. And yeah, you then send out the cells, I said in in kind of as a bundle with the with the column. And and once you get, you know, to the tester, the tester is going to use this to to verify that they actually got the right piece of information. Um, but yeah, so as I said, it's not ideal that this is in the critical path. In fact, it doesn't need to be in a critical path and part of the

design and this is really like a part that's been actively um worked on in the sense of basically the spec is being finalized around this. There's kind of active discussions around more like details of this but it's you know now really part of the of the uh pureas design for for fusaka um to actually take this out of the critical path. Um and the way this is done is basically that you push proof computation to the senders. Um so you you know there's someone that's submitting this transaction in the mele um they can actually just do the computation for their blob like they can extend the blob compute all the proofs submit all of the proofs to to to the mele because we have this asymmetry like computation is uh is

slow verification is fast so if someone does the the job kind of at the edge when when sending the transaction then the proofs going around the network isn't so bad like verification is fast they're also very small one very good thing about KZG is that um basically yeah the the proof size is is is really tiny it's like 48 bytes so yeah it adds like very little overhead uh and yeah it's 48 bytes for a cell which currently is two kilobytes so it's it's really small u kind of overhead um and yeah so with uh this kind of architecture uh the cell um proof computation would be both out of the critical path path and

distribute it like now there's no single actor which actually needs to it's not the builder anymore it's not the proposer it's not even like specific nodes in the me pool it's whoever is actually uh originating the transactions and these are different actors and also like generally well resourced but even without thinking about that it's just something that you know can be done by a lot of actors and it's not time critical even because when you send a transaction well if you need to do the proof computation it's just going to take a bit longer to to send the transaction um and So yeah, this basically is what the current design looks like, which is you you still have um these proofs like on I mean on the CL everything is the same. You eventually do want, you know, everything to go with the proofs and so

on, but the proofs are coming in from here. They're not computed in this part of the of the design. So yeah, basically, you know, maybe this is like uh the startnet sequencer or something. It's generating a transaction. It's computing the proof and it's submitting it to the meo. And you know, this is another rollup or whoever. and they're they're doing the proof themselves and then here there's only verification happening and everything is passed to the proposer which again only needs to do verification and uh yeah so now we've kind of okay gone through the mele um part of things gone through this proof computation kind of sit um and we're back to the to the problem of on the CL side of being able to actually uh push

out the data fast enough and so this is a bit challenging Because now I mean well there's kind of two reasons. One is we're adding a 2x factor. So there is this you know orange thing that you also need to send out. Uh and so yeah we've already coded the data. This is great for sampling reconstruction so on but when you need to send it it's now 2x the original amount. Um and also because uh yeah you need to send it to eight peers like um as as I said before in in each of these uh subnets you kind of when you propagate you send out to to eight peers and so you have this like big factor of kind of overhead from that. So at the end of the day you essentially need to

send out 16 times more than the original size. So actually you know even 50 blobs it's not that much data. It's 50 megabits. Um so you know not not even uh uh like yeah whatever that is you know 66 megabytes. Um but once you add the 16x factor that actually becomes quite a lot like if you need to send out all of the data in one second to all of your of these eight peers and so on now it would require 800 megabits upload which starts to be pretty significant. I mean for you know for local blog builders is for the most part pretty much I mean on unattainable there's people that have gigabit but like of course this is not something that um that that's within like reasonable requirements essentially

um at least again for local block builders and so yeah the question is is that the bottleneck and kind of jump ahead a bit a little bit yeah one important thing to to keep in mind is uh whether we're worried about local block building versus building in general I would say let's just worry about local block building specifically. Um when it comes to like a meth boost builder, they can kind of figure it out. It's yeah, even even that amount of um even that bandwidth is is is really not uh something like too prohibitive for for them. Um and I mean and there's other things that you can do to it can be pushed out from different nodes. But yeah, like um there's there's also I mean there's even other kind of

techniques that they make things better. You actually need to send the data to to eight peers strictly speaking, but kind of beyond beyond the scope of this. But yeah, the main worry is really local block building like to some extent, you know, whatever you think about local block building, we at least want to enable um building to to be possible not in a data center. It doesn't mean that everyone should be able to to do it like no matter what kind of bandwidth they have. Um but yeah certainly not that that kind of uh the requirements cannot be as high as that or like nowhere near as high as that. Um if we want to ensure that no matter what someone will be doing uh will be doing the job essentially. And so one thing that's kind of critical here is that when you build locally all

of the blobs are coming from the vampool. Like you don't have access to kind of private mess. If you imagine like a an entity like a just just you know solar staker or uh something like that they just get blobs from the mele that's the only source of blobs that they actually have. And so one thing that um that we can do with that is well the blobs do already exist in the mele essentially. So you know the whole problem here is the proposer needing to push out the blobs to other people but actually the proposer is getting the blobs from a network this separate like yell network where they already exist. So they they in some sense there there's maybe no need to actually recropagate them. And so one thing we can do which I mean actually this already happens in 4844 um but it's

like not as critical let's say is that uh yeah you could just send out the block then anyone that gets the block now knows which blobs they're supposed to get because the blob has these the block has these commitments and you can basically request it from your node. So your node is running a mempool hopefully already has all the blobs and can just basically send them to you um and you know you kind of get all whatever columns you you might need. Um now this is a bit of a naive version of this. There's a big limitation which is that if you miss even a single blob then you're unable to reconstruct the columns like you know let's say you miss this one now you would miss this whole kind of row. uh it's it's not useful to have

only like a partial set of blobs. Uh like all of your sampling would fail. Like you would not even be able to get a single column because you're missing some pieces for all of them. Um and so yeah, we can actually do do better than this. Um and uh yeah, this is what's called distributed blob publishing. Um basically, it's the same idea, but um with the twist that you don't rely on everyone just being able to do this pulling successfully and get everything that they need. uh you rely that on someone being able to do this like someone having all of the blobs or like a few nodes and and so on um and then helping to propagate the columns. So you know someone should successfully be able to do this and then kind of repropagate

the data on this yell. Um so one way to think about is basically the proposer sends out the block and it's almost like a request to uh help him with the propagation. So it's you're kind of outsourcing the task of propagating columns to anyone that actually is able to get all the blobs from the me pool. Uh so yeah basically in this case okay you get the blobs from me poolool maybe you only use blobs that have been there for like a little bit of time so that you're you're more sure that uh someone should have all of them. Then there is at least this one node that uh is actually successfully able to do to go through this process. Request the blobs from its yellow node to get the whole matrix and then this one person now kind of acts as the proposal was supposed to

be. Um so it actually does this step of like propagating everything. uh so there's kind of this like you know extra step before getting to this to this actual u propagation on the CL and eventually I think still get to it to the tester and they test um and can actually look better than this even because it doesn't have to be a single node that that does this um there could be a bunch of nodes they're all kind of successive successfully able to do this and then uh these nodes don't even need to be subscribed to all of the subnets uh for example this node Even though it does get all of the data from the Y, again, there's this a bit of symmetry between the Y and the CL where on the CL you might care about only a subset of

the data. On the Y, you always have all of the data at this point. So you get all of the data, but then maybe only propagate this one column and then these other uh node propagates this column, this other node propagates this column. So different parts of the network can kind of act as repropagation points for specific uh portions of the data that they care about. And then maybe there's some nodes that just care about all of the data and they kind of push it all out. So there's there can be a bit of like u essentially a bunch of repropagation points throughout the network that actually make this whole essentially much more uh efficient. So as long as there's like enough uh people that have all of these blobs, so as long as these blobs are well distributed in the meool, then on the CL things should

basically take care of themselves. like people will kind of boost propagation uh to the to the to the nodes that maybe they're missing some of them and yeah I mean with this I think and this is very much again part of the active um you know things that are being um worked on actively and and where there's a lot of interest in in doing testing. I'm really like it's something that's worked on being worked on right now being being tested right now. Uh but yeah, the idea would be that through uh this kind of mechanism, it should still be possible to local block building even at relatively high blob counts without actually needing um a lot like a a big upload um

connection. And yeah, with that, I think we've gone through a lot of the the the bottlenecks, a lot of the uh you know, kind of uh interesting parts of the design and and motivated why they're they're there, what's left. Um but basically just a bit of implementation as I said this get blobs part that I just mentioned is is actively been worked on there's a little bit of work on the uh this transaction proof computation and like pushing it to the edges u but really like I I would say most of it is just a lot of testing a lot of stress testing a lot of um figuring out what actually is the the you know the real behavior of some of these things in a in a network and in a

real network at scale. Uh and then yeah figuring out from there what things need to be kind of tweaked and and you know how far how far even um we can push things like how many blobs can we actually support safely and so yeah there's there's definitely a lot of work but I think uh the design is essentially done at this point for for Fusaka um and and there's just a lot of work left in in actually um basically figuring out what's the best shape that this can that this can be shipped in. uh and uh yeah and then I mean there's a lot of uh still research work and and and so on or R&D work in figuring out then what's what's next.

There's there's a lot of ideas which unfortunately I didn't have time to to cover. I I decided that it would be interesting given where we're at in in the cycle to more focus on the details of the current design. Um but yeah, for for another uh time, I think there's there's a lot to say about the the next steps essentially and I think maybe yeah, in a few months we might have even a clearer idea of what's the right thing to to kind of do next. And uh yeah, thanks everyone. Um again, thank thanks for inviting inviting me. Uh you can find here at DAS.wtf WTF uh a lot of resources that that uh people have helpfully compiled um other talks and other kind of there's also a link to another website that's called das.atatm

ATM uh where it's it's much more like in the process like it it it's almost like a PM kind of website with like what's going on with peer dust right now like really current issues breakout calls like all these things. I I don't know exactly what the audience looks like, but uh maybe there's someone, you know, some some of you out there that really um are kind of taking interest in in seeing like what's what's going on. Even maybe some of you can help and you're welcome to. Uh so yeah, lots of resources here if if you um yeah, and I mean of course if you have any questions about anything um you can reach me on Twitter possibly. Uh that's uh at f r a d a mt. Sorry didn't think to to write it on the slides but yeah somehow

I hopefully you can find a way to to reach me and feel free to to ask questions. Um again thank you and uh I should I show the the slides you let's do questions first and then show show the end slides. Um okay so I know I've got some questions. So first of all thank you for a great talk. Um I I I know I learned heaps and um it's really good to see where things are headed and um yeah it the design makes a lot of sense to me. So that's really good. Um Chris, Paul, Joanne, do you have any questions? Okay, let's say we've got no

questions from them, I think. So one thing that you didn't talk to is so I know that some of the blobs end up being only partially full because essentially a say a rollup they sign up to a particular blob you know and then so they might only half fill a blob. So is there some sort of compression technique to handle the fact that the blob might only be half full? Um you know thus reducing the amount of data that needs to be pushed around.

Uh interesting. I it's a good question. I I mean I I definitely don't think we we have something uh along these lines. I mean, there is um um snappy compression, I think, pretty much used by by all clients for for sending things around in the network. Uh but I'm not sure if it would actually help in the case of um in the case of blob specifically. Um yeah. Yeah. Cuz I mean, normally the transactions are compressed and if it was base rolling up, you know, chances are they're going to have com full logs. But then some of the other rollup systems, you know, they might

post a blob, but it might not be full. So there might be a whole heap of zeros, you know, in it, which, you know, you could just run length code or something something really simple. Um, but yeah, I I I'm not sure. I actually that's a good good question. I I don't think there is something like that right now. I mean there is this like compression by by default that's used on all networking but I don't think there's something like specific for blobs. Um yeah because I mean I guess you're going to do the read Solomon coding to expand up the data and when you do that that it's you know it's like it's gone through a hash function or something. So it'll be yeah I mean yeah on the on the expansion side. Yeah I think there's

probably nothing. So you'd have to before the expansion, but then I guess if the whole system assumes full blobs and it maybe it just makes things more complicated. Um yeah. Um so another thing I wrote earlier in the talk which I think I've answered my own question but so if there were if the blobs were split up you know rather than 128 pieces it was 256 pieces would that be better or is there some tradeoff about the number of splits?

Yeah. So there is a tradeoff. I I think it's likely that at least for 256 it might be it would be better. I think the idea partially with 120 is to start with something a bit closer to what we're already kind of used to like we have 64 subnets for when it comes to stations like to I mean that that's one thing but also just um to yeah like not kind of immediately try to push things too much. Uh but yeah, the part of the trade-off is proof computation. So the more you you split, uh the more proofs you have because you have a proof for each cell like for each like minimal thing that that uh can be kind of go by itself essentially. Um and so yeah, proof

computation like um just would take uh quite a bit longer the more you split. I think we consider up to 512 and uh yeah, it's it's much worse. I don't remember the exact factor but yeah I think it might even be like about linearly worse. Um yeah which would make sense and uh so that's that's one thing. The other thing is you get a bit more overhead uh on the networking side because now you have a proof per smaller unit. So you know before it was 48 bytes proof uh plus also 48 byt commitment uh per 2 kilobytes cell and now it's 1 kilobytes and you know with 512 it becomes like uh 500 by uh bytes so it becomes even like

actually pretty real overhead like between the proof and the commitments it becomes like 20% overhead um so that's another thing um and then I mean one other point is it it only helps the samplers um to to to make the the columns thinner, not the nodes that are kind of participating into all subnets. For them, it's actually just worse um because now you have more overhead, but still the same amount of data. But still, I think it's it's quite possible that we might uh in a later um iteration like change this parameter to to to something bigger um precisely to help the samplers. Yeah. And and you could imagine I guess if you had say you know so there are bases obviously you know

like a huge roll up but there are others and so maybe maybe if there were multipleized um blobs you know like so there were a large blobs and normal blobs and cuz at the moment I think um you know bass uses something like two or three blobs per block every block as um in its roll up and I think um um arbitrum one uses a whole heap as well like one or two per block every block. So you could imagine for those um those rollups with lots and lots of transactions if there were was a big blob. So, I know it again complicates the design if you had two sizes of

blobs, but um I wonder if that that would then mean if you had 256 splits, the the actual data size would still be okay for instance. But yeah, any thoughts for the future though, I guess. Um yeah, it's good to see that. Anyway, sorry. No, no, yeah. Sorry. Go ahead. Yeah. Yeah. It's good to see how close it is and a lot of the how a lot of the thought and things have been really um Yeah. Well, anyway, one one other question that I had was around um block times. So I know in the road map the longerterm road map for Ethereum um

we've got you know three slot finality but another part of it was trying to reduce the actual you know like slot time down and um I guess if as that slot time goes down that 4se secondond propagation time for blobs would start to become you know like problematic if you were trying to get your slot time down to 1 second or 2 seconds. So um yeah so your thoughts on that? Yeah so I mean one thing is that um if we did decrease the slot time it would also come with a proportional uh decrease in the in the total amount of data. So like let's say the number of blobs that is pushed. So from like the bandwidth

perspective things should be essentially stay the same. There is the latency part of it which is still like you know there's a few hops that things need to go through the network and like part of the cost is just the bandwidth but part of the cost is just the latency and the latency part doesn't matter how you know how big the data is it still exists so you kind of have this fixed cost of latency that becomes bigger um so it definitely you would impact things if you if you shrunk uh the the time um a lot um but uh yeah I also I mean I think it's it's uh still quite like I think there's a lot of work to to do to really figure out like concretely realistically like what is a good way to to uh reduce the

slot times and much in a system like Ethereum like of course there's other blockchains that have much shorter slot times but they also have like very different assumptions and so on. So I I I would say there's yeah there's definitely a lot of a lot of work to do there. Yeah, for sure. For sure. And and I think yeah this is the thing I think everyone needs to remember Ethereum is not Salana um you know Salana has tradeoffs different tradeoffs to Ethereum and um yeah I think that's where it is. Um are there any other questions? Okay let's flick to the next slides then please.

Uh yeah, sorry. Just give me one second. I'm going to plug in my laptop because I think it might dive before we get to the end. Okay. All right. Not a problem at all. Um yeah, so whilst Francesco is busily plugging in his laptop, I think everyone will agree that was a fantastic talk and that we know a lot more about Pier Daz than we did before. Yes. Some claps all around. Yeah.

Okay. Uh, yeah, back in business. I should have just gone to the to the slide before that's okay. Okay. All right. There is a merch store. Um, there is an appearance shirt though. Maybe once the tech um goes goes um live in Faka, I'll come up with a new t-shirt. I'll think of a interesting. Wait for the spec to to be frozen. Yeah. Wait. Yes. Otherwise, if I have the block diagram, it'll all be wrong. All right, next slide, please. All right. Um, so in two weeks time, we've got a talk on the three slot finality. Um then um more two weeks more after that we've got a deep dive

analysis by our pseudo anonymous um auditor who um has given us some great detail in a previous talk on hack and is going to again do a really deep dive on how bybit worked and try and talk to the strategies being used and you know like look for patterns there. Um after that we've got a talk which is about the intersection of AI and blockchain. Um after that there's a talk by a ZK rollup um infrastructure company. Um but anyway we're just finalizing that. And then finally in June I'm going to talk about the other

part of Fusaka that is huge that will um change things dramatically which is EOF and um give a deep dive on EOF and looking at exactly how that works. Uh next slide please. And so if you're watching this live it's available on YouTube. Um if you're watching this on YouTube and you want to ask the um speakers tricky questions uh join the meetup. Um there is a Slack workspace and finally um for some things there is example code which will be in that repo and that is all. So Franchesco thank you again. Um that was an

excellent talk. Um I think we've all learned a lot. So um yeah thank you. Thanks again for inviting me. It was it was a pleasure. Yeah. All right. Well, have a great day and um I'll send you the link to the video on YouTube when it's available, which should be in about 24 hours. All right. Great. All right. Thank you. Bye. Bye.