Ethereum blob gossip and validation before and after PeerDAS

Transcript

[Music] Hello. Hello. Testing. Hey everyone. Cool. Yeah. Uh, thanks for coming. Excited to be here presenting um, blobs in the future of the blob mempool. And this work was joined with Julian and Franchesco at the Ethereum Foundation. Um, so just to set the stage a little bit, I'll kind of outline what the agenda of the talk is. We're going to start by talking about blob services. And I think it's important to kind of ground our discussion in what do blobs do for L2 users and for um the roll-ups themselves who are consuming the blobs when they post to Ethereum.

And uh kind of once we've grounded the discussion in what blobs are for and kind of what the the main point of them is, we'll go through how blob gossip and validation works on Ethereum today. And this is important because this is kind of the two modalities in which people have to download blob data. And um it's kind of good to understand how it works today before jumping into what changes with pure. And puras is kind of one of the next upgrades to the Ethereum protocol and it aims to kind of scale the amount of data that Ethereum can handle. And we'll kind of go through how that works and why. And then we'll talk about what we can do moving forward. And in particular, we'll talk about how Purdas introduces this kind of asymmetry between how blobs are are processed in in two different parts of the stack. And

I'll outline some of the things that I think are important moving forward. So yeah, just to start with blob services, I think it's helpful to kind of think of them in terms of confirmation rules. And I informally define a confirmation rule as a signal indicating the status of a transaction. So this can mean a couple things. Um, one of the kind of default ones is when your transaction on Ethereum L1 is actually included in a block, right? And etherscan reflects this by saying this block um is unfalized and may be susceptible to reorgs, but it also gives you that success green check mark that means it was included in an Ethereum block, right? But there's that's not the only confirmation rule you get um or the only confirmation you get rather. Um you also get the finality which is a much stronger guarantee. This is something

that everyone's probably very familiar with which is the block is finalized and cannot be reverted without slashing at least one-third of validator stake. So this is kind of the second much stronger confirmation that L1 transactions get. But L2 transaction also get confirmations. Right? So this is a picture from base scan and we can see that it it has the green check mark. It says success. This transaction has been sequenced and executed. Um but the the confirmation is actually coming from the sequencer itself. Right? And this is kind of one way that that people get assurances about their L2 transactions. And the the takeaway from this slide is that blobs actually provide confirmations for L2 transactions. Um, but the the type of confirmation and the importance of that confirmation to the

user depends a lot on the sequencing model of the L2 itself. So I'm going to walk through two different versions of sequencing for L2s and show how that kind of results in very different blobs. In particular, blobs that have very different time preferences for when they need to land on the L1. So, as I described in the in the previous slide, the let's start with like the centralized sequenced L2. This is kind of the dominant model of of roll-ups that exists today. And we have a user that sends their L2 transaction to the centralized sequencer and they get this confirmation number one from the sequencer, right? And for this is kind of the green check mark I was showing in the previous slide, but for most people, this confirmation is sufficient like I trust Coinbase to sequence my transaction fairly, whatever. And I I

trust that they're not going to reneg on that promise that it was executed in in the order that that it was confirmed to me. But of course, this isn't the end of the life cycle of the L2 transaction because all I have right now is kind of a trust me bro handshake from the sequencer. There's also the process of getting that L2 transaction data posted to the L1. This is kind of like the whole point of rollups in the first place is so that that transaction data makes its way to the to the base layer and and kind of inherits in in some sense the security of the L1. So the the kind of dominant way that centralized sequencers land these blobs uh with L2 transaction data on the L1 is by sending them to the public mem pool. And once they're in the public mem pool, they get picked up

either by a builder or proposer who then includes it in an Ethereum L1 block. So you kind of have this nesting where the L2 transaction gets packaged into a blob and that gets packaged into a block and that block is sent over the consensus layer network. And this is actually another confirmation, right? So the confirmation now is that oh I actually see my L2 transaction data posted to Ethereum. But of course the second confirmation isn't that important and the the takeaway is that the first confirmation is actually sufficient for most users and there's kind of no urgency for the centralized sequencer to actually land the blob on the L1. And for this reason, I call these these blob transactions patient, right? And in particular, they're maybe happy to go through the

public mem pool so long as they get included within a slaughter 2. Um, and they they're not worried about kind of getting um front ran. They don't carry me. They don't need to play timing games and stuff like that. So on the opposite side of the spectrum, there's a set of blob services that actually do lead to a very different paradigm for blobs. And in particular, that is the sequencing model of total anarchy. And in total anarchy, we basically say, okay, anyone who sufficiently, you know, um, lands a blob on the L1 with some L2 transactions is the L2 sequencer for that slot. So, everyone's kind of competing, racing to get an L1 transaction L or an L1 blob transaction landed with all the L2 transaction data in it. So, I use Tao as the example here because they're the

first um rollup to kind of try this and and play play around with this experiment. But basically once the L2 transaction gets sent to the Tao public mempool now you have a a slightly different set of actors who are searchers on the L2 who are looking at this transaction and trying to sequence it right so there might be me they can capture there might be transaction fees that get paid to them for including this L2 transaction in an L1 blob but that's kind of how how the process is playing out and in particular those searchers are sending blobs with the L2 transactions sequenced to the L1 uh builder or proposer And from there the flow is the exact same but now this confirmation is the first confirmation that the user has received. Right? So they didn't they didn't get that green check mark from

the centralized sequencer. The only time they have any confirmation about the L2 transaction being sequenced and being executed is when they see it in an L1 block. So this is a very different paradigm and it leads to what I call urgency or like impatience for the blobs because now there's a competition. everyone's racing to to be the first person to land a blob on chain and this kind of results in some sort of me in the L2 space playing out on on the blob uh market. So kind of to summarize these two discussions, there's kind of two different paradigms for which blobs can be posted. There's kind of the the public patient blobs that are happy to sit in the public mem pool and then

there's these private blobs. I guess, oh, one other point I wanted to make on this slide is that the searchers in the the number two label there are sending their blobs directly to the Ethereum builder and proposer. They're not going to the public meool. And the reason for that is because they're worried about someone else listening to the memp pool and potentially kind of like rugging them, going in front of them um or unbundling some of the transactions that they sequenced. So, um this this uh plot here shows the percentage of private versus public order flow. I guess it it just shows private and the it's a little hard to see but on um the blue shows the percentage of blob carrying transactions that are private order flow and we see it's about 20%. And the orange shows the share of blobs that are going through

private channels and that's about 7%. So I guess the takeaway here is that actually most blobs go through the public memp pool and I think the the explanation that makes the most sense to me is that the blobs are patient generally and coming from these centralized sequencers and thus don't need the type of guarantees that going through private channels would provide to them. And I guess this is this is important to understand because when we think about designing the protocol to scale blobs, it's it's useful to think about how these blobs are actually flowing through the network. Um and that helps inform the the protocol design. So the the TLDDR here is we expect patient blobs to be content using the public mem pool as long as it provides a relatively good inclusion service. And relatively good is kind of intentionally vague, but it might mean that if I post

a blob to the public meool, it gets included within like two or three Ethereum blocks. And for most roll-ups who are doing like this batching process, they get a bunch of L2 transactions, they construct the blob, they send it to the blob pool, and it gets included. that's like probably enough time um or like a a sufficiently good service for them versus someone who's actually like competing and needs it landed that slot um the public blob pool probably will not offer that that service for them. And I guess to kind of not bury the lead here, I believe that blobs distributed through the public memool are probably better for both builders and for low resource attesters. So, I'll kind of try and justify that claim, but the the takeaway here is that I think we should strive to maintain the public memool for

blobs to be the the dominant way that they um go through the like basically land on the L1. Cool. So, now we're going to talk about that actual process. So, there's kind of two ways that blobs get there's two kind of roles that that validators need to perform to validate blobs. The first um note here is that Ethereum nodes run both execution and consensus layer clients. And actually both of these clients are connecting to different peer-to-peer networks. So there's different peers, this is a different way you can source blobs either from the consensus side or from the execution side. And the blob mempool, which I sometimes refer to as the blob pool, is an execution layer object. And just like the normal mempool, it maintains a set

of pending but not yet included transactions. Right? Of course, these are blob transactions. So the way it handles them is um slightly different because they're 128 kilobytes, but generally speaking, it's just this set that that everyone maintains in their local nodes view. The block validation and a testing flow is a consensus layer operation and it checks for availability of the blob data. Right? So the the way this works today is there can be like let's say there's six blobs included in an Ethereum block. Um, in order to check that the block is valid, you also have to download those six full blobs and confirm that they're available and and have fee paying transactions behind them in order to confirm that the whole block is valid. So, a testers who are voting on the block and deciding which um which

fork they view as the head of the chain will kind of need to download all that data in order to uh make that decision. Right? So, how does this play out kind of in time? So I have like this two slot diagram here to to try and help visualize. And we have four public blobs which are these these green uh blobs on the left. And then the two private blobs are the purple on the right. And the important thing to note here is that since the public blobs aren't depending on kind of being released just in time as the builder publishes the block, they're able to kind of be spread out and amvertised over the entire previous slot. And once the block actually gets published, any blob any blobs that haven't been seen seen in the me pool

need to get gossiped before the add astation deadline. So if I'm in a tester and that I see a this this red block and it has all six of these blobs. If I've seen the four public blobs beforehand, all I need to download before the addestation deadline is the two private blobs that were referenced by the block. And the important note here is that blobpool blobs can be used for block validation. Right? So there's this kind of coupling between the two where if the blob pool is providing me a good view of the set of blobs that were being gossiped around, I can use those to validate blocks on the consensus layer. So it's kind of like this nice synergy between the two.

And the reason for that is fundamentally because the DA check actually needs to download the full blob data, right? So there's kind of no getting around it. You have to see the entire contents of the block before attesting to it. And that's why the the two are working together well now. But as might be kind of clear from the direction I'm going here, this doesn't actually stay the same with Purdas. And let's just like remind ourselves what Purdas is. Purdas is slated to be the headliner EIP of the Fusaka hard fork. And it's aiming to massively scale the blob count by reducing the DA check to a subset of the columns of each blob. So right now blobs are these kind of commitments to arbitrary uh or they're commitments and and they come in these terms of these uh 128 kilobyte kind of data format and

postpass you kind of break up this blob vertically and only need to check that a subset of the columns for each blob are available to confirm the block. So how does that look? um EIP4844 downloads kind of all of the green space there um kind of represented by this diagram but on the right PDAS says okay we have many many more blobs but you only take a slice of the blob and that's what you have to download to check the availability and the reason this works is um basically everyone gets assigned different columns and if everyone attests that their columns are available together as a as a whole the network confirms the availability of the full set of data so this is what puras is aiming to do and now let's look back at our slot diagram

and see okay what's what's actually playing out here. So now in the in the first slot in slot n we have these public blobs but now there's many many many more of them right and because we increase the number of blobs that that happen per slot all of these blobs could get gossiped through the public meool but when the slot actually begins and the block is published the purple represents the private blobs from before but now we only need like small columns of each of the private blobs that were downloaded uh to confirm that the block is valid. So the key observation here is that the types now mismatch, right? So on the left we have public blobs going through the public meool and that's the entire 128 kilobyte object and on the right it's just a column and this is kind of what leads to this asymmetry

between what the execution layer is providing and what the consensus layer actually needs to download to a test to a block. And basically the the summary here is without changing anything the blob bandwidth increases by 8x while the DA check remains constant in terms of bandwidth consumption. So, we kind of threw the baby out with the bathwater little bit. We said, "Okay, we're going to only subsample parts of these blobs on the on the consensus layer, but on the execution layer, if you want to stay connected to the memp pool, you're going to have to download the full blob data." Um, so this is kind of where that that core tensions arising from. And yeah, I guess now we can zoom out a bit and say, okay, what do we actually want to do anything to address this asymmetry? And the first point is that no protocol

enshrine changes will imple be implemented for the blob in Fusaka, right? Fusaka is already set. The scope of the hard fork is committed to and it's not going to change this asymmetry. So this asymmetry will exist at least for one hard fork um kind of no matter what. The second fact is that the number of validators self-building so constructing their own blocks is so low that the blob inclusion service might not significantly degrade. So what I'm saying here is that like maybe that asymmetry doesn't matter if we trust that the 95% of blocks being built by builders can contain up to the 48 uh blocks then for roll-ups who are happy sending their blobs through the public blob pool this service remains sufficiently good.

Um, second or I guess thirdly, there is not an cool five minutes. Um, there's not an obvious reason we will shift to a private blob only regime and that's exactly for the reason I started at the beginning with which is centrally sequenced blobs are happy to go through the public blob pool. Like there's no economic forces pushing them to private channels because there's no me available in these blobs. And that leads to claim one which is that we should probably keep the blob pool around to amvertise the blob bandwidth uh consumption for low resource testers. So that's that's my first claim and then the second claim is solving the fundamental asymmetry of blob validation and gossip is important to continue the scaling road map right so pure may be fine like the blob pool will only consume maybe 10x the

resources that it consumes now but once we go to 2D sampling it gets another order of magnitude increase and it just like it doesn't make sense to to say oh the ex uh the execution layer is continuing to scale linearly in the number of blobs but the consensus layer is only scaling kind of um sublinearly like that that feels uh directly at odds to me. So, right, one straw man proposal is what what can we do? What if we just shard the blobs that we sent to the blob pool? Right, we're saying we vertically shard the consensus layer DA check. Why can't we just shard the gossip itself?

And the reason we can't do that is because this provides a free DOSS vector on the blob pool. And the the reason for this is previously the memp pool can check that any transaction that gets gossiped is fee paying and um comes from like you know is is a valid transaction basically. So in some sense the people who are sending those transactions are paying some economic weight behind the the transaction that they're sending and thus they they can't just like spam arbitrary bytes into the into the memp pool. But if you just subsampled a blob into columns and gossiped those columns directly, there's no way to like formally prove that that column belongs to a fee paying transaction on the blob.

So that's one problem. And the kind of key thing here is that we need to ensure that columns that are gossiped pay for the service that they consume or the resources that they consume like that bandwidth has to be paid for by someone. So what is one idea for what we could do moving forward? Um I uh I like to call this the blob pool ticket auction with refunds which is kind of a mouthful but the idea is okay we have this scarce resource we want to allocate why don't we run an auction right and we can auction off not blobs themselves but access to the blob meool and you could simply do this kind of in protocol with with something like this where you a few slots before have bids submitted as L1 transactions and the bids are processed and um kind of consumed that way. But

what this allows us to do is say when a blob gets gossiped, we only verify that it comes along with a winning ticket from that auction. And now we know that whoever has uh oh sorry um whoever uh does the gossip itself has like already paid for that access. And importantly, we also include this refund thing because we want write access to the blob pool to be free in expectation because right now it's free, right? And um if if the blob itself also lands on chain, then it did end up paying a fee to the protocol. So we can probably refund the access that they paid for um to the memp pool. So again, this is this is a little complicated. This is just one design.

There's like a ton of kind of optionality here, but I think it's it's worth considering at least a little bit. Um I'm going to skip this slide just um in for the sake of time, but I'll post the slides later and um happy to to discuss more after. But I think there's a few questions that are worth highlighting just for the key stakeholders or the constituents of blobs and and this whole blob pipeline. So there's questions for rollup teams like how do you think about public versus private blobs? How much latency do you um are you okay with for landing blobs on chain? Um and would you participate in the blob pool auction?

Then you have the builders who are actually the ones creating the L1 blocks and we could ask them how much do you depend on the public blob pool when um deciding which blobs to include and what guarantees do you offer to private blobs? And then for staking operators and people actually doing their testing to blobs, we can ask how they plan to handle the bandwidth increase resulting from larger number of blobs and would they continue downloading blobs in either of the um sharding worlds that we discussed. So um I'll wrap here. These two links at the bottom are are articles that kind of should help fill in some of the details from this talk. But thank you very much for your time and attention.

Sure. Yeah. If there's any questions, happy to seems like no. Cool. All right. Thanks, everyone.