DeFi by Design EP #123: Data Availability in The Modular Stack Explained

Transcript

welcome back to the rollup High Roller Community as always nothing on the rollup is to be construed as Financial advice we are just an Educators in the defi space toss us a quick like give us a share a repost send it to a friend who may want to watch this and finally give us a sub or a follow it means the world to help support credibly neutral educational content in def5 we're here to educate Empower and enrich you and let's jump in welcome to D5 by Design we talk all things blockchain and cryptocurrency while striving to educate Empower and enrich what is going on guys welcome to defi by Design today we've got very special guests we've got Nick white from

Celestia Chief modular uh Celestia uh leading the modular wave um and uh we are fully embracing the the concept and philosophy of modularity um nick man I mean I'm not going to do too much of a preamble I'm just GNA hand it over to you and I I think like where we'd like to start is modularity in general what does that mean to you well first thank you Robbie and Edy for having me on I've seen you guys uh very much embrac the modular uh community and Vibe and memes and uh you know it Wars my heart and so I'm I'm soaked to be here and get talk to you guys about all this stuff

so modularity is this idea I it's something that's you know a much bigger idea than just like the crypto space right it's kind of like applied across various different Technologies but generally the concept is that instead of trying to do everything all in one package you can unbundle things and decompose them into specific components and then those components can be designed to you know specializ in one specific function and then those pieces can then be recombined to achieve the same like overall results so you know you can think of it sort of like Lego pieces or like different parts of a of a

car you know that all P fit together to make the complete package but each one of them is like a separate you know specialized component that one you know team or company can you know specialize in making that the best as possible and then you can also because things are split apart and able recombine you could get all this flexibility of different combinations of different components and so when we apply that to the blockchain space you know historically blockchains were built in a monolithic architecture meaning that like you know every piece is glued together fused together like welded right and so the blockchain is just this very rigid uh thing and uh kind of limits what people can do on

that watch shade if you're a developer user and it ALS also you kind of binds you to certain fundamental constraints around scalability or verifiability in most cases and so modularity blockchain it designed is about hey well actually let's like split those apart rather than having this one rigid monolitic you know thing we're going to split it into different layers and different protocols and those protocols again specialize in these different functions so um that is what modularity is all about it's about a new paradigm for watching design and it's going to result just like I said in things like a lot more flexibility and expressivity for for developers I think it's going to be uh something that um

matures the space and gets us where we need to be to actually developer like sort of like develop and deliver compelling applications right like everyone's always talking about well where are the users where are the apps I think uh and everyone's like oh ever's overly obsessed with with like infrastructure why are there so many l1s um and rollups and things like that and and I think the reason is that we've been held back by the infrastructure we've been building on and the modular movement kind of promises to to sort of like bring us up to a level of maturity where we're ready to go for the big leagues it it's very cool to see the the blockchain space go through this transition this like this transformation

from monolithic to modular like the you mentioned car s which is like kind of my when I was like just learning just in general like just going through my schooling there was this like case study on interchangeable parts from Henry Ford and my understanding is that this led to the Industrial Revolution in in America and that by by allowing interchangeable parts then each piece of the manufacturing process was able to specialize on the one part that that it was building and it was It was supposed to to focus on um there's another philosophy that that we we talk about sometimes which is divide and conquer and I think this is also applicable to

modularity because once we're able to divide we're able to conquer and specialize on these individual pieces and as long as those pieces are composable and you can kind of like weld them back together then you kind of have like this this more specialized stack at each level that also accomplishes like the the goal of the oall ho and and this is what I see as like modularity continues to expand is like these specialized interchangeable parts that projects and groups are able to incorporate into their own blockchains for an overall more mature stack and like we hadn't seen that in blockchain

like other Industries had gone through that that maturation process how did how did you and the Celestia team find this this movement like how did how did you get this idea to to install interchangeable parts into blockchains so the idea really goes back to uh 2018 and 19 so um and it involves um one of our co-founders Mustafa alasan he wrote he co-authored a paper with metallic that described this new cryptographic primitive called Data availability sampling which kind of solves one of the core scaling

bottlenecks of of blockchains and we can like you know talk about that later but as a as sort of like a followup to that uh initial idea he wrote another paper in 2019 called lazy Ledger and in that paper he kind of he had this Insight which is that you know we assume right all like blockchains from you know Bitcoin to ethereum to salana every B chain that existed up until that point it was assumed that every time there's a new new block of transactions added to the to The Ledger added to the chain that all of the nodes of the network would actually have to uh not just sort of like vote on consensus of like adding

the block but also would have to verify all the transactions before accepting that that block should be added so what that meant was there was this bundling this coupling of consensus which is again how you how you how do you decide what data gets added to the Ledger which transactions gets added and then execution which is how do you decide which transactions are valid and how do you know what the the state of the chain is and in every blockchain he like those two things happen at the same time and were performed this functions were performed by the same set of nodes and then and he he had this Insight he's like actually there's no reason why that has to be the case it was just sort of like you know Satoshi

did it that way and then never just kind of took that for granted and carried on that way but when you go back to the basics and the fundamentals you realize that you don't need to do that you can actually just have a chain that is lazy meaning that it just adds raw data and it doesn't look at the data at all it just comes to consensus meaning it orders that data and adds it to the chain and then other people can read that data and run their own execution on top of it and this is actually before the idea of rollups existed really and there was ideas about l2s like plasma and things like that but um so the core idea was essentially like just make build a blockchain that

doesn't do any execution and uh it's such a counterintuitive idea right everyone else at the same time was trying to pursue bing bing architectures that actually made things more complicated like sharding or even like salana with all these like really intense optimizations around execution and block propagation all these things um so rather than going more comp complex musaa went the other way he went back to the uh first principles and it was like actually I'm going to solve all these problems by actually just like going back down and and start like creating a new path that no one's ever thought of before and simplifying simplifying and Distilling what B's

early about and so um that's really where where the idea came from and then in parallel what one of our other co-founders um John Adler was was working on layer to scaling Solutions and he came up with this construction of the optimistic rollup and those two ideas ended up being perfect compliments for each other basically um where you know the like a lazy Ledger like Celestia is all about only consensus and data availability no execution and a rollup is all about only execu ution no consensus and data availability will inherit that from another chain and so like each one was kind of incomplete without the other and when those two kind of merged it was like oh wow we

have now this new paradigm for building blockchains where we have separated consensus and execution to separate protocols and then that idea has now and so that was like the birth of this notion of modularity there was a blog post in 2019 that Mustafa wrote when he introduced the project which was called leder at the time and Al Celestia where he said like our vision is a modular you know blockchain ecosystem with you know consensus and data layers and execution engines and that was sort of the the birth of this this notion of modularity now now it's really extended even Beyond just you know consensus data availability execution there's things like settlement there's things like proving layers shared sequencers and all

these different modular components that have emerged kind of in reaction and the extension of this original vision and really you know the way that I think about it is that modular boads are sort of like protocols as a service it's like I'm going to build a protocol that actually um provides a functionality that other protocols can tap into you know like people are always talking about you know software as a service is a major movement in like web 2 and I think protocols as a service or like modular blockchains are going to going to provide a similar kind of re revolution in in the web 3 space very interesting and there's some there's some funny memes about Mustafa

which were in too much about but that I've been seeing circulating and I think there's a this this Aura boy genius in a sense an Untouchable V uh it brings his Aura of of of um of of higher power to celesa just a quick Insight from what the outside looking in brings which is which is funny to kind of have on a on on blockchain and on a crypto protocol me with the with the Des end choice that Celestia has made um you know being a being a d protocol we talk bit more about why we think data availability may not be the

best but for now go with that you know you GL shows to be a cosmos chain at least build on a COS SDK not necessarily affiliate yourselves too much with the ca ecosystem but keep the properties of of the ca SDK but that's not necessarily a requirement to do data availability for the modular sta so and's kind of um about that design choice and kind of what went up to that um you know and how that allows Celestia to be a better data good question so um you're right Celestia is built using the cosmos SDK and tender it or

now what people call Comet bft and on the reason that that choice was made was first of all um Cosmos uh kind of was the originator of this multi-chain vision for the the future of of the blockchain industry and like you know it's you people take that for granted now but you know if you rewind back to 2017 18 even up until like 20120 maybe people kind of assumed that it would just be like this world computer world that we live in you know and ethereum would scale and you know there there's no reason to have any other Chains It's just like all going to be on one chain and some people still cling on to that Vision like salana they

they really believe in this like one world computer thing U but Cosmos you sort of were they Originators of this idea of like no there there there will be uh it will be like a multi-chain future and there's a lot of different reasons for that there's sovereignty you know you want to like be able to control your own State like you don't want to be beholden to the social consensus of some other community and also just like application specific blockchains and be able to customize your chain for your use case and so first of all like Celestia is very much aligned with that Vision because one of the things that Celestia enables is the goals was to enable anyone to launch their own blockchain extremely easily and be able

to customize it and be as Sovereign as possible so anyway there's there was always like this deep sort of a philosophical alignment and and alignment of Visions like you know cosos talks about the um what's there it's it's the the internet of BL chains right and I I like to say what celes is kind of about like what part of the vision is the internet of modular blet so we're kind of like extending the cosmos vision and hopefully making it even more more possible but the reason that we chose to build specifically using the cosmos technology stack is that first of all Let's see we had to start over it's a it's a net new chain

no one had ever built you know a data availability layer with data availability sing before and so uh we have very specific needs it is sort of like an application specific BL shade in a sense uh where the application is data availability um not necessarily because data is like a very generic thing it's not really it's not like an app like a DEX or it doesn't have any logic but the what I've getting at is that it made sense to use a software stack that was first of all a battle tested Cosmos you know uh tender as a consensus engine and module what is still to this day the most battle tested widely used consensus

engine out there and so it's like why are we inent that we all just to use something that works and um you know tap into this broader community and you get IBC out of the box so we get interoperability and all the like wonderful things you get with Cosmos and um then you know the cosmos SDK is nice because we as a you know D data availability chain only needed specific functionality and a very minimal State machine and so it CMOS us allowed us to customize that and just only use the things we we need it they have this interface called ABCI Plus+ which allowed us to um basically we had some kind of interactions between what's happening in the tender layer and the estate machine so having this like

really good interface there allowed us to customize things and and um build our our build Celestia in a really like uh nice way essentially that wouldn't you know you know would have been really hard on you know using other Stacks so basically Cosmos has the best tech out there and um but you also get a shitload of things for free aside from IBC you get wallets for free you get explorers for free you have all these validators out there who already know how to run tendermint noes and so it's kind of a no-brainer to me like if you if you are building your own blockchain from scratch and you have to which Celestia it has to be his own blockchain then to the tenderman stack the cosmo stack is

is like the best thing out there now in the future one of the the things that celes is built for is to to make it so people don't actually have to build their own blockchain right they can launch their own chain without bootstrapping a new consensus Network and validators and all that so um you know but hopefully people will continue to use the cosmos SDK that's why we've done roll kit which we can talk about but it's a it's a sort of a rollup framework that uh is a sort of replacement for tender with um that allows you to build was SDK um rollups yeah very very very cool so so just to like make sure that we have like the entire like Topography of Celestia

like kind of on table gear um you've got you've got the you you you kind of laid out like these these three parts of the blockchain you got the consensus part you got the data part which has multi like there's all there's all these other parts that are embedded in each of these sections data has data availability storage uh broadcasting all within the data layer and then you've got the execution layer on top uh which is where like the rollups uh are are hosting these applications so quick quick question like does celestia it's it has it done blockchain but does it have its own validator set

or it doesn't have its own validators because you guys use 10 like how does how does that work as far as how consensus is derived yeah so what tenderman is is just like the software it's a consensus protocol and um so you still need a set of validators to run that protocol and and come to consensus you still have to issue your own proof of stake token which you know is is is the thing that allows you to actually have this silver resistance to come to consensus and things like that and um so yeah like celesia is very much its own blockchain with its own validator set um and its

own proof of State token uh which is Tia and um and so like tenderman what it what it is is just that people whole Community have written software that allows you to have a very secure consensus engine essentially and so rather than having if everyone had to write their own consensus algorithm from scratch then uh there'd be bugs everywhere and uh things would be really ugly and also you wouldn't have these sort of Standards like IBC where it's really easy for any any tender based chain to bridge to another one because they have this kind of common language of IBC that they can communicate through yeah so everyone would have to everyone

would be Satoshi if they all had to write their own uh consensus protocol um yeah we stop and as much as I want to ask you about all the Future air jobs T stakers I will just simply refrain and stay in the tech weeds um getting Greatful thank you yeah right getting to the uh the nuts and the bolts of um of kind of what uh Celestia enables for uh for blockchain is that a lot of this moding a idea is around solving the original trilemma of model the chains um could you talk us through kind of how you see

uh Celestia's light nodes or or like clients playing a big part in kind of solving that scalability and cost TR wups or just for blockchains in general yeah great question so people may have seen on Twitter or elsewhere uh you know people reading light notes and tweeting about it and um you know it's kind of a meme going at one point about like running a light note on a refrigerator or you know all these like different devices and actually people were doing that people ran them on you know Kindles and uh Nintendo switches and uh old computers or uh you know TVs and things like that and what that me

was about was basically this new node type that Celestia has enabled uh that has never existed before and it which is a light node right it's kind of a port Monto of this of these two other node types that every chain not every chain but a lot of chains have which is you have a full node right which is basically a a node that um a lot of people think that that has that means it's a validator no like a validator is like a consensus node a full node doesn't participate in consensus but it ver fully verifies the chain so it downloads every transaction or re-executes them makes sure that they're all legit and and so when you're running a full node you are like fully verifying the chat and so you secure as you could possibly be you're not trusting the

validators that they're you know that they're being honest you if something goes wrong you're going to see it and you don't have to follow the chain anymore so it's a full node and then on the other hand in in monolithic watch Chads you have a light client and a light client is something that is very light you know you could run it on on a phone theoretically or your laptop like any user should be able to run it very cheaply but it takes a shortcut which is that it just trusts the validators of the chain or the monitors of the chain to be honest so the you you you're kind of like in a monolithic or like the old style of building blockchains you're stuck in this world where you either pay a lot and have to

and and run a run a full node and have the maximum security and least trust assumptions or you have to sacrifice to security uh you know if you're a user and you can't afford running running a full node and but then you're making these trust assumptions right so then you're kind of defeating the whole purpose of laes to begin with because laes are all about having these verifiable systems that you don't have to trust other people for so what's so cool about Celestia is that there's this new not type which is kind of the combination of The Best of Both Worlds and so it's a light node so it's a and you get the security of a full node basically with the uh node requirements the resource requirements of a like

client and so this is enabled by this enables all the users all over the world to run a um run a do directly on their phone and directly verify Celestia and so they don't have to trust the validators of Celestia to be honest they can know for themselves what the ground truth is and this is enabled by this new technology that I talked about earlier which is data availability sapl so the reason why you can't run have a light node on a normal chain is that if you wanted to verify the chain you used to have to download all that data right that's what a full Lo does it has to download every transaction reexecute them and um what data availability samping does is it

makes it so that instead of having to download every single transaction or all the data in in a given block you can just sample from that block randomly so you only have to download less than 1% of the overall data but you get a Almost 100% guarantee that the block is valid so it's sort of like a probabilistic method it's very similar to how actually how ZK proofs work for example so you were able to directly verify the Chain by doing a significant you know orders of magnitude less work and the other magical property of it is that uh as more light nodes start to run and verify the chain the bigger the Celestia block size can grow while still remaining

trust minimized so Celestia that's why was kind of I don't know if I mentioned this earlier but one of the unlocks of Celestia is this uh scaling property where the more users there are in the network the more people running these light nodes the bigger the blocks can be so you don't end up you don't have a fixed block size you don't have this fixed capacity where you know in ethereum or other monolithic chains there's a maximum block size you can't go beyond that right the only way to go beyond that is to increase the node requirements for everyone of the network and that again leads to centralization whereas in Celestia there is no fixed block size because as more and more people run nodes you can increase that block size uh elastically and so um this is this is

really cool because it's the first time where we can really have abundant block space for everyone to send their transactions transactions can be cheap uh you know in perpetuity in in a system that's designed this way and has this this kind of property and again every user can now be a first class citizen of the network they can verify things directly themselves and not have to trust anyone else they don't have to use an RPC which um is essentially how most people interact with blockchains today is there's an intermediary the blockchains are meant to be peer-to-peer networks where you know I send my transaction directly to the rest of the nodes but actually what's happening in today's uh infrastructure is you know I send my transaction through metamask and

it goes to some other centralized RPC provider before going to the real Network so it's kind of like this layer between the users and the real the real thing and the the reality the truth if I want to the state of the B is I can't verify that myself I ask the RPC to tell me what it is so uh light nodes are really I think an evolution in the right direction for for the space in so many different ways in terms of scalability as well as in terms of decentralization when one gigabyte blocks sir that's a good question well um we are currently so uh for those who don't know we launched uh back in October end of October uh with two megabyte blocks

with the ability to increase the size the box size up to eight megabytes via onchain governance um because that's sort of what the protocols currently expect to handle um but um we have the ultimate goal of eventually meeting and hopefully even exceeding uh gigabyte size blocks um now there's going to be a lot of you know you can't just get there overnight and um aside from just the the bottom neck of needing enough light nodes in the network to secure that size of block there's also just you know straight up engineering constraints but we have a lot of really interesting exciting uh new approaches on the way and um I think we yeah we'll be able to

scale block size pretty significantly um in the near future um and so but but I don't know gigabyte specifically I don't know but what you know maybe I think 64 64 megabyte blocks or or you know I I don't want to necessarily pin down the number but I feel like those sort of smaller nearer nearer term goals are very achievable I think um but yeah thanks for asking yeah it it it it's uh it's dependent based on what you're seeing on the amount of of uh people running light mod so the in order to to get to those those level of storage in Block Street we would technically need a

a increasing lot of usage of the diffusers light notes and just like blockchains using celesti in that kind of how it would scale like I guess aside from just like when but like what is the4 maybe blocks is it just a matter of scaling amount of users and light noes or a demand question it's the so there's a lot of different moving components and I don't want to like Alpha leak anything and I think there will be a blog post that we publish I don't know in the coming couple months or something like that that we'll sort of like describe a more detailed road map of of power scaling

log size but um there's a few different qus one obviously like I said number of light nodes you need to have enough light nodes to actually securely sample that size of block right I don't I think that I think we're going to be pretty covered on that um that the the other bottlenecks are more on the engineering side so there's two two components one is just the block production side like can the block producing nodes the consensus nodes actually build blocks and like vote on them and gossip all the transactions and things like that uh as blocks get bigger and uh there are certain bottlenecks that need to be fixed in order for that to uh sort of be achievable the nice thing though about

Celestia is that we because we have light notes we can increase the validator requirements without causing centralization so like you know Salon a solution to scaling blockchains I mean side they do do some optimizations um but a lot of it is is also just increasing the node requirements for for validators right and so that's kind of like I wouldn't call that cheating but it is cheating if you don't have a lightning right because by increasing the the validator requirements you're you're just C centralizing the chain now in Celestia's case we can do what salana does but as but if people are sampling then we're not actually causing centralization so we can have bigger block producing nodes

without the side effect the negative side effect if you will so there's there's always that option but like um so aside from that um there's also the the the the bottom neck of making sure that sampling works intended as the number of light nodes increases and as the block size increases um so there there's also work to be done on the data availability sampling protocol side of things aside from this block production yeah yeah yeah the uh I I draw two tiles here first we we had a podcast with say Network they are more or less a monol lipic chain using a paralyzed evm um but um they're they're able to scale in a sense using

realization which is kind of interesting subject at the at the V level and then the other one is uh with with regards to samplings is um similar but different is DVT this distribute valida technology wherein you know you can own just a key to a a cluster of notes of full notes for ethereum which uh allows you to um you know be a small state still own part of full all these different things in in blockchain which are democratizing absess and decentralization or that for um I think Bush specifically on your Da Side on j s

side um what's it between single onedimensional G and two dimensional and is there is there Focus for celesy to or is that kind of for this yeah well first I want to say um paralyzation is is something that's super powerful at the execution layer and um you know really excited about you know obviously Salon virtual machine does that monad is building that I know say is big on that and interestingly enough one of the things that um is useful even for scaling a DA layer like SL is paraliz but at the Block production level and so there's this notion of internal sharding so it's kind

of because data vill sampling was invented to basically like uh make charting Obsolete and and it's a much more eligant solution um but we're kind of like coming back to charting but rather than charting the chain you're charting sort of the block production internal to each node that's something that I'm sure we discuss more on that that is a very cool concept uh ishill from lrange kind of opened my eyes and he called it embedded rollups like because it's kind of like internal sharding it is it's yeah it's a nice concept yeah I don't know I I wonder if he was uh sort of like talking about the

same the same thing as sort of what what I referring to probably um anyway what were we oh yeah you were talking about DVT and things like that and I I think DVT um is interesting in terms of like democratizing access to people running validators but even something that's interesting about Celestia's vision for like the future uh or how you know what's what what what are we optimizing for where we're building these systems is that you know the valid a validator set doesn't really have to be like focusing on decentralizing the validator set or making like having like let's say 10,000 validator nodes is actually kind of a uh a mistake in the sense of that

there's there's bigger fish to fry more important things to prioritize such as end user verification right or like people be able to verify the the chain directly and so as long as you have light noes you have end user verification then you're not really trusting the validator set for very much you're trusting that they're uh you know going to come to consensus uh you know and they're not going to Fork the chain but the thing is you you have if you're a proof of stake you have uh you know like I like forking the chain is is a attributable fault so you can slash them always and that's really the I think the best you can do in terms of consensus and then other thing you're trusting them for is censorship resistance right um and that's a big that's a big deal right you do want these systems to be censorship

resistant but how many like uh how many validators do you need to ensure censorship resistance and really the the reality is it's not about the number of validators it's about the distribution of stake in general like so uh like you know you can have 10,000 validators but if like the stake is centralized among like the top you know 30 or something like that which is something that you see across almost every chain out there um it doesn't really matter so anyway I just wanted to like uh note that and really so really the the the answer is about end user verifiability so you don't actually rely on the validators very much now your question what was what was the question is as regards to

the differences and we don't have think to technical here but I think it's it's interesting because there's going to be more and more da uh protocols coming up um that are going to be talking about this which is a onedimensional data availability simpling and two-dimensional oh right simpling the differences in what's powerful and kind of how you how you're looking at that and even just what it means for our audience to go too deep absolutely yeah okay so let's let's talk about data availability samply let's get a little bit more in into the weeds here I'll try to explain it as simply as possible because it's really not that complicated um so okay so here this this is the problem right you have this data right that's all this transaction data that's

in a block and now you want to be able to verify that all that data is actually there right but you don't want to have to download all that data yourself because if let's say that let's say the block is a gigabyte right if you have to if if I had to download a gigabyte of data on my phone every 12 seconds and every time there's a new Celestia block that's never going to work right so we need to have a W for me to download like I don't know maybe like 10 kilobytes or 100 kilobytes of data and but verified have a have a statistical guarantee that all the data that that validator that block producer tells me is in that block was actually published I want to know

that the data was published so that's so other people uh could download and see that data if they wanted to right that's that's the problem statement of of data availability um and so know naively you know you would think like there's no solution was just like well I can only the only way for me to know that all the data is there and is actually available and published is for me to look at every single you know bite of data and verify that it's there myself right and that's what that's what every monolithic existing blockchain does they download the full blog right now the cool thing was that Mustafa and vitalic and this guy Alberto sonino wrote this paper

where they introduced this concept of data availability sandline so the and it solves this problem it's kind of mind-blowing it's pretty magical so what what you do is you take the original data let's say that one gigabyte of data and you use this technology uh called eraser coding in eraser coding is used in things like CD ROMs it's used in like streaming it's used in like every web web connection you know when you're when you're when you're sending packets like a lot of gets dropped and so what you and and like a CD ROM gets scratched all these these things uh you know it's very easy to lose data right and so what ratio coding does is it adds redundancy to the data such that even if you lose parts of it you can always recover the original data so in you know CD ROMs and

in in like in you know internet connection um you're you're losing the data and this block production so of like blockchain world it's that the the validator might be trying to hide data from you right so it's like a like a sort of malicious intent and but but eraser coding does the same thing which is that if the validator tries to hide certain data from you you can always still recover the original thing as long if he wants to hide the original data he has to hide a huge amount of the overall data that includes the Eraser coded portion so you take the original data you add this redundancy which is the Eraser coded portion and

then you sample from the eras you sample randomly from the Eraser coded uh sorry the the the extended data it's called which is the original plus the Eraser coded and what you know is that if he's going to try to hide the original data he has to hide let's say uh 50% of the overall data so every time I I sample randomly if I get a actual response that's valid like he shows me I'm like hey show me the data behind you know uh you know index 1,000 show me the data behind index 9,988 or something like that you and you just do that randomly every time he

actually comes back to you and has a has a real like real data to show you then you have a increasing confidence that the whole block data is available because every time you have a 50% chance of Landing somewhere that he's trying to hide if that makes sense it's kind of like flipping a coin where every time it lands up heads you feel like more and more confident that it's always going to land up heads does that make sense whereas as as soon as it lands Tails right then you know it's like okay well I just found I found an index of this block where you're not willing to tell me what's there so you're hiding data so now I'm not going to trust this I'm not

going to trust this block that makes sense so that's how data availability s works now what I described was sort of like the simplified version which kind of is in one dimension so or 1D now the problem is so with with um this scheme the validator could also do something malicious which is they could uh extend the data incorrectly so you have to in order for this in order to be able to recover the original data you have to the the validator has to have actually use the right eraser coding scheme to extend the data they could have he could have if he wanted to be malicious just made gibberish right so that even if you sampled enough and had enough data you

couldn't ever recover the original data to begin with and So to avoid that situation you have to have some kind of mechanism to alert the light nodes that hey this is a like a badly encoded block and so for that reason you have to uh and that mechanism uh is basically if if it's all one-dimensional that fraud proof mechanism essentially requires all the light notes to download the entire block because that's the only way to verify that the encoding was incorrect or like a significant amount of it so instead what you do is you you take the original data and you chunk it up and and you make rows and columns and you

extend all those in a two-dimensional array and and then basically you saball over that two-dimensional array and now what happens is if there's bad if there's a if there's bad encoding um you can just uh the the fraud proof size is a square root of the overall data so it makes the fraud proof much smaller so it's very important for like the scalability of system hopefully that made sense but feel free to ask you that question it it does make sense it me it means that this 2D uh uh uh data availability sampling process is able to sample uh well it's able to verify more data by sampling less like you only have

to pick out you can pick out fewer points but still understand the validity of the entire block um I want to I want to make sure that I understand like kind of what what's happening here like great explanation by the way so like What's Happening Here is that the light nodes are sampling the validators right and their request in these indexes like um you know let's say all right validator show me index number 69 all right valid like show me index number 420 okay valid and like we can we can sample a statistically very this really random samp plan there yeah very very

rek proven sampl numbers right there been doing this exercise on like a panel at IST stul and I raised my he was like give me a number I raised my hand I was like P he's like no got to be an integer so I was like all right let me pick my numbers more carefully um and is it is it like one light node like if I'm running a light node am I picking multiple indexes or is it like your light node picks 420 like Andy's you know picks like a different index I pick a different index like is that is that what's happening where all like all of the light nodes are picking their own samples or or is each light

node picking like their own their own samples is it like a collective approach or is it like a each individual uh does the sampling good question so it's completely random and has to be for the security of the protocol so each light node there's no coordination it's not like okay I'm light node number one so I do samples one through 100 you do sample you light two you do you know 101 to 200 it's it's uh it's each one is totally independent of each other but it is collective in the sense of as long as there are enough light nodes sampling randomly then you you have a uh a threshold at which you know that a

minimum amount of the block has been sampled and because yeah you need what you need is that in the case where so like we can go really deep in this I actually give a talk uh and stumble at L2 days about different data availability schemes but one of the uh one of the attack vectors here we haven't talked about yet is that the uh the the the block producing node could actually let all the light noes sample but not let any not let any one node get enough data to reconstruct the original block so then what you need is you but so all the light bus have sampled it so they all trust it they

think it's available but then no one's able to actually take the data back together and get the original thing so these what these light not need to be able to do is actually talk to each other and and an honest full node needs to be able to gather the samples from every light node around there and reconstruct the orual block so the reason I bring this up is that what you need the reason why you need a minimum threshold of these light nodes for a given block size is that you need to have a statistical guarantee that there are enough samples out there to reconstruct the original block but yeah but you're not doing it in a way where oh each light node coordinates with each other they're still totally random I got a I got a weird mental model here and I'm curious if this is if this is like

uh reasonable to think to think about this in in this way so you've got validators who are running proof of stake they've all got some Capital at at stake and then they're coordinated consensus that way uh like and and the cost of corruption is the cost of this stake um then or Loosely there's like it's not exactly the amount of state but it's it's like based on the amount of state that validators have then you've got the light nodes who are gossiping like their samples to one another is it fair to think about this light node valid this light node set of this these light nodes as its own sort of consensus mechanism

yet it's not it's not stake or Capital at stake it's data at at stake is like is that yeah yeah there's there's there's there's an element there I think there's a thread of of Truth in the sense where um so leg nodes are and this is something that a lot of people don't understand about watch kads and how they work um a lot of people assume right it it's partly partly because of for example the n like people say validator right I think validator is a absolutely terrible name because it includes this word valid validate it implies that those

nodes are actually intended to verify the chain right but we just talked about how especially in Celestia for example like that's not necessary like that like you don't have to uh validators don't have to verify the chain they can just be dummy data right like in in cel's case and um what what validators really are are just consensus notes what they're really what they're really doing is just adding transactions to the chain adding data to the chain pending data and um and that's really their their function right now um in and and and and they don't validate you're not supposed to trust them that all the that whatever's

on the chain is is valid you are supposed to run a a full node or some other kind of node that allows you to verify it for yourself right because the whole point of bling so you we get out of these trust models where yeah maybe that like yeah trusting I rather trust as a decentralized set of of nodes than like one like Google right sure that's a step in the right direction but you're still there's still trust in the system those nodes could collude against you and L to you right so what blockchains are always all about is trust minimization and the way you minimize trust is by people verifying things directly themselves right so when I run a full node I'm not trusting the

validators um and so anyway the reason I mentioned that and because ccle back to light notes is that light nodes are the same in that like you are kind of voting right everyone who's participating in the chain does sort of vote on the blocks in the sense of like they're not voting which blocks get added right but they are voting in they're deciding every time there's a new block that gets proposed like the validator just proposed the block but no one at the network has to accept it right they have their own choice of they can inspect the block verify it and then say you know what this block Breaks the Rules or this block has unavailable data and they can

discard it and throw it away and that's really where the security of blockchains comes from is from people verifying it so in a sense yes they are they are fundamental to the security they are voting in a sense with their feet of like you know you try to send me this glal try to send me this block I'm not going to trust that go through yourselves you know what I mean so yeah it's all about social consensus that's what we we that's why we say you know social consensus is out of the a zero you know yeah Rob and I came to the conclusion with sloy the bear that bear chains Community is actually this applies to all especially let as you seen kind of the the memes the staking and the air drops

all that stuff kind of come to light is the community is almost like its own set of validators even if they're not staking or participating in the network they're almost their own nodes just by achieving that social consensus right amongst the know about the bread we image fing all these things and the means right very interesting how you can have this offchain or onchain social validation happening you you bring out a couple point that I I saw on a proposal on celeste's form from maybe a couple months ago which was that to try to figure out a way to go from the current nor of data availability to data publishing after this conversation it's almost like data

verification confirmation and Publishing the data is available but it's not it's not about that it's about verifying it and passing it on almost publishing it to R the main chain or to see a settlement letterer so like what are your thoughts on this are we too far down to change St availability or know is there is there reasoning to want to change it or kind of how do you think that people should think about this uh whether or not we can you know change the term is there is there a better is data publishing better way or how do you think about theology you know data um publishing is very much a better phrase the thing is

data availability has built up so much mind share and even you know people abbreviate it to Da and and so it's it will be hard it would be hard to replace it but I but I think ultimately data publishing uh DP will be potentially a better meme or not meme but like a more uh a much easier thing for people to grasp and the reason is that it just gets to the heart of what this is really all about which is I have some data that I want to publish and I wanted people to be able to verify that it was published without having to download all of it that's really all that this comes down to and data

availability uh availability is such a vague um word right and a lot of people it's so easy to misunderstand that um like you know what that means and so a lot of people oftentimes and think and this still happens this happened like for you know the first three years when I started talking about data availability everyone would be like oh well you know well there's you know filecoin is a data availability layer like why do you need Celestia and I i' be like well no it's not it's a data storage layer and they're like yeah but like you know the data is available it's there I push put it on uh you know you know um file coin and anyway can now download it right and and and so like there was this Con constant confusion

about what data availability meant and even these teams would be like hey we're a new data availability layer right and and we would have to be like hey you're using the word incorrectly even I remember like back in the day christic Chris dixen tweeted about like oh uh um what's that it's like an inferior like service I can't remember what it's called Uh but it's basically like an RPC type of or like a indexer and he was like oh this indexer is like Al he was like Alchemy is a great data availability layer and I was like dude that's not Alchemy has nothing to do with data availability just let you like just like uh see the data right just like yeah it allow it basically indexes the data so

like analyzes it so you can query and like you know look into what's happening on chain in an efficient way but it has nothing to do with publishing it's not even a protocol it's layer points to the data that's there right yeah exactly it's like a service an offchain service so anyway um so for that reason I do feel like data publishing would be a better phrase and and oftentimes that's that's kind of how I I'm trying to explain to people what data availability is I say like just think of it as data publishing because that's a you know okay here here's like a here's an analogy that I haven't really used before but I've been thinking about which is you can almost think of Celestia like a newspaper right um and I know the newspapers are like totally out

of date now and like like so you know uh don't don't read into that side of the analogy but uh I think I think it works which is like okay I what does a newspaper do right it's a it's a publisher and what happens is people come to the newspaper and they say hey I have this article right I have this advertisement and I want to publish it on on your newspaper Network basically and and and I'll pay you know this x amount of dollars it for you to publish it from me and the newspaper that's great and it it packages it takes all those articles all those advertisements bundles it into one package which is like right and then it sends out that block all over the world to everyone

who's subscribed right so like I subscrib to the New York Times okay I'm GNA get the New York Times in my mailbox every day right so every block time is sort of like you know a day in this analing right and so every I'm going to get the newspaper no way am I going to read the whole whole thing but like someone tells me oh by the way the rollup that I'm using they tell me by the way you know I updated the state up and like you don't worry you don't have to read the whole article but like just look in at attend this issue of the New York Times look and see that there was an article published and it's in there like you don't have to read the thing but just like look at the table of contents you'll see there's a little

like you know it says you know rollup B publishes this thing right and so all a I can verify it by the fact that I'm already subscribing to this like newspaper Network I get that kind of for free I can do that for as many rollups as there are participating in this in this um newspaper so that's it is the news is block right and each one of these articles advertisements is the data behind the specific rollup and you're like a reader like the reader is like the light note that is like s dark and it's like the like the uh 12-year-old kid who comes tosses the newspaper that's your mailboxing about about

voger extra extra extra oh little bit um come on um I think um something that we can pull ourselves out from the into the uh that that today and some of the uh the application of s about we SE notably Manel has um but they're claiming as the first modular rollup that Celestia for da or for data publishing should we say um think we're gonna start using data publish um and yeah I think that I heard any the founder of man on a video with box mining with regards

to so um I'm curious to understand first of all um how um significant using Celestia for da or other da layers is in reducing settlement costs and and data storage costs um you know for 100x 90x is 00x Dragon L is 500x so I'm really just trying to get to the bottom of like what us could expect as we push forward this or push this forward into 2024 I also why there's this one or two month like startup period or or like War up period where men slowly gets gradually cheaper and cheaper as Celestia gets fully ined and

like kind of what that is what's happen under the G why that is so know a quick primer on some of the uh reductions and cost and then yeah curious about the explanation or or the oneup period yeah absolutely I'm really glad you brought this up because now we're you know I think we went really deep on the the theory side of modular blockchains so far you know talking about the technology you know why is modularity powerful and what I'm so stoked about is that we launched Celestia last year right and now the network is live and so all these things that we used to have to talk about and the hypothetical are real and are

tangible right and so one of the things I'm most proud about is um that we have already had some of the first rollups migrate over from uh eum da to Celestia da so Manta was actually the very first rollup to take that leap and um it's been honestly uh incredible because they first of all the the transition was absolutely seamless so users and like there's no downtime the network just uh shifted over seamlessly and users were totally impacted you would never have known the only thing you would have known realized is like wow my gas feed just dropped by literally like 100x and so um it's it's it's pretty awesome and

and one of the and and it's made all these applications like there's this really cool application bu on meta called ZK Holm so I don't know how much people know about meta but just a quick plug on them and and I think they're representative of powers of the modular stack but basically Matta is it's an OP stack rollup so it's optimism it use has evm but it has a custom sort of evm implementation where they've added functionality to natively verify certain kinds of ZK proofs and so they have uh a sort of built-in sort of Randomness as well as like privacy features and this so this is like it's really cool because this is exemplary of what modularity is all about which is like you can customize the execution

model of your chain to enable the use cases and so that's like that's what manda's done and one of the new use cases they've enabled was this uh game uh called CK Holdem which is like an onchain uh poker type applic and they use the randomness right so you can actually have like you know verifiably random Game of Poker and the Privacy so you don't have to actually verify you don't have to reveal your cards to everyone else like if everything is on chain right but then you can just see what cards they have then it's not going to be very interesting poker gate so what's so cool about Manta is that they uh you know are enabling these kind of applications so anyway as ZK hold them before the transition to Celestia was too expensive to be feasible basically like no one was going to play this poker game if it's

like literally a dollar every time you want to make a move right uh you know what I'm saying so what's cool is the transition reduce those fees by 100x all of a sudden people can actually play CK and all these other applications were running out that so I don't know that just is very meaningful to me because it's it's tangible it's real it's like okay the technology we built is actually you know delivering Real Results and enabling new use cases and enabling more people to to use the chain and so um to to to to sort of wrap up on that there's a Tracker that Manta has also deployed which tracks their da fees and you know in in ethereum world there's this meme

of like watching the fee Burn Right which is like we're like uh I I was gonna say something a little bit uh not not appropriate but basically they are they're like they they circle around this thing and they're just like watching it and and just getting stoked about this idea of like Mor or fees getting burned and we've kind of uh with Manta and with you know other rups that are planing in WRA kind of flipped that on its head that narrative and instead of watching the feat burn we talk about watching the fees saved essentially like the anti-burn if you will and um in this case meta has a Tracker that tracks basically the Delta the difference between how much uh gas they would have paid on ethereum versus what they're

paying on Celestia and So currently they have already since they launched which was sometime about a month ago I think December 12th they have saved over a million dollars in gas fees so they had been running their rollup on ethereum they would have paid a million dollars by now and so um I think that's really why there's such an incentive for all these other teams to migrate is that like that million dollars is money that can stay in your users pockets it's money that you can also internalize via your sequencer and you can use that money to you know fund more development of the protocol you could put it into the Dow treasury

so that people can spend it and however they want and so that's that's something that's really um really amazing so people want to check that out it's met. social scan. da- fe- tracker or if you like just Google Manta data availability fee tracker something like that it should pop up um but yeah I'm pretty pretty excited about that lovely um is the madin network at full capacity with regards celest da is it ever up capacity like does it reach a point where it's like it can't steal past how does that how do that transition yeah I think that they're

pretty far from reaching so of the capacities the capacity for uh Celestia um Celestia da capacity think so far the total data based on like the uh sort of lock Explorer thing is they posted around 700 megabytes um of data so far and the the history of the chain and so if we were doing like two megabytes every 12 seconds that's like 10 megabytes every minute um and so you could have posted all that in basically like an hour it's only like an hour a little over an hour of Celestia BL space um over the last month or so so they're they're far from sat

however I know that they have had like people minting inscriptions and things like that um which you know I I'm sure you've seen this in other ecosystems but like I don't know I don't really get it but whenever this's the inscription mint there's like a race basically to Mint as many as you can you know in that that period of time when it gets released and so there have been these moments where there's been congestion because everyone's just like spamming you know trying to try to admit as many as possible but yeah I mean long story short celesia still has a lot more capacity to supply data availability to a lot more rollups uh and and also we can increase the loock size to 8 megabytes um as needed without any like further protocol

upgrades yeah lovely and um see as this pod will be released after these updates um what what is the what is the tldr on Avo Avo as well as frax how is Celestia going to uh intertwine there um anding better yeah well I don't know um how much I can say about that um other than you know Celestia for any project that is building or is using their own L2 or their own chain Cy has a very concrete uh value proposition which is

you can run your chain extremely cheaply with high security and verifiability and um I think that there's a lot of one chain that I can you talk about uh sort of in that context is is Lyra which is like an onchain uh sort like options uh exchange and um you know as you know options exchanges have are very data hungry right there's a lot of data that has to be posted on chain and frankly that if you're running on ethereum it's just going to be extremely expensive uh and so lra is migrating to Celestia and that's going to again just like mt to reduce their their costs by like 99% or more and also I actually

mentioned by the way um because we didn't talk about it earlier we have this uh bridge this da Bridge with ethereum called blob stream and what that enables you to do is still have a uh a bridge to ethereum so your roll up can still be an e L2 and it can get uh users and liquidity from ethereum but it doesn't have to pay the expensive da fees on ethereum we can pay the da fees on Celestia so you get kind of get the best of both worlds uh in this situation and so um uh but yeah basically like you know expect a lot more uh I would say chains to start to uh migrate there was a good episode on Bank list with um

Andrew Wong from conduit and he was talking about how out there he just just seeing a lot of chains um sort of demanding to to migrate uh I think a lot of our other role of Business Service Partners are are seeing a similar pattern um and um yeah blob stream is also something that is not yet live on mainnet but will be soon uh we're getting very very close and um I think that's going to be a big step forward in this whole like modular world oh by the way I also want to mention which one of the cool things that you can do um if you're using celesia for da a lot of a lot of folks have this fear of like well Celestia is a new chain you know if I'm going to migrate my L2 to Celestia what if Celestia goes down or

you know there's congestion or for whatever reason I'm not able to post successfully post my data to Celestia then my chain would like halt basically right um and that's a ballot fear you know like ethereum has been around for a long time more battle tested chain um that Celestia so you feel a little bit safer right from that perspective and the nice thing is we've built this capability to have a fall back to ethereum so you can actually uh if Celestia for whatever reason uh you're unable to post data there you can always post your data back on ethereum uh so your chain doesn't have to Halt or or go through any kind of like service issues so um there's a lot of really cool things like that that I think make make

the celesia integration uh really appealing to a lot of different teams come on Nick we've done a little research buddy come on this did our first we gonna want to ask about the tradeoffs right people you know there's an argument about decentralization you guys have a really nice graphic that shows you know vums um then to dat committees which your Tweet recently uh an committees heleniums and rups so there there's you know there's a bit of a eum alignment question but fallback mechanism seems to bring the Best of

Both Worlds as far as scalability cost and as centralization just brings me back to the original Point as we wrap up here light noes St ability sampling it's um modular rollups um different pting mechanisms L all that all these you guys Tred other blood on top of and we're now entering era of abundant block Stace and Sol this tremma issue one static time without going back on the first principles about of why blockchains exists so um it's you you you couldn't I couldn't have said it better myself I mean um again I I want to emphasize well first

of all yes what this whole modular blockchain movement is about is among other things is we can build these systems that are actually scalable without compromising on the core values of cryptoid blockchains up until you know these Innovations it felt like okay well we can scale these systems but in order to do so we have to sacrifice un trust minimization and user verification and all these things and the reality is actually no we don't have to and I think that's just it's amazing like otherwise crypto may have just kind of like fizzled out right like we might not have ever we may have just been stuck with these very congested but

trust minimized chains or these really scalable but just very trusted chains and not been able to get sort of The Best of Both Worlds um and I also want to emphasize though that you know CIA may have been sort of the first to you know plant the modular Watch chain flag and to to sort of promote this idea and this architecture but overall you know we're we're contributing just this this you know a foundational component the consensus and data ability layer but it's really a it's so much bigger than Celestia you you know you have teams you all these different rollup framework teams you all these different teams buing l2s shared sequencers you have other da layers you have you know people working just purely on briding you know like hyperlane or

succinct or Union people working just on ZK proving at ckvm and stuff like there it's a insane amount of talent and and projects out there that are all contributing to this and like it's bigger than any any any one of us or any one project you know and that's also why frankly it's like the people who are all about this monolithic thing I'm just like I don't know how you can compete with this like how are you GNA compete with literally you know 100 plus extremely talented extremely motivated teams all building into this this modular future like I just don't think it's like it's it's it's like pricious Innovation and uh

experimentation and there's just no way to compete with that and incentives are actually aligned you know and and that's why we say modalism not maximalism it's really about you know we each contribute our part to this greater whole and um you know crypto if we're GNA if we're going to be successful in building building this new internet um you know that is web 3 you know it's going to take a village it's going to take you know literally the entire world to to to move mountains to get us there and so you know I we're we're we're happy that we were able to play a small part in this thing and hopefully inspire people to to continue to build modular and um so yeah I'm grateful for for the whole

Community cre the entire modula narrative humbly claims that they're going to play a small part thank you Nick thank you for that optimistic end off it's been an absolute pleasure to have you on likewise yeah thanks Eddie thanks Robbie um you know big fan of you guys and um excited for you guys to keep covering the modular space and um uh yeah I just you know hope hopefully maybe we should do it again sometime maybe like a year from now or I don't know should keep checking in maybe CL