EthStorage - Building a Modular Storage Layer for Ethereum on EIP-4844

Transcript

[Music] hey hello everyone uh my name is chi today I'm going to present is storage building a modular storage layer on top eum and so could inro myself have been working here for 6 years a primary interesting infrastructure especially L to and data babil and so also closely working with etherum Foundation receive a couple grants so for is storage is also and also today's talk is actually sponsored by the ethereum ecosystem program so what is is a storage it's basically another type of layer to but

rather than scale ethereum in terms of more uh transactions or more computation power it aims to scale the etherum inms more space more storage capacity using the uh Rob approach so compared to exis storage solutions for example famous ones like filecoin RV we have a lot of unique features here so first is minimum trust um which users just need to send a transaction to the ethereum and everything's done rather than I have to submit an individual transaction to maybe filecoin or RV and then submit the corresponding data hashes on chain so everything here is minimize the the trust that required by the users and second is permissionless Network so

everybody be able to run the correspondent dat know and receive corresponding storage incentive from uh from the network and also it is a modular so that uh we do not require any protal upgrades of eum uh than to S snark we are able to verify the offchain storage um similar to the F proof and proof that to verify the execution result of offchain we have a storage proof that Implement on top of aan smart contract and lastly which is really important is scalable which we are able to achieve ped of capacity with 1 to the 1,000 transaction uh storage cost versus current uh ethereum

has so first of all how we able to achieve minimum trust so the the key idea is that to use the the latest EIP 4844 blobs that if AER just launched about two months ago which dramatically reduced the uh the code data cost to right now is almost negligible uh with much higher capacity with especially a lot of Advanced Technologies that is going to developer like PS uh which uh for the previous speech that we able to uh upload about 80 terab of data um per year so for the user it just need to send those EIP 444 transactions to the

eirm uh where the transaction will call our storage contract uh and record corresponding block hash so that we know what kind of data the user would like to store and then the rest of thing will be automatically Handle by the nodes in our storage Network that we built dedicated for the ethereum so in this way we are able to eliminate the centralized Bridge basically every know here will be incentivized to download the data from ethereum and store a local version broadcast to the peers and then store this data and prove to the Ean saying that I really maintain a copy of data

and receive corresponding reward from The Ether Mar contract so with this way we are able to minimize the trust and also review of the eum security that is the uh major principle of e layer to Second feature is that we are open open network with storage incentive alignment so uh first of all is a permissionless P2P Network by reusing for example Li P2P all these libraries and so that we are able to offer accept the data from the peers and everybody is able to join the network and become one of the nodes and be able to download data and also send the data

to others and it has some key features to implement C Storage incentive alignment the main problem here is like ethereum storage model is that all the storage fee for example pay by as a store opic Cod or create contract will be paid to the uh basically the PO blog proposer uh one time so the blog proposer rece receive all the storage fees while all the full notes running the as validator they will not be receive any fees even though they run the full node or full R cut of the etherum nodes so in is storage because we have so much data then we need to have a better way to incent to align the incentive between

the the users that pay the fee and then the corresponding data notes so that the user just pay one time storage fee which is similar as the current ether model and at the same time this storage fee when received and deposit in smart contract will gradually distribute to those data nodes that can prove that they really make a copy a replica of the data of chain and receive the corresponding fee over time so that we can align the storage contribution that is number of the that's that's the size of the data times the time and then proportionally to the to their

corresponding uh reward and be able to verify their contribution we need to also developer the onchain story verification so that uh we are able to distribute the fee proportionally to their contribution that is the size and time over time and to achieve that we we need every data node to prove their offchain storage by sampling their local data it's kind of like a uh pure D but it's constantly sample this data over time and then using snark to

compress those results those procedures in a very very simple proof Su same proof and once there such a successful proof is being generated then the St data node is able to submit this proof to network or to the to the smart contract and tell the hey I really store this copy of data over this time I I would like to receive Ro and this knock will be verify on chain using uh eclip curve paing and then we are able to check that the corresponding node is really maintained copy so that distribute correspond world and lastly it is scalable because

we only need to have two as a store key value payers that is on chain so that we can maintain what who owns the data and what is the data hashes of the corresponding 128 kiloby blob and so that we need 128 key value pairs that store on chain but be able to be points to the 128 kiloby of data so that we can achieve 100 one uh one 1,000 a and assuming the e no capacity is about 1 terabytes then we able to achieve paby capacity and that's be how we get this number and another common question ask

ask us is that what is your differences with existing F existing systems like f RV so here at list uh gave a table L this list of differences uh with different uh angles one two thing I would like to highlight is first is storage is working on kind like offering like a key value programable database thanks to the smart Contra so that we allow the users to use the smart Contra to determine who owns the data who is able to modify data who is able to append data while filecoin R they more design for static files so for

example in R there's no way to update a file there's no way to delete a data but in is storage we are able to thanks to Smart contract we are able to programmably to manage how the underlying key value database look like where the value corresponding to 128 kilobytes of blood and so this are able to enable a lot of applications for example decentralized social network like we can have a blob we can Implement on top of is storage and if there smart Contra we can have a lot of interesting applications that will give give some uh examples later and another major difference is that everything on top of storage is built on top of ethereum it

can can use ethereum wallet to send transaction and upload a data it use ethereum uh account and also easer as a stor fee and also its development testing too is all on top of etherum so that we are not only be able to reuse eeran security but also be able to reuse eer development stack eer ecosystem and this is really really really important that we able to maximize uh the E the value at the same time lower the friction of the new user to on board using is storage and so here is the road map of

is storage uh for last year we received the grants also secure the first um the first place of Aton super demo and and this year um so we have a couple of major uh Milestones one is that we have a a public test net that is now running up top of uh the Ean sepolia test net and it has run been running for two months everything is really quite stable and definitely welcome everybody to be able to join Network and try to crash and if you find some way to hopefully to crash this network in the early stage so and also uh we are have some proposals

to for example integrate with Network and with the OB stack so um here is some dashboard and also the terminal of our pu test net and is on spolia and we uh have about 100 rabid cards with which most of the notes are run by our s party so that they able to join Network download the corresponding blobs synchronize data and then do the periodic exampling can press this proof and collect reward so uh feel free to go to our website there's a link to uh to tell you how to join the network and um and how to run this uh uh notes in in the in the

network and so with is storage we are able to imagine build a lot of new applications on top of airm one interesting application is that it can serve a so great if there align solutions to adjust data expir expiration issue of ethereum so right now for ethereum EIP 444 it allows each block block to carry six blobs that is 6 times 128 kilobytes of data unfortunately because it generates so much data The Ether protocol specify that only this data is only saved in The Ether Network for about two weeks that

means after after about uh 18 days all these eum blobs will be discarded in the network and is storage will be a natural way to help saving all this data using e align approach and this problem will be become severe when the etherum is upgraded with for example PS which will eventually be able to accept about 80 terab of data per here on top E network and where to store this data is Bally how it storage can work and so this are some specification

uh for examp E 4844 that right now is about 18 days and E 444 is also going to remove some historical uh blocks that is older than uh one year and so right now the major users of ether blobs are L tws which they compress batch their Layer Two transactions and then submit to the E network and using the blob as the way to carry the B uh carry all the data on top of air so thanks to op St we also secure gr to work using it is storage so that we can able to secure all this this layer to transactions in the blobs using

its storage so that um when those data can be pruned after pruning by the ethereum protocol they're also still available on top of is storage and right now we have a test net that is testing all this feature so that we are able to be able to uh basically we run get the latest dat of this dat Tu by running the the all transaction l two transactions from Genesis to the latest day with part of data that is uh stored in this storage even of ethereum has per Auto data and another interesting application is is that we are able to combine with

the computation power of aeran with storage power so that we are able to enable a lot of fully onchain applications that's why we have proposed this rep stre assess protocol which I will give another talk tomorrow about the details but in general the idea is to turn the evm as a decentralized HTTP server so that now I can Implement all those front and logic in a smart contract and wear the points correspondent data on top of the is storage so that now we are able to enable new type of applications with Dynamic content that is managed program by the smart contract it's very similar to http but instead of calling a

centralized IP address or DNS is basically calling a smart contract and tells hey I want to retrieve this data with this code data and then this smart contract will return corresponding uh data uh for example we using EIP 44 together with rep 3 URL we have be able to upload uh MFD images on top of ethereum network or even more powerful applications uh for example one application is we upload the whole vitalic block that is about 104 megab of data and then send this all this data using EIP 444 transaction to

carry all these articles and send to a Smart contract that is mapped by vitality. block. eth that is from En and now we are able to upload the who vitalic block that is now running in top of of The Ether rather than for examp ipfs which can discard the block at any time because ipfs is not guarantee the availability of the data so this is really really crazy ideas but there's more more ideas that can build on top of inst storage plus the uh smart contracts uh another interesting idea is fully onchain AI so that we are able to

for example upad upload a large amount of the AI models or AI training data and using for example smart contract to determine the quality of data or to determine what is the relationship of the promote that's input of model and the corresponding output um so for example for the in on inference we can able we could work with uh Aura on top of the called optimistic machine learning so that we are able to almost instantly to tell what is the corresponding uh inputs uh on an outputs of a given AI model so that now we are able to fully deploy this AI on top of blockchain

without any third party uh centralized parties um so this is uh all the talk today um so happy to answer any questions regarding R storage thank you [Applause] much thank you so much are there any questions hi thanks great talk you mentioned um the incentives and then there is a onetime fee for storage but how you kind of like make sure like um or how you kind of distribute that over time and how you know frequency like what if um I would like to store something for 10 years let's say like not just like for a few weeks or a year but like let's say

10 years can I like top up or like how do you make sure that the incentives are still there for the notes yeah that's a really great question so basically how be able to cover a long time maybe 10 years or hopefully 100 years or maybe 200 years uh using one time payment uh because this is a bit counterintuitive versus our current fear U model and second is how what's what is the interval right so for the uh for the distribution we basically using a called discounted cash flow model which means that when the user pay the first time of the fee um let's give an example like 5% of the fee will be distribute it in the first year and for the next year we will distribute

the remaining of the fee and times another 5% and another 5% the underlying assumption is that first the stor fee will decrease versus the the fear and also the Easter price will be gradually like according to our current observation is going high as long as this ratio is greater than five five five% then even we basically the eer will be will be basically paid to the uh to the store providers but the rest of Fe will be able still be constantly cover the rest of the uh storage nodes uh in the following while maintaining the same

level of replications of security because the security is one/ n where n is number repus so this is the basic under underlying assumption actually this also applies to bitcoin because Bitcoin has Assumption of having the B uh the block reward every four years which implies that in order to secure the same security the Bitcoin price must be double every four years uh unless there's a significant amount increase of the uh transac transaction fee so we basically employ the same model but we Implement all this model using smart contract so it automatically calculate this discounted flow and what how much fee to should be distribute in for

example a second level uh calculation but given the whole discount rate is 5% every year so you can check out the smart cont how we uh achieve that so this is answer for first question for the second question is regarding the interval so the interval is a probabilistic process and so for example right now our design is I think 3 hours for the interval the main reason for choing such long interval is because we would like to amortize the fee over this 3 hours time and also all the rapid cards in the network so that the corresponding fee is able to cover the SN verification fee because that will be

expensive on top ethereum so uh in the I think let me see if there uh if you go to our dashboard you can see that there's a basic mining summary tells uh how many block be mind and what is the corresponding reward versus the corresponding story fee and at the verification fee that we will like to reach in some level of 10 over 1 over 10 so that the reward is about 10 times of the verification fee that's how we choose the number yeah and that is really really like important questions awesome any other question uh I have a question up about I

wonder what is your base data structure of the of uh of your scheme is storage data structure is still M po TR or verle tree or some other Dynamic trees and so on that's uh that's great great question so actually uh in the smart Contra the stud data structure is many back by ethereum key value mapping by itself we main maintain two mappings one mapping is to give a list a rate of the all the data hashes that the option should store like like this this is array of data hashes and second mapping is basically a

way to tell who what who is the owner of these data hashes of this array so that the owner is able to for example modify the data or maybe remove the data or maybe create new data so there are two mapping that takes two key value pairs on using S store on top of ethereum and so with this two mapping now we are able to for example uh allow the applications to using these key value interfaces to modify the data and the same time be able to verify this list of array of data has been corresponding has been replicated uh in the offchain network so I'm I mean not layer one uh smart

contract storage but the layer two storage so which data structure are you organizing the uh off chain data uh and uh and the base data structure which you um generate a snar to to to prove the correctness of the data Pro the validity of the data I mean the the layer two data structure uh so for the layer two we are not like for example execution Layer Two which maintains its individual lger for the layer two you can imagine it just maintain a array of the blocks that is mapped the same after this array of the hash hash list that is on on top of

blockchain so it's really simple it just maintain a array of the s which for example when there's a new data hashes that is append to the to the to the to the this smart Contra and then all these nodes will download the corresponding data from the E network and append to the DAT local storage um in this way uh when they are performing the verification they submit approve they are be able verify is really matches the onchain data hashes so this is the current very simple code design but really powerful thanks to Smart contract because we can dynamically change this data using key value interfaces on Smart

Contra so this is the uh how the data structure looks like I believe we do have time for one final question if there is one if not thank you so much for joining us and Shing sharing youres with us [Music] [Applause] every