Garrett MacDonald - Decentralized Storage Landscape in 2024 - the rise of native VMs

EthereumZuri.ch 2024 · April 2024 · 22:14

Garrett MacDonald surveys the decentralized storage landscape in 2024 and the rise of native virtual machines on storage networks. Protocols such as Filecoin and Arweave are building VMs that let decentralized applications access and compute over massive amounts of stored data, and he looks at the new capabilities this unlocks for Web3 developers at EthereumZuri.ch 2024.

Speakers

Topics

From EthereumZuri.ch 2024, hosted by EthereumZuri.ch.

Transcript

uh yeah I'm Garrett uh yeah big fan of like retro computers and uh and crypto uh shout out to the guy in the back with the Commodore 64 shirt uh my first programming language was like 6502 assembly so it's great to see a lot of VMS uh native to decentralized storage um uh networks and protocols that support assembly uh web assembly specifically so yeah um I thought we'd start by just giving a you know brief history of uh decentralized storage um yeah you know there have been a lot of great projects in the past um that have come around and a couple of them have uh fallen off or pivoted into decentralized compute in general um but there are a

few which have stayed um since since the beginning um and and a couple others that have that have stayed on Swarm is definitely the oldest one um eth storage is a great project that was started more recently um both of those two I would highly recommend checking out um and diving into um they're they're great teams working on interesting things um but for the purposes of this talk um I'm just going to focus on filecoin and rwe just because they're the only two which have started to develop um Native virtual machines um which are going to unlock a lot of new a lot of new possibilities um in decentralized storage as well as compute so um the the the two kind of off offerings put

together by the by the teams of r fcoin r AO and and fvm um they both have very similar intentions and actually are causing kind of a convergence of um of rwe and filecoin which you know are quite different in their architecture and intentions as protocols but the native VMS allow for you know each one of them to kind of make up for for the sacrifices that or the trade-offs that it had to make in the in the original design just because you can do so much more with file with or with virtual machines now um so this is uh this is really fascinating and we're going to dive into the ways that they're converging as well as um yeah kind of

the but we'll start by you know uh just going over you know how they are as protocols before the VMS um so if you're interested more in like the protocol level um comparison um this is a link to a talk I gave uh last year about that specifically um yeah uh basically just diving into all of those things from a protocol perspective so you know just to do a quick recap from that talk um yeah these are the main the main differences between the two protocols um not considering the VMS of course um yeah so there's different implementations um there's different kind of Hardware that's required for each of them um

different tokenomics um different goals this fourth one is probably one of the most important differentiations um you know rwe by default or like as a protocol without a VM is geared only for uh Perpetual storage where filecoin is just for um uh term-based storage so um yeah this is this is probably the biggest differentiation between the two um but today we're going to talk about um yeah the virtual machines that each of them are going to be able to run um ethereum virtual machine thereus web assembly wasum um as well as general virtual machines which each of the the two protocols both AO and fbm are working on

implementing their mechanics um and then we'll go through all of these topics um on applications uh I wanted to cover a lot more actually but uh didn't think we'll have time so I will put a link to the um this presentation at the end which then uh will update probably by the end of next week going into a lot more depth with different applications um because I think there's a lot a lot to cover there um yeah so we'll start with with virtual machines so the the the great part about both of the offerings so far is you know uh filecoin virtual machine is started out by offering just an ethereum compatible virtual machine underneath it um and of

course it inherits these same uh properties advantages and limitations as every other ethereum virtual machine out there um it's very you can think of it a lot like L2 that settles down to the the filecoin blockchain um and then web assembly is you know the big advancement that a lot of other uh networks are moving to um it's a lot more free in what you can do um you can program in a lot of loc in a lot of different languages the as long as it compiles down to wum um it's an amazing in terms of what kind of memory you can throughput in applications things that just wouldn't be possible due to gas fees on uh evm implementations um and particularly

interesting um both the you know AO as well as fvm will eventually be able to support more than just uh 4 G gigabytes of memory which will be super fascinating because then we can load entire uh AI models like very large and complex ones as well into um virtual machines that are running natively on these networks leveraging data stored on the respective networks so this is like a massive massive step forward forward for um decentralized compute as well as autonomous agents um which is you know a super hyped topic these days um so yeah just wanted to specify that yeah both of these are being worked on by uh by both and we'll we'll dive in a little more

specifically on the nuances there as well um so as well as you know supporting both evm and web assembly um both AO and fvm are working on being VM agnostic so that means as a developer eventually you'll be able both of them don't have this implemented yet but eventually you'll be able to uh load up whatever kind of virtual machine you like so instead of wasum maybe you want to implement something else and then you'll be able to do that and have all of the same kind of guarantees that the applications you intended are running or what's running and so on um interv communication uh message passing is uh basically this we we slide after this kind of defines what um like based

Computing is but um inter VM communication um is really key part of that and um yeah so wanted to just Define Actor model for everybody here um basically it just means that you know instead of having a global State you have a bunch of independent actors and they're acting they're doing out whatever they want they they're doing whatever compute they want and the only way that you know another Act One actor is aware of another is when a message is passed so um so it's another way of computing that's very different from traditional blockchains that we're all used to where we have a global State um now we have a bunch of individual states that are are you know evaluated independently and

only communicate with one another as they're programmed to do so um of course this uh this makes it a little bit more difficult to build kind of like a decentral decentralized exchange or something like that but what it does enable is you know massive massive uh parallel compute which means we can do like insane things that we PR previously could have only done in web 2 um but now now through just like independent instances and message passing we can accomplish um yeah compute on big data sets in web 3 um with similar trust assumptions so um yeah so then where it starts to get a little bit different is um is in the mechanics

so um so AO uses a single system image um and there's three kind of separate units like uh schedule unit compute unit message unit and they all have different roles schedulers kind of uh schedule when when things should happen so you can pre-program that at a certain date or Upon A Certain event happening um the the uh a virtual machine can instantiate itself and kind of wake itself up um to do a to do a task and this is something that's new that you couldn't previously do in ethereum or evm based um systems because those unless you're depending on kind of a external scheduler running on a web 2 server or something like that um and it's also has it basically AO is

focusing on you know wasm first and we have a separate Network for running like uh that's basically ethereum virtual machine that is native to uh to rwe um and that's kind of similar to what the fvm already has they have the FM um and that's the thing that you can deploy smart contracts to and so on already um so yeah the main difference is uh in this kind of thing is um the fbm is prioritizing being more like a traditional blockchain First where uh AO is prioritizing uh message passing first um the advantages of each are the trade-off is mostly because um filecoin I think wanted to build a system which

can be used um really really quickly um for like file coin defi um so that way they can have you know liquid staking and um lending facilities for the different uh miners because miners are required to stake tokens um AO is being more experimental um you know they're creating this whole new paradigm of decentralized computing by focusing on message passing first um that will be eventually implemented in in FM um but for now AO is is uh leading on that side while fbm has a more kind of secure uh traditional approach um yeah so so far both protocols will allow you to um yeah start or renew

storage deals and we'll dive in a little bit more on the uh AO side how that's uh really interesting because previously that wasn't relevant because R weave is by default a Perpetual storage protocol so whenever you upload data to it it's uh there are mathematical guarantees that it'll be there for a very long time um and both of course will have native storage access so so you know this is uh this is hugee because in the past you would need an oracle like chain link or or some other to you know grab data from a decentralized storage provider or or centralized storage provider and bring that down onto um whatever kind of compute later you're using if it's ethereum or or or whatever else um so having that kind of like native storage access where smart contracts will be

able to grab data um directly from the from the system and they don't have to trust some kind of third party Oracle um will be huge as well as um just because they'll be able to ingest massive amounts of data into the memory of the applications um whereas like you know in the in web assembly one which is using like 32-bit memory pointers it's limited to 4 gabt of of memory but you know that will expand to much much larger as soon as uh like wasum 64 is implemented and you know tooling gets better for that and so on um a big Implement a big uh feature of both of these things as well is like uh is compute um you know bandwidth is what costs the most when doing compute especially on large data sets of course um uh so having the

compute facilities at the same place as the storage facilities just makes a lot of sense for for large use cases particularly AI um so yeah this is this is a really big Point as well um so for AI inference um you know AO is going to create kind of a decentralized compute marketplace where anybody can join in as a compute unit some of them can be specialized in running gpus that are good at AI inference um FM has this uh nice kind of I probably I don't if it's an advantage but it's just a design difference there where currently U they happen to have a lot of miners which have gpus in the same location as storage so um yeah those facilities already exist in the same data centers

most often so it's really exciting to see how you know that compute will be leveraged for something really productive as well um the uh yeah and and here's like I think the most interesting part of this whole phenomenon that's happening is where AO or R weave started as just Perpetual only storage and filecoin started as only term-based storage meaning you know you agree with the storage provider for a certain amount of time and then when the contract expires somebody has to be there to renew it otherwise the data is just deleted overwritten whatever um now the other can be implemented in each so it'll be interesting to see what the reliability is of those uh different

tradeoffs because R weave and fqu are coming from these these different approaches in the beginning and now kind of you know stepping on each other's toes if you will like uh implementing what the other can offer through virtual machines so it's pretty interesting um yeah so AO will you know use the VM to enable you know temporary storage contracts on compute units um FM will enable um automatic renewal of storage cont contracts with the storage providers so yeah that's uh I'm pretty excited about this because now are we even file coin will actually be able to compete once this is up and running um yeah so uh just to summarize like key

similarities you know uh actor model um meaning you know it's all based on message passing both of them are really trying to implement this um Native storage access is going to unlock you know um large scale compute on on big data sets that wasn't possible in web 3 before um each of them are trying to implement you know evm wasm and the ability to implement whatever VM that you desire as a developer um the difference there is just that you know filecoin is prioritizing evm uh AO is or rwe is prioritizing wasum um and yeah and the convergence and storage offerings I mean that's uh that one's

really huge um and the big differences are actually not so much in kind of where you want to go but it's in more of just like the road map so both of them will imple will like in in their ideal end States I believe uh end up looking pretty similar just with different trade-offs because of those original because of the design of the original protocols um so the only the only difference is kind of the order of you know what's being built um and and yeah the the but it's it's of course going to be subject to all of those original design decisions from each protocol so um we're still really early in this and it's not very clear you know which will end up being the better offering for any

given um use case but you know on FM people are most excited I think about you know kind of like liquid staking of the tokens and automatic storage renewals with storage providers and on AO people are more excited about like autonomous agents doing compute on large data sets um it could be the case that the other is better at either of those but um but I think it's really exciting to uh to follow these two protocols as they as they go on this uh this journey of convergence um yeah so that's all and thanks very much appreciate it and yeah these slides will be uh yeah available there thanks thank you so much is anyone has

any questions there's one question over that yeah I uh yeah thank you so in my limited understanding um rwe is mostly based on like permanent storage with a blockchain right and uh and then the um um um sorry the file coin is more is more based on the uh what's the name of that interplanetary ipfs yeah right with like mixture of like consensus algorithms so those are like uh very different in nature how do we like compare them going forward and then I mean besides what

you've already mentioned today and then also what what's your view on um swarm because when it originally when I originally heard about it as that was one of the original ethereum storages and then after that I didn't hear much about it and I was really uh rooting for it really wanted it to succeed so if you have maybe mentioned a few words about it as well yeah yeah for sure um so the M the main two questions are you know about how to compare the two going forward uh being you know filecoin and fbm and are we even AO um as well as swarm um so yeah how to compare the two I think I mean it's going to be uh it's going to

be really interesting I think yeah it's a it's a t question like how to compare the two I don't know what perspective are you asking kind of as a developer interested in building on one of them we BL permanent storage with blockchain right in in my view it's going to be more expensive to sustain it forward with as far as like crypto economics goes yes rather than something like file coin with actual miners there and yeah it's kind of broad question sorry but uh yeah yeah it's really just so different in nature yeah they're very different which is why it's so interesting that now they're kind of going toward the same place with the virtual machines but the uh but R we actually isn't a blockchain per se it's it's more of a block weave which is why

it's called like U our weave and um there is like a blockchain that kind of manages the transaction history but the data itself is stored through like U you know economic like guarantees and it's it's a lot of kind of game theory that goes into that um the prior presentation that I gave there um goes more deeply into the economics of rwe and yeah how that works exactly but um yeah happy to you know chat more oneon-one about that as well and um and swarm as well yeah I've been following it since the beginning as and been super um super eager to see them integrate you know they've tried a few different things over the years and um I think

yeah there there's still kind of this like more quiet team building in the background but they've been working on a lot of interesting things recently so I'd recommend yeah check them out awesome thanks as long as there as long as the Swarm is not dead yeah yeah know very well in life and I don't think that'll change I mean I think they've you know kept a lot of the crypto that they raised at the beginning so you know I think they're there to stay financially it's just a matter of yeah uh adoption yeah thank you yeah any other questions uh thanks for the presentation um have you seen any increase in usage uh driven by decentralized compute on R weave or

by liquid staking on filecoin yeah not yet on either but that's only because um yeah you can't uh do kind of General decentralized compute on either yet I mean AO is still in a testet and fvm you know hasn't implemented like a like like arbitrarily uploadable you know wasm code yet so um yeah it's too early to tell unfortunately uh yeah liquid staking yeah that's been driving like a ton of volume in like defi on F coin um that's like the majority of it I believe there's like yeah like I think 500 million or so as of a stat that I read of a couple months ago staked on the filecoin um and like 300 of that is like

liquid staking um for you know facilitating kind of uh you know liquid staking so then miners can take those tokens and um and stake them for their like storage capacity to mine yeah hey yeah I just wanted to ask you what else would you like to see get built on R weave yeah going forward yeah right that's a good question and thing I'm most excited for is like yeah autonomous agents which are automatically building themselves I mean we're starting to see this with like uh you know different like AI tools that are are coding you know able to kind of like code an entire application um just through like a a text prompt so if like

we can make a agent which prompts like another um like AI model to code something that might be useful or profitable or something like that we can have a whole ecosystem of autonomous agents you know going around in in the in cryptand you know acting by themselves um this is this is going to really change the way that we all use crypto and eventually as that TAPS in into the real world I mean it'll it'll change you know the way we interface with with a lot of things I mean just the like robots so far have been quite Limited in like what they can access but you know I think I really believe that that AO has a potential to you know enable like autonomous agents to act on

chain as well as in the real world in a really meaningful way that we haven't seen before so it is kind of a potential for a a paradigm shift so that's that's what I'm more most excited for of course there's two sides to to every coin but uh yeah I think if we're responsible in the way we deploy these things it could be really really positive yeah thank you so much yeah thanks appreciate it talk thank you [Applause]