Is decentralized cloud storage the next big web3 industry

20 Feb 2023 · 20:47

Jay Hamilton presents a primer on decentralized storage, drawing on a write-up by CryptoShuriken. It explains why centralized clouds such as Google Drive and AWS are open to censorship, attack and single points of failure, and how peer-to-peer nodes paid to store redundant copies solve this. The Web3 Academy episode compares IPFS, Filecoin, Swarm, Arweave, Storj and Sia, and spells out IPFS versus Swarm: distributed hash tables versus a content-addressed chunk store, and Swarm's built-in incentives versus IPFS relying on Filecoin.

Speakers

Topics

Transcript

GM GM welcome to another episode of web3 Academy your trusted source for useful and legitimate information to remain on the Forefront of the internet Revolution I'm jay bird and here web3 Academy we believe that web3 is going to change the world and that's why I'm here to guide you the world's top talent into this space to contribute get ahead of the curve and capitalize on the opportunity in today's episode we are talking about the rise of decentralized storage so in recent years many of us know the world has witnessed tremendous growth in the amount of data being generated

stored and accessed every day with this explosion of data comes the need for more efficient and Secure Storage Solutions decentralized storage is a promising technology that has emerged to address the limitations of traditional centralized storage systems unlike centralized storage which relies on a single entity to manage our data decentralized storage leverages a network of computers to store data in a distributed manner this approach offers several advantages including improved security privacy and resilience so today we have crypto shuriken who is the founder of his own blog and has been writing about web3 for a while and he's

here to share his knowledge in today's deep dive here's what we're going to cover we're going to talk about what is decentralized storage why we need decentralized storage how decentralized storage works and the options for decentralized storage protocols and platforms so it's going to be a good one for everybody out there because this is really the basic information it's not the sexy stuff that gets you to make money it's not going to get you a new nft or the next Alpha on which token you should be buying but all of us have data a lot of data in fact online and we need to all understand how this data is

managed and how it is stored because if we don't we really put ourselves in position for a lot of risk so you're going to want to tune into this because it's a really important episode for everybody to understand before we jump in let's just take a minute to hear from our awesome sponsor the future of social media is here and that future lives in web 3 on top of lens protocol web 2 social platforms are bro broken and right for disruption you see the epicenter of social media is the creators and yet they are the most neglected web 2 platforms like Facebook Tick Tock and Instagram are all essentially robbing creators of their worth creators are a new type of entrepreneur forming new types of businesses yet with web 2 platforms

creators don't own their content to the profiles and that's their product in business instead they are tied to the platforms they choose to create on well just like how crypto is freeing us from Banks web3 is freeing us from these centralized platforms on lens protocol creators own their content own their profile and even their social graphing followers in the form of nfts this allows you to move freely from one social application to another with your content profile and followers moving along with you lens protocol enables self-saverty for your social graph and interoperability across the internet at web3 Academy we believe this is the future of Social and that's why we've partnered with lens to ensure that the path of social media is heading in the right direction visit lens.xy Z to learn

more today are you building a community around your brand well listen to this at web3 Academy our motto is Community First profit second why because engaged communities tell you exactly how to improve your product and ultimately Drive growth they act as team members recruiting new customers and providing crucial feedback and they become brand super fans sticking by you through thick and thin but to engage your community you must first understand them that's where Chasm comes in Chasm is our go-to web3 tool for managing and understanding our community members it combines both on-chain and web 2 metrics all in one user-friendly dashboard with Chasm you'll know things like which other

communities your members are part of and which of your campaigns are truly driving results that's why at web3 Academy we use Chasm to launch campaigns optimize growth engage our community members and automate workflows with this all-in-one tool if your community is already on chain and get to know them better with Chasm head to chasm.xyz using the link in the description and discover why top brands like a mutable Nifty labs and collab land are using Chasm so what is decentralized storage well as its name implies decentralized storage is a type of storage system that does not rely on a central server or authority unlike a centralized storage

system which is managed and operated by a single centralized entity decentralized storage is operated by a peer-to-peer network of user operated nodes Each of which stores a copy of the data creating a resilient file storage sharing system this makes it much more resistant to censorship and attack than traditional storage systems the need for decentralized storage really comes down to the fact that in modern days cloud services such as Google Drive AWS Dropbox have become the go-to option for hosting files and websites while these Services have helped individuals startups in large companies relieve the hassle of managing their own storage infrastructure to centralize nature of

such Services presents some deep flaws one of the many flaws of centralized Services is that they are easy to censor if a government or other authority does not want certain information to be out there they can simply order the centralized storage providers to remove it and not store it in the future furthermore they can even make changes to the contents of the data stored resulting in poor information Integrity now another flaw is that centralized storage services are vulnerable to attack if hackers can gain access to the service of a centralized storage provider they can potentially access and even delete all of the data that is stored on those servers since these

Services have data hosted on centralized servers having a single point of failure means any outage can result in temporary or even permanent loss of data and we've seen this many times one example that comes to mind was about a year ago when Uber was hacked and a lot of sensitive information was obtained by a hacker one single hacker was able to get into their system and once he's in he can access a ton of their data because of that centralized entity so it's very concerning the risks are real now decentralized storage provides a solution to all these problems by

providing a censorship resistant secure distributed efficient robust cost effective and resilient form of data storage so question on everyone's mind how does decentralized storage work well let me explain the decentralized storage model works by Distributing a copy or a portion of the entire data across a peer-to-peer network of nodes Each of which is incentivized to store the data by storing data redundantly across multiple nodes the decentralized storage system ensures all data is secure and accessible when the same piece of data is stored across multiple nodes even if

a couple nodes go down the data can still be retrieved from the rest of the storage nodes so what's in it for the node operators well we all have unused use space available in our computer and mobile device storage the decentralized storage system makes use of available storage space on the Node operator's disks drives and node operators are incentivized to rent out this unused storage space on their devices to the decentralized storage networks now let's learn about different decentralized storage protocols and Platforms in their inner workings for starters we can even consider the ethereum blockchain itself to be as decentralized storage system

and to some extent it is for code smart contract storage but when it comes to storing large amounts of data jpegs GIF MP4 a lot of the data that is being used in nfts when we think about music nfts when we think about image indices or art nfts it becomes a bit unfeasible not to mention gas inefficient since this isn't what blockchain was originally designed for to store large amounts of data so we need to look at other solutions that are tailor-made for storage purposes first ipfs so ipfs is a peer-to-peer protocol for storing

accessing and sharing data in a distributed file system what does the mumbo jumbo even mean well let's look at an example we all love Twitter the Little Bird app that is so cool to access Twitter you put in HTTPS colon forward slash forward slash twitter.com into your browser and since it points to Twitter's IP address we get what web page the server on that address stores but if we put Twitter on ipfs we don't get an IP address instead we get a Content identifier or CID for short to access Twitter that looks something like this it's a ipfs address that looks like

a wallet address so instead of finding data by its location ipfs finds it by its contents so how does ipfs work right now there's a problem that exists on the internet that content is found by its location you want to Netflix and chill cool go to netflix.com you want to read web3 Academy cool go to W3 Academy Dot i o but what if the location of the content changes for some reason so content addressing fixes this problem by content addressing every piece of content has a CID which is based on the data's cryptographic hash that points to

the data in ipfs that means two exactly similar files will have the same CID any minute difference in the content will generate a completely different CID ipfs uses the sha-256 hashing algorithm by default to generate cids ipfs uses content addressing to identify and find content rather than looking where it's located so take a look what happens when let's consider what happens when you add a file to ipfs your file and all the blocks within it is given a unique fingerprint called a cryptographic hash

ipfs removes duplicate across the network each network node Stores Only content it is interested in plus some index information that helps figure out which node is storing what and then when you look up a file to view or download you're asking the network to find the nodes that are storing the content behind the files hash so you don't need to remember the hash though every file can be found by human readable names using a decentralized naming system called ipns there's so much to these D Storage or decentralized storage protocols that each of them deserves a detailed blog post for

example ipfs uses Merkel dags for representing files and directories but for today we're just going to stick to the basics in this podcast second decentralized storage option we're going to talk about is file coin filecoin is a peer-to-peer Network built on top of ipfs created by the same team behind ipfs that incentivizes users to rent out unused storage space by rewarding filecoins native fil token in filecoin users pay for storage space and anyone who wants to store other users files can join the filecoin network and get paid so here's how it works I have a device

and a user wants to store it the user pays the miner to store their file the miner commits publicly via filecoins blockchain to storing the file the network constantly verifies that miners are storing files correctly and then the user pays the miner to retrieve their file back again when they might want it in the future let's talk about another option swarm swarm is another decentralized storage protocol that is part of ethereum's Holy Trinity ethereum for computing power whisper for messaging and swarm for storage swarm provides a completely decentralized storage infrastructure that allows

people all over the world to become storage providers and get paid while doing so swarm is designed designed to be highly scalable and resilient and to provide a platform for applications that require a high degree of security and censorship resistance the idea for swarm was presented by Gavin wood and its development is mainly funded by the ethereum foundation so let's talk about a few things about swarm first it's permissionless and private second it's leak-proof messaging made real this means that the third element of this ethereum Trinity allows for swarm to be incorporated with whisper which is bringing the possibility of truly

leak-proof messaging for full nodes swarm is also built for Builders it's completely decentralized it's scalable with zero hosting costs and it's public interest since anyone can contribute to swarm it establishes the system for important data Commons such as openstreetmaps data and is collectively funded by the public it's a public good so what's the difference between ipfs and swarm well both protocols may look similar from a high level there are subtle differences that exist when we dive deeper into the inner workings and philosophies Behind These protocols some of them are swarms core storage component uses an immutable content addressed chunk store well ipfs uses

distributed hash tables to find which peers are hosting the content a hash table is a database of key to values a distributed hash table is one where the table is split across all the peers in its distributed Network since swarm is a part of the ethereum Holy Trinity it has deep integration with the ethereum blockchain for the incentive system whereas IPS s has no incentive system of its own and utilizes filecoin to add an incentivization layer from a development standpoint ipfs is much further along in code maturity adoption and community engagement as compared to swarm as a result ipfs has a lot of content

available in terms of documentation whereas swarm doesn't have a large documentation base I mean even while doing the research for this article we found very difficult to find information on Swarm and had to go through a lot of third-party blogs because their documentation just isn't quite there yet so there are a lot of similarities and differences between both these protocols and for more details make sure you check out our newsletter where we have more links next up let's talk about our weave our weave is a decentralized storage platform that uses a new type of data structure called block weave block weaves allow our weave to offer scalable resilient and efficient storage so what are block weaves block weave is a core

technology behind our web we all know that a blockchain is just a chain of linked blocks that contains transaction data block weeds are similar to blockchain and that they are a chain of blocks but it contains storage data linking to multiple previous blocks from the network block weaves work by enforcing that the miners provide a proof of access to old data in order to add new blocks unlike a more traditional blockchain where miners spend compute power and electricity to compete to mine a block in order to earn tokens the rweave network encourages miners to store and replicate valuable data in order to earn tokens the Perma web now

the Perma web just like the traditional web is a collection of intellect documents and applications that are stored permanently the permanent web sits on top of the rweave data storage layer since the rweave network is built on HTTP just like the traditional web web browsers have access to the data stored in the network so imagine you the users and then we have the Perma web which is US accessing the web and then underneath that we have our weave which is the network that stores all the data next Let's Talk About Storage that's s-t-o-r-j storage is a decentralized

storage platform that provides secure scalable private efficient and S3 compatible decentralized Cloud object storage similar to other decentralized storage platforms storage connects people with unused bandwidth and storage space to those who need cheap accessible and private filed storage storage uses s-t-o-r-j storage they're native erc20 token to incentivize people who are renting out bandwidth and storage space one more Sia or s-i-a Sia is another popular decentralized cloud storage platform that connects renters with and hosts together in a peer-to-peer Network Sia has its own

blockchain and the hosts get rewarded with Sia's own native utility token see a coin once the renter uploads their file to the CN Network it gets split up encrypted and sent all over the world the network ensures the files are always accessible by making multiple copies of them and since the files are split up into multiple pieces and encrypted they're inaccessible to the host so in conclusion with the mainstream of adoption of centralized storage networks in the past decade and with centralized cloud services like Google and AWS providing a fast and cheap storage infrastructure to individuals and organizations threats of poor information Integrity privacy and censorship have been omnipresent the

decentralized storage model addresses the issue of centralized storage by Distributing the data across a network of distributed nodes Each of which stores a copy of the data this makes it much more resistant to censorship and attacked with better information integrity and data availability than traditional storage systems thanks so much for listening in everybody I hope you've learned about decentralized storage systems how they work and the available options look forward to seeing you in Discord on Twitter or at a conference coming up until then have a great day thank you for listening to web3 Academy we hope this helps you along your web 3

journey and if it does please share this episode And subscribe so you don't miss the next one nothing in this podcast was Financial advice crypto and web 3 can be risky you can literally lose it all in fact if you invest on account of what we say you probably will lose it all so don't do that in all honesty the point of this podcast is to remove the noise of markets and price and focus on utility implementation anyway so you should not take any of this as Financial advice thank you friends and see you in the next one