David Vorick : Decentralized Frontend and Storage Hosting for Dapps

ETHCC 4 · 21 Jul 2021 · 29:41

Sia founder David Vorick discusses decentralized frontend and storage hosting for dapps at ETHCC 4 in Paris, addressing how applications can avoid single points of failure by hosting both code and data outside centralized cloud providers. The talk touches on Decentralized Storage and Tooling within the wider Ethereum stack.

Speakers

Topics

From ETHCC 4, hosted by Ethereum France.

Transcript

so okay hello everyone thank you for coming to the talk um today i'm going to be talking about skynet and in particular on a new product that we're building to help decentralize front ends um targeted at d5 but but just in general uh front ends within the crypto ecosystem so before we dive into that um i wanted to give some background about myself so i'm

the co-founder and ceo of skynet labs uh we built the scion blockchain starting in 2014 uh and then in 2019 we came up with a product called skynet we've been so we've been around for a long time i've been in blockchain since 2011. uh mostly since about 2013 i've mostly been dialed in on decentralized storage um and then one of the things that i tend to focus on is points of failure so kind of asking the question what can go wrong so it's just a little bit of background about me okay so before we go into what we can do for the centralized i want to talk a

little bit more about what skynet is and kind of the base layer of skynet is this immutable content so you upload files you get these hash links these links are merkle trees of the content data so after you download the file if you have the link you can verify that you got what you're expecting and it's good for all types of data um so rather than open it in the browser just to keep things smooth i just took a bunch of screenshots you'll have to trust me that it works very fast it's under a second for every single every single query um but this first link here is a is an image the second link is a 4k movie um that streams and so whether

whether you're talking about like a little you know 80 byte json file or like a 20 gigabyte movie file um skynet can handle all sorts of data um it's supported by ens and metamask that's pretty recent and then just in general you can kind of think of skynet's immutable content layer as analogous to ipfs and then kind of on the backend it does things pretty differently but but the user experience is very similar so the first link is just my profile picture this this is me on twitter and then the second link this is a screenshot of a movie playing obviously screenshots don't don't have motion but this is this is the movie tears of steel which is an open source film uh so it's it's free to share and distribute and present

um and so it's one of my one of my favorites um this is a six gigabyte file with a hundred megabit per second bitrate it streams from skynet just fine um on the back end uh rather than being uh like a peer-to-peer dht routing solution the way ipfs is we use the psi of blockchain there are a bunch of storage contracts you put data on hosts those hosts uh you pay the hosts so they get revenue they also put up collateral so they can be slashed if they don't hold the data and then we don't just give files to one host we we erase your code it we spread it out so the typical scheme is 10 of 30. that means you put it on 30 different hosts and any 10 of those hosts are sufficient

to recover the file it doesn't matter which ten and so this is kind of analogous to file coin although uh i really feel like file coins kind of kind of behind the curve on this so we've we've been doing this since 2015 and we've really over the past six years really worked on like performance and reliability and stability in the repairing code and so this stuff is like kind of our core competency so when you combine these two things together you can use skynet i already showed you the images you can also do web pages just like with ipfs that can include things like web apps and then we also support dns links so i've got two links here the first one is um you can see a base 32 version

of a hash dot scienceguy.net scisco.net skynet portal you can sub out that that tail for any other skynet portal it'll load the same exact thing so the first link looks like this it's just a landing page it'll collect an email the email goes to a centralized server so that's that's the centralized component but the the web page itself is loading from a decentralized cdn and then the second one this is our dns link this is our uh the dashboard that we use to see how our servers are doing see if things are running see you know what's low on one on balance well how the performance metrics are doing um so just two things you can do um okay on top of that we have mutable storage uh kind of

analogous to ipns you can link a public key to essentially an immutable data so you you get a public key you can share that link with your friends and then as you update the content under it um yeah the when they go to that link it'll load the latest thing you've posted so this is useful for example like blogs if you are doing immutable storage you give someone a link to your blog it'll never change if you give them the mutable link or what we call a resolver link when they when you add a new blog post all of your friends can see that new blog post using the original link and so again this is this is analogous to ipns and it's useful not just for things like

blogs but also for storing just general uh [Music] like rich state for a web application it allows you to build interesting things without needing a uh any any sort of back end um so your back end is just the skynet network um so when you deploy an application on skynet it has access to the apis that allow it to the application can talk to skynet and can upload it can download it can use these mutable resolver links and so i'm going to show you two two examples of applications that people have built um so our default uploader on skynet is not encrypted someone made a web application that's basically the same thing as the uploader

except it has uh encryption and so if you upload a file here you will get back an encrypted link that you can share no one can see what the file is even the portal that you uploaded it to doesn't know what you've created and what you've shared and this is entirely hosted on skynet there is no centralized back end this this whole application exists completely decentralized you can run your own portal and the whole application will just work um even if the uh the creator disappears so if you want to play around with it i put the uh the link up here in nice big text it's skysend.hns uh we use handshake for humanreadablenames.cysky.net

okay another app that i just wanted to showcase is called note to self this is really it's not meant to be a useful app it's really just to show off the use cases of the mutable storage so you create a passphrase this will be used to derive a public key so i'm going to put in ecc1234 then it'll take me to a note at the time i took the screenshot there was no note um so we got to we got to put in a note i wrote you know hello this app was made using a fully decentralized backend submit we saved it opened up a private browser and then if you want to follow along you can just go to note to self you can put in the password eat cc-1234 all lowercase and you will see either you'll see the note i put in or

if someone else has already griefed it has changed the note you'll see whatever the person change it to because obviously you guys have the secret key now um so you can you can manipulate this note okay yeah and then and then here's the link in uh in big text so we have one more thing to go through before we get to the application to defy and decentralize front-ends this is user accounts so we've built a technology called my sky it's actually entirely a web browser technology so all the primitives that i've described so far this is built on top of those anyone anyone could have come along and made this it's a seed-based single sign-on for all of skynet so

previously we kind of typed in ecc1234 instead what we could have done is we could have logged in with my sky when you log in with my sky you get a seed you save it kind of similar to metamask and then you can just automatically log in to all of your applications so you log into my sky one time and then when you visit various applications in the skynet ecosystem you're just already logged in with all of your user state and so my sky also adds some nice abstractions for developers when you want to store and save data it just gives you like nice nice functions you can use another thing that i'm not going to go in with into detail today is that it gives you really nice abstractions for sharing data between applications so like

application a might want to read data from application b application b wants to you know share that data but not give right access my sky kind of encapsulates all that and really convenient apis that uh developers can use to have this like common common data but that's that's not nearly as relevant so we're going to stop there um let me give you final example um this is a leaderboard that we built for a hackathon that we ran so you can see right now i'm not logged in this is in private browsing over here we have the the my sky login we come in when i click log in it says i'm logged out i can click the sign up button i get a seed i i don't know if you can

see it on the projector but there is kind of a grayed out seed in that box so we can copy the seed save it somewhere safe and then we're logged in here we have a sky profile down in the bottom left so since it's a new account it hasn't made one skype profile is one of those data sharing things once you make an avatar you put a username all of the applications can go to the sky profile and load that avatar and username and so you just do it one time on the web ever and then every application can access that in the future and this also includes centralized applications so you don't need to be a fully decentralized application to query the skynet api some something like github could just drop in some code to query skynet and pull

pull your avatar down so we could we could have like a decentralized gravatar using sky profile today and so uh once once you've filled it all out i won't go through that process this is kind of what it looks like you can see my profile my username um and then this this will follow me around on the ecosystem and then the reason i picked the leaderboard is because if you want to see some of the other applications that have been built using my sky this leaderboard kind of tracks which things are being used the most and so you can just explore it around so here is the link for that okay so we've gone through all of the like primitives that we need to we can talk about um decentralized applications and so just

as a quick recap we can store immutable data that immutable data can be applications the applications have access to apis we also have mutable data that allows us to store things like avatars and like persistent user state when you log into mysky from any machine you get just the previous state right so it's this like really nice seamless cloud and then you kind of can build all this together to build rich social applications um which can help us attack problems in defy and specifically um one of the things that i think i've been kind of focused on with d5 are the centralization

around the front ends so you have things like app.uniswap.org or curv.phi it goes through a centralized dns server usually talks to a centralized hosting server some some of them dns link through to ipfs most of them don't and then you have like a whole bunch of issues with this which is like domains can be seized by the registrar you have government letters um domains can be hacked the dev team can go rogue the dev team can be like forced to make changes or they may have like profit incentives to take features away from you or add fees that you don't want and so as a user you're really not in control of the defy apps that you use if you're going through these centralized services some of you might be thinking

ens and so i want to address that like ens is not a complete solution it solves the registrar problem and the dns problem and it really makes the relationship between the user and the developer which is a very good step so it eliminates points of failure but not all of them and so we still have the issues with rogue devs or keys getting lost or you know prof you know misaligned profit motives or governments sending regulation letters like you know you got to start doing kyc whatever so if we really want to decentralize the defy ecosystem we need to put the user in the driver's seat make it so that the user is always the one deciding what code is running

on on their machine or in their cloud and so we can use all the stuff we talked about in skynet earlier to make an app that i've been calling home screen something that we're debuting today it is a skynet app that functions similarly to like a smartphone home screen or like uh like a desktop background and you have all your applications you can click on these things are fully owned by the user when you click on an application it's not loading the application from say app.uniswop.org but instead it's looking at your skynet storage looking at a saved version of the application pulling that from skynet

and loading that and so that puts the user in control apps can be updated at any time you can downgrade at any time so we will keep a version history as you go from you know unit swap two to three or if there are like ui changes or ux changes you can revert any of them just by going to a previous version of the front end that was rolled out and so we have a screenshot of what this might look like everything else i've showed you today was um like real live mainnet uh and you can do it now this is just a mock um it's still in in alpha um but here's an example of a home screen with a bunch of apps on it and you notice that uniswap is missing from this one so what we can do

is unit swap can actually add a button that says add to home screen they can put it on their readme or they can put it on uh like their landing page or you could even put it on something like coin gecko or coin market cap and it's just a simple button that a user can click you get a pop-up that asks are you sure you want to install uniswap the user says yes and then boom we just have we have uniswap in our home screen app and so um the real like advantage of building something like this is again the user is the one in the driver's seat oh yeah so if if you then click on uniswap it'll take you to the uniswap frontend you can see in the

url bar that's a hash link so it's it's specifically loading an exact version of unit swap it's not it's not talking to ens it's not talking to dns or any part there's there's no part of your chain your supply chain that is interfering with what what application you're loading the user is fully in control so if you're excited about this you want to get on the beta list you can go to homescreen.hns.cisguy.net if you want if you're a developer and you're interested in having your app support home screen we also have we also have information for you at this page so i invite you guys to come uh and check it out

and then finally um yeah the the point is to put the user in control and and take out as much interference as possible especially if things start to get more intense on the regulatory side we want to make sure that the the users have the power um to to do what what they think is right so i got one more slide that i just want to talk about skynet and the decentralization i want to cover the portal model so you saw in all these links we have scienceguy.net science guy is the portal that we run it's kind of the the official and the biggest portal but the portal is open source infrastructure every piece of code

that we run to make our portal work is available in a github repo anyone can run it it has well-documented setup scripts it has upgrade scripts and we've really been working on making the versioning as as smooth and as clean as possible it's permissionless in the sense that like you don't need to talk to other portals to run like you can you can open up your portal in a vacuum you can be on a private network it will have access to the full skynet network it's also stateless something that we've we've worked very hard to achieve which means if a user switches portals even mid-session like science guy goes down mid-session you switch over to skyportal.xyz

which is a community-operated portal everything will still be intact everything will run as fast like just as fast it did on science sky we've we've worked very hard to not be dependent on a caching layer or like a local stateful database to make things fast everything goes directly to the skynet network and this allows us to do things like mid-session you can switch portals and the user might not even notice and then finally uh something that i think just in general the whole crypto ecosystem could use more of every single api on the skynet portal responds with cryptographic proofs so if you do like a range request you get cryptogra you get a uh like a merkle proof

that the data in the range is correct if you do like a pub key lookup it'll give you the signature from the pub key um and then there are there are a couple other like api queries that you can make that will give you proofs that the data you're getting is correct so if you have a like a verifier engine in your web browser it can verify every single request coming from the portal you do not need to trust the portal you were talking to which i think is is a big step up and and something that uh it's a standard we we like holding ourselves to okay uh so if you want to learn more uh there's a home screen uh url we also have scisco.net docs or you can email us at hello

cyrusguide.net uh we're pretty responsive we have we have a couple of people who are making sure that every every email gets read responded to um and then if you want to get involved with our community probably the most active place is our discord uh discord.g skynet labs and then we also have our github repo our website and a twitter handle okay uh so i believe that wraps up everything i had um so i'm happy to take questions thanks for the presentation is it possible to remove the data from the skynet so the way that data works on skynet is through a pinning model

when you pin data to skynet basically you're paying hosts to keep the data those contracts have a time duration you can if you're the owner of the contracts you can delete data from the contract but at someone else if they get the data can repin it and then you have you don't have control over pulling that data office kind of so it's like kind of a yes um but if someone else is painting all of your data then you can't make it disappear uh just wait for the microphone impressed by all the different layers that you guys have kind of built out for the usability aspect of it but how do you see this kind of

going from here as far as interoperability with ipfs and all these other systems how do you go from all these new styles getting created for similar solutions versus something that's more valuable for the end users yeah so i think at the end of the day the end user doesn't care what uh protocol they're using like whether it's http 1 or http 2 or now http 3 um and i think skynet ipfs r weave all these other things are going to be like in that same bucket where when they're working well users aren't going to care what it's going to be on and really it's going to be about the devs and so i what we want is to get all of these apis to the place where they're all verifiable you have this engine in the browser this outer scope of today's talk but we are we are

working on uh this kind of kernel that can do be arbitrarily extended to add any set of apis and also do the verification of those apis and so then uh it's really in the control of the application developer to decide whether they want to do skynet ivfs are we have a mix of both or like if they want to add support for cosmos they can write the verification module for cosmos they don't even need to talk to us that's that's not deployed today that's still kind of in the in the conceptual stage but i do i see this moving to a place where the user is not really thinking about it and just all these apis are supported on the back end at like i think of it like an operating system kernel you don't really think about what drivers your operating system has or

what what the file system is but yet you know all the applications make use of it continuously how does your economic model differ from falcon at this point um yeah so the the major thing is that everything on the sci network is paid for so every every request every upload every download goes over payment channels actually when when you download a file it does something like 500 payment channel transactions um and it just it just pays a whole bunch of hosts for all the data it's fetching different pieces from different hosts um and the way that hits the end user is that cya sky.net operates on a freemium model so if you're a user who's never heard of it and you haven't

signed up you get some some speed um and then as the internet kind of fills out and more people are making use of skynet you can upgrade to cya sky premium you'll get faster speeds and then sci sky pays its bills it covers the cost of the free users with the premium users would you mind talking about some of the traction numbers that you're seeing and also comps towards like how hosting files on cnn compares to centralized file storage and then maybe where you see it going in the future yeah um so in terms of traction numbers uh we're actually the the file storage

end of things is doing extremely well i think in the month of june we had something like a million unique visitors 1.1 million unique visitors accessing this uh the sci network or the skynet network most of it is for file sharing and like not even the illegal type of file sharing it's just like on the internet today it's a pain in the ass to send if you have like a pdf or a log file and you need to send it to a friend um skynet works really well for doing that like we take this pdf i don't know if you saw me before the talk started i was downloading it from skynet because that was the the most efficient way i had to send it from my phone to this laptop that's it's not my laptop um and

we're seeing a lot of that um we also have some companies using skynet as an actual cdn um so uh i don't know if i can i can share the name but um yeah like they have a bunch of users who browse web pages and then and then some of the media assets on those web pages are coming directly from skynet um and then your final your final question was yeah so skynet is an open marketplace or i should say is an open marketplace that makes the prices extremely competitive it's about i think at the moment three to four dollars per terabyte per month for

uh bandwidth is about one to two dollars per terabyte so the bandwidth is just absurdly cheap and then the performance which i think is really really like the key place that we shine relative to all other decentralized storage projects um the median response time for a one megabyte download i believe is around 300 milliseconds and that's that's like behind the cache layer so if you hit the cache layer it'll it'll be more like you know 30 40 milliseconds but with with the cache miss again yeah like so two more questions in fact um so one in the freemium model uh following back to my previous question

first question in the freemium model uh was the guarantee that the data will stay yeah so if you are using a portal uh that is not not yours you're kind of at this point in time you're depending on the portal to keep the data pinned for you cya sky makes a public commitment to holding files for 90 days a minimum of 90 days we've actually never deleted files so we've everyone at least up till now everyone's had permanent storage because we've just it's been financially sustainable um and then if you run your own portal uh you can get those pinning guarantees you have the control yourself we're also working on a technology in the background this has kind of been it's not super heavily requested so we

haven't been it hasn't been super prioritized but we can give you control over your own file contracts on the scion network okay thanks and the second is about the verifiability i was using skynet like half a year ago i mean the skynet uploading some website and like playing with it and i was not using any private key so you mentioned i mean some new feature how does it work who is owning the private key and who is signing it who's signing the data what um so the question is less like how the verification works and who's yeah so when you so when you upload data to so if it's if it's if you're using the immutable part of skynet

it's all just hash based right and so you upload data you get a hashback and you share that hash with friends they can they can just verify that what they're downloading matches the hash if you didn't put in a seed and you're using the so if you're using the mutable or like the resolver part of skynet somewhere there's somewhere there's a seed usually the application developed usually the application either uses sky id my sky or it's straight up asks you for a password and it uses one of those three options to generate the public key that it then uses to sign the data um so if you didn't put in any seed at all it probably means you're using the immutable part of the storage oh i see okay thanks i'm wondering uh what would happen if

someone were to post something some illegal content on skynet like do you guys like have some blacklist or are you not liable for it yeah so it is a decentralized network uh we can't go and like take things down from saya however we run scisky.net and we can control what we serve and so we do have a um a block list and we're working on a malware filter um that will just scan scan things for malware and automatically block them if it detects malware and so if you're using a public portal it's going to be the portal operator's choice what they either allow or don't

allow and then of course if you if you disagree with our policies for example we most of our blocked content is copyright infringement if you disagree with that you're free to run your own portal all the all the code is open source okay it looks like uh we're done with questions so thank you very much i uh i was interesting yeah is think