Decentralizing WalletConnect and Why Low Latency Is Hard

Transcript

good morning fellow plebs fellow Maxis fellow jpeg aficionados and last but not least all the millions of people watching from home on their televisions of course so by means of introduction I'm I'm Derek my work at World connect and I like wallets I like to connect wallets but I also want to get to know you a bit so please raise your hands if you have a desktop Chrome something browser extension wallet okay that's very few of you who has a hot wallet on their phone and no worries like I don't think this

is recorded so you're not going to get kidnapped or anything so yeah who has a hot wallet on their phone who has more than one hot wallet on their phone who owns a hardware wallet who has like a exotic wallet like a smart TV wallet who has a wallet in their fridge or smart toaster or something um okay so maybe in a couple of years um your toaster is gonna your AI toaster is gonna hate you because it you don't you don't pay it thank you um all right so today we're going to talk about decentralization but in some other some other shape or form like

we're going to talk about how you can decentralize a more Orthodox service a more Orthodox web 2 type service so before we kick it off let's get into what's World connect most of you are probably somewhat familiar with World connect the open source project was started about five years ago the motivation was at the time we mainly had the apps on desktop and we didn't really have remote wallets or hot wallets and then hot walls started but there was still no good way to connect them so basically World connect started to connect desktop dabs browser dabs to hot wallets

but this has changed like even this basic use case that we started with has changed nowadays you have wallets in in many types of devices so hot worlds obviously Hardware wallets and um in unity unreal in many different places there's there's no wallets and the same is happening for dapps obviously we're not going to bring a billion people online on desktop but if we bring a billion people they're going to be on mobile devices and other types of devices and we have sdks for all of these platforms and I think we can expect this mobile Revolution mobile first that happened many years ago and to also at some point happen in crypto

and we're we're building the infrastructure for that and so what's the motivation to decentralized World connect in the first place I mean for most people here in the room that that's probably obvious the crypto ethos mandates this supposedly but just to bring it back to reality like other similar messaging services to wildconnect let's say email are somewhat decentralized but the real incarnation of emails that you choose a provider like Gmail and if you would want to like that's decentralized I can hypothetically move my email to another provider but it's actually pretty hard like AI needs to update my DNS and then I need to move my data and in reality

it's like not a really smooth solution so we want to do better than that and provide end users the the option of like obvious things like um redundancy availability of one world connect no node goes down you can just use another one your data is distributed everywhere and like um censorship resistance and why as well connect maybe a bit different compared from blockchains from for instance so we need we have different latency and throughput requirements and also some requirements on consensus that blockchains have we don't have and also the systems We compare ourselves with

like email or Whatsapp are are very cheap which blockchains aren't necessarily and also they have way higher throughput than blockchains so when you look around of what other types of systems solve these Solutions while Meeting those requirements there's not much around obviously blockchains too expensive level lower lip P2P waku and also too slow too expensive to like solve and meet all the throughput requirements latency requirements and then traditional databases aren't decentralized enough so let's take a look at the current high level architecture so basically you have a

wallet or a dab and that connects to through a lot balancer to a set of services so you're let's say it's World connect chat one end user is connected to Singapore another one might be in the US and and so it's different load balancers and then you're connected to different nodes they send your messages back and forth via http and the end user devices are connected via websockets you have a bi-directional channel and these things you can sort of decentralize not so not so hard but then in our scenario we have a routing table which is a yeah it's basically a hash

map that says which end user device is connected to which node and whether it's online basically such that if both parties are online we can just send the message directly but if one of the parties is offline which is very common in chat but also in our signed use case when you sign messages and connect a wallet and then the message goes into what we call the mailbox so next time the client comes online it just picks up the messages and that it's that it missed and then there they are deleted all of this obviously is end-to-end encrypted so we can't we can't see any of the of the contents of whatever that is being passed through the system and yeah so there's nothing off the shelf available so what we came up with

is what we call nibilung and all the Wagner affectionators of you are going to be very excited so we draw some inspiration from The Ring of nibilung obviously Wagner's big Epic from the 19th century it has its roots in the 13th century when nebulon was just a poem and then the German Romantic Movement and the 18th century made it popular and then it became very famous in the 19th century with Wagner so on a high level the story is about a ring and we'll get to that in a second with the architecture design and then Wagner's or in the in the poem The Ring is about whoever owns the ring has world domination or controls world domination we are not the delusional

that we think anybody in this case will have world domination but in the story The the power of the Ring obviously motivates and Boots cat is a catalyst for a scramble of both good and evil actors to get possession of this of this ring and that's the interesting story that reverberates into pop culture even today and I think what's kind of interesting about the story is the romanticist movement in Germany obviously made this very popular in the 18th century and Romanticism was a rebellion against against the enlightenment which was rational and I think that's kind of an interesting analogy for crypto and for

what we're doing here because in the enlightenment you would be very rational and this is somewhat irrational because why are we decentralizing this in the first place works just fine so but obviously um like our ethos like drives us to do something maybe irrational and decentralizes so that's why it's sort of rooted in Romanticism on a architectural layer this is Cassandra inspired and Cassandra uses an idea which is called consistent hashing so basically you have a hash ring and I'll explain this and the idea is any of the notes of the Cassandra database or of any consistent hashing

database Cassandra doesn't invent this and this model many other databases use this consistent hashing model but the general ideas that it's decentralized by Design you there's a virtual ring let's say that goes from zero to ten thousand and then it restarts and every node has an ID and that ID is hashed to a number that's placed on a ring so let's take node a with ID something that hashes through let's say the number 2000 and another note that hashes to 1000 and now usually you have a counter or clockwise

scheme in this ring let's say we have a clockwise scheme then the nodes at 0.2 000 would be with two nodes one or two thousand one or one thousand then order two thousand would be in charge of all the keys from and all the data from two thousand to one thousand so let's say I want to read from this database then I get a key so it's a KV a key value store I wanna I get a key I hash it also to some number let's say it falls at 1500 so now I know I need to ask node at place two thousand for this value same when I write I also hash it and I know which node I can ask so there's a couple more challenges here

like what happens when a node comes online or offline and when node comes online it gets an ID it hashes to the ring and then it can know let's say it hashes itself to 1500 now it knows that it can ask no the 2004 the keys from 1500 at some point in order two thousand is going to shed the keys from 1500 to 2000 and then there's a similar scheme going on when the node goes offline um I hope that roughly makes sense and so that's the the concept and because there's this ring it's decentralized by Design if you take a database like Cassandra they actually don't care about decentralization they solve different design goals such as notes coming online

and offline in a in a good and performant way um and Cassandra is not decentralized the way it's used because there's an orchestrator node and obviously an application that at the end of the day then calls this orchestrator node in our case we don't have an orchestrator node every node is the world connect application itself and has all the logic the websocket logic Etc that you saw in the previous slide and that's why we are fully decentralized and you can connect to any of these nodes so the instead of just the database package being on the on the ring like the entire application sort of is on the on the ring and to let the other nodes know whether they're

online we use a popular Library called lip P2P probably very popular in this in this room ethereum also and many other blockchains also use lip P2P for gossip basically to broadcast in a decentralized fashion which nodes are online which nodes are going offline we use a popular database called roxdb for storage it's a Facebook technology and Cassandra itself uses roxdb has many features Bloom filters and LSM trees for performance charge so it's state of the art and database we also made some contributions to it at World connect and we have some fancy things to even further push the state of the art of like Cassandra Cassandra for instance

uses Regional quorums in a data center we're gonna probably push the envelope a bits in the state of the art of databases if we use the latency based Quorum like to my knowledge nobody has done that before and we would provide better cross-region latency so um what are the Milestones here so what we're building right now is a permissioned decentralized database it's not a permissionless database yet so you need to get access from wallet connect to be able to run the system because we need to trust actors that they're not malicious but we want to remove this obviously and allow anybody out there to to run a node so this year we are battle testing those Network and what you just

saw and then we're currently designing how we're going to make this permissionless with a bunch of Partners you're also more than welcome if you're interested in this to reach out and take a look at the current design dock of how we're going to make this permissionless it has some novel requirements and problems to make this permissionless so for instance in blockchains you have a certain set of things that you need to have consensus on and but in our case it's different like you don't have we don't have to have consensus on all the data that pass through the system but probably we only have to have consensus on which messages were actually delivered and we need to know that because probably we won't be able to

just cryptographically secure the system we will probably have to have an incentive layer and in order to have an incentive layer we need to have the the consensus and probably reward participants so the question is what do we reward people on how do we find or how do we penalize Bad actors some ideas how to even detect Bad actors somebody use a snitch notes I think there's common ideas and how do we design this to be in line with the with the fallbacks and the drawbacks of the consistent hashing approach such as the consistent hashing approach other than other Technologies like demdia

which are also like ipfs users cademlia and which is built for notes coming on and offline all the time very quick in our consistent hashing approach approach we can't do that we get benefits for that but we can do it so we probably need to have some staking slashing layer or other ideas in order to prevent notes from from doing this and yeah that's it thank you again I want to say if you're interested in becoming a partner running a node then reach out or if you're interested in reading the design docs for the permissionless network also reach out and we'd love to hear your feedback do we have time for Q a or

[Applause] [Music] [Applause] if anyone has a question feel free to ask Derek well you have his time and his attention also ask anything about German Romanticism movement in case you didn't get it I enjoyed talking about that well awesome then how can be it heavy are gonna be the

nodes like what's the size of the note you're gonna turn it to so the question was what's the size of the node so there's different parameters that we're taking into account so one one thing is we want to scale CPU memory independent of scaling storage so what is the storage size and like basically how much data can a can a node hold and what are the CPU requirements and we don't really know yet so currently the centralized system that drives all of the current world connect traffic the CPU and memory requirements

of the actual nodes is very low and um but the the requirements on the the data storage are a bit higher so I mean we're shooting for very very little nodes at least in terms of CPU memory and maybe a bit more in terms of in terms of data but how exactly it's going to work we don't know yet and for the permission Network where we're going to control this and we're going to mandate certain size of nodes and we also don't know yet awesome thank you [Applause]