From Whisper to Waku

Transcript

so give a brief refresher of whisper for people who might not have heard of it so whisper is sort of this part of this back in the day you had this Holy Trinity with aetherium swarm and whisper sort of to compute storage and messaging and whisper sort of it it's sort of the messaging layer and it's it's cost based messaging particulars locators kind of ephemeral key value store and it has this sort of ambition to preserve some metadata and have this sort of darkness property as well as being a spam resistance to the use of proof-of-work it hasn't seen a lot of usage and it

hasn't been developed a lot I think the main users of it is most likely sort of the status app and a few reasons for this so first of all is using proof-of-work for spam protection which is quite bad for heterogeneous nodes so if you have a mobile phone it's it's quite expensive to still do some proof of work which means you need to lower the requirement and it makes it very easy for someone with a more powerful computer to sort of overwhelm the network it also has a bunch of sort of fundamental scalability issues that we'll talk a bit about more detail later but has to do with sort of the lack of routing how it's using bloom filters and so on this was no incentive to really

run a note and and sort of this darkens property and the anonymity claims is making or kind of not very rigorous and it doesn't correspond to anything in in academia and to the extent that it is sort of privacy-preserving in some way it sort of quickly breaks down in production environments and finally so the specification is it's there is one but it's not sort of machine readable there's differences between implementations and it's not clear exactly what guarantees it provides um so what's good about it well it's actually it's running a production it's running on top of ter PTP it is decentralized if you have this sort of topic based messaging system which can

you for desensitize group chat and public chats which is what studies are using it to for introduction it also provides some type of encryption if you don't have a layer it takes care of that and it does provide some privacy guarantees even though they are fairly weak so last year we were here at the panel together with NIM valid labs and web free foundation and back then a lot of talk was about sort of mixed knits and these extreme privacy guarantees and that's not what this talk is about and it's also not what vacu is about in terms of privacy guarantees and so on a whisper occupies this kind of suboptimal spot so

this is Fink old enemy to trilemma is four points here but essentially you have strong enemy T bandwidth and latency and you you have to pick tubes if you want strong anima T you have to sacrifice bandwidth or latency and then it's all fourth dimension is the user distribution which if you have more users in the system then you have sort of a larger name et cetera and whispers can have been spot where you have very high bandwidth usage but you don't get very much strong enemy T in return and because of this high bandwidth usage it also makes it hard to sort of scale the network and that also sort of impacts things um so that doesn't mean that we are not sort of interested in this and sort of these sort of mix nests that are being

researched by name and so on but it's a matter of order operations whether of our priority is to support the scaling of these protocols and it's really difficult to sort of and unsolved this transference or problems when it comes to running these protocols on mobile phones and then we'll talk a bit more about what we mean by research data devices here so yeah I said probably we're probably the only people using whisper right now which is status and that's suboptimal because status is essentially for resource to restricted devices we define resource restrictive devices as devices which have generally less efficient CPUs aren't able to have

that much bandwidth there's battery concerns memory concerns up tying concerns etc for smartphones for example the problem here is we're mostly offline there's very short connection windows so you can for example sync a DHT first of all that except effect battery usage it also is impossible to do in like a 5 second window that you're in a messenger app replying to a text message and limited bandwidth you don't want something like whisper which has a huge overhead where you end up sinking all this data over your 3G it's just draining on your device we wrote some scalability assumptions which Oscar will talk about when it comes to whisper sure so it

becomes this kind of meme that whisper doesn't scale and while the intuition is correct it's very useful to be able to UM send exactly why and in all this sort of change to polygons such a way that it could allow sort of a reasonable amount of users we sort of decided to make like a theoretical scalability model for whisper and I'm not going to sort of the details of all of this but essentially what we did is we just encoded best understanding of the protocol and how it behaves under reasonable assumptions and what this allows us to do is that we can sort of if there's something here that someone disagrees with then we can sort of reason about that in isolation and play around with the model and have a very quick turnaround time in terms of the kind of changes we want to make that

would have a big impact so we encoded things such as how many sort of messages are floating in a system how the bloom filter works so that we bloom for this work in whispers that used to get some kind of plausible deniability because you listen to more misses than you actually want sort of how many duplicate message you get how often notes offline and so on and by sort of encoding these assumptions we got this sort of model where essentially we have different types of cases starting from sort of the most naive and then building up with more layers in terms of things that we observe the things that we intuited existed in a real world production scenario so this

is a log-log graph and you see on x-axis its number of users in the network and in on y-axis you have receive bandwidth megabyte per second air per day and you see a bunch of the sort of graphs up here and very quickly quickly at like 300,000 years something like that you quickly run into very high bandwidth consumption which is a big issue on a mobile phone especially with limited data plans you see this line here that sort of the vacuum mode and essentially a lot of that comes from the fact that it's it's using topic interest as opposed to sort of blue filters correctly and that allows you because with blue filters you have this false positive rate and that's

sort of a function of all the messages that are floating around in the whole system and by sort of being more direct about that the system scales a lot better there's also a bunch of it change this that they will go through this take here where it increases again that has to do with how this status app is is using a whisper or wagyu at directly and it's all sort of the topics are partitioned but that's also deployment specific okay so now that we have this introduction on whisper let's quickly get into what whack you is I said it's a fork of whisper which is e IP 627 it's optimized for better usability for resource restricted devices so our main

focus is lowering all these bandwidth problems and ensuring that it's usable on mobile phones because I said our users are mainly mostly offline bandwidth constrained smartphones like we're trying to build this so that it's as efficient as something like Watts up how do we do this we add support for things like light nodes so there are mobile clients which don't have to reap transmit messages they simply receive the messages which are meant for them we have historic messages which you load from external servers or mail servers as we call them you do this by having your topic interest or bloom filter which you provide to the server and it returns the

messages which are meant for you and we implement rate limiting because proof-of-work as previously stated it's just it doesn't work like we'd have to lower the work required on a mobile phone so far or yeah we'd have to lower the work so far that anyone could spin up an AWS server and just start spamming the out of all of our whisper nodes the trade-off here is metadata protection versus scalability but a set the metadata protection which you get from whisper isn't very rigorous there's like no academic study on how much it actually provides so how we went about going or how we decided to do these whack you specs is we split up this back into three separate parts so we have the

whack you spec which is the main whisper fork then we have things on the envelope data format and then we have a spec for the mail server so these are three separate components one thing that was very important to us is that we formalized the specs more whisper doesn't do this right now so we have things like a B and F grammars which properly describe how the packets look what is it contained in a message how it serialized at cetera the entire spec is split up in capabilities so a whack you node can have multiple capabilities and nodes can choose which capability they support so not every node is a mail server for example and not every node needs to be like client friendly so you can choose which ones you implement

depending on what you need you can also choose which nodes you connect to depending on what you need there is explicit security considerations and implementation notes in the spec which is also something which is missing from whisper so that developers actually know what the security considerations are and how things are implemented or how they should be implemented with examples leading or from all the implementations there are of whack you what's interesting as well as we're compatible with whisper we do this through bridging we can easily bridge packets from whisper to whack you and vice versa and we use clear language which we also don't often see an EIP

meaning we follow RFC to one-one-nine which has very clear definitions of terms like must should may etc so that our specs are fully understandable where are we now on this so we've recently released our latest spec V 0.4 with all these necessary components which should be going into status relatively soon Nimbus and status go already implement this version there's more details on the actual specs page on which versions they support and what capabilities they support we made tweaks to expressing topic interest more dynamic and less error-prone so like I said this optimizes above the bloomfield

or this is an optimization versus the bloom filters we have a simulation supporting the third theoretical models that we've built which we use the Nimbus client for asset core implementation or a core integration is work in progress and it's expected to be experimental in the 1.1 release of status meaning that users can decide on if they want to use whisper or wack you and we make those trade-offs very clear on what the privacy trade-offs are etc there's we wrote a blog post about that and there's quite a lot of detail on that and like what we're doing right now in the future etc cold so we have this sort of theoretical model but that's kind of

living isolation and it's nicer to have some better sort of guarantees of this thing working out or not so we made this simulation which is actually running the protocol it's written in Nimbus and you have sort of these full notes and then we have whisper light notes or the previous protocol and a way of acolyte note and you see here that we have maybe ten thousand ten thousands of valid envelopes a flurry on the system and then attempt sent false positive rate which you see here so that means that you get a lot of messages they're not for you and then you have sort of one message for the backer clients there's a huge difference in terms of BAM will

receive bandwidth between the difference of modes from vacu to whisper light note and then full note you can also see that it's about to be invalid meses and has to do with the lack of routing and whisper so depending on topology you you're likely to get two messages both times so in terms of sort of ball necks we talked a lot about the first one this is the obvious one that he notes as a user which is to receive bandwidth for sort of mobile phones which is a sucks on a limb on a limited data plan the second one which team will talk a bit more about more soon it's sort of the no discovery because you sort of if you're on a mobile phone you

need to find notes I can help you stuff really messes to it and get relayed message from and then if the third one which is something that we haven't directly observed but very likely to to kick in a sort of tearful no traffic so even though you used to salt endpoints sort of dead their mobile phones and then you have all these other notes and you can find all these odd notes you still have sort the issue of traffic between these full notes because there's no real sort of routing and that's something that sort of in early work in progress and it's likely that in future versions will move to some more structured kind of routing some versions of caramely and so on but that's still

yeah multicom on that later yeah so one of the things we did so that we can move away from this Kadhim lea rendezvous like discovery bloat which we were using before is use VIP 1459 which is a dns based discovery protocol written by Felix long a which was done for disk v5 it essentially stores a Merkle tree within DNS text records and the nice thing about it is it adds authenticity to this so the Merkle tree or the merkel root is signed within the tree so we know if some adversary changed DNS records because we know that the signature is

not signed by the public key we expect it to or we can easily check whether one of these Leafs is actually in the root this is super efficient because asset we just need a query DNS text DNS text records which is a lot easier to do on a mobile device versus synching with academia DHD so what's next for us we're working on improving routing as previously said we're working on a separate piece of work which is we're trying to figure out better storage solutions this helps us clean up the mail server situation so that we use some kind of decentralized storage solution behind

that and we're working on optimizing that for all these issues we have with low bandwidth devices etc additionally we're working on incentivization so that nodes are actually incentivized to store other people's messages which is not the case right now we're also working on better spam resistance where we're looking into things like semaphore written by Barry white hat which is a zero knowledge proof spam resistance mechanism which we can probably use and beyond that or further down the line what we will probably be doing is removing def p2p and switching it to a lit p2p based protocol this will allow us to use things like WebSockets which will allow us to easily write browser clients easier mobile clients etc for

those of you interested we publish quite a lot on our site including monthly blog posts about our progress what we're working on along with all of our specifications you can find those on our website are there any questions ok thank you [Applause]