Whisper: the Multi DHT Messaging System with Routing Privacy. Vision & Roadmap.

Transcript

[Music] so whisper there's been a fair amount of misconception as to what whisper is what it does how it works and what it isn't um so a lot of this talk is going to be dedicated to explaining um the goals of whisper spread a little bit of background um whisper started as a project at least conceptually so uh about 20 months ago so um April

2013 um it was uh devised basically as an application um for uh doing implementing an exchange a decentralized exchange so you know around about the time Bitcoin became really famous in the price 2013 January February um I like oh yeah Bitcoin I remember that from slash. in 2010 or whatever it was um yeah that seems to be worth a bit more now um and uh I thought well why don't I why wouldn't The decentralized Exchange be great we got this decentralized currency but we got these centralized this centralized infrastructure so I looked into doing a decentralized exchange and just for is a sort of interesting um thought project

um wondered what a back end might be like um when I considered how it might be implemented um immediately of course I thought about uh what abstraction this was what what you know underneath underneath the application what was really going on and and is there a pattern there that could be reused elsewhere and uh that really provided the basis to to whisper at the time I wanted to do it as a pure JavaScript thing so the idea being that you would go to a I don't know GitHub site or something download a web page which had a JavaScript portion in it and the JavaScript portion would connect to a peer-to-peer Network something that we similar to what we now know was uh that

Alex described um in the last talk um and on that it would it would it would set up its own it would set up what what what I'm going to describe to you now and it would use web RTC in to connect all the nodes together so basically you'd end up with a web experience but it would be um an entire the content wouldn't be actually stored anywhere singular it would be washing around the pier network all hosted by browsers so that's the background coming to ethereum so fast forward 12 months and we uh what we're beginning to see is like yeah we got this blockchain we got true and complete

you know um consensus everything's synchronized everything's in lock step and actually with a 12C block time you can do you know reasonably decent Communications with it you know you can you can uh you could do a reasonably uh uh highish frequency exchange for instance um the the issue is that if you move state in lock step which is what a blockchain does blockchain provides you with a decentralized service to move state in lock step across the entire network um it's really expensive right to the point where a single transaction that would be done in microsc on a modern machine probably nanocs um is costs you know well we don't know exactly how much

but a probably uh no less than half a half a Cent and probably up to 5 cents um this is this is really expensive so we don't want to use the blockchain as the answer to everything blockchain is is an excellent answer to some problems but it's definitely not the answer to all problems uh certainly not the moment and probably not for the foreseeable future we need alternative ways of of doing Communications so when we think about massively multi-user I should say MMA um but anyway massively multi-user applications and when I say massively multi-user applications I'm talking about the sorts of things that we see implemented on the web um these days so

I don't know email Twitter uh but also going into things like U Facebook obviously going into things like uh MMO rpcs so uh RPGs where you've got um uh games based on the notion of having lots and lots of people all interacting with each other um so the sorts of comms patterns that we see are static content publication so where you've got a single piece of information that you want that people need to know about maybe some more than others but nonetheless um it's uh it's static it stays there for a very long period of time if not forever um another uh Communications pattern is

the uh the notion of a connection so the notion of two uh two people or two applications or two end points um uh wanting to communicate either unidirectionally or usually bir directionally um this is the sorts of things that you you rely on when you use a Voiceover IP application when you use video on the internet and then finally the sort the other sort of uh pattern that we see is this this notion of transient datagram so uh these are the things that might sit behind an email instant messaging thing where typing message on a chat um IRC sort of um Twitter's a good example of this so Twitter is kind of different to the rest because Twitter is a

publication thing but it's um it's a publication thing where the things that you publish are very small and uh the things that you publish also are um have an identity that you can then look up um and filter efficient so if we just go to the datagram pattern um we see we see some subpatterns within the datagram pattern so uh I've called them wholly directed going to a filterable all Global so uh wholly directed I'm I'm talking about where you have one or a small subset of recipients and the notion is that nobody else is going to see this so although you the tech most of us know that when

you send an email the server is not going to be encrypted the server on the way so the SMTP servers are probably going to be able to read your email unless You' taken explicit steps to encrypt it um but nonetheless when people send email they generally expect it to be private between sender and recipient instant messaging roughly the same um partially directed so this is where we've got the notion of multiple recipients or a a distribution list so use net subscription services and again the idea is that there is some privacy within this group so it's a bit like email again uset anyone can join it but there is a notion for instance on forums of a

password protected Forum or an invitation only group for instance in Skype and then finally what I've called filterable Global so this is where you've got um information is actually plain you know anyone one can join it there's not um nothing uh nothing special about sort of being able to read it uh no uh no restrictions on it but there's so much of it and you're only interested in a relatively small subset of that it needs to be filterable so Twitter's a a good example here yeah we use hashtags because there so many twitting Twitter feed Twitter feeds we use hashtags in order to um collate from each uh registry is kind of another example okay

so for static content um in um when we do massively multiuser applications in um in ethereum or the ethereum framework uh we have we have some piece of technology called DHT called it form whatever and Daniel's going to mention some of his thoughts on this after this talk for um real time Communications so connection oriented between one or between a particular pair wise set of identities um we actually sort of we have the one of things that we probably build into the P2P framework pretty soon is a uh an ability to sort of say right

I'm interested in this particular node set me up make sure it's one of my peers and I I'm then going to um actually Institute a connection this of course is encrypted because everything on PTP is encrypted uh but other than that it's a raw connection you're just sort of pumping data in and data comes back out but for the other this is what uh the rest of the um the rest of the use cases that's what this talk is going to be about so the other the other sort of thing that to some degree whisper is designed to solve or at least mitigate is um privacy and in particular rooting

privacy although there's been a lot of um a lot of focus on yeah everything should be encrypted https all the way yeah but the problem is of course that um a lot of the time the metadata gives away enough information that your your privacy is is severely hampered um furthermore if there is an attacker of some sort that is reasonably well resourced um by knowing which end points are communicating they can Target end points and and there are other attack vectors once end points are known and metadata provides this um initial information that can that can be used for such

attacks so really we we really want to work if we're truly serious about about guaranteeing our privacy we really need to work towards rooting privacy what's also generally referred to as this sort of dark dark connection dark internet dark whatever Darkness it's a bit it's a it's not a good marketing term I mean I prefer routing privacy but depends on the market well um okay so what what is whisper um so whisper is an odd combination because it handles a a selection of use cases and it's not specific to any one sort of

primitive communication primitive that we've already got and so it's difficult to really wrap your head around um in the first in the first sense I I originally called it a messaging system because it you know it can do messaging but actually that's and that's a relatively small Corner case of the full thing that it does so to really understand it you've got to accept that it's a hybrid it's a hybrid Communication System part of it is is messaging for sure so it in includes things like authentication and encryption in order to uh do certain provide certain guarantees about uh that you'd expect from a messaging system um but it it also um provides other services more

reminiscent of a DHT um that uh kind of make it muddy the waters a bit so I'm going to I'm going to a couple of slides I'm going to treat it from both of these uh both of these directions so that you can you can sort of see how it differs from each one um it's dark so there's rooting privacy this is this is done by probabilistic message foring which I'll go into a bit later and um uh one of the other sort of key things is that it's um it's oriented via um keys not um uh not destinations or Origins so this is something to bear in mind as you as you're looking at as you're thinking

about what this is doing so origin and destination really just it's dropped completely it was never a thing and this is why it really doesn't fit in with the traditional notion of a messaging system we don't they're not first class Concepts in in whisper subject key however is a first class concept and I'll I'll explain a bit more what I mean about that uh later so is it is it a DHD is it a distributed hash table kind of uh it's uh if you treat it as a DHT what we can say is it's a multi- key DHD which is to say each entry in the hash table doesn't just have one key it has many keys okay so it's like a composite key it can have

arbitary numbers of keys it's also multivalue which is to say um there's a uh not a one toone mapping not a on to many mapping between um uh the domain and the range so it can have many keys that are the same that uh that each reference different values um and when I say many keys what I actually of course mean is many composite Keys many of the same sets of keys and finally it's transient the entries are transient so they they they stick around for a bit and then they sort of go away I have an next an expiry date basically so okay it's kind of a DHD is

it how about does it fit any better with a datagram transport later a messaging system well yeah it's got the notion of broadcast multicast and unicast um they're not really first class entities in the protocol but nonetheless they're they're very well um uh supported um and each one through being a broadcast or a unicast or a multicast it's it's necessarily secure and authenticated and it's also um it's also asynchronous so treated as a datagram transport layer you can fire off a datagram you can go away you can you know go offline and uh you know there's a fairly large probability that uh it will be delivered assuming that your time to live is sufficiently large such

that the other node comes online before your UH your message dies it's got some features that make it uh um not impervious but resistant at least to a denial of service attack or a um a Spam attack so methods these these these measures so one of them is a proof of work attached to um a particular message or communication um record Another is is uh what I've call good behavior incentivization which basically means that peers peers should um try to forward messages which are most valuable to their peers and this is incentivized

I'll go into it a bit later but it's basically incentivized by saying right well every minute I'm going to kick off the pier that is uh noticeably least useful to me so if if you're all basically the same sort of usefulness then I'll probably keep you all but if one of you is noticeably underperforming we can do statistical modeling to get that then I'm I'm going to kick you off and replace you with either own peer chose at a random from a registry or one of my peers peers or one of maybe your peers I'll find another pier and I'll replace you next important sort of tenant of this is that it's um we don't deal with end points end points are deal dealt with at the PTP

layer so what PTP gives us is a is a mesh Network so we were checking out this earlier this is sort of comes into this whole how do you do a lookup to we don't deal with we don't care about ports and ipss that's abstracted far long time ago what we deal with is identities so when we say uh if we ever use the notion of recipient or or origin which we don't really as a first class CI but when we do um what we're talking about is an identity we're not talking about anything that maps onto anything physical because as soon as we get physical we run into problems with routing priv another aspect of whisper is it's

configurable so when we um when we take this notion of um privacy if we maximize privacy at least at the moment I don't know maybe there's going to be some super awesome improvements in computer science and they're just going to solve this completely but at the moment when we maximize privacy um we sacrifice efficiency um if you've ever used any privacy pre preserving um tools um tour freet whatever it is um you'll not you'll notice that you these aren't as fast as as the the services that they seek to replace so there's there's necessarily at least at the moment a a trade-off what whisper provides the user is to make that trade-off configurable

so slider bar on the one end private completely private nobody knows what it is that you're doing but it's slow and on the other end um you know maybe a sufficiently well resourced attacker might be able to have a guess at what you're doing but it's much faster um the next tener is that it's topic based this is when I was talking about subject key before so you don't watch out for messages that are addressed to you specifically that would again um compromise rooting privacy instead what you watch out for is the sorts of entries in this DHT or messages that are floating washing around the network that look like it's it's the sort of thing

you're interested in now how do you say well it's the sort of thing I'm interested in well there's an API for I'll get into that later but um basically each entry has a bunch of keys as I mentioned before these keys are derived from topics so they're actually just the hash of the topics and um topics are just arbitrary pieces of data that describe um something to do with the message now maybe it's only something that you know and that your intended recipient knows yeah so maybe for instance if you want to communicate with an intended recipient what you do is you hash your public key and hash their together with their public key and this gives you a topic and this topic is sort of a unique identifier that's unknown to anyone outside unless they

can guess that these two particular people do want to indeed engage in the conversation so if they know that you two want to engage in the conversation they' be able to get guess this but otherwise it would just be a random a random Ash a random topic other examples of topics are things like um application name with particular part of application for instance application name IRC particular part of application particular chat room ethereum Dev right uh taken together those are two topics um you could even hash them into one single topic and then you can then look out for these sorts of messages or put it another way they they get indexed into a global DHT and you can look out for any values that have

that index it's secure this is important this is something that you shouldn't I shouldn't have to say right protocols should be secure by default but unfortunately many many are not because they weren't designed with that in mind at the beginning I don't know maybe they they didn't evolve especially fast and now you know I have to point out that yes it's secure so when we get to the difference between unicast and broadcast so this notion of um I want to send a particular identity this information versus I want to send everybody this information or everybody who's interested anyway um it's the difference

between encrypting it or not encrypting it that's how it should be in email it's the difference between selecting two or you know not putting two in I really Mak sense for email but you you you get the point it should be uh it should be this it's not in present Technologies too often and it should be so it's this by default it doesn't have any other way of of differentiating between unicast and broadcast if it's broadcast it's plain if it's unicast it's encrypted and of course it's encrypted with PRI the public key of the uh intended recipient if it's a multicast we have uh kind of still working this out but the basic idea is to pre- distribute um on a u individual uh session key for each of

the individual um intended recipients um they then remember that session key and when this uh this this this message comes up they can decrypt it with a session key and if they want to send to everybody else they of course encrypt with a session key and mail it it's authenticated as well it's like this is another really important thing that should have been in there in the first place it should be there by default but all too often it's not well in whisper it is the difference between it being anonymous versus it having an origin is the difference between it containing a signature that's how a signature is the only way sensible way of giving it An Origin well no because there's

the plausible inability I'm getting to that I'm getting to that with the exception so the signature will be signature of something not necessarily the the the S um okay so what isn't it it's not Point too right we're not dealing with a point-to-point connection here that is a use case for some things but it's not what whisper is designed for that that's what some other sub protocol can do it's not high bandwidth what high bandwidth is important for some things it's less important for the sorts of use cases we have whisper for Twitter it doesn't

matter whether your Tweet gets distributed to the to the person who wants to read it in well once they found out about it in you know a microsc or 5 seconds it doesn't matter you're not you're not powering out through with 5,000 tweaks a minute um well I guess some people do but they're probably sad lonely people you don't want um it's not low latency it doesn't again it's not a big problem um for the sorts of use cases we're dealing with it's not application Level this is not designed for one particular application it is a an abstract uh Communications uh layer for doing a particular pattern of communication that we see a lot um but

that hasn't been thought of particularly at least in these terms most of the mmas myly multiuser applications that that use the sorts of things that this could address um actually use it via a server's actual table like using SQL or whatever they actually store particular entries in the in thing and then everything goes through um a server level piece of software which is backed by a database so this is is kind of getting to the point of a sort of a a fast real time distributed database but it's a fast realtime distributed database for some particular use cases nonetheless it's not an application Level thing so it's not an alternative to email so that's how it differs from

some things like bit message for instance um and it's not connection oriented okay so it's not there isn't this notion of uh of having an ongoing connection that can be you know you can do that so you can have a session key and uh you can set up a session it can be uh it can be placed over this but it it's a fit only for one or two quite specific uses okay so let's get on to rooting two roots to rooting I was tempted to say two roots to Route I think quite a shoehorn that um so we can either do it passively or actively if we do it passively we give

away much less of our privacy so passively um we were sort of Alex was was mentioning some of this in the last talk so this is where we have um we have this pier set these peers are just throwing messages at us throwing entries in the DHT at us and most of the entries are probably not going to be interested in so some of some peers will probably be closer to the sources of messages that we are interested in than others there will probably be um a difference in the distribution of the likelihood of getting a message that's interesting in a difference in that distribution for each peer so this is where we say right well if you're a peer that tends to give me messages that I'm just not interested in or maybe messages that that are old so that's another way of not being

interested in a message maybe it's you've already received it from another Pier if that's the case I'm going to kick you off I you know I don't I'm not interested in you from what you've got to say I'm going to move to uh this other pier and give them a chance so that's a passive way of doing it and eventually over a sufficient enough time and we need to do modeling of course to work out what this time period would be over a sufficient period of time you'll eventually arrive at a peer set that more or less probabilistically delivers you message messages that you're interested in the other way of doing it so that's kind of like Evolution the other way of doing it is active this is kind of like learning so when you know you got Evolution to evolve particular sort of um uh traits particular ways of

thinking about things and then you've got sort of learning to actually learn skills so it's kind of like talent that you're born with stuff that you learn so active is where you actually tell your peers I'm not going to kick you off or anything what I'm going to do is ask you to deliver this sort of stuff these sorts of messages this portion of the DHT stuff that roughly matches this pattern now if you do that you're going to be giving away a bit of what you're after to that Pier so what we do is we don't ask for explicit entries CU that's that's too too specific and we don't need to ask for explicit entries what we do is we say right here's a mask um am I going to yeah I'm going to go into this slightly does every when I say a mask who know who knows what I'm

talking about like a bit mask okay a few so the notion is that you've got a topic a topic is a uh actually I think it's 32 32 bits so it's got It's like0 one0 0 0 one one0 one he goes 32 32 of these zeros and ones I'm interested in a particular topic yeah cuz it's it turns out to be the topic of uh someone that I don't know is tweets interesting messages or a particular hashtag it represents a particular hashtag that I'm interested on the Twitter D um now I could say to my peers hey this is this is what I'm after this particular set of series of zeros and ones this particular topic now if I do

that they know I'm interested in whatever that hashtag is and that hashtag I don't know might be Smiley kittens and I might be really like I don't want anyone to know I'm into Smiley kittens so um what I uh what we then do is we form a mask now a mask is is just another one of these things it's another 32 bit 32bit um series um it's mostly zeros so it's like z z z z and then maybe a one and then lots of zeros and maybe another one and zero and all the rest Z and what this does is it says ignore these bits so these now get set to zero right we we we multiply them um but we ignore but what we do is we don't multiply the ones that are above a one

okay so now all of these are zeros bit wise multiply except for these and we provide both of these so what we're doing is we're hiding most of the information because they're all they're all set to zero so it's like getting a combination lock and then when you've locked the thing you're only leaving maybe one or two of the numbers there and the rest you're just resetting back to zero yeah and with only one of the two numbers left they can match it against all the various other combinations and you know maybe there'll be a few messages that match and one of them will probably be the one that you're after and that will get forwarded to you and it won't give away that you that you necessarily have to Smiley kittens because there might be another one that's like Fierce looking dogs and that's got that's a bit of a different

message but it still has a one in this position and a zero in this yeah a zero in that position the rest of the bits will be different but then your peer will never know which which one of the two it is so that's masks that's a way of telling your peers what you're after without giving away actually what you're after or increasing the probability that you'll get what you're after without giving away what you actually and that's a uh that's a way of rooting uh of getting things to be rooted to you what uh what they risk if they don't do this is is disconnection as I said before now how do they determine which messages

should come to you so I went over the interesting topics so you know you mention which topics you're interested in they tend to root those parts of the dhts to you the other two things that that that should influence whether you get forwarded a particular entry in DHT is uh the time to live we we we prefer to forward messages that have a shorter time to live because they've obviously got less time to be distributed across the network before they die and everyone drops them the other thing is the to make sure it's the highest proof of work this is this is an anti-spam measure so this is if someone's pumping loads and loads and loads of messages onto the network trying to bog it down well they won't unless they're very well resourced they won't have the um the compute power to

place a decent proof of work for each of these whereas the user that only has maybe one or two messages a minute can afford to spend a couple of seconds doing a half decent proof of work and that's what allows them to um to rise above the crowd now the masking and filtering as I mentioned before you're able to stack the odds the idea is of course that you here I mentioned that only two bits were set to the one and thus you know it's it's reduced from uh uh you basically split the amount of uh total messages that will come to you from all of them to a quarter of all of them by uh if there were only one bit set it' be half of all of

them um we can configure that number the more bits we set the more we narrow the search down the more the appearer knows this number can be customized per Pier or per application it might be that some peers you trust more you give them more information some applications may be that you know they're more sensitive maybe it's like the Wikileaks application or something and uh you you really want to have greater privacy for that uh for that application and so you give away nothing or this forms what I call a topic vortex in the network so if you view the network as a mesh you get your point is like a point of hey I'm after this topic your peers

know that you're after something like that topic the idea is that they take that and they consolidate it with all of their peers and the topics that their peers are after and all of the other peers they then tell hey this is the sort of stuff that my peers are after so I'm after this by by uh uh by by proxy and this is so you get sort of this notion of a transient mapping of uh topic uh filtering it becomes um reduced the further out from the center of the topic you go and of course it may be that the several nodes end up collecting in the middle in this topic Vortex because they're all interested in the same stuff and all using the same applications you end up with topic

vertices forming um yeah okay so how do you use it it's stupendously easy to use it has three functions the first one doesn't really do it just creates an identity right so new identity that's how you use it there's not much to it it's it's what it does it makes it it makes an identity it gives you the public key for it you never allowed secret keys of course that would uh that would be too insecure U but it remembers that you've created this identity and it it sets the public

herey aside and the sort of implementation part and uh whenever you want to author a particular message or hashtable entry it you author it with this key and it will sign it for you next one is post post as you might imagine that posts a message onto the DHT onto the system you can optionally specify a from and a two you don't have to have to if you don't want to just specify a from if you want that will make it a broadcast message just specify a two that will make it Anonymous you could specify neither and then it would be sort of this weird Anonymous broadcast that you know it's probably really useful for things like um well

things like Wikileaks yeah so anyone can come pop a message on nobody knows where it's from it's not directed to anyone in particular but everybody can read it and maybe you got a topic you know um uh thinkable too tired okay so you got your set of topics okay and this is a set not an array so the series is the the order is UN important it's just a set um this is actually just arbitrary data in in our web3 JavaScript thing it's the data arbitary data is prefixed with ox or it's a number um and what it does is it takes the hash and then to get a 32bit thing it just Chucks off the last 200 24 bits and uh you're left with

32 bits um the reason it's 32 bits and not the full 256 is firstly we don't really need 256 bits there aren't that many topics second a collision is unimportant and thirdly we've got data to save we've got space to be saved then you got your payload payload is again just unformatted data so again in the JavaScript API it's ZX but we have a bunch of stuff that allows you to convert from aski and stuff and I guess eventually there'll be ways of um converting arbitary binary data into this this thing but we don't have yet um the last two are the time to live integer number of seconds you know it could be days it could be just 5 Seconds it depends on whether you know it's important for you that your message

actually gets to its recipient if it's very important you have a higher time to live um and it depends whether you want the message to stick around or not so if it's something that you want like a job posting or something you probably want it to stick around for a day or two before you go online again and refresh it so that's that's where you have time to live maximum I'd say two or three days um but I haven't mention should be an expiry time because like you know you don't know later when that message was ini so a very good point but expir time is calculated as time of calling post plus TTL and that's actually put in the message itself so when the message gets distributed on the whisper network using

the Y protocol it does indeed include an um no I no so this is the user API rather than the uh rather than the sort of internal uh time St the user s it and that's it crushed I guess it's a future time and um well you know if a node that has a what would it be a clock that's further ahead of yours sends you a message that they think is valid and you say well it's further ahead you'll you'll just say right mark down this node I mean if it's only a couple of seconds ahead then it's like well maybe their clocks a bit skewed you know you

probably won't mark them down much um if it's two days ahead then it's like this node is really trying to pull one over on the network you know disconnect this gu is bad so you know it really it really depends I think actually I've written a proposal for the clock skew adjustment anyway so it should should really be built into the P2P Network level and then um nodes can peers can sort of um negot iate if one of them has a vastly skew clock CU it plays plays absolute merry devil when you're um when someone's mining with a a computer that has hasn't been corrected for summer time God it's an hour ahead and then suddenly none of the blocks work really bad um and then finally priority so priority is basically how much of a

proof of work do you want to do so it defaults to a a a small proof of work of like I 100 or 200 milliseconds so you barely see it in US level applications but um it's still Pro provides a decent amount of spam protection but it's the sort of thing which ultimately as the network grows you probably want to raise because there'll be lots of people who are trying to send lots of messages for TTL if you use number of blocks does that get rid of some problem is it problem um The Whisper protocol is entirely independent so uh the idea is that it can be used alone the final the final piece of the API puzzle is watch what watch allows you to do is to surprise surprise watch

for things that people have posted so when someone posts something on the whisper Network you get the notification that that it's happened now actually you'll get a notification at some point after it's happened and how long it takes for that notification is how what you know how efficient things are being rooted and therefore uh to some degree what's security level whether you put it up at a private in which case it'll take a long time before you notifi because you'll be filtering through you know loads and loads of of random other stuff or um uh efficient in which case you'll be telling your ndes what you're after it appears what you're after you'll be steering really fast and uh this stuff will come relatively quickly um again uh you've got the filter so the filter is the topics now this is a subset of the topics that the

that must have been posted with so it might be posted with like uh I don't know IRC uh IRC application topic one topic two might be chat room chat room name topic three might be um original origin so original name of poster right and it might be that you're in the chat room you're not filtering on the original name of the poster because you don't care because you want to go to receive everyone's posts uh but you are interested in the other two so you put the other two in here and when you're posting you put all three that way when you say actually I am only interested in alistar's posts because alist says really interesting things then uh you'll filter on all three and you you'll only be notified when those guys come optionally you can also specify one of the public keys that

you created with a new identity earlier now what that does is it says even though I can never tell who actually signed something assuming that they weren't using topics to tell me um I know from this filter that the the messages that come through will have been encrypted or to or will have been sent to me so being encrypted to me and being sent to me are the same thing in this it doesn't make a distinction so what you do is you say right I'm expecting if these are the topics that this will be the key that unlocks the data and it obviously unlocks it before it gives it to you so then you say right when something comes in watch that

change here's a function m is the message in here is the decrypted message so M now has things like data expiry um time to live how much work was proved all the rest of it topics you can also say right well what messages are there for this particular watch for this particular filter tell me all the messages that are arrived and this can well in future versions this may have a way of specifying um time date times from and two so I'm only interested in messages that came through from this point in time to this point in time and finally you can install your watch you get a change notification do you get the the H topics back or the

original topics uh you get the original topics so the topics of the message they're sent in plane they're sent plane text Cu they're needed for the routing and the security filter will just add random filters sorry the security bar will add random filters to your filter list uh the say again so you said it was like a slider bar for how much uh yeah add Rand filters so what it will do is these this number of ones that you include in The Mask will be fewer so at the maximum privacy this will all be zeros you won't you won't tell your peers that you're interested in anything they'll just take everything and then it's up to you to try to steer your Pier set to the peers that tend to give you stuff that's useful yeah when we watching you do you also set a minimum proof of work they

accept uh that's possible yeah so that that's it's not in the API yet but that's that's a reasonable thing to do as well my my immediate idea would be the the primary use dcem three three messages so I to say what was it to field don't ah yeah well the thing is that you have to know that the messages that come through are indeed encrypted with your public key now you can't two on its own doesn't filter anything all all

mentioning back all mentioning two does is decrypt it with your public key with this public key right here what it doesn't do is check first that it's that it's sent to you because it can't check because it's never mentioned it's completely opaque as to whether something is sent to you or not and that's why you use the filter so if you would if you wanted to do that wait the post which means that it's probably S no no it's not that's it so this isn't what this is also just the direction so this is this is an API right this what this isn't is the wire protocol the wi protocol doesn't include

from and through so what's what's it using it for it's just the encryption exactly encryption and signing so is it possible to send to the subset of people so several public you send to everybody uh so multicasting indeed so for multicasting you would set first set up um a subset of identities that you're interested in that you wanted to have have read things as a group or receive things as a group you tell them all a session key and then you would only encrypt not to each of them individually but when you want to send a message you encryp with the session key but it's still always the topics that the things that provide the routing No One Ever Knows who it's encrypted to unless they happen to know that through the topics so what can you see from the

outside is basically the topics yeah so if you're mishing the whole network but some of these topics tend to be aggregated to certain parts of like the mesh or the no graph and this changes over time but you can't tell literally what the topics is because of the Mask um you can't as peers advertise that they're interested in particular topics you can't tell what those topics specifically are well not unless this mask is all ones but assuming it's only a few dotted ones then can't you dve a hell of a lot of information about what just monitoring that over time as to you know you can you can note that something is common on Twitter at the moment and then map that because there's a certain

pulse of topics in this area and then like two months later it's another and then statistically sort of really pinned down the individual's topics and interests or not it's a possibility it's something that we definitely need to look into for modeling um and this is something I'll I'll bring up a bit later but um but the idea is ultimately you can you can ramp down the pier steering and you can ramp down the um uh the asking and in that case sure it will take a while before you receive stuff that you're interested in but it will you'll never it will never be anything other than plausible deny it's always PL and one more thing that if you're really interested in privy then one thing that you could do is you send so you you

somehow agree with your recipient in advance in a large range of topics and then you send to a square root of those of that range and they have search in a square in a different random square root and then one of those messages will be there so the external Observer has no chance so following from that there is actually a protocol that I came up with that allows a um this to so the basic idea is someone posts a broadcast they say right I'm interested in such and such um I'm I've got a chair for sale right um a someone who might want to buy the chair says right I'm interested in buying this chair I will send it to this this recipient um but what they do is

they um uh put it on so they they uh they put the topic in okay so now someone knows someone might know that there was a message being received but they can't read it in that in that topic in that message so it's now encrypted you put a subject in you put another Topic in that's just a random hash yeah and now both of the individuals both of the both of the daps if you like know this random hash and that's then used as the next topic and that's just the random hash no one can track what that topic mean what what it means at all and so obviously that can then be probabilistically

changed as the session continues um yep so what's what's the over head that I'm creating with form anonymities like I don't have a for that um so or like fresh discovery of a of some message that I'm interested in like how much time the honest answer is I don't know it's something that needs to be modeled and it's something that hopefully some of our modelers can can look into it some any um I would hope with a combination of proof of work a modest amount of um filtering um and uh peer peer steering

it's reasonable for the sorts of applications we're going to have on one point not and then um for two point not we can sort of do some proper research and work out ways that we this can be optimized further with the same API but I the API is the most important thing for 1.0 to get people used to thinking in this Paradigm and placing their applications in this Paradigm Alex has there been uh thoughts on um uh growting and you might you might have covered this I may miss it um preventing uh pack storms and um and also preventing the service

attacks now it's so obviously somebody can always um do a higher proof of work but then what if somebody just happens to have I don't know needs is really more proof of work than what is out there right now not necessarily than everyone else because it's only going to be more than who's currently using it so um so this person could just use up everybody's bandwidth and then and then everybody else it becomes expensive for anyone else to use and now it's hard to use uh yeah so if they can afford to um put more proof of work in than all the other

m is combined on the network then it's going to grind to a hole it's a it's a the question then is well how much proof of work is that and would anyone really be interested in setting up a supercomputer or a massive collection of nodes to do a denial of service attack on the network in that what what would they gain by doing it and you know that's a separate question as is the question of whether it's actually affordable for for anyone to do it to do this for high frequency tring I think uh I mean it's true but the the question is how much is there to be profited from it and is this profit um is this profit sort of the same order of magnitude as the the cost that it would that it would take and that's something that does need to be modeled I don't

have the answer and if the tated how are the so every message is broadcast to every Noe that to the no so the messages wash around the network you you all the you're aiming for every message to be broadcast at every node but that will never happen yeah yeah at least not within a with a small period of time so it's not that every node has you know will will have all information ultimately they're not building up to that what you're just sort of exchanging messages the messages are washing around and eventually uh you'll get something that you and and how do how do we

prevent from some message washing around but it ends up just washing around in a circle was um well if a peer a peer won't forward a message to uh the same message to another Pier twice so it won't go in a circle it might die out early um but that just that generally means that it wasn't given enough priority or it's time to live was too small so if I want to send you a message but I don't want to reveal the contents to anybody I want the message to be private then I have to agree with you on a topic but then that means that I have to have sent you a message already you know each other's public

keys can agree on a secret without without any secret communication that's the agreement all is it fair to assume that if I'm not connected to the right there's a good chance I'll miss of messages um possibly especially if you're not broadcasting uh in sufficiently um precise terms what your interested but um in general again it needs to be modeled as to precisely what what that limit is um it may be that for for the forceable

future it's not a massive it's not a massive problem because people have very decent very high bandwidth and um you're connected to a sufficient number of peers and the peer Network can be um spread across uh properly well well balanced that actually it doesn't and we you know it's just something that these three implemented a model is there any physical or biological process that this whisper is um most closely related to I mean some kind of combination between whispering and active listening um but anything else uh like it's not I me it feels like a forward backward diffusion I don't know weird it's an interesting question and probably not one I'm going to be able to

address uh in a tractable time right now uh yeah is whisper just a working name for the the project or um it was it was not a working name but it might become a working name given I know I know it's a shame I really like the name yeah yeah like small question um Regular expression filter somethingand a regular expression yeah I want to

listen oh I see um well the thing is filters are a binary well okay the topics that they're filtering are not but the underlying because the hash is taken the under line thing is binary because you want it to be cryptographically secure so what you don't want is for the uh by putting ones in particular places it actually to be clustered around um things of roughly similar topics so you hash it so that the the the things that is cluster the Clusters are just completely arbitary it might be Smiley cuts it might be face dos if you restart your de you watch for the same topic which received a message with a TTL that hasn't been expired yet or you rece again

next time you start your Des to store it somewhere yourself you won't be notified of its arrival but you will get it back when you ask for the messages assuming assuming that it hasn't been thrown out because it's not it's not believed to be interesting to any DS but this is this is something that'll probably be handled at the uh a level underlying where you say right well this D was interested in these sorts of messages I'll probably not Chuck them out yet at least not until the that runs again and gets to register interest the Cent persist yeah I mean there's a possibility a later sort of proof of concept or a later version where we can actually allow clients to say these messages I know that the TTL I know they're supposed to

expire in an hour but I'm actually really interested in keeping these forever and they take a sort of archive flag and they every now and again they have to sort of reassert that they their wants to be archived and then they get back in messages whatever the filter um maybe maybe you said it in but but I missed it so the topics that are in their filter are they basically are those do they all have to match are they are they end subet no sorry yes they all have to match but they don't have to match all of the top um they don't have to be a complete set of topics for the entry they only have to be a subset of the topics for the entry so it might be

that you provide two and the entry actually has three topic topics or four or five or whatever but it includes this topics the to be have to be a superet isn't a tag a better term because these topics are really tags uh yeah tags uh subject uh key uh you know yeah I had to land on more Alex is the um is the topic DED from a single hash or double hash a single hash is that a trick question make double hash absolutely if there's a good reason um so it's sort of

not a good reason but um when there's so at the beginning when there's not a lot of topics um yeah I don't know okay then well think about it and get back to me you do a pre-image attack on 32 minutes of a hash um pre-image yeah you just just create a whole bunch of like just big table of hases table yeah not so not keys but for if it's like a topic it's like um IRC or something

um oh yeah that's a good point yeah you probably could yeah that probably needs to be uh um yeah oh no that said if you got a rainbow you'd be able to do it with a with an actual hash it wouldn't get you anywhere because you would just you could do it anyway you could just search on the first 302 bits of an actual hash with okay anyway um yeah yeah that's that's one of the things that so something like Skype uh Twitter email that's that well I don't know but with email I

get Laten up to like a day so yeah probably Che I mean it's true again this this is a dark level so what this doesn't do is say it give it doesn't give you a direct line to me yeah okay so uh just I'll just get to the end and then oh oh sorry okay um example oh no no I've I've only got yeah

I've only got a few slides um okay so this is just a quick example so we make an identity there we go new identity we we what we're going to do is we're going to uh post a message to everybody broadcast message going to say Hey you know I'm online what's your name so uh what we do is we say the topics are just the application name and this is this is a string right this is somewhere there it's like the hello world application um the payload is my name so gav or whatever um and then the next thing is is uh what is your name right so it's just like the application is saying right well here's this is like a dock level protocol so the first the first field is going to be um so we can

we can make the payload I should say out of a series of 32 uh 256 bit um chunks of data and all it does is a pen so my name and then what is your name right and the idea is that on the opposite side the app will then the Dack will go o this is this is Gab he's after somebody he's after some friends okay so we'll reply to um and then we got like 100 seconds for the time to live and just give it a default priority of a seconds worth of proof of work okay and then what we do is we set up a reply watch right so this is watching out for these messages right the reply watch has a simp a simple filter um oh hold on uh uh right jump to the next slide we

set up a broadcast watch and this is a simple filter and we're looking out for anyone who's broadcasting with the topic of this application so the hello world application so everyone running this code will will be running it with the same application name so they'll all be um seeing those those posts that that happen and what we say is when something has arrived with this filter um we'll check to see if it's us so if m. from is my identity well we're not interested if it is cuz we're just going to reply to ourselves that wouldn't make any sense I we'd have to have very few friends for that to be interesting um and if it's not us then let's let's do something so what we do we got a new message so it's it's uh it's some we know that from this this topic it's someone asking for our

name so firstly we grab their name right that's what this stram does it just takes it takes their name from the um from the data we say oh look we've got uh we found a broadcast from this this this this this guy um they've told them we're going to tell them that our name now so what we do is we sign we we sorry we make a post we say it's from us that's that's our key right we put it to them so this this as a the data the bit that goes in the actual DHT will now be both encrypted and signed signed then encrypted so no one can tell it's actually from us um the topics are going to be the application name again and then also um

who it's from so this is the way of rooting it back to them right this is a super simple way because we're just putting as one of the topics their their ID right their public key so they can they can look out for their public key as a topic um the payload is going to be our name we are and then we we're replying so we know that they're probably well we don't actually know they're online but anyway time for the two seconds we we it's a transient thing yeah we just don't we don't care if they don't reply if they're not online saing it for a long time and then a bit of a priority half a second so how do we know in from I thought you said it was not part of the the message of broadc from is part of it from isn't

it's not quite part of it it's in the signature so it's it's retrievable from the signature so if you if you say it's from an identity it will sign the the payload okay so that's that's the broadcast um um post and the watch for it and what we can do is set up a reply watch and the reply watch says right well let's assume they got this and they've replied they were looking out for it and they've replied so we saw what that code was now we're looking at a reply which we're watching for the reply the reply is right well again our app name just to make sure that it is actually this application and my identity if you remember before we set one of the topics to this to the the sender's identity so now we're looking out for our own identity in the topics

um and what we're saying is here this two we're saying if it matches this filter so if if it's on this application it's to the filter in the topics include my identity then we are asserting that it was actually sent to us and thus we're saying to my entity and this is the only thing that prompts the underlying implementation to actually decrypt with our public with our private key if we didn't put this two in the data of the message would just be a load of gibberish because it it would be unencrypted uh sorry undec cpted uh data and then finally we say right when when it arrives uh we'll just we'll just tell

the user hey you know I've got a friend so this is the name so you take make okay um so things that I haven't really covered in detail mainly because they're probably not quite sort of finished or worked out the multicasting that's vaguely there's a plan there's a strategy might need tweaking but um Daniel has some ideas on adding to the uh the signatur specifically for plausable deniability which I don't know maybe give to if you want um and then finally as I mentioned before security privity and scalability we need to do modeling to work out how scalable the network is what what the chance you know how long it's going to

take uh for a message to generally go Network well formedness will be a play a large part in making sure that um that this is that we have a decent uh upper bound on those times um but basically how good actually is it when when it goes into the real world okay any questions um what's my incentive in choosing a smaller time it it cost more proof of work for longer no incentive is that peers will bu by um convention forward them because there's a greater chance of another Pier U liking the fact that you gave them that message so that the other peer that you forward too might increase your R and that's what you're as a as someone who wants to keep

their current Pier set you're trying to get your peers to um keep you not disconnect you so you w having incentivized to provide them with useful information um um said the the filter needs to be a subset of the topics of the message so in the example we could uh actually only use one watch call to get both types of messages um that is correct yes you're right yes is it to actually get the second topic from the message we receive in the call back it is yeah but the problem is with having a single Watch call you have to specify the two in the

watch call so the underlying layer can decrypt the message for you and one of them was plain the other was encrypted but you're right in the the um there was an issue with that that example the one that only had one the one app name Topic in would uh try to give the application um encrypted information so you would put another Topic in that said you know the the the initiation the broadcasting just so you could differentiate between the two y um I have a question regarding proof of B in this U hope because for blockchain I kind of see the the value of pro of work so we get consensus um with mind blocks and stuff and I'm not sure if I understood correctly why we

have proof of work in in bper is it just for um avoiding spamming and stuff you need to do calculations yeah just in order to keep volume down uh pretty much yeah it's a a a basic strategy against that it it's still precisely what proof of work is used again is a something to be tweaked the time uh why is it important to be able to choose the B The Mask why not just tell how many of the front bits do I yeah you could do um it's it's reasonable so if I have a public

interest on the blockchain and I decided to use that as my would that leak any information about who I was I was yes it leak all the information about so it would link those two now you might want to do that it might be that you've got an address on the blockchain and you want to prove to somebody offchain that you you are indeed associated with this address and yeah that you're rich or you high reputation or that you did do that transaction with them but likewise it maybe that you want a new identity and both are supported um if someone doesn't give you any information in their filter can you kind of like encourage them to give you information by making a whole bunch of messages and just sending them to them

and then they'll be like Oh I'm getting too many messages let me just you know give you a little ah very good point now this is mentioned in the wiki I didn't mention it here um one of the things that will need to be done to avoid this kind of attack so this is the attack oh I got a half decent name for it as well but yeah this is the attack whereby oh it's the yeah like a black hole attack so that are everything but attack so you give you know what they're interest or you think you know what they're interested in right because they gave you some sort of a thing so what you do is you send um everything but what you think they're interested in yeah and then you you're basically pushing them to them to tell you more and more about

what they want in order to for you to actually send the thing that that that they after and make messages are you're sending them then definitely not you're looking for uh say again you can just make up the messages oh yeah yeah that's it you can you can just make random messages so um how do you get around this well you can reasonably assume that uh the information that you have from the other peers I mean okay if you assume all your peers are are bad then you pretty anyway so so you assume that the other peers are giving you roughly decent information and then you do a statistical analysis on the traffic that comes from the other peers how much of it is actually useful how much of it isn't and check to see if that matches what that PE is giving you so if they're

doing a really sort of U um heavily uh uh conscious heavily sort of Tainted everything but attack you'll be able to detect it and if they know you if they know your peers then you're more vulnerable if they know your well certainly if they've compromised your peers you're very vulnerable but if they were to just send those messages to your peers they might send them to you and it might look like those messages are going through something to um this is slightly reduced by uh con Pier um steering so you just keep getting random peers sorry might missed it but when you you use the API where do you set your masks uh where do I where do you set

your oh right The Masks are done actually is an underlying level so the D never actually sets The Masks it's something that the user specifies in the interface um of the of the client like the browser or whatever like how much information you're will to and then they could you know there's OB the idea of a big slide of art but then you could also drill down and say right for this app I want more for this less whatever you could call it Zoro instead of whisper because of I'm asking oh uh we should have H we should have a competition shouldn't we Alex was asking about how how you manage that the message won't go in a circle and jutah was asking for some kind of biological counterpart and it's

it's very similar to the way like SP communicate using diffusion but normally when you use diffusion animals that use diffusion also has some have some kind of feedback to to say okay this this place where the diffusion coming there is food is there a way if I if I am connected to a pier and then I find a message that I I want from this pier is there a way for me to tell that peer I you gave me a message I want please bring me more from whatever your your source so so we can we can find this connection like automatically you could do yeah um the issue with that is of course that you're giving the peer

effectively you're telling that particular peer you're giving them all the ones you're saying right that message that that topic set there that's great give me more like that is is telling them this is the topic you're after so if you want to do that you just give them this this information in the first place you can get just tell me tell that peer that he he's he's a good peer Rel have good sources so you don't you don't reply after one message like you respond after a thousand messages in the past yeah I mean that's reasonable so you could do that as a as a peer rating so it could just be hey what's my rating with you on this protocol and you say right your rating is currently 10,000 because you're super awesome but but that's not really a rating all the

protocol it's just saying that there is a connection if there is a big big Network then the best way to find a connection between those spe is is to we're not dealing with end points yeah okay true anything else oh cool thank you e e