P2P Apps Deployed P2P

Transcript

Uh oh, there we go. Okay, nice. Hey, come on in. Uh my name's Dave. Uh DMC. We've made something really exciting. I'd like to show you. Uh, and we can play with it a bit as well if you'd like to engage or I can also just demo it. We've got an hour and a half. Um, but I wanted to take a little bit at the top to to break down what this is because it requires a a paradigm shift. Um, it requires groing that this is a different way to to build. This is a different way to

solve. Um the headline is no servers, no cloud, no masters build peer-to-peer apps. So four years ago we started working on uh a concept which is can we build peer-to-peer applications and systems that require no server infrastructure.

And it turns out that you can. We've built an app called Keat on top of on top of uh pair. Pair is the thing that allows you to build peer-to-peer apps. Keat has a million new users per month. It scales to the planet and the infrastructure cost is essentially zero. Compare that to Telegram with a cost infrastructure cost of 700 million a year. And you can see why this is a good idea just on practical grounds. We can build applications that currently exist.

We can build new types of applications that aren't even possible with a centralized approach that are real time uh that work almost identically to their equivalent apps but with no infrastructure cost. Imagine you're saving 700 million a year on infrastructure cost. Where are you going to put that? Right? It goes it goes to things like business strategy, marketing, compliance checks, all that kind of stuff.

So it also eliminates um a huge amount of communication because when you have infrastructure that requires people to oversee that infrastructure and that requires many layers of communication around that. You eliminate all of that. You eliminate a lot of noise with within uh an application's development and deployment. You eliminate that and you can with say four friends make an app that can

uh do something uh much high at much higher scale than you could have before. If you can build a UI and you can think a little bit in the peer-to-peer paradigm, you you can build things that don't have any infrastructure cost. If you don't have any infrastructure cost, you don't need to exploit your users by their data because you're basically taking a fire hose of user data exploitation and then you're giving that you're sending that fire hose straight through to pay cloud providers. So with this, you don't have to do that. And just on a basic human level, that feels right.

Here's how it works. Use a module. Deploy peer-to-peer. No cloud involved. That's it. We have a module. You take it, you put it into your into your project. It enables peer-to-peer over-the-air updates on desktop, on terminal, and coming very soon on mobile as well. So that means that you can deliver your application via the normal channels, but then any updates to that application come through as peer-to-peer updates. So you can deploy an application to billions of people around the world with no servers

and have peer-to-peer overthe-air updates today. This is open source and available and it's how we build. This is Keat. Um the beginning of this year we had an uptick in usage and uh whenever there's uh difficulties and challenges in a particular country uh and they start locking down communication and locking down internet access and different things we see a big rise in that country uh of usage of ke. We've not been marketing it. We've not really been advertising it, but as I said, we have a

million new users a month currently and growing. >> Sorry, what are the >> Oh, sorry about the confusion. Yeah, that one's velocity, I think. But look at the uh green line. I need to get a better graph. So pairs.com peer-to-peer embedded application releases stack. Uh, if you'd like, you can scan that. It might still work. If not, I can sort you out later, but we can connect on Keat and you can come into the pair development room. You might need Ke on your phone first. I don't think I've updated that QR code. It'll be on the bottom left of the screen. You might not be able to scan it, but don't worry.

There's not that many of us in here. We can we can figure it out between us. the stack. We have a library called pair runtime. Uh you mpm install that. This is a JavaScript thing right now, but the paradigms and the protocols apply to anything. The command line interface is pair. You use that to deploy your application. You can also use it to install applications peer-to-peer on desktop. Then bear. Yes, we have pear and we have bear. Uh the logo for bear is a bear, but it's not spelled that way. It's spelled uh in terms of bare metal. Uh and the reason it's called that is it's

a NodeJS like JavaScript runtime with no core library as a very specific design choice. If you have no core library, then you're not granting any capabilities from the outset. If you have no core library, then it turns out getting mobile support is much easier. So we we run uh anything that you that you need uh nativewise uh with bear you just install it as a dependency. So if you need to use the file system in node, you use the core FS module. With Bear, you use the Bear FS module when you install it. Um with that, uh it works on mobile on desktop terminal. we can uh deploy a single peer-to-peer backend

uh on bear on mobile desktop terminal and then the uh whatever view layer so it says react native on mobile and electron on desktop that can talk to the same uh code that runs in the bare back end on that machine. So this is really nice we found as an architectural choice because most people are used to the front end backend paradigm. So we just say okay well there is a backend but it's local on device. You talk to it over RPC and then that local backend does all of the peer-to-peer magic.

Then there's the bare module ecosystem and then there's the hyper module ecosystem which cont which is where we get all of the peer-to-peer capabilities. There's about like 2,000 modules on the whole punch GitHub repo that that this all extends down into. This all comes under the stack. I've said some of this already. I can skip it. Uh the pair runtime module. The only thing I'll say is it has a run method and that's for encouraging this architectural approach I'm talking about. You can uh mpm install uh pair runtime and then use uh pair run within an electron app or a mobile app and it will start a bare worker which is your

pitto back end. This is the workflow overall. This is I'm just giving you an overall image of this and then we're actually going to uh go through a little workshop and and play with the tooling. But this is the full picture. uh you use pair touch to create a pair link. That's that's basically a deployment target. You use pair seed to expose hypercourse to peers. So the way that it works is it's um it's next generation bit torrent called cadmia uh and that's seeded over a DHT uh by and then a topic the the pair link is actually a topic which is then discovered via the DHT. So when an

application connects to the seeded hyper core uh it's connecting by the pair link key which is a topic on the DHT which allows peers to define each other. Uh there's pair build which creates a multiarch deployment folder. So you've got you know Linux, Windows, Mac, you put those all in one multiarchc folder and then that's what you actually deploy. pair stage synchronizes the uh files on your disk into a hyper core. Uh that's that's the equivalent of the staging environment.

So when you stage you've got you're actually at the point we're in the workshop we're only going to take it as far as stage because that just shows the functionality. So, but when you stage, you're actually at the point of doing peer-to-p peer deployment because by synchronizing from disk into hypercore uh and seeding that to the network uh you're exposing uh uh uh the the builds, your user installs the build, it connects to that uh to you as a peer. And when you do when you stage the next time and you do an update, the length of the hyper core increases and then that's that is uh replicated to the peers. The peers receive that as an update event

and then they can click a button and it applies the update does a swap on disk. That's how it works. Pair provision, sorry, I wasn't sure if it was a question but it was a yawn. It's all good. Pair provision is a pre-release step and it it basically strips down the uh the hyper core because every time you uh stage it if if it's doing a deletion then that's actually an append. So when you're staging to a hyper core it kind of gets bigger and bigger and bigger but when you do a provision uh it's it strips out all the intermediate steps and it's basically like a snapshot block sync right? So that that becomes a pre-production drive that that link that

pair link uh you you you you give to like QA teams and uh uh that that's that's where you're getting really really serious like we're about to go into production. This is how we this is how we make ke and then there's pair multisig and that's the production step with pair multisig you you have uh this this pre-release key this pre-release pair link that that you've that you've been qaing and all of that and then you have stakeholders who are responsible for what's going out. So say you've got like five stakeholders and then three out of five have to sign in order for that to actually hit production. So all of your users are using the multisig production link in their application.

Uh and the the three out of the five say in the if it's three out of five quorum who sign each one of those people who sign are saying I'm putting my career on the line to say there are no supply chain attacks. The diffs look good. I don't see any bugs. I don't see any vulnerabilities. Right? It's a very thorough process that is then signed off cryptographically with your public key and people know that that's your public key. We've developed that process over the years because it doing a peer-to-peer deployment, it's really really important that it's done right

and then it's done seriously. And the pair multic command really uh hones the focus. So that's that's the full flow of how you deploy uh a pair app. It works like this. You have this dual distribution. Uh so you do your builds like you normally would. You build your mobile app, you build your desktop app, whatever. You put it on a website, you put it in the app store. The user gets it, they start running it, and then it connects peer-to-peer. From there, it's peer-to-peer updates.

We've talked about Bear, so I'm going to skip this. This is the Bear ecosystem. I I don't know if you guys have used Node or whatever, but the things in green are basically the equivalents of of Node Core. And then there's a bunch of stuff beyond that as well, like bare f, which we use in Keat for um certain things. Then there's the hyper ecosystem. Uh you have the bedrock primitives. I've already mentioned hyper core. Uh you also have hyper DHT, which is peer discovery. And then you have hyperswarm which is peer discovery via topic. Then you've got the data structures uh hyperbe, hyperdrive and autobase.

Hyperbe is a key value store. Hyperdrive is a peer-to-peer file system and autobase allows you to do multi-writer. The really cool thing about autobase and this is what we use in ke for conflict resolution of messages is it has this apply function. So everyone in if you're on ke and you're talking to someone else you and that person both have an appendon log. I write a message it appends to my append only log and then uh you receive that message it appends to your appendon log right uh but it goes through this this linearization process via autobase. So the autobase runs that through an apply function and the apply function determines whether it's written and how it's written and

when. Uh and with that what you actually have is tokenless smart contracts. Just something to think about. Um storage and sync core storage is for managing multiple hyper cores. Local drive is the equivalent of hyperdrive but for disk. That's how we do a stage. We just we just have a local drive that mirrors to a hyperdrive. Uh which is what we use mirror drive for. So like I said, you have these deployment layers. You have this build, you have this build flow, you have a deployment directory which is your multiarchc folder and then you have uh the the staging uh layer which you can

just pair stage to. You could have a shared staging key. You could also just touch a new link stage to that give it to a colleague and say hey can you just check this over or like I'm struggling with this or whatever. that's really really easy to to create variations internally that no one else ever looks at even internally just between you and someone else. But then you can also have like these are the official links between us. So we have uh we have a development stage key, we have a staging stage key, and we have a release candidate stage key. And it goes that's that's an an additional process that we have where we're going through each uh stage until uh once we get to the RC key, it it stays on there for couple of weeks and then once we're like happy

with it, we we do a provision and we we create the pre-release key. the pre-release key is like like I said a very very serious moment like at that point we have QA check the RC but then QA check the pre-release key as well if there's any problems at that there shouldn't be any problems at that point because what you're saying with the pre-release key is like if we' have gone to production today right now this would be it right and then uh once that's been verified again then you do the multisig and you go into production uh this is just a release cycle every time you uh we're going to look at some uh the hello paralleer boiler plate. Uh but the way but basically every time you version uh every time you bump the

version that's uh to create a new build and the updates are based on the version increasing. So you version it, you make the distributables, you build the deployment directory, you stage, you provision, you prepare the request, you sign, you verify. Once you've got the signatures, you commit, you're in production. So, uh, we've got different quick start guides. The one we're going to be looking at is the Electron boiler plate. Uh, don't worry, we'll come back to this. We also have a, um, bare boiler plate, which is for creating, uh, peer-to-peer uh, terminal applications.

So CLIs, ripples, dammons, services, uh uh uh hooks, um whatever whatever you can make in the terminal, you can now make as a uh application that has peer-to-peer overthe-air updates. Pair Pass is a product that we've made which is a password manager. You can't leak clear text passwords if the thing doesn't have a server. like you eliminate all of that and you just have it between your devices. It's about personal data custody, right?

UAC is a uh uh local LLM uh project. There's a lot of uh optimization for running LLMs locally. This has a really nice parity with uh our peer-to-peer stack. Uh one of the things that QAC can do is remote inference. So if you have a if you have a nice beefy machine with some LLMs on it, you can use QAC with our stack and use it on phone for like the easy stuff and then offload compute for the remote inference and of course that's all completely private and it's all completely your machines your data and then of course Keat and like I said we've we've discussed that uh if anyone wants to uh connect with me on Keats uh we can uh later on uh same for that uh

there's a news uh pairs.com news. Check that out. Uh docs.pairs.com to learn how to build and careers.te.io because we are hiring like actually uh a lot. Um we need people of all kinds, frontenders, backenders, cryptographers, whatever you are, we're probably hiring. So, uh have a chat with me if you're interested in that. Um I certainly would be interested in that. So, um, any questions before we get into the workshop?

Yo, >> thanks. I really like what you guys have been since the beginning. >> There is one thing that might >> Yes, thank you. Uh there's one thing that might be uh stupid question based on what you just described, but um I'm building an app that has quite a few users at the moment, but in order to make it as accessible as possible to the general public, it's a web app.

>> Uh it's a web app. We try to keep everything local on your browser and when you need you sync with end to end encryption, etc. Are you thinking of introducing anything in uh in your tech that could be applied to web apps like mine that try to maintain the best security and privacy as they can? >> Yeah. Uh so that's that question has been answer asked many times. Um and the answer is unfortunately always the same. Browsers are controlled by specs.

Specs are controlled by organizations who have a vested interest in remaining centralized. In order to do the kind of peer-to-peer that we do, you need UDP. Browsers don't have UDP. I don't expect they will ever have UDP. It's not a technical problem. It's a political problem. So our response to that is to go 100% on apps. However, you can just like like I said, you can create like a a bear a bare process with like OTAA updates or whatever. So you could

create a server that's a portal into a peer-to-peer landscape and then progressively enhance from there. So you could have a web app that gives your users as much as you can give them and then after a certain point when you have the scaling need or whatever whatever that feature is then you onboard them to the app which is a pretty normal flow. Um and that's we haven't decided to do that for us because Ke is a messaging app and it it it doesn't necessarily make quite as much sense as as a web app then. Um, but I I do think that there's other uh other types of apps that you might build where you you you're going to want a

portal probably. >> Super. Thank you. Uh, and last question. So, we're indeed also preparing some desktop and mobile apps. Uh, so slowly taking the users into something that's much much higher friction. Um, but for us it's uh it's document collaboration, spreadsheet collaboration. So, you have 100 people editing the same stuff. you need CRDTS etc. Uh do you think that at the moment uh uh what you've built is already at a good stage to handle applications that require I don't know a thousand edits per 10 seconds by uh so many sources >> 100%.

>> 100%. uh not that it's easy uh but uh the the key is that every edit needs to be uh you what you you don't store the latest state you you store every edit that's made and if you do that then you've got undo and redo and all that built in uh but you're taking advantage of the append only append only log uh and then the autobase apply you would need autobase for this for sure and then the autobase apply function function would then be deciding who wins or it might even have you can you can do it any way you want. You can even do it for certain situations where it sends it back to the user and it makes the user decide who

wins, right? But there's many ways you you can do that part. Uh as long as as long as you're not trying to store latest state, as long as you're actually storing every single edit, then 100%. >> Thanks. >> Sure. Any other questions? Yeah. I mean >> oh >> thank you >> thank you so yeah super interesting first of all um who's running the DHT server nodes um maybe >> so uh every pier can be a bootstrap node >> but uh you need the IP addresses of the bootstrap nodes so the ketap for

instance has a a couple of bootstrap nodes hardcoded >> and and we run those, right? But if you're making your own app, you can set up your own bootstrap nodes and and run them or you could make it so that your users could supply. >> Would these networks then or these DHTs overlap? Is it like one giant DHT where all the apps would register themselves or is it does each app have their own? >> By default, the first but if you'd like to customize the second, >> right? Okay, got it. Um, and how's the research usage of a DHT client on on mobile? So, this has you >> how do you mean?

>> So, for example, there's so back in the day, I'm not sure if they're still around, Bertie, the they used to run a DHT client on mobile devices and what they were suffering is like they this was just draining the battery because you had to keep the routing table up. >> Yeah. Um, so, so, uh, just I'm going to answer this in a roundabout way. >> Mhm. >> Uh, because it hopefully it makes sense at the end. So, um, if you're not online and I send you a message, you can't receive that message because it's peer-to-peer. But if we have blind peers, then those blind peers can receive the messages and forward it on to you encrypted, which is how we solve the availability problem. But, um, mobiles will never be blind peers. So,

they're not not every peer has to be the same. mobile will probably always stay as client peers >> and by doing that you reduce a lot of the battery and energy cost and all of that but it's not you know it's not a foregone conclusion it's it's still there's still definitely some trade-off there but I think the cap well I know like because I use keys all the time the capacity is sufficient that it works >> yeah I thought I thought so that it's it's fine I I I guess um what else do I have here ah um yeah what whole so I I mean P2P networking is hard. So you're probably applying some whole punching techniques. I guess >> when we first started doing this, we called the company hole punch

>> because our secret source is the hole punching technique, >> right? >> Um I can't speak to that so much because that's the bit that I that's where I came in once once that was built, >> right? Um but as far as I understand it, it's a statistically guaranteed method to opening the uh the gates on the net routters and uh from what we can see it it it it works very well. Uh which you can see in the data because a lot of those million new users a month are actually coming from Russia. uh we have some conversations with uh people inside China with having conversations outside of China. So um there's uh it's effective

>> and then but it's not 100% effective for the like 1% that it's not then it goes through uh a blind pier actually but as a relay. >> Exactly. Yes. This is exactly what I wanted to know. So what the fallback mechanism looks like. >> Yeah. It's just a message so it works via a blind pier as well. >> Yeah. Sure. Um you you so you mentioned UDP earlier um is so does pair support like TCP transport or plus quick or is it are these just raw UDP datagramgrams that you're sending around? >> It's our own flavor. It's a it's called UTP I think which it's just UDP plus. So it's it's it's TCP like but it's not TCP because it's optimized for what we're doing.

>> Okay. Um but bear supports TCP. I don't know if it supports quick, but you could write a native add-on just like easily, right? >> Is this all written in in uh JavaScript or is it just a shim or like a >> JavaScript and C? >> Sorry, >> JavaScript and C. >> JavaScript and C. >> So, uh we find JavaScript's really good at um rapid prototyping, rapid iteration. Um, we use it. We obviously we have practices and and have a pretty strict subset, but uh it's fantastic for for getting there quickly. It's fantastic for on on boarding people. Uh it's such a commoditized language. Uh but all but and then with pairing that with C as

well. So C is for like the native add-ons in Bear and then you have this thin, you know, JavaScript layer on top. Uh >> yeah to me just sounded like an unintuitive unintuitive choice of yeah using JavaScript for like s such a system low-level system um uh problem to solve. Well, but you mentioned C as >> Yes, there's C. But I I think for something like this, JavaScript as an invented language is excellent for as a med for the mediator pattern and in peer-to-peer it's all mediator patterns, you know.

>> Um so so it I think it works really well particularly for the mental model. Um but we are uh as we move forward like more and more of the stack is being moved into C right >> because of efficiencies and and uh cross cross compat and things like that. >> Cool. Okay. Uh maybe one last question. So is there um some so you may have already touched on this but in other P2P libraries there's next to DHT which is used for peer discovery also some gossiping protocol.

Um, is there anything like one of a module that supports any kind of gossip? >> I a little bit out of my area, but I believe Cademia is responsible for the gossiping. The Cademia, it's a it's a next generation bit to algorithm that we use like right down deep in the stack. You can find that under hyperdh. >> Okay. So the the DHT is just a key value lookup thing. So if you want to disseminate a certain message across within the entire network, there are like probably other protocols. I will I will look it up if there's anything. We have something just on that just as a side note we have something really really exciting coming that's not been released yet still in the works but um search via DHT as well.

>> Interesting. Okay. Okay. >> So that's that's a global search engine at that point. >> Good. Okay. Anyone >> any any other questions before we get to the actual workshop portion? >> No. >> Okay. I would have yet another one then. >> Okay. >> What did you want? >> Okay. >> We can take one more and then one more if you want. >> Right. Okay. If if yeah um the hyperdrive that you mentioned is this in any way content addressed is content addressing take is playing a role in all in in this whole stack that you mentioned. >> Uh when you say content addressing you're talking about content hashing of the content. So we so pair links are um a standard.

>> Mhm. >> And it's uh the pair protocol and then uh a key string but it can also be the pair protocol fork.length length keystring and then it can also be the pair protocol fork.length keystring uh hash content hash or something like that. Right? >> So we have we have all of the forms and if you if you want to use the full content hashed version you can. But with the fork and the length uh you you you're actually you're pointing to a very very specific thing in a hypercore at that point. So there I think there are probably like if you're ultra paranoid you can use the full content hash but mostly we just use forklength.

>> Okay cool. Good. Thank you. >> Yeah. Sorry I I missed the beginning but uh what I I've heard is that you're making a a distributed hasht kind of storage >> sort of but more much more than that. It's a it's a way to build applications at planet scale without servers. >> But how does that differ from from something like swarm or >> swarm as in DHT swarm?

>> Ethereum swarm the >> right. Yeah. Uh not not too keed up on that. We have uh I don't know can't compare them. Uh but there's no tokens involved. Maybe that's the the key point. Thanks. Uh stick around for the workshop and we'll get into it. Okay cool. So, if you'd like to follow along, uh take a look at github.comholepunch2.

Uh, wait. Where's it gone? Pear workshop. Let me put this in um bigger letters so it's easier to see. Everyone got that? Anyone need more time? Yeah. Cool.

Okay. So to do this workshop you need to have uh node installed if you because we use npm. We don't use node on with pair actually bear is everything but we use node and npm because those are the things that people use. We we don't use node, sorry, we just use npm, but you need node for npm because that's what people use to install dependencies. Um, so if you don't have it, you can get it uh on Mac with nvm.sh or and Linux and on Windows you can get it with NVS. See it in the workshop. Uh oh, good. To install pair, um, you just

run Oh, shoot. Hold on. Let me sort myself out of it. you run MPX pair. Now, if I run that, it's going to do something different, I think. Um, you know what? I'm going to remove pair from my system just to show you how to do it. So this is this what what this is doing is it's uh executing uh an npm module that downloads pair from peers

uh and installs it onto your system. The pair itself is actually a bear program. Uh, and the next the next version that we're going to release is also going to be using the same um PPOTA updates with bear approach that we uh published recently. Once you have it, you should be able to uh you're going to you're going to need to export you're going to need to modify your path. Um you can do it by running this command here. I don't need to because I I've already done it. And then if I run pair, I should see some help output.

Come on. There we go. Um, that's a Mac thing. That's not a pair thing, by the way. When you put a new binary on Mac, it actually does a call back to Apple, I think. Um, so this is this is the uh the pair uh help. You can see some of the commands I talked about there. Touch, seed, stage, build, provision. There's some other cool stuff. Info dump is really cool because you can just you can dump whatever you staged. So it's it's synchronizing from your hypercore back onto disk. Uh change log is really cool, really fun. Um all right. Anyone need help getting pair

installed because it's, you know, not as easy as it it as it could be? Everyone good? All right, don't forget to do this part. If you don't set up your path, you can't you can't run it because it's pointing at a custom bin folder. In future, that's going to change. So, once you're set up, uh you can go to the first exercise. Uh, and what we're going to be doing is we're going to take the boiler plate.

How much longer do I have, by the way? Like uh 45 minutes or so. 50 minutes. >> Well, officially the workshops end at 3:45. >> Great. Perfect. Lovely. Thanks. Okay, so we've got this boiler plate. Hello, Parel Electron. And we're just going to get clone that into a chat app. So I'm going to go get clone. Oh, did it twice.

Get cone. Hello paralleer into chat folder. Let me make that a bit bigger. And then uh we're going to do a uh run this command. And what that does is just sets the package JSON name uh the package JSON product name and the package JSON version. Um the product the product name is what's used when we actually do a build. So typically it has an uppercase letter in it and the package name is has to be lowerase. That's why we do that. And then I'm going to mpm install.

Uh while that's happening let me talk through the boiler plate. So the boiler plate is uh the the read me of this boiler plate has the full explanation of how to do this end to end. Um talks about the architecture, talks about how the updator works, uh talks about where the storage lives, how you can set your own storage flag, which is important when you're developing peer-to-peer applications because if you it's all about where it where something's stored. So if you run two applications locally with different storage targets, then you're you're simulating a peer-to-peer interaction locally that they actually go they actually go and connect outside uh and then whole punch back into yourself. Uh when you when you that's why that's important. Uh the workers

which is the backend stuff um explain it talks through the deployment stuff I've already been talking you about. You can see some of these diagrams are quite similar. So, uh, this is this is something you can work through like end to end to get to a full production deployment of, uh, a par app. Uh, now that's installed, I'm going to do mpm start. Oh, my bad. I missed a step. So, the package JSON has an upgrade field and it says your key here. So, uh, we need to we need to give it, uh, a key that we're going to stage to later to do peer-to-peer updates. So, I'm going to run pair touch.

And that's given me a key, which I'm now going to take, and I'm going to set on the upgrade field like this. Uh, and now MPM start should work. Great. So, that's launched uh an Electron app and it just says what the version of the app is. I hope someone hasn't died.

That would be terrible if they have after saying that. Um, so that's that exercise two. Now we're going to make the actual chat app. So um what we have here is uh a little diagram of the flow between the pieces of the Electron app and a remote pier. So the Electron app it has renderer app.js. It has a preload.js which is actually part of renderer. Uh but it goes through this context bridge thing.

It's Electron stuff, don't worry about it. And then there's Electron main.js JS which talks to the renderer and then there's workers main.js which is bare runtime. That's what's running the actual peer-to-peer code, right? That's all talking over uh IPC RPC. Then from that it's doing a write to the peers uh via Hyperswarm. Every time Hyperswarm uh detects that a new pier has joined, you get a pier object and then you can listen for data on it and you can write to that object and that writes to the pier. So uh so we're getting the data flow is going from the

front end UI the web view over to electron main over over to the bear worker over to the remote pier back from the remote pier and all the way back into the renderer. So to do this we're just going to install some dependencies. We need hyperswarm for the dht uh hypercore crypto and uh uh b4a which is just a a buffer manipulation uh dependency. So I'm installing those now and then uh copy paste. So we're going to take workers main and we're put going to put

this code in it. You can see here that we have a hyperswarm. We set it up con swarm equals new hyperswarm. When the swarm gets a connection, it gives us a peer object and then we're listening for data on that pier and then we're calling our uh send function that we've made with uh this object. The send function is here. Uh and all it does is it writes to uh this pipe um the which is a uh frame stream over a socket essentially bear.c is the socket when you start bear

as a worker bear.c IPC is the is the the socket to the parent. Uh and then uh when data is coming in, we take the message uh and then uh depending on the message type, we either join a swarm uh with random bytes or we join a swarm with provided bytes. So that means like that's the same exact flow from there for creating a chat room or joining a chat room. And then uh if it's a type message, then we loop through every pair that we're connected to and we just write the message text to them.

Uh and that's that's it. That's pretty much that's that's the whole thing uh in terms of all of the peer-to-peer logic for a ephemeral chat app. There's no persistence here, but that's the whole that's all that's everything. So I'm going to put that into uh workers main.js JS like so. Oh, wait a second. Someone's updated this. Uh, I'm going to keep the um what we also have in workers main right now and it's a good thing but

this needs updating in the workshop um is the the updater is running in the worker. The updater used to be running in electron main. So um I'm going to copy paste that code but it's going to go between these lines here. Hopefully this works. I'm not sure if it's going to work because it's a bit Have to debug it if it doesn't. That's fine.

And then uh got some HTML here. Uh renderer index.html HTML which we're going to uh just replace verbatim. All it you'll see it all it all it is is uh some chat stuff uh for joining and creating rooms and then uh renderer.app.js will give us the um the glue code for receiving uh data from the worker and and vice versa. making that work with the UI. So that is renderer.app.js update.

Okay, I'm not going to prove updates for the chat app. That's just not going to happen. Uh, right now that's done. npm start. Now then we've got a chat app and then if I create another process mpm start with a storage flag. Oh, that's the wrong wrong um now I have two instances of the chat app

uh with two separate stoages. So it you could also if you've if anyone's got it running on your you can also you can also join but uh let me know if you have if you have it there. So if I create if I create a room here. >> Yeah. So when you press create it it joins the room with uh the random bites. Uh so we're creating a topic on the DHT join room. That should be happening faster. But I've just realized I'm on my uh uh I need to get on. Let me try on Dapcon guest. Uh, and then the idea is that you then join

that topic on the other one. I don't know why that's being weird right now. Let me try creating one on this one instead. Cool. Just checking. Yeah. So, the the boiler plates been updated and the there's a little mismatch here. I'm gonna have to debug this real quick. >> Does it? >> Fantastic. >> Yeah, you didn't do the little thing that I did. Let me just do that and see if that works.

Okay, cool. So, I've got this topic and then I join that topic and now I have Um, are you on KE at all? >> So, I'd share the link with you on Ke. That's the thing. You have to share the topic out of band some way or another. Um, but if I gave you this topic, it it would work. A lot of the time we do this uh uh as a remote workshop, so it's really easy then because you're like in a Zoom call or something, you know, but trust me, it works. Um any any questions about that about that build?

>> What the bearipc part I have suspicion that maybe okay thank you. Yeah good. How does the bearc IPC thing work? I said I have a suspicion how it works but maybe you can explain it again. >> Um >> um so uh electron main.js JS right >> uh we have pair runtime run and that what it's actually doing is it's running uh bare sidecar

>> bare sidecar is just a duplex stream uh that spawns uh a a bear that's packaged in as a dependency, but it's just a native it's still just a native binary uh per OS you just install it in. And then uh the IPC is uh sdddio3. So you've got, you know, zero for input and then one for out and two for error. We do FD3, right? And that's just the that's just the pipeline.

>> Okay. Yeah, >> nice. Anyone else? >> I was just wondering if this I don't know if if this is a Unix domain socket or how you connect all of these pieces together, but okay. This is different from what I expected, but it's cool. >> Mhm. Mhm. This is after like four years of iterations and it's where we sort of landed for uh you know uh it's not just about like the technology side of it. It's about what makes sense with lots of people working on something at the same time, you know, um and and and you know, helping people uh framing things in a way where it just makes sense to like onboarding people

quickly because that's what we've been doing a lot, you know. Um let me also show you. Sorry, I'm just going to go off piece here and just go do whatever. So, I want to um I'm going to clone the the boiler plate again and just straight into Hello Pareleron um because then I can show you the update flow because of that little wrinkle there. So, uh, npm install and we're going to do pair touch again. Create a link.

I'm also going to set the version to zero just to confirm it and then mpm start. Great. So, uh, now what I'm going to do is I'm going to do, uh, mpm run make make a build. So, what I'm doing is I'm going to go I'm going to go through the productionizing flow and then uh do a peer-to-peer update. Uh, and you'll see the version hopefully change if uh if I don't mess it up. So, I've got some artifacts here. Um, great, that works. So now that so once

once you have once you have uh a build of an app, you can uh use the pair build command to piece it together. It's a really simple command. All it's doing is copying, but it's about structure, right? Um the point is that you want to be able to build on Mac, on Windows, on Linux, and then put them all into one folder. Um, at some point we'll extend the functionality of pair CLI to make that easier because we could totally leverage our peer-to-peer stuff uh for making like a build farm or something.

Um, but but at the moment it's like manual on purpose uh because there's less that can go wrong. Um, so uh this was I'm building on a Mac M1. So this is a Darwin ARM 64 app. So I'm going to do pair build Darwin ARM 64 app out hello pair blah blah blah hello pair app uh and I'm going to set a target folder also for out uh out build let's call it and uh I also need to provide a uh path to the package JSON uh and that's really important because what a what a build folder is is

the multiple architectures and then a package. JSON at the top that has that upgrade field on it. And that's that's what it all comes down to is that package JS at the top with the upgrade field on it. So uh if I run this now in the out folder I should have uh build uh yeah so this is this is it. You got the package JSON at the top. Uh, and then you've got the uh the bark folder just with Darwin ARM 64 in it right now. But you you you know, you want to have like for a full production app, you're going to have everything.

So with that, what I can do is I can cd into the build folder or actually I'll do it from the outside. Hold on. So out build is the is the deployment folder right now. Uh so I'm going to pair stage to the uh upgrade link which is this. And the pair stage command looks like this. Pstage flags link dur. So, I'm going to say pair stage with the link and then point it to uh well, sorry, I'm

going to do a dry run first. Always dry run first. Uh, and then that's out build. That's my that's my deployment folder, right? So, once I've run that, you can see that it's it's printed out this diff. Um, pretty big because it's an Electron app and it's the first stage. But the point of the diff is to to verify um that every change that's happening is intended. Um and that's why I dry ran it because it's not actually written. It's just shown me what changes it would make. So now I'm going to run it again without the dry run.

Cool. Uh and now uh I should be able to uh seed this link and that's now exposing that link uh to the DHT. So if I there's something else really cool that I just want to show you that's on the side of this. Um, we don't have this out yet, but there's come there's it'll that will fail.

There's a there's a pair install command coming, but you can preview the command with mpx pair-install and then a link. So, if I run that, it will install the build that I just made from myself. Uh you can see the there's a join on the seed there. That's that's this that's going to install it into my applications folder. Uh so it it will install UI applications and it will install uh uh command line applications. So there's a side advantage here that the same update drive that you're using everything actually can be a peer-to-peer install

source as well. So if I open applications hello pair now don't be like this. That's not the one. Try again. Great. Okay. Maybe that wasn't one. Now when I opened it, there was a join that that came that popped up on the seed. If you would if I was to give you this app and you opened it, you would also join in it, right? So now I know that the this this this pier this app is that's now a pier is ready

to receive updates. So what I'm going to do is I'm going to make a change. Uh, and all I'm going to do because it prints the version, I'm just going to bump the version, right? So, I'm going to do MBM version patch. Oh, the git directory thing. I'll just uh do it this way. Um, yeah, I'll just do it manually. So, I'm going to bump the version and then we go through the build flow again. So, it's mpm run make. And then once we've made it, we build it again uh for the uh uh the multiarch

folder and then we stage it again. And then once we've staged it, the update should go out. So pair build again. Uh everything's exactly the same this time. And now uh P stage going to dry run it first. And we can see this time the diff is much smaller, right? because we've only made a small change. Uh the things that have been updated are updated by the Electron make flow, uh some P list stuff, some that Windows stuff, the package JSON itself, uh inside the app, and then the top level package JSON.

They're the same package JSON. So that all makes sense to me. So now I'm going to run it without the dry run. Uh but before I do, let me make sure to have this app visible and the seed visible. So, I'm going to hit return in this one. So that's staged. Yes. Good. So, the uh that should come through, but it's not yet. This is always the bit where I'm like, ah god, it's not going to work, is it? But uh I might have to restart it to get it to work. Um, I'm not sure why that is, but it it

generally happens when I'm on the same machine, but if I'm using two machines, it doesn't happen, you know. Um, so you see that the length increased here from 1263 to 1273. That's that's the thing. That's the version. That's the the the peer-to-peer version, right? Uh if I do pair info of the link then it should tell us okay it's 1273 so we know that it's been updated uh but the app has not received the event. Of course it's not.

Uh so we got any errors? Like I say, this is updated recently, so it might have been broken. Don't see errors. Let me try shutting it and starting it again. What is that?

Sorry, I'm just trying to think about why this might be. My bad. Um, you could just trust me that it works. It's how we It's how we do updates and keep all the time. We know it works, but I think uh we have a lot of guys that are that are contributing all the time and and improving things. And it's possible that there's there's a bug that's come in here between the last time I checked and now that's the pair join. Oh, okay. That's

weird. I restarted the seed and then it started working. So, I'm going to take that back to the team and and figure out what the hell's going on there. But you can see that it's actually picked up the update this time. And it says updating. And you can also see that there's a upload from the seed happening because it's uploading to this this pier over here. It says the update is ready. Still downloading. That's weird. Or uploading. And then if I press apply update, it restarts with a new version. Uh that intermediate bit, I need to research exactly what went on there. Um but when like I say, when it's on the same machine, it's sometimes a little diceier than on different machines, but

I'm not sure if that's something to do with it. And with that, you have peer-to-peer updates. Um >> so this update Yes, it went out and back in. Yeah. Yeah. Yeah. Any other questions? We've got a cool snake like on the workshop. You can keep going through and then we've got like this cool game that we've made um called um Snake. I don't know if you've heard of Snake before, but um this works on exactly the same concept as the chat app. Um but instead of it being chat messages, it's structured data uh that determines the

position of a snake, but it's the same architecture. It's the same uh same exact thing all the way up and down. Here you've got like the frame stream. Uh you've got the updator. Uh you got join game instead of join chat, right? Um but it's all the same thing. You're just joining a swarm with a topic. The two pier connect and then they can play snake with each other as well. Uh There's a whole part two that you can

take a look at here uh probably in your own time. But if you if you work through uh part two of this workshop, you can go through the same deploy flow that I just showed you without but you can go you can take it further and go to uh provision and multisig as well. Um ah I could show we got a few minutes I could show the provision stuff. So, let's say that um we're at a point with uh this link that we're like, "Okay, good. Uh version 0.01, that's the production version, right?" Uh so, the next step then is to go, okay, well, it's time to it's time to um

create a provision drive. So, I run pair provision dry run uh and uh it takes a versioned source link, a target link, and a versioned production link. When you're when you're creating your first pre-release uh key, your first provision key, there is no production link yet. So for you just use a a zero versioned link and it but what it does is it it synchronizes everything from the version production link onto the target link and then on top of that it overlays everything from the version source link onto the target

link and that way you have a replica of production first so it's always it's always correct and then you have your changes on top. So, uh, I'm going to do I need to create a new a new link for this. So, I'm going to do pair touch. And I actually need to create a new version now because and and this is sort of uh C a problem for CI or whatever when it comes down to like a proper deployment. Um, but I now need to update the upgrade link. I think actually we've got a thing where you can do I'm not going to do it here but you can do like develop uh like stage and then

production and then have a link per um but the raw way is to uh update the update the link. Uh so I'm going to update that. But then I need to also stage that because we're going to be syncing from the stage link as a source. So I'm going to go pair stage dry run. Oh wait, sorry. I need to do a make first. So npm run make and then I need to build and then I need to then I need to stage dry run and then it's got the the provision link that we've not provisioned to yet.

So, uh, now build and now stage dry run. Yep, that looks correct. And then without the dry run. And now uh I can do the provision. So my stage drive is is uh aligned with the future provision drive. So now I go pair provision.

The source link is the versioned stage link. So you can actually find the version stage link in the stage output kind of discreetly here. And now the target is what we've just set the upgrade field to and that doesn't need to be versioned. And then finally, uh there's a production link which we don't have yet. Uh so I'm going to pass in the um the source link at 0 0. So it syncs nothing to start off with. And oh don't

forget to dry run first. Uh I think because this is the first one there should be a diff but uh because it's the first one there's not. Uh so this this is saying uh yeah good to go now provision. So, this this is getting more serious and it says like, "Are you sure you want to do this? I'm waiting 10 seconds for you to bail." And then uh when it's done, it it uh should then have synced and we can see that.

And what I'm going to do just to make sure that it's worked is I'm going to remove Hello Pair from my system. Oh, because it's uh running. And now I'm going to try to do that install again with the provision key instead of the stage key. See the join. See the upload.

We didn't need to bump the version for that build. Generally, you always bump the version, but because we're we're changing we're swapping the upgrade key out, you don't need to bump the version because you're starting with a new key, a new a new link. Now, it's been installed to the applications folder. Uh, open it. Great. Cool. So, this is this is now a pre-release link. The step after this would be uh pair multisig. Um and that this is where it gets a little bit heavier. We're not going to do this uh we don't have time. But the flow is this. You uh you create signing keys. So uh each signer needs to generate their own key.

So we would say like myself, yourself, maybe you, right? Uh and then yourself, and then yourself. And that's that's five of us, right? And then uh we're going to say, okay, all of each person is going to to generate a key uh with a password. Uh and that's uh then that that writes that out. It's sort of like SSH, right? Uh to this pair folder in your your home. So that's your that's your public key for signing. And then what you do is you create a pair.json file in the project route.

Oh, it's already there with uh those those public keys. So um I've just lost that pair multi-IG keys get cool. Yeah, it it creates it if you don't if you don't have it. Uh otherwise get will just you know give you your your keys. So that would be the first key and then uh maybe I take two more keys and we just do a two out of three corum whatever. Um once once you have that you then run pair multic link.

Oh I need to hold on let me just fudge this. I'm going to do a one out of two chorum and hopefully it lets me use the same key. Huh?

See if pet mod works. Most link works this time. God damn it. Cool. So once you've set up your pair.json adjacent with your quorum, you use pair multisig link because uh instead of pair touch because it's determined by your multisig configuration. If you change the signers, you you you need a new link. If

you change the quorum, you need a new link because it's it's basically a contract at that point and the link represents the contract. Um and then that would be then set that pair multisig link would then go on the upgrade field, right? And then you'd go through that whole process again where you'd like stage the new multisig link and you'd provision the new multisig link because this is now about production. And then uh then you do a then you'd go through the multisig flow uh write it to that link and then at that point you've actually reached the production piece where you can actually give this to your users. Um so uh you would version you'd make the distributables you build the deployment directory you'd stage. You've seen me do

that a few times already, right? and then you provision uh and then you go and write uh pair multisig request, right? So, um I wonder if this will give me a little preview if I do it like this. Pair multisig request. Going to version it as 00 and see what happens. It's going to freeze. No, it needs to be seated. But yeah, so you you you you you I've not written to it yet, so I can't do this part, but uh with what what pair multisig request does is it it creates a um a signing request that you then give out to your

signers and then each signer runs pair multisig sign and paste that sign in request in and it signs it with their keys and they give their uh uh responses back to whoever's whoever's actually running the release, right? So, say it's me. I I I get my response from you. I get my response from you. We put them together, and then I run pair multisig verify uh with the source link, the the multisig link, the signin request that I sent out, and then all of the responses. Uh that's basically a dry run, but we we've made it a separate command because it's that important a step that it should have its own command. Uh once that's once you know that that's all

good then you can run pair multisig commit with the source link signing request and responses. Uh and at that point it actually writes to the link and then any users that are using your app will receive uh an update and get the update and go through that flow that we have already seen with stage. But that that takes us to the that takes us to the conclusion that takes us to the full end to end flow of creating production and deploying and installing uh production peer-to-peer apps.

So uh questions before we wrap or um how we feeling? It doesn't have to be a question. It can be a a take if you like. >> Pretty cool. Ah, that's very nice of you. Anyone else got any compliments for me? All right, cool. Um, does anyone if anyone wants to connect on Keat? We can uh just do it. Uh, we can wrap now. We can finish this and then as soon as I figure out where I've put my phone, uh, we can connect on Ke. Here it is.

Uh all