ATProto/Bluesky: The Personal Data Server vs the Collectivist State Machine

Transcript

[music] Okay, thanks. So, I mainly build stuff on Ethereum. I'm going to talk today about Atroto, which is not on Ethereum, and I didn't build it. Um, I don't in any way represent um the Blue Sky team who did build it. And the way I'm going to talk about it is not really the way they talk about it. Um, but I I think it's kind of important to talk about because it's really not well understood among web free people.

Um, and I think it's not well understood mainly for kind of vibe and tribal reasons and not for sound technical reasons. So I want to talk really, you know, about how it how it works and how it's designed. So I'll start at the beginning of time long long ago with web one. When I wanted to publish something on the internet, I would make my HTML page and then I would upload it to my web server. Um then if my friend wanted to look at it, they would access my web server and and and download it. Now this was not a peer-to-peer system, right?

But it does have some nice properties that we are chasing when we build peer-to-peer systems. So anyone can publish. No one can stop you publishing. You can self-host your web server if you want to. But usually people don't bother because if you use shared hosting, the shared hosting is unopinionated, right? The shared hosting doesn't do like your web host doesn't do like editorial control of your of your stuff. And if they do mess you around, you can just like restore from back up to another server, move your DNS over and carry on.

Okay, so that's quite nice. But then the web went a different way. Um, we went to web 2.0. And this happened because there was a load of stuff missing in the original web one design. We didn't have social connections in there. We didn't have identity. Um, we didn't have structure to the data. So if you've got something like a a tweet, you've got like a tweet and a reply and a retweet. um that wasn't built in and we didn't really have resilience. So often early on if you had a website that went viral then you you would just get too much traffic and the website would fall over.

Um so we solved that with web 2.0 by having like a really small number of really big websites. All of our content goes in somebody's database. So something like Twitter or Facebook, all the content is in that database. And now that guy has editorial control over your life, right? The people who access it are accessing it through his website or through the API that he provides and he can now decide what he wants to let you see. So this is the lesson I think we want to take from this and what happened there.

If your decentralized protocol is missing features, if you have a protocol that doesn't completely solve the user's problem, then the market will actually give them those features, but it will do it with centralization. And that's something we'll see again. So, we were we were quite lucky with Twitter that um Jack Dorsey, who was in charge of our who had editorial control of of our lives in Twitter, um was a libertarian, like a crypto guy, and he thought that we should do better. He thought that Twitter should not be uh just a platform controlled by him. It should be an open protocol. So he got a bunch of P2P people together and said, "Design me a way to decentralize Twitter." So they

did that. But then just as they were getting kind of close to to something maybe usable, Elon Musk bought Twitter. They went to meetings with the people at at Twitter and nobody showed up. It turned out that everyone they were supposed to be talking to at Twitter had been fired. So it just turned out Elon Musk had no interest in in decentralization of the protocol. So what they did was they shipped their protocol. They bought they built a Twitter clone on top on top of their protocol. But the point of this was not just to make a Twitter clone. The point of this was to to decentralize the web.

Um so there are now loads of other applications also built on approto. Um, I will just mention one here, Tangled. Um, because it kind of illustrates what I'm talking about. Um, Tangled is a replacement for GitHub. So, this is another another case where Git was a completely peer-to-peer protocol. Right? With Git, I can sync directly to the person who uh I'm collaborating with, but there was a load of other stuff that you need that wasn't built into uh Git. So like you need CI, you need um uh pull request tracking, all this other stuff that wasn't those features were missing from git. So the market filled them in with centralization, fill them in with

with GitHub, which ends up being bought by Microsoft. So how does it work? This is the heart of at proto. What I've done here is on my blackboard, I've rubbed out web server from my web one slide and I've written in PDS. Otherwise, it's just the same. So, how's that different? A PDS is for structured data. So, everything that goes on your PDS is going to uh conform to some lexicon. I will tell you about lexicons. It will be attached to an identity. It will be connected to other people's identities so you can follow your social graph. and it will be content addressed and cryptographically signed. Um, now this is normally stuff

that we do in peer-to-peer because we want to get our data from a peer who we don't trust. Um, but here we're not necessarily going to get all our data from an untrusted peer. Um, and we're not going to do that because this was built by people with experience of P2P systems and they realized that getting you p good UX with the current state of peer-to-peer is really hard. So they said potentially we're going to use some semi-rusted servers to actually deliver our content but the origin of the content is all going to be content addressed and cryptographically signed. So let me give you an example. Um so this is I'm going to be so this is me uh posting on blue sky. Um so I'm replying to a post by Kira. Um Kira by the way is an AI bot. She's running on somebody's

GPU in Tokyo somewhere. Um, I heard that the guy who bought Farcaster wants to get rid of the bots and like do like proof ofish humanity or something. I think that's really sad. Like on Blue Sky, we have loads of bots just wandering around like farmyard animals, you know, and you know, we learn from them and they learn from us. It's a it's awesome. Kira has loads of fans. She um she sells her post bath coolant water um to her fans because everyone loves them. Anyway, so so she's done this brilliant post. So I want to post banger in reply. So this is the record I'm going to make.

Um, this record has a type. This type locates it in the lexicon. So it defines the structure of the data that we're going to write to the user's PDS. Um, I've written my text banger. And then I've got a reply since it's a reply to to Kira. Um, I've got the hash of the thing that I'm replying to. And I've got the URI which consists of her identity. That's the DP PLC part and a post ID inside her PDS. Okay. So once I've made that record, I'm going to write it to my personal Merkel tree.

This is a slightly exotic kind of Merkel tree called a Merkel search tree. Um I'm going to add the record to my to my tree and then I'm going to sign the root hash. So once I've done that, anybody can now get that record. They can get a Merkel proof and they can verify that the thing that I posted uh is actually posted by me, no matter who they get that from. So this is what we did. This is the model. Everyone in the world gets a self-s sovereign cryptographically verifiable semantic data store. A self-s sovereign cryptographically verifiable semantic data store. This is the correct way to do the internet. Right? This is what we should have done after web one when it turned out that what we had didn't really fill our needs. So um if

we just did this, this would actually work, right? Um, so we could make a client. Um, it could have maybe a list of all the people whose PDS's I want to look at, all the people whose data I want, and it could just fetch all of the all of their records from their PDS's. Um, this looks actually looks a little bit like our RSS if that's what we're doing. Um, but that's not very efficient. So there's some more infrastructure in between. The first thing is a relay. The job of the relay is to um connect to all these PDS's, slurp in all of the new content from all the PDS's with its Merkel proofs and then splurge them all out in a massive fire hose that anybody can act to. Um and then beneath that we can do our app

views. The app view is basically some kind of front end, right? So it could be a web two like website. it will have its own index of the the data it thinks it's relevant and it's organized in a way that it thinks is relevant to the user. So let's look at what happens if we want to make an app. So I'm going to take the example of a recipe application. Um so this isn't blue sky. This is a a different app using approto for for recipes called uh this one's called buttery. Um so the first thing we want to do is define our lexicon. So this is the structure of the data that we're going to write to everybody's pds. Um, since this is about recipes, it's going to have things like ingredients, uh, cooking time, stuff like that. Um,

basically all the things that all the fields that we need to to make sense of our data. Um, if you're the first person to use this to write this kind of data to people's BDS's, then you'll probably define your own lexicon. So, you decide for yourself how it should be structured. If it's something that people are already doing, like I think here they've reused somebody's lexicon um from recipe exchange. If it's something people have already done, then you can use an existing lexicon for your application. And that means that all of the data that is already written to people's PDS's you can immediately use.

So that really helps with the the cold start problem. Um all of the data belongs to the users and not the application. So if if there's already data out there, you can access it. You can use it however you want and you can display it how you want. So we we've defined our structure of our data. Now what we've got to do is we've got to get the users permission so that we can write to their personal data server. So we do that with oorthth. Um we're going to make a button. It will flip the user over to their PDS. They will um give permission for writing data of this particular type. So they'll say this application wants to write recipe data on your behalf. Uh they will give

permission and then it'll take us back to to our application. Then we want to make like a read interface. So what we're going to do here is we're going to listen to the relay. We're going to index that data, put it in our own database in a way that is optimized for serving to the user in the way that we want to serve it. And then we're going to um serve that either through a website or through like a um an API back end for a for a for a client's app. Um but what we didn't do here is we don't own the authoritative data, right? The data all lives on the user's PDS. Um lots of different people can run can run these relays. Um so we can always like back fill if we need to if we if we want to if we let's say we lost all this data

we could always recreate this data um from from the original source um and the the nice thing about this design is our users are not actually generally um connecting to the PDS and that means that the PDS can be really really lightweight you can host your PDS really really easily um and you don't have to worry about getting slash dotted because because in practice the the the data is being served to users through a couple of intermediaries. Okay. So, conceptually, how is this different to the blockchain? Um, because with the blockchain, we took we looked at the same kind of problem, right? We said that we've got these big web 2.0 databases. They've got everybody all this we've got these big private databases of all our data. And we said, okay, let's make a big public database

of all our data. Right? So, we made this blockchain that belongs to everybody. Um, and sometimes that's really what you want. But the the the downside of doing that is there is now a cost to writing this data. If you make an Ethereum transaction, I have got to download that data to my node over my rural broadbound in Tokyo. So there's this commons that we've now got to manage, this collective database that we've got to manage and that means that we have to prevent spam at the right level and that means that we have to charge everybody gas and that gives you this huge hurdle when you're trying to onboard a new user that they can't actually write anything and until they've got some like some of them made

up money. Okay. Um the the other big difference here is that on the PDS the user owns all the data. So they can delete stuff. They can rewrite stuff at will. It's entirely belongs to them. With the blockchain everybody owns the data. And even if you're the person who wrote the data, you don't have the right to um to change it. And sometimes that's actually what we want, but sometimes it's not. So conceptually like the the blockchain has got this commons. It's like a collective. Um it's like all this data is is shared by everybody. Um whereas the PDS is more like the sovereign individual, you know, it's the guy on his shotgun with his ranch. He he go on his ranch with a with a shotgun like he owns his own data. Um he he does

what he wants and if if if nobody wants to go go there, nobody has to, right? So it's this kind of radically individualist conception. It's funny to me that these projects have really different vibes. like the blockchain has somehow in the American culture wars become kind of MAGA vibes and then the the the PDS because it was seeded by people who are mad at Elon Musk. The PDS is kind of kind of leftwing in the American culture wars or well blue sky is kind of leftwing in the American culture wars but those are not the properties of this technology. Okay. Okay. I should say a little bit about identity and this will also kind of show us the limits of this model of sovereign data um self- sovereign data as opposed to consensus collective data. So there

there are two schemes for identity in app proto. One is called did web um with an identity what you need is you you need to make a document. It's going to have things like your username. It's going to have the address of your PDS where you're hosting it. Um, and then it's going to have the signing key that you use to sign those records uh on your PDS and and sometimes a load of other stuff. So, you make that document, you put it on a website, the address of that website is your identity. That's one method. Um, and that's great as long as you trust DNS, right? Uh, if you if you trust that you can hold on to that domain, then then that's awesome. The other method is one made by the App Proto team, which is called DID PLC. Um, this starts off looking a lot like sort

of IPF, IPFS with sort of self-certified data. So, your ID is the hash of the original version of your identity document. Okay. Um, if you want to make an update, you chain it on top. You put the hash of the f of the first um document into the second document and you sign it with one of the um the controller keys that you put in the first document. And then you can make a change to the content of that. For example, I might to change my username or change my sign in key. Okay. Or change my PDS address. Um, so far so good. The problem is what if you want to change the control of your identity?

Like what is the key that is allowed to to manage this identity? Um, so what you can do there, it's the same process. You put in the new key, you sign with the old key, you make an update and friend on the the new key is used. The problem is that now we have the Bitcoin double spending problem. um somebody could make a um an alternative history using the um the original key that I've rotated off if presumably like if it's if I rotate it off it it's because it's going to get get compromised eventually. So so they can make this alternative history and there's no way to tell which is the real history. Um so looking at how different protocols deal with this Nostra does it by not doing changes of control. Right? In Nostra,

your public key is your identity. Um, you can change your handle and stuff, but but you can't change the control of that. And that's because Nostra can't use a blockchain because because Nostra, it's like Bitcoin maxis, right? And um all blockchains that aren't Bitcoin are a scam, but also Bitcoin is not really suitable for for for this. So, so in Bitcoin, they just don't let you change control. Um, in the blockchain native things like Farcaster and Lens, it it's very nice. Like the blockchain was just designed for this kind of problem. uh the blockchain will let you do it. Um in app proto the changes of content are great. The changes of control they do with a centralized sequencer. And this is the only like explicit point of of centralization in app proto. Um you can avoid it by just treating your identity

like Nostra and just not doing changes of control so that you only ever have one uh chain of of identity and the and the sensor can't help you. Okay. So I I I I feel like you know the the these two approaches are both really useful in creating um decentralized applications and we really need both of these. Often in the same application there will be some data that belongs to like the self- sovereign model and some some data that belongs to the the the commons model. Okay. Um I think I can squeeze in a couple of Ethereum things um that people have built. Um the uh app pay. So this is a a way to do um to attach an Ethereum address to your um to your

approto account. Um some Nosis safe integration I did. So this is so that you can do things like you can take a blue sky message or any other approto um action and you can use that to drive um a nois safe on chain so that you can execute stuff on chain based on uh things that happened in the approto world. Um, somebody much smarter than me, um, ZKU Crypton Anarchy Network, um, built a ZK version of this, or not quite the same thing, but a way to do proofs, uh, on chain of of stuff in your PDS Merkel tree using, uh, Groth 16. Um, finally, I made a thing for like cryptonative onboarding. Um, so if if you on board with Blue Sky, firstly, by

default, they will control your uh your identity keys. um wi-i which you know since they're going for the mass market that's understandable but if you're crypton native you you can handle a seed phrase right you can do that um so so so we want to be able to make your own identity with your your own seed phrase um and then the other thing we do is in um in app protos there are really incredibly tools for decentralized content creation and moderation. So for example, you can make an algorithmic feed which will ba based on whatever algorithm you like very very easily serve it very very cheaply. All it has to do is feed a bunch of ids out to the

client and you can really easily add this to your client software. So you can get the curation experience that that you want rather than being dependent on the developer of the application. Um you can also do the same with moderation. So you can make block lists about who you don't want to see. Um, and the defaults that Blue Sky gives you are awful. Um, so this will give you like onboarding, but it's actually nice. Um, so I've put a bunch of links up here. I'm goat navy on blue sky. I'm Eban Edgar. Ethan Farcaster, but I don't know if you'll be able to see it depending on which file you're on. Okay.

Yeah. Let's move to the questions. Yes. Thank you. Thank you, Edmund. So let's start from the first one. Uh yes. So actually currently we have a question blue sky still run mostly by one company. How the centralization at at pro in practice today? >> Yeah. So the there's kind of two two levels to answer that. Um blue sky in practice as you say the the vast number of users have their accounts with blue sky. they have their keys also invested with blue sky and they use a client run by by blue sky. Um so you have kind of a similar similar situation that you have

that caused this fiasco with Farcaster where in practice if everybody is using the same clients and everyone has the has the same everybody is is experiencing the network through through the same um through the same gateway then like blue sky could just say you know what we've decided appro is we're not doing appro anymore um and and they would basically get the network right. Um me and my friends would still be able to talk to each other because like we would be using Approto the stack like originally there were concerns that for example they were operating a lot of the intermediate infrastructure. So there was only one relay originally that was run by Blue

Sky and people saying oh this will need loads and loads of like uh hardware to run. That stuff has all got really really good. So there are now like plenty of independent relays. Um there were also some kind of independent communities who were kind of survived. So for example, Black Sky and and Euros Sky, I I think they would potentially like survive, you know, Blue Sky doing something weird. Um but but it's certainly true that as far as the Blue Sky network goes, I I think that they kind of have defect de facto control just by the virtue of everybody using their application. At proto applications, it's a bit different. It's a much smaller subset of people who are who are I think less dependent on blue sky.

>> Got it. Let's move to the next question. So next question is what's about incentives? Any thoughts? >> Um okay. So what about incentives? Okay. So so um what's nice about this model is um I I think it works well without really needing you you don't need to do like storage rent and stuff. the these need you know need needing gas in the first place is really a function of having this design where everybody has to hold everything. Um there is when it comes to things like the dur the decentralized curation and decentralized moderation right now I think curation is actually working really well um curation there's

there's one guy who runs the for you feed um he does it on his gamer PC gaming PC it's just he does it incredibly cheap serving this brilliant feed to to loads and loads of people he's just announced like a very simple like slightly premium business model you know premium uh like $5 charge or something I I think that will work really well. Um the decentralized moderation to be honest I think is not really working well and needs some kind of an incentive structure. Um because the problem with moderation is that the kind of person who is interested in making a list of all the bad people who you don't want to see is generally somebody who has like some rich collection of like eccentric beefs. So like th those people th those kind of um

kind of volunteer run moderation lists seem to be not working very well to be honest. Um so I think probably we need something there that will make the the decentralized moderation work better. I don't think that's quite working. >> Got it. Got it. Moves forward and actually next question. So how make easier for people to switch Yes PD PDS to avoid censorship. And so let's yeah you actually recently had a story with blocking links. Yeah. From activities and yeah can you explain last these questions?

>> Yeah. So I um I I I don't I don't know about this like this specific case or what exactly happened in that case. Um but generally what they do when they um when they ban some when if they if they suspend an account um what they do is they ban them from their PDS. But then if they've got if they've got their um their keys, they can fairly easily flip it over to to to another PDS. And actually, this is part of the goal that I think of of the thing I mentioned where I made this front end with the the goal that you you initially already uh control your own rotation key. Um I'm not the only person working on this. You know, there are some other really good tools to do this. Um but but but I think that if we can onboard new users and

particularly when we onboard um users who are potentially going to be the targets of censorship um you know if we can improve the tooling for them to control their own keys um if you control your own keys honestly there were quite good tools to to flip feed. Yes. >> Okay. Thank you. [music]