Private Shared State

Transcript

Cool. Thanks everyone. Uh nice segue from Zach, but yeah, here is Lucas. Lucas was down in our uh London office recently whiteboarding with the rest of the team. I think um a lot of hot ideas there. So interested to hear what you got, Lucas. Good morning. Uh hi everyone. Um it's great to be here. I've been at all three norans and this community is really something special to me and us. Uh I first felt this during EFCC last year at the trust infrastructure day and again in full-blown effect at a rather spontaneous workshop where we come together at the Aztec office later in the summer in London and from this like very tight group and this really nice people. It's amazing to

see how the community evolved over it's only 10 months now and today I want to talk to you about something I believe is critical where we go next around two years ago I was at the uh the science of blockchain conference walking around with a question in mind at that time everyone was talking about CK in the programmer cryptography land and it felt like a bit off because people were kind of alluding that CK would solve all the problems that we are attributing that can be solved with programmable cryptography but this doesn't feel felt right for us why because CK doesn't let you for

example collaborate on private data it's inherently stateless and most valuable applications on the internet today from marketplaces to messaging to governance are all collaborative and stateful in nature. So I spent the conference asking everyone are you thinking about the limitations of CK and here I mean like the limitations of CK beyond performance and developer experience turns out most didn't everyone was really focused on proof system and lookups like this was the time like the first open CKVM uh lasso chol came out and also ser knowledge machine learning

or CKML was like at really its hype cycle even though CH GPD didn't come out so far. Now CK felt just synonymous with and this bugged me. Why is no one seeing the obvious or just is it us that we are just dumb and don't see like how this all works out in the end? Then on one of the last days I met uh a person who was like the head of cryptography for a big project has just launched and we were talking and they were already using Ethereum as their root of trust. They were using client side

proving uh CK all over the place but they only have launched one or two months before and already run into the problem we were alluding to. They needed confidential compute and they couldn't find any infrastructure that they can use. Even worse, they couldn't even find tooling that met their requirements. And that's the case because confidential compute in general had been overlooked and neglected for years compared to CK. So this moment validated what we felt. something fundamental was missing and most didn't see it because either their projects didn't launch and didn't run into these problems or they were just too deep into this CK narrative that CK

would solve everything we need. Like to sum up in a world like let's let's pretend CK has no overhead and we can like program in angry language and like you can even vip code it if you want like if we do not get confidential compute like as a second pillar we'll never reach the full impact we want now I want to zoom out a bit how to see like this observation of the CK space fits into the broader blockchain and uh internet space.

We often like to describe the internet in three phases like the read phase like the early web where you could browse you get your information on the second phase where the read write where we're currently like the web two you can participate like you have social medias you have marketplaces like you can text people that's really like what most people are using today and we know all that like we have built great networks in this internet but those networks got more closed over time and more extractive. So we need something different. We all agree on that and that's sometimes called readr own sometimes we web free and at the heart of it is that like users content creators should take back control and

that the networks should be positive some and open and at the heart of it are blockchains that make this possible. That's at least a promise. But real control isn't just about self-custody. Self- custody is not enough. It's also about how your data is being processed and who is processing your data. If you're looking at web two like how do we do there like we get some kind of privacy in web two like at least we have privacy from our peers in web two applications. We get this by trusting or an operator like Google Meta or whoever runs the application.

If you're building an application currently on a blockchain, it's even worse. Like if you want to handle some form of private data, your options are quite limited. You either disclose everything uh which is worse than the web two model so far or you go back taking the data off chain and trust just like being a trusted operator like web two. And you can't blame the app builders for that. There's no infrastructure. There is no infrastructure for secure scalable um private compute and we need that more than ever today.

Like AI we all know is changing how we interact with information with each other with the web. Like the open internet at least for me is collapsing into a prompt bar and this is a huge centralization force. We often describe or we hear that like blockchains are this kind of counterbalance counterweight to these centralized uh tendencies for AI. But for me this doesn't make sense at least not now because like blockchains today can't handle private data. So how would they handle my prompts? Some people say yeah just run the AI locally. But I think this goes beyond the point like we're seeing already that if you want to have like a really powerful uh AI or ch large language model a lot of advantage

is gained by having memory context context over time over your habits your programming habits or like whatever you do like how you interact with an LLM. So you need this context be across devices, across sessions, across models and we already can see this will default back to the cloud. So what we need is a network that let us that let us store access compute and collaborate on private data without giving it up. Without that I have the feeling the promise of blockchains will fall short.

So if you want web 3 to shape the internet's next chapter, we need confidential compute. And that's what we're building. A network for encrypted compute. One that lets you collaborate without exposure. One that keeps privacy and verifiability really together. And one that should be as seamless and universal as HTTPS. It's quite an ambitious plan and it won't happen overnight, but it's essential and it's possible. Now let's dive a bit deeper into the technical aspect of this and why we can't build a network for encrypted compute with CK alone.

If you think about Ethereum like now as this world computer and Ethereum is like the gold standard for blockchains is essentially a censorship resistant permissionless globally accessed computer we all trust. And how do we gain this trust or how do we trust this? It's by implicitly making all data public and agreeing on it in a sophisticated form. But for most applications, what we actually need is something like a trusted compute and storage environment inside this world computer that nobody can look into, but still has the same properties that we would expect from a blockchain.

For quite some time, the narrative, as I told you, was that CK will solve this, but this isn't true. Why? What? What is like the fundamental fact why this can't be done with CK alone? It's because whoever generates the CK proof needs access to all the underlying data. Now, who should be this entity? We are sending all the data and then does the proof. Are this the validators? Like if the validators get all our data, we are back to like making everything public. And that breaks the whole point.

Now we know that CK brings us private state. How does this fit in? Like we know about ads tag aliden. This private state is owned by the user. The user generates the proof using their own data and this works quite well and you can build a lot of nice application. We're seeing a lot of them today or in the workshops. But this form of private state is fragmented. It's fragmented by users. So the moment you want to build something collaborative and collaborative is not very abstract like think about a decentralized exchange about a darkpool about a dating app something that has something to do with matching for example

at this point you have to open up the private state to the application basically the operator and as soon as you do that your private state becomes public again. So what we need in addition to this user owned private state is a private state that is not known to anyone. It's a state we call private shared state and this is what enables collaboration without exposure. That's the missing infrastructure we need. So how do we actually build such a network for private shared state? Let's start with the obvious properties we want. It should hold private data or private state that is not known to

anyone. It should allow us to compute on private state although nobody knows it. And the state update that results from this computation should be verifiable by anybody including of course blockchains. So in short we need persistent encrypted verifiable compute. The probably easiest way to realize such an infrastructure would just literally mean using trusted execution environments. But it's such a critical piece of the infrastructure. We want to rely do not rely on trusted execution environments.

So instead we want to rely on cryptography being having the layer more robust. And for verifiability, the obvious choice is CK. And for computing on multiple inputs while keeping them secret, that's the field of multi-party computation or short MPC. And when I say MPC, I mean this rather broadly. I don't mean any specific protocol. And like you can look at for example FHE as a two-party MPC protocol. So it fits also like into this uh field of cryptography.

Now combining CK and MPC seems like quite a headache. Both are fields that quite complex, quite an overhead. Now you want to like merge them together. This can't be good. But against intuition, combining them can be surprisingly efficient. like this was first shown in the original paper in the original collaborative snarks paper or short co-nox by Alex and then bonet like the tlddr the the main insight is like if you look for for uh at your knowledge proof generation from a computational point of view it's mostly linear operations so and you have to know that those linear operations do not introduce

overhead in MPC systems so this paper really showed us a way how to combine encrypted compute and verifiability in a practical manner. And at Tatio, we have been working on coarks for the last one and a half years. We first presented our ideas at like the CK summit 11 a bit more than one year ago and then started to build tooling like a proof of concept in circum so that you can compile any circumc if you're interested like this is

already live you can use it and there will be a workshop at 12:15 later down in a small workshop room. Now, this was a lot of words and of course a big vision visions are only like meaningful if you can ship. So, I want to share with you the concrete next steps that we are taking at that to make this network real. We're only a few weeks out uh from launching our production network with its first use case private proof delegation. under the product named Tatsio proof. It will be for CK apps and protocols that suffer from bad UX due to long client side proving times. At launch will support circum but closely followed from

nor. And later this summer we plan to integrate fully into the Aztec network which then enables any application on the network to just use that co proof out of the box and get a great UX while still preserving privacy when they generate the proofs. Like as you see this first phase of the network is kind of ephemeral like it doesn't persist data between states. So we'll upgrade the the network later this summer that it can hold persistent encrypted state and this will make an evolution from CK as a service network to a comprehensive encrypted compute solution and the biggest leap forward will happen towards the end of the year. This is

then private shared state at large scale. At the core is a primitive we're calling oblivious sparse merkel tree. So it's like very similar how ethereum uh interacts with data and stores data. It would enable uh an efficient encrypted reads and writes on secret shared data uh on secret shared data at really at the scale that is needed for real world deployments and it will serve as like the foundation for private shared state deployments across the Ethereum ecosystem. So that's the papon from private proof delegation to persistent encrypted state to private shared state. But that's not just a techly road map for me. It's a

bet on what kind of internet we want to build because without encrypted compute, we're kind of stuck. And coming back to my initial question and how this question is now perceived, I see that people react differently like if I talk about them like about the limitations of CK and like if they feel we have to build this confidential compute in order to reach our common vision and this has really changed over the last two years. Like one example is like I getting a slot here to talk about this to you for 20 minutes.

Now to close I want to say like we have managed as a community to build public state and we just witnessing how to master private state thanks to Aztec. There's just this one frontier left, the one thing we have still to master in order to have widespread real world adoption and it's private shared state. And with KNX, with MPC, with this network and like with the help of this community, we finally have a real shot at changing that. We can we can give people and applications the ability to work on private data together without giving up control. And it has to become as universal as seamless as HTTPS in the

end. And that's what I'm building. And uh we couldn't be more excited that our journey and the journey of Conor and private shared state started really in the heart of this community. Thank you. [Applause] Hey, uh great technology. I have a question about oblivious um sparse Merkel trees. Do you have some limitations in how you access the state like for example in fullorphic encryption you can do this uh additions and multiplying by scalar or is it completely arbitrary interactions?

It will be completely arbitrarant actions but like the oblivious sparse mercury is something that we haven't disclosed so far. So it's the first time mentioned it public and like uh it's still under heavy development but it's like a general purpose programmer like how you would usually uh interact with a merkel tree at the beginning it mostly read writes updates. Cool. Thanks. Uh great talk. Thank you. Uh I was wondering what type of overhead you have uh compared to a normal execution on chain if you want to do private shared state uh compared to a normal um transaction. How does that look? uh or do you have communication with the computing committee or how does that work?

Yeah, so private share state is handled offchain like you like you can think of it like this network like just rolls the new state update on chain. So you the one thing that happens on chain is like commitments and verifying the proofs that come from the network. So you're technically not like doing too much work on on chain like uh if you're for the proving part like going from normal nor to cornor like there is like no theoretical overhead that we occur or very little as like most uh operations are linear there's still like some engineering work to catch up with the great nor team like all these

performance improvements you have to kind of translate also to the MPC world and for private shared state like the biggest bottleneck that you want to get right is what we call the witness extension or the witness generation like this part of the proof generation that does the plain execution of the program now has to happen in an NPC network which is like the biggest bottleneck and for the user themselves do they have to do anything else when they want to post a transaction for example no like for the users they don't like see if it's private shared state or like it's private state or public state this doesn't change any it's for the developers and they're like that's why we target in nor that we do not introduce a new way how they have to

think about programs. Hi, thanks for the talk. Uh I'm curious if you consider a viability of introducing a peer-to-peer private shared state because this is what a dark pool is. So you still uh have an intermediate in the form of a blockchain storage. So have you considered the ability to operate without this? You mean like that the MPC network is really peer-to-peer between like all the participants? Yes. Like they exist only there. Yeah. We have considered it in the design space like it's quite hard because like everybody has to be online all the time or like you can do some asynchronous protocols. uh but like you're then limited by the bandwidth by

the latency of every single individual that uh participates in this network. Uh it would be great if we get there sometime at some point like this would be like the ultimate end goal like to have like a network of users like peer-to-peer uh engineering rights is I guess like the hardest part that you can target for. Thanks Lucas. Thank you for having me.