Privacy in Cosmos Live: Finch (Penumbra)- Full-Stack Private Protocol Design.

Transcript

and now I would like to introduce to the stage Finch from penumbra [Applause] hello so um today I want to talk about this thing that we've been calling full stack private protocol design because um all right so I'm from penumbra Labs uh just uh this is our logo it's the best that I can do drawing it by hand because I decided to draw my slides by hand because I thought it would be fun to do so I'm from penumbra those of you who don't know what penumbra is I'm just going to give a really quick overview so that I can ground the talk and chat about some of the cool stuff that we're

doing this talk is going to be a bit of a grab bag and I think that's going to be really fun but the grab bag has a theme and I will introduce the theme momentarily but first um so what is penumbra penumbra is a tendermint L1 app chain written in Rust so we have shielded transfer staking governance as akibu just was just saying in his talk all of these operate on a Model that is different from the cosmos SDK which means that interoperability is kind of an interesting question I was just having some cool conversations about private interchange accounts and how we can make that work with the penumbra model which is not an account model under the hood it's a utxo model

like zcash and accounts from the user perspective are sort of a polite fiction presented by the the client interface but that's sort of an aside also we have IBC to and from anywhere and so you have this interchange shielded pool that penumbra gives you in the cosmos ecosystem and Beyond um as well as in the center of that a shielded Dex that uses a lot of the really cool privacy features that we've built in penumbra to get um some really unique features of a DEX stuff you couldn't build without privacy uh in that environment and I actually give a talk about that at Gateway on Sunday morning uh very first talk slot

for the day so some of you may not have seen it because you were asleep and that's fine because it's recorded and it's on YouTube already so you can go look it up and watch it and I encourage you to do so it got some good reviews um so this talk is about three things when you're building a private protocol and you want it to be used by people like actually used you want it to be faster easier and cheaper than everything that has come before because if your private protocol is slower harder and more expensive then no one is going to ever use it regardless of how much they care about privacy except for wearers like me right so and probably probably quite a few of you but we are weird right we're not the norm so I want

to talk about this this is a post that I made uh recently um I was talking to so I was talking to the the um penumbra uh group chat that we have inside the company um and I had this like little uh brain worm um I was thinking about some of the different architectures that we have in penumbra um and I realized that we've got these different structures that represent different parts of our flow so we've got uh something called a transaction plan which you can use your human intent and the what we call the view service to construct and then this gets turned into a transaction So the plan is like the plain text version of a transaction the

transaction is the encrypted version um then once you submit the transaction to the chain what you actually look at Once you pull it back down from the chain is a transaction View and then sort of hypothetically could translate that back into a sort of abstract representation of the human intent behind that thing and I realized this Four Element cycle looks exactly like this thing called an ooda Loop um and those of you who are not familiar with an ooda Loop it was this idea that got um invented by some some guys in the Department of Defense or something back in like the 1960s or something don't quote me on that um but there are a lot of memes about this

um and uh there's this very uh funny Twitter account called defense charts um that makes uh that sort of posts uh Declassified slides from the Department of Defense briefings that look like absolutely insane PowerPoints on drugs um and so like I I sent this to the group chat and everyone was like this is a great post we should post this on our uh on our alt account which is called penumbra devs and uh so we did that and then I started thinking about this and realized this is actually not just the post this is actually kind of an interesting way of thinking about the way that private systems work this is an example of what we call post-driven development

and it turns out post-driven development is actually a really great way to figure things out for real because when you're posting you don't uh restrain yourself to just thinking in a sort of like orderly and straightforward manner you let your mind just cast out for associations and oftentimes what that means is that you realize that there are certain symmetries that manifest as jokes but are real and when symmetries are missing you realize that there's something missing in the design of your system so I am like a full-heartedly and unironically in favor of post-driven development I encourage you to try it out and I will see you on

Twitter when you do that so anyhow um this ooda Loop um I'm going to talk about sort of all the different components of this flow and how when you construct like when you're constructing a private protocol like penumbra you have to think about how to make it faster easier and cheaper not just at one part of the flow where say perhaps you're processing a transaction on the Node but also where you're viewing a transaction on the client or you're planning a transaction on the client like there's this whole Loop that goes from the client to the node and then back again maybe through a relayer there's all sorts of stuff right okay cool so faster and this is where we get into the grab bag stuff

most clients most of the time they're not sending transactions actually and I give a talk about this at ZK Summit eight um which has some pretty cool visualizations that I did not draw by hand but wrote a computer program to generate um so you can watch that with more details about this but they're actually doing private State sync that's to say that when you have a ZK system you can't just query the chain of course to find out what your balance is you need to download the entire history of every transaction that's ever happened and you need to decrypt it locally you need to synchronize a shielded commitment Tree in order to get the root hash so that you can prove inclusion of all of your state fragments right so this is slow and hard because you have to hash

everything from the beginning of time into now so we didn't want to do that instead we built this optimized tiered accelerated Merkle tree that we call the tiered commitment tree that's what my talk is about at zk8 but we also realized that this isn't enough because it's because fundamentally even though this solves the computational problem of not having to compute every hash since the beginning of time you still have a bandwidth limitation problem because you still have to download every transaction since the beginning of time so we built into penumbra some cool cryptography to do fuzzy message detection for private delegated scanning

so what this means is in your like key hierarchy you have a particular sub key that's called the detection key and the detection key is something that you can hand off to a third party not a trusted third party but just any third party and say hey can you please scan probabilistically for all of my transactions and that party will hand you back a set that's it's like an encrypted Bloom filter so that party hands you back a set of say 10 times more transactions that are actually yours and then you locally filter those down and the third party can't tell the difference between the transactions that are yours and the

like chaff that's generated so it's still more or less protects the metadata privacy of the transactions that you're doing because the third party doesn't know which is you in the like restricted stream that it's filtering out of the fire hose but you get to download only only need 10 times more data or whatever like a factor of 10 I'm just sort of like waving my hands about as opposed to the entire fire hose and that's really cool and that means that you can take even better advantage of the optimized accelerated Merkle tree that we have so that you can just like Zip right ahead and synchronize your entire state so that you can use the wallet on your cell phone or something like that all right so but sometimes also you don't just want

to know how much money you want to have you want to actually send that money somewhere else weird right um so uh we we also want to make this fast so we realized that when you are authorizing when you're when you're constructing a transaction you're sort of doing two things at the same time you are proving the transaction you're forming a zero knowledge proof um and you're producing the witness data that's necessary to create that proof but you're also authorizing the transaction by signing it now uh signing the transaction is a pretty quick thing to do computationally the proving actually takes quite a lot of time by which I mean like less than a second for a lot of our penumbra transactions but

still um yeah so um but you're also authorizing the transaction and so what we did is we said well do these things concurrently we architected things so that you can have a concurrent authorization flow which means that you um which means that you have a um you have the user say use their Ledger device to authorize the transaction and the the uh simultaneously in the background you're doing all of this work on the computer to prove it so that gives you more responsive ux okay so easier right um this goes back to the ooda loop that I had up there before um so uh intense we're all talking about intents right this is not necessarily

the intents that everyone else is talking about but we do have this notion now of a transaction intent and we want to build that so that we have a use like an easy representation of uh what does it mean to want to do something in a private context because these transaction plans that I've been talking about they're the the plaintext of a transaction but they contain all sorts of stuff like uh references to which notes you're spending and randomizers and ZK not ZK proofs but like um like you you've got all this like extra chaff that doesn't really represent actually what you're trying to do it sort of represents like the implementation details so instead you want to be able to have a data structure that says you know uh I

want to send like um 10 usdc to ZK validator because they're doing a great job organizing this conference um or something like that and that's all right and then you want to be able to translate this into a plan so that this helps developers to write better interfaces so that they don't have to manage like actually building these transaction plans and scanning through Note data and all of this stuff but it also helps users to have a more comprehensible understanding of the contents of a transaction um so what about Advanced users what about Advanced biddlers right um so we've built the penumbra State model in a kind of interesting way so internally we've got this uh way that

we've got a a copy on write State model um that means that um when we are internally within the application we're making modifications to the state we don't actually commit those changes down to the underlying storage what we do instead is we hold them in memory and we've got like a layered stack of modifications we can Fork the state we can do arbitrary things to it and then at some point we can decide to commit it down but if we don't commit the state all the way down that means that we can actually just use the exact same mechanism to offer an RPC that says you can query a full node and ask it what if I did this what would happen and it can give you a

hypothetical answer about what if you did that transaction using the exact same transaction logic and validation stuff as if you actually submitted the transaction which means that for people who are doing like trading watts and stuff like this this is actually like really useful okay cheaper um so uh fees have to exist because we don't want people to be using up all of our Network bandwidth right so um we need a fee model and there's this cool paper about multi-dimensional fee markets that we're going to be implementing on penumbra it's very cool um but we need a specific fee model within that kind of Paradigm that's tuned for a zero knowledge chain Like Us

in particular we want to place a premium on uh we want to place a premium on uh any actions that contribute to the stream of things that you have to download when you are synchronizing your client State because we want client sync to be fast so as a consequence we're going to charge more for actions in your transaction that don't just bloat the state on the full nodes because full node operators have a bunch of resources but actions that bloat the state that you actually have to download as a client because clients necessarily have a lot of resources um but we realized that every penumbra transaction is actually secretly or not

so secretly a micro ZK roll-up that consists of well the micro ZK roll up that's to say the commitment and zero knowledge proof of a single transaction we're rolling up a one transaction into one tiny little commitment to it and also the encrypted contents of said roll up and we thought to ourselves huh well why do we need two well we need two because we want any client that has their seed phrase to be able to recover the entire contents of all of their state and we also need the recipient of the transaction to be able to decode uh the the notes that they have been sent but what if you're keeping track of all of your own State off chain and what if the recipient is always yourself so

that's actually like a pretty reasonable scenario that scenario happens a lot if you're doing a bunch of Market making actively because that just consists of like open positions closing positions claiming rewards all of this cyclically to yourself so what if you could keep track of your own State and you don't uh and you're only sending to yourself then you don't need to publish the encrypted data to the chain you can just use penumbra as a like micro ZK roll-up chain for all of your trading activity well you that would be great but you also need a way to durably track your own state um because you don't want to lose that state which means that you would make it unspendable like if if penumbra isn't holding all of your state then that

means it's not just if you lose your seed phrase you you lose access to your funds it's if you lose your entire database of all of these tiny State fragments which seems a bit scary right because it's a lot easier to back up a seed phrase than it is to back up uh like a database that's in motion so we're building narsal narsal is a private uh or like a permissioned personal custody chain for penumbra it's based on tenement consensus and What it lets you do is it does a threshold uh Keys it you do like a threshold signing using it you split your signing key across all of the shards of your narsal cluster um so that when you want to submit a

transaction to penumbra you submit the transaction plan with for your penumbra transaction to the narcel cluster it does within the uh within like and that is a transaction in the narsal blockchain and what it does is it signs the transaction and it gives you back the signed transaction for you to then submit to penumbra and that means that everything that you do on penumbra using the key that's distributed across your narsal cluster is itself tracked by the narsal blockchain so this is kind of an example of like you know blockchains are useful for things even in a less adversarial environment like some people are like oh

well you know you don't need blockchain for just keeping track of a database you could just use a replicated database well this is actually quite a useful replicated database and so we just are going to use temperament consensus to keep track of all of the shards of data and so you get threshold signing and you get keeping track of your private data and as a cons sequence what you can do is save on sending all of the encrypted bits to the penumbra blockchain so if you're an active Market maker and you use this thing you save everybody's work downloading all of your private state that they don't care about and you save your own fees because you don't need to pay for the block space to store the encrypted blobs

that you're storing locally in your own narsal cluster right so uh full stack private protocol design how hard could it really be well just do this think about all the bits Trace all of your data through this think about how to make it faster easier and cheaper and draw the rest of the auroburos thanks thank you very much Finch maybe there are some questions yes questions anybody yes I

questions us um which uh I wonder um was the surprises content that you need to be approved by uh by witness and during our transactions such as for for instance for the uh protocol is managed to earn highs um receiver but for but but according to your um according to the presentation and Europe seems to be it seems to manage to hide the whole transactions

so how to how to deal uh how to do that um can you repeat the last bit [Music] um uh what is I mean what part of this and and of the context of the transaction you need to be put into the business offices oh I see so when we're witnessing things um the stuff that we're witnessing is um the zero knowledge inclusion proof within the shielded commitment tree so commitments to the existence of uh notes that you possess so that's in spends um it's also in swap in swaps in uh and in um

uh in governance oh okay so the community is um because of the house the house value of each and is on coin or uh some um you're proving that you have custody of uh of a note cover note of the American tree or it's a yeah so you're proving you're proving that um the the that a particular note did get uh included in the Merkel tree yeah I'm sorry okay I got um this might be one for offline but

as soon as you I'm very interested in penumbra using um interacting with other chains and using interchange accounts I think I may have incepted Henry to to push this forward um but the resource estimation is gonna seems like it's going to be very challenging um it's maybe not an issue now because it's all subsidized but um you're doing all this work to uh you know multi-dimensional fee markets to to kind of estimate the resource consumption within penumbra um you kind of have a limitation that you don't want to call out to any clear text chain to understand

the resources consumed there um have you what is the end game there if you if you want to interact with other Chans and actually you know actually be able to estimate how how that's going to be paid for well you're the one that incepted Henry to do that so let's chat afterwards yeah that's a really really good question and I I'm super curious to figure it out I do think we can I do think we can do something there but I I don't know the details yet any other questions okay thank you very much Finch

thank you [Applause]