Episode 217: Information Theory & Blockchain with Sreeram Kannan

Transcript

[Music] welcome to xero knowledge i'm your host anna rose in this podcast we will be exploring the latest in zero knowledge research and the decentralized web as well as new paradigms that promise to change the way we interact and transact online this week tarun and i chat with sriram khannan associate professor at university of washington where he runs the uw blockchain lab in this conversation we look at how information theory and blockchain intersect and talk about sriram's early work on p2p mobile

networks mapping bitcoin security and how this evolved to work on consensus fair sequencing and more we couldn't get through everything in this episode so hope that we can have sriram join us again in the future before we start in i want to let you know about a section on the xeroknowledge.fm website that might be interesting for you if you're looking to jump into zk professionally it's called the zk jobs board and there you can find jobs posted from some of the top teams working in ck find the project or team you want to work with next over on the zk jobs board i also want to make sure that the upcoming zk hack mini is on your radar last fall we did a seven week event with puzzle hacking and zk tool workshops now we bring it back for a mini version of

this event it's set to last two weeks and starts on march 1st 2022 if you missed the first edition or if you attended and just want to join again be sure to sign up today for the newsletter you can also already sign up for the march 1st kickoff event there you can get updates on our partners and schedule speaking of partners i'm very excited that our puzzle building partner for this edition is polygon we are very much looking forward to once again bringing the zk community together around zk hack mini we hope to see you there so now i want to hand over the mic to tanya the podcast producer to tell us a little bit about this week's sponsor today's episode is sponsored by electric coin company the inventors of zcash and halo cryptography they're building the next generation of zero knowledge tech

with network upgrade 5 happening this april z cash moves to the halo zero knowledge proving system removing trusted setup and becoming shielded by default they believe privacy is essential to delivering on the promise of a thriving and equitable web 3. visit electriccoin.com to learn more about the relationship between privacy self-sovereignty and creative freedom that's electriccoin.com so thank you again electric coin company now here's anna's interview with sriram [Music] so in today's episode we're here with sriram kanan associate professor at university of washington where he runs the uw blockchain lab welcome to the show sriram

thanks rano i actually wanted to first start this episode with a question to tyrone tarun really brought this episode together and introduced me to sriram so tyrone can you share with us what we have planned for today's episode yeah so you know one interesting thing i think about um the cryptocurrency in blockchain space is that you know it a little bit like ai is able to draw uh talent and ideas from a lot of different backgrounds and you know one interesting sort of view from an academic lens is that you know cryptocurrencies really started mainly with cryptographers and then sort of distributed systems people uh but over time it's sort of branched

out into other academic fields over time and um you know personally i came from sort of this kind of like mixed background as well and i i've always you know when i enter the space i kind of observed like hey there are all these cryptographers who are really bad at probability like there's got to be something that like someone who knows how to how what a measure is can do uh and you know that sort of is actually what nerd sniped me into this space and around the same time i started observing sriram and his advisor and some other folks at different institutions like starting to write papers that were a lot more rigorous from sort of the probability standpoint than had been in the space prior so i think the the idea that you know you

can take tools from other fields and apply them to kind of like subsets of problems that are actually really interesting and widely applicable um is sort of the main theme that we'll go for and i think hopefully with this episode you'll get to see kind of the variety of work that true ram has done and also if you're so if you're say someone who's not in the crypto space but you like studied math or something or you kind of know you know these other fields you'll get an idea for like hey there's actually a ton of open problems here where you can like apply tools from other parts of the world and i think yeah kind of getting a little yeah i think tree rom's background and history will give you kind of an idea of like how you know certain tools and

applied math and and probability can be applied to a really large variety of fields if you take kind of this very generalist lens instead of being a super specialist so shoe ram i now really want to hear a little bit about your background and maybe what nerd sniped you to jump into this absolutely uh thanks rana thanks taran for the kind introduction so i got into blockchain uh in four years back but the story of my interest in this field or peer-to-peer networks actually dates way back to my phd thesis my phd thesis was actually thinking about

how to construct peer-to-peer wireless networks what are the fundamental limits today when we look at blockchains like bitcoin or ethereum and so on one of the big questions we look at is what is the maximum throughput that you can achieve with these kind of like peer-to-peer networks what is the latency that we can achieve with these networks these were exactly the same type of questions we studied but in the context of peer-to-peer wireless networks they used to be called ad hoc networks at that time and there was a massive interest in figuring out can we build wireless networks within the absence of any infrastructure no base stations no wi-fi access points can we just create self-organizing wireless networks

peer-to-peer they just talk to each other and then uh organize themselves into a network and the kind of math problems we were interested in are what are the fundamental limits of such networks can you achieve like you know what's the maximum throughput capacity that you can achieve what is the minimum latency and uh it was in my phd thesis my advisor professor pramod viswanath at illinois urbana champaign and i worked on some of these problems which were basically uh theoretical limits of peer-to-peer networks particularly peer-to-peer wireless networks and so our interest in this uh started already at that point however what happened was peer-to-peer

wireless networks did not take off it's it's because the massive deployment of infrastructure in a wireless including access points base stations optic fibers the massive availability of infrastructure obviated the need for constructing peer-to-peer networks and at the end of the end of my thesis even though we solved some open math problems inside these fields the real world application was lacking so i was sitting around rethinking what should we do now that you know we solve this math problem it all looks pretty amazing but uh in reality nobody's actually interested in

deploying a peer-to-peer wireless network so you had the solution without the problem and that's right we definitely had a theoretical problem which is uh how would you go about constructing it but there was really uh nobody who was actually particularly interested actually qualcomm at that time had a big project which it shelved right around the time when i finished my phd thesis called flashlink which was trying to build peer-to-peer wireless networks my advisor used to work with them and so on so we were thinking that could become a thing but it didn't become a thing because of the massive deploy deployment of infrastructure so it gives me a lot of satisfaction to see helium today which is basically

brought about that idea back into wireless peer-to-peer networks instead of i think a paradigm shift that helium brought in is instead of building just a peer-to-peer network they built a peer-to-peer network and the pool is basically i think this is the kind of fundamental transformation that we're seeing in decentralized exchanges for example order book exchanges appear to appear right a pair trades with another peer but fundamentally the new paradigm you know we're seeing in amm taxes and so on is peer to pool to peer cool so that's exactly the same paradigm shift i think helium brought in they said well it is

infrastructure-based networks but why should only like monolithic companies build these infrastructures can we get coordinate regular people to put up infrastructure put the right in centers and actually create these pools from which we can start providing service so i feel like you know the world has come a full circle they're starting from peer-to-peer wireless networks back in uh 2008 i think 2002 to 2008 2012 there's a lot of academic interest in that and then that died down and i think now for example my advisor promotion he is actually studying how to build these kind of peer to pool wireless networks what are the right and centers what are

the mathematical primitives that need to go in again a lot of interesting random processes uh and analysis showing up there very cool you're talking a little bit about this sort of lull in the p2p research or p2p interest would you put that firmly in that p2p winter that we've kind of heard talked about absolutely so what this is 2008 would you say 2010 when it started to just die out okay yup yup so 2008 i started my phd and i finished my phd around 2011 2012 and the the interest was just veining so there was so the interest in p2p wireline networks was already went by 2008

but we were thinking because wireless is infrastructure was not built out at least all over the world there was still a possibility that peer-to-peer wireless networks could take off but that that didn't turn out to be the case i i think one of the most important things to uh realize from listening to this and it took me a long time to understand this uh is that a phd is really like a venture bet it's like investing in something and then like wait waiting for five years and then seeing if the thing is still alive after five years and choose wisely absolutely sounds like sriram yours has worked out at least the the field has come back a lot of the work that you did i guess becomes relevant again so let's

let's hear about that move maybe into blockchain for you what what was that where did that happen so so after peer-to-peer wireless i didn't at that time it's 2011 so i i hadn't had the same exposure to blockchain at that time i was looking around and seeing what were what are some of the most important things happening and potentially how can we bring this understanding of information theory and networks and graphs into this field so just for our viewers who may not be familiar with information theory one of the most celebrated icons in computer science is alan turing i would say equally important was claude shannon who fundamentally pioneered this view

that all information can be represented digitally without any basic loss of performance it was not at all obvious during the time that he pioneered these ideas back in 1945 48 at that time you know communication was all analog you know radio stations used to use frequency modulation and things like that which were analog methods and he pioneered the idea that almost all information can be represented by bits he pioneered the idea that there is a way to communicate bits without error he pioneered the idea that you can bring up very sophisticated tools from

mathematics to actually solve some of the real engineering problems in communication compression uh information processing and i think it has laid the groundwork on which much of our modern society is built and definitely modern information technology so coming from that field uh the inspiration that we had was can we bring these ideas into other emerging important fields and one of the things that we took a bet on several years again like taran said another venture bet was in computational biology so we spent several years five years working on computational genomics trying to understand dna rna sequencing things like that and it was in 2018

january that my former advisor at the time promote called me and said hey do you know about bitcoin i said yeah i heard of it why are you asking and he said no you know there's this thing called bitcoin it's like super interesting and you know the two basic problems in bitcoin one is bitcoin has low throughput only can process a few transactions per second and it has very high latency it takes like uh several hours to confirm a transaction with higher reliability he said all our life or rather at least my phd we definitely spent all our time thinking about throughput and latency in peer-to-peer networks and the question

was can we redesign bitcoin to actually be very high throughput and very low latency and this kind of immediately caught my attention and so that's that's really uh that the technical in for me into the area and what was the work that came out of that problem like did you end up solving that is there something that we can like point to a paper or anything that describes it yeah absolutely i'll explain uh what we ended up doing so before that i i think you know i was going through a more uh a philosophical problem in my head you know again like thorough and alluded these are like venture beds and to take

a venture bet it's to get into any area and do something useful it's gonna take at least three to five years so that's a basic minimum compounding required to get any kind of results so i was considering wow we're working on all this cool genomic stuff should we give that up to work on on blockchain is this equally important one of the one of my reasons of getting into genomics is you know there's all this crazy things happening in synthetic biology we're going to be able to engineer genomes re-engineer ourselves seemed like a massive paradigm shift and i wanted to be a part of a smaller part but a part of it uh and i was wondering whether blockchains are anywhere near the same scale of

interest and i wasn't convinced for a few months i was dawdling around looking through it and then well it clicked one day and here's what clicked meant for me one of my favorite authors is you all noah harari his famous book sapiens and there he makes this thesis that humans have taken over this planet he summarizes the humans evolutionary advantage in one line he says humans took over the planet because we cooperate flexibly in large numbers and we believe in myth we're we're able to believe in myths absolutely in chad myths help us cooperate flexibly large numbers but the

fundamental primitive he lays out is this ability to cooperate flexibly in large numbers is what has led us to this evolutionary advantage as i started chewing on that it became obvious that any mechanism which helps humans cooperate flexibly in large numbers more efficiently will have a very basic fundamental evolutionary advantage to us as a species and worth investing deeply in whether bitcoin crashes or not so until i got to that conviction i could not jump all in on this new venture bet yeah wow because that is such a shift like you're working on

when you were saying kind of genomic is that mapping genes and it's like biology like super important medical like knowledge and research into knowing kind of how we're built and how it works but and you made a shift but you actually could see that even i mean when you think of blockchain sometimes you do think of like the speculative nature of it and it's a little bit like of a shame if you move from medical to like oh i'm going to make money over here but i think the way you just put that is really cool this is this idea of the the organization of society and you can't even have that deep medical research if you can't organize society correctly so absolutely absolutely i think just like the internet built

information super highways i think blockchain is building the cooperation so super highways cool and i guess is this what led you to look at consensus because that's like the deepest cooperation side of things on the one side yes on the other side that's also our distinguishing advantage that we've actually thought deeply about how to organize peer-to-peer networks how do you get information across these networks at high throughput low latency so for people coming from distributed systems already come with a deep understanding of the consensus problem we did not we came with a deep understanding of the network

problems how do you get maximum throughput how do you get low latency also we come up with the come with a very deep understanding of the random processes that tharon highlighted which played a massive role in information theory and all these wireless networks because one of the most important things you're dealing with when you're dealing with a wireless network is noise noise and noise is random and you have to understand the randomness quite carefully so the first thing that struck me is bitcoin mining is random okay you mine a block who gets to mine a block there is some randomness and if you view the bitcoin blockchain as like a ch chain or even a tree potentially because there are forks because two people can mine blocks near

simultaneously sounded like hey that's a random growing tree can we bring a lens of probability theory and information theory to actually sharply understand this problem so once we understand why these things are actually secure then we can maybe expand it to say how do we get both security and high throughput security and low latency so that was the direction in which we actually came in and i'd like to give a shout out to uh both my advisors you know pramod vishna that illinois urbana champaign who got me into this area and professor david shea at stanford who was my postdoctoral advisor

and the three of us got into this in in 2018 and we used to meet like every day several hours because we didn't understand head or tail of what is going on here and we dropped almost everything else to just uh study this area from like a beginner that sounds so cool and we were one of the questions i just asked before was like is there work that came out of this first problem so yeah what's what's it called so the the paper is called prism okay the idea that we were actually trying to pursue was how do you get low latency as well as high throughput with bitcoin and we realized so just delving a little bit into what what

we did there so first to understand why uh bitcoin takes a long latency to confirm so the idea is blocks are made every 10 minutes right so blocks are made every 10 minutes and just because a block is inside the bitcoin blockchain does not mean you should accept any transaction that went into that block why because if there was an adversary the adversary could potentially mine a competing block at the same depth and take over the chain and your transaction may get double spent on the other chain so the basic fact of bitcoin is the closer the block is to the tip of the blockchain

this the more likely it is to get reverted the more deeply embedded a block is into the blockchain the less likely it is to get reverted and so this is a basic fact and so what this implies is if you want to confirm a transaction at a certain level of reliability then you need to wait for a certain amount of time so this is all there in satoshi nakamoto's original bitcoin uh white paper and we went through all of this and found that the latency is accruing fundamentally because to think of this in probabilistic terms what you're doing is let's say tarun is an attacker and tharon has like 20 percent mining power and

the rest of the bitcoin miners have a 80 mining power now if you just ask what is the probability that tarun will get the next block first that's like 20 because you know it's a race between a 80 mining power and a 20 mining power but if you ask what is the probability that tarun will get the next 10 blocks first as opposed to the rest of the network then the probability is much smaller because you know the other guys are winning the race every time and they're kind of compounding on their wins and over so what is called the law of large numbers which basically says if you average enough then you will always see that the 80 percent wins over the 20

percent so that's the basic reason you wait for a long enough time in order to confirm a block in bitcoin and the question we asked is can we get this averaging this law of large numbers to work in a completely different way which does not require latency so here i want to highlight that there are many other methods people have thought about how to reduce latency in approval work system and one of the simplest is to say that take the last few proof of work miners and then form a committee and let this committee vote on blocks right because once you have a committee you just send the thing to the committee they all sign and then you're done but to us this felt highly

unsatisfactory and the reason it's highly unsatisfactory is one of the really beautiful things about bitcoin is nobody's power everybody's power is evanescent the power is transient and vanishes immediately so for example when i'm trying to make a block you know i'm racing raising racing to make a block once i made the block i don't know when i'm going to make the block and then once i made the block i have no further power in the network special power in the network everybody's power is evanescent so what that means is even for a short time scale nobody has special powers everybody is competing fairly all the time but if you elect the miners from the last thousand blocks to go ahead and

form a committee and then let them run a blockchain or just sign blocks now you have a vested powerful intermediary this committee which can just collude and take down the network or do whatever so mathematically the way we think about it is we think of this as a dynamic adversary or an adversary that can corrupt people after knowing what their identities are and a dynamic adversary cannot corrupt bitcoin it's really magical in that sense in the sense that you know you have power proportional to your actual computational power and you don't get any further power in the network and so this was a very pivotal property for us when we thought about how do you get low latency without giving up

this absolutely fundamental property of bitcoin one thing i always appreciated about trump's research is i i also at one point was doing computational biology and thinking about things in terms of like probabilistic processes generating graphs and graphical models and generative models it's kind of the way that you know i spent probably my first five years of doing research and i think one interesting thing is that in crypto it's actually really hard sometimes to see that those things map as cleanly like things in ml and ai are almost always stated directly in terms of like graph processes and things that like look like these types of well studied statistical objects um but i think a lot of the hard part in

the cryptocurrency industry is a first filtering through the because there's a lot of like nonsense b filtering through the wrong math so like you know the bitcoin paper their poisson distribution assumption was wrong right like for and a lot of what true ron you know i'd say like there's three lines of work that have really tried to fix all of the really shitty math in the original bitcoin paper and don't get me wrong like clearly it worked and whatever but like you know it should be told the truth should be told that the math is wrong like a lot of the assumptions made don't work uh and like this kind of poisson expectation kind of assumes sort of like some extra smoothness on the on the kind of state space that you're looking at um but there's sort of three lines of

work the first was this paper like um and this was the thing that kind of nerd sniped me which was bitcoin backbone protocol which is paper and kind of overly complicated but like it did get the right probabilistic idea of of like how to analyze these probabilistic blockchains and get kind of cap theorem like guarantees that from traditional databases that that people kind of wanted the second was sort of like the elaine she and raphael pass like improvements on that and simplifying it and then the third was really i feel like tree ramp's work which said like hey instead of like trying to analyze a specific consensus protocols what if we took the space of all consensus protocols and then said what properties do you need uh to make things

work and i think a lot of the kind of famous cryptography professors who made you know blockchains that went live like kind of ignored this problem when they made their chains they they really focus on like here's a single consensus protocol and here's why it works and i think like the beauty of kind of the way sriraman collaborators have thought of it is like they went the other way around right they're like what is the set of all of these things and like how do we distill them down and i think maybe talking a little bit about how you got into that how you you guys flipped the problem in that way like going you know kind of top down versus bottom up would be but right before you do that what's the name of the paper you just mentioned uh the there are two papers one is

actually doing the low or low latency stuff that's called prism deconstructing blockchains the other one is called where we actually look at the fundamental properties of bitcoin and many other fam a large family of consensus protocols that's called everything as a race and nakamoto always wins so everything is a race and nakamoto always wins the name was chosen uh by our collaborator david shay who just uh has this enormous respect of course for the deep math stuff but also for somebody like nakamoto who who had an intuitive sense of actually how this thing works even

without actually formalizing it in the same rigorous ways but we were so shocked in in the number of ways in which the intuitions were spot-on while the the math was far lacking was actually amazing you know you have to have a very deep intuitive understanding of what all ways in which things can go wrong and had a mechanism to fix almost all of them i'll highlight a couple of them as as we go on but i think to reinforce what taran said i think when we got into this area we were looking around all kinds of papers they're all looking at very narrow formulations one of the things that we like in information theory you know this is really uh claude shannon's school

what what shannon did is instead of saying hey i'm going to communicate using this method and then let me just try to analyze it which is what everybody was doing at that time instead he says what is the set of all possible potential communication algorithms then can we find the absolute best among all of them what does it even mean right it seems like a simply mind-boggling thing to say that there's a space of all possible communication algorithms and then you know what is the absolute best among the family so this is really a information theoretic thinking on on blockchains so we just brought the same fundamental question which is what is the space of all consensus algorithms and what is the absolute limits on what you can do if

you optimize across all of them like what would be the best one can you actually find something which is close to the best one and this is really this paper everything is a race and uh david went on and brought on some really strong probabilists uh amir dembo uh of zaituni uh who i uh we had the privilege of working with uh absolutely a fantastic probabilist and great students you know david's student nasrat and ashwacha who was promote student formed a big group who actually ended up working on this problem um the basic thing just to summarize what kinds of problems need to be thought about

so when i stated the simple heuristic about how nakamoto did the calculation he's saying publicly everybody is mining 80 percent of the mining power is doing a public mining and the remaining 20 percent uh the attackers mining a private chain and then trying to displace the private chain because in in the blockchain the longest chain wins so this is the attack that he considered but then when we looked at it it was not at all clear why this is the only attack what if i as an attacker try to create blocks on both sides try to balance the chain now there are two chains both of them roughly equal length all the time

and you know the blockchain just gets confused people get confused what is the longest chain and leave you under perpetual confusion is it possible or why is it not possible so even modeling what an attacker can do seemed pretty mind-boggling and this is where the i would say it's an absolutely brilliant paper the bitcoin backbone protocol just came in and distilled it out to its very essence they said an attacker can do four things they can decide what to mine a block they can decide when to release that block they can decide what to put into that block

all these degrees of freedom and all of these of course can now depend on the particular state of the blockchain or if one chain is winning then you do this if another chain is winning then you do that so it's a very very very complex state space of what an attacker can potentially do and now you want to prove security against this massive potential that an adversary has and this is really what was already done in the bitcoin backbone protocol what they showed is if your mining rate is slow enough if your mining rate is slow enough then you can be secure as long as the adversary does not control more than 50 percent then the blockchain

will be secure so it's an absolutely fantastic uh general result which says that you cannot execute all these other complex attacks even if you tried you will not be able to succeed as long as you have less than 50 and so we use that as our baseline to build on our work and what we said in this paper is can we think about this problem even more sharply instead of saying that if your mining rate is really slow then you can actually solve this problem can you find what is the maximum mining rate at which you will continue to be secure was it sort of like were you testing out sort of the parameters of the security of the system are you just like trying

to break it did you break it did you find those limits or so this is this is like theoretical right like it's i think like one one important i think piece of information maybe here that i think to listeners who maybe have more of a background in like traditional distributed systems is traditional distributed systems work has these properties that like are you know safety and liveness of like hey if i send a transaction does it stay in the network and hey if i keep sending transactions can the network keep processing transactions and one of the the kind of fundamental results of sort of like the 1990s was that you kind of can't have your

cake and eat it too you sort of can't have safety and liveness perfectly you have to make some trade-off in terms of like how much the network can partition or things like that and blockchain sort of promised this like holy grail of like actually you can get all three it's just that one of them is going to only hold with like 99 probability one of these properties you want and the rest will be maybe deterministic or also a high probability and i think the hard part in that is that adding in this probability thing blows up your state space at to shreem's point to be this like infinite dimensional sort of search space right you go from this very finite dimensional i can work with the stuff thing to this infinite dimensional thing but in some sense the infinite dimensional thing has a lot of constraints on it it has some

types of decay properties some types of like you can't really like predict the future more than a certain amount of time and that all adversaries kind of somewhat have that property and so in in some sense that if you can formulate that correctly you kind of can get around these sort of traditional database problems and the bitpoint bitcoin backbone protocol is the first to really show that hey there are some different types it's a different sort of like rotating your way of framing the problem such that you can you can write this simply so i think the context is like that was sort of like the 1990s early 2000s bitcoin happened but then like no one gave a good reason for why it happened

and then you know bitcoin backbone protocol came out and then people started being like i can build better than bitcoin right like everyone and their mom's consensus algorithm right so i think then it became this problem of like who's the taxonomist and i think that's where our story goes next so you mentioned it was like a taxonomy move but like isn't it like i guess the question i would have now is like from the work you did were you also developing a new protocol from that did you actually want to build something else because what it sounds more like is that you're mapping the space yeah so part of engineering or at least principled engineering is understanding and seeing

through what is happening so we did both to uh summarize briefly is we actually first map the space try to understand when things are secure when things break and now you have these atomic units that you can combine into new protocols that can actually give you different kinds of performance metrics we did end up the the protocol prism uh some of our collaborators and you know people at mit they ended up building it and actually uh showed that you can get you know insane throughputs for example 60 000 transactions per second you know uh low latency confirmation within one or two block times things like that actually ended up getting

built so it's not just uh a taxonomy it's it's kind of uncovering the principles that underpin the design of these systems we absolutely felt what tarun said which is that it seemed like a simple enough thing you just have a longer strain why not have a longest graph or longest tree or make it a directed graph why not play with that the the space with which you were playing seemed intuitive and simple but the properties were extraordinarily non-intuitive i'll give you some examples a little bit later when we started getting into proof of stake and proof of space where some extremely non intuitive things happened

where if i tell you the protocol it seems like it should work it should be even better but actually it turns out far worse in terms of security so there are all these very non-intuitive things in fact you know one of one of our starting points was there was this really interesting protocol called the ghost protocol by aviv zohar and collaborators which said you know you can get much higher throughput by instead of just looking at how many blocks are in your chain you look at the entire tree and calculate weights and stuff recursively so that protocol claimed to improve the throughput and in fact was adopted in ethereum among other things and when we did a much more formal analysis what we found is the protocol does not actually improve the throughput

it gives you the same throughput as but the longest chain but it does not improve the throughput and it will never be obvious without actually going into the security analysis because there was this balancing attack which was in in ghost if you try try to improve the throughput by making more blocks simultaneously what will happen is you can try to uh you will not be able to figure out which one of the two chains actually got confirmed because i'll keep flipping between the two chains so you never get any uh agreement on to what the real chain is so there are all these very interesting things that can happen with these security things that many many many people were overlooking and it so that was the

taxonomy part is kind of understanding when a protocol is secure what is the limits of security and as you phrased it can you say in this parameter regime which it is secure and in this parameter regime it is simply insecure we call this in in in statistical physics and in information theory sharp thresholds sharp thresholds are saying basically exactly in this regime it's all safe and exactly outside of this regime it's not at all safe okay so don't go don't venture into these regimes and no gray zone it sounds like and this is why it's no gray zone that's exactly that's very sharp that's why it's called a sharper shape absolutely because you know what happens is as things go on for infinite time any bad thing that can happen will happen

i think one thing that's funny is the first time i was on the zk podcast in 2018 or whatever i i remember talking about phase transitions then uh and which which is the same as as this notion of sharp threshold like different fields we'll call it different things like combinatorics and other places will say shrek dash holds other people will you know physics you'll say phase transitions um but you know it's like ice turning into water uh right it's a very sharp change in in in some sort of macroscopic property that averages over all sort of participants particles whatever yeah i think like people didn't have as much of an appreciation for this until maybe like 2020 2019 when like people actually were writing these things in

code and like the network test net would always keep falling over and they couldn't explain why i kind of want to go back to this idea though that a lot of different people were looking at these problems would it be that the the way that they defined the security would actually all like that would then define the type of protocol that they would develop but they might not be looking at all of the axes like who's to say exactly but that's exactly correct but who's to say that like why was your what act like the axes you were looking at were they broader better like why would you also have the same kind of blind spots in the work you were doing i guess no researchers should say yes but no no absolutely absolutely right like cause

self-doubt is very good if you're if you're a researcher and so here is uh uh here is one thing uh one reason why at least what we were looking at is to summarize what other people were looking at they were mainly focusing on because satoshi nakamoto looked at only the private attack which is i grow a chain in private and then i try to overtake your public chain everybody else thought that that is enough so everybody else we're building protocols which are fundamentally secure against that attack but what it turned out is what satoshi nakamoto did is uniquely correct in that that attack was the absolute worst thing that could happen for that protocol

but not at all true for most other protocols that people were designing so so that's why you know the paper is called everything is a raise and nakamoto always wins because there's something like absolutely stunning i'll i'll give you another example which which uh which is also a line of work we did and you know actually pioneered by aguilos quieres and and the people who did the big bitcoin backbone protocol so one of the things going on in bitcoin is variable difficulty which means the difficulty gets adjusted every thousand 24 blocks so people are not looking at what is the longest chain but what is the most difficult chain or what in which chain has most work accrued

so if you look at the mining power in bitcoin right when it started and now it has gone up like 10 to 13 times or something some absolutely insane times so even after all of this the system works and is fully secure because the system keeps recalibrating the difficulty it is required to make a block so blocks are made roughly every 10 minutes and the system every 1024 blocks has a recalibration okay so this is called difficulty adjustment and instead of following the longest chain you follow the most difficult chain or the most worked chain okay and this was already kind of thought in until this point it's intuitive okay you should follow the

most difficult chain that seems like correct okay and then it turns out that there is an attack on this system if you just define it like this and the attack is again let's bring tharon or attacker darwin comes in and he has 25 percent mining power and tharon wants to overtake the longest chain so what does he do what he does is he says i'm going to raise my own difficulty how do you rai how does darwin raise his own difficulty by creating fake timestamps you can artificially you know show that blocks are arriving too frequently in his his chain and because blocks are arriving too

frequently the algorithm will say you should increase your difficulty you should increase your difficulty and if he keeps increasing difficulty then what he can do is if his difficulty is so much higher that one block that tarun makes can overtake an entire chain that the honest guys are making so what this does is there was a law of large numbers which said that by the time the honest guys make thousand blocks they have 80 mining power so they're going to be much ahead of tharon who only has 20 mining power okay but now he has claimed back his fair odds his fair odds was like he should win 20 of the time by saying that one block he makes which is a highly stochastic event

can completely take over the chain so he's instead of saying that he has to make 1000 blocks he's just making one block so he doesn't operate with the law of large numbers he's operating in a random regime so the law of large numbers is what makes things uh deterministic or roughly deterministic and tarun is by raising his difficulty he is bringing it back into the random regime where he his chance of winning is now 25 that's completely unacceptable to run a system but you can't execute this theron can't execute this attack on bitcoin because satoshi nakamoto had an absolutely brilliant fix to it

i don't know if they understood it or not but there was a six which is basically difficulty will not be raised by arbitrary amounts but it can only raise 2x or 4x every interval and and that just makes everything completely different it's one of the most amazing things that i found that you know if you just go ahead and design a protocol you would absolutely not think of anything like this this is an insane kind of attack and you know and and the fact that it has withstood attacks like this uh speaks a lot to the amount of engineering insight that went in into the design of this protocol so so far i feel like a lot of what we've been talking about is very very proof of work

focused but i know that your work also starts to touch on proof of stake does that change any of this like yeah i'm just curious about like where did you evolve into almost like where did you start on the proof of stake train too because that's a whole other kind of realm of research that's right so all of the stuff was proof of work uh and this was 2018 when we started and the end of 2018 there was too much buzz about how this proof of stake is going to be the real thing and you know proof of work is going to go away and all of this stuff and having studied the longest change so deeply our natural impetus was to actually follow the train of work in proof of

stake which worked based on longest chain much later again going back to the taxonomist hat we synthesized the space much more clearly by kind of understanding what the fundamental like trade-offs are when you design a proof of stake protocol so i'll start with that so then we can delve into what the longest chain offers and doesn't offer so fundamentally there are two different kinds of proof of stake protocols looking at it practically you will say that there are longer chain protocols and then there are bft protocols but fundamentally what we are rivers engineer and other people like tim rufgarden and others have also pointed this out that what is actually going on is there is a fundamental tension or a

trade-off between two sides of algorithms one class of algorithms offers you availability what is availability mean available or dynamic availability dynamic availability means even when a small fraction of participants are present the protocol will remain and continue to make progress the protocol continues to be running even when only a small fraction of participants are present so these are called dynamically available protocols would this be like when there's like 30 active or like what what's the 30 or even three percent active the protocol needs to continue to make progress for example bitcoin of course continues to make progress even if the mining power like the china thing

happened you know miners moved out of china and bitcoin is running effortlessly because it's a completely dynamically available protocol as the mining power increases or decreases there is no denominator on what the world's mining power is so nobody knows what to normalize it by and so it's self-normalizing so it just keeps adjusting itself and moving on so we call these family of protocols dynamically available but if that's one side of a spectrum is it sort of like are there some pos chains which are more dynamic have like how does that work actually like they have a lower threshold needed in order to remain dynamically available or something that's right that's right so there are two absolutely like it said it's at the very core of when you design

a proof of stake protocol do you want to be dynamically available or do you want to be finalizing okay so that's the dilemma is either either you're dynamically available or you're finalizing what is the finalizing protocol is you get uh safety even on even when the network is asynchronous when you know some nodes are if when the network latency is really really high you still have the system to be safe so those are called finalizing protocols usually the class of bft protocols asynchronous bfd protocols will fall under finalizing which is you know tendermint which is hot stuff so

i'll give you examples of real world blockchains which took one or two one side of the fork or the other side uh aguilos and others who were pioneered the longest chain bitcoin backbone protocol and and and they went on to build a whole family of probably secure proof of stake protocols called auroboros underpinned much of the rigorous longest chain protocols cardano is one of the the biggest uh ones following that but also polka dot uses longer strain protocols there are smaller blockchains which use proof of stake which use these longest chain protocols on the other side of the spectrum are protocols which are finalizing dapper labs flow the flow blockchain uses hot stuff which is a finalizing

protocol tandem and cosmos is a finalizing protocol and there are others like ethereum two which want to be both okay okay where does ava fall in that i just had i was more on the finalizing finalizing side so it's closer to hot stuff flow that's right tendermint that's right okay cool but with with some weaker guarantees right like it takes more liberty with like its voting mechanism so there's more variance in like the expected latency for instance and stuff like that but it also means you get to use fewer resources and potentially lower communication complexity so it it anyway there's maybe that's like too much detail but i feel like that they're kind of in the middle they're they they have some things that

are a little bit unfortunate so i want to just like state the the the sort of spectrum again so it's from the do we say this is like dynamically available or finalizing and they're sort of on either side i i had an interview um some time ago that i've cited a few times with the tai abraham and there he mentioned synchronous and asynchronous are those the equivalent terms are those different yeah they're related but not equal okay so synchronous means network uh network has a guaranteed time to deliver all messages so you're making a very strong assumption on the network that it will deliver all messages within a certain synchrony bound which is a a latency let's say like in 10 seconds

all messages will be delivered a protocol which makes such an assumption will be called a synchronous protocol an asynchronous protocol is one that will not make such an assumption the family of asynchronous bft protocols would be finalizing because you know they are you cannot have a protocol which is safe under asynchrony and also available because you don't know whether you got enough messages to make progress right because when you say oh only ten percent of the nodes have signed i'm still going to go ahead and make progress you don't know is it because the remaining 90 percent were honest nodes and they were trying to sign something and their messages got delayed how are you going to go make progress you need

synchrony in order to get dynamic available okay so that's the that's the thing but i think as a fundamental property for the end user operational property it's much better to think about are we dynamically available or are we finalizing okay and and the benefit dynamic availability is obvious as users come and go you know it's a come as you please kind of like a model and the benefit of finality is also there which is if somebody does something wrong you can penalize them because it's usually based on signatures when you know the committee or size or whatever they sign something and if you double sign something then you can be penalized

so so the taxonomist in us is basically trying to categorize this space and say our protocols falling in this space or protocols falling in that space is it possible that a single protocol can be both that was another question actually this is another one of those things you know in academics actually david shea and his students started working on this and promote and i started working on this simultaneously independently and we were both working on exactly the same problem can you have a single protocol that is both finalizing and dynamically available and there are some basic theorems which tell you that it's not possible it's called the cap theorem and and and i think thorn alluded to that so the cap

theorem says you can't be both in in modeled in this blockchain language it says you can't be both dynamically available and finalizing i wanted to ask you since since you had just mapped out where certain protocols fall i had a few others i kind of wanted to ask you about yes please um where does salona fall i never know what it is is it tendermint is that no it no no it's just pbft and the proof history thing is kind of like a heuristic i wouldn't call it like you know yeah so it's it's basically a finalizing protocol with some protection against this thing called the long range attack okay so and i'll come to that when we explain some of our work in in that space basically so if you look at the tree of all

protocols you have pro work and then you have proof of stake in proof of stake you have either dynamically available or finalizing so by the way you can't have a finalizing proof of work protocol because you never know what the size is that is enough to finalize something okay because you the total amount of mining power in the world is unknown and unbounded so you can't have a finalizing proof of work protocols approval work protocols are all dynamically available on the other side your proof of stake protocols and pro-stake protocols could either be dynamically available or finalizing but you know one difference between prover work and proof of stake is proof of work protocols uh do not have the same level of long range attacks which is history reversion

attacks that are quite possible in proof of stake protocols okay so that that's another like meta thing and solana has some defense against this kind of long-range attack using this proof of history but but it is heuristic it's not like using a vdf where you have a guarantee it's it it's much better from an engineering standpoint than a vdf but it is that's it you can't really prove that it works it has it it makes a lot of weird sort of assumptions when it's used let's now take that step towards answering the question can it be both dynamically available and finalizing actually we it was not our original idea

it was actually vitalik's original idea for trying to figure out if you can get both and the way he was thinking about it you know i don't know if people remember back in 2018 there was this thing called casper casper was supposed to be something built on top of top of ethereum proof of work which gives finality so this is called casper the finale yeah the friendly finality gadget exactly and when we looked at it i said man he's trying to have his cake and eat it too is it possible or not possible or is there like some trade-off they were not complete proofs for these things you know they were hints that this may work may not work what's going on okay and actually the the casper is kind of

built into ethereum too now called gaspar it's a combination of ghosts and casper called gaspar and i i think it's not called ethereum 2.0 anymore as of yesterday but uh yeah i think the tm foundation i think the aetherium revolution okay okay yeah i have heard before that that they were distancing themselves from the word from the name yeah i know it i think it's called just consensus update or consensus layer or something but the consensus update which is basically called gaspar what happened was david shea found that there are many attacks on gaspard so this was a paper they wrote early this year and uh early in 2021 that got a lot of attention and so david's

working with the ethereum foundation on how to fix several of those things we actually we did an episode i think tyrone you and i did this like years ago or two years ago maybe or one year ago two years ago i think it was like the beginning of the pendulum yeah i think it and it was with um one of the co-authors of gaspar yeah we did an episode on casper so i'll try to i'll actually try to dig that up if people are curious but i didn't i haven't since heard about this this work finding the problems yeah so the thing is gasping was trying to get both basically the way ethereum 2 works is there is kind of like a longest chain but people eventually vote on the chain enough and then that becomes final so it's kind of like having the features of both longest

chain and finalizing so that that's the basic class of things and the fundamental questions we asked is can you be both finalizing and dynamically available and the way you can be both is you have a common underlying protocol but different ways to confirm things inside that protocol so anna you are a very let's say conservative person you want to only make very safe claims let's say you're running a bank or whatever so have a common underlying protocol and you'll only accept somebody's payment if it has gone through a very safe rule whereas tharon is running a gaming engine and he's like saying hey you know let's get things moving man so what taran is doing is it's the same

underlying protocol but he's coming at it and applying a different confirmation rule on the protocol which keeps the system running live and the the theorems that you know david showed and we showed in separate papers by the way the end of the story was david beat us to the race and we wrote our paper like uh two weeks later with uh uh promote student church and suya and we ended up basically uh bringing more properties to this kind of a protocol basically can we build a protocol which is both dynamically available and uh finalizing fundamental blockchain is run by the same protocol but anna can use a different confirmation rule and tharon

can use a different confirmation rule in the same but at the same time like can you do that at the same time but at the same time and and so you're led so the safe rule will always be lagging the live rule because you know you're only making things that have been like signed off very very finalized somehow yeah very finalized so the way you can think of it is already in bitcoin you know you can use the confirmation rule which is like 10 blocks deep yeah and tharon can use a confirmation rule which is only one block t okay okay and so it's it's like that but much more complex and much more general and so that's the space of things we figured out and so of course now david is working with the ethereum foundation trying to understand how you can make sure the prop protocol has

both properties but different confirmations what was that work you just mentioned when you said you released this paper but later yeah so the paper is called uh blockchain cap theorem allows user dependent adaptivity and finality okay it's basically user dependent adaptivity and finality so either you're adaptive which is dynamically available or finalizing depends on the user but the protocol itself is the same underlying engine and actually other blockchains have also been thinking about this polka dot had something called grandpa which is their finality gadget and so we kind of integrated all of this into like a common picture which is again the taxonomist one one thing that's also interesting to

note here is that a lot of layer twos effectively offer this sort of similar promise of you choose what level of quality of service you want oh yeah and so you probably can generalize this to some of the layer twos uh although of course you need to make different latency assumptions because there's two different chains and sequencers and stuff like but but like you could argue that layer twos were already trying to go in this direction but like didn't know why they were doing it other than it was easy engineering wise i think it was because it was so slow to actually write to the main chain right you needed a faster approach well i i guess i meant in terms of like their security guarantees right like how how people design layer twos based on security guarantees is still quite heuristic right there's no there hasn't

been the bitcoin backbone protocol paper for layer twos in a lot of ways a lot of layer twos are driven by engineering needs and not like mathematical soundness needs which is why you know i think they've been somewhat hard to get to production uh so far absolutely i think um so here i've you know at this point we'll also ask something anna raised earlier which is what are you not doing you know you're saying you know you're modeling all the adversary thing and all this what are you not doing what are you sneaking under the carpet i'll tell you exactly what we know we are sneaking under the carpet okay we may not know other things what we do know is fundamentally when you try to mathematically model these complex blockchain systems

again there is there are two fundamentally different views one view says i'm going to assume that x fraction of nodes are protocol obeying you can call them honest but i think maybe the right thing to call them is altruist they'll just do or you can say obedient if you want to appear differently but basically these nodes just simply tow the party line basically they're going to follow whatever the protocol founder whatever team says they'll just run that particular instance of and there's a fraction of nodes which are non-protocol following and they can deviate arbitrarily they can do whatever they want so this is one kind of model we can think of this as an adversarial model of blockchain some group of nodes

are adversarial the remaining are altruist they're called honest but basically altruist they're just doing what they're told and they're doing it because it's good for the network they're doing it because they're they're already happy with the revenue they're getting whatever we don't care we don't model why some nodes are honest and some nodes are adversely we just plop them into those groups and say either you're honest or you're adversarial okay so that's one kind of modeling there is a completely different kind of modeling which is saying nodes are rational okay now when you say nodes are rational they're trying to maximize kind of some utility which is how much maybe money they're making it's it may be factors in some of their

real world reputation we don't know what a utility function is so the problem with the second class is it's much much more difficult to state things very sharply everything that you state is under somewhat weak models because it's very difficult to model the full space of what utilities are and what rationality is so what we had done for several years of say 2018 to 2021 is completely focus on the honest versus adversary or altruist versus adversary models but over the last one year or i've been thinking a lot about the incentives which seem to drive a lot of

user behavior that's a fundamental disclaimer fundamental disclaimer is when you see a paper and it says hey you know some nodes are honest some nodes are adversarial okay it's good in some sense because it can allow the adversaries can do arbitrarily bad things and still it's going to work but the honest notes have to absolutely follow protocol and then you can model it rationally for example um amin good sire and and all their selfish mining attack is basically uh rational deviation right it's not a honest adversary thing it's basically i'm going to maximize my like revenue source so when you look at a mathematical model you have to first calibrate is the

assumptions what you want or will these assumptions hold are there reasons promoting the holding of these reasons all of these assumptions or not and so that that's a that's a basic uh fork in the modeling choice when you talk about this rational though like is there not still sort of the like irrational following their orders honest in that like do you really believe that all will act purely based on incentives no no no i'm not i'm not not at all saying that everybody is incentive driven and also incentives have a meta layer which is super hard to model and the meta layer may be the following like social in bitcoin there is a very easy

there is a very easy bribing attack but if you execute this driving attack you may collapse the value of bitcoin measured in bitcoin maybe you make more money but measured in u.s dollar you make less money so there are all these meta things that drive into incentives which are very hard to formally model so i'm not at all suggesting that this is the absolutely correct modeling or anything like that but the reality is there are byzantine nodes who don't care they just want to take your maybe they're a competing protocol they're coming and shilling on twitter that hey your protocol is is bad and i'm going to take it out so there are byzantine nodes there are altruist nodes and there are rational nodes so the fundamental correct

overarching mathematical paradigm will be what i'll call a bar model byzantine altruist rational all mixing together in like one big party and trying to characterize what fraction of byzantine what fraction of ultra is what fraction of rational can your system endure and can you really slug it out this was actually our original formulation when uh when david and promote and i were trying to formulate the mathematical problem of blockchain but we said this is way too complex for our heads and the math to handle simple it down to just let's say there is bython time can we solve it now let's say there is rational can we solve it let's combine everything together can we solve it so and and we see this kind of

playing out in in practice in like various ways for example ethereum one of the main things that ethereum two now called just their consensus layer is thinking through is the crypto economics and cryptoeconomics to me is just how to make sure it's rationally incentive compatible to just follow what people are telling you to follow and can you make it more and more compatible to actually follow it maybe if you attack you're going to get slashed this is one of the basic primitives that casper introduced and like many others followed suit the idea of slashing which is that if you deviate you're going to lose your money and that's the economic game there that's the economic game there exactly

and and and by the way you know even though in theory people are talking about all this i don't know of any real large-scale protocol that actually does slashing it has happened i mean in cosmos it definitely happened i remember that somebody got slashed but it was usually accidental that's the problem it wasn't really malicious that's the reason that people are worried when they want to bring slashing is are you getting slashed for a real reason or you know it's just some kind of like discordance in the version you're running and you can get slashed so absolutely that's uh the risk with building slashing and so it's going to take time before like slashing becomes like an automated thing rather than you know it's a con it's a social consensus thing that if somebody gets slashed i

can go and unslash them because you know they were all just honest nodes out of sync yeah yeah i mean i mean so people do a variety of very weird kind of heuristics to avoid slashing right now for instance i think in solana they have this like thing where they keep track of like validator statistics and i think it's actually like central i'm not actually sure who how they get collected and disseminated but there's some notion of like what the distribution of validator uh times is and then you kind of get penalized based on like how far you are in that distribution in general yeah designing the slashing conditions are is really hard because like at some level right it has to be just as dynamic as like the fastest type of attack in your system but like the

system is trying to also slow things down and so like it becomes like very like multi-time scale optimization problem which sucks absolutely i think we've we observed this kind of central difficulty in slashing and the idea is this you know slashing is applied for safety failures when you have two blocks you know which are simultaneously confirmed actually we have a paper on when can you slash this paper is called bft protocol forensics uh this appeared in this year's ccs and was selected as a best paper runner runner up but the the core idea here is the when can a protocol have slashing and what we found is that people

normally think you know if you have enough signatures and people double sign you can slash what we found is that's not absolutely true because there are scenarios where you can create safety failures two things getting confirmed without actually double signing and for example algorand is not slashable fundamentally the way they have designed it you even though it's it has signatures and quorum and all this it's not slashable so looking deeply into the mechanics of what makes a protocol slashable is another line of work that we kind of embarked upon in this in this taxonomy of consensus protocols so so like i i like to view a lot of this as as like the natural maturation

of the space right like if you think about things in 2017-18 there's a lot of wild west like hey i wrote this paper i wrote some code like like as a prototype like hey it's gonna work put hundreds of millions of dollars into it and i think you know at that time everyone was like the math doesn't matter like who cares whatever the engineering matters more and like you know there's always this interplay in this industry i think in general where like the engineering can oftentimes get ahead of the theory but then the engineering hits a roadblock as we're seeing with layer twos where i actually think like the theory is actually going to be very important for ensuring that they actually can get to production unlike a lot of layer ones and you know there's some roadblock and then

it takes a while for people from other fields like provide theory you know like if you some of the people mentioned earlier like kiaya i can never say his last name correctly yeah yes thank you uh and elaine she and stuff you know they were kind of working in related fields but like tangentially and then brought in new thoughts and i i think in general you know it's like each kind of wave sees like this improvement in theory i i think like the moral of the story is there's kind of this like meta meta game between the engineers and the at like you sort of more mathematical researchers in the space competing over who's uh whose design actually works and prod better [Music] yeah so so just uh uh i think there

when we look at security right performance can be determined by engineering you just run things and check you know you you know if solana is running 120 000 transactions per second or not by just running it but you don't know if solana is secure or if you don't know ethereum is secure by running it because the space of all possible attacks is simply too mind-boggling for like people to even realize this is the basic thing in cryptography was you know you cannot just say that you're a signature scheme or like a zero knowledge scheme is secured because nobody has attacked it yet instead you rely on very basic axiomatic assumptions saying okay calculating the discrete logarithm is

hard therefore you know this scheme is secure so this principle somehow did not yet fully manifest itself in the blockchain world because protocols seem like engineering things they are engineering in one dimension but the dimension that they are engineering is just performance there's no you know to prove that one thing is more performant than other is not that useful i mean it's useful to some extent to understand the taxonomy and principles but i'm running it and i'm actually getting better performance nobody cares about your theory in some sense but to prove that something is secure you cannot just run it and show you know

what did you do you know did did your friends attack it did your enemies attack it did you put 100 billion dollar on it and let it sit for 10 years and did you move the entire wall street into your application and see if it's secure it's not possible right it's simply not possible to do any of this you have to have mathematical methods to think through security in a way that you do not necessarily need for performance so when people come in from an applied systems view and say hey i've built this system it's great in engineering we can absolutely give them credence for building high performance systems but whether their system is secure has to be saved through a theoretical lens

and here i want to give like a big shout out to like uh aguilos elenshi raphael paz all of these people who brought in this lens you know uh and and in particular this was a very cowboy academics you know at that time a lot of their peers were like you're going into the scam thing yeah yeah exactly and and they've they stood there and wrote paper after paper after paper seeing through the ab you know core principles in this it's absolutely amazing as you're analyzing a lot of this work i mean i think a question that inevitably will come up is what happens when you think about mev like this other vertical this other

i mean maybe it was taken into account i missed it but i feel like it hasn't been fully kind of chat like brought into any of your modeling and obviously it's a very big topic right now so what what are you thinking around that what work are you doing absolutely so when we started seeing this whole uh mev thing explore i remember first time i think ari jules was giving a talk on flash boys 2.0 how these things were happening and how they blogged about it and more people started just like front running and you know arbitrage bots all of this stuff and thinking that this there must be like a fundamental mathematical problem underpinning why mav is happening and

one way to think about mev is it's really an incentive thing right if you're a miner you are obviously going to prioritize transactions that pay you more fees and put them ahead than transactions that don't pay you that much so that leads to this whole idea that you can front run transactions by just increasing the fee that you're paying the miner okay this is an incentive driven way of thinking about it and of course we were much more familiar or deeply embedded in the honest and adversarial way of thinking about it and the adversarial way of characterizing this problem is the following saying that if you are an adversary and you are mining a block you can unilaterally determine the order of transactions in

this block if you're just running the get node right you're going to order transactions in a certain way but if you're running some other version of the node you could order transactions in the way that you want so that's an adversarial behavior and the point is anybody can mine blocks and an adversary can mine blocks and their mind blocks will have a very different order so then we went back to the basics of like distributed systems and we see that there are really two pillars of distributed systems safety the security one is called safety which says that a confirmed transaction should not be reverted another is liveness which is an honest transaction should eventually get confirmed

but what we are looking at here is a fundamentally third dimension and the dimension is is the order of transactions in the final ledger how is that related to the order of transactions entering the network so it's a concordance property between transactions flooding into the network and transactions entering into the ledger so people had thought about it a little but no deep work had gone on to create systems which give strong guarantees on these uh on this concordance between the arrived order and the finalized order and would you say like high concurrence or i don't know if i'm saying that word right but

like if it's high in that dimension does that mean that it's more similar so there's less kind of like funny business happening in the mev camp exactly exactly right imagine so in fact this is one of the big properties that a centralized exchange offers today like a new york stock exchange will take enormous efforts to make sure that transactions are processed in the order in which they are arrived going so far as trying to synchronize their servers across different cities to millisecond resolutions and so on okay so this ordering fair ordering is also a mandated property by something like the sec because you want to make sure that the big guy who can pay for order or

does not outrun the small guys who cannot pay for orders yeah and we were seeing you know ripples of this in things like robin hood doing order flow things like that i'm just realizing like early early stock market probably had like the mev bought equivalent which was like the person who ran the note from one place kind of all still happens that having having worked in hft it's the same thing it's just that it's more competitive on a hardware level than it was before i i'm picturing like when orders were like written on papers and like run across the room i just want to like have a vision of what a like mev bot would look like if they were corporeal and i'm like picturing it's

like that's why i'm saying like early early like pre-digital but yeah i guess it's it's something that's always been there it it it's it's always that but once it arrives at the exchange there is no unfair ordering but to arrive at the exchange you can always kind of try to get things close to the exchange or whatever but coming back to the blockchain i think the guarantees offered by today's blockchains are far worse than the guarantees offered by an a centralized exchange because because you can front run just by like putting more transaction fee and that's just like a zero bar on like what is needed to do front running so what happens in this world is if you're doing large trades you can put in a

higher transaction fee if you're doing small trades you cannot put a higher transaction fee so you're obviously going to get front run and to me this goes against the spirit of decentralized finance what are we decentralizing if we cannot offer at least some protections for people whose orders are small okay so what we are saying is can we create a system which does fair ordering and fair ordering means if the or timing in which transactions entered into the system is something then those transactions if a transaction a arrived into the system earlier than transaction b and you have to define it and so on then it should appear ahead and in fact our our

work shows that on both types of protocol longest chain protocols as well as confirming protocols finalizing protocols like bft you can actually modify the protocols to have very little overhead but still have a fair ordering so that's the line of work the although one thing one one caveat here right still is in the themis result you still have some level of trust in this aggregator where there's like some it's like somewhat permissioned right oh yeah so so let's just give her to to give a quick overview of what that is so normally in these bfd protocols there's a leader and the leader makes a block and sends it to everybody

instead the leader does not make a block or is not supposed to make a block unilaterally instead the leader has to collect everybody's viewed orders because you know the order arrivals are different at different nodes it's a distributed system so what the leader has to do is to collect all the different people's orders of different transactions and then come up with an aggregate order and that's what they have to put it on the on the chain and if they did something wrong it's a slashable offense because anybody can challenge them and then if they don't show it then they can get slashed so the point the turn is mentioning is of course now it's still dependent on one node but if the node does do something wrong they can get slashed and what you

can do is you can increase the scale of slashing by saying that not just one node needs to sign on it by 10 nodes need to sign on it so 10x the collateral can be slashed if you want the entire network to be slashed you can say that everybody needs to sign on it so there's a kind of natural trade-off between complexity which is how many people need to sign on stuff versus like the guarantees that you're deriving out of it or the the slashing that you want to derive out of it so that's the finalizing protocol work called themis famous that was the that was what i was looking for what's the name of it themis that's called themis and team is built on top of some of the work done by ari jules and a student mahima kelker earlier in collaboration with them my student

shawbik and i worked with them to actually come up with this protocol called themis which is a way more efficient they had already come up with the basic principles of what it means uh that protocol was called equitas equidas is a bfd protocol which is offers some kind of fair ordering what we did is both improve the scale of fair ordering as well as reduce the complexity back to what it would have been without adding fair ordering okay so that's one line of work the other line of work is on the longest chains and this is something also another thing happening in in things like ethereum two is short reorgs which is can miners kind of reorg the last few blocks

and you know our protocol called hamurabi now going back to ancient babylon i mean you have venus you have babylon thermos and hammurabi you're you're just like uh you're really going archaeology protocols it's kind of ironic to like name the newer protocols after older things like it's like you're going backwards in time to make future things you know further back no but i think some of these things actually organized and created law and order and organization and society at that time and potentially newer protocols that's an alpha leak i don't think anyone ever explained to me your choice of paper names until this

moment absolutely so we also have longest chain protocols where you do not have a miner make ordering unilaterally the ordering comes out of an emergent understanding or a consensus among many people presenting their views of how the order should be and then that gets synthesized into the finalized ordering so that's the line of work we have on on this direction all of this stuff like in just an interview i just did recently it was actually talking to a team building the bots that like exploits mev and i'm wondering like you know here you talk mainly about the miners and like

the work for the miners but like do you actually remove the ability of these bots to use strategies to actually get something maybe it's not all the attacks but are there still some like attacks that get through absolutely i think the scope of attacks is something you know if you you can broadly categorize front running as targeted front running which is you know i want to kind of sandwich your transaction anna uh and and you know extract some mev from it that i would call as targeted front running and then you can say i want to do non-targeted front running imagine the uh price fall of bitcoin like a two to three days back what i could do is i say i don't i don't want to particularly

front run anybody i just want my transaction to go before other people's transaction because i know the price is crashing i just want to get out so there are a scope of methods which can defend against targeted front running basically by saying that i obfuscate or or just send a commitment of what i'm going to send and then reveals like the threshold decryption type thing this is exactly threshold encryption absolutely so that'll solve targeted front running because i don't know your transaction to target it but still there is a massive scope of non-targeted front running which is price is crashing i want to take all the price advantage i'm a big trader i'm

going to put up a big transaction fee and go ahead of all other small traders even though they actually sold ahead of me this is basically like creating an inside system mechanism for order flow which you know people don't like about robin hood and citadel and all these guys coming together but what we feel is that the protocol should offer the absolute strongest guarantees on timing the protocol should offer the best standards of timing on top of it when you build your d5 applications you could absolutely do things like you don't respond to each transaction separately you batch them up because you know you don't want just because you know taran

sent an order early i don't want to give him like an advantage i'm going to batch things at a slower resolution in fact this has been suggested over and over for things like the stock exchange and you could absolutely build it once you have a fair ordered system you could build these batching things at a higher layer but you have a non-far ordered system you can never retrieve the people who got screwed in the unfair ordering so sriram i know that you have a lot of work a lot of papers a lot of like kind of various works happening and we probably won't be able to you know get everything into this episode we're really at time at this point but

i'd love to hear what are you working on right now like after all of this work behind you like what is what are the focuses maybe you can highlight a few things and maybe we can bring you back some time to go deeper on these absolutely so one thing is you when we talked about slashing you know how do you build the system you know even more basically you think about proof of stake on the one hand has certain kinds of properties proof of work has some kinds of properties can you build a system which is better than both so this is a question that we've been kind of ruminating over and one particular direction that's culminated in is a protocol called babylon which lets you derive security from bitcoin to protect yourself you know your proof of

stake network against long-range attacks so suppose you want to build a cosmos chain but you want to also be resilient to like long-range attacks you also want to be resilient to let's say your own token market cap is only like 100 million dollars but you don't want your security to be bounded by your own token market cap instead you want to borrow security from like a security powerhouse bitcoin how can you do that so that is a theoretical project we did called babylon and david shea was a professor at stanford and my former advisor has taken it up and actually building a project around it another archaeology project just a note no actually building it this engineering project actually they're trying to to borrow bitcoin security on the other hand uh i myself

have been on leave over the last eight months building a company called layer labs and layer labs is trying to solve a problem which i goes which goes fundamentally to the root of governance and blockchains if you look at blockchains today like ethereum or any of the other blockchains how does innovation get in and move the practical blockchain industry it's very difficult for innovation to find a place in existing blockchains because blockchains upgrade through unanimous or near unanimous agreement which is a very rare thing in society what do we all as a society agree on unanimously it will be almost nothing but you know blockchains have to have unanimous agreement or near unanimous

agreement to get new innovation in this has bugged me a lot why because we come up with these cool algorithms we don't want to necessarily start companies or anything we just want to have other people adopt it if bitcoin were a company absolutely they would have adopted ideas like prism inside it but bitcoin is not a company and that is to its credit because it's a decentralized network so how do we build decentralized networks that do not have unanimous dynamics we need to build decentralized networks which have opt-in dynamics for example can we take the ethereum network and say oh twenty percent of ethereum stakers like my new protocol or like my new idea

they can provide service on this what does it mean how do you build systems like that i'm just trying to solve the problem that we faced as innovators in this space again and again when you come up with an innovation you have to build your own trust network let's say solana comes up with a new virtual machine why do they have to build a new network why not just run the virtual machine on ethereum because there is no way to run a new virtual machine on ethereum so this we face again and again and again and i think this is massively slowing down the pace of innovation in blockchain because new innovation needs to build new trust and by decoupling innovation and trust you can actually if you can get ethereum have opt-in dynamics if you can have

solana have opt-in dynamics on new innovations you completely change the equations of adoption so that's what we're building in layla it also sounds like you're almost like rethinking the role the agents that actually do a lot of this kind of consensus building like the validators would potentially change or the miners i guess in some cases like would they dramatically be like a different beast in this i i think so i think we have to give credence to the free will of these validators so today you say oh i'm you everybody uses a terrible ethereum virtual machine or whatever you know this concerns this protocol because it's universally accepted by all of us is good but why not each of these ethereum staker also has a

free will to say i am putting my eat at double jeopardy not only providing the service but also running like a new evm 3.0 or like you know a solana c level virtual machine on top of ethereum what does it mean so basically addressing the the scale of what is agency here in blockchains validator is not being passive no they just like run the thing you tell them to run instead having active agency in deciding their own risk reward trade-offs and potentially earning additional yield by actually providing much more fruitive crazy uh you also start to create almost like subsets in the chain itself almost sub communities

sub applications this is very interesting this also goes to something that theron asked about l2s how do you build l2s which have strong value alignment and full security of l1s is is something i think we can address then we'll we'll hopefully talk about all this yeah so i think we're going to have to invite you back maybe in the next few months to continue this conversation we've we've covered amazing ground so far i think in this episode and a lot of your background what led you to a lot of this work um and yeah i guess you've kind of just teased the audience a little bit here with what we can talk about next absolutely i'd love to come back i'd really enjoyed the chat uh thanks taran and anna for bringing me here yeah

yeah i mean you know i just want in a lot of ways you know i think a lot of the there are a lot of engineers who listen to this podcast who you know find ideas or or things to implement or or things like that from listening to this and maybe aren't so deep in the theory so i think sometimes it's really good to give some exposition of what's been happening especially since there haven't been conferences in two years or like really many conferences like soon like live conferences so i feel like it's always good to hear hear about these things absolutely yeah thank you so much for coming on the show and i want to say a big thank you to the podcast producer tanya the podcast editor henrik and to our listeners

thanks for listening you