Tarun Chitra: Gauntlet – The Simulation Platform for Blockchain Protocols #356

Transcript

this is epicenter episode 356 with guest taron chitra hi i'm sebastian cuccio and you're listening to epicenter the podcast where we interview crypto founders builders and thought leaders on this show we dive deep to learn how things work at a technical level and we fly high to understand visionary concepts and long-term trends if you like the show and you'd like to support us the best way to do that is to leave a review on apple podcast it helps people find the show it helps people know that we're one of the best podcasts in the crypto space

and we're always happy to read your reviews so if you're on a mac or ios device the easiest way to do that is to go to epicenter dot rocks slash apple today our guest is tyrone chitra he's the founder and ceo of gauntlet gauntlet is a simulation platform for building financial models of blockchain protocols and applications so when you're building a blockchain protocol these days it's become expected to have the code audited this is like you know an essential thing now and it's a good thing of course and a security audit will look at the code to make sure there's no bugs and usually a security audit goes into mechanism design but conley goes even further and they do simulations on the mechanism and on different layers

of the stack so they use machine learning methods to simulate different environments with various user behaviors and to see how the system holds up under those conditions so they perform analysis on things like the core mechanism to test for liveness and block propagation and they can also do higher level analysis and test entire markets to see how the market will react to a certain protocol so this was a really cool interview because it's heavy on machine learning and statistical analysis which is not an area that i'm immensely comfortable with but i find it really fascinating nonetheless and when it's applied to blockchains i think there's a potential here for this to become you know much like security audits something that becomes expected by users by investors and by the

community for new protocols and you know that's probably a good thing because as we've seen recently these protocols can lock in a lot of value they're also doing research around transaction fees which apparently is an entirely new space in the area of mechanism design a lot of the mechanism design research apparently focuses around things like auctions for like google ads and things like that and of course this is an area in which like you don't really need to consider transaction fees because you know transactions are abundant and free whereas with blockchains that's of course not the case and so given the the current surge of transaction fees for example in ethereum it's an area that's going to become increasingly interesting and increasingly relevant

as bull markets have you know these these effects on the transaction fees in a network so at the beginning we talked a lot about chip manufacturing which is since tarun was previously in that sector he's got some interesting stories there and we also briefly very briefly touched on algorand which is convenient because they are sponsoring this episode so if you haven't heard our episodes on algorand i would encourage you to go back and listen to them we did one in january with steve coquinos and silvia mckelly and one way back in 2017 with salevio this was before al grand was even a company he'd just written the white paper at this point anyway they're doing really cool stuff to improve developer experience specifically around building defy applications

and i'll tell you all about that a little bit later on during the interview but for now i give you our conversation with tarun chitra we're out today with tarun chitra he is the ceo and founder of gauntlet gauntlet network is the domain name is it actually a network or is it uh sort of just a company so i tried to get all the gauntlet domain names unfortunately the only one that was really open was dot network i think now we're going to try to see if there's a tld.degen and by gauntlet to gen but we have gauntlet.5 but basically in 2017 and 18 i bought the dot network name and

it kind of stuck and we had to incorporate with some name so i chose networks so great to have you on you have a really interesting background that gives you a bit of a different perspective from a lot of the people in the crypto space or especially at least in the defy space could you tell us a little bit about it what were you doing before you got in involved with crypto after i graduated college i worked at this billionaire's research institute it was called de shra research and this person who worked in in trading he wanted to spend his money on building asics so application specific

integrated circuits so these custom hardware devices for doing computational biology and drug discovery and physics research is this the same company as like the esa shaw investment group or is this a separate it's the same thing basically one of the branches of the investment group was working on this building asics for physics research so basically the story of david not to kind of recursively add some stories is in the 1980s he was an assistant professor i think he didn't get tenure and so you know if you think about people you know who are really smart but like they don't get tenure for some reason or another in his case it was he was really working on non-von neumann computers

and 1986 was the time when intel was kind of like about to hit moon at that time no one wants to hire someone who's doing kind of custom computer architecture everyone's like no intel's gonna win so we're gonna like hire people who are gonna make intel hit moore's law and so then he went and worked in finance somehow secretly hidden underneath him working in finance and trying to make money was this idea that he like still wanted to work on non-vulnerable computers so non-von neumann computers means that like a normal computer architecture kind of separates memory and compute potentially separates those two uh von neumann computer architecture is kind of are what you have in your computer or

your phone right now for the most part where uh compute and data are processed in the same sort of pipeline and so i think in the 80s people didn't know what would win like what hardware would be in your devices that you own he he was kind of always interested in that and one of the applications of these kind of very esoteric architectures was building super computers that were really good at solving physics problems and also solving sort of like computational biology problems at large and he kind of in the back of his mind he was always like hey if i become rich enough i'm just going to spend all my money on building these custom computer architectures for something useful for society

and so once he became a billionaire he started trying to prove theorems about whether it was possible to do better than these like intel style architectures and he proved this theorem in sort of 2004 which is kind of great because it uses only high school math to make this point that you can actually do significantly better like very much better in the sense that intel style processors if you try to parallelize this computation will always take a finite amount of time even if you had an infinite number of processors but there exists kind of crazy architectures that would take you know basically zero time as the number of processors goes to infinity so we actually once he he kind of proved

this theorem i guess you know once you're a billionaire you're like i'm gonna build hardware to prove that my theorem is correct so that's the genesis story i worked there and in 2011 there weren't many people building asics most of the people building asics were certainly not bitcoin miners um they were mainly telecom companies so one of the things that's interesting is that the only people who really needed custom hardware were people doing really low latency uh fast fourier transforms ffts and most of the people doing that were in telecom there was no neural nets there was no self-driving cars there was none of that type of stuff when we were talking to suppliers

because we're not someone who's like trying to be samsung or apple building a whole infrastructure on it we want to kind of like pay them for their excess supply so it's like we want to build 10 000 chips intel and apple and samsung are each building a billion ships but the factories that you that build that stuff they have excess supplies sometimes like the one factory might have an excess 10 000 and they're like oh well can we sign someone who wants to buy that and so the way that excess supply gets sold is the sword gets auctioned off to different market participants and at that time we were literally the only ones buying this type of space it was like us and like a random telecom company in japan

in 2011 and 12 one of the companies that sort of became avalon started buying a bunch of this chip space and so we were like talking to a supplier and like hey we sent you you know 25 million dollars when are our chips coming and then we got ghosted this is getting ghosted pretty tinder so that was weird and at some level we were like what the hell we just gave you a bunch of money you just disappeared like and then they came back eventually and we're like sorry we're gonna do your chips you know the next batch how's the 10 discount sound and that was the first time i'd really thought seriously about bitcoin after that i kind of started mining how did that go into mining was so you found out the people who like

took your order were basically bitcoin miners who were like a hardware fab let's say they have 10 000 spots like imagine a physical wafer now cut up the wafer into ten thousand like one centimeter by one centimeter little units let's say apple takes up nine thousand of those so there's one thousand left and what they do is they auction off the physical space imagine block space but physically auctioned off usually it's kind of a fair auction like you're like okay i put in a bid i wanted you know 8 000 of those spots for a hundred dollars each so we put in our bid and then we got told hey you won

and then the person disappeared and so once you win you put you have to put money in escrow so you post the money that you're supposed to post as collateral and then you're supposed to eventually get paid or eventually get your hardware and then they take your money they just kind of kept it in escrow for like six months way longer than it was supposed to be and we were like what the hell and so what happened was this bitcoin miner went to the supplier because the bitcoin miner was in taiwan and the supplier was in taiwan and they like just i guess they knew each other and they were like look i know you closed this auction but we need to make these miners buy x date and we will pay you anything it's kind of tangential to this conversation but people don't really

realize the strategic importance of chip manufacturers because there aren't that many i think you probably know this more a lot better than i do but like intel for example is a chip designer and a chip fab and in the last 20 years or so there's been a shift towards you know like more the chip fab model and like the designers and the fabs are now separated and so like companies like tsmc which is like this big chip fab in taiwan have kind of like won the market and apple and all these companies get their chips made there and the question here that's kind of really interesting in the current geopolitical context and this kind of example that you have kind of exemplifies this very thing is that you know these chip manufacturers are

on arms reach away from china and they're very far away from you know the us or europe and other western countries and they're strategic to you know critical military and industrial infrastructure in west the amount of chips that you know the u.s can produce on its own without these suppliers is like insanely small compared to just like what tsmc can put out or like samsung for example what your thoughts on that like knowing this ecosystem a lot better than i do i think the us is actually completely uncompetitive at this point there's global foundries which has this kind of big fab in upstate new york and long island i guess now they're kind of like split

into two and then there's like the intel's intel has their own fab so they have one in arizona and the uf government especially in the current kind of strong arm administration type of nonsense is trying to be like hey you have to like build your chips in the u.s if you want to sell them here or something that's not a very good point of leverage in the long run right because one of the more impressive things about moore's law is moore's law actually is a self-fulfilling prophecy gordon moore said this kind of apocryphal thing of like oh every 18 months your chip frequency is gonna double it turned into its own kind of war right so like every 18 months these companies would have like the chip designers benchmark

themselves on how close they were and then once they had a design that could achieve that sort of doubling rate then they would go through this entire process of finding suppliers who would like be able to do that and the suppliers also had to follow the like hey we need to double every 18 months kind of rule and you had this cycle of like chip supplier gives you design suppliers who are like oh man we need to source this really rare material to like make this happen like we're gonna spend all of our money trying to do that leads to successful moore's law thing leads to lots of chips sold leads to chip designer forcing supplier to do this and there's an ecosystem effect kind of

not unlike cars where like the car manufacturer isn't really the true end-all be-all manufacturer there's this whole network of suppliers who is necessary for it to kind of you to get the final product and the chip designers the intel's of the world plus the suppliers who are making the little sub components had to work together cooperatively for this like very long time period in order to achieve the current kind of status and the suppliers not just the fabs themselves are all in asia right like the entire supply chain is completely in asia there's literally nothing in the us i think it's a farce when the us is like we're going

to take back all this like it's a 30-year effort of building out multiple industries right like one of the things that's very important to getting to sub 10 nanometers chips is something called extreme uv so it's building these really crazy lasers are they the same uvs that prevent the coronavirus exactly it's like these really crazy lasers that are very like hard to build and intel has claimed that hey they've been working on it for 20 years of like we can build these like really crazy lasers the reason you need these lasers is that when you have a chip you have a piece of silicon and then you build what's called a mask so the mask covers a piece of silicon

and then you shine some type of electromagnetic radiation on there and it etches it like kind of etch-a-sketches out your circuit design but there's this whole industry of these like extreme uv lasers in order to get like the size of the little thing to be really small so you can pack more transistors on a trip you need to build these custom lasers the only place in the world that you can make the kind of crazy glass that you need for the laser is in sort of mongolia inner mongolia and china and there's just like little tidbits like that like oh well we need this type of glass for this type of thing or we need this type of silicon or we need this rare earth material those are all things that you need to

build if you want to like vertically integrate the chip stack and i think the geopolitical thing is like well asia has spent 20 to 30 years building the whole supply chain around this industry and you're not just going to like uproot the whole tree that's like saying that mycorrhizae or rhizome of like this industry is gonna like migrate in two seconds i just don't think that's possible i think maybe like tsmc has been due to open a fab in the u.s in like some time or there's some kind of thing like that but like the fab itself isn't able to produce like these you know 10 nanometer chips i mean i know very little about this but i just from what i know it seems like a very kind of interesting thing that most people don't realize the geopolitical impacts for sure that's like this olive branch

that was given to trump because he's like i want to have chip manufacturing in the u.s and it's like that's not happening by the way the same thing happened to boeing i know this is really off topic but part of the reason we had this whole 787 fiasco is that boeing tried to decentralize its supply chain and then like they stopped having control over like batteries and then batteries exploded whereas they used to make their own batteries before back in january we interviewed steve coquinos and sylvia mccally of al grand and during our conversation we talked about how algorithms unique design makes it easy for developers to build sophisticated applications on their platform so what's great about algorand beyond the fact that it's fast it's secure its

scales and it has instant finality is the fact that they've designed a layer one protocol with primitives that are purpose built for d5 so what that means is that they've taken some of the most common things that people do with smart contracts and they've embedded them right in the system right in the layer one so things like issuing tokens atomic transfers well these are built into the layer one and smart contracts are first class citizens on all grant so with these essential building blocks at your disposal you can build fast and secure define apps in no time to learn more about what algorand brings to the table and how to get started i would encourage you to check out algrand.com epicenter that lets them know that you heard about it from us and it takes you where you need to go to

learn about their tech and with that we'd like to thank al goran for supporting the podcast i think our audience would also like to hear about gauntlet and what you guys are doing so i guess i didn't even explain how hardware got into crypto bitcoin miners front run us i started mining and then in 2013 i sold all my bitcoin because i was like this is going to blow up this is going to be a ponzi scheme very dumb idea obviously in retrospect but i started really paying more attention to the papers um because we worked in distributed systems we were building this type of you know when you're building these asics we we built this data center to kind of run like millions of these machines so

we had to kind of think about this type of stuff i started really getting convinced that there was something novel here when the ghost paper came out so ghost is kind of this fork choice rule that was in the early versions of ethereum that kind of promised you that you could handle like faster block production times if you chose a different fork choice rule and ghost was one of the first papers that thoughtfully thought about the incentive design and also the networking and also this sort of like basic like architecture of like if i wrote this code how would i write it and that was when i was like wow there's something serious here it's not just like oh haha like a bunch of people on the internet

like made tried to you know usurp leslie lamport's paxos it was like oh there's actually some novel thing here so you know i think before that i was like very you know maybe bitcoin maximalist i think the ghost paper was one of the first papers that i was like oh wow there's like cool ideas that the bitcoiners are not paying attention to for sure it also made me realize like oh man the design space of this thing is like bigger than anything that humanity has ever had like you have to like just combine so many things to say a simple result like that's kind of insane right like you know in other fields you don't have to do as many things like that and then i worked in high frequency trading afterwards

and there we actually would do this type of thing where we would make models of you know our trading strategies and then we'd make models of other people's trading strategies and we'd have them run kind of think like alphago style where they would like play against each other and you try to optimize your strategy and that was around the time i think the algorand paper came out and i remember reading the algorithm paper being like this is amazing from a cryptography standpoint in the sense of like wow like you can actually generate these verifiable random functions i didn't study cryptography i had to go read the classical papers at the time because i didn't really know that existed but at the same time i was like this seems like a little bit like a derivative more than it seems kind of like proof of

work like a one-way function like burning energy is a true is like nature's only one-way function that we know like a perfectly one-way function whereas like in cryptography we try you try to like emulate that but it's never perfect and so it's kind of i was a little bit like surprised that there's this whole proof-of-stake thing but like people didn't really think about the financial aspects and then 2017 happened and then i kind of started being like hey maybe this is a real deal am i writing simulations for fun based on the type of things we were doing in finance and then in 2018 i kind of kept talking to a bunch of layer one protocols because i was curious if anyone was doing this financial modeling i quit trading

and then started consulting for later ones and then the big bad libra tried to like buy my consultancy and that was when i was like you know what i think there is enough room that there are a lot of people who probably need financial and actuarial modeling for this stuff and i met my co-founder along the way because he also was in trading for a while and then he actually worked on designing like incentives for drivers at uber so we were both like yeah you know like i think there's like a way to make this rigorous so we started initially focusing on proof of stake especially because i think that was the genesis of my kind of interest in really committing 100 of my time to this and then over time it became much more

clear that defy is really the place where there's crazy amount of financial incentive modeling from multiple agents that exists and there's just this open space of both research as well as like actually deploying into production and you know the 2010 to 12 shift from nai from like it's like half academics who are washed up from the 90s and half like people who are just making random stuff and like calling it like sentient but we don't know if it works was really this thing where like these kind of hooligans turned into like the people who were correct i really feel like that's starting to happen right now in crypto that's a very

long-winded explanation of how i got here and so gauntlet really is about taking these tools from from finance actuarial modeling agent-based simulation and tooling them towards the kind of new problems in incentive design and crypto could you give us an example of like so what are the sort of things that you would model like let's say i came to you with a new proof-of-stake a consensus protocol are you modeling like are you testing the safety and liveness of my consensus protocol is it at some higher layer than that like what specifically are you testing i think it starts in a bunch of different levels i think that's certainly the first level uh one

of the things i remember that tipped me off when i was first reading about proof's sake was this idea that there were many different synchrony assumptions in all of the different papers but they were quite inequivalent so some people would say you're live if you eventually were able to process the block some people would say you're live if greater than x percent of nodes agreed that a block had been produced and some people would say you're live if and this is sort of the way the avalanche paper kind of eventually got proved if like in the limit of an infinite number of blocks a non-zero fraction of them were actually reached by a large percentage of the nodes

now those all kind of sound equivalent but mathematically when you're trying to write these proofs they're not so the types of things i was really first interested in simulating were things like how long does it actually take on different network topologies for these blocks to actually have disseminated enough such that the network reaches consensus and one of the things i was very realized you could only kind of answer by simulation and would be very hard to prove is given a network topology what is the true partial synchrony constant and what i mean by that is like what's the constant at which if everyone receives all of the blocks within a certain time window how long does that time window have to be for the network to like

achieve liveness and safety and so simulating that on different network topologies actually convinced me that even bitcoin has a lot of problems if the network topology is like too disconnected and so mathematicians have sort of ways of defining what it means to be too disconnected for without getting into too much detail i think like the spectral gap of a graph is something that measures how long random walks take and so the idea is like if someone who's randomly walking on your network topology gets lost because they're too drunk then your block may never reach everyone and so you kind of like assume like hey i put a drunk person on the network graph and i see how long it takes them to reach everyone

that's kind of this this model of like time that mathematicians use and i was trying to map the model that people have formal proofs in that land to to like what cryptographers and distributed systems people were using and simulation was the tool for that so we start by assessing kind of some of these types of issues i think safety is not the type of thing we assess i think safety is a purely cryptographic property but liveness of proof-of-stake protocols is very much a sort of statistical property it depends on the network topology it depends on the latencies how random they are what the 95th percentile of the latencies look like etc i'll use example of tendermint because obviously that's what i'm most familiar with you know we also have this whole live

and partial synchrony how it works is we basically have these rounds and each round we say there's a timeout which nodes will wait currently on most tenant networks it's one second by default but then if we don't reach consensus in that one second we go to the next round and we increase it by the time out by a certain amount so i think we increase it by a quarter of a second every time so we do a one second timeout then if that round doesn't work we go to a 1.25 second timeout then we go to 1.5 second timeout and so these numbers for us we just pulled these numbers out of a hat you know we did a little bit of testing but if i was still at tendermint we'd go to you and basically say like

hey help us figure out the right numbers we should be putting here because if one second is too long then we're we're wasting time that we could be making faster blocks meanwhile if it's too short we're causing unnecessary rounds for no reason and so you would basically be able to help us parameterize that correctly exactly yeah it's like a bandwidth first latency trade-off of like how much communication do i have to do there's an expected number of rounds and the distribution of the number of rounds in the thing you're talking about imagine you have 100 million blocks that were produced and for each block i looked at the number of rounds it took before the network agreed and i look at that distribution now that distribution is a function of these parameters you chose but the

problem is that distribution also depends on the network topology it depends on some lower level details of the protocol and so yeah the type of thing we'd stress stress test is like how does that work under different models of users because you can have different types of users who affect that behavior one type of user might be the type of user that drops a lot of packets because their computer goes off a lot they don't care about getting slash because they don't even know they're getting slashed for being offline or something another type of user might be one that's malicious who's purposely forwarding bad packets another type of user might be one who is kind of a big block producer and like just is like trying to get not even just honest but hyper rational and that

they're just trying to like flood the network so that their block always is first one the different composition of users also affects this distribution of like the expected number of rounds it takes and that's kind of where we model when it comes to proof's sake but we also model over time we realized that we started with these networking models because that's what people in high frequency trading do a lot in high frequency trading you model like here's exchange one here's exchange two here's exchange three here's all the routers in exchange one to exchange two if i send a packet how long does it take and if you know you kind of model the topology in in sort of a similar way you would think about modeling validators and then you say hey if i have an adversary who's also thinking the same way as me

are they also sending the same number of packets and will it caught will who will reach first it's a similar type of home it sounds like you're doing analysis at like different layers of the stack you're doing the mechanism analysis of the systems themselves in order to looking for liveness and availability and things like that so this is like the mechanism design part and this might take place when the team is building the system but you're also doing research and analysis and simulations at a higher level up the stack so i know like you're also doing say research on market participants risks like in the compound protocol so this is happening at the economic layer at the market layer

is that right yeah so i like to think of when you do simulation i think one of the reasons people oftentimes think like hey this can never be real or it doesn't replicate reality or how do you know it replicates reality is that a lot of people try to simulate everything all at once and you really need to think of it like an onion where here is a particular problem that i'm trying to solve and here is the particular instance of it and here are kind of the bounds of like the worst case and best case and i'm going to try to analyze that in isolation and then i add the next layer of the onion and i have it interact with that layer and then i add the next layer of the onion and i have it interact with that layer

i think if you do it kind of in this incremental way you can actually try to reason about the whole complex system you know we start with things like this layer one liveness type of stuff but you slowly build up to the economic incentives so how much that complexity gets injected into that once you start thinking of things like interoperability between blockchains for instance the problem gets exponentially more difficult if you start factoring in multiple blockchains and interactions between all these different systems that doesn't fit in my brain space it is certainly exponentially bigger i mean the you're taking address space one address space two you've doubled the number of bits definitely increasing in an exponential manner but the key is

to try to like isolate the points of complexity that are most tangible to think about how humans would interact with these systems because fundamentally okay i went from 128 bits of entropy to 256 bits of entropy for a two-chain interactive system but humans are still using this right and like methods and interfaces that you provide to the human as a developer also dictate what usage you're going to get and so you try to model things that replicate what humans who are using those interfaces would look like and then you kind of say okay let's say i have 10 different versions of the same human how would they use the system

100 different versions of the same human how would they use a system and you kind of build from sort of a bottoms up approach where you try to like identify behaviors figure out which of those behaviors are consistent among a group of people then figure out what math describes them like what their utility function is what value they're getting out of calling this function across blockchain transaction what value they're getting out of hey i'm willing to pay a transaction fee that's higher than the one on my chain to move across chain then after that what decisions do they make like given this sort of notion of how they can value a cross block chain transaction what actions can they take one action is

certainly make cross-blockchain transaction another one is don't another one is is there a way for me to do it on my current blockchain that gives me 80 of the same value or 70 or 50 breaking it down in kind of this hey there's still a human using this thing or there's a human writing a bot that's using this thing there's still this notion of like people's ux habits are not uniformly random right they're not just like a fuzzer they're really kind of like using these interfaces in a very concrete way and reasoning about how different people would use it is really how you mod kind of try to start modeling these types of things

in consensus protocols we usually like think of it like okay the three types of actors we have are like byzantine rational and altruistic but so what you're essentially implying is that this is like way too simplistic and that we need to be much more specific it's not just these three categories it's way more of a spectrum or many more types of users or actors so how do you know you've modeled all the actors possible or like how do you know you've covered the entire space and how do you like deal with like things that were just you couldn't predict like imagine you wanted to try to predict like the distribution of snx and how much what it would be collateralizing but

like you know in what world could you have predicted that like a million snx would be sitting here farming yams craziness how can you possibly build all of these into your models for sure so i think one thing to remember from consensus protocols is this bar model as byzantine altruistic rational is very uh unfair in one way which is that byzantine and altruistic are like one-dimensional things so in the space of all possible strategies if i represent a strategy that a user takes given an interface so an interface let's just say it's a set of functions that you can call and let's assume that all users have valuation model

and based on the valuation evaluation model means a utility function and traditional economics or kind of an objective function in machine learning you have objective function and you have decision function so the objective function gives you a single number or some set of numbers the decision function takes those numbers and gives you an action like what the user does and there's an entire you know not to tutorials but you know if you read philosophy there's a whole argument of whether this is how humans act or not but ignoring the kantian type of stuff that's how almost all models uh in machine learning for like alphago and stuff think about the world right there's this kind of value function decision function now byzantine the value function is choose a

random number choose an action altruistic the random number it's like choose the same number and choose this particular behavior always rational is much more like i'm actually observing the environment and and like trying to figure out evaluation and the value of these actions and changing over time fundamentally byzantine and altruistic are actually very one-dimensional in that way one is purely random and one is purely deterministic but rational is actually this adaptive type of adversary and it's an infinite dimensional space of functions like the set of functions that can give you utilities and values is infinite dimensional for rational but the other ones are actually zero

dimensional like one one dimensional like you know there's like a single function that everyone knows in advance and so the problem is by saying byzantine altruistic rational you're kind of assuming hey there are three equal categories but that's not true because the rational category from a mathematical perspective is significantly larger like infinitely larger and not just countably infinitely larger it's uncountably infinitely larger i think there's a lot of kind of classical functional analysis theorems that prove this i'd say nash won his nobel prize was kind of related to proving this but the point is that like you can't actually you know when you when you say hey i've analyzed this protocol under

byzantine altruistic rational 99.99999 percent of people who've done that are like hey we're just going to say rational is optimizing quasi-linear utility uh like keys keith or hey rational is like they they are like only caring about a certain type of thing and the problem is no matter how you reason about any of these systems you are fundamentally imbuing a notion of what you think rational is and you can never perfectly simulate these such things i will be the first person and the last person to tell you that but i do think you should be foliating the set of rational actors with a much broader

set of views than what you can do with formal proofs i think in formal proofs the problem is really like it's very hard to prescribe a model that's that you know kind of can cover this infinite dimensional space and i think cryptographers have this willingness to suspend disbelief of like hey we're just going to pretend that the rational actor only does one type of rational action but that's just not true right and game theory and algorithmic game theory and stuff have tons of examples of this happening in practice so i think the better better answer and you know i think this is what the biggest algorithm that game theory systems that are in production like google ads and facebook do

is they they do tons of numerical stress testing of different types of users trying to commit fraud different types of users trying to do xyz type of action and then you run these simulations and say like ah okay this parameter for our auction is correct or like that's what we're going to use tomorrow and you keep updating it as you get new data so the yam thing happens okay snx whales are suddenly into into vegetables like didn't predict that but now that we add it to our little repository that's a new type of rational agent and so then the next time so the day after the yams happen we can say hey look here is this thing that actually causes this crazy amount of risk to your system

because people who have smx they're already leveraged they print printed some susd and then they took their susd bought more snx and put in yams now your system even though you say it's a 700 collateralization ratio it's actually much lower because people have been doing this recycling and kind of like sort of weird weird sort of uh financial engineering that they they might not even realize they're doing and now we have a strategy that replicates that so now when someone else wants let's say yams add atoms i don't know let's pretend there's like a synthetic atom on each such that you can deposit ndm we can run the same strategy and say like this is the amount of risk the atom

holders are taking and so my point is it's an incremental thing where you're not going to predict everything but you're going to try to make your best guess by building the biggest library of possible things and then stress testing against it it's a lot like security auditing where you say either there exists formal verification as this dream that will predict everything or here are the set of things i know that could happen and i'm going to try to carefully look through each line of bytecode and say like this might happen this might not happen it's much closer to that if that makes sense and so how does like historical data play a role into this do you like when you have the simulation do you like run it against historical

data and like modify your simulation models until they fit the historical data and then you start to use them to predict or yeah how does that work yeah not another kind of philosophical dichotomy that exists in the traditional world is the difference between the financial world and the actuarial world so in the financial world people really care about what are called point estimates so point estimates are what your neural net does they give you an answer they say like hey here's a function here a bunch of examples train it on those examples so that it gets the right answer and then in the future take a new example and give me a guess

of like what the output is it doesn't give you any estimate of what the uncertainty is it doesn't give you any estimate of like what how wrong can this function that's mapping things be neural nets don't give you that but like sort of more traditional statistical methods do that and so in finance people care about point estimates because they're like i want to maximize my expected reward or economics microeconomics in general in actuarial studies people care about like hey i have this life table for this insurance i'm underwriting and i care about the variance of how much i have to pay or like hey like yeah sure on average i only have to collect a hundred dollars in premiums from each person

but like there's this one dude who has asbestos poisoning and he it's going to cost us a billion dollars to cover his health insurance i'm making something egregious but so in insurance and and actuarial studies you care about this kind of like distributional effect whereas in finance you care about like the eve and and you know i would say machine learning in general you care about kind of like predicting the average although in finance you care about the tail events blowing you out but there's kind of this dichotomy and so one way we do this is we fit some of the rational actors behaviors based on historical data so we tried to take the historical data and say hey these actions were done by this address

repeatedly you know let's say you tagged these addresses and we're like this is a you know a dextrader that does this type of action so can we try to infer their utility function in further utility function now that's one of the libraries of the one type of user who's fit to circle data another type of user is one where we leave we say hey this user we're going to parametrize in this way we're going to say hey they have a value function but we're not going to say it's precisely these numbers and then we sample all the numbers we're like hey there that we have a parameter that says how risky they are and when it's one they're a complete gambler they just pull the slot machine every time and when it's zero it's they're very risk-averse

and then we have another parameter that says hey how much do they value high growth versus how much do they value kind of like safe growth so like they they're like oh like i'm willing to invest in the s p 500 versus like i'm willing to put all my money into nicola or i don't know i don't know what the hot like now thanks to robin hood i don't even know what the like hot stock thing like you know port noise stock thing is anymore but the idea is you tried to say hey here's how we parametrize how this agent thinks about risk and then we search through the whole parameter space so we say we're gonna grid search from zero to one

on their risk level and then show kind of these heat maps or like these plots or these kind of more descriptive statistics about how at each parameter how the system behaves um so you kind of have to do both uh that's maybe a long-winded answer to to that where you want some historical types of users but you also want to try to make sure you parameterize a space in a flexible enough way that you can search it i want to ask you about transaction fees and if you're doing any research there and how important that is in this mechanism design space yeah so i think in traditional mechanism design it's it's not quite it's not quite well studied and i think

you know we're you know one one of our advisors is tim rufgarden and we're were constantly educating him a lot about the this type of stuff and he's really been like hey yeah like we just didn't really you know we spent the last 20 years building auctions for google because that mechanism designers and algorithmic game theory folks sorry when you say when you say in traditional mechanism design you mean transaction fees applied to other mechanisms than blockchain like i mean what other in other in what other places do we see like transaction fees as part of mechanism design for sure yeah so uh the biggest i would say practical user of mechanism design that exists in the world

is online adoptions okay well sorry what i meant by mechanism design was specifically like sorry i i was talking about like crypto cryptocurrency mechanisms like basically like this cryptocurrency design yeah what i i guess what i mean is a lot of the math that has been invented for traditional mechanism design doesn't include transaction fees in the way that blockchains use transaction fees and what i mean by that is when i'm say buying an ad or i'm connecting to a futures exchange i don't pay per message i send to the exchange right but in crypto i have to actually pay per message that i send um and so that actually changes the dynamics of of a lot of the a lot of the math and a lot of the math

that works for ad auctions is completely invalid for blockchain that makes sense because of this it's like a new research space essentially is what you're saying yeah it's a 100 new research because people don't think about this pay-per-message aspect of it it's assumed that like any user can send as many messages as they want to google or facebook and that they don't have to pay like they're kind of paying in like there's some ddos prevention but there's not like a like hey you you actually have to to pay for spam prevention and so what we do is we spend a lot of time modeling this we don't model it say in the way that we could probably prove a theorem about it but we do try to say how should i value if i'm a minor

and i get i have a mempool how should i value a certain permutation right because a mempool is a set of unordered set of transactions and the the kind of notion of a i chose some subset of it and b i chose an ordering of that subset that's the value that's extracted by the minor right um and so we we tried to take kind of the more machine learning ish more statistical approach to it which traditional mechanism designers would say oh well like how do you know it's optimal we just try to say like hey well any local optimum is good enough which is sort of the machine learning approach of like what permutation what subset can you pick that will

maximize your value and then what ordering will maximize your value so we measure that both in terms of trying to predict distributions of delays so submit transaction how long is the delay given the fee and then we also tried to say what permutation is like most likely so we but the problem is prescribing value functions over permutations is very difficult because a very large you know it's n factorial space uh so you have to kind of like come up some heuristics for that but roughly speaking that's what you do the good news is that everyone who's writing front running bots is still a human and so like they they write a certain set of strategies right like it's not

like it's not like there's like they're really looking at the strategy that's like compute the ackermann function divide it by the maximum value that it could have been and then use that as a random number to flip a coin to decide on the ordering right they're not going to choose some crazy thing whose complexity is like super factorial or something right so so so far you know we've been discussing this in the context of like simulating an existing designed game do you guys also work on designing new games all together so in hft for example you could simulate the hft

or you could solve some of the problems by like inventing batch execution so when it comes to like you know for example on ethereum there's like crazy gas spikes that we've had for the past couple of weeks you know we could continue to simulate this game but it's probably not sustainable like there's probably a good chance that the game design itself is broken and we need to rethink how we do block space auctions in the first place and so would you be able to use similar methodologies to construct new games or is that or is the construction of new game sort of something that has to be just intuitive and then this stuff is only used to test them out

i think it's sort of there's a feedback loop right of like i have an idea i run a bunch of simulations i see if it works and then i see what doesn't work and then i mutate my idea until like i get to some type of minimum or like optimum solution i think a lot of the problem with things like designing box-based auctions is like they there's a really well-established theory that's very attractive to people to use which is the theory of adoption so a lot of the papers on that i would say that especially by crypto professors are just like cribbing algorithmic game theory results and saying like hey they apply here

but i think that like a lot you there is certainly some theoretical innovation you have to make first i i think you do have to write the correct mathematical framework and equations before you can really simulate but i do think simulation tells you when you're wrong it doesn't tell you you're right but it definitely tells you when you're wrong so it's kind of like a property test like you know you say this model should do x kind of like in formal verification except to add a statistical level you say on average this type of block space thing should do x and you use simulation to verify and then you find hey it doesn't work so like i must have made the wrong model so now i have to change something and

when i say model here i mean first price auction second price auction weird like auction mechanic for block space that you choose right like you somehow have to kind of you know you can think of you should really think of simulation as a way of doing this property testing and verification so what piece of the cryptoeconomic field or stack do you think would most benefit right now like today from some of this simulation work so it would it be like the proof of stake protocols is it the fee models is it some of these on-chain device stuff like lending dexes

i used to think it was proof of stake itself um i think i think the problem for proof of sake from a more practical standpoint is that people are just more risk-averse which is good you should be very risk-averse for your base layer but that also means you're like way too slow to like try to like update like you know simulation should be used in like we did something we observe something we try to predict what will happen given the new observations and then we update and you kind of have this feedback loop repeatedly applied that's when it works best so like that's what happens in trading that's what happens in chip design when when simulation twos and other places but i think proof of stake is like very

very slow like i and and like d5 is basically copying proof of stake except it's replacing proof of stake as with an insurance fund type of thing and i think yeah the d5 parameters are really the biggest deal right now for sure because like people are doing all the stuff that they said they would do in proof of stake except they're doing it like recklessly so i think in the long run proof of stake will learn a lot of the lessons of failure from these d5 things but yeah right now it's just so much more you know you can like make a prediction someone does it see how it works use that as an example to add to your simulation and and like that's just happening in all in d5 right

now i just don't think it's really happening in proof of stake how much does like sort of governance actually impact a lot of this stuff so so when you do these simulations or like this mechanism design you have like some model of what like some socially optimum utility is for the entire system and if i was like a benevolent dictator and i wanted to maximize like the social optimum for this thing you know you could figure that out tell me what what the best mechanism is and i can go deploy it but now what happens when you know there's a governance token and so sometimes the holders of the governance token are not trying to maximize socially optimum

they're trying to maximize you know they themselves are rational agents so it seems like it becomes this like very weird meta thing thing that you have to also account for you have to model first you have to model the game that's the the mechanism so let's say you know it's curve but now you also have to maxim model like the incentives of the curve governance yeah i think the key is to inject simulation into the decision-making process so like when someone is proposing a vote you run a bunch of simulations and you say like here are the set of outcomes given these types of utility functions put yourself in one of them and if you don't find yourself in one of them then

you can complain but assume you have these sets of value functions we've run these simulations under different edge cases and here are the properties that hold and here's the probability they hold with like if everyone's a degen gambler the probability of the system going to zero went from point is went from previously before this vote one basis point to five percent like okay that's something right like and i can give you an uncertainty estimate so one of the things that i was saying before about like point estimates finance machine learning versus uncertainty estimates actuaries insurance statisticians is that if you can provide good uncertainty estimates if i tell you it's

hey i'm increasing from one basis point of a chance of s and x going to zero to five percent of a chance of s and x going to zero but five percent plus or minus point two percent then you actually have a lot more con like if i can give you more and more confidence that like this is an increasing thing then governance actually you can impact governance in a way that's quantitative and not like this emotional view of the world because like at the end of the day it's a new field people don't really like their voting kind of blindly and at least giving some sort of uncertainty estimate lets people be like okay well i'm

rational but i'm not am i rational and iterated game right like if i'm taking a single step it's rational for me to say increase the fees 99 because i'm an snx holder and i want all those fees but this kind of iterated simulation says if the game lasts long enough we might go to zero if if i try to collect too many fees and if that happens is that really worth it and so it gives people a way to figure out their own valuation of how much they want to risk adjust so i think simulation's not going to be able to predict perfectly these governance actions but it's going to show you the outcomes under what happens when you choose them and so

it's an integral part of giving quantitative justification for these things and in the normal world it's much harder actually to impact governance in a quantitative way whereas in crypto it actually feels like it's quite tangible that you're like giving people risk assessments based on a lot of like a lot of very clear financial data in a lot of stuff you do in the real world you have to infer whether the data is real whether it's accurate sometimes you're like well someone may have been like kind of injecting noise into the data but the on-chain data being something you trust in is valid is actually quite important for us so i'd like to come back to an earlier point which is let's imagine

that someone is like building a new blockchain as as that happens these days right and um you know a lot a lot of times i think teams are kind of focused on on like building the product growing the community and then of course one of the things that often comes up at some point is doing a security audit and a security audit will entail like a bunch of things but there's some like design aspects that are also part of that audit i'm curious how you consider your work to be complimentary to that or you should like all that replace it is it better or is it a little bit somewhere in between like where you put yourself in that sort of

early stage research when one is building a blockchain i think it's it's pretty complementary to to both normal audits and formal verification because i think one of the problems for formal verification is related to the thing you're talking about which is that naively there's an exponential state space blow up once i start interacting to a system like k systems once k systems interact you have this the naive notion of the number of bits you need is blowing linearly in k so the space is kind of blown up exponentially but simulation is more about like well what's the behavior that's not if i don't have to sample every possible action

if i sample the most likely actions as well as the ones that are near the most likely actions what happens and so there are two different types of scenarios right one is the pure worst case but might take infinitely long to search through the set of tests and the other is how do i kind of use the expected behavior to estimate risk in a way that is intuitive and interpretable to the non-developer and both of those i think are are valid ways of stress testing but they're very different and they are extremely complimentary so like when in exchange when like the cme the chicago mercantile exchange which is like the biggest futures market in the world

when they build a new piece of software they of course get audited and they do kind of traditional cyber security ads but they also do simulation and they stress test like hey did we choose the right tick size as a parameter did we like are we resistant to kind of certain types of malicious trading strategies that try to like block everyone out of the market and there's a whole literature of this like the sec themselves spends a bunch of time doing these stress tests on exchange code to show that they meet compliance and so they're very complementary but they they stress as very different things one is really stress testing user behavior and the other is stress testing like

code behavior and user behavior is about probabilities code behavior is about determinism pure determinism um but they're they're related so in all of our simulation research and i think this is what distinguishes us from other people that who've kind of tried to do this is we run everything against the real code so we build kind of think of open ai gym or like you know the alphago training program what happens is people build a harness around the real piece of code and then the harness has a way has a set of interfaces that you can model the different types of

users and you make a domain specific language that you can program the different types of users and then the users interact with the real code and i think i've seen a lot of simulation especially in 2017 i saw a lot of kind of less rigorous simulation stuff that kind of was like hey well we think the model of how the blockchain itself works is this and we're going to say this is a poisson process and this is this thing and this is this thing and then we have models of user interact with them the problem is in reality like a lot of these code things that cause problems for formal verification or security outdoors also will affect the economics in super edge cases so you want to minimize the amount of surface area

that you seed to your model you want to say hey look we're running this against the real code as much as possible and this is something people in trading do a lot and i think that i only really respected this once i saw the difference in trading between hey like this exchange happens to use only 18-bit fixed point integers and like all of a sudden the strategy loses money right like that's that's the type of detail that you know i think a lot of people who are like oh well i just like learn python and use pytorch and like i made a model of your blockchain don't kind of like are missing like they've never seen like people lose money because like

the you went from floating point to 18 bit fixed point integer randomly and i think that that's why you need to actually run this stuff against the real code because you you there's just like tons of weird developer decisions some random if statement somewhere that completely like takes all the money out and you don't realize why until you like actually are running like tons of so i it's complimentary though i i just don't see i think security authors are really focused on like binary objective functions of like does it property testing of like yes no and you know i think what we're focused on is this kind of like statistical version of that of like but we still want to run against the real code right

we still you know compile against whatever docker image you give us we run it against whatever kernel modules you say it should be running against um because i think you never know when like some random piece of the code just changes the economics completely what kind of tools do you use to do this like how do you are these like custom in-house built tools or are there like some you know public tools that you kind of use to do these sort of simulations both you know with dummy code and then also when you want to test with the real code yeah so similar to kind of security auditors uh we kind of have we kind of build a lot

of our own versions of the virtual machines themselves and like add in extra tracing functionality and extra kind of like tracking functionality for like agent submits uh a trade to uniswap and we track kind of like where through the client that transaction goes and like oh did it get halted at a certain point or oh did the networking layer like look malformed at it but we spend a lot of time i mean certainly now we're pretty much only ethereum we were doing a lot more other chains but honestly it define ethereum has the most sort of need for this but so we we kind of have

written our own we kind of have our own fork of geth where we have optimized a bunch of things for doing simulation one thing to remember when you're doing simulation is you're controlling the threat model you're describing all of the users in the system so by controlling a threat model you can actually reduce a lot of the cryptography burden and by doing that you can make the performance a lot better and so we've we spent a lot of time building this client with extra tracing and kind of ways of like having multiple agents interact with the same node multiple agents kind of work off the same kind of simulated blockchain state stuff like that

and so yeah so we we a have that and then b we have sort of a domain specific language that we is mainly in python because i think from a data science perspective it's just like still too hard as much as i love julia i'm sorry julia fans it's just still not quite quite there but it's always one year away right but uh i'm sorry julia has this scientific programming language that's like way faster than python it's like this like compiles to rust in c pulse plus like supposed to be like the the real deal but yeah if you talk to every data scientist in the world they're gonna tell you they use python or r or something so we have these python bindings we have

this dsl dsl compiles to some byte code that uh basically gets run against the virtual machine directly so there's like kind of a layer in between that takes the compiled agent code and interact how does it interact with the virtual machine i think in a world you know kind of in the same way that it took trail of bits forever to to open source a lot of critic i think we want open source some of it but it's just going to take us a while but yeah right right now it's not only that type of stuff a lot of what we use is based on a lot of the work that google and and facebook have done on uh compiling python models to

cpels plus that that type of stuff has been is like really deep in our in our stack for increasing performance so before uh we wrap up here i'd like to ask you a little bit about gauntlet the business and what does the current business look like i mean you guys put out all these reports and all this research and um but who do you work for and then also you know what's the sort of roadmap and plans looking forward yeah so you know i think a lot of what we we do right now is uh you know putting out reports working with the protocols themselves kind of close to security auditors although we've been taking a lot more of an active role in governance

so we're sort of the the third largest comp holder in governance by votes and we have a a bunch of stuff we're working on right now to try to automate the actuarial predictions i was telling you about earlier so imagine that there's a governance vote someone says hey we want to change the collateral factor on compound for wbtc to this value we will basically auto-generate a bunch of simulations and risk estimates for like what this what the before and after of this particular vote look like to our best estimates and then present

them to the user in a way that's intuitive so you can be like oh well you know by making this change we decrease the probability of default by this amount but then we also lower the revenue that the the cash cash flow that the network gets by this amount and then you know a user who's like maybe more financially educated but not so like in the weeds on like how the protocol works can be like oh okay i kind of get that this is what this change does and we're also working on sort of what we call autogov which is a way where we monitor the markets and then auto submit proposals so we do we have you know sort of some of the proposals are more

simple but some of them need a little bit of like program synthesis where we we generate the code for the proposal based we run a bunch of simulations we say hey there's way too much risk in because a bunch of yield farmers decided to like mint too much at susd and we're gonna submit a proposal that says like increase susd minting fee by x and here's the reasons why and here's the code for doing it and so the dream is to have the sort of automated system that can monitor these things and submit proposals to governance in a fully automated fashion and then the smart contracts pay for this

service i don't know if that it's it's it's a little bit of the opposite business model of most blockchains most blockchains and smart contracts want to make their coin worth a lot whereas we want to kind of reduce the tragedy of the commons and have be kind of paid as a service provider but it's automated and how do you get the time to like spit out all these papers you know one of my you know i just remember i had this idea like a couple of months ago of like oh you can combine ideas from stellar and avalanche to create this new consensus protocol and then like you like you're like oh me and my brother we wrote this up like two months ago for fun and i'm like what like where do you get all the time to like write all these

papers and like is that part of the work you do with you know for example you wrote this paper on like uniswap or just like in or amms in general is that also work you're doing with these companies or is that sort of just something you do on the side uh that is something we i i do on the side but i think it's very closely related in the sense that you know how you were talking earlier about like can you discover new mechanisms by pure sort of simulation methods i i think that there's kind of this interplay between the theory and actual discovery of these things so you actually need to make the theory so

that you can simulate it right like once you have the theory you can start saying like here's where the theory breaks and that's where we're going to simulate and that's where we're going to do kind of these stress testing type of things and i feel like right now the way that things look especially in defy it feels a lot like the kind of late 2000s early 2010s in machine learning it feels a lot like quantitative finance in the 1980s where if you can figure out how to make the valuation model that people use then you will actually impact the usage

of these systems and so that is related in that like yes we use the same models in simulation but also we have more people using these things because like they understand these financial aspects so there's kind of this dual play between like doing research and convincing people that these risk metrics are correct and i think i think writing the research is is quite crucial to that it's it's the equivalent of open source software for this type of stuff cool so where should people go to find out more about gauntlet and your work so our twitter is at gauntlet network for me my twitter is at my name chitra so t-a-r-u-n-c-h-i-t-r-a

you know we're we're we we publish a lot of stuff but i think we're gonna be coming to a governance vote near you soon so cool if you're if you're in that realm you will you will see us or you've already seen us in compound but i think that's the uh that's the story great thanks for coming on tyrone yep thanks it doesn't end here there's more to this conversation and you can hear it on epicenter premium as a premium subscriber you'll get access to a private rss feed where you can hear the interview debrief and get enhanced features like full episode transcripts and chapters which allow you to easily skip to specific sections of the interview you'll also get exclusive access to

roundtable conversations with epicenter hosts and bonus content we put out from time to time go to premium.epicenter.tv to become a subscriber and support the [Music] podcast