Episode 228 - Catch-up at DevConnect AMS with Tarun, Guillermo and Brendan

Transcript

[Music] welcome to xero knowledge i'm your host anna rose in this podcast we will be exploring the latest in zero knowledge research and the decentralized web as well as new paradigms that promise to change the way we interact and transact online for this week's episode we have something a little out of the ordinary recorded near the end of the amsterdam dev connect week guillermo tyrone brandon and i jumped on the podcast for a spontaneous and pretty chaotic conversation we do not stick to the script and manage

to explore how blockchain and ai thinking differs where it might intersect how science could benefit from more adversarial testing why peer review is broken math theory stuff and lots of random odds and ends we may or may not have been drinking while recording this episode so you'll have to listen to find out but before we kick off i just want to highlight the zk jobs board which is a spot where you could find a new gig working directly with ck tech and i specifically want to highlight one job post which would have you working with my team over at the zk validator we are right now looking for junior researcher who is interested in exploring and writing about zk proof of stake general meta blockchain topics and ways these intersect with privacy

ideally you'd have some technical background because we do want to go deep on these topics so if this sounds like something you would be interested in check out the job ad and get in touch now i'll let tonya the podcast producer share a little bit about this week's sponsor today's episode is sponsored by zcash community grants zcash community grants is a grants program within the zcash ecosystem that funds projects that advance the usability security privacy and adoption of zcash their primary areas of focus include wallets corn security interoperability zcash apps ongoing services education ecosystem and community with that said they also welcome novel ideas outside of these areas of focus zcash community grants are huge about inclusivity and the grants committee will also consider

grants of all sizes from individual hobbyists to large organizations that build on or contribute to increased integration of zcash previously awarded grants have ranged in size and scope from a three thousand dollar grant all the way to a 1.2 million dollar grant to apply for a zcash community grant and contribute towards the development of the zcash protocol visit the zcash grants hub at zcashgrants.org that's zcashgrants.org you can find the link in the show notes so thank you again zcash community grants now here's the devconnect chat with anna taroon guillermo and brendan [Music] we're here jorina should we still call it blockchain weeks i don't know what they are whatever weeks full of

conferences in various cities currently the city that we're in is amsterdam uh i think i i'm actually on day seven or eight oh nice and it is a marathon not a sprint you went to the cosmos thing i put on the cosmos thing on friday and uh i feel like these weeks are incredibly fun you meet a lot of people it's great but it's also like there's this battle for sleep and not losing your voice and trying it's like you're trying to get around a city you're not familiar with so there's like a lot of logistical thinking and yet you're zoned out and it's really hard so i do feel like every time i go on one of these adventures it is an adventure and it is hard actually there's like a it's it's you know there's a journey that we're

going through so i do wonder guillermo how at what point of the journey are you right now because you're long you're six days yeah yeah well six days is like kind of a weird lie right because uh i feel like it's the the flight here and with the delay i feel like i don't know if time has gone faster or slower because of like my brain being turned into a mush by the time zone change that's the other thing that it gives you is the jet lag that's right add a little dollop of jetlag and much too much alcohol what do you have this bunch uh yeah just to say at least that's one way of putting it what about you uh where are you at on your journey i've been to like too many of these lately but one thing i will say that i i

think is um maybe uh an observation that seems positive for the ethereum ecosystem is i say this in just but it does seem to be quite true which is the more hackathons that happen the more that actually any protocol development gets done it feels like people need to have hackathons to actually and like there's like almost one every two weeks now so they really want to get the merge done this is this is quite spicy yeah that's true though i mean i feel like it's 100 true yeah they get everyone in the same place they're not like a company and they get them building with the tools that are there fair enough i mean that's what was really missing i think over the last few years well i think that's the difference between like vc chains and organic chains right it's like they have to keep having conferences

whereas like you know more corporate like blockchains end up not having to do that because they have like okrs and stuff right like it's like like a normal company so middle management for hackathons are a form of middleware that's right that's right that's right what about you what at what stage of the journey are you uh so i had a bad week because i procrastinated uh preparing a talk that i had to do on the first day that i was here for the second day and so i like stayed up really late and so i i just never had a shot no it was getting more for you from the beginning sorry yeah so yeah i mean it's been good but but also bad uh okay so i think

what i what i think we can start to explore is like what what have you been up to this week like what are the things that have happened we can start easy what are the events that you went to you kind of remember those yeah let's start with like what are the events maybe some of the folks and then let's go into some of the ideas oh no so what are the events that you've been to touring you want to want to go spoken at or enjoyed this i mean there's making a face approximately like 70 events and at each event he's given like three times because you don't like the question or you can't remember uh oh you know uh i i think um the events have been interesting i

didn't really actually go to the main event like the actual hackathon but uh i think the campus off whatever i don't know how to do this uh we're pretty interesting there's definitely a lot of people at them you know i i don't think i heard too many new ideas to be honest i feel like it was a lot more you know i think a lot of stuff that was like coherent it you know i think if you're a new person in the space it was probably really educational you would have learned a lot i think any v day was actually probably the best organized event i've gone to perhaps ever in crypto i thought it was like really well put like professionally put on that's cause you didn't go to seek a summit yeah uh yeah

i mean i knew i was i was walking into this one with that that's why i was like oh you're like oh do you what do you want to talk about oh talk about every event other than the one related to those podcasts that we can lend base to you that's right that's how it is i think that's actually exactly what it was so taron didn't make it to the zk summit but i think the the reason being is that he's another event on the same day that asked to room to speak like three times which meant he could not leave that space oh right and i my bad was i should have been way more communicative with you about the date like i i think you knew it was happening but i wasn't like this is the date you have to be here and so i'm sorry actually because i

really wanted you to be there it's all right um everything is chilling i think that like i said i kind of feel like i didn't get and maybe maybe dk summit was the one place there was lots of new stuff but i do feel like the some of the other conferences i feel like were more about bringing in new developers than it was about say like some new kind of crazy idea the starkware thing today i think actually i heard some cool projects coming out of that so i'm kind of interested to hear go like watch some of the talks from that um i think it's a hackathon isn't it it's hackathon plus people are like kind of giving talks on stuff so and it it's like um i feel like the interesting thing and i think you kind

of pointed this out too and i was like we went from maybe like zk events being mainly like hey here's a paper for like a new thing that i've worked on and like hopefully we find investors and or engineers who want to build it to like hey actually we're building this thing on top of something uh so you know that change i think i'm not sure exactly what precipitated it you know things things i've seen that i thought were interesting or like people who made tools for using a lot of kind of not so fun zk software like ides for circom which you know a couple years ago people would just be like well why aren't you hand wiring circuits i think i literally remember once like

talking to howard and he was like howard from familia like just like 2018 or 19. he's like yeah you know like i i don't i don't know why i don't know why people complain so much like everyone could just like write the circuits manually like you need tools to do that just write the polynomial constraints by hand it's fine i think it's like interesting to see that hey people are like building dev tools for zk stuff like that actually wasn't i don't think people gave a [ __ ] as much about that before okay what about you uh so i went to the layer 2 like the l2 beat uh conference and uh zk summit which i thought was the best that was probably the best produced event that i've been he has to say that no no i i think he was paid to say that

independent i was not paid to say that and actually true i'm actually on this episode is to the it's say that but uh but that was super cool because the venue was excellent the food was excellent and i thought the talks were really good i liked uh producers talk on delegated um proving and my teammate jacob's talk was awesome oh you didn't come to my talk i'm just disappointed in you so i think it conflicted with something actually no it's good you didn't come to my talk actually it's probably for the better for your health generally speaking so so guillermo guillermos i'm sure was excellent uh it it either conflicted with uh with a polygon talk or with the time that i spent on an adirondack chair like sleeping in the car

oh i see so you were sleeping in my talk i get it what struck out to me from layer two and this might be kind of controversial was how i think expectations were being dialed back a little bit in terms of road maps and i i think like you know unnecessary dose of realism was was like refreshing to see so are you talking about the l2 beat that yeah yeah and you mean like everyone had been saying we are we are about to enter this like full roll up world and it's actually because i didn't i wasn't there so maybe you can yeah yeah so i think at least um some of the zk teams were sort of candid about things taking a little bit longer than

they'd anticipated and yeah i mean nothing obviously against those teams but i think it was good to like be realistic about when things are coming out guillermo oh um yeah what else was i i guess i was mostly just at zk summit and the celestia event which i actually don't even know the official title modular summit modular summit there we go that's the name which i think i found out i was speaking to that it's speaking at that one or in some panel or something because of a tweet thread like that i was like that's right yeah and i had no idea i was speaking and i you know afterwards i was told like oh yeah i was on this telegram group with like 150 messages i'm like i'm sorry guys i didn't mean it

i promise um and then afterwards yeah so those are the two so those were on the same day they were on the same day two on the same day that's right the reason jaroon couldn't come is because they were on the same day i you know i really struck it out i actually went into both and heroically showed up approximately like 20 minutes late to the other one and ran on stage and everyone i didn't know that yeah i ran off honestly though it could have been a little more dramatic i know i i i was waiting for the background music yeah you were kind of like not rockying it i know you know what did you do he was supposed to rocky and he just kind of like i just kind of like hobbled off he like jogged like a 50 year old man

in the neighborhood okay in a neighborhood in the suburbs yeah wow you could have done like some crowd work like oh i could have gotten everyone yeah you should have done that i should have gone around clapping been like yeah you know i i was i was worried well i was first of all hoping there was gonna be some background music while i was hopping on that would have been ideal but i didn't get any of that so i feel like that day you had told me you were pretty tired too like that's a rough day i was like i literally i don't remember actually my zk summit talk wow like i just like have no idea what happened my brain is it was really good all good was it actually yes there's a recording are you are you saying that just because you're on this podcast you know i think it's

the only reason you're on this podcast because you know i'll say that or you must say that yeah that's my law you brought me on this thought that's right that's right that i said so you said that no i think i think we're good okay okay you were good oh thanks it was really great and people can actually now see it i mean it's already on the live stream we're gonna have videos soon so that's right because you're like cropping them to the individual things yeah i don't think you need to worry you also you had a few people in the audience who like i could tell there was like a language that you were speaking to them about some kind of theorists oh yeah the measure theory a few people going like yeah he knows what he's saying but it was very niche no no the the best

[ __ ] on this one particular type of map there's there's a couple types of mathematicians like he he girmo likes a like smoother version of the world than is exists and so one about to get roasted one of the ways of doing that is to [ __ ] on people who have uh who have somewhat complicated mathematical theories that in practice you don't actually you know you might be computing something that is related to this but you actually never use the theory but like if you go talk to like a math person they're like no no this is obviously the thing you're doing in production like you're obviously competing a little big integral it's like no you're not like none of this actually matters right right right so but but yeah yeah but it does matter filtrations matter man they do not what kind of math was that it's called

measure theory i mean and i i should be very careful because i suspect that there's a non-zero number of people who are going to listen to this and be like oh my god like this dude is either absolutely insane or which is true i guess that's fair um but but secondarily like has no idea what he's talking about no i promise i've studied measure theory it's it's unfortunate for everyone involved i'm sorry if you have as well i would not recommend it i love measure theory i know that's exactly why i would not recommend i mean imagine the kind of people that study it is like tarun so i don't know well it's just like so there's a lot of very well-known paradoxes in like the formalism and formal logic of math where like math is like not consistent in a lot of ways like the kind of basic rules of like how

you count i don't know numbers or proofs about the natural numbers uh don't work and one of the most interesting set of these counter examples that like say that like hey one plus one might not equal two in like all scenarios that you might care about is usually these examples for measure theory and so like logicians really care about this stuff um most people who are practitioners who like care about something like in the real world don't but if you care about like the foundations of math like does is can math be logically consistent from like a philosophy standpoint not like a practitioner's standpoint you know this is this is actually like a very interesting form of i still don't know what you're talking about i think minok tarsky the fact that you can decompose a sphere into four spheres makes perfect sense a hundred

percent makes but these logical paradoxes teach you about the limits of your abstraction so like that's a that's a worthwhile that's a worthwhile endeavor like philosophy like it's not like you're thinking of it from the perspective of like can i can i put it in the turing machine that's not the point the point is like the epistemological foundations of like a lot of stuff that we talk about has to somehow be consistent but it's not and the these counter examples are worth understanding from saying reality i completely agree i feel like i'm not saying the term i have for this is uh i believe mental masturbation i'm not saying that that's a that's necessarily a bad thing it's a worthwhile endeavor i mean yeah at some point we were just talking about math as art or something but you know like it's great uh i'm just saying you

know if someone tells you but not tarsky and prod you should run away immediately or even well sure sure sure if we start talking about zfc i'm leaving i didn't this is why i didn't mention any of the security actions i just like went straight to like epistemology and philosophical arguments directly but i i think i think like the the the mental masturbation aspect of it is you know why do people like reading fiction they like this like imaginary notion like the notions of like i have some core abstractions i can build up these complicated things and i can run into conflicts dilemmas that i would maybe myself never run into and for mathematicians this is the same thing

yeah i mean like even in crypto people like thinking about things that are imaginary like like developing apps for cardano all right well i guess uh wow what a spicy taste maybe shall we let our listeners know what we're drinking right now or shall we not i we could leave it we'd leave it as a surprise at the end of the episode i don't think it's going to be that surprising i mean given current state of being but you know all right well actually a kind of related question to this which is a debate i had with this old coworker of mine who like i mean i guess maybe you actually worked with him with with uh is my my friend who who works on sort of like building these

like human level performance game bots at facebook and deepmind and like the things that like beat humans at like starcraft or go or whatever and he's been working on this game called diplomacy which is like which is like a really annoying like it's one of these games where like you collude with other people playing by like sending them text messages so like you're you're you know the game is like you want to like kind of like control all the property in this like virtual world but you can't do it on your own and you have to like form these allegiances and then betray people at certain times it's a form of like game where it's like it looks like a go board or stratego but because of the spoken language thing the state space of the game is way bigger because people

can suddenly collude and like when they collude they like change like how many moves are actually accessible to the different types of users stuff like that so like in real world so from uh right from this kind of like hey do we how do we build automated bots that can beat humans at this but also like predicting collusions all these things all the mechanics within it too right and a lot of it a lot of the success of like being able to beat the humans comes from these like large language models like gpt3 style stuff and the problem with that like we were just having this argument at this bar in new york and i was like obviously like drunkenly yelling i was just like like that's what you usually are the reason you he was just like yeah like we don't have theory for this but like just make it

bigger like just make the model bigger i don't really give a [ __ ] why it works and and like we were just we had this like interesting kind of philosophical argument between of like the difference between like ai and crypto where like in crypto because you actually care about the adversarial behavior existing against your model you're not just like oh i i don't care if my model is like perfect in some sense um up to some like very very well controlled error you basically need a lot more mathematical formalism uh and and you know my friend was he was just like yeah like who cares if we don't know the theory if it works and it's like good enough if it quacks like a duck it looks works like a duck it looks like a duck then like it must be a duck and i was just like yeah but like do you

have any sort of like there's no epistemology for why this thing even behaved correctly you're just like okay yeah i was able to sort of predict that like anna when she plays diplomacy talks a lot about sparkling water when she's trying to like say that she makes these like sparkling wine you know like you know those jokes that people write that are like x is sparkling oh i love that joke that's my favorite joker in the world template what is that joke what have you never seen anything you explained so the the classic joke goes uh it's uh it's only a frankenstein if it's from the frankenstein region of france otherwise just a sparkling monster yes that's the classic and so got it and then it's like it's like you know yeah sure they're learning some

pattern matching on this but like you don't actually understand like why people are colluding or like the strategy that's actually being learned of the different players in the game um and yes it's effective right like it somehow has worked but yeah you have absolutely no understanding how to transfer any of this knowledge and you have to like basically increasingly just throw more and more computation at it and like less and less like strat actual strategy and so this is a game this is a diplomacy game and is the point of the game to show you that you're just throwing a lot of computation at it instead of being able to do this or is this more like this is how this game this is kind of no human humans have played this game for like almost a century oh you mean to diplomacy like the concept of diplomacy

no no no no this is a game it's another game specifically a game called diplomacy okay and it's like a board game that like can last like 20 hours it's like a really like intense organ and it can play up to like i think eight players and basically what happens is like in the middle of the game people will start colluding with each other and they kind of can communicate with each other in certain ways uh and so a lot of the game strategies not just like the pieces on the board it's also like trying to figure out if like two players are actually one now or like if they're breaking up and that opens up the state space right because like you you as you're playing the game you might have gone from eight real players only two because there's two cartels right and so

that that changes how you design how you try to beat this versus like something that's like go where it's yeah it's like sort of partial information but it's actually really a a game of complete information that's just that like it's too big of a search space so you're just trying to like not pretend it's incomplete and try to learn as much as you can from a small number of moves here it's actually incomplete because it depends on our spoken language like we're talking to each other and communicating and like that changes the kind of action space again it's also quite imperfect too and it's right it's almost emotional right it's like you're gonna collude right that's why i made this kind of bad joke about like hey maybe like you when you like you you make a joke to someone when you're colluding with them you like say the sparkling

france region yeah maybe it was a little too yeah there's there's a lot of references yeah but but uh but but i get the reason i bring this up is like there there's some sort of notion of like one of the reasons i like crypto even though this is not a reason guillermo does clearly is that is that like i i believe in like very strongly in the value of like the epistemological foundations of like whether you can reason about why something works and you just in ai we're just like completely just we went we started the early days of ai actually we're really focused on that right and we just threw it out of the water because we're just like yeah we're just going to throw more gpus at it we're just going to use the energy consumption of kenya to just like you know to train

this diplomacy model and it but also not have any understanding of like why certain strategies work or or the reasons for anything and and that i think uh you know one thing about cryptos i feel like actually true i think you think it does do it so like okay but this gets down to the the one philosophy is you should care about epistemological foundations including measure theory which um i mean no i care about measures i like measure theory it's a very pretty thing it's just irrelevant in a lot of cases but i think the big thing is that unlike cryptographers right like ai machine learning you know none of this has had to deal with the fact that like north korea is trying to like screw with your like system right like in order to do that you have to like the hackers

yeah right like my point is you know crypto has this like daily attacks on it that's like crypto i mean cryptography generally although i suspect crypto as well but but you know like no one's going to be like ah yes i'm going to screw with your gpt3 model in order to like make you say something you didn't intend ha like whatever you're going to do about that people do do that for these image models no people do do that and this is great i love love love adversarial not adversarial learning but in fact like adversarial like yes someone said someone someone i do think actually we should get on this uh podcast is this guy who is um [ __ ] i'm forgetting his name but he's uh he's one of bones he's co-advised by bonnet and someone in in the theoretical computer science department in stanford

okay and he he has like his entire phd has been he he actually did a bunch of like monero attacks in the beginning of his phd but the end of his phd was just like adversarially breaking like lots of um ai things and like trying to generate like kind of treat it like a crypto problem uh and then like destroying like the facial recognition thing or like turning like hot dog into not hot dog yeah with like single stickers or single pixels or something like that you can do that absolutely and and i think like yeah you know the point is like if you don't understand how your model works you can never actually make it advertising resistant and that's just like i don't i think that's like a fundamental truism of nature and like we haven't well some people might tell you not that's not true right they might

just like be like oh it's just fine it's trained on the distribution of like generated samples of gender every serious temples then afterwards retrain again until you reach your face you don't even know if you have the right distribution look i'm just i'm just giving you the argument i'm not saying yeah yeah look look a lot of people got very rich from convincing people that like that was all you needed to do to do self-driving cars so like i i mean yes this is true right i it has not happened yet so let us let the point be known that there's no self-driving cars really they're not many like that and most of it is still like sfa area where it's always beautiful and sunny and there's no problem well phoenix really oh yeah well that's just phoenix apparently really easy no no no pedestrian traffic because it's like a

suburban city so there even though they had one accident one person killed myself so it's a lot of heuristics right it's not like this like single model that like knows how to drive a car they're just like throwing them around edge cases and but the investor pitch decks it's all they all say about this shot right like and like this is why i like my my friends in ai i feel like i'm just like philosophically drifting further and further away from them because of this like this fact that like none of them seem to give a [ __ ] about this anymore they're just like must just like throw more data and prod it's fine it'll eventually fix itself is there a moment where these two fields merge enough that like the attack surface is attractive so that it could be like adversarial oh i really hope so

that would be interesting that you actually like have to be serious well i think maybe a good question is like what types of applications of ai do you think that would happen like right the nation states [ __ ] with facial detection so that someone else can't detect their spies like a simple i'm sure look given how many papers have been written on this in the last five years the machine learning literature i would not be surprised if like there is some nation state that's tried to do that yeah yeah for sure it like that's the easiest low-hanging fruit but like you could imagine a number of other things right like you could manage like more complicated systems where you're just like by putting in just the right input like congratulations you're like now in or like you do like more interesting things like just end up being wrong at a

structural level that are really hard to notice until like way later yeah and one of the problems i think like philosophically between something like classical optimal control which is like what's used to do autopilot and planes where you do have they're quite weird but you know quite strict guarantees about what it does versus like the self-driving car doing reinforcement learning and like trying to like teach itself from its mistakes it's like you don't have any guarantees in the ladder case uh and in some ways like i i just like think people kind of underestimate the more and more we have the internet of [ __ ] i mean the internet of things the more the more uh more and more of these like ml attacks that will happen and the more that maybe krypto

and ai have like but it's going to be ai having to like eat some humble pie i really hope andre does not listen to this podcast uh or which andre tesla oh i mean come on the dude wrote a [ __ ] blog post of writing a bitcoin note as if he's like paying attention to you in cryptography very recently too that's right yeah that was actually reason finish him no he's like you know he's like he's kind of like he's really [ __ ] smart though he's like the guillermo of the the the self-driving car revolution i can't tell whether this is an offensive statement or a really flattering one i i think both hey well thank you but hey

i am now wondering though what is the overlap like of the ai community like you were talking about like your ai friends yeah have some of them moved over no a lot of them just view me as the weird like guy in crypto who who like has nephews yeah that's right you're like uh you're creating systems to attempt to topple governments yourself personally well they all still kind of only think of it in terms of this like whatever bitcoin maximalism like everything about like escaping a country and stuff like that oh no but i do think they've like started to appreciate stable coins like so for instance like my mom has started sending stable coins uh i'm bucketing my mom in this because like you know that she she's like statistician and she's more like she's ai

no but she's like classical statistics like you know clinical trial stuff yeah my mom sends me stable coin stuff but for a very different reason well your mom's a dj yeah because it's true guillermo's mom is like farming alongside of the big dogs and on is she then i have no idea she has not told me i i didn't i did get the privilege of finally meeting her recently at the european graduation and she thought i wasn't real so it's like it's the same as the alt thing where people she really did yeah for the whole for like a lot he's talking about this fictional character called true this is just telling him to do things yeah yeah i don't know it's a joke it's like it's definitely like turning's not real like every time we're gonna meet him like he just somehow dips the last at the last

second so he like obviously can't be real even my girlfriend was thinking this too and she was like are you serious like like i've seen pictures but like clearly probably a paid actor so wow that's such a flip on the twitter narrative i know it's exactly the opposite exactly irl on twitter not the same who would have thought i mean and then then who is who's out right and then you ate her arepas actually that sounds weird but i literally do mean like venezuelan arepas like yeah you know we're just for my uh hur gary on his mom's food you gotta eat it all right there you go confirmation shout out but i guess one thing that maybe the zk stuff i think in general maybe and i don't know if you guys i'm curious of like whether you think this will be the way that

ai and crypto actually merge is that there's just like a lot more complicated applications that are sort of closer to not not the complexity of like a gpt3 style model but like simpler ai models that i think we're going to start seeing people make zk proofs of validity for so like one way of saying hey this ai model was not manipulated by some adversarial example is it was like trained on like this particular data set you can verify here's a proof of the data set here's a proof of like the training model the inference so one of the cooler examples i saw was this topology which is like this like starkware game engine disclosure tiny investor but but uh they made a neural net on

starkware that only inference so like it you had to train it elsewhere but it would like generate a proof that of like given a set of input data points that like it generated the right data point and like you know in crypto that amount of computation they're doing like pretty dense matrix matrix multipliers like the matrix sizes were like you know a thousand by thousand like it was something that like you could never even imagine doing on a public chain right now and somehow the there's also this future world where the zk stuff enables these adversarial things to somehow be more transparent and like that i don't know who in the world is thinking about that but like there's got to be some genius somewhere in russia like thinking about or like maybe just left russia i think

uh i i know of one i think uh world coin is looking at uh oh that's a good point actually like on the so on hardware yeah and i was like no yeah i love remco though yeah very similar too yeah although we had a weird twitter thing with uh what does it mean that's right that's right that's right but but anyway i guess my point is i i think like i hope that maybe maybe if i actually make it to the zk summit and it lets me give a talk i'm actually gonna try to do a talk of like zero knowledge proof for ai people and like try to give these like examples because i actually think like there is a lot of potential overlap if the zk stuff

gets to the point that you can like help enforce some like hey adversaries didn't do more than x well but it also yields really funny things right where if you have an adversarial example for a fixed network you could just put it in and have like a very nice little proof that says i computed it right i promise which you did well the key is the key the key though let's say let's say you're doing if you have a robust algorithm yeah yeah yeah yeah okay okay we are we are pre yeah like assuming that right yeah we're assuming like here's one one example that like um i have this you know i have this friend who does um like research that i find very like it's never gonna it kind of is always gonna be an oraboros in some sense but like sort of like ethics and ai and like how

do you reduce bias stuff because it's actually quite hard to do but imagine if you could actually say like i trained this algorithm on this data set here's a public data set and here's the proof that i did not edit the weights and i did not retrain or augment any of it and like you can see that like hey like this is the sample has this much diversity or of like you know population sampling right and like that way you can publicly verify that hey it's not biased in some way and somehow like the ethics and ai people aren't super technical don't know about crypto and so they don't think about this type of stuff it's the same for like a lot of adversarial learning people and someday there will be that a different type of emerge the apology one like the thing is with

that example that you were giving were you when it says like is the validity proof like the this idea of like using validity is it in the protocol itself or is it just the fact that it's on a zk roll up the fact that it is no like yeah the contract actually does generate a proof of each constraint on like each matrix on each multiply fuse multiply add text i mean they also did like integrators on chain and stuff right like rk for like like they're definitely like of all the zk it's kind of crazy they're like the only ones from like wow you're really trying to do something that's going to take 10 years it's like kind of crazy like do a full game engine this way but like if you can do that then like this ai type of stuff is going to be similar

it's going to have a similar so there's like a whole world of these things people don't think about and i guess open-ended episode is a good time to pose that question yeah that's good if you have a plan for that talk i'm so down i mean i'd probably do it with my friend i'd basically be like he's a skeptic and i'd try to like basically bring him and be like we're gonna we're gonna we're gonna we're gonna show you some stuff kid we're gonna no no we're gonna have a duel a duel talk we're gonna have like oh okay like i present the crypto side he presents the ai side and it's like one of those like thesis antithesis synthesis type of talks and like he does i do the first thing he does a second thing and then we kind of merge and so like where could they merge

you have not been drinking enough muscle to not be able to say ancient this is yet antoine oh [ __ ] i already got it right away well if people are listening we'll know that okay there you go are there any other types of technologies you guys are like oh i wish like crypto interest that was exactly the question i was just about to ask you i kid you not you just literally we just had like like i was about to say i was what i was to say is like i know this episode is about us being in amsterdam but are there any other fields basically that we should be having adversarial models in like why aren't we using a lot of this stuff is it like and you go to ai because it's like so super technical a lot of data it's very exciting also influences like

everyone on earth's life now at this point oh yeah right are there in a weird way that's terrifying yeah i don't know i don't know who was it told me and i have no idea to rasterial story but apparently like auto land on planes now actually uses neural networks really the other is the only yeah that's right maybe the challenge is every time every other time you're trying to like mix crypto with something you end up with these sort of like we're gonna tokenize [Laughter] oh a coin for dentists to use with each

other didn't even have like a good pitch wouldn't that so i'm i'm uh i guess full disclosure like a crypto investment idiot um yeah but if you had something identicoin that was just notorious wouldn't it sort of make sense to just accumulate a large position oh like like wait when it was at the bottom because you know that people will always be talking about the question oh it might just be it the next dog coin why not from dogs to dentists i mean i uh i agree but uh it it depends on whether you're an anon crypto investor it's not financial advice or whether you're a self-respecting one and you just put in your ems oh no yeah exactly well you know i think uh you

know if we're thinking about other technologies or at least things in science there's kind of this funny weird thing where crypto and ai maybe that's the reason they keep popping up and like you see really bad icos that don't really make any sense around this is that they have this they both are are like the ultimate narrative storytelling right like they both have these like different versions of the world they're easy to tell a lot of people there's been a lot of advancement you can be like ah i can impact people's lives more directly there's a point to specific things too right like that that have happened in these things right yeah yeah exactly like you can you can point some impact that like you know maybe not every person on earth can see exactly what they're doing but you're like okay i kind of can get like a large

person portion of the planet's population appreciate them right but one thing that's interesting is like adversarial models don't exist in science in some ways adversarial god just uh [ __ ] with your experiments i mean that's awesome what is adversary in this case like what would it look like peer review system is adversarial okay guillermo guillermo can tell you a lot wait could that be then i mean but you don't want to you don't want it decentralized science there you go learn you anything every day is it cool no okay but decentralized peer review or like

having a way of paying people who do peer review somehow would maybe that might be interesting yeah the question is how do you align inside well actually about you as the only actual phd in this room oh god you don't have one here right okay use only actual phd in this room what you should explain the problems and pitfalls of peer review the pros and cons yeah explain how it works because like people might know okay that's right that's right because a lot of people never like publish their papers they're just like they just put an article archiving yeah so so yeah yeah so so the peer review process is you know you submit a paper usually to some cranky old people who are uh a conference yeah engineered tenure and tenure and what happens usually is you know you

get some pretty good comments back so so often often people are actually quite nice and they're like oh this is good or interesting and then every once in a while you get one reviewer who just absolutely shits on your paper and says sorry like it doesn't matter what you do but like there's no way you can ever get this in because like x y z reasons this is not always true often people are generally nice but the point is you know there's this weird system also by the way the fact that i'm gonna go on a little side right here is that peer review is the golden standard it's a standard it is a [ __ ] standard and i'm not saying like there are like necessarily better standards i can think of immediately but it is a garbage standard and everyone tells you like

it's not a peer-reviewed study you should tell them like any number of things are not peer-reviewed that are useful like jumping out of an airplane uh probably causes depth of high probability is a particularly useful thing that we have not peer-reviewed well well an interesting thing is that like we've actually developed these kind of like meta adversarial things against peer reviews like p-value hacking so uh it's beautiful so so p-values are just like under a lot of very stringent assumptions a way of like measuring like hey like i did a hypothesis test that says um is it possible for the opposite of my conclusion to be true i still don't think most people can define what a p-value is who use p-values in their papers that's my hot tub actually i agree with that certain actually yeah but but the idea of a

p-value is it tells you sort of an estimation of the probability that your your prediction of your solution to the hypothesis is incorrect and people interpret it as a percentage so if i say p equals five percent that means that i have 95 confidence uh that the hypothesis true but the problem is the definition of a p-value assumes a certain model for how you generate the data how you actually compute what the error probability is it's just central limit theorem dude it's just like it's obviously a caution everything's everything everything is clt independent id clt humans exist because all their molecules are independent that's exactly right okay i think that's a good one here's a small question has anyone explored like peer reviewing the peer reviewers

like has there been an evaluation technique on peer reviewers the answer is attempted uh maybe not evaluation on peer reviewers but often like open peer review is a thing that happens for example in ai a lot right where you peer reviewers like when they post their review so you have to post your paper online okay publicly accessible for everyone to see and reviewers have to post their reviews online also publicly accessible even though they are pseudonymous oh so they do have an identity but that's like no done okay the point is that you can't often you can't track thumbs up thumbs down don't you worry about that a little bit like because then that's gameable that's just a little bit

this is exactly why like there has to be some weird solution that maybe it's not like crypto tokenize it tokenizing there you go that's a terrible idea nailed it but we should make the jingle oh no we'll just come in once in a while i have a sound the craziest thing about peer review is it reminds it's it's um you know there's a lot of like labor abuse that exists in a lot of like established very like high garbage fields so i didn't say that no no hi no no not that i i mean more like high sort of like societal value fields even though they might not be like a large part of the economy so like in fashion in music in like art there's like tons of unpaid internship

intern i'm putting error quotes on it like there's all these unpaid jobs and people are or like people even pay for them in some cases and they basically effectively are jobs where it's like hey i get to say i like status work it's the saddest thing right right you're paid in status but the weird thing is usually that happens to people early in their career the peer review system is actually kind of opposite people later in their career don't get paid to review all these papers for free and then elsevier and publishers are the ones who sell it back to the university is it status there though are they status peer-reviewing why do they do that uh in part i think they have to well no no there's a collusion effect you actually want other people to basically be saying that your papers are good and there's kind of like a like

wait are you saying like you be right you don't become a pure reviewer by reviewing you're able to like express your preferences not necessarily objectives okay and about what you're doing about your own papers you're not yeah you might you might say things like hey cite people me that all the time but do you get to shape the narrative in a weird way you get to direct ideas towards your ideas it's like the difference in producer and director in a movie that's right but you also got to say yes they are the right producer i think they're kind of the producer right okay yeah they don't create the the art they don't create the world that's right they're like i want you to give it like a little bit more superheroes for this conference the the the sort of um

essence of this conference is x and this paper does not meet the essence of this right right well what does the [ __ ] essence of the conference mean well it's like whatever that reviewer thinks once but also whatever their work may represent exactly you also got to do it via like kind of quasi-censorship effect right like you also get to vote down on papers that you don't think should make it independent of like there might be many reasons why this is true of course many papers are usually garbage even those that are published but generally speaking right like there is a censorship idea of like someone tries to publish something that you kind of don't think is interesting or doesn't fit the narrative even though it might be interesting in like a very general sense but like of course we only have our own weird subjective things

right and you just got to like downvote and be like sorry actually uh this is not that interesting at the end of the day even though it might be a very interesting topic and and again this is always interesting there's like these voting you're voting yeah there's there's some news there's something this is why a lot of yeah that's why a lot of the crypto stuff somehow should intersect here yeah and also get rid of a lot of the labor abuse because like right in my mind there's this huge labor abuse going on yeah yes it's her status but also someone is making money off this right and it's [ __ ] crazy when you're a professor at a university you're reviewing and then you go to a publisher like elsevier or nature or science and then the publisher is the one who's like hey give us reviews for free and

then the publisher sells that back to your institution yep like that is the ultimate scam like like they're taking free labor and it's all based on the brand of the journal and the impact factor and some [ __ ] metrics that are games i mean okay one are they making us a lot yes their margins are like they're like over 50 percent yeah okay there you go eighty percent i mean it's not like they're not like big growing businesses but they're like ten billion dollar business they're like stable yeah businesses three very high skilled labor in fact the like medium skill labor the ones who are like distributing in the sales and whatever they're getting paid but the highest skill labor isn't it's the opposite of the art music industry though they're public companies

oh they're okay they're a huge problem are they yeah yeah we're in the netherlands yeah actually you're totally right journals yeah yeah yeah yeah yeah this is the aggregation company nothing about this journal i mean i know they exist i know that like but like think about you do you get why the labor thing is like kind of an abusive labor thing right like it's like why are the professors and grad students doing this for free and then their institution is paying to buy back their labor labor at the same time like right now as you describe this all i can think is it's it's going to go out the door like there's no way we're all and how annoyed will they be when that happens ai and high energy physics and math have basically moved off the journal system

they do everything in pre-prints there are journals but only when it's like really seminal like you know terence towel proves some weird kind of crazy so wait who's on it then uh well no everything is just done on the internet more publicly yeah and then afterwards submitted to journals like if you go to biology it's like no you didn't publish in nature like what are you doing like your career is [ __ ] like there's there's a lot of the status stream stuff that like you see in like art music finance but our art music um sort of like the crazy part though is it's not just the reviews though it's literally like the papers themselves are like essentially done for free right like like the a lot of the work that goes into not just like reviews but also the

papers like just gets done for these like publishers for free and then they go and sell it back and i mean in a lot of cases in math and computer science and physics it's very very common to upload your preprints first so it's always available publicly online but in some cases it's actually not like you will get sued if you like publish your papers online for free and then like try to publish it like afterwards and stuff like wait you just said like a bunch of fields moved off it that's right how does that work then they just started publishing on the internet so archive um which is sort of the main prim archive was created by paul ginsberg who is a physics professor um at cornell yeah uh who famously i remember um when i was in undergrad there he uh they built a new physics

building and uh he he's kind of a true true hippie like and he was like in his office i guess he like passed out in his office and then like the police came and like were like oh there's a homeless guy like sleeping and then there was yeah so i remember that was like one of the funniest things that happened this is like 2007 or eight oh god that's really funny but um he's an interesting character and so like the idea was that a lot of high energy physics research um a little bit like ai uh at that time like obviously in a different scale was stuff where there was stuff that was done in empirical stuff so people of particle accelerators were kind of writing

technical reports on like things they saw that were sort of like the way guillermo and i started writing papers we just like post them online we see what feedback is before we refine them yep um people would just post those and they'd usually only share them within each accelerators like cern particles switzerland they would have their own technical that's like a mailing list that's right their peer review because it's peers yeah appear but it's huge right it's sort of like push not pull or or vice versa pull not push like you just post it and then if people review it it's not like a and so then a bunch of the different particle accelerators like started realizing like hey we actually should be like sharing our technical reports yeah they're not finished but if we like

shared some of that we would like probably save someone like some dead ends that they couldn't go to i also catch early mistakes too a lot catcher yeah i mean like whatever superluminal neutrinos and all this stuff right right or well we were at them at a time where like baryons were still not classified this is like this is jumping ahead on the top but it's like this was caught before it like went out i i think people who are like at the boundary of theory and production like people who are trying to take theory and like do really hard experiments like validate really complicated theory they oftentimes need and it's not not true that like other fields don't have this but like for some reason that ends up being the space where people want to make these kind of

like right very quick uh intellectual works that they share quickly rather than like have to go through the whole full peer review process and so that was where it started and that sort of was sort of yeah the formation the genesis of archive but then of course the most famous thing was perilman who's who who won the fields metal and kind of went to go live in siberia yeah yeah true true goat none of us are around the uh he uh you know he posted this proof of like this this very famous conjecture uh on archive never never got peer reviewed and then it turned out was right and like he won this million dollar prize and he hated he hated

society for giving him a million dollars so then he went to go live as a recluse with his mother in siberia that's right i thought i thought he was working on the riemann i bought there no no point crazy no no so i know that he proved the poincare conjecture but i thought that people speculated that he hadn't actually like retired entirely i'm sure he's not retired but he hates institutions he basically was like why does it matter that i am an academic institution if i publish something that was correct and there's truth and there's both like good things and bad things institutions are a heuristic they let you like assume a lot of things without having to like actually spend years figuring them out at the same time he has a good point of like there was like he was able to kind of

like do a lot of stuff on his own so i i but i don't know if he would be doing rebound hypothesis because like his background is more in like richie flows and like probability stuff measure theory all the [ __ ] that like guillermo hates hey no it's true it's like literally literally like one of my undergrad advisors like proved this measure theory inequality that was used in that paper and like when i looked at the proof of that i was like this is all like in reality like no [ __ ] thing that's like really you know homotopia equivalent to three sphere would ever [ __ ] look like this edge case and that's like the guillermo like measure theory is useless thing no okay now i'm on the measure theory thing i want to set the record straight i don't hate measure theory i actually

find it quite beautiful right like i have studied measure theory stochastic you know whatever processes the stochastic differential equations the whole thing right but like my point is what i'm pushing back against a lot and i do this a lot is like in papers it's very easy to go down a [ __ ] rabbit hole of just like proving a bunch of [ __ ] that turns out to kind of be irrelevant and it's really obvious if you know like basic real analysis or you know basic measure theory but like reviewers will come back to you and be like oh but like how do you know this thing is like dude it's literally three lines like if you if you took an undergrad class in this thing you would know what i one place i will disagree with this is that you know there's a lot of like counter examples

that generally come up from you not stating the correct assumptions no no but but like okay so being very clear here in in our papers we we state the exact assumptions yeah and we say ours are not like real matters it's very different right yeah i mean like actual like yeah strategically pretty mathematics i mean like even even whatever like the jordan curve theorem read is really easy to prove in the case where you assume that everything is differentiable or like sure point wise i mean it's a homework problem but that no no but it's not a homework problem now right the general case is pretty hard the general case is even isn't even true sorry can i actually bring us back to peer review because i was so down yeah guillermo just doesn't like me making fun of this but this is actually

something i've personally faced in peer review where we're like usually people are just like oh like you like didn't mention this complicated counter example and i usually i'm like oh yeah you're right guillermo on the other hand's like no no my definition should have restricted it so that you never had that [ __ ] that's right that's right yeah absolutely but no that's a philosophical difference no but my point is like i i don't it's not that i don't think that people shouldn't be careful is that like i think people should be extremely careful but in the paper it does not matter like if you knew the thing you needed like if you knew the exact for example in one of our cases we have something that like is technically not necessarily even like inequal in like a usual sense it is like the big

integrable but not like normal riemann integrable oh you mean the optimal fees paper no and i love that we've fed you this much mezcal sorry and you're arguing right now about a very specific very part this is like this you may have to you may have to cut this because this is like so in the weakness i know i know we're going to leave it but i think i might bring us back all right because actually it sounds like what you were actually proposing we i don't think we've proposed a solution i think we're going to the period yeah a problem right i think we've said hey there's a problem it's also i mean it's a problem for zk papers as well yeah it's a problem uh we've actually run into but oh no no

no no no no no no i'm not gonna wait wait wait wait wait wait but even even if he's not gonna say one i think the the the the planck thing is a really great example right where like a mistake in the paper was never verified until like later by a auditor trail of bits the research yeah yeah and like to some extent i remember when i first convinced guillermo that crypto wasn't a scam i was like hey like you know what he's like what's the cool thing i was like there's all these papers they use a lot algebra so you're not gonna like it but it's like zero knowledge stuff and he was just like yeah i can't even [ __ ] verify any other [ __ ] proofs like they they like they're still trying to read them yeah their standard their standard of proof is like weird even

even the notation is no it's yeah yeah there's a whole other subscripts and superscripts and you know you have functions that depend on eight different things even though like seven of those are completely irrelevant yeah sorry anyways but besides the point sorry because everyone likes to write out the proof as like they write out the full algorithm instead of like analyzing the components which like it's like a cryptographer's style that difference between like math people between normal people and cryptographers you should just uh teach one cryptographer einstein summation notation no that would be it'll spread like a virus actually i think i i i don't think it would though i feel like they like they don't like they like writing like five million okay my hottest take is that i think teaching cryptographers like a normal linear

algebra course would do like wonders for humanity where you don't actually have to write out the sums for 90 of these things you just define them as linear operators anyways another another another thing i think that's like kind of a stylistic difference you oftentimes see when you like read applied math papers from different fields is like some people like writing papers in a sense of like oh like here is actually the most general possible mathematical thing that like sort of correlates to this scenario even though like if i look at the set of all possible things you defined less than one one millionth of a basis point of the possible things is actually like what you think that you care about

and the other type of people are like people who define things too narrowly and they're like are like it's true to my use case and then like you can't use it somewhere else and there's always this kind of trade-off in applied research of like do i make the most general thing or do i make the specific thing and you kind of like have to learn that in like some ways and i just feel like zk people haven't figured out their sweet spot for that and ai people were the same like they it took them all [ __ ] long they have 20 years i would i would say they don't but it's much better than it was in the 90s no this is this is quite true i even i wouldn't even say the early 2000s i mean some like look you read a lot of the

papers like even random kitchen sinks which is like a pretty simple paper was kind of illegible the original time around i feel like there's two things happening here though it's like one question here is like how do we better peer review zk paperwork and the other question is how do we fix peer review all right this is how we can talk we're we're doing do we use zkp's to fix peer review no no well you didn't have a [ __ ] post on this didn't you yeah yeah i i i wrote that making fun of the design people though really but like it was a subtweet i don't think anyone really got it unfortunately it's fine it's fine i got it yeah you're all understood really i was pretty i think i retweeted the the the alt harem cartel cheers to that cartel

[Laughter] um i guess like the the main main thing of this these like stylistic differences yeah though for things that have to be very precise is that what are we talking about by the way we're talking about like zk being peer reviewed well we'll well we're getting back to that but we're kind of talking about like this idea that like what does written in the paper how do you actually verify the proof is correct right and and like guillermo's philosophy is that the implementation needs is the actual proof like if the implementation works which is definitely true like that's the most true form but there's also a sense in which the style in which things are written

can also be adversarial to the reader and then peer reviewers just go lgtm looks good to me plus one lazy thumbs up like code review has the same problem like yeah for the record and and i think this like is actually really important because especially in frye there's a lot of like sort of ambiguity and soundness and there's some conjectures and like i feel like there could have been more attention placed on that if it was a little bit more clear that those existed and like how they functioned and real protocols and so it's just you finding this out because of like actually trying to implement it yeah yeah so yeah so so we rely on a soundness conjecture that has not been formally proven um i believe really star queer does as well um but it it's like it would be nice if there was

there were more attention and formal proofs but that that last friend is annoyed because we're just pouring more he's he asked us at the beginning of this interview actually do i have to and the answer is all of it the answer was always yes yes um because uh as of now there is a bottle of mezcal that is officially empty um very nice just to the listener this is not the future of the ck podcast we will have normal ck podcast episodes soon but this week it's a week of amsterdam and honestly there's a lot happening here i i guess like one other thing that's like kind of interesting about this the

trailer bits report is just that it kind of says like oh well the peer review for these zk things was this auditing firm deciding to like go read the implementations right all right and so that's where i i think there's there's like kind of a notion of peer review that we haven't figured out push verse pull and it it it's like the academic version of the world is push like you send a centralized authority the journal which is some institution with some cultural definition for itself very much like nft dao's but hopefully smarter but not usually they're still humans too too early okay uh and then and then usually then they they are like push here's the here are the papers to review here's what we think is cool here's whatever the like the program

committee type of thing and then people review and then you have the opposite like the zk world or the high energy physics world was like this before where there was a lot more like pull like we just shoved the [ __ ] online and like the problem is in the 80s there was no one else posting content internet so but in the case of an auditing firm doing it was were they hired to do an audit and then found it oh they were hired to do one of the audits and then they started doing the rest yeah there's no way this was like uh just like oh we got hired to do this and therefore we like went into this gotta it had to be like an internal thing right i don't know i mean yeah so i i i believe that they were hired i i

think another wrinkle that's sort of interesting in this case is that this bug didn't exist everywhere so certain teams had fixed it because like the the they spotted it uh no no the tr i think trail of bits spotted it but but yeah so so i i think there's like a deeper like maybe structural problem in zk where there's like not always that level of uh collaboration right all right and so here yeah yeah and so i think that's like interesting thing i mean because like the bug was fairly like straightforward it was just not including public inputs in feature mirror um oh it's just like a pretty cool yeah basically but actually just a question on that

like what you just said was like it wasn't the i just asked like where was it that like some teams had implemented it and then found that problem and changed something like well i mean the bug was in like some overarching like research paper that was implemented in the way it was supposed to be but you just like like you just said like not everyone had the bug i actually don't understand why they didn't yeah so i think my understanding i could be wrong is that in the planck paper it wasn't uh like it wasn't explicitly hashed into some uh input for fiat mirror and or it wasn't included as part of the input and that was true in both bulletproofs and in planck but there were a number of

teams that just knew to do that yeah right right yeah this one my guess is that's what i thought it was implementation but why did they not notice the problem no no no the problem is people just open the paper and these crypto papers are written as like here's the algorithm algorithm right but it's not just exactly but i don't have to think about it because i assume the one in the paper is correct but there is no no no i get that but i what i don't get is like why did somebody implement it correctly and fix that and not just not not like disclose but why like why was that not communicating somehow it's common that people have a lot of context when they read these papers right it's not like in a lot of cases right you the paper is supposed to represent like a fairly high

level but reasonable idea of how one might implement this concretely right in a lot of cases like if you have enough context on the paper it's very obvious to you that this is a thing you have to do but one might call it an implementation detail right is it almost like because of who's actually making the changes they're just not like super tapped into like the comms team they're like they're not going to be the people who are like raising like i mean like why would you deleting it up they're just like we're going to oh this doesn't work i'm going to fix it well but then they don't it doesn't go higher than right like people who have like decided to use the paper they're not aware so in some sense like when you read the paper and you have enough context it's like almost an obvious

thing right it's a thing you just read and you're like yeah i mean of course like they didn't do this check in the paper because it's like you know why would you do it you should already know to do it right but in general this is not true i mean as we found clearly right you know if you just implement straight algorithms on the paper in the same way as if you implement like a lot of like mathematical papers by just like reading the thing kind of front to back and being like all right here's what i need to do um it's not immediately obvious that you have to do these like extra checks these like even like in a lot of cases you're doing kind of a lot of reductions down into like something that you can actually use so it's um i think it's a lot of

contextual questions that are not easy to answer this is this is a bit of a like left field but like coming back to amsterdam where we are right now oh no there were a few topics we had meant to talk about that we have not uh because we've kind of run out of time but we've also had a whole bottle of mezcal i also think the conversation we had was pretty amazing but i am wondering what are in a short moment like what are the takeaways from this week what are the the ideas you sort of said earlier on like idea-wise you didn't feel like necessarily inspired but like what do you think are the topics at least that are being discussed right now or something that

you're thinking about um i think any v day again and not to to harp on it being but i thought it was a really great event um congratulations to tina uh for also giving her i want to give her a shout out like clap and shout out because i was tina's been putting on events for a long time just like you anna and like i feel like she had started by being renegade and ragtag and like would like do these free events and then what the [ __ ] d5 yes what the [ __ ] there's the other one at wtf mev yeah i mean so she like did these kind of like very grassroots events that like to see it grow to something is actually like i i

just like feel like as as her friend i just feel like she like has clearly like built this institution around it and also just like i do think they got a lot of interesting talks i i don't think anyone answered the question that i care about personally and guillermo which is just like how do you actually write down some type of like mathematical theory about mev that like actually combines things that we know about like i mean as we found it's a hard questioning it's very hard to answer because there's some notion of like subjectivity of value but there's also some notion of like optimality when you have many parties like defining the social welfare of like all the participants is actually quite hard it might actually be in some cases and i

think this is something i really kind of felt a lot from that mev days like the existence of mev for a lot of chains actually is a good thing it like drives volume there and it actually has these like weird ironic effects of like oh volume increased because people are getting sandwiched oh people add more liquidity because they're like oh there's more volume here and it like has this weird like growth hacking bootstrap effect bad metrics but also are you an ex what is an accelerationist kept saying i i am not per se um i do think dean had a very funny comment which is like i would never use a chain that i make money from oh that is that is actually that was like they're all centralized

that was like a very harsh that was actually yeah yeah he's like i will not mention that you know but then he just said bsc like almost like the neck he's like there's 21 validators and 11 of them are owned by an exchange actually said that on the episode we did with him that's all clear yeah yeah i think it's pretty lovely i feel like it's the mev chain though like as far as i know there's like several people on and everyone is just like there just to make money but that it's not just that it's also just there's a lot of like retail usage in south america and asia is there on bsc yeah i mean oh [ __ ] it's like finance finance has extremely high in asia and south america so in south america busd is actually people basically do remittances by just

sending busd between binance not ust no apparently because like most of them their only gateway is like finance bias has a huge penetration in south america wow holy [ __ ] because like yeah sure bitso and stuff definitely own mexico and that area but it is it is is actually quite interesting that a lot of the capital inflows are basically latin america and asia and the reason there's so much busd demand comes from from people who are basically using binance as their bank i mean you don't like whatever 5 000 inflation in venezuela you know there's there's better inflationary uh not

inflationary is venezuelan so he has firsthand experience with uh ohm like inflation and that's right that's right that's right and it did not have provable bounds on like our most recent paper or whatever you know the thing that was interesting is like i think one thing that has been kind of the narrative around mev is generally like it's predatory it's bad it's whatever but i i i think like there are conditions and like if you could understand the theory you can actually understand a little more of like when it actually is useful for a network it's not necessarily always bad which i think maybe is like a contrarian thing to say but it actually is useful for some i'm not like in some cases it is extracting value from users no doubt but in some cases it's actually like

increasing liquidity for users so like if the amount that's taken from them versus the like decrease in transaction costs is different yeah it actually can be positive and listening to a lot of the people talking especially the traders i was actually like more surprised at like some of the things they're saying where like they they effectively increase their participation in certain networks because they were doing mev on them which improves user quality and something i mean it certainly improves like with the bad comes the good right right so like i just don't like this kind of like you know look everyone yes humans i still put them in the category of that so humans humans love false dichotomies they love this idea of

this binary classifier that you can just like always just you rely on as the ultimate heuristic the worst one of the internet has produced is the word cell and shape rotator one no you're not mentioning this in this episode no you're not you are not bringing up the other terror in this episode uh i'd say the worst binary uh sort of classification dichotomy that humans have that have made on the internet in this last year is the word cell versus shape rotator cell word cell versus version shape rotator shape rotation don't worry i'm not going to vibe camp

i'm sorry i'm sorry everyone brooke i promise i promise i'm not in the center for applied rationality well i think there's everyone who is like x center for applied rationality right yeah so i i guess the the point i i want to kind of maybe put across is like me v day as a really interesting event having worked in uh high frequency trading before it was like you would never have that type of conference and people would never be that honest about kind of like they do and you would also not get the feeling that there were a lot of people there who are like oh yeah like actually uh there's some positives to it for certain networks and certain types of applications do do better when you have

that and as application and base protocol security start getting more intertwined you we're going to inevitably see that like there's going to be good mev and bad mev and it's kind of going to be too much of a catch-all phrase that doesn't actually capture the nuances i did i did actually notice just generally this week the use of the word mev kind of constantly being confused it's even like it's like zk and mev both get used when they're not they're like supposedly yeah it's like mev equals sandwiching equal and that's the only way and it's always bad and it's kind of ruthless and then like someone's like no it's more complicated and then it's complicated to actually describe why it's more nuanced my favorite i've seen

that kind of you know of the like six or seven whatever panels i i in talks i did uh the one that was did not attended was on and moderated yeah intense was one where um phil diane was a moderator and he brought eric buddish who is a famous academic from uchicago who invented this concept of the batch auction which was supposed to kind of destroy hft and like of course we've it's turned out it didn't it didn't work it didn't work it didn't it in fact it like didn't work but fully like like like i think the theory has a lot of merit but one interesting thing i learned from this talk you know i've had a lot of respect for buddhist work uh but interestingly i learned phil first of all was quite inspired by buddhist and

that's sort of how he got into thinking about flashbots because he was like how do we make a batch auction for this stuff because i know i know but like look that's just someone's life their story and so and and so and then i basically was kind of we were on this panel and like it's very interesting because like you have me and lev uh and some felix from cow swap and the three of us are like way more in the weeds and like thinking about like the kind of transaction level details technical details and then we have like phil who's kind of like the philosopher king moderator and then we have eric buddish who's like economist who like probably couldn't write co like couldn't tell you anything about ethereum transaction ordering to save his life

but he clearly had this kind of view that he's like yes this is going to like we're going to like implement all my ideas and then we had like you know lev and me and and felix giving a lot of interesting counter examples and one thing that was very clear at the end of this was that like buddhist left this thing thinking that like he came in thinking that like flashbots was just his idea kind of like implemented and like it provided the same benefits i think he left feeling like oh [ __ ] actually actually it's like more complicated it's not always bad like like there was like all these like interesting and so that's kind of where i think sometimes the this like being able to kind of have the dialectic ironically given dean uh the

dialectic and multi-lectin perspective on these things is actually extremely important fair enough cool i think this may bring us to the end of our episode i i feel like we could continue forever if we're not careful we might we might alcohol dilates time yes that's fair enough but it was it was an incredibly fun time and i'm so glad we had this conversation and even though we went a little off the zk podcast track a little bit i think it's good why not no i think it's phenomenal i also feel like isn't it isn't it time for this show to also like expand and evolve and allow for more spontaneity and more spicy take yeah thank you

misskell anyways thank you guys for doing this interview yeah it was fun and it went completely off what we had planned but i think it's better this way i think uh you know if you've never actually gone drinks with me the number one thing you've hopefully learned from the episodes i love saying the word epistemology the more alcohol the more it's like more he drinks the more i drink i say like philosophy kind [Laughter] well thank you for having me it's been lovely uh thank you for coming back and brandon thanks for coming on for this like very spontaneous insane interview

yeah thanks jenna for having me it was a lot of fun we uh in the process we did we did anoint the polygon polynomials which is which is good i heard that that happened on twitter and now tarou and i heard it's real the head of marketing oh yeah congratulations congratulations one tweet all it took all right i want to say thank you to the podcast team and to our listeners thanks for listening you